slicktune.metrics

Training and evaluation metrics tracking.

Classes

MetricsTracker	Collect and persist metrics across a run.
TrainingMetrics	Snapshot of metrics for one training run.

Functions

count_parameters(→ tuple[int, int])

Count trainable and total parameters.

Package Contents

class slicktune.metrics.MetricsTracker

Collect and persist metrics across a run.

Parameters:

output_dir (str or Path) – Directory where metrics.json is written.

__post_init__() → None

Ensure output_dir is a Path and exists.

load() → TrainingMetrics

Load metrics previously written by save().

Returns:

TrainingMetrics – Restored metrics object.

Raises:

FileNotFoundError – If metrics.json is missing.

output_dir: pathlib.Path

save(metrics: TrainingMetrics) → pathlib.Path

Write metrics to metrics.json.

Parameters:

metrics (TrainingMetrics) – Metrics snapshot to persist.

Returns:

Path – Path to the written JSON file.

class slicktune.metrics.TrainingMetrics

Snapshot of metrics for one training run.

Parameters:

• strategy (str) – Parameter strategy name (e.g. "lora").

• objective (str) – Objective name (e.g. "sft").

• model_id (str) – Base model id or path.

• train_loss (float | None, optional) – Final reported training loss, by default None.

• eval_loss (float | None, optional) – Final evaluation loss if computed, by default None.

• train_runtime_sec (float | None, optional) – Wall-clock training time in seconds, by default None.

• train_samples_per_second (float | None, optional) – Throughput reported by the trainer, by default None.

• trainable_params (int | None, optional) – Number of trainable parameters, by default None.

• total_params (int | None, optional) – Total parameter count, by default None.

• probe_pass_rate (float | None, optional) – Fraction of probe checks that passed after training, by default None.

• eval_perplexity (float | None, optional) – Holdout perplexity from Phase-2 eval, by default None.

• judge_score (float | None, optional) – Mean judge score in [0, 1], by default None.

• extras (dict[str, Any], optional) – Additional key/value metrics, by default empty.

eval_loss: float | None = None

eval_perplexity: float | None = None

extras: dict[str, Any]

judge_score: float | None = None

model_id: str

objective: str

probe_pass_rate: float | None = None

strategy: str

total_params: int | None = None

train_loss: float | None = None

train_runtime_sec: float | None = None

train_samples_per_second: float | None = None

trainable_params: int | None = None

property trainable_percent: float | None

Return trainable parameters as a percent of total.

Returns:

float or None – Percent trainable, or None if counts are missing.

slicktune.metrics.count_parameters(model: Any) → tuple[int, int]

Count trainable and total parameters.

Parameters:

model (Any) – PyTorch module.

Returns:

tuple[int, int] – (trainable_params, total_params).