slicktune

SlickML fine-tuning toolkit for LLMs.

Submodules

• slicktune.callbacks
• slicktune.cli
• slicktune.data
• slicktune.eval
• slicktune.metrics
• slicktune.models
• slicktune.objectives
• slicktune.recipes
• slicktune.strategies
• slicktune.tuner
• slicktune.types

Attributes

__version__

Classes

AdaLoRAStrategy	AdaLoRA: adaptive-rank LoRA that prunes toward target_r.
DPOObjective	Direct Preference Optimization (TRL DPOTrainer).
DoRAStrategy	DoRA (Weight-Decomposed LoRA) PEFT strategy.
FitResult	Result of a completed Tuner.fit() call.
FullStrategy	Full fine-tuning: update all model parameters.
HoldoutEvalResult	Holdout negative-log-likelihood / perplexity summary.
JudgeReport	Aggregate judge outcomes.
JudgeResult	Outcome of judging one generation.
KTOObjective	Kahneman–Tversky Optimization (TRL KTOTrainer).
LLMJudge	Ask an LLM to score a completion from 0–10, normalized to [0, 1].
LoRAStrategy	LoRA PEFT strategy (default production PEFT path).
MetricsTracker	Collect and persist metrics across a run.
ORPOObjective	Odds Ratio Preference Optimization (TRL experimental ORPO).
QLoRAStrategy	QLoRA strategy: 4-bit quantized base + LoRA adapters.
SFTObjective	Supervised fine-tuning on instruction / chat pairs.
SubstringJudge	Deterministic judge: pass if must_contain appears in the generation.
TrainingMetrics	Snapshot of metrics for one training run.
Tuner	Composable fine-tuning entry point.

Functions

compute_holdout_perplexity(→ HoldoutEvalResult)	Compute mean NLL and perplexity on a holdout SFT JSONL / dataset.
run_judge_on_probes(→ JudgeReport)	Generate replies for probe prompts and score them with judge.

Package Contents

class slicktune.AdaLoRAStrategy

Bases: slicktune.types.Strategy

AdaLoRA: adaptive-rank LoRA that prunes toward target_r.

Parameters:

• target_r (int, optional) – Average target rank budget, by default 8.

• init_r (int, optional) – Initial rank before pruning, by default 12.

• alpha (int, optional) – LoRA scaling alpha, by default 32.

• dropout (float, optional) – LoRA dropout probability, by default 0.05.

• target_modules (list[str] | str | None, optional) – Modules to adapt, by default "all-linear".

• bias ({“none”, “all”, “lora_only”}, optional) – Bias training mode, by default "none".

• tinit (int, optional) – Warmup steps before rank allocation, by default 0.

• tfinal (int, optional) – Final fine-tuning steps at fixed rank, by default 0.

• deltaT (int, optional) – Rank allocation interval, by default 10.

• total_step (int, optional) – Expected optimizer steps (required by PEFT AdaLoRA), by default 1000. Prefer setting this close to steps_per_epoch * epochs.

__slots__ = ()

alpha: int = 32

apply(model: Any) → Any

Attach AdaLoRA adapters to model.

Parameters:

model (Any) – Hugging Face causal LM.

Returns:

Any – PEFT-wrapped AdaLoRA model.

Raises:

ValueError – If total_step is not positive.

bias: BiasType = 'none'

deltaT: int = 10

dropout: float = 0.05

init_r: int = 12

load_kwargs() → dict[str, Any]

Return empty load kwargs.

Returns:

dict[str, Any] – Empty mapping.

name: str = 'adalora'

target_modules: list[str] | str | None = 'all-linear'

target_r: int = 8

tfinal: int = 0

tinit: int = 0

total_step: int = 1000

class slicktune.DPOObjective

Bases: slicktune.types.Objective

Direct Preference Optimization (TRL DPOTrainer).

Parameters:

• beta (float, optional) – KL penalty coefficient, by default 0.1.

• loss_type (str, optional) – TRL DPO loss type, by default "sigmoid".

__slots__ = ()

beta: float = 0.1

loss_type: str = 'sigmoid'

name: str = 'dpo'

required_columns() → list[str]

Return required preference columns.

Returns:

list[str] – Preference triple column names.

class slicktune.DoRAStrategy

Bases: slicktune.types.Strategy

DoRA (Weight-Decomposed LoRA) PEFT strategy.

Same knobs as LoRA with use_dora=True for magnitude/direction decomposition.

Parameters:

• r (int, optional) – LoRA rank, by default 16.

• alpha (int, optional) – LoRA scaling alpha, by default 32.

• dropout (float, optional) – LoRA dropout probability, by default 0.05.

• target_modules (list[str] | str | None, optional) – Modules to adapt, by default "all-linear".

• bias ({“none”, “all”, “lora_only”}, optional) – Bias training mode, by default "none".

__slots__ = ()

alpha: int = 32

apply(model: Any) → Any

Attach DoRA adapters to model.

Parameters:

model (Any) – Hugging Face causal LM.

Returns:

Any – PEFT-wrapped DoRA model.

bias: BiasType = 'none'

dropout: float = 0.05

load_kwargs() → dict[str, Any]

Return empty load kwargs.

Returns:

dict[str, Any] – Empty mapping.

name: str = 'dora'

r: int = 16

target_modules: list[str] | str | None = 'all-linear'

class slicktune.FitResult

Result of a completed Tuner.fit() call.

Parameters:

• output_dir (Path) – Directory containing the saved adapter / model and metrics.

• metrics (TrainingMetrics) – Collected training metrics.

• model (Any) – Trained model (PEFT or full).

• tokenizer (PreTrainedTokenizerBase) – Tokenizer used during training.

metrics: slicktune.metrics.TrainingMetrics

model: Any

output_dir: pathlib.Path

tokenizer: transformers.PreTrainedTokenizerBase

class slicktune.FullStrategy

Bases: slicktune.types.Strategy

Full fine-tuning: update all model parameters.

Parameters:

None

__slots__ = ()

apply(model: Any) → Any

Enable gradients on all parameters.

Parameters:

model (Any) – Hugging Face causal LM.

Returns:

Any – The same model with requires_grad=True on all params.

load_kwargs() → dict[str, Any]

Return empty load kwargs.

Returns:

dict[str, Any] – Empty mapping.

name: str = 'full'

class slicktune.HoldoutEvalResult

Holdout negative-log-likelihood / perplexity summary.

Parameters:

• eval_loss (float) – Mean token NLL on the holdout set.

• perplexity (float) – exp(eval_loss).

• num_examples (int) – Number of evaluated examples.

eval_loss: float

num_examples: int

perplexity: float

class slicktune.JudgeReport

Aggregate judge outcomes.

Parameters:

results (list[JudgeResult]) – Per-example judgments.

property mean_score: float

Return mean score across judged examples.

Returns:

float – Mean in [0, 1], or 0.0 when empty.

results: list[JudgeResult] = []

class slicktune.JudgeResult

Outcome of judging one generation.

Parameters:

• prompt (str) – User prompt.

• generation (str) – Model completion.

• score (float) – Score in [0, 1].

• rationale (str) – Short explanation from the judge.

generation: str

prompt: str

rationale: str

score: float

class slicktune.KTOObjective

Bases: slicktune.types.Objective

Kahneman–Tversky Optimization (TRL KTOTrainer).

Parameters:

• beta (float, optional) – KL penalty coefficient, by default 0.1.

• desirable_weight (float, optional) – Weight for desirable (label=True) examples, by default 1.0.

• undesirable_weight (float, optional) – Weight for undesirable (label=False) examples, by default 1.0.

__slots__ = ()

beta: float = 0.1

desirable_weight: float = 1.0

name: str = 'kto'

required_columns() → list[str]

Return required KTO columns.

Returns:

list[str] – Unpaired preference column names.

undesirable_weight: float = 1.0

class slicktune.LLMJudge

Bases: Judge

Ask an LLM to score a completion from 0–10, normalized to [0, 1].

Uses digit-constrained decoding so small models cannot wander into Yes / True instead of a numeric score.

Parameters:

• model (Any) – Causal LM used as the judge (often the same trained model).

• tokenizer (PreTrainedTokenizerBase) – Matching tokenizer.

• max_new_tokens (int, optional) – Unused for constrained scoring (kept for API compatibility), by default 2.

__slots__ = ()

judge(*, prompt: str, generation: str, **context: Any) → JudgeResult

Score via an LLM rubric prompt.

Parameters:

• prompt (str) – User prompt.

• generation (str) – Model completion.

• **context (Any) – Optional criteria string.

Returns:

JudgeResult – Normalized score parsed from the judge reply.

max_new_tokens: int = 2

model: Any

tokenizer: transformers.PreTrainedTokenizerBase

class slicktune.LoRAStrategy

Bases: slicktune.types.Strategy

LoRA PEFT strategy (default production PEFT path).

Parameters:

• r (int, optional) – LoRA rank, by default 16.

• alpha (int, optional) – LoRA scaling alpha, by default 32.

• dropout (float, optional) – LoRA dropout probability, by default 0.05.

• target_modules (list[str] | str | None, optional) – Modules to adapt. "all-linear" lets PEFT discover linear layers, by default "all-linear".

• bias ({“none”, “all”, “lora_only”}, optional) – Bias training mode passed to PEFT, by default "none".

__slots__ = ()

alpha: int = 32

apply(model: Any) → Any

Attach LoRA adapters to model.

Parameters:

model (Any) – Hugging Face causal LM.

Returns:

Any – PEFT-wrapped model.

bias: BiasType = 'none'

dropout: float = 0.05

load_kwargs() → dict[str, Any]

Return empty load kwargs (bf16/fp16 decided by the trainer).

Returns:

dict[str, Any] – Empty mapping; LoRA does not require special load flags.

name: str = 'lora'

r: int = 16

target_modules: list[str] | str | None = 'all-linear'

class slicktune.MetricsTracker

Collect and persist metrics across a run.

Parameters:

output_dir (str or Path) – Directory where metrics.json is written.

__post_init__() → None

Ensure output_dir is a Path and exists.

load() → TrainingMetrics

Load metrics previously written by save().

Returns:

TrainingMetrics – Restored metrics object.

Raises:

FileNotFoundError – If metrics.json is missing.

output_dir: pathlib.Path

save(metrics: TrainingMetrics) → pathlib.Path

Write metrics to metrics.json.

Parameters:

metrics (TrainingMetrics) – Metrics snapshot to persist.

Returns:

Path – Path to the written JSON file.

class slicktune.ORPOObjective

Bases: slicktune.types.Objective

Odds Ratio Preference Optimization (TRL experimental ORPO).

Parameters:

beta (float, optional) – Odds-ratio penalty coefficient, by default 0.1.

__slots__ = ()

beta: float = 0.1

name: str = 'orpo'

required_columns() → list[str]

Return required preference columns.

Returns:

list[str] – Preference triple column names (same shape as DPO).

class slicktune.QLoRAStrategy

Bases: slicktune.types.Strategy

QLoRA strategy: 4-bit quantized base + LoRA adapters.

Requires CUDA and optional extra slicktune[qlora] (bitsandbytes).

Parameters:

• r (int, optional) – LoRA rank, by default 16.

• alpha (int, optional) – LoRA scaling alpha, by default 32.

• dropout (float, optional) – LoRA dropout probability, by default 0.05.

• target_modules (list[str] | str | None, optional) – Modules to adapt, by default "all-linear".

• bias ({“none”, “all”, “lora_only”}, optional) – Bias training mode, by default "none".

__slots__ = ()

alpha: int = 32

apply(model: Any) → Any

Prepare a 4-bit model and attach LoRA adapters.

Parameters:

model (Any) – Quantized Hugging Face causal LM.

Returns:

Any – PEFT-wrapped QLoRA model.

bias: BiasType = 'none'

dropout: float = 0.05

load_kwargs() → dict[str, Any]

Return bitsandbytes 4-bit quantization config for model load.

Returns:

dict[str, Any] – quantization_config suitable for from_pretrained.

Raises:

• ImportError – If bitsandbytes is not installed.

• RuntimeError – If CUDA is not available.

name: str = 'qlora'

r: int = 16

target_modules: list[str] | str | None = 'all-linear'

class slicktune.SFTObjective

Bases: slicktune.types.Objective

Supervised fine-tuning on instruction / chat pairs.

__slots__ = ()

name: str = 'sft'

required_columns() → list[str]

Return required dataset columns for SFT.

Returns:

list[str] – Chat messages column name.

class slicktune.SubstringJudge

Bases: Judge

Deterministic judge: pass if must_contain appears in the generation.

__slots__ = ()

case_sensitive: bool = False

judge(*, prompt: str, generation: str, **context: Any) → JudgeResult

Score via substring match.

Parameters:

• prompt (str) – User prompt.

• generation (str) – Model completion.

• **context (Any) – Must include must_contain.

Returns:

JudgeResult – Score 1.0 or 0.0.

class slicktune.TrainingMetrics

Snapshot of metrics for one training run.

Parameters:

• strategy (str) – Parameter strategy name (e.g. "lora").

• objective (str) – Objective name (e.g. "sft").

• model_id (str) – Base model id or path.

• train_loss (float | None, optional) – Final reported training loss, by default None.

• eval_loss (float | None, optional) – Final evaluation loss if computed, by default None.

• train_runtime_sec (float | None, optional) – Wall-clock training time in seconds, by default None.

• train_samples_per_second (float | None, optional) – Throughput reported by the trainer, by default None.

• trainable_params (int | None, optional) – Number of trainable parameters, by default None.

• total_params (int | None, optional) – Total parameter count, by default None.

• probe_pass_rate (float | None, optional) – Fraction of probe checks that passed after training, by default None.

• eval_perplexity (float | None, optional) – Holdout perplexity from Phase-2 eval, by default None.

• judge_score (float | None, optional) – Mean judge score in [0, 1], by default None.

• extras (dict[str, Any], optional) – Additional key/value metrics, by default empty.

eval_loss: float | None = None

eval_perplexity: float | None = None

extras: dict[str, Any]

judge_score: float | None = None

model_id: str

objective: str

probe_pass_rate: float | None = None

strategy: str

total_params: int | None = None

train_loss: float | None = None

train_runtime_sec: float | None = None

train_samples_per_second: float | None = None

trainable_params: int | None = None

property trainable_percent: float | None

Return trainable parameters as a percent of total.

Returns:

float or None – Percent trainable, or None if counts are missing.

class slicktune.Tuner

Composable fine-tuning entry point.

Parameters:

• model_id (str) – Hugging Face model id or local path.

• strategy (Strategy) – Parameter-update strategy (LoRA, DoRA, AdaLoRA, QLoRA, full, …).

• objective (Objective) – Training objective (SFT, DPO, ORPO, or KTO).

• output_dir (str or Path) – Where checkpoints, adapter weights, and metrics are written.

• max_seq_length (int, optional) – Maximum sequence length, by default 512.

• num_train_epochs (float, optional) – Number of epochs, by default 3.0.

• per_device_train_batch_size (int, optional) – Batch size per device, by default 1.

• gradient_accumulation_steps (int, optional) – Gradient accumulation steps, by default 4.

• learning_rate (float, optional) – Learning rate, by default 2e-4.

• logging_steps (int, optional) – Logging frequency, by default 1.

• save_steps (int, optional) – Checkpoint frequency, by default 50.

• seed (int, optional) – Random seed, by default 42.

• eval_data (str or Path or Dataset or None, optional) – Optional holdout SFT JSONL/dataset for perplexity after fit.

• probe_path (str or Path or None, optional) – Optional probe JSONL judged after fit (substring or custom judge).

• judge (Judge or None, optional) – Judge used with probe_path; defaults to SubstringJudge.

eval_data: str | pathlib.Path | datasets.Dataset | None = None

fit(data: str | pathlib.Path | datasets.Dataset) → FitResult

Run fine-tuning for the configured objective.

Parameters:

data (str or Path or Dataset) – Path to objective-specific JSONL, or an in-memory dataset.

Returns:

FitResult – Trained artifacts and metrics.

Raises:

• TypeError – If the objective is not supported.

• ValueError – If required dataset columns are missing.

gradient_accumulation_steps: int = 4

judge: slicktune.eval.Judge | None = None

learning_rate: float = 0.0002

logging_steps: int = 1

max_seq_length: int = 512

model_id: str

num_train_epochs: float = 3.0

objective: slicktune.types.Objective

output_dir: str | pathlib.Path

per_device_train_batch_size: int = 1

probe_path: str | pathlib.Path | None = None

save_steps: int = 50

seed: int = 42

strategy: slicktune.types.Strategy

slicktune.__version__

slicktune.compute_holdout_perplexity(*, model: Any, tokenizer: transformers.PreTrainedTokenizerBase, data: str | pathlib.Path | datasets.Dataset, max_length: int = 512) → HoldoutEvalResult

Compute mean NLL and perplexity on a holdout SFT JSONL / dataset.

Parameters:

• model (Any) – Causal LM (base or PEFT).

• tokenizer (PreTrainedTokenizerBase) – Tokenizer.

• data (str or Path or Dataset) – Holdout SFT data with messages (or loadable JSONL).

• max_length (int, optional) – Truncation length, by default 512.

Returns:

HoldoutEvalResult – Loss / perplexity summary.

Raises:

ValueError – If no examples are available.

slicktune.run_judge_on_probes(*, model: Any, tokenizer: transformers.PreTrainedTokenizerBase, probe_path: str | pathlib.Path, judge: Judge, max_new_tokens: int = 128) → JudgeReport

Generate replies for probe prompts and score them with judge.

Parameters:

• model (Any) – Model under evaluation.

• tokenizer (PreTrainedTokenizerBase) – Tokenizer.

• probe_path (str or Path) – Probe JSONL (prompt, must_contain).

• judge (Judge) – Scoring strategy.

• max_new_tokens (int, optional) – Generation length, by default 128.

Returns:

JudgeReport – Aggregate scores.