slicktune.objectives

Training objectives (what the model learns).

Classes

DPOObjective	Direct Preference Optimization (TRL DPOTrainer).
KTOObjective	Kahneman–Tversky Optimization (TRL KTOTrainer).
ORPOObjective	Odds Ratio Preference Optimization (TRL experimental ORPO).
SFTObjective	Supervised fine-tuning on instruction / chat pairs.

Package Contents

class slicktune.objectives.DPOObjective

Bases: slicktune.types.Objective

Direct Preference Optimization (TRL DPOTrainer).

Parameters:

• beta (float, optional) – KL penalty coefficient, by default 0.1.

• loss_type (str, optional) – TRL DPO loss type, by default "sigmoid".

__slots__ = ()

beta: float = 0.1

loss_type: str = 'sigmoid'

name: str = 'dpo'

required_columns() → list[str]

Return required preference columns.

Returns:

list[str] – Preference triple column names.

class slicktune.objectives.KTOObjective

Bases: slicktune.types.Objective

Kahneman–Tversky Optimization (TRL KTOTrainer).

Parameters:

• beta (float, optional) – KL penalty coefficient, by default 0.1.

• desirable_weight (float, optional) – Weight for desirable (label=True) examples, by default 1.0.

• undesirable_weight (float, optional) – Weight for undesirable (label=False) examples, by default 1.0.

__slots__ = ()

beta: float = 0.1

desirable_weight: float = 1.0

name: str = 'kto'

required_columns() → list[str]

Return required KTO columns.

Returns:

list[str] – Unpaired preference column names.

undesirable_weight: float = 1.0

class slicktune.objectives.ORPOObjective

Bases: slicktune.types.Objective

Odds Ratio Preference Optimization (TRL experimental ORPO).

Parameters:

beta (float, optional) – Odds-ratio penalty coefficient, by default 0.1.

__slots__ = ()

beta: float = 0.1

name: str = 'orpo'

required_columns() → list[str]

Return required preference columns.

Returns:

list[str] – Preference triple column names (same shape as DPO).

class slicktune.objectives.SFTObjective

Bases: slicktune.types.Objective

Supervised fine-tuning on instruction / chat pairs.

__slots__ = ()

name: str = 'sft'

required_columns() → list[str]

Return required dataset columns for SFT.

Returns:

list[str] – Chat messages column name.