slicktune.data

Dataset loaders for fine-tuning objectives.

Attributes

DEFAULT_SYSTEM_PROMPT

Functions

load_kto_jsonl(→ datasets.Dataset)	Load a KTO unpaired-preference JSONL file.
load_preference_jsonl(→ datasets.Dataset)	Load a DPO/ORPO preference JSONL file.
load_probe_jsonl(→ list[dict[str, str]])	Load probe questions used to verify fine-tuning worked.
load_sft_jsonl(→ datasets.Dataset)	Load an SFT JSONL file into a Hugging Face Dataset.

Package Contents

slicktune.data.DEFAULT_SYSTEM_PROMPT = 'You answer questions about Amirhessam Tahmassebi, SlickML, and slick-tune accurately and concisely.'

slicktune.data.load_kto_jsonl(path: str | pathlib.Path) → datasets.Dataset

Load a KTO unpaired-preference JSONL file.

Parameters:

path (str or Path) – JSONL with prompt, completion, and boolean label per line.

Returns:

datasets.Dataset – Dataset with KTO columns.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If the file is empty or a row is invalid.

slicktune.data.load_preference_jsonl(path: str | pathlib.Path) → datasets.Dataset

Load a DPO/ORPO preference JSONL file.

Parameters:

path (str or Path) – JSONL with prompt, chosen, and rejected per line.

Returns:

datasets.Dataset – Dataset with string preference columns.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If the file is empty or a row is invalid.

slicktune.data.load_probe_jsonl(path: str | pathlib.Path) → list[dict[str, str]]

Load probe questions used to verify fine-tuning worked.

Parameters:

path (str or Path) – JSONL with prompt and must_contain (substring checks).

Returns:

list[dict[str, str]] – Probe rows with prompt and must_contain.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If a row is missing required keys.

slicktune.data.load_sft_jsonl(path: str | pathlib.Path) → datasets.Dataset

Load an SFT JSONL file into a Hugging Face Dataset.

Parameters:

path (str or Path) – Path to a .jsonl file. Each line is one training example.

Returns:

datasets.Dataset – Dataset with a messages column.

Raises:

• FileNotFoundError – If path does not exist.

• ValueError – If the file is empty or a row is invalid.