One chapter covers whether to fine-tune at all; another covers the mechanics and the evidence. This one is a single run, all the way through, with the code.

Turning free-text support emails into a strict structure. A customer writes a paragraph; we need a category, an urgency, a product and the action required, in exactly the same JSON shape every time so a downstream system can act on it.

This fits the profile from the decision table: it is narrow, the labels are consistent, and what we need taught is form and taxonomy, not facts. A good prompt gets most of the way and fails on consistency — roughly one response in fifteen adds a field, renames a key or wraps the JSON in prose. At scale that is the whole problem, and it is precisely what fine-tuning fixes.

Training data is JSONL — JSON Lines, one complete JSON object per line, no commas between them. Each line is one worked example in the chat format the model was trained on.

{"messages": [{"role": "system", "content": "Extract the ticket fields. Reply with JSON only."}, {"role": "user", "content": "Hi - the card reader at our Andheri branch has been showing E-04 since this morning and we cannot take payments. Three tills affected. Please help urgently."}, {"role": "assistant", "content": "{\"category\": \"hardware_fault\", \"urgency\": \"high\", \"product\": \"card_reader\", \"action\": \"dispatch_engineer\"}"}]}
{"messages": [{"role": "system", "content": "Extract the ticket fields. Reply with JSON only."}, {"role": "user", "content": "Could someone tell me how to add a second user to our account? No rush."}, {"role": "assistant", "content": "{\"category\": \"how_to\", \"urgency\": \"low\", \"product\": \"account_admin\", \"action\": \"send_documentation\"}"}]}

• The system message must be identical in every row, and identical to what you will send in production. The model learns the mapping from this input shape to that output shape. Change the shape at serving time and you have quietly changed the task.
• Only the assistant turn is learned from. The trainer masks the prompt so the model is not being taught to generate customer emails.
• How many rows: 300–500 is enough to see whether the idea works. 1,000–3,000 is where a narrow task like this usually lands. Beyond that, returns fall off sharply unless the extra rows cover cases the first thousand did not — and adding more of the same is the commonest way to spend a week for nothing.
• Split it three ways before you do anything else: train, validation (used during training to spot overfitting), and a test set you do not look at until the end. Roughly 80/10/10.

# Check the dataset before you spend money on it. Takes seconds.
import json, collections

rows = [json.loads(l) for l in open("train.jsonl")]
print(f"{len(rows)} rows")

systems, labels, bad = set(), collections.Counter(), 0
for i, r in enumerate(rows):
    systems.add(r["messages"][0]["content"])
    try:
        out = json.loads(r["messages"][-1]["content"])
        labels[(out["category"], out["urgency"])] += 1
    except Exception as e:
        bad += 1
        print(f"  row {i}: assistant turn is not valid JSON -- {e}")

assert len(systems) == 1, f"{len(systems)} different system prompts -- must be exactly 1"
assert bad == 0, f"{bad} rows have an unparseable target"

for k, n in labels.most_common():
    flag = "  <-- too rare to learn" if n < 10 else ""
    print(f"{n:5d}  {k}{flag}")

That last line matters. A label with three examples will not be learned; it will be ignored, or worse, absorbed into a neighbouring label. Either collect more of it or merge it into another category deliberately — but decide, rather than discovering it in production.

Before training anything, measure how well a good prompt does on your test set. This is the number the fine-tune has to beat, and producing it costs an afternoon and no GPU.

import json
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="unused")
test = [json.loads(l) for l in open("test.jsonl")]

exact, parseable = 0, 0
for r in test:
    reply = client.chat.completions.create(
        model="qwen3:8b",
        messages=r["messages"][:-1],      # everything except the answer
        temperature=0,
    ).choices[0].message.content

    try:
        got = json.loads(reply)
        parseable += 1
        if got == json.loads(r["messages"][-1]["content"]):
            exact += 1
    except json.JSONDecodeError:
        pass

n = len(test)
print(f"BASELINE  valid JSON {parseable/n:.1%}   exact match {exact/n:.1%}")
# BASELINE  valid JSON 93.0%   exact match 71.5%

Two metrics, deliberately. Valid JSON measures whether the output is usable at all — the consistency problem. Exact match measures whether it is right. Fine-tuning typically moves the first one dramatically and the second one modestly, and knowing which you actually needed tells you whether the project succeeded.

This is LoRA: the original weights are frozen and a small set of new parameters is trained alongside them. The mechanics chapter covers why that works; here is what running it looks like.

from datasets import load_dataset
from peft import LoraConfig
from trl import SFTTrainer, SFTConfig

data = load_dataset("json", data_files={"train": "train.jsonl",
                                        "validation": "val.jsonl"})

peft_config = LoraConfig(
    r=16,                    # adapter capacity. 8-16 for style/format, 32-64 to teach more
    lora_alpha=32,           # keep alpha = 2r so changing r doesn't also change the step size
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules="all-linear",   # safer default than hand-picking layers
)

args = SFTConfig(
    output_dir="ticket-extractor",
    num_train_epochs=3,                    # 1-3. If you are reaching for 10, see the loss curve
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,         # effective batch 16 without the memory for it
    learning_rate=2e-4,                    # LoRA wants ~10x a full fine-tune's rate
    lr_scheduler_type="cosine",
    warmup_ratio=0.03,
    logging_steps=10,
    eval_strategy="steps",                 # <-- without this you are flying blind
    eval_steps=50,
    save_strategy="steps",
    save_steps=50,
    load_best_model_at_end=True,           # <-- keeps the best checkpoint, not the last
    bf16=True,
    max_length=2048,
)

trainer = SFTTrainer(
    model="Qwen/Qwen3-8B",
    args=args,
    train_dataset=data["train"],
    eval_dataset=data["validation"],
    peft_config=peft_config,
)
trainer.train()
trainer.save_model()          # writes ONLY the adapter -- a few megabytes

Loss is simply how wrong the model's predictions are — lower is better. You get two numbers. Training loss is measured on the data it is learning from, and it will fall no matter what, including when the model is doing nothing useful. Validation loss is measured on data it has never been shown, and it is the only one that tells you anything.

When validation loss stops falling and starts rising, the model has stopped learning the pattern and started memorising the specific rows. Everything after that point makes it worse at the actual job while looking better in the logs — which is why load_best_model_at_end is in the script above.

Run the exact baseline script again, pointing at the tuned model, on the test set you have not looked at.

BASELINE   valid JSON  93.0%   exact match  71.5%
FINE-TUNED valid JSON  99.8%   exact match  84.2%

# Now the check almost nobody runs -- did it forget how to do anything else?
# Ask it 20 general questions unrelated to tickets and read the answers.
#
# A model that answers "what is the capital of France?" with
# {"category": "how_to", "urgency": "low", ...} has been over-tuned.
# It is still fine for THIS task -- but it is now a single-purpose
# component, and anyone reusing it will be surprised.

Read that result honestly. Valid JSON went from 93% to 99.8% — the consistency problem, which was the actual reason for the project, is essentially solved. Exact match improved by about 13 points, which is real but smaller. If the business case had rested on accuracy rather than consistency, this is a much weaker result than the headline suggests, and you would want to know that before presenting it.

You have two options, and they are genuinely different.

# Option A -- merge, then serve like any other model
from peft import AutoPeftModelForCausalLM
model = AutoPeftModelForCausalLM.from_pretrained("ticket-extractor")
model.merge_and_unload().save_pretrained("ticket-extractor-merged")

vllm serve ./ticket-extractor-merged

# Option B -- one base, many adapters, swapped per request
vllm serve Qwen/Qwen3-8B \
    --enable-lora \
    --lora-modules tickets=./ticket-extractor billing=./billing-extractor
# then call it with model="tickets" or model="billing"

Every major provider offers hosted fine-tuning — you upload the same JSONL, they return a model identifier you call like any other. No GPU, no library versions, no memory arithmetic.

The honest comparison. Hosted is faster to a first result and the right default for a first attempt. What you give up is real, though: you cannot download the result, so the tuned model exists only while that provider offers it; you are tied to their base model and its deprecation schedule; and per-token pricing on a tuned model is usually higher than the base. Self-hosting costs more effort and gives you an artefact you own — which matters exactly as much as your answer to whether the model must still exist in three years.

• The dataset disagreed with itself. Covered above, and it is the number one cause. The model learned your inconsistency perfectly.
• No baseline, so no verdict. The tuned model works; nobody can say whether a better prompt would have.
• Trained to teach facts. It will produce confident, fluent, wrong answers in exactly the right format. Facts belong in retrieval; this is the field's most expensive recurring mistake.
• The serving prompt drifted from the training prompt. Someone improved the system message afterwards and quality dropped for no visible reason. Pin them together.
• Too many epochs. The logs looked excellent and the model memorised the training set.
• No plan for the next base model. A better one arrives within the year and your adapter does not transfer. Budget for redoing it, or accept being pinned to an ageing base — but choose, rather than being surprised.