Lakshya

AI from First Principles · Chapter 2

The four things people call “training”

The most expensive confusion in AI governance, and entirely avoidable.

Someone says “we want to train the AI on our customer data.” Before you answer, find out which of four completely different things they mean. They differ in cost by six orders of magnitude and — the part that matters to you — in whether the data can ever be removed.

FOUR THINGS PEOPLE ALL CALL “TRAINING THE AI ON OUR DATA” 1 · Prompting seconds, pennies Changes: the request only The weights never move. Data lives only in that one call. data leaves only for that call deletion is a retention question 2 · RAG hours to set up Changes: the request, plus retrieved text Your documents sit in a search index, not in the model. your index is the crown jewel access control belongs THERE 3 · Fine-tuning days, thousands Changes: a small slice of the weights Your data is now absorbed into numbers. cannot be un-learned cleanly a deletion request is a real problem 4 · Pre-training months, millions Changes: all the weights, from nothing Essentially nobody outside a frontier lab does this. not your decision to make if someone proposes it, ask why When someone says “we trained it on our data”, ask which of these four they mean. The answer changes every control you need.
Only the bottom two change the model; the top two change the request. That distinction decides your entire control set: for prompting and RAG the data stays in systems you can govern with existing tools, while fine-tuning absorbs it into weights that cannot be selectively un-learned.

Why deletion is the sharp question

If you are subject to any right-to-erasure obligation — GDPR Article 17, India's DPDP Act, CCPA — the four options are not equivalent and only two are comfortable:

ApproachWhere personal data ends upCan you delete it?
PromptingThe request, and whatever logs captured itYes — a log retention problem you already solve
RAGA search index you controlYes — delete the document, re-index
Fine-tuningAbsorbed into weights as numbersNot cleanly. The honest answer is retrain without it
Pre-trainingThe same, at enormous scaleNo, and it is not your model anyway

This is why “can we fine-tune on customer records?” deserves a different answer from “can the assistant search customer records?” — even though to the person asking they feel identical.

The questions to ask, verbatim

  • “Are we changing the model's weights, or changing what we send it?”
  • “If a customer asks us to delete their data next year, what exactly do we do?”
  • “Where does the data physically sit between now and then?”
  • “Whose data shares that location, and what separates them?”

Fine-tuning is usually not what they need

Teams reach for fine-tuning because it sounds like the serious option. Most business problems brought to it are better solved by retrieval, because the requirement is usually know our current facts rather than adopt a behaviour. Fine-tuning teaches style, format and task shape; it is an expensive and poor way to teach facts, and the facts go stale in the weights.

So the governance-friendly option is frequently also the technically better one. That is a good position to be in, and worth saying out loud in the meeting.

← What a model actually isWhy it sounds certain when it is wrong →