This comes up in interviews constantly, and most answers are a shopping list of techniques. What distinguishes a good answer is the order and the reasoning — which levers are free, which cost you quality, and how you would know whether any of it worked.

• Input tokens — everything you send: system prompt, tools, retrieved context, the conversation so far, the question.
• Output tokens — what comes back. Priced several times higher than input on every model.
• And the fact that surprises people: the API is stateless. In a ten-turn conversation, turn ten resends turns one through nine. Cost does not grow linearly with conversation length — it grows roughly with the square of it. This is the single biggest driver of a runaway agent bill, and it is invisible if you only look at per-request cost.

Anthropic's current published rates, as an example of the shape: Opus 5 at $5 input / $25 output per million tokens, Sonnet 5 at $2 / $10, Haiku 4.5 at $1 / $5. Note that the spread between tiers is about five times, while the levers below routinely save more than that without changing model at all — which is why ‘use a cheaper model’ is a weak first answer.

Caching lets the provider keep the processed form of a repeated prefix so it does not have to reprocess it. The economics, using Anthropic's published multipliers: writing to the cache costs 1.25× a normal input token for a five-minute lifetime (or 2× for an hour), and reading from it costs 0.1×.

Work that through, because it sets the break-even. You pay a 25% premium once and then pay 10% forever. You are ahead from the second request onwards, and by the tenth you have paid roughly a fifth of what you would have. For a chatbot with a long system prompt, or a retrieval system pasting the same documents repeatedly, this is the difference between a viable product and an unviable one.

# Structure for the cache: stable first, volatile last.
response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    system=[{
        "type": "text",
        "text": STABLE_INSTRUCTIONS + POLICY_DOCUMENTS,   # never changes
        "cache_control": {"type": "ephemeral"},           # <-- breakpoint here
    }],
    messages=[
        # everything volatile lives AFTER the breakpoint
        {"role": "user", "content": f"Today is {today}. {user_question}"},
    ],
)

# The one line that tells you whether any of this is working:
print(response.usage.cache_read_input_tokens)      # ~0.1x cost -- want this large
print(response.usage.cache_creation_input_tokens)  # ~1.25x cost -- want this rare
print(response.usage.input_tokens)                 # full cost -- want this small

50% off both input and output, in exchange for asynchronous processing with results typically back within a few hours. It applies to work nobody is waiting for: nightly classification, bulk summarisation, backfills, evaluation runs, report generation.

And it stacks with caching. A batched request against a cached prefix pays half of a tenth. The reason to name this in an interview is that most people treat these as alternatives rather than as multiplicative — and the honest follow-up is that far more workloads are latency-tolerant than teams assume. ‘Real-time’ is usually a habit, not a requirement.

6. Lower the effort before you change the model. Current models expose an effort or thinking control that trades thoroughness against tokens within one model. This matters more than it sounds:

7. A smaller model, per route. Classification, routing, extraction, reformatting and simple summarisation rarely need a frontier model. Per route, not globally — and measured on your real traffic, not on the vendor's benchmark.

8. Routing and cascades. Run a cheap model first and escalate when confidence is low or the output fails a check. Genuinely effective, and subject to the caching caveat above. The hard part is not the routing; it is the confidence signal, and a bad one gives you both bills.

9. Fine-tune a small model. Only for a narrow, high-volume, stable task where the cost of the tuning run amortises. See the worked fine-tune — and note that the base model will be superseded within a year and your adapter will not transfer, so put that in the arithmetic.

• Retrieval versus long context. Pasting a whole 200-page manual into every request is enormously more expensive than retrieving the three relevant pages — usually a hundredfold — and it typically produces worse answers, because the relevant passage is buried. Retrieval is a cost optimisation and a quality improvement at the same time, which is rare enough to be worth saying out loud.
• Agent loop discipline. An agent that re-reads the same files every turn pays for them every turn. Context editing, compaction and simply not putting large tool results back into history are where agent bills are actually won.
• Do not call a model at all. The cheapest token is the one you never send. A regular expression, a lookup table or a cached previous answer handles a startling share of production traffic, and a semantic cache — returning a stored answer when a new question is close enough in meaning to an old one — catches the repeats that exact-match caching misses.
• Cap the output. A sensible max_tokens and an instruction to be brief. Runaway generations are a real cost line.

This is the part that separates a strong answer from a list, and it is where most candidates stop too early.

• Log token counts per request, split into cached, uncached and output, and tag them by route. Without the split you cannot tell a caching problem from a context problem.
• Track cost per completed task, not per request. A cheaper model that needs three attempts to get a usable answer is more expensive, and per-request metrics report it as a saving. This single reframing is the most useful sentence in the chapter.
• Have an evaluation set before you optimise, so a quality regression from step 6 or 7 is a number rather than a complaint three weeks later.
• Attribute cost to a feature or team, not one line item called ‘LLM API’. You cannot make a decision about a bill you cannot decompose.

The one-sentence version, if you only have a minute: take every free win first — caching, input and output hygiene, batching — because they change the bill without changing the output; only then trade quality, starting with effort rather than model; and measure cost per completed task, because per-request cost will happily tell you a lie.