Lakshya

Software Engineering · Chapter 29 of 31

Prompting, for people who have to ship

What actually works, what is cargo cult, where to put instructions so a coding agent reads them, and how prompt shape drives your bill.

9 min read1 diagramAll 31 chapters

Most prompting advice is folklore that was true of one model in one year. This chapter is the part that has held.

THE FIVE PARTS — AND THE ORDER IS NOT ARBITRARY 1. CONTEXTWhat the model needs to know that it cannot infer.The stable part — put it FIRST, because that is what gets cached.2. THE TASKOne clear instruction. Not three bundled into a paragraph.If you cannot state it in a sentence, you have not decided what you want.3. CONSTRAINTSWhat it must NOT do, and what to do when it cannot.“If the answer is not in the context, say so” — the highest-value line you can write.4. FORMATThe shape of the output you can actually consume.Better still: use a structured-output schema and stop asking politely.5. EXAMPLESOne to three worked examples, if the task is at all subtle.Worth more than any amount of describing what you want. STABLE FIRST, VOLATILE LAST. This ordering is worth real money — see the cost chapter. Caching is a PREFIX match, so anything that changes per request belongs at the END, after everything reusable.
Note the last box. The ordering is not a style preference — it is the difference between a request that hits cache and one that does not.

What actually works

  • Be specific about the output, not the persona. ‘Return a JSON object with keys category and confidence’ does real work. ‘You are a world-class expert’ does almost none on a current model.
  • Give it an approved way to fail. If this single line were the only thing you took from the chapter it would be worth it. A model with no permitted way to say ‘I don't know’ will produce something plausible instead, because that is what it was optimised to do.
  • Show, don't describe. Two worked examples beat two paragraphs about what you want. This is the highest-leverage technique on the list and the most consistently skipped.
  • Put the reference material before the question. Long document then question, not question then document — better for the model, and it is what makes the long part cacheable.
  • Say what to do at the edges. Empty input, ambiguous input, conflicting sources. Most production prompt failures are unspecified edge cases, not bad phrasing.
  • Use structured output when you need structure. A schema the API enforces beats asking nicely for JSON, and it removes a whole class of parsing failure.

What is cargo cult now

Things that were once good advice and have aged badly

“Think step by step.” This was a genuine 2022 discovery. Current reasoning models already do it, and on some the instruction is counterproductive — you are asking for a behaviour that is already on, in a way that can interfere with it. Use the model's own effort or thinking control instead.

Elaborate role-play preambles. ‘You are a senior engineer with twenty years of experience’ buys very little now. What used to be personality is now capability.

Threats, bribes, and emotional pressure. There was a period when ‘this is very important to my career’ measurably helped. Treat any such trick as a temporary artefact of one model generation.

Very long prompts as a default. Length is not thoroughness. A 2,000-word prompt with three contradictory instructions performs worse than a 200-word one, and costs more every single request.

The general lesson: prompt tricks are model-specific and they expire. Prompt structure — context, task, constraints, format, examples — has not.

Coding assistants specifically

Claude Code, Codex and GitHub Copilot all read a project file for standing instructions, and this is the single highest-return thing you can do in a repository. You write it once; every session in that repo gets it.

ToolFile it readsScope
Claude CodeCLAUDE.md at the repo root (and nested, per directory)Read at session start; nested files apply to their subtree
CodexAGENTS.mdSame idea; the name is becoming a cross-tool convention
Copilot.github/copilot-instructions.mdApplies across the repository
markdownthe file pays for itself the first time it prevents one wrong assumption
# CLAUDE.md -- what actually belongs in one

## Commands
- Test:      pytest -q
- Lint:      ruff check .
- Typecheck: mypy src/
Run all three before saying a change is done.

## Conventions
- Python 3.12, type hints on every public function.
- Errors: raise, do not return None. We have no null-return convention.
- Tests live beside the code as test_*.py, not in a tests/ tree.

## Do not touch
- src/legacy/ -- frozen, migrating away, changes need an ADR first.
- Any *.generated.py -- regenerate with `make codegen` instead.

## Context you cannot infer from the code
- `Account.status` has five values but only three are reachable in prod;
  ARCHIVED and PENDING_MERGE are historical. Do not write logic for them.
- The retry decorator on payments is deliberately NOT idempotent-safe;
  ask before changing it, there is a ticket explaining why.

The most valuable section is the last one, and almost nobody writes it.

Commands and conventions are things an agent can partly infer by reading the repository. What it cannot infer is the history — which fields are dead, which ugly code is deliberate, which module is being migrated away from.

That is exactly the knowledge a new human hire gets in their first month from conversations. Writing it down is the highest-value thing in the file, and it helps the humans too.

Review it like code and keep it short. A 400-line instructions file is read every session, competes with the actual task for attention, and costs tokens on every request.

Prompting an agent versus prompting a chat

A distinction worth making explicitly, because it changes what a good prompt looks like. In a chat you are asking for text and you will read it. With an agent you are delegating a task it will carry out over many turns, and you may not read every step.

  • State the goal and the acceptance criterion, not the steps. ‘Make the failing tests in test_billing.py pass without changing the tests’ is a far better instruction than a list of things to try — it lets the agent verify itself.
  • Say what is out of bounds. Which files not to touch, whether it may install packages, whether it may change the schema.
  • Give it a way to verify — the test command, the build command. An agent that can check its own work behaves entirely differently from one that cannot, and this is the biggest single lever on agent quality.
  • Ask for a plan first on anything large. Reviewing a plan costs two minutes; reviewing a wrong 900-line diff costs an afternoon. This is the same argument as spec-driven development, at the scale of a single task.

Personally, and at work

Personally: keep the handful of prompts you actually reuse in a file rather than retyping them. Get specific about output format — most disappointment is a shape mismatch, not a capability gap. And when a result is wrong, fix the prompt rather than arguing with the model in follow-ups; the follow-up fixes one answer, the prompt fixes every future one.

At work, the shift is that prompts become code:

  • Prompts belong in version control, not pasted into a dashboard. They change behaviour, so they need review, history and the ability to roll back.
  • Every prompt change needs an evaluation run. A prompt edit is a behaviour change with no compiler and no type system. Without a test set, you are shipping untested code and calling it copy-editing.
  • Separate the stable part from the per-request part, deliberately. It is cleaner, and it is what makes caching work.
  • Never build a prompt by string-concatenating user input into the middle of your instructions. Keep instructions and untrusted content in separate blocks, and say in the instructions that the content is data. See prompt injection.

Prompt shape is a cost decision

This is where prompting stops being a writing skill. You pay per token, on input and output, on every request — and in a conversation the whole history is resent each turn.

  • Order for the cache. Stable content first, volatile last. A cached prefix costs roughly a tenth of a fresh one. Reordering a prompt so the timestamp moves from the top to the bottom can cut the bill by most of it, and changes nothing else.
  • Nothing that changes per request goes early. A date, a request identifier, or a username in the system prompt invalidates everything after it, every time.
  • Ask for less output. Output tokens cost several times input. ‘Answer in one sentence’ is a real cost control, and it is usually also a better answer.
  • Trim the context, do not pad it. Twenty retrieved chunks instead of five costs four times as much and usually gives a worse answer, because the relevant passage gets buried.
  • Long prompts cost forever. A 500-token instruction block is paid on every single call for the life of the feature. It is worth twenty minutes to cut it in half.

The cost chapter has the arithmetic, the caching mechanics, and the traps that silently stop caching working.

Check yourself

What are the five parts of a prompt?Context, the task, constraints, format, examples — with stable content first and volatile content last, because caching is a prefix match.
What is the highest-value single line?Telling the model what to do when it cannot answer. Without a permitted way to say 'I don't know', it produces something plausible instead.
Why is 'think step by step' now cargo cult?Current reasoning models already do it, and on some the instruction interferes. Use the model's own effort or thinking control instead.
What is the general lesson about prompt tricks?Tricks are model-specific and expire. Structure — context, task, constraints, format, examples — has not.
Where do standing instructions go?CLAUDE.md for Claude Code, AGENTS.md for Codex, .github/copilot-instructions.md for Copilot.
What is the most valuable section of that file?The context you cannot infer from the code — dead fields, deliberate ugliness, modules being migrated away from. It is what a human hire learns in their first month from conversations.
How does prompting an agent differ?State the goal and acceptance criterion rather than steps, say what is out of bounds, and give it a way to verify its own work — that last one is the biggest lever on agent quality.
Why do prompts belong in version control?They change behaviour, so they need review, history and rollback — and every change needs an evaluation run, because a prompt edit is a behaviour change with no compiler.
How does prompt order affect cost?Caching is a prefix match, so anything volatile placed early invalidates everything after it. Moving a timestamp from the top to the bottom can cut most of the bill.
← Factory, Blitzy, and buying at the top of the spectrumAPIs, webhooks and MCP →