Software Engineering · Chapter 29 of 31
Prompting, for people who have to ship
What actually works, what is cargo cult, where to put instructions so a coding agent reads them, and how prompt shape drives your bill.
Most prompting advice is folklore that was true of one model in one year. This chapter is the part that has held.
What actually works
- Be specific about the output, not the persona. ‘Return a JSON object with keys
categoryandconfidence’ does real work. ‘You are a world-class expert’ does almost none on a current model. - Give it an approved way to fail. If this single line were the only thing you took from the chapter it would be worth it. A model with no permitted way to say ‘I don't know’ will produce something plausible instead, because that is what it was optimised to do.
- Show, don't describe. Two worked examples beat two paragraphs about what you want. This is the highest-leverage technique on the list and the most consistently skipped.
- Put the reference material before the question. Long document then question, not question then document — better for the model, and it is what makes the long part cacheable.
- Say what to do at the edges. Empty input, ambiguous input, conflicting sources. Most production prompt failures are unspecified edge cases, not bad phrasing.
- Use structured output when you need structure. A schema the API enforces beats asking nicely for JSON, and it removes a whole class of parsing failure.
What is cargo cult now
Things that were once good advice and have aged badly
“Think step by step.” This was a genuine 2022 discovery. Current reasoning models already do it, and on some the instruction is counterproductive — you are asking for a behaviour that is already on, in a way that can interfere with it. Use the model's own effort or thinking control instead.
Elaborate role-play preambles. ‘You are a senior engineer with twenty years of experience’ buys very little now. What used to be personality is now capability.
Threats, bribes, and emotional pressure. There was a period when ‘this is very important to my career’ measurably helped. Treat any such trick as a temporary artefact of one model generation.
Very long prompts as a default. Length is not thoroughness. A 2,000-word prompt with three contradictory instructions performs worse than a 200-word one, and costs more every single request.
The general lesson: prompt tricks are model-specific and they expire. Prompt structure — context, task, constraints, format, examples — has not.
Coding assistants specifically
Claude Code, Codex and GitHub Copilot all read a project file for standing instructions, and this is the single highest-return thing you can do in a repository. You write it once; every session in that repo gets it.
| Tool | File it reads | Scope |
|---|---|---|
| Claude Code | CLAUDE.md at the repo root (and nested, per directory) | Read at session start; nested files apply to their subtree |
| Codex | AGENTS.md | Same idea; the name is becoming a cross-tool convention |
| Copilot | .github/copilot-instructions.md | Applies across the repository |
# CLAUDE.md -- what actually belongs in one
## Commands
- Test: pytest -q
- Lint: ruff check .
- Typecheck: mypy src/
Run all three before saying a change is done.
## Conventions
- Python 3.12, type hints on every public function.
- Errors: raise, do not return None. We have no null-return convention.
- Tests live beside the code as test_*.py, not in a tests/ tree.
## Do not touch
- src/legacy/ -- frozen, migrating away, changes need an ADR first.
- Any *.generated.py -- regenerate with `make codegen` instead.
## Context you cannot infer from the code
- `Account.status` has five values but only three are reachable in prod;
ARCHIVED and PENDING_MERGE are historical. Do not write logic for them.
- The retry decorator on payments is deliberately NOT idempotent-safe;
ask before changing it, there is a ticket explaining why.The most valuable section is the last one, and almost nobody writes it.
Commands and conventions are things an agent can partly infer by reading the repository. What it cannot infer is the history — which fields are dead, which ugly code is deliberate, which module is being migrated away from.
That is exactly the knowledge a new human hire gets in their first month from conversations. Writing it down is the highest-value thing in the file, and it helps the humans too.
Review it like code and keep it short. A 400-line instructions file is read every session, competes with the actual task for attention, and costs tokens on every request.
Prompting an agent versus prompting a chat
A distinction worth making explicitly, because it changes what a good prompt looks like. In a chat you are asking for text and you will read it. With an agent you are delegating a task it will carry out over many turns, and you may not read every step.
- State the goal and the acceptance criterion, not the steps. ‘Make the failing tests in
test_billing.pypass without changing the tests’ is a far better instruction than a list of things to try — it lets the agent verify itself. - Say what is out of bounds. Which files not to touch, whether it may install packages, whether it may change the schema.
- Give it a way to verify — the test command, the build command. An agent that can check its own work behaves entirely differently from one that cannot, and this is the biggest single lever on agent quality.
- Ask for a plan first on anything large. Reviewing a plan costs two minutes; reviewing a wrong 900-line diff costs an afternoon. This is the same argument as spec-driven development, at the scale of a single task.
Personally, and at work
Personally: keep the handful of prompts you actually reuse in a file rather than retyping them. Get specific about output format — most disappointment is a shape mismatch, not a capability gap. And when a result is wrong, fix the prompt rather than arguing with the model in follow-ups; the follow-up fixes one answer, the prompt fixes every future one.
At work, the shift is that prompts become code:
- Prompts belong in version control, not pasted into a dashboard. They change behaviour, so they need review, history and the ability to roll back.
- Every prompt change needs an evaluation run. A prompt edit is a behaviour change with no compiler and no type system. Without a test set, you are shipping untested code and calling it copy-editing.
- Separate the stable part from the per-request part, deliberately. It is cleaner, and it is what makes caching work.
- Never build a prompt by string-concatenating user input into the middle of your instructions. Keep instructions and untrusted content in separate blocks, and say in the instructions that the content is data. See prompt injection.
Prompt shape is a cost decision
This is where prompting stops being a writing skill. You pay per token, on input and output, on every request — and in a conversation the whole history is resent each turn.
- Order for the cache. Stable content first, volatile last. A cached prefix costs roughly a tenth of a fresh one. Reordering a prompt so the timestamp moves from the top to the bottom can cut the bill by most of it, and changes nothing else.
- Nothing that changes per request goes early. A date, a request identifier, or a username in the system prompt invalidates everything after it, every time.
- Ask for less output. Output tokens cost several times input. ‘Answer in one sentence’ is a real cost control, and it is usually also a better answer.
- Trim the context, do not pad it. Twenty retrieved chunks instead of five costs four times as much and usually gives a worse answer, because the relevant passage gets buried.
- Long prompts cost forever. A 500-token instruction block is paid on every single call for the life of the feature. It is worth twenty minutes to cut it in half.
The cost chapter has the arithmetic, the caching mechanics, and the traps that silently stop caching working.
Check yourself