Book
LLM Systems Engineering
Seven chapters on the engineering underneath applied AI: the roofline and KV-cache arithmetic that explain every serving decision, quantisation with the evidence, serving stacks and the prompt-cache traps that cost real money, retrieval internals and whether long context replaced RAG, agents at the token level, evaluation with actual statistics, and what won in post-training.
The technical companion to AI from First Principles. Every claim carries its source quality — measured study, vendor documentation, or flagged as unverified.
CHAPTER 1
Inference economics: one calculation that explains everything
Prefill and decode are different machines. Once you can compute the roofline and the KV cache — the model’s running notes on the conversation so far — every serving decision follows.
CHAPTER 2
Quantisation, with the evidence
Which formats are actually lossless, why the kernel matters more than the algorithm, and why perplexity hides the damage.
CHAPTER 3
Serving stacks, and prompt caching
Five engines and what each is actually for, plus the caching mechanism where most production money is quietly lost.
CHAPTER 4
Retrieval, technically
Index algorithms and their real trade-offs, what the chunking evidence actually says, and whether long context has replaced RAG. It has not, and the reason is measurable.
CHAPTER 5
Agents, at the token level
How tool calling actually works, what MCP became in 2026, the measured failure taxonomy, and a bug where a correctly-configured agent silently stops calling tools.
CHAPTER 6
Evaluation, with statistics
Judge biases with measured effect sizes, the sample size you actually need, and the result that should make you distrust every agent leaderboard.
CHAPTER 7
Fine-tuning, and what actually won
LoRA mechanics and the rank trade-off, the evidence on when tuning beats retrieval, and how reinforcement learning modularised into three methods with different jobs.
CHAPTER 8
Managed agent runtimes — what you are actually buying
AWS, Azure and Google all shipped one inside twelve months, and they solve narrower problems than the marketing implies.
CHAPTER 9
AgentCore in practice — the contract, the code, the limits
What your container must actually do, how it is deployed and configured, and the numbers to know before you commit.
CHAPTER 10
Running an open-weight model yourself
Where to get one, what the files are, what hardware it needs, and the five genuinely different ways to change its behaviour.
CHAPTER 11
A fine-tune, start to finish
One real task taken all the way — the data, the baseline, the training script, the loss curve, the evaluation, and serving it.
CHAPTER 12
A RAG system, start to finish
The two pipelines, a working system in forty lines, the tools you would actually choose between, and the two upgrades that move the needle.
CHAPTER 13
RAG in the enterprise
Permissions, real data sources, keeping the index honest, and an unsentimental look at building versus buying.
CHAPTER 14
Cost optimisation, and the interview questions about it
The levers in the order you should pull them, the caching mechanics in detail, and the reasoning that separates a good answer from a list of tips.
CHAPTER 15
An eval suite, start to finish
Where the cases come from, the five kinds of assertion, a runnable pytest suite, how to use a model as a judge without fooling yourself, and how to run it in CI.
CHAPTER 16
Building an agent, start to finish
The loop in twenty lines, why tool design is the whole skill, how to stop it running away, and the honest case against multi-agent.
CHAPTER 17
Context engineering
The window is a budget, not a container — what competes for it, the four things you can do about it, and why more context is reliably worse.
CHAPTER 18
Harness engineering
The layer around the model — what the term means, what has actually been measured, the framework worth stealing, and how it is being adopted.
CHAPTER 19
Loop engineering
A term about three months old, what it actually names, how it differs from the harness, and why its author's warnings are the most useful part.
CHAPTER 20
Enterprise agent platforms
The thing banks are building — where business users describe a process and get an agent — what it is architecturally, and why most pilots do not reach production.