LLM Systems Engineering · Chapter 18 of 20
Harness engineering
The layer around the model — what the term means, what has actually been measured, the framework worth stealing, and how it is being adopted.
A term that went from nowhere to everywhere across 2026, and one of the few pieces of new vocabulary that names something real. It is also the chapter that ties several others on this site together, because the harness is where tools, context, evaluation, guardrails and observability stop being separate topics.
The definition, and it is unusually clean
Agent = Model + Harness
The harness is everything supporting the model except the model itself — the loop, which tools are exposed and how they are described, what context is carried and what is dropped, the guardrails, the verification, the observability.
It is the third phase of a maturity arc people now describe as prompt engineering → context engineering → harness engineering. Each phase widened the unit of design: from the words, to what is in the window, to the whole runtime around the model.
Why the word is useful: it names a thing teams were already building without having agreed it existed — and things without names do not get owned, budgeted or reviewed.
A name collision to get straight before anything else, because it will come up in exactly the conversations where this term does.
Harness (harness.io) is a well-established continuous delivery platform — it builds, tests and deploys software, and it sits in a great many enterprise pipelines alongside Terraform and an artifact registry.
“Harness engineering” in this chapter has nothing to do with it. The word here is the mechanical sense — the rig around a thing — not the company.
If someone in a platform or developer-experience conversation says ‘are we using Harness’, they almost certainly mean the CD platform. The AI sense is newer and narrower, and assuming the wrong one is an easy way to answer a question nobody asked.
The most substantial treatment is by Birgitta Böckeler, a Distinguished Engineer at Thoughtworks, published on Martin Fowler's site in April 2026. Her framework is the part worth taking, and it is below.
Why it became a discipline
Because model capability converged and the variance moved. When several frontier models are close on the tasks you care about, the choice between them stops being the interesting decision — and whatever is still producing large differences in outcome becomes the thing worth engineering. The harness turned out to be that thing, and the size of its effect surprised people.
Read the third row again, because it is the most instructive. The treatment was not a better model, a bigger context window or a cleverer prompt. It was mechanically shortening older tool results as context pressure builds, plus detecting repeated failed patterns. One context-management mechanism, and the same weights solved nearly twice as many tasks. It transferred unchanged across four different model architectures.
Now the correction, because you will meet the overclaim constantly.
“The harness matters more than the model” circulates widely and is attributed to exactly these papers. Neither of the two I read makes that claim. Harness-Bench states it does not quantify it; the second describes it as an acknowledged research gap.
And Harness-Bench contains a finding that points the other way: stronger model backends tend to achieve higher mean scores while exhibiting lower cross-harness variance. The better the model, the less the harness matters — which is the opposite of the popular reading, and it is in the paper people cite for it.
The claim that survives: harness effects are large, were previously unmeasured, and are big enough that a benchmark result which does not specify the harness is not a result about the model. Both papers converge on the same recommendation — report at the model–harness configuration level.
That is a stronger position in an interview than the overclaim, and you can defend it.
The framework worth stealing
Böckeler frames a harness as a cybernetic governor — a system combining feedforward regulation (anticipating a problem) with feedback regulation (detecting it after the fact). Every control you add is one of the four quadrants above.
The design rule follows directly. A type system is a computational guide: it makes an entire class of error impossible, instantly, for free, forever. A review agent is an inferential sensor: flexible, slow, costs money, sometimes wrong. Both have a place. But reaching for the second when the first would have worked is the same mistake as reaching for an LLM judge when json.loads() would do — see the eval chapter, where it is the same ladder.
The three categories, and the honest gap
| Harness | What it regulates | State of the art |
|---|---|---|
| Maintainability | Internal code quality — structure, duplication, complexity, conventions | Most mature. Linters, formatters, complexity limits, architecture tests. You largely already own this tooling. |
| Architecture fitness | Performance and architectural characteristics — does it stay within budget, respect the boundaries | Workable. Fitness functions, performance budgets, dependency rules enforced in CI. |
| Behaviour | Functional correctness — does it do the right thing | The weakest, and acknowledged as such. Current practice leans on AI-generated test suites plus manual testing, which Böckeler says plainly is not sufficient. |
That last row is the most useful admission in the field right now
The controls we have are strongest exactly where the stakes are lowest. We can automatically enforce that code is tidy and fast. We cannot automatically enforce that it does the right thing.
And neither computational nor inferential controls catch the highest-impact failure at all — a correctly implemented, well-tested, beautifully formatted solution to the wrong problem. A misdiagnosed requirement passes every check you can build.
Which is why the mature goal is not to remove humans from the loop. It is to build a harness that directs human attention to where it is most valuable — and the machine-checkable things are precisely the things humans should stop spending review time on.
How it is being adopted
- ‘Keep quality left.’ Fast, cheap, deterministic controls run pre-commit — linters, type checks, a quick review agent. Expensive controls run post-integration in the pipeline. Drift is watched by health sensors running outside the normal workflow entirely. Same tiering as the eval suite, for the same reason: the expensive checks cannot run on every keystroke.
- Iterate the harness on repeated failures. The harness is not designed up front; it accretes. Something goes wrong twice, and that becomes a rule, a test, a lint check, or a line in the agent's instructions file. This is the single most transferable practice here, and it is the same discipline as growing an eval set from production complaints.
- Reduce variety deliberately. Böckeler's point that a comprehensive harness is only achievable if you commit to a small number of service shapes. Fifty bespoke architectures need fifty harnesses; five templates need one. This turns an AI-tooling question into a platform-engineering decision, which is where it actually belongs.
- Legacy is the hard case, and it is honest about it. A codebase with no tests, no types and inconsistent conventions gives an agent nothing to be regulated by — which is precisely the codebase where teams most want to point an agent.
Building one, in the order that works
- 1. Start with the computational controls you already have. Tests, types, linters, the build. If they are not wired into the agent's loop so it sees the failure and reacts, that is the first and cheapest win available.
- 2. Give the agent a way to verify itself. The single biggest lever on agent quality is whether it can check its own work — see the agent chapter. An agent that can run the tests behaves categorically differently from one that cannot.
- 3. Write the guides down where the tool reads them. The instructions file, the tool descriptions, the constraints. The prompting chapter covers what belongs there.
- 4. Add inferential sensors only for what nothing cheaper can express. Groundedness, tone, whether the change matches the stated intent.
- 5. Instrument it. Trajectories, tool-call counts, token spend, failure modes. You cannot iterate a harness on repeated failures if you cannot see the failures, and this is the layer teams skip.
- 6. Put the gate on irreversible actions, not on every turn.
Notice that four of those six are things a good engineering organisation already does. That is the quiet reason harness engineering caught on so fast: it is mostly existing software-engineering discipline, pointed at a new consumer — and the teams with strong test suites and tight conventions turned out to have been building harnesses for years without the word.
The interview angle
If asked what makes an agent reliable, the weak answer is a better model or a better prompt. The strong answer names the harness, gives the shape — deterministic guides first, sensors that let the agent self-correct, bounded context, gates on irreversible actions, observability — and cites the evidence accurately: harness choice moves benchmark scores by twenty points or more on identical models, which is why a result that does not name the harness is not a result about the model.
And the sentence that will separate you from everyone who read a blog post: the papers do not actually show the harness matters more than the model, and the best-measured one suggests the effect shrinks as models improve.
Check yourself