Software Engineering · Chapter 28 of 31
Factory, Blitzy, and buying at the top of the spectrum
Three differently shaped products, what the vendor numbers do and do not support, and how to evaluate one without running a year-long pilot.
The previous chapter put these products on a spectrum. This one looks at the top of it properly — because the three best-known names are shaped differently enough that comparing them on a feature grid tells you almost nothing.
Factory
Factory's product is Droids — agents specialised by workflow rather than by task or by repository. The named ones are the jobs every large estate has and nobody wants: migrating between framework or language versions, generating test suites, keeping documentation in sync with code, and dependency upgrades across many repositories at once.
The shape is the interesting part. Droids run inside your existing tools — editors, pipelines, ticketing — rather than in a separate destination you have to visit. That is a materially easier adoption story than a new place to go, and it is why Factory reaches individual developers at $20 a month and enterprises on custom contracts with the same product.
Its enterprise control list is worth stealing as a checklist, whoever you buy from
Single sign-on, SCIM provisioning, zero data retention, model selection, autonomy controls, organisation-level deny lists, audit logging, data residency, session retention policy, network policy.
Every one of those is a question your security review will ask. Take the list into the first vendor conversation rather than discovering it in month three — and note that a vendor who cannot answer the autonomy controls and deny list items has not thought about enterprise deployment, whatever the logo wall says.
Blitzy
Blitzy is the fleet. It ingests a repository, produces a unified technical specification, and then runs a large number of agents for hours to batch-generate and pre-compile the implementation — targeting the estates that traditionally went to a systems integrator on a multi-year contract. Legacy modernisation is the pitch, and it is a real and expensive problem.
Note what that implies about fit. A fleet is worth its overhead when the work is large, repetitive and specifiable. It is the wrong tool for a small change, and its eight-to-twelve-hour cycle means you cannot iterate with it the way you iterate with an editor.
What the numbers do and do not support
Everything published about outcomes in this category is vendor-sourced, and the framing repays attention.
“Tripled development velocity in three months.” Velocity of what, measured how, against what baseline? An unmeasured denominator is not a result — and if the baseline was a team that had never had a paved road, the tripling may be an argument for the paved road.
“Devin writes 89% of its own code.” A striking figure, and note that a company's own codebase is the most favourable possible test case: written by the people who built the tool, in the style the tool is best at, with every engineer motivated to use it.
The customer logo walls. A logo means an account exists. It does not distinguish a production deployment from a team trial, and the distinction is the whole question. Ask which team, on which repository, for how long.
And the valuations — billions across the category — are a statement about expectations, not about whether the product works on your codebase.
The one genuinely independent signal available is that a frontier lab named Blitzy among the testers of a new model's coding capability. That says a serious party found it substantial enough to use as a test harness — which is real, and is not the same as evidence it will modernise your estate.
What this tier is actually good at
The pattern across all three, and it is more useful than any product comparison:
| Works well | Works badly |
|---|---|
| Repetitive and specifiable — a migration applied 400 times | Novel design, where the hard part is deciding what to build |
| Machine-verifiable — the tests say whether it worked | Ambiguous requirements that need a conversation to resolve |
| Large enough to amortise the setup | Small changes, where you would have finished before it started |
| Low judgement per unit, high volume | Anything where being subtly wrong is expensive and hard to detect |
Which produces the uncomfortable correlation from the previous chapter, sharpened: the work that suits this tier best is work with strong tests and clear conventions — and the estates most desperate for legacy modernisation are precisely the ones without either. That is not a reason not to buy. It is the reason the first project should be building the verification layer, not the migration.
How to evaluate one in weeks rather than a year
- Give it a job you have already done. A migration you completed last quarter, on a branch. You know the right answer, you know how long it took, and you can diff. This is the single most informative test available and it costs almost nothing.
- Measure wall-clock to merged, including every round of human correction — not generation time, and not lines produced.
- Count the review hours honestly, including the senior engineer who had to read it. That is the cost the business case usually omits.
- Test the failure mode deliberately. Give it something under-specified and see whether it stops and asks or produces something plausible. At this tier the second behaviour is expensive, because nobody watched it happen.
- Check what it does to a codebase you care about but do not want changed. Deny lists, branch protection, blast radius — before the trial, not after.
- Ask for a reference customer with a codebase like yours, and speak to an engineer rather than a sponsor. The sponsor bought it; the engineer reviews its output.
Build, buy, or neither
A great deal of what these products do can be assembled from a coding agent, a script, and your existing pipeline — a migration codemod run across four hundred repositories is a loop, and loop engineering is exactly that. What you are buying at this tier is the orchestration, the enterprise controls, and somebody else's problem when it breaks.
- Buy when the work is large, the controls list matters to your security review, and you do not want to own the orchestration.
- Build when you have one or two well-understood repetitive jobs and a platform team — you will get better results because you can tune to your own codebase, and you will spend less.
- Neither, and this is the most common correct answer, when the honest blocker is that you cannot verify the output at volume. Fix that first, with tests and conventions, and the choice above gets much easier — and cheaper.
The one-sentence version: Devin does a task, Factory's Droids do a workflow inside your existing tools, and Blitzy runs a fleet at a whole repository — they answer different questions rather than ranking; every published outcome number is vendor-sourced and most have an unstated denominator; and the fastest honest evaluation is to hand one a migration you already finished and compare.
Check yourself