Lakshya

SDLC, DevOps & Developer Experience · Chapter 4 of 15

DevSecOps and the software supply chain

Where security checks belong, the attack at each link of the chain, and the one question a supply-chain programme exists to answer.

8 min read1 diagramAll 15 chapters

‘Shift left’ became a slogan and lost its meaning. The defensible version is narrow and worth stating precisely: a security finding costs less the earlier it is found, so put each check at the earliest point where it can run reliably. Not everywhere, and not as early as possible regardless of noise.

Where each check belongs, and why

CheckEarliest reliable pointWhy not earlier
Secret scanningPre-commit hook, and again in CINothing earlier exists. And it must also run in CI, because hooks are trivially bypassed
Dependency scanning (SCA)On the pull requestNeeds the resolved lockfile, which needs the change
Static analysis (SAST)On the pull request, diff-scopedFull-repo scanning on every PR produces a wall of pre-existing findings nobody reads
Infrastructure policyOn the plan, before applyThe code alone does not tell you what will happen
Container image scanningAt build, and again on a scheduleA vulnerability disclosed after you built is invisible to a build-time-only scan
Dynamic analysis (DAST)Against a deployed environmentNeeds something running
Penetration testingAgainst a realistic environmentNeeds a whole system and a human

The rule that decides whether any of this works

A check that blocks a build must have a very low false-positive rate. A check with a high false-positive rate must not block.

Get this backwards and you produce the standard outcome: developers acquire a habit of adding suppression comments, or the security team grants a blanket exception, and the control becomes decorative while still costing everyone time.

So: scope static analysis to the diff rather than the whole repository, block only on high-confidence high-severity findings, and route everything else to a backlog with an owner. A control that is routed around is worse than no control, because it also produces a false sense that the risk is handled.

And the corollary: the first thing to do with a new scanner is not to turn on blocking. Run it in report-only mode long enough to see its false-positive rate on your codebase, then choose what blocks.

The chain, link by link

EVERY LINK IS A PLACE TO INJECT CODE. NAME THE ATTACK, THEN THE CONTROL. developer laptop, credentials source repo branch, review dependencies transitive, huge build runs arbitrary code artefact registry, tags deploy who can push to prod stolen tokenmalicious commitforce-pushunreviewed mergetyposquatdependency confusionpoisonedbuild steptag overwrittenimage swappeddirect pushto production signed commitsshort-lived credsMFA on gitprotected branchrequired reviewno force-pushlockfile + hashpin versionsreserve namespacesephemeral runnerno secrets in PRhermetic buildsign the artefactimmutable tagsdigest not tagverify signaturepolicy gateno human push THE ONE THAT IS TRIVIAL TO FIX AND USUALLY IS NOT: DEPENDENCY CONFUSION If your internal package name is not reserved on the public registry, an attacker publishes it there with a higher version number, and your build may prefer it. Reserve your internal namespaces publicly, and never let the resolver fall back to public for internalscopes. The question a supply-chain programme has to be able to answer in minutes, not days: “a critical vulnerability was announced inlibrary X — are we affected, and where?” That is what an inventory of what you actually shipped is for. Not compliance — the ability to answer that question before yourcustomers ask it. And the build step deserves its own emphasis: your CI runs arbitrary code from every dependency you install, with whatevercredentials the job holds.
Every link is a place to inject code. The middle box is the one that is trivial to fix and usually is not; the line at the bottom is the one people underestimate.

Two links deserve more than a row in a diagram.

Dependencies. A modern application pulls in hundreds of packages, most of them transitively, most of them never read by anyone on your team. Three attacks recur:

  • Typosquatting — a package one character away from a popular one.
  • Dependency confusion — and this is the one that is trivial to fix and usually is not. If your internal package name is not reserved on the public registry, an attacker publishes it there with a higher version number and your resolver may prefer it. Reserve your internal namespaces publicly, and configure the resolver never to fall back to the public registry for internal scopes. Both are one-time changes.
  • Compromise of a legitimate package — a maintainer's account is taken over, or a maintainer hands the project to someone who turns out to be hostile. This is the hardest to defend and the argument for pinning versions with integrity hashes and for reviewing what a dependency actually does before adding it.

The build. This one is consistently underestimated: your build system executes arbitrary code from every dependency you install — install scripts, build plugins, code generators — with whatever credentials the job holds. A build runner with a long-lived cloud credential and a production deployment token is a very attractive target, and it is reachable by anyone who can get a package into your dependency tree.

  • Ephemeral runners. A fresh environment per job, destroyed after, so a compromise does not persist into the next build.
  • No secrets available to pull-request builds from forks. This is the classic path: an attacker opens a pull request whose build steps exfiltrate your secrets. Most platforms now default to withholding secrets from fork builds, and the default is worth verifying rather than assuming.
  • Short-lived, federated credentials rather than stored keys. If the build exchanges its identity for a token valid for fifteen minutes and scoped to one action, a stolen credential is worth very little. This removes the class rather than protecting it.
  • Least privilege on the job itself. A test job does not need deployment permissions, and separating them means a compromise of the common path does not reach production.

Artefacts, and why a tag is not an identity

Once you have built something, the questions are: can you prove it came from your build, and can you be sure the thing you deploy is the thing you tested?

  • Deploy by digest, not by tag. A tag is a mutable pointer — the same tag can be overwritten to point at different content. A content digest cannot. :latest in a production manifest is an unreproducible deployment and a supply-chain gap in one line.
  • Sign artefacts and verify at deploy. Signing without verification is ceremony; the value is entirely in the admission-time check that refuses an unsigned or unrecognised image.
  • Generate provenance — a signed statement of what was built, from which source commit, by which builder. This is what lets you answer ‘where did this binary come from’ without trusting anyone's memory.
  • Immutable tags and retention policies in the registry, so history cannot be rewritten and old vulnerable images do not linger indefinitely.

The question the whole programme exists to answer

“A critical vulnerability has just been announced in library X. Are we affected, and where?”

That is what an inventory of what you actually shipped is for. Not compliance — the ability to answer that question in minutes rather than in days, before your customers ask it.

Organisations that had a generated, stored bill of materials for every artefact answered the last few industry-wide dependency crises in an afternoon. Organisations that did not spent a fortnight, and could not say with confidence when they finished.

So the test of a supply-chain programme is not whether you generate the document. It is whether you store it, index it, and can query it across every deployed artefact — and whether anyone has ever actually run that query. Generating a bill of materials and discarding it satisfies an auditor and helps nobody.

The rehearsal worth doing: pick a library you use, and time how long it takes to list every running service that contains it and every version. Most teams discover the answer is ‘we cannot’, and that discovery is more valuable than the tooling purchase it prompts.

Where AI has genuinely changed this

Two changes worth naming, one on each side.

Code generation increases dependency volume. Assistants readily suggest a library for a problem, and the review burden on what enters the dependency tree has risen correspondingly. The control is unchanged — someone must look at what a new dependency does before it is added — but the rate at which that decision arises has gone up.

And scanning got better at the part it was worst at. The historic weakness of static analysis was context: it could see a dangerous pattern and not whether the input was reachable from an untrusted source. Model-assisted triage is measurably better at that judgement, which matters because false-positive rate is the single variable that determines whether a security control survives contact with a development team. That is the honest case for it — not that it finds more, but that it argues more convincingly about what it found.

Check yourself

Where should a security check run?At the earliest point where it can run reliably — not as early as possible regardless of noise. Secret scanning pre-commit and in CI; static analysis diff-scoped on the PR; image scanning at build and again on a schedule.
What determines whether a blocking check survives?Its false-positive rate. High-noise checks that block get suppressed or exempted, and a control that is routed around is worse than none because it also creates false assurance.
Why is dependency confusion worth fixing first?Because the fix is a one-time change — reserve internal namespaces publicly and forbid public fallback for internal scopes — and the impact is arbitrary code in your build.
What is a bill of materials actually for?Answering 'are we affected by this vulnerability, and where' in minutes. If you generate it and do not store and index it, you have satisfied an auditor and helped nobody.
← Designing a pipeline people do not route aroundA testing strategy that survives a real codebase →