Security Engineering & Tooling · Chapter 1 of 8
What the tools are, and what they structurally cannot do
Security tooling is sold on what it finds. The useful question is what each category is blind to by construction.
Every security tool has a shape, and the shape decides what it can never see. That is not a maturity problem that a better model fixes — it follows from where the tool sits. An interview is largely testing whether you know the boundaries.
SAST — static analysis of source
Genuinely detects: syntactic and dataflow-reachable defects where source and sink are both in the code it can see — injection (SQL, command, log), server-rendered XSS, hardcoded crypto misuse, unsafe deserialisation entry points, path traversal. Modern engines follow taint across function and file boundaries within one repository.
Structurally cannot detect: authorisation and access-control logic, business-logic flaws, anything whose correctness depends on runtime configuration, and anything crossing a service boundary. There is no syntactic signature for “this endpoint should have checked tenant ownership” — which is why broken object-level authorisation, consistently among the most damaging real-world web flaws, is still a manual-testing problem. A tool cannot know your intended authorisation model unless you tell it.
The false-positive problem is the adoption decision, not a footnote.
The mechanism: developers receive findings they judge to be wrong, learn the tool is unreliable, and then ignore true positives at the same rate as false ones. Recovery is very hard once that trust is gone.
Do not quote a single number. Published rates range from about 20% to over 90% depending on language, ruleset and codebase — a 2025 study reports ~79% on a standard benchmark corpus and up to 38% in real-world multi-language cases; a Python analysis found 99.5% of flagged command injections were false. Java and C/C++ do better than Python and JavaScript, because static types give the analyser more to work with.
The “NIST found 78% false positives for Java” line is folklore. It is cited everywhere and traces to no specific NIST publication. Do not attribute it.
What actually changed the category: the move from pattern and AST matching to interprocedural taint analysis. “Flag eval()” produces noise; “flag eval() reachable from an HTTP parameter without passing a sanitiser” produces far less. That line is also the commercial dividing line — it is what most vendors put behind the paid engine.
SCA, and the reachability shift
The old model: build an SBOM, match against a CVE database, report “you have 4,312 vulnerabilities.” Technically accurate, operationally useless. It generates a backlog nobody can work, ranked by a score that measures theoretical severity in the abstract rather than risk in your application.
The one prioritisation result with real evidence behind it is EPSS — the Exploit Prediction Scoring System, governed by FIRST. At a threshold of 0.088, version 3 gives roughly 82% coverage of exploited vulnerabilities for 7.3% of the remediation effort, against a CVSS-based strategy at 69.9% coverage and 18.5% efficiency — about double the coverage at triple the efficiency. Be precise about what that proves: it is a prediction-quality result. Nobody has shown that EPSS-driven patching reduces incidents.
Where static reachability loses precision: reflection, dynamic dispatch, dependency injection, eval-style loading, native and foreign-function calls, and service boundaries. Dynamic languages are materially worse than statically typed ones.
Vendor reachability numbers — do not repeat these as fact
“Filters ~92% of false positives.” “Up to 95%.” “Up to 80% reduction.” All three are vendor marketing: self-measured, no stated baseline, no independent replication.
There is no independent academic or third-party evaluation of commercial reachability precision and recall that I could find. Academic call-graph research reports improvements in the 8–84% range on precision/recall — for research call-graph construction, which is not a validation of any product.
The question that separates a serious answer from a repeated one: reduction against what baseline, and what did it do to the true positives?
Vulnerable, malicious, and legally risky are three different things
The interview-grade distinction. A vulnerable dependency and a malicious dependency are different threat models needing different detection. CVE matching cannot find a typosquatted package published yesterday, because there is no CVE for it — only behavioural analysis of what the package does at install time (network access, filesystem reach, obfuscation, install scripts) will. The npm worm attacks are why this split now matters, and it is the entire reason a vendor like Socket exists alongside the CVE scanners.
And there is a third category that a security-shaped tool will never report, because nothing is broken. Every dependency arrives under a licence. Most are permissive. Some are copyleft — the GPL family — and can require that if you distribute software containing them, you make your own source available. The code works perfectly. There is no vulnerability. The exposure is contractual.
This is what Black Duck was built for, two decades before its vulnerability scanning grew alongside it — and it is why a dependency scan is a standard step in acquisition due diligence. A buyer’s lawyers want to know what is inside the product they are paying for, and a copyleft library deep in a commercial codebase is the kind of finding that moves a price. If someone asks why you would buy Black Duck when you already run a vulnerability scanner, that is the answer. Fuller treatment in the SDLC tooling chapter.
The tools, and one licence story people get wrong
| Tool | Model | What it is actually for |
|---|---|---|
| Semgrep | Engine LGPL 2.1; registry rules under the Semgrep Rules License | Fast, writable rules. See below — the engine was not closed-sourced |
| SonarQube | Community (LGPL) + commercial | Origin is code quality; security is a later layer. Strong on maintainability, weaker on deep taint |
| Checkmarx | Commercial | The regulated-enterprise incumbent. Heaviest configuration burden. Now also owns ZAP |
| Snyk Code | Commercial, free tier | Developer workflow is the product — IDE and PR integration, not the engine |
| CodeQL / GHAS | Free for public repos, paid for private | A query language over a code database. Most expressive engine here, steepest learning curve |
| Opengrep | LGPL 2.1 fork | Forked from Semgrep CE in January 2025 by a competitor coalition, restoring cross-function taint in the open fork |
| Coverity | Commercial | Deep static analysis with a C and C++ heritage — safety-critical work, automotive and medical. Slow and thorough rather than fast and conversational |
| Black Duck | Commercial | Dependencies, including their licences — see above. Same company as Coverity since October 2024, when Synopsys sold its Software Integrity Group for $2.1bn and it took the Black Duck name |
| Fortify | Commercial | The other long-established enterprise scanner, now under OpenText |
What actually happened to Semgrep, since it gets told wrong. The engine remains LGPL 2.1. What changed in December 2024 was the rules: the registry moved to a licence limiting use to internal, non-competing, non-SaaS contexts, and “Semgrep OSS” was renamed Community Edition. Individual engineers, consultants and internal corporate use were never restricted — the change was aimed at vendors reselling the rules. If you are an end user, nothing broke.
And a pricing fact worth having straight, because it is a billing trap rather than a licence one: GitHub Advanced Security was split in April 2025 into Secret Protection and Code Security as separate SKUs, and a “committer” is anyone who contributed to an enabled private repo in the last 90 days. A contractor who pushed once in a quarter is a billable committer.
Check yourself