Security Engineering · Chapter 8 of 12
Control frameworks, and what an auditor actually wants
NIST, SOC 2, ISO 27001 and PCI from first principles — what each is for, how a control is tested, and where the whole apparatus turns into theatre.
A control is a thing you do, repeatably, to reduce a risk — and a control framework is somebody's organised list of the things worth doing. That is the whole idea, and it survives being stated that plainly.
This chapter exists because the vocabulary is a barrier and the concepts are not. An engineer who can explain what a SOC 2 Type II opinion covers is immediately more useful in a security organisation than one who cannot, and it takes an afternoon to learn.
The four you will meet
The four are different KINDS of thing, which is why comparing them is confused
NIST is a catalogue. A very good, free, structured library of controls. Nobody audits you against it and there is no certificate. You use it to decide what to do.
SOC 2 is a report. An independent auditor writes an opinion on controls you chose. You hand it to customers who are diligencing you.
ISO 27001 is a certificate — and it certifies your management system, not your security. It says you have a working process for deciding, implementing and reviewing controls.
PCI DSS is a mandate. Prescriptive, specific, and imposed on you by the card networks if you handle card data. You do not get to choose the controls.
So ‘which framework should we adopt’ is usually the wrong question. The right one is ‘who is asking, and what will satisfy them’.
SOC 2, since it is the one you will meet first
SOC 2 is an audit against up to five Trust Services Criteria: Security (mandatory), plus Availability, Confidentiality, Processing Integrity and Privacy if you choose to include them. Most reports cover Security alone, which is worth knowing when someone waves one at you.
| Type I | Type II | |
|---|---|---|
| What it tests | Are the controls designed properly? | Do they actually operate, over time? |
| Evidence | A point in time | A window — typically 3 to 12 months |
| What it proves | You wrote it down | You did it, repeatedly, and there is evidence |
| Worth | Very little on its own | The one that matters. Ask which you are being shown. |
“We are SOC 2 compliant” is not a sentence that means anything on its own, and knowing why is a genuine interview differentiator.
There is no such thing as being compliant with SOC 2. There is having a report, and the questions that matter are: Type I or Type II? What period? Which criteria? What is the scope? And were there exceptions?
The exceptions are the interesting part and they are in the report — a clean opinion can sit above a list of noted deviations. Most people who receive a SOC 2 report never open past the first page.
A report is an auditor’s opinion about controls the company chose for itself. It is meaningful evidence that a process exists. It is not a statement that the company is hard to break into.
What a control looks like when it is real
Abstractions are why this subject is hard to learn. Here is one control, concretely.
CONTROL: Production access is approved and time-limited.
RISK: A standing admin credential is a permanent path to customer data.
OWNER: Head of Platform Engineering (a named person, not a team)
DESIGN — what should happen:
No standing production admin. Access is requested, approved by the
service owner, granted for a maximum of 8 hours, and logged.
EVIDENCE — what an auditor will ask for:
1. The policy document, with a version and an approval date.
2. The IAM configuration proving standing admin does not exist.
3. A full list of elevations in the audit period.
4. For a SAMPLE of those: the approval record, and proof it expired.
TESTING:
Design effectiveness — read the policy and the config. Is it capable
of working? <- a Type I question
Operating effectiveness— sample 25 elevations from the period. Did each
have an approver, and did each expire on time?
<- a Type II question
FAILURE: 3 of 25 had no recorded approver.
-> that is an EXCEPTION. It goes in the report. You get a
finding, a remediation plan and a date, and the auditor
re-tests next period.Notice where the work actually is. Not in having the policy — in being able to produce the evidence, on demand, for a period that has already passed. Teams fail audits because the control worked and nobody can prove it, far more often than because the control was absent.
Which is why this is an engineering problem
- If the evidence is a screenshot, the control will fail eventually. Screenshots are taken by a person who was busy that quarter.
- Make the system the evidence. If elevation runs through a tool that records requester, approver, scope and expiry, the audit is a query. If it runs through a chat message, the audit is an archaeology project.
- Prefer preventive controls to detective ones, and both to a policy. A configuration that makes standing admin impossible is worth more than any amount of quarterly reviewing — and it is cheaper to evidence, because the absence is the proof.
- Sampling is the whole reason automation wins. An auditor samples 25 because reading 4,000 by hand is impossible. Continuous control monitoring tests the population instead — and the argument for it is not rigour, it is that you find out in March rather than in the audit.
Where it becomes theatre, honestly
Any treatment of this that does not say the following is selling something.
- A framework measures whether you did what you said. It cannot tell you whether what you said was the right thing. Both a good and a mediocre security programme can produce a clean report.
- Scope is where the meaning leaks out. A report covering one product in one region says nothing about the rest, and the scope statement is the part nobody reads.
- Control counts are not security. An organisation with 400 controls and no asset inventory is worse off than one with 40 controls it can evidence, and the first will audit better.
- The compliance calendar can crowd out the work. When a team spends the quarter assembling evidence, the evidence is the output. This is the strongest argument for automating it, and it is an argument about attention rather than cost.
And the fair defence, because the cynical read is also incomplete: a framework gives you a checklist that survives staff turnover, a vocabulary shared with your customers' risk teams, and a reason to do unglamorous work that never wins an argument on its own merits. Asset inventories and access reviews get funded because an auditor asks for them. That is a real benefit even when the motivation is unedifying.
Check yourself