Lakshya

Security Engineering · Chapter 8 of 12

Control frameworks, and what an auditor actually wants

NIST, SOC 2, ISO 27001 and PCI from first principles — what each is for, how a control is tested, and where the whole apparatus turns into theatre.

8 min read1 diagramAll 12 chapters

A control is a thing you do, repeatably, to reduce a risk — and a control framework is somebody's organised list of the things worth doing. That is the whole idea, and it survives being stated that plainly.

This chapter exists because the vocabulary is a barrier and the concepts are not. An engineer who can explain what a SOC 2 Type II opinion covers is immediately more useful in a security organisation than one who cannot, and it takes an afternoon to learn.

The four you will meet

FOUR FRAMEWORKS, FOUR DIFFERENT JOBS — THEY ARE NOT ALTERNATIVES NIST CSF / 800-53a catalogue, not a certificateA structured library of controls you pick from. Nobody certifies you against it.Free. Voluntary. US-federal-adjacent work often requires 800-53 alignment.SOC 2an AUDIT REPORT you sellAn independent auditor’s opinion on controls YOU chose, against 5 trust criteria.Asked for by your CUSTOMERS. It is a sales document before it is a security one.ISO 27001a CERTIFICATION of a systemCertifies you run a management system (an ISMS) — the process, not the controls.Asked for in UK, EU and Indian enterprise procurement. Recertified every 3 years.PCI DSSa MANDATE with prescribed rulesNon-negotiable, prescriptive requirements. If you touch card data, you comply.Imposed by the card brands. Non-compliance has contractual and financial teeth. THE DISTINCTION PEOPLE GET WRONG: SOC 2 does not tell you WHICH controls to have. You choose them. The auditor tests whether you do what you said. A clean SOC 2 report over weak controls is entirely possible — and it is why “we’re SOC 2 compliant” answers a different question from “are you secure”.
Read the red box before the table. It is the single most common misunderstanding in this whole subject.

The four are different KINDS of thing, which is why comparing them is confused

NIST is a catalogue. A very good, free, structured library of controls. Nobody audits you against it and there is no certificate. You use it to decide what to do.

SOC 2 is a report. An independent auditor writes an opinion on controls you chose. You hand it to customers who are diligencing you.

ISO 27001 is a certificate — and it certifies your management system, not your security. It says you have a working process for deciding, implementing and reviewing controls.

PCI DSS is a mandate. Prescriptive, specific, and imposed on you by the card networks if you handle card data. You do not get to choose the controls.

So ‘which framework should we adopt’ is usually the wrong question. The right one is ‘who is asking, and what will satisfy them’.

SOC 2, since it is the one you will meet first

SOC 2 is an audit against up to five Trust Services Criteria: Security (mandatory), plus Availability, Confidentiality, Processing Integrity and Privacy if you choose to include them. Most reports cover Security alone, which is worth knowing when someone waves one at you.

Type IType II
What it testsAre the controls designed properly?Do they actually operate, over time?
EvidenceA point in timeA window — typically 3 to 12 months
What it provesYou wrote it downYou did it, repeatedly, and there is evidence
WorthVery little on its ownThe one that matters. Ask which you are being shown.

“We are SOC 2 compliant” is not a sentence that means anything on its own, and knowing why is a genuine interview differentiator.

There is no such thing as being compliant with SOC 2. There is having a report, and the questions that matter are: Type I or Type II? What period? Which criteria? What is the scope? And were there exceptions?

The exceptions are the interesting part and they are in the report — a clean opinion can sit above a list of noted deviations. Most people who receive a SOC 2 report never open past the first page.

A report is an auditor’s opinion about controls the company chose for itself. It is meaningful evidence that a process exists. It is not a statement that the company is hard to break into.

What a control looks like when it is real

Abstractions are why this subject is hard to learn. Here is one control, concretely.

textone control, all the way through — this is the shape of all of them
CONTROL:      Production access is approved and time-limited.
RISK:         A standing admin credential is a permanent path to customer data.
OWNER:        Head of Platform Engineering        (a named person, not a team)

DESIGN — what should happen:
  No standing production admin. Access is requested, approved by the
  service owner, granted for a maximum of 8 hours, and logged.

EVIDENCE — what an auditor will ask for:
  1. The policy document, with a version and an approval date.
  2. The IAM configuration proving standing admin does not exist.
  3. A full list of elevations in the audit period.
  4. For a SAMPLE of those: the approval record, and proof it expired.

TESTING:
  Design effectiveness   — read the policy and the config. Is it capable
                           of working?            <- a Type I question
  Operating effectiveness— sample 25 elevations from the period. Did each
                           have an approver, and did each expire on time?
                                                  <- a Type II question

FAILURE:  3 of 25 had no recorded approver.
          -> that is an EXCEPTION. It goes in the report. You get a
             finding, a remediation plan and a date, and the auditor
             re-tests next period.

Notice where the work actually is. Not in having the policy — in being able to produce the evidence, on demand, for a period that has already passed. Teams fail audits because the control worked and nobody can prove it, far more often than because the control was absent.

Which is why this is an engineering problem

  • If the evidence is a screenshot, the control will fail eventually. Screenshots are taken by a person who was busy that quarter.
  • Make the system the evidence. If elevation runs through a tool that records requester, approver, scope and expiry, the audit is a query. If it runs through a chat message, the audit is an archaeology project.
  • Prefer preventive controls to detective ones, and both to a policy. A configuration that makes standing admin impossible is worth more than any amount of quarterly reviewing — and it is cheaper to evidence, because the absence is the proof.
  • Sampling is the whole reason automation wins. An auditor samples 25 because reading 4,000 by hand is impossible. Continuous control monitoring tests the population instead — and the argument for it is not rigour, it is that you find out in March rather than in the audit.

Where it becomes theatre, honestly

Any treatment of this that does not say the following is selling something.

  • A framework measures whether you did what you said. It cannot tell you whether what you said was the right thing. Both a good and a mediocre security programme can produce a clean report.
  • Scope is where the meaning leaks out. A report covering one product in one region says nothing about the rest, and the scope statement is the part nobody reads.
  • Control counts are not security. An organisation with 400 controls and no asset inventory is worse off than one with 40 controls it can evidence, and the first will audit better.
  • The compliance calendar can crowd out the work. When a team spends the quarter assembling evidence, the evidence is the output. This is the strongest argument for automating it, and it is an argument about attention rather than cost.

And the fair defence, because the cynical read is also incomplete: a framework gives you a checklist that survives staff turnover, a vocabulary shared with your customers' risk teams, and a reason to do unglamorous work that never wins an argument on its own merits. Asset inventories and access reviews get funded because an auditor asks for them. That is a real benefit even when the motivation is unedifying.

Check yourself

What is a control?A thing you do, repeatably, to reduce a risk. A framework is somebody's organised list of the ones worth doing.
How do the four frameworks differ in kind?NIST is a catalogue you pick from, SOC 2 is an auditor's report you show customers, ISO 27001 certifies a management system, and PCI DSS is a prescriptive mandate imposed by the card networks.
What does SOC 2 not tell you?Which controls to have. You choose them; the auditor tests whether you do what you said. A clean report over weak controls is entirely possible.
Type I or Type II?Type I tests whether controls are designed properly at a point in time. Type II tests whether they actually operated over a period. Type II is the one that matters — always ask which you are being shown.
What questions should you ask of a SOC 2 report?Type I or II, what period, which trust criteria, what scope, and what exceptions. The exceptions are in the report and most recipients never read past page one.
What is the difference between design and operating effectiveness?Design asks whether the control is capable of working — read the policy and config. Operating asks whether it actually worked, tested on a sample from the period.
Why do teams fail audits?Far more often because the control worked and nobody could produce the evidence than because the control was absent.
Why does automation win?An auditor samples 25 because reading 4,000 is impossible. Continuous monitoring tests the whole population — and the real argument is that you find out in March, not in the audit.
What is the honest defence of frameworks?They give a checklist that survives turnover, a shared vocabulary with customers' risk teams, and a reason to fund unglamorous work like asset inventories that never wins an argument alone.
← Container and Kubernetes securityIncident response and threat hunting →