Model a control catalog in YAML/JSON and OSCAL so it can be queried, diffed and tested.
45 min 180 XPSkills: Data modeling, OSCAL
A control is a record, not a paragraph
Written as prose, a control can only be read. Written as a record, an id, a title, an owner, a frequency, a list of framework references, and the query that produces its evidence, the same control can be queried, diffed, validated and tested. Nothing about the safeguard changes; what changes is that machines can now participate.
The moment your catalog is data, questions that used to take a week become one command: which controls have no owner, which map to no framework, which claim to be automated but have no evidence query, which changed since the last audit period.
If you cannot answer "how many controls do we have" with a command, you do not have a catalog. You have a folder.
The same safeguard, expressed so that tooling can act on it.
The fields that earn their place
Every field should have a consumer. `id` is the join key everything else references, so it must be stable and unique for the life of the program: never reuse it, never renumber. `framework_refs` is the crosswalk that lets one implemented control satisfy SOC 2, ISO 27001 and 800-53 at once. `owner` must resolve to a real identity, not a team alias nobody reads.
`automated` and `evidence_query` are the pair that keeps the catalog honest. A control marked automated with no query is a claim; a control with a query is a control that can produce its own evidence on demand.
OSCAL, and when you actually need it
NIST's Open Security Controls Assessment Language is the standard machine-readable format for catalogs, profiles and assessment results. It is verbose and it is worth it when you exchange control data with someone else: a regulator, a customer, a federal program, an assessor's tooling.
For an internal catalog, plain YAML with a schema you validate in CI is usually the right first move. Design your fields so an OSCAL export is a transform rather than a rewrite: keep ids stable, keep framework references explicit, and never encode meaning in free text.
Integrity checks are the whole point
Data you never validate rots faster than prose, because nobody reads it. Run three families of check on every pull request: schema validation (every record has the required fields, of the right type), uniqueness (no two records share an id), and referential integrity (every framework reference resolves to a requirement that exists, every owner resolves to a real person).
The failure mode this prevents is quiet. Two catalog branches merge, an id ends up on two records, and from that point mappings, evidence and test results attach to whichever record the tooling happens to read first. Nothing errors. The numbers are simply wrong.
A catalog that cannot fail a build cannot be trusted in an audit. Wire the checks before you grow the catalog.
Duplicate ids never announce themselves. A check has to go looking.
Catalog reality, not aspiration
The last discipline is reconciliation: periodically comparing what the catalog claims against what the systems actually show. Controls disappear from reality long before anyone edits the record: a scheduled job is disabled, a repo is archived, an owner leaves.
Treat the reconciliation itself as a control. It runs on a cadence, it produces output, and its output is evidence that the catalog you handed the auditor describes the company you actually run.
Tasks
Answers are graded on the server: the answer key never reaches your browser.
Task 1: Two controls, one id
Scenario sim+60 XP
Name the record property the merge broke, then the check family that would have refused it.
Incident brief
Two engineers split the catalog work. One rewrites the account-management control in the `iam` branch, the other rewrites it in `access`. Both merge cleanly on Friday: YAML merges rarely conflict when the edits are in different files.
On Monday the coverage report shows 61 controls instead of 60, the SOC 2 mapping for CC6.1 points at a record with no owner, and the evidence job for account management runs twice with different queries.
Both records read `id: AC-2`. Nothing errored. The pipeline is green.
1.1Name the record property the merge violated. (two words)
1.2Name the check family that would have refused the merge. (two words)
Task 2: The mapping the customer asked for
Scenario sim+60 XP
Name the catalog structure being requested, then the standard format for exchanging it.
Incident brief
A federal-adjacent customer asks for "your control catalog, machine-readable, with the 800-53 linkage". The security lead exports the internal YAML to a spreadsheet and adds a column of comma-separated NIST identifiers typed by hand.
Two weeks later the customer's assessor rejects it: the identifiers do not parse, three point at withdrawn controls, and there is no way to tell an implemented control from an inherited one.
The engineer's response: "It is machine-readable, it is a CSV."
Term bank, not all of these are right
crosswalkoscalprofilebaselineschemacsv
2.1Name the structure the customer is really asking for. (one word)
2.2Name the standard format it should be exchanged in. (one word)
Task 3: Automated, allegedly
Scenario sim+60 XP
Name the missing field, then the recurring activity that would have caught the claim.
Incident brief
The catalog reports that 38 of 60 controls are automated, and leadership quotes the number in a board deck. The auditor samples six of the 38 and asks each owner to produce this quarter's output.
Four produce it. One produces output from eight months ago. One owner says the check "is in the catalog, so it must be running somewhere": the record has `automated: true`, a title, an owner and framework refs, and nothing else.
No record in the catalog has ever been compared against the systems it describes.
Your compliance lead sends you a control lifted from a Word policy: "Access to production is reviewed regularly by management to ensure appropriateness." She calls it "already automated" because a Jira ticket reminds someone to do it every quarter.
A reminder is not automation, and "regularly" is not a frequency. Rewrite the control as structured data a pipeline can act on: it needs a stable id, an owner, a machine-readable frequency, the system it runs against, and an assertion an evaluator can test.
Source control (as written in the policy)
title: Production access review
text: >
Access to production is reviewed regularly by management
to ensure appropriateness.
owner: "whoever runs the review"
evidence: "screenshot of the review meeting"
1
2
3
4
5
6
7
8
9
10
11
12
Checks
Has a stable, uppercase control id
Owner is a team, not a person or a role-in-prose
Frequency is a machine-readable enum value
Assertion states a testable condition
Evidence is a query, not a screenshot
Marked automated once it has a query and an assertion
An auditor asks you to "map the MFA control to SOC 2, ISO and NIST". A colleague has already started and mapped everything to CC6.1 because, in his words, "CC6.1 covers all the access stuff".
Mappings are many-to-many and framework-specific. Add a mappings block that names each framework and its own control identifier: a SOC 2 trust services criterion, an ISO 27001 Annex A control, and a NIST 800-53 control id.
Framework identifier formats
soc2: CC6.1, CC6.6, CC7.2 (trust services criteria)
iso27001: A.5.15, A.5.17, A.8.5 (Annex A, 2022 numbering)
nist80053: AC-2, IA-2, AC-6 (control family + number)
Control under mapping: multi-factor authentication
enforced for all administrative console access.
Six control records have been merged into the library as YAML. The team lead says they are all machine-readable because they parse.
Parsing is not the bar. Accept the records a scheduler and an evaluator could actually act on; reject the ones whose fields are prose in a data costume.
Verdicts are accept or reject. Every rejection needs one reason code from: prose-assertion | unstable-id | unschedulable-frequency | evidence-not-rerunnable.
controls/*.yaml (field summary)
Y-1 id: AC-PROD-MFA | owner: platform-security | frequency: daily | assertion: "zero admin accounts without mfa" | evidence_query: okta-api /admins
Y-2 id: AC-REVIEW | owner: platform-security | frequency: quarterly | assertion: "access is reviewed appropriately by management" | evidence_query: okta-api /admins
Y-3 id: control 7 (new) | owner: risk-office | frequency: monthly | assertion: "no bucket is publicly readable" | evidence_query: aws-config query
Y-4 id: EV-RETENTION | owner: data-platform | frequency: monthly | assertion: "every evidence object is retained at least 2555 days" | evidence_query: s3-api lifecycle rules
Y-5 id: VM-PATCH | owner: platform-security | frequency: "as needed" | assertion: "zero critical findings older than 24 hours" | evidence_query: scanner-api /findings
Y-6 id: VN-REVIEW | owner: vendor-management | frequency: annually | assertion: "every tier-1 vendor has a current assessment" | evidence_query: screenshot of the vendor tracker
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Checks
Every line in the fixture carries a verdict
Accept and reject land on the right lines
Each rejection cites a valid reason code
Next on your path · Automation & Code
Policy as Code with OPA/Rego
Write Rego policies that evaluate cloud configuration against your control catalog.
