Fail a pipeline on a control violation, and design the exception path that keeps engineers moving.
50 min 210 XPSkills: CI/CD, Guardrails
A guardrail is a control that runs at commit time
Detective controls tell you about a violation after it reached production. A pipeline check refuses to let it get there. Same control objective, radically different cost: the engineer learns in ninety seconds, in the context they were already in, from a message that names the fix.
Placement matters. Run the policy check before deploy and after build, so it evaluates the artifact that will actually ship rather than the source someone hopes it came from.
The check is only a control at the point where it can stop the deploy.
Three ways a green pipeline lies
First, the swallowed exit code: the scanner runs inside a pipe or a subshell, the shell reports the status of the last command, and a job that found fourteen violations exits zero. `set -o pipefail` costs one line and closes it.
Second, the advisory check: the job genuinely fails, but nothing depends on it: `continue-on-error`, or a status that was never marked required, so the merge proceeds anyway. Third, the skipped path: a conditional means the check does not run on the branches that matter, and a job that never ran shows up as neutral, not as failure.
Test your guardrail the way you test a smoke alarm: push a deliberately violating change and confirm the pipeline goes red.
Fourteen violations, one green tick, zero errors in the log.
Blocking is a branch setting, not a job setting
A failing job stops nothing on its own. What stops a merge is the branch protection rule marking that job a required status check, and required means it must have run and passed, which is also what closes the skipped-path hole.
Keep the required set small and fast. A guardrail engineers wait eleven minutes for is a guardrail somebody will find a way around, and the way around is usually worse than the violation.
The exception path is part of the control
Some violation, some day, will be the right thing to ship. If your design has no legitimate way to do that, engineers will invent an illegitimate one, a bypass flag, a direct push, an admin merge, and it will be invisible.
A designed exception is requested, approved by someone other than the requester, recorded with a reason and an owner, and, this is the part that matters, carries an expiry date. Exceptions without expiry are how a temporary risk decision becomes permanent architecture.
Count your exceptions and their age. A rising median age is a control being eroded politely.
What the pipeline gives the auditor
Well-built, the pipeline produces its own evidence: every run logged with a commit, a result and a timestamp, covering the whole population rather than a sample of it. That is stronger than any quarterly screenshot.
It also produces the exception register: the complete list of times the control was consciously overridden, who approved each one, and whether it expired. Hand over both and the operating-effectiveness conversation is a query, not an argument.
Tasks
Answers are graded on the server: the answer key never reaches your browser.
Task 1: Fourteen violations, one green tick
Scenario sim+70 XP
Name the shell option that would have propagated the failure, then the workflow setting that also had to be wrong.
Incident brief
The IaC scanner has gated main since February. In August an engineer opens the artifact out of curiosity and finds `report.txt` listing fourteen high-severity findings: from a run that the pipeline recorded as passing.
The job's single step is `scan.sh | tee report.txt`. Above it in the workflow, added during a noisy first week and never removed, sits one more line the team had forgotten about.
The scanner is working perfectly. It has exited non-zero on every run since February.
Term bank, not all of these are right
pipefailcontinue on errorrequired status checkadvisoryexit codeset -e
1.1Name the shell option that would have propagated the scanner's failure. (one word)
1.2Name the workflow setting that also had to be removed. (three words)
Task 2: The check that failed and nobody stopped
Scenario sim+70 XP
Name the mode the check was operating in, then the setting that makes it enforceable.
Incident brief
The policy job is correctly configured: no pipes, no continue-on-error, and it goes red on violations. Over the quarter it went red 40 times.
All 40 pull requests merged. The team lead is adamant: "the check is blocking: look, it fails." The repository's branch protection lists exactly one entry: `build`.
For nine of the 40, the job did not run at all because of a path filter, and the merge screen showed the check as neutral rather than failed.
Term bank, not all of these are right
advisoryrequired status checkblockingbranch protectionpath filterexception
2.1Name the mode the policy check was really operating in. (one word)
2.2Name the branch setting that makes it enforceable. (three words)
Task 3: Temporary, since 2023
Scenario sim+70 XP
Name the record that authorises a bypass, then the property these ones are missing.
Incident brief
The gate is genuinely blocking. To keep releases moving, the platform team added a repository label that skips the policy job, used "only for urgent fixes, with a ticket".
The register lists 31 uses. Twenty-nine have tickets. Eleven of the tickets are closed and the underlying violation is still in production; the oldest is from November 2023 and is described in the ticket as "temporary pending migration".
The auditor's finding does not mention the label at all.
The pipeline already has a policy step, and the release engineer insists it works: "nothing ships without approval". Last week a deploy went out approved by the person who wrote it, and another shipped with the test stage skipped.
Extend the gate so production deploys are denied when they lack an approval, when the approver is the author, or when tests did not run, while leaving staging and clean production deploys alone.
Deploy events the gate will see
# env approved approver_is_author tests signed
A prod false false passed true
B prod true true passed true
C prod true false skipped true
D prod true false passed true
E staging false false skipped false
A, B and C must be denied. D and E must ship.
Fields arrive on input with exactly the names above.
Eight production deployments went out yesterday. The control gate is configured but the report shows it was not always the deciding factor.
Raise the deployments where the gate did not actually control the release. Leave the ones where it did.
List only the lines that qualify, worst first. Severity is one of: critical | high | medium | low. Add or delete rows as needed.
Deployment gate report
V-1 Deploy 4471: gate skipped: the job's continue-on-error is true, so a failed policy check did not block the release.
V-2 Deploy 4472: gate passed, all checks green.
V-3 Deploy 4473: released from a local machine, bypassing the pipeline entirely.
V-4 Deploy 4474: gate failed, release blocked, fixed and redeployed.
V-5 Deploy 4475: gate passed after an engineer edited the workflow in the same PR to remove two checks.
V-6 Deploy 4476: gate took 11 minutes, longer than the 5-minute target.
V-7 Deploy 4477: gate passed; evidence artefact retained 90 days.
V-8 Deploy 4478: gate ran after the deploy step rather than before it.
1
2
3
4
5
6
7
8
9
10
11
12
Checks
The right findings are raised
Benign lines are left out
Severity matches the actual exposure
Ranked worst first
Next on your path · Automation & Code
OSCAL Profiles & Baselines
Tailor a catalog into a profile without destroying it, and keep the SSP, assessment results and POA&M pointing at the same control ids.
