What makes evidence sufficient, appropriate and reproducible, and why a screenshot is the weakest artifact you can hand over.
30 min 170 XPSkills: Evidence, Audit
Brief
Every control you will ever automate ends in the same place: something has to prove it ran. That proof is evidence, and most compliance programs produce the weakest possible kind of it: a screenshot, cropped, undated, pasted into a folder by the same person who owns the control.
Evidence is judged on two axes an auditor will never stop applying. Is there enough of it to cover the period and the population? And is it the right kind: relevant to the control and reliable in origin? Those two words, sufficient and appropriate, decide whether your artifact ends an argument or starts one.
A screenshot is a picture of a claim. A query result someone else can re-run is a fact.
The quality ladder
Artifacts are not equal. A screenshot sits at the bottom: it has no verifiable timestamp, no provenance, and no way to tell whether the filter was set to hide the bad rows. A manual export is better but human-touched. A system export carries a timestamp. A query result someone else can execute is reproducible. An automated check with a retained run log is the top rung, because it produces evidence as a side effect of doing the work.
Climbing this ladder is most of what GRC engineering does in practice. You are not making auditors happier; you are removing the human step where the artifact could have been shaped.
Five rungs of the same claim. Only the top two survive a sceptical reviewer.
Sufficient and appropriate
Sufficiency is a quantity question: does the evidence cover the whole period and the whole population? One screenshot of one day against a twelve-month period is insufficient no matter how clear the image is.
Appropriateness is a quality question with two components. Relevance: does this artifact actually speak to the control being tested? And reliability, where did it come from, and could the tested party have altered it? A vendor's marketing PDF asserting encryption is relevant and unreliable. A config export from the platform itself is both.
Two independent tests. Failing either one sends the artifact back.
Provenance, timestamps and the Excel problem
Evidence has a chain of custody: where it came from, who handled it, and whether anything changed on the way. The most common break in that chain is entirely accidental: someone opens a CSV in a spreadsheet program, it reformats dates or truncates long identifiers, and the re-saved file no longer matches the system it came from.
The engineering answer is a hash. Record a checksum at the moment of extraction and any later alteration is provable in one command. Pair it with an extraction timestamp and a named source system, and the artifact defends itself without anyone vouching for it.
Designing evidence before you need it
The teams that survive audits cheaply decided, at control design time, what artifact the control would emit. The teams that suffer decided during fieldwork, under a deadline, using whatever they could screenshot.
For each control, write the evidence spec alongside the control statement: source system, extraction method, retention period, and the field that proves period coverage. It takes ten minutes and it removes the single largest recurring cost in the program.
Chain of custody and retention
Evidence is only as trustworthy as the path it took. Chain of custody records who collected it, when, from which system, and every hand it passed through, ideally with a hash taken at collection so any later change shows up.
Retention is how long evidence must be kept. It comes from the strictest obligation that applies: the audit standard, contracts and laws. Deleting evidence before the retention period ends can turn a passed control into an unprovable one.
If you cannot show where it came from and that nobody changed it, it is a story, not evidence.
Tasks
Answers are graded on the server: the answer key never reaches your browser.
Task 1: One day, twelve months
Answer challenge+40 XP
Two properties are missing. Name them.
Incident brief
The PBC item asks for evidence that MFA was enforced on the admin console for the full year. The control owner uploads a PNG: the security settings page, MFA toggle green, cropped just below the toggle.
There is no date visible. The file's created timestamp is yesterday. Asked how the auditor might confirm the same state on, say, 14 March, the owner replies: "You can log in and look: it's on right now, and it's been on the whole time as far as I know."
The setting genuinely is on today. Nobody is lying. The artifact still proves almost nothing about the period under audit.
Term bank, not all of these are right
sufficiencyappropriatenessreproducibilityrelevancematerialitychain of custody
1.1Name the evidence property broken by covering one day of a twelve-month period. (one word)
1.2Name the property missing when nobody else can re-run the artifact and get the same result. (one word)
Task 2: The vendor said so
Scenario sim+40 XP
Name the failing attribute, then the component of it that failed.
Incident brief
The control requires that customer data at rest is encrypted in the analytics warehouse. The evidence submitted is a two-page PDF from the warehouse vendor's sales site, titled "Security & Trust", containing the sentence "all customer data is encrypted at rest using AES-256".
Your compliance manager defends it: "It's absolutely relevant: it's literally about encryption at rest on that exact product. Relevance is the test, and it passes."
The warehouse admin console can export the actual per-dataset encryption configuration in about forty seconds. Nobody has run it.
2.1Relevance alone is not the test. Name the full evidence attribute in question. (one word)
2.2Name the component of that attribute the marketing PDF fails. (one word)
Task 3: Opened in Excel
Scenario sim+40 XP
Name the trail that broke, then the field that would have proved it.
Incident brief
An analyst pulls a 12,400-row entitlement export from the IdP, opens it in Excel to "tidy the columns", saves it as .xlsx and uploads it. The auditor's copy has 12,400 rows but every employee ID beginning with a zero has lost it, and 41 date fields have been reinterpreted as US format.
The analyst is certain nothing was changed: "I didn't edit a single cell. I only widened the columns and saved."
The original extract still exists on the jump host. Nobody recorded anything about it at the moment it was pulled beyond the filename.
Term bank, not all of these are right
chain of custodyhashtimestampaudit trailprovenanceretention
3.1Name the evidence concept the analyst broke by re-saving the file. (three words)
3.2Name the value recorded at extraction that would prove the file was altered. (one word)
Task 4: The screenshot with a past
Answer challenge+50 XP
Trust depends on the path, not the picture.
Incident brief
The access-review evidence for Q3 is a screenshot. It was taken by an engineer, pasted into a slide by a manager, exported to PDF and emailed to compliance three weeks later. Nobody recorded the source system or took a hash.
A tester notices the date in the screenshot uses a different font from the rest of the page.
Separately, the team's storage rule deletes evidence 90 days after the audit ends, but a customer contract requires keeping it for three years.
Term bank, not all of these are right
provenanceintegritysufficiencychain of custodyretentionsampling
4.1What record of collection and handling is missing? (three words)
4.2Which requirement does the 90-day deletion rule break? (one word)
Labs
70 bonus XP
Hands-on work. You edit the artefact, the server runs the checks: the expectations and the reference solution never reach your browser.
The auditor emails: "Please send a screenshot showing MFA is on." Your predecessor answered this by opening the admin console, screenshotting the toggle, and pasting it into a folder called Q3-FINAL-v2-real.
A screenshot proves a setting existed on the day someone looked. Specify the evidence properly: what artefact, from which source system, covering which period, produced how, and retained for how long.
The request thread
From: auditor
"Send a screenshot showing MFA is on."
Audit period: 2026-01-01 to 2026-03-31
Source of truth for MFA enrolment: Okta API
Retention expected by the audit firm: 7 years
A control owner has uploaded six artefacts for the Q3 access control test and marked the request complete. The audit is in nine days.
Accept the artefacts that would let an independent reviewer reach the same conclusion you would. Reject the rest: the ones that are out of period, cannot be tied back to a source, cover the wrong population, or are not evidence of the control at all.
Verdicts are accept or reject. Every rejection needs one reason code from: unverifiable | outside-period | wrong-population | not-evidence.
Uploaded bundle
E-1 okta_admins_2025-09-30.csv: API export, run id 8841, covers all admin roles, period end date.
E-2 screenshot_admin_page.png: a photo of a laptop screen showing the Okta admin page.
E-3 access_review_signoff.pdf: signed 2 Oct, lists reviewers and decisions for the Q3 population.
E-4 okta_admins_2025-06-30.csv: same export shape, dated the end of Q2.
E-5 production_users.csv: every user with any production login, not only administrators.
E-6 policy_access_control_v4.pdf: the access control policy document.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Checks
Every line in the fixture carries a verdict
Accept and reject land on the right lines
Each rejection cites a valid reason code
Next on your path · GRC Fundamentals
Policy, Standard, Procedure
The document hierarchy that governs everything you will later encode, and how to keep it from rotting.
