Review the policy test suite

Objectives

A team submits a policy test suite for review with a green badge and a note that coverage is 100%. Your job is to decide which cases give the organisation real assurance and which produce a pass that means nothing.

Accept the entries that could genuinely fail if the rule broke. Reject the rest.

Verdicts are accept or reject. Every rejection needs one reason code from: empty-fixture | asserts-execution-not-decision | no-negative-case | default-allow-untested.

G-1  A pair of cases on the same rule: one violation expecting deny, one compliant input expecting allow.
G-2  A suite that globs fixtures/*.json and loops; the directory is empty in CI and the suite reports pass.
G-3  A case supplying an input with the field under test absent, expecting deny.
G-4  A case that wraps the evaluator in try/except and passes when no exception is raised.
G-5  Fourteen cases on the encryption rule, every one of them a violation expecting deny.
G-6  A rule that falls through to allow on unrecognised input shapes, with no case exercising that path.

Step-by-step hints

The guided walkthrough, in full. It is free: no XP cost, and it does not touch the paid hint ladder or the reveal gate.

1. Step 1

   Read the request before the artefact

   Coverage tells you which lines executed. Review tells you whether the suite can distinguish a correct rule from a broken one: that is the only question here.

2. Step 2

   Judge every line, including the boring ones

   Six entries, six verdicts. Two of them are the strongest things in the suite; name them as accepts so they survive the next refactor.

   Every line in the fixture carries a verdict

3. Step 3

   Separate what is wrong from what is merely ugly

   Run the same thought experiment on each entry: break the rule in the most plausible way and ask whether this entry goes red. A suite of violations survives an always-deny rule; an execution assertion survives almost anything.

   The absent-field case looks like an edge case and is in fact the most valuable entry present, because it pins the engine's default behaviour.

   Accept and reject land on the right lines

4. Step 4

   Say why, in the vocabulary the programme uses

   Reserve the empty-fixture code for the suite that iterates over nothing, and the default-allow code for the untested fall-through path rather than for the all-violations suite.

   Each rejection cites a valid reason code

Rules summary

• XP on pass

  40 bonus XP the first time every check goes green.

• Paid hints

  2 available. Each costs 25% of the lab XP (about 10 XP).

• Reveal solution

  Scores 0 XP and unlocks only after 3 check runs.

• Attempt cap

  60 check runs per learner. The runner warns you in the last 10.

• Drafts

  Your editor content is saved in this browser, so a reload will not lose work.

• Walkthrough

  Always free and never penalised: use it before spending a hint.