Write a suite that can actually fail

Objectives

A policy author asks you to "just add a test" for the public-bucket rule. The existing suite has nine cases, every one of them a violation, and it asserts that evaluation completed without raising.

Write the minimum suite that can distinguish a correct rule from a broken one: cases that name their input and their expected decision, a violation, a compliant input, an input with the field under test absent, assertions targeting the decision rather than the run, and a suite that refuses to pass on an empty fixture set.

rule: storage.deny_public_bucket
  denies a bucket when acl == "public-read"
  denies a bucket whose policy grants read to every principal

observed inputs:
  { "acl": "public-read", "encryption": "aes256" }     # violation
  { "acl": "private",     "encryption": "aes256" }     # compliant
  { "encryption": "aes256" }                            # acl absent: engine default applies

incident INC-4471: a bucket with no acl field was allowed through.

Step-by-step hints

The guided walkthrough, in full. It is free: no XP cost, and it does not touch the paid hint ladder or the reveal gate.

1. Step 1

   Ask what the current suite could never detect

   Nine violations and an execution assertion means an always-deny rule passes, an always-allow rule passes as long as it does not throw, and a rule that matches nothing passes silently.

   Each case you add should close one of those blind spots: write them in that order and the shape of the suite falls out.

   Cases name an input and an expected decisionAssertions target the decision, not the run

2. Step 2

   Fix what the suite asserts on before adding cases

   The top-level assertion target currently refers to the fact that the rule executed. Change it to the thing an auditor cares about: the verdict the rule produced for a given input.

   Then turn on the guard that makes a suite with nothing to iterate over a failure rather than a success.

   Assertions target the decision, not the runAn empty fixture set fails the suite

3. Step 3

   Give every case a name, an input and an expectation

   Each entry needs three parts: a name that describes the behaviour rather than the number, the input object itself, and the expected decision.

   The expected decision is one of the two verdicts the rule can return: the fixture describes both.

   Cases name an input and an expected decision

4. Step 4

   Cover the violation and the clean input

   Take the first observed input for the case the rule must catch, and the second for the case it must leave alone. The second is the one that catches an inverted condition.

   A violation the rule must catchA compliant input the rule must leave alone

5. Step 5

   Reproduce the incident

   The third observed input has no acl field at all: the shape that got through in INC-4471. Add it with the input object genuinely missing that key, not with the key set to an empty or null value, and expect the verdict the incident says it should have received.

   An input with the field under test absent

Rules summary

• XP on pass

  40 bonus XP the first time every check goes green.

• Paid hints

  2 available. Each costs 25% of the lab XP (about 10 XP).

• Reveal solution

  Scores 0 XP and unlocks only after 3 check runs.

• Attempt cap

  60 check runs per learner. The runner warns you in the last 10.

• Drafts

  Your editor content is saved in this browser, so a reload will not lose work.

• Walkthrough

  Always free and never penalised: use it before spending a hint.