grceng.academy
YAML lab 45 bonus XP

Make an AI RMF record operable

Start labRead the room

Objectives

The AI inventory record for the triage model says the framework is applied: risk rated high, status managed, evaluated before release in March. In July the vendor upgraded the underlying foundation model and nobody noticed until complaint volume rose in September.

Rewrite the record so each function produces an artefact: an accountable owner, the context the system operates in, a scheduled evaluation with a threshold, change triggers that include a model version change, and a dated treatment decision.

What happened, and what the platform can observe

system: clinical triage support (vendor foundation model, fine-tuned)
march   : pre-release evaluation, accuracy threshold documented, signed off
july    : vendor upgraded the base model; no code change, no deployment, no ticket
september: accuracy for one language cohort down 11 points, found via complaints

observable without a deployment:
  model_version, prompt_version, vendor release notes feed

register today:
  risk: "degraded output for minority-language users" -> rated high, status managed,
        no measurement, no threshold, no dated decision

Checks the server runs (5)

  • An accountable owner is named
  • Purpose and affected people are recorded
  • Evaluation is scheduled with a metric and threshold
  • A model version change triggers re-evaluation
  • A dated accept, reduce or stop decision exists

Check names are all you see here: the expectations that decide pass or fail, and the reference solution, stay on the server.

Step-by-step hints

The guided walkthrough, in full. It is free: no XP cost, and it does not touch the paid hint ladder or the reveal gate.

  1. Step 1

    Govern and Map first

    Name a durable owning team rather than a person, then record what the system is for and who is affected by its output: the fixture describes both in its first line.

    An accountable owner is namedPurpose and affected people are recorded

  2. Step 2

    Turn a one-off evaluation into a control

    Change the cadence from pre-release only to something that repeats, and carry the metric and the documented threshold from the March run onto the record so a later run can pass or fail against the same bar.

    A statistical system needs a threshold, not a boolean.

    Evaluation is scheduled with a metric and threshold

  3. Step 3

    Watch what actually changes

    List the triggers that force a re-evaluation. Code changes are not enough here: the fixture names the observable that moved in July without any deployment at all.

    A model version change triggers re-evaluation

  4. Step 4

    Make Manage produce a decision

    Replace the status word with one of accept, reduce or stop, and date it. An undated status is a conversation; a dated decision is a control.

    A dated accept, reduce or stop decision exists

Rules summary

  • XP on pass

    45 bonus XP the first time every check goes green.

  • Paid hints

    2 available. Each costs 25% of the lab XP (about 11 XP).

  • Reveal solution

    Scores 0 XP and unlocks only after 3 check runs.

  • Attempt cap

    60 check runs per learner. The runner warns you in the last 10.

  • Drafts

    Your editor content is saved in this browser, so a reload will not lose work.

  • Walkthrough

    Always free and never penalised: use it before spending a hint.