How the evals work.

Why the cases are organized by check, not by stage.

The judge's unit of judgment is the dimension — a stage is just a bundle of dimensions. If a case were a whole stage artifact and the judge got it wrong, you wouldn't know which check broke. So each case isolates one check: a snippet with exactly one planted defect (or its clean mirror image), and a miss diagnoses itself.

The golden set — 14 cases

Dimensions 4 (strategic fit) and 6 (falsifiability) have no PASS/FAIL pair — they are substantive: the judge may only emit a question with an owner, never a verdict, so there is no verdict to label. Dimension 4 appears only inside FW-01, where the absence of a verdict is what's being tested.

Structure, never quality.

Not one property judges whether the writing is good. They check presence, provenance, and vocabulary — the objective skeleton of each artifact. That's deliberate, and it draws the honest boundary: this suite catches a prompt that stops producing its contract (untagged figures, blockers without owners, a missing scope boundary). It does not catch writing that got worse while still ticking every box — quality belongs to the judge at runtime and to Layer 3, the designed-but-not-yet-run outcome evals with human labels.

The contract — 26 properties across 8 stages

Stage 0 (intake) is interactive — it needs a scripted human — and is out of scope here; its behavior is exercised by the judge suite and the challenge-budget rules in SKILL.md.

def pct_lines_tagged(t):
    """Every % figure must have a provenance tag or a sanctioned gap nearby
    (within 160 chars after it — markdown hard-wraps lines, so a line-based
    check false-fails). This is the never-invent-a-figure contract."""
    for m in re.finditer(r"\d+(\.\d+)?\s?%", t):
        window = t[m.start():m.end() + 160]
        if not re.search(r"\[(OBSERVED|REPORTED|ASSUMED|UNKNOWN|NEEDS EVIDENCE)", window):
            return False
    return True

Now watch them run.

Everything above is the mechanism — the answer keys. The dashboard shows what happened when both suites actually ran: every case's snippet and the judge's verbatim response, every stage's generated output and which properties held.