10 domains × 50 held-out tasks (500), plus a 100-task distribution-shift probe. The library arm is a hand-written procedure set; the 500/500 is a ceiling on this constructed suite, not a VAGI claim.
Paid model baseline (2026-11-18) — the ceiling test
Claude Haiku 5.5: 481/500 (96.2%) · Claude Fable 5.2: 96/100 (96.0%), same policy card, no tools, no memory, one attempt. The hand-written library: 500/500.
Reading: a frontier model already solves ~96% of the in-distribution suite. The 500/500 is a procedure ceiling, not a moat. The 5/100 distribution-shift probe is the honest gap, and the model arm did not run it.
Raw rows, cost, README and verifier →