# VAGI Behavior Suite v1 — paid model baseline (2026-11-18)

**What this is.** A raw frontier-model baseline on the published VAGI Behavior Suite. The model receives
the task data and the same policy card the library arm codifies. It gets **no tools, no memory, no
retries**, and one attempt per task. Invalid JSON counts as incorrect. This is an external baseline for
the suite, **not** a CAIN arm and **not** evidence of AGI/VAGI.

## Results

| Arm | Model | Tasks | Accepted | Rate | 95% Wilson | Cost |
|---|---|---:|---:|---:|---|---:|
| `haiku-5-5-full` | Claude Haiku 5.5 | 500 | **481** | **96.2%** | [0.9414, 0.9755] | $0.0475 |
| `fable-5-2-stratified` | Claude Fable 5.2 | 100 | **96** | **96.0%** | [0.9016, 0.9843] | $0.5819 |

Published context on the same 500 held-out tasks: no-procedure floor **72/500**, one fixed procedure
**112/500**, hand-written library **500/500**, governed **500/500**. Distribution shift (100 unfamiliar
variants): library **5/100**.

### The honest finding

The raw model scores about 96% in-distribution. The hand-written library scores 100%. Therefore the
**500/500 is a procedure ceiling, not a frontier-model discriminator**. The interesting number is the
**5/100 distribution-shift result**, where the library collapses. The model arm did not run the shift
probe, so this bundle does not show whether a frontier model transfers better. It shows only that the
in-distribution suite is easy for a frontier model when the policy is supplied.

### Per domain

| Domain | Haiku 5.5 | Fable 5.2 (10/domain) |
|---|---:|---:|
| business_decision | 35/50 | 10/10 |
| data_analysis | 48/50 | 9/10 |
| governance | 50/50 | 10/10 |
| memory_retrieval | 50/50 | 10/10 |
| planning | 50/50 | 10/10 |
| recovery | 50/50 | 10/10 |
| science_calculation | 48/50 | 10/10 |
| security_triage | 50/50 | 10/10 |
| software_repair | 50/50 | 7/10 |
| tool_orchestration | 50/50 | 10/10 |

## Cost

Total spent: **$0.629442** of the **$1.91** budget
(remaining **$1.280558**). Prices: Haiku 5.5 $0.10/$0.50 per MTok;
Fable 5.2 $8/$40 per MTok.

## Reproduce

```
python3 -I verify_offline.py .
```

The verifier regenerates the 500 held-out tasks, re-scores every raw answer with the
published suite checker, recomputes the counts, intervals and cost, and checks that the
prompt view contains no `fixes`, `hidden`, `bad` or reference answer. It imports the
suite source bundled here, not CAIN. This is offline recomputation with the published
checker, **not** an independent reimplementation and **not** a third-party audit.

## Limits

- Constructed, deterministic suite; not a public benchmark.
- Policy card supplied; this measures procedure application, not open-ended reasoning.
- No tools, no memory, no retries.
- One attempt per task; invalid JSON = incorrect.
- No independent verifier or witness.
- Not AGI, not VAGI, not production, not a certification.
