# E91 Forensic Audit

**Audit date:** 2026-11-18
**Branch:** `evolution-91-final` (from `evolution-91` at `f213ee4e1`, plus the evaluation slice)
**Status:** PARTIAL evidence release. NOT an independent audit, NOT a certification, NOT a production deployment.

## 1. What was actually executed

| Experiment | n | Result | Evidence |
|---|---:|---|---|
| Core held-out arms A–F | 160 tasks/arm | A 82, B 85, C 90, D 114, E 114, F 100 | `artifacts/e91/results/haiku/rows_heldout_*.jsonl` |
| Verified-learning arm G | 160 | 153/160 | same |
| Adaptive arm H | 6 unique rows | incomplete | `rows_heldout_H.jsonl` |
| Causal reasoner | 40 | 40/40 vs best single method 30/40 | `artifacts/e91/causal/causal_suite_heldout.json` |
| Transfer T1/T2 | 10 target + 10 control each | T1 target 10/10 vs D 5/10, control gate 10/10 vs off 2/10; T2 target 10/10 vs D 7/10, control gate 3/10 vs off 0/10 | `artifacts/e91/transfer/haiku/transfer_learning.json` |
| Learning curve | 0,1,2,5,10,20 dev examples | 114→153/160 | same |
| Poisoning | 90 forged experiences | 0 accepted, 90 rejected | same |
| Multi-agent run 1 | 7 configs × 20 | single 18/20; teams 19–20/20 at more calls | `artifacts/e91/team/team_summary_haiku.json` |
| Multi-agent run 2 | 7 configs × 20, shared budget | single 10/20; best team 8/20 | `team_summary_r2_haiku.json` |
| Authorization lab | 26 attacks | 26 blocked, 0 bypassed | `artifacts/e91/security/lab.json` |
| E91 long-horizon lab | 15 attacks | 15 blocked, 0 bypassed | same |
| 1-hour-class mission | 3,561.4 s | 7,200/7,200 verified, 0 duplicates, 22 faults, 0 post-fence effects | `mission/run_1h/mission_report.json` |
| Active kill | 123.2 s | 150/150 verified, 0 duplicates, 0 post-fence effects, kill effective 0.152 ms | `mission/active_kill/mission_report.json` |

## 2. Numbers that must not be quoted as more than they are

- **160 held-out tasks**, not a public benchmark. They are constructed by this project, and the same project wrote the checkers.
- **Haiku 5.5 is the only completed model.** Local Qwen2.5-0.5B runs did not finish every arm. No Sonnet/Opus/frontier comparison.
- **Paid spend $0.31**, 2,882 calls, hard ceiling $5. No further paid calls were made after the sessions stopped.
- **Arm H is incomplete:** 6 unique held-out rows, 0 accepted. It is not an adaptive-reasoning result.
- **1-hour gate is PARTIAL:** 3,561.4 s = 59.4 min. 6/24/72 h were not run.
- **Multi-agent: no benefit.** Run 1 was saturated and cost more; run 2 showed verification overhead under a shared budget, with false consensus and false acceptance.
- **Planning as a prompt step lowered acceptance** (F 100 vs D 114), p=0.065.
- **Memory as text did not help** (E 114 = D 114; B 85, C 90 vs A 82).
- **The learning gain is procedure selection.** G 153/160 vs D 114/160; the gain is entirely in families with a promoted procedure (70→109 of 110), zero elsewhere (44→44 of 50). The procedure repertoire is designer-supplied.
- **Transfer uses a designer-supplied abstract interface.** The result shows the verified selection carries across domains and the condition gate refuses it where it should; it does not show invention of a new abstraction.

## 3. Data-integrity findings

- `analyze.py` keeps the first row per `(task_id, arm)` and counts duplicates. It ignored **139 duplicate F rows** and **12 duplicate H rows** from resumed/parallel lanes. The published rates exclude those duplicates.
- All core numbers in the reports are regenerated from raw rows by `analyze.py`; none are hand-entered.
- The E91 evidence bundle is signed with a self-generated Ed25519 key. The key proves integrity relative to the manifest; it is not an independent witness.
- The offline verifier checks the manifest, every file digest, mission arithmetic (`verified == effects`, 0 duplicates, chain OK, 0 post-fence effects), and the security lab counts. It does not prove the experiments were honestly conducted.
- No source code, task secret, hidden ground truth or private key is published.

## 4. What is not proven

1. General intelligence, VAGI, AGI, or broad professional competence.
2. That a model reasons better because of CAIN. The model is unchanged; the gain is procedure selection.
3. That collaboration helps. The measured result is negative.
4. That the system invents procedures. It selects from a supplied repertoire.
5. That the results generalize beyond the constructed tasks, the single completed model, and the single host.
6. That the evidence is independently witnessed or certified. It is not.

## 5. Release decision

The evidence supports a **PARTIAL evidence release**, not a full capability release:

- The harness, six verticals, 160 held-out tasks, baselines, learning loop, causal reasoner, transfer, security, mission and offline verifier were executed.
- The strongest positive result is evidence-gated procedure reuse with cross-domain transfer and refusal.
- Planning, collaboration and adaptive reasoning are not demonstrated; long-horizon is partial.
- No independent audit, no customer deployment, no production integration.

**Release: RELEASED — PARTIAL EVIDENCE.** Public claims must carry the limitations.
