# E91 Causal Reasoning Report

Held-out split, run once after the design was frozen (commit 39840d749; prereg artifacts/e91/causal/HELDOUT_PREREGISTRATION.json). Acceptance = verticals.check only. No language model. Per-row answers: artifacts/e91/causal/causal_suite_heldout_with_outputs.json (a re-execution of the frozen code that adds outputs; acceptance identical row by row).

| family | tasks | CAIN causal | controlled experiment | stratified log | naive log |
|---|---|---|---|---|---|
| hypothesis_test | 10 | 10 | 10 | 5 | 0 |
| logged_ab | 10 | 10 | 0 | 10 | 8 |
| scenario_test | 10 | 10 | 10 | 9 | 0 |
| underpowered | 10 | 10 | 10 | 0 | 0 |
| **total** | 40 | 40 | 30 | 24 | 8 |

Evidence classes assigned by the reasoner (held-out):

- hypothesis_test: INTERVENTION_SUPPORTED 2, REPLICATED 8
- logged_ab: RANDOMISED_BY_DESIGN 10
- scenario_test: INTERVENTION_SUPPORTED 3, REPLICATED 7
- underpowered: INSUFFICIENT_EVIDENCE 10

Root cause vs symptom, and a rule that does not transfer:

| family | tasks | bisect | linear probe | prior order | symptom blame |
|---|---|---|---|---|---|
| chain_diagnosis | 10 | 10 | 5 | - | 0 |
| skewed_prior_diagnosis | 10 | 2 | 3 | 10 | 0 |

## Capabilities tested

1. Cause of a failure / root vs symptom: chain diagnosis, bisection vs blaming the observed symptom.
2. Competing explanations and a discriminating experiment: A-causes-Y vs C-causes-both, separated by a balanced 2x2 intervention.
3. Belief update: the ledger records BELIEF_UPDATE when an intervention contradicts the correlation.
4. Replication: two independent halves of the budget; REPLICATED only when both agree.
5. Calibrated abstention: underpowered tasks answered INSUFFICIENT_EVIDENCE with the sentence 'I do not have enough evidence to establish this conclusion'.
6. A rule that does not transfer: bisection is optimal on uniform chains and poor under skewed failure priors.

## Limits

- Designer-written deterministic procedure; this is CAIN infrastructure reasoning, not model reasoning and not learning.
- One dev-split fix before freezing (covariate-adjusted estimate on randomised logs; dev logged_ab 7->10).
- Synthetic same-project tasks (session ec's generators); 10 held-out tasks per family.
- Overlaps session ec's strategy repertoire (reuses its log/decision helpers); adds evidence-class selection and replication.
- Avoid-repeating-a-disproven-strategy across tasks was not separately measured.
