# VAGI distribution-shift probe — paid model baseline (2026-11-18)

## The result that matters

The hand-written library scores **5/100** on the 100-task distribution-shift probe. Three raw
frontier models, given the probe's data and the intended policy card, score far higher:

| Arm | Model | Scored | Accepted | Rate | 95% Wilson | Cost |
|---|---|---:|---:|---:|---|---:|
| haiku-shift | Claude Haiku 5.5 | 100 | **74** | **74.0%** | [0.6463, 0.816] | $0.0149 |
| sonnet-shift | Claude Sonnet 5.5 | 100 | **81** | **81.0%** | [0.7222, 0.8749] | $0.2603 |
| fable-shift | Claude Fable 5.2 | 65 | **54** | **83.1%** | [0.7218, 0.9028] | $0.7511 |
| fable-shift-replication | Claude Fable 5.2 (seed 9503) | 23 | **20** | **87.0%** | [0.6787, 0.9546] | $0.2859 |

**The 5/100 is substantially a library-coverage gap, not a general-intelligence gap.** The probe is
adversarial and partly underspecified: the software hidden target (x % 3) is not inferable from the
public tests, and **all models scored 0 on tool_orchestration**. That is a probe/prompt problem, not a
general model failure. The result does not show VAGI; it shows the suite's shift probe is a coverage
map, not an AGI test.

## Cost and the cap

Shift spend: **$1.312174** of the **$1.280558** remaining. Overshoot: **$0.031616**
because concurrent requests passed the budget check together. The Fable arm stopped at
65/100; the replication stopped at 23/100. Rates are on scored tasks only.

## Per-domain

| Domain | Haiku | Sonnet | Fable (scored) | Library |
|---|---:|---:|---:|---:|
| business_decision | 10/10 | 10/10 | 7/7 | 0/10 |
| data_analysis | 10/10 | 10/10 | 7/7 | 0/10 |
| governance | 10/10 | 10/10 | 6/6 | 0/10 |
| memory_retrieval | 10/10 | 10/10 | 6/6 | 0/10 |
| planning | 7/10 | 6/10 | 6/6 | 0/10 |
| recovery | 10/10 | 10/10 | 6/6 | 0/10 |
| science_calculation | 10/10 | 10/10 | 7/7 | 0/10 |
| security_triage | 2/10 | 10/10 | 7/7 | 0/10 |
| software_repair | 5/10 | 5/10 | 2/7 | 5/10 |
| tool_orchestration | 0/10 | 0/10 | 0/6 | 0/10 |

## Verify

```
python3 -I verify_offline.py .
```

Recomputes counts and cost from raw rows, regenerates the shift tasks for the two seeds, re-scores
every answer with the published shift checker, and checks that the prompt view contains no `expect`,
`hidden`, `fixes` or `bad` field. It imports the suite source bundled here, not CAIN.

## Limits

- Fable arms are partial (budget cap). Rates are on scored tasks, not 100.
- The probe is deliberately adversarial; it is not a fair AGI benchmark.
- Software target not inferable; tool task failed for every model.
- No tools, no memory, one attempt; no external reviewer.
- Not AGI, not VAGI, not production, not a certification.
