CAIN-42 · VAGI BENCHMARK & SOAK BOARD
Every test, every soak, every benchmark — with its verdict
VERIFIED WINS — 6-hour mission PASS 21,600 s · 39,449 tasks · chain intact (33,813 solved) ·
24-hour soak PASS 24.003 h · 0 anomalies · 0 divergences · 67 faults ·
VAGI E118 0.264→1.000 · E119 0.333→1.000 · E122 0.146→0.917 · E123-OW 0.188→0.906 (Strategic 1.000) ·
8/8 bundles verify · 192/192 checks.
—PASS
—FAIL
—RUNNING
—test runs
Last win: — · Next: — · Updated: —
Live agent feed
Waiting for the first heartbeat…
AGI: NOT ACHIEVED. This board shows what passed, what is running, what failed and what is planned. It never upgrades a run. The 48-hour soak (2026-11-18) failed its own signed verdict, and that is shown.
| Test | Started | Duration / state | Result | Evidence | Limits |
|---|---|---|---|---|---|
| Mission E119 six-hour governed mission | 2026-11-20 01:30Z | 21,600 s | PASS | bundle →verifier → | 39,449 tasks; 0 duplicates; journal intact True. 33,813 solved / 5,636 science failures. PASS = duration + journal integrity, not task success. |
| Soak 24-hour multi-region soak (cain-mr-01) PASS under the criteria in force on 2026-11-02; predates the 2026-11-16 streak criterion and the 2026-11-15 PBFT incident. | 2026-11-02 21:39Z | 24.003 h | PASS | bundle →standalone verifier → | 25 checkpoints; 0 anomalies; 0 divergences; 67 injected faults; 10,994 ALLOW / 177 DENY; all four nodes at height 142,689. |
| Verification Offline verifier sweep (E118–E124) | 2026-11-20 11:08Z | 7 bundles | PASS | test report → | 162/162 checks passed across 7 bundles, run with the published standalone verifiers that import no CAIN code. |
| VAGI benchmark E118 unseen primitives + search acquisition | 2026-11-20 | sealed suite | PASS (26/26) | bundle → | OOD 0.264 → 1.000 (gain +0.736, replicated +0.778); finite grammar; constructed OOD; no external reviewer. |
| VAGI benchmark E119 learned macro grammar | 2026-11-20 | sealed depth-3 suite | PASS (16/16) | bundle → | fixed 0.333 → learned 1.000 (p≈1e-7; replicated +0.583); base language supplied; constructed tasks. |
| VAGI benchmark E122 neural abstraction synthesis | 2026-11-20 | n=48 sealed | PASS on G05 (37/37 verifier); overall PARTIAL | bundle → | E120 symbolic 0.000 / neural alone 0.146 → CAIN neural organism 0.917 (Holm p=3e-11); local CPU models; three gates remain PARTIAL; no external reviewer. |
| VAGI benchmark E123 Open-World capability acquisition (32 unfamiliar families) | 2026-11-20 | 160 sealed instances | PASS (G02–G03 · 30/30 verifier) | E123-OW bundle →verifier → | Full CAIN E123 0.906 vs compute-matched 0.188 & no-causal 0.469 (Holm p<1e-12); 29/32 verified 13-field capabilities; overall E123-OW PARTIAL (16 PASS / 1 PARTIAL / 3 BLOCKED: G11, G18, G19); constructed families; AGI NOT ACHIEVED. |
| VAGI benchmark E123 Strategic Test (Software → Cybersecurity IAM transfer) | 2026-11-20 | n=36 sealed (+12 adversarial) | PASS (1.000 vs 0.389 base) | E123-OW bundle → | Autonomously acquired temporal precondition-barrier capability from software traces → transferred to sealed cybersecurity IAM chains: 1.000 vs base Qwen2.5-0.5B 0.389, compute-matched 0.556, E120 frozen 0.000 (Holm p=6.10e-5); constructed benchmark. |
| VAGI benchmark E123 Causal World Model & 6-vertical transfer + negative rejection | 2026-11-20 | 48 worlds · 14 pos · 12 neg | PASS (G04–G07) | E123-OW bundle → | Active EIG+HMM 0.917 vs 2x passive 0.000 (shift det 0.833, noise abstain 1.000); 10 positive transfers (5 structurally distinct) + 12/12 harmful negative transfers rejected fail-closed; 9-role discovery (4 cycles, 2 rejected). |
| Governance E123 Metacognition, Poison Defense, Collective & Multiprocess MCPGate | 2026-11-20 | 14 TCP scenarios · 25 poison · 24 SI | PASS (G09–G10, G12–G16) | E123-OW bundle → | Metacognition Brier 0.125 (vs 0.312); 0/25 memory poison accepted; SIGTERM crash recovery PASS; 7-arch collective 1.000 vs 0.458 (0/30 Byzantine); 24 self-improve candidates (4/4 governance attacks blocked); 14/14 TCP MCPGate + WAL kill-switch (0 effects after KILL). |
| Soak 48-hour multi-region soak (cain-mr-03) — NEW | 2026-11-20 11:05Z | starting | RUNNING | live bundle → | Started 2026-11-20 11:05Z; first checkpoint due ~1 h after start; verdict only at the end. |
| Soak 24-hour multi-region soak (cain-mr-01) — NEW | 2026-11-20 11:05Z | starting | RUNNING | live bundle → | Started 2026-11-20 11:05Z; first checkpoint due ~1 h after start; verdict only at the end. |
| Mission E120 12-hour governed mission | 2026-11-20 02:44Z | 12.00 h of 12 h | COMPLETED | bundle → | restarts 3; journal records 8299; original start/deadline preserved across restart. |
| Mission E122 12-hour governed neural mission | 2026-11-20 04:38Z | 12.00 h of 12 h | COMPLETED | bundle → | restarts 2; journal records 1559; original start/deadline preserved across restart. |
| AGI benchmark SWE-bench Verified (500 tasks) | 2026-11-20 07:35Z | 500 tasks | COMPLETE (awaiting publication) | E124 bundle → | 0/500 resolved; official evaluator; one attempt. |
| AGI benchmark ARC-AGI-2 public evaluation (40-task expansion) | 2026-11-20 11:08Z | 0/40 tasks | RUNNING | E122 bundle → | Public ARC-AGI-2 evaluation set; 0.5B CPU model; 2 attempts per output; not the private leaderboard. |
| Soak 48-hour multi-region soak (cain-mr-03) | 2026-11-18 04:43Z | 48.008 h | FAIL | bundle → | Signed final verdict: cluster status unreadable for 71 consecutive checks with no injected fault in progress (limit 5); replicas ended at different heights: [345052, 345053, 345054]. The published verifier did not check the streak criterion and returned PASS; that verifier gap is disclosed, and the run is recorded as FAIL. |
| VAGI benchmark E123 Open-World real LLM 6-arm generalization (Qwen2.5-0.5B) | — | 24 sealed cases | 0.000 → 0.292 (E vs A) | evidence → | Real local Qwen2.5-0.5B-Instruct (0 weight updates, 0 authority, 48-token CPU budget); honest measured score. |
| VAGI benchmark E123 Open-World AMA-Bench fresh slice (MIT, n=12) | — | 12 items [20:32] | Token F1 0.2444 vs 0.2560 recency (5W/4L) | evidence → | Non-overlapping slice with SmolLM2-135M-Instruct on CPU; honest neutral delta (-0.0116, McNemar p=1.0). |
| VAGI benchmark E123 Open-World ARC-AGI-2 fresh slice (Apache-2.0, n=6) | — | 6 tasks [8:14] | 0/6 exact match | evidence → | Non-overlapping slice with SmolLM2-135M-Instruct on CPU; honest raw score. Not won. |
| VAGI benchmark ARC-AGI-2 public sample (n=8) | — | 8 tasks | 0/8 exact match | evidence → | Local 0.5B CPU model; honest raw score; not the private leaderboard. Not won. |
| VAGI benchmark AMA-Bench (MIT, n=20) | — | 20 items | Token F1 0.3158 → 0.3538 (+0.0380) | evidence → | Small local model; token F1 only; LLM-judge accuracy blocked without frontier credit. |
| VAGI benchmark E124 long-horizon 70B agent | — | 18 attempts | 13/18 solved; fitted 50% horizon ≈2.0 h | evidence → | CAIN-native 6-task suite, not METR HCAST; frontier METR 50% horizons are 6.4–17.4 h. |
| VAGI benchmark E123 METR time-horizon comparison | — | — | UNRANKED on METR | evidence → | No METR suite run; governance is a separate axis; not an intelligence score. |
| VAGI benchmark E79 model-backed collective (historical) | — | 104 tasks | 35/104 vs 10/104 deterministic | evidence → | One 0.5B model; collaboration gave no significant gain; constructed tasks. |
| VAGI benchmark E75 held-out code repair (historical) | — | 8 tasks | 8/8 vs 0/8 previous | evidence → | Narrow single-function repair; frontier tasks 1/4; local model alone 1/8. |
| VAGI benchmark GSM8K small probe (historical) | — | 10 items | 1/10 | evidence → | Ten items with a tiny local model; not a benchmark result. |
| Planned ARC-AGI-2 full public set (120 tasks) | — | planned | PLANNED | test board → | Harness ready; 40-task expansion running first; 0.5B–1.5B CPU models. |
| Planned GSM8K 50-item evaluation | — | planned | PLANNED | test board → | Local model; exact-match scoring; small sample, disclosed. |
| Planned HumanEval / MBPP code benchmarks | — | planned | PLANNED | test board → | Local model; pass@1; not a substitute for SWE-bench. |
| Planned 72-hour multi-region soak | — | planned | PLANNED | test board → | After the fresh 48-hour run; same signed checkpoints and verifier. |
| Planned 6-hour permit-enforced mission | — | planned | PLANNED | test board → | E119's endurance loop used 0 permits; this closes that gap with single-use permits. |
| Planned 10,000 aggregate worker-hours | — | planned | PLANNED | test board → | Long-horizon workforce campaign; not yet achieved. |
| Planned External human reviewer | — | planned | PLANNED | test board → | An independent person runs the verifier and signs findings. |
| Planned Multi-org 2-of-3 trust quorum | — | planned | PLANNED | test board → | Independent organizations sign Slots B/C; not yet met. |
| Planned Production MCPGate path | — | planned | PLANNED | test board → | The boundary so far is loopback/one host; production path not tested. |
| Planned Hardware attestation | — | planned | PLANNED | test board → | SGX/SEV/Nitro not available on this host; software attestation only. |
| Planned Full pytest suite green | — | planned | PLANNED | test board → | Last recorded full-suite state has failures; a clean run is still to be produced. |
How to read this
- PASS on a soak = the signed checkpoints verify under the published criteria. It is not a third-party audit.
- PASS on a mission = real wall-clock duration plus an intact hash chain. It is not a claim that every task succeeded.
- MEASURED = a benchmark score, including negative results (ARC-AGI-2 0/8 is shown as 0/8).
- PLANNED = not run. Nothing in this bucket is evidence.
- The 48-hour run's signed final checkpoint says FAIL; the published verifier's PASS was a verifier gap and is disclosed.