CAIN-42 CAIN Studio
CAIN-42 · VAGI BENCHMARK & SOAK BOARD

Every test, every soak, every benchmark — with its verdict

VERIFIED WINS — 6-hour mission PASS 21,600 s · 39,449 tasks · chain intact (33,813 solved) · 24-hour soak PASS 24.003 h · 0 anomalies · 0 divergences · 67 faults · VAGI E118 0.264→1.000 · E119 0.333→1.000 · E122 0.146→0.917 · E123-OW 0.188→0.906 (Strategic 1.000) · 8/8 bundles verify · 192/192 checks.
—PASS
—FAIL
—RUNNING
—test runs

Last win: — · Next: — · Updated: —

Live agent feed

Waiting for the first heartbeat…

AGI: NOT ACHIEVED. This board shows what passed, what is running, what failed and what is planned. It never upgrades a run. The 48-hour soak (2026-11-18) failed its own signed verdict, and that is shown.

TestStartedDuration / stateResultEvidenceLimits
Mission
E119 six-hour governed mission
2026-11-20 01:30Z21,600 sPASSbundle →verifier →39,449 tasks; 0 duplicates; journal intact True. 33,813 solved / 5,636 science failures. PASS = duration + journal integrity, not task success.
Soak
24-hour multi-region soak (cain-mr-01)
PASS under the criteria in force on 2026-11-02; predates the 2026-11-16 streak criterion and the 2026-11-15 PBFT incident.
2026-11-02 21:39Z24.003 hPASSbundle →standalone verifier →25 checkpoints; 0 anomalies; 0 divergences; 67 injected faults; 10,994 ALLOW / 177 DENY; all four nodes at height 142,689.
Verification
Offline verifier sweep (E118–E124)
2026-11-20 11:08Z7 bundlesPASStest report →162/162 checks passed across 7 bundles, run with the published standalone verifiers that import no CAIN code.
VAGI benchmark
E118 unseen primitives + search acquisition
2026-11-20sealed suitePASS (26/26)bundle →OOD 0.264 → 1.000 (gain +0.736, replicated +0.778); finite grammar; constructed OOD; no external reviewer.
VAGI benchmark
E119 learned macro grammar
2026-11-20sealed depth-3 suitePASS (16/16)bundle →fixed 0.333 → learned 1.000 (p≈1e-7; replicated +0.583); base language supplied; constructed tasks.
VAGI benchmark
E122 neural abstraction synthesis
2026-11-20n=48 sealedPASS on G05 (37/37 verifier); overall PARTIALbundle →E120 symbolic 0.000 / neural alone 0.146 → CAIN neural organism 0.917 (Holm p=3e-11); local CPU models; three gates remain PARTIAL; no external reviewer.
VAGI benchmark
E123 Open-World capability acquisition (32 unfamiliar families)
2026-11-20160 sealed instancesPASS (G02–G03 · 30/30 verifier)E123-OW bundle →verifier →Full CAIN E123 0.906 vs compute-matched 0.188 & no-causal 0.469 (Holm p<1e-12); 29/32 verified 13-field capabilities; overall E123-OW PARTIAL (16 PASS / 1 PARTIAL / 3 BLOCKED: G11, G18, G19); constructed families; AGI NOT ACHIEVED.
VAGI benchmark
E123 Strategic Test (Software → Cybersecurity IAM transfer)
2026-11-20n=36 sealed (+12 adversarial)PASS (1.000 vs 0.389 base)E123-OW bundle →Autonomously acquired temporal precondition-barrier capability from software traces → transferred to sealed cybersecurity IAM chains: 1.000 vs base Qwen2.5-0.5B 0.389, compute-matched 0.556, E120 frozen 0.000 (Holm p=6.10e-5); constructed benchmark.
VAGI benchmark
E123 Causal World Model & 6-vertical transfer + negative rejection
2026-11-2048 worlds · 14 pos · 12 negPASS (G04–G07)E123-OW bundle →Active EIG+HMM 0.917 vs 2x passive 0.000 (shift det 0.833, noise abstain 1.000); 10 positive transfers (5 structurally distinct) + 12/12 harmful negative transfers rejected fail-closed; 9-role discovery (4 cycles, 2 rejected).
Governance
E123 Metacognition, Poison Defense, Collective & Multiprocess MCPGate
2026-11-2014 TCP scenarios · 25 poison · 24 SIPASS (G09–G10, G12–G16)E123-OW bundle →Metacognition Brier 0.125 (vs 0.312); 0/25 memory poison accepted; SIGTERM crash recovery PASS; 7-arch collective 1.000 vs 0.458 (0/30 Byzantine); 24 self-improve candidates (4/4 governance attacks blocked); 14/14 TCP MCPGate + WAL kill-switch (0 effects after KILL).
Soak
48-hour multi-region soak (cain-mr-03) — NEW
2026-11-20 11:05ZstartingRUNNINGlive bundle →Started 2026-11-20 11:05Z; first checkpoint due ~1 h after start; verdict only at the end.
Soak
24-hour multi-region soak (cain-mr-01) — NEW
2026-11-20 11:05ZstartingRUNNINGlive bundle →Started 2026-11-20 11:05Z; first checkpoint due ~1 h after start; verdict only at the end.
Mission
E120 12-hour governed mission
2026-11-20 02:44Z12.00 h of 12 hCOMPLETEDbundle →restarts 3; journal records 8299; original start/deadline preserved across restart.
Mission
E122 12-hour governed neural mission
2026-11-20 04:38Z12.00 h of 12 hCOMPLETEDbundle →restarts 2; journal records 1559; original start/deadline preserved across restart.
AGI benchmark
SWE-bench Verified (500 tasks)
2026-11-20 07:35Z500 tasksCOMPLETE (awaiting publication)E124 bundle →0/500 resolved; official evaluator; one attempt.
AGI benchmark
ARC-AGI-2 public evaluation (40-task expansion)
2026-11-20 11:08Z0/40 tasksRUNNINGE122 bundle →Public ARC-AGI-2 evaluation set; 0.5B CPU model; 2 attempts per output; not the private leaderboard.
Soak
48-hour multi-region soak (cain-mr-03)
2026-11-18 04:43Z48.008 hFAILbundle →Signed final verdict: cluster status unreadable for 71 consecutive checks with no injected fault in progress (limit 5); replicas ended at different heights: [345052, 345053, 345054]. The published verifier did not check the streak criterion and returned PASS; that verifier gap is disclosed, and the run is recorded as FAIL.
VAGI benchmark
E123 Open-World real LLM 6-arm generalization (Qwen2.5-0.5B)
—24 sealed cases0.000 → 0.292 (E vs A)evidence →Real local Qwen2.5-0.5B-Instruct (0 weight updates, 0 authority, 48-token CPU budget); honest measured score.
VAGI benchmark
E123 Open-World AMA-Bench fresh slice (MIT, n=12)
—12 items [20:32]Token F1 0.2444 vs 0.2560 recency (5W/4L)evidence →Non-overlapping slice with SmolLM2-135M-Instruct on CPU; honest neutral delta (-0.0116, McNemar p=1.0).
VAGI benchmark
E123 Open-World ARC-AGI-2 fresh slice (Apache-2.0, n=6)
—6 tasks [8:14]0/6 exact matchevidence →Non-overlapping slice with SmolLM2-135M-Instruct on CPU; honest raw score. Not won.
VAGI benchmark
ARC-AGI-2 public sample (n=8)
—8 tasks0/8 exact matchevidence →Local 0.5B CPU model; honest raw score; not the private leaderboard. Not won.
VAGI benchmark
AMA-Bench (MIT, n=20)
—20 itemsToken F1 0.3158 → 0.3538 (+0.0380)evidence →Small local model; token F1 only; LLM-judge accuracy blocked without frontier credit.
VAGI benchmark
E124 long-horizon 70B agent
—18 attempts13/18 solved; fitted 50% horizon ≈2.0 hevidence →CAIN-native 6-task suite, not METR HCAST; frontier METR 50% horizons are 6.4–17.4 h.
VAGI benchmark
E123 METR time-horizon comparison
——UNRANKED on METRevidence →No METR suite run; governance is a separate axis; not an intelligence score.
VAGI benchmark
E79 model-backed collective (historical)
—104 tasks35/104 vs 10/104 deterministicevidence →One 0.5B model; collaboration gave no significant gain; constructed tasks.
VAGI benchmark
E75 held-out code repair (historical)
—8 tasks8/8 vs 0/8 previousevidence →Narrow single-function repair; frontier tasks 1/4; local model alone 1/8.
VAGI benchmark
GSM8K small probe (historical)
—10 items1/10evidence →Ten items with a tiny local model; not a benchmark result.
Planned
ARC-AGI-2 full public set (120 tasks)
—plannedPLANNEDtest board →Harness ready; 40-task expansion running first; 0.5B–1.5B CPU models.
Planned
GSM8K 50-item evaluation
—plannedPLANNEDtest board →Local model; exact-match scoring; small sample, disclosed.
Planned
HumanEval / MBPP code benchmarks
—plannedPLANNEDtest board →Local model; pass@1; not a substitute for SWE-bench.
Planned
72-hour multi-region soak
—plannedPLANNEDtest board →After the fresh 48-hour run; same signed checkpoints and verifier.
Planned
6-hour permit-enforced mission
—plannedPLANNEDtest board →E119's endurance loop used 0 permits; this closes that gap with single-use permits.
Planned
10,000 aggregate worker-hours
—plannedPLANNEDtest board →Long-horizon workforce campaign; not yet achieved.
Planned
External human reviewer
—plannedPLANNEDtest board →An independent person runs the verifier and signs findings.
Planned
Multi-org 2-of-3 trust quorum
—plannedPLANNEDtest board →Independent organizations sign Slots B/C; not yet met.
Planned
Production MCPGate path
—plannedPLANNEDtest board →The boundary so far is loopback/one host; production path not tested.
Planned
Hardware attestation
—plannedPLANNEDtest board →SGX/SEV/Nitro not available on this host; software attestation only.
Planned
Full pytest suite green
—plannedPLANNEDtest board →Last recorded full-suite state has failures; a clean run is still to be produced.

How to read this

AGI evidence index · all bundles · home