FIRME AI CORPORATION, a three-person team. Baseline: Evolution 79 at f4449aa7913623bc034fa4434ca96c2a2919a889.
Every number on this page is read from the signed artifacts below, and the bundle's verifier recomputes them from raw per-task rows.
Download the evidence bundle (ZIP) Read the offline verifier How to reproduce Final report
c08_evidence_first, gained +0.1042.
The routing direction repeated on 24 replication tasks (10 vs 8), without significance. These results are published as measured.13 PASS · 10 PARTIAL · 1 FAIL · 5 BLOCKED · 3 NOT EVALUABLE (32 gates). NOT RELEASED. Evolution 80 is a development-branch result. Production deployment is not authorized.
| Gate | Status | Basis |
|---|---|---|
| 01 Baseline reproduction (E79 144/144, clean room, second host, 25 tests) | PASS | docs/evolution-80/EVOLUTION79_REPRODUCTION.md |
| 02 Public evidence reconciliation | PARTIAL | provenance established and branch files corrected; live sites unchanged (deployment not authorized) |
| 03 Real local model inference, governed and signed | PASS | 1154 calls, 153132 tokens, 0 cloud |
| 04 Adaptive routing beats the best fixed path (H1) | PARTIAL | router 22/48 vs m15_direct 19/48, p=0.647606 |
| 05 Collaboration advantage (H2) | FAIL | best multi-agent arm c08_evidence_first delta 0.1042; significant arms: [] |
| 06 System recognises when collaboration hurts | PASS | arms measured worse than one agent: ['c07_specialist_committee']; the adaptive graph communicates only on disagreement |
| 07 Replication in a separate process (H3) | PASS | {"a_correct": 8, "b_correct": 10, "n": 24, "diff": 0.0833} |
| 08 Heterogeneous model registry | PARTIAL | 3 local CPU models measured; larger and other-family models NOT EVALUATED |
| 09 Remote model providers | BLOCKED | no credential or funding; purchase prohibited; never mocked |
| 10 Specialist capability profiles | PARTIAL | 10 of 20 measured; 10 NOT EVALUATED |
| 11 Skill acquisition (LoRA) with held-out, transfer and regression checks | PASS | held-out 0.0 -> 1.0; transfer delta 0.125; NO_REGRESSION; promotion AWAITING_HUMAN_APPROVAL |
| 12 Memory answer accuracy (constructed set) | PASS | answer accuracy governed_retrieval 1.0, naive_recent 0.2143, no_memory 0.2143, ungoverned_retrieval 0.7143 |
| 13 Public memory benchmarks (LongMemEval, LoCoMo, MemoryAgentBench, AMA-Bench) | NOT EVALUABLE | not run |
| 14 Autonomous research project end to end | PASS | pre-registered, frozen policy, raw rows, separate-process replication; local fixture-free literature review of abstracts only |
| 15 Self-improvement candidate evaluated, not self-promoted | PASS | benefit demonstrated: False; state AWAITING_HUMAN_APPROVAL |
| 16 Agentic 75 audit and composition | PARTIAL | 75/75 engines executed on bundled samples; 0 model-backed; 0 independently evaluated; 4 composed in a mission |
| 17 Durable execution under injected failures (one host) | PASS | injected 8/8; duplicates 0; stale commits refused 2 |
| 18 Cross-host scheduling, checkpoint replication, network partition | NOT EVALUABLE | not run in E80; E72/E78 evidence preserved |
| 19 Security adversarial lab | PASS | 51/51 blocked; same-project lab, bubblewrap only |
| 20 Kill switch | PARTIAL | in-process gate refuses decisions and pre-issued grants; no distributed halt tested |
| 21 Stronger isolation (gVisor, Kata, Firecracker, Kubernetes Agent Sandbox) | NOT EVALUABLE | not installed |
| 22 Mission controller with evidence-based completion | PASS | COMPLETED_WITH_UNVERIFIED_ITEMS_EXCLUDED; nodes {'VERIFICATION_FAILED': 4, 'VERIFIED': 9}; verified-but-wrong 0 |
| 23 ClawX command-center backend | PARTIAL | 20/20 views backed by artifacts; local token-gated endpoint; not deployed; no live process view |
| 24 MCPGate enforcement | PARTIAL | reference gate enforces single-use grants on every model and tool call and passes bypass tests; the production MCP proxy was not modified or re-tested |
| 25 CAIN Studio unified public explanation | PARTIAL | architecture and status documented; no site change deployed |
| 26 Endurance 1 h | PASS | 3600 s actual elapsed in one process; 80 cycles; 640 governed tasks (560 local model calls, 80 sandbox executions); 0 failures; 0 duplicate task ids; checkpoint chain intact; duty-cycled (44% busy); accuracy 252/640 is a small-model score, not a quality claim |
| 27 Endurance 6 h / 24 h / 72 h / 168 h | BLOCKED | not run |
| 28 Signed evidence with offline verifier | PASS | see evidence.json; self-generated key |
| 29 Second-environment verification | PARTIAL | 166/166 checks on 45.76.169.191 (development replica); same administrative domain, not an independent witness |
| 30 Transparency log / independent witness | BLOCKED | publication not authorized; no Rekor entry; no independent witness |
| 31 Third-party audit | BLOCKED | none commissioned |
| 32 Production deployment | BLOCKED | prohibited for Evolution 80 |
| Arm | Correct | Accuracy | Model calls |
|---|---|---|---|
c01_single | 11/48 | 22.9% | 48 |
c02_parallel_vote | 11/48 | 22.9% | 144 |
c03_pairwise_debate | 11/48 | 22.9% | 139 |
c04_critic_reviser | 11/48 | 22.9% | 96 |
c05_planner_executor | 13/48 | 27.1% | 96 |
c06_hierarchical | 15/48 | 31.2% | 181 |
c07_specialist_committee | 10/48 | 20.8% | 144 |
c08_evidence_first | 16/48 | 33.3% | 96 |
c09_parallel_synthesis | 13/48 | 27.1% | 192 |
c10_adaptive_graph | 22/48 | 45.8% | 60 |
det_exec | 6/6 | 12.5% | 0 |
fixed_pot05_else_direct05 | 16/48 | 33.3% | 48 |
fixed_pot15_else_direct15 | 27/48 | 56.2% | 48 |
m05_direct | 11/48 | 22.9% | 48 |
m05_pot | 5/29 | 10.4% | 29 |
m15_direct | 19/48 | 39.6% | 48 |
m15_pot | 13/29 | 27.1% | 29 |
oracle_any_path | 32/48 | 66.7% | 42 |
router | 22/48 | 45.8% | 42 |
1154 real local CPU model calls, 153132 tokens, 0 cloud calls, $0.00. Three small models (Qwen2.5 0.5B and 1.5B, SmolLM2-135M). Constructed closed-form tasks: unseen instances of known templates.
PASS: 3600 s actual elapsed in one process; 80 cycles; 640 governed tasks (560 local model calls, 80 sandbox executions); 0 failures; 0 duplicate task ids; checkpoint chain intact; duty-cycled (44% busy); accuracy 252/640 is a small-model score, not a quality claim. ACTUAL elapsed wall clock; no virtual or accelerated time. Campaigns of 6 hours and longer were not run.
| Shipped offline verifier | 166 / 166 checks on the build host |
|---|---|
| Second host | 166 / 166 on 45.76.169.191 (development replica). same administrative domain as the build host; not an independent witness. |
| Separately authored verifier | 52 / 52, verdict PASS. Written by a different session, imports no CAIN code; it was executed by the session that finished the evolution, on the same host. report |
| Merkle root | 1ae86e51063095240c42f436eda8787edbbe8a49539c4ed55da584a20093bc2a over 117 artifacts |
| Signing key | Ed25519, fingerprint 4acfa1815616155b7f5ebc1b70c9501fa189d7a9e2e5ffc934f0422c5f1f1e33. Self-generated key. The fingerprint has not been published out of band. No transparency-log entry. No independent witness. |
| Bundle ZIP SHA-256 | 81828ba427869e0299056c556cc9276b97f23f437ef3015ceda33455e32d4e3d |
Three benchmark runs were made on 2026-11-15. Runs 1 and 2 are preserved in the repository and superseded; only run 3 is evidence. Run 3 was interrupted when its session ended and was resumed with the unchanged command; no stage ran twice. Record: run_lineage.json, explanation: EVIDENCE_AND_REPRODUCTION.md.