# E125 — The Generalization Breakthrough (honest verdict: PARTIAL)

**Entity:** FIRME AI CORPORATION · **System:** CAIN-42 · **Evolution:** E125
**Frozen engine:** `be3494d96413bfd29f3f63ce621733cd2972ec19` before the confirmatory run.
**Preregistration:** `config/e125_generalization.json`, frozen 2026-11-21T12:47:46Z.
**Confirmatory challenge commitment:** `9f5d67555e04e99fbea80f1ec15b54c4228978a969d2fe8954ac90a6113a12e6`.
**AGI status:** NOT ACHIEVED. This is not a VAGI claim.

---

## 1. What was asked, and what was actually done

E125 asked whether CAIN-42 can acquire reusable capabilities, transfer them, learn from feedback,
remember, predict interventions, collaborate, and improve itself without weakening governance.
The work was split into a **confirmatory challenge** and eight supporting experiments.

**The confirmatory challenge is constructed.** The same three-person team wrote the engine and
the generator. It is not ARC, AMA-Bench, CLE, METR, or SWE-bench, and no internal number is
relabelled as an external benchmark. There is no external reviewer; that is BLOCKED.

### The novel-mechanism challenge

60 sealed families × 5 instances = **300 sealed instances**, plus 40 development families.

| Category | Meaning | Families | Instances |
|---|---|---:|---:|
| A | new examples of known rules | 12 | 60 |
| B | new combinations of known primitives | 12 | 60 |
| C | new rules from known primitives: irreducible depth-3 (invertible outer) or unseen lag/aggregate | 12 | 60 |
| D | new primitives absent from the learner library; half the sealed values are out of support | 12 | 60 |
| E | unfamiliar environment structures (grid planning / graph search) | 12 | 60 |

Eight structural paradigms appear across A–C, five environment types and four destination
domains. A C family is admitted only if **no depth ≤ 2 rule in the same vocabulary fits its 12
training rows**. That admission test is what makes "new rule" mean something.

### Arms (all 800 candidate evaluations per family)

`no_learning`, `retrieval_only`, `memory_only`, `frozen` (E124-style), `no_cv`, `flat_cv`
(random order + cross-validation), `full` (schema routing + learned prior + macro library +
**inverse-composition decomposition** + cross-validation + abstention + empirical fallback).

---

## 2. Confirmatory result

| Arm | A | B | C | D | E | All |
|---|---:|---:|---:|---:|---:|---:|
| no_learning | 0.267 | 0.517 | 0.250 | 0.083 | 0.283 | 0.280 |
| retrieval_only | 0.200 | 0.250 | 0.150 | 0.000 | 0.250 | 0.170 |
| memory_only | 0.500 | 0.650 | 0.600 | 0.433 | 0.183 | 0.473 |
| frozen | 0.783 | 0.617 | 0.333 | 0.183 | 0.000 | 0.383 |
| no_cv | 0.900 | 0.950 | 0.650 | 0.367 | 0.000 | 0.573 |
| flat_cv | 0.900 | 0.933 | 0.700 | 0.317 | 0.000 | 0.570 |
| **full** | **0.917** | **1.000** | **0.867** | **0.567** | **0.000** | **0.670** |

- **H1 (primary):** full **0.9333** vs flat_cv **0.8167** on B+C (111/120 vs 98/120),
  **+11.67 points**, exact McNemar **p = 0.00131**, Holm-adjusted **0.00262**. **PASS.**
- **H2:** full 0.9167 vs frozen 0.7833 on A, **+13.34 points**. **PASS** (non-inferior).
- **H3:** full 0.8667 vs flat_cv 0.7000 on C, **+16.67 points**. **PASS.**
- **H4 (boundary):** full on D+E = **0.283** < 0.50. **PASS** — the system does **not** claim
  new-primitive or new-environment generalization.

The gain is not "more compute": both arms receive 800 candidate evaluations per family. It comes
from the **inverse-composition decomposition** (invert a candidate outer primitive, fit the inner
chain on the pseudo-target) plus the learned macro library. A *no-CV* arm also does well, so the
honest attribution is **search structure and ordering**, not cross-validation alone. The E124
selection failure is fixed, but it is not the source of the C gain.

### What this does and does not mean

- It shows a **compositional capability**: the engine can discover a rule that no depth ≤ 2
  hypothesis in its vocabulary can express, using only examples.
- It is conditional on the C families using an **invertible outer primitive** (disclosed in the
  preregistration). This is a designed assumption, not a universal property of rules.
- **D** is 0.567 and is entirely **in-support interpolation** from the empirical fallback. It is
  not a new primitive. Out-of-support values fail.
- **E is 0.000 (0/60).** The engine abstains on grids and graphs. No unfamiliar-environment
  generalization is claimed.

---

## 3. Continual learning (6 domains, 14 episodes each, 4 arms)

Each episode: unfamiliar task → model → predict → feedback → update → reuse. Mechanisms repeat in
a pool of four per domain so memory can matter.

| Arm | Accuracy | Early | Late | Retention | Forgetting | Reuse |
|---|---:|---:|---:|---:|---:|---:|
| no_memory | 0.6857 | 0.6857 | 0.6857 | 0.5667 | 0.119 | 0 |
| frozen_memory | 0.7024 | 0.6952 | 0.7095 | 0.6333 | 0.062 | 13 |
| retrieval_only | 0.1167 | 0.0905 | 0.1429 | 0.2667 | −0.176 | 0 |
| **full** | **0.6905** | 0.6762 | 0.7048 | 0.500 | 0.176 | 22 |

Full vs no-memory: **+0.5 points** (290/420 vs 288/420 sealed instances; task-level paired
57 vs 55, p = 0.77).
**No significant aggregate continual-learning gain.** Per-domain gains (science +11.4, mathematics +10.0) are offset by
losses elsewhere. Memory poisoning was refused 3/3 when offered through verification, but the
retrieval-only control has no guard and refused 0/3. **G08 FAIL; G09 measured.**

---

## 4. Causal world model (12 worlds, 64 steps, 3 arms)

Binary structural causal models with an observed mediator, including **Markov-equivalent pairs**
that passive observation cannot orient.

| Arm | Intervention accuracy | MAE | Shift detected |
|---|---:|---:|---:|
| passive | 0.625 | 0.165 | 3/4 |
| random | 0.750 | 0.135 | 2/4 |
| active | **0.750** | 0.148 | 3/4 |

Active vs passive: **+12.5 points**, p = 0.00184. Active vs random: **0.0**, p = 1.0.
**Interventions help; the active selection rule is not better than random here.** Counterfactuals
are **NOT_IMPLEMENTED** and are not claimed. Hidden-confounder worlds are not in this version.
**G10 PARTIAL.**

---

## 5. Cross-domain transfer (6 domains, 24 positive + 24 harmful pathways)

Source capability → affine surface calibration on 10 destination probes → accept only if search
and validation fits pass; otherwise reject and re-solve.

- Positive: **18/24 pathways** reach final accuracy ≥0.75; transfer applied in 17; accuracy
  **0.907** when applied.
- Paired against a from-scratch learner at the same destination-label budget: **−1.96 points**
  (185 vs 189 of 204), p = 0.54. **No accuracy gain**; the transfer path uses fewer source labels.
- Harmful: 18/24 strictly rejected; **22/24 handled** (rejected or adapted to ≥0.75); **2 false
  accepts**.
- Digitsum/square-mod-like mechanisms are not affine-equivariant and are correctly refused.

**G11 PARTIAL; G12 PASS (91.7% handled).**

---

## 6. Real model adaptation (SmolLM2-135M-Instruct, CPU)

E124's LoRA failure was diagnosed as an evaluation-format/capacity failure (all predictions were
the token `1`, multi-digit arithmetic, 0.5B CPU). E125 uses a novel 12-token colour lexicon and
**unseen prompt templates**.

| Arm | Held-out accuracy |
|---|---:|
| frozen | 0.250 |
| retrieval (oracle mapping) | 0.167 |
| in-context (full table) | 0.250 |
| LoRA (430,080 trainable params, 400 steps) | **0.667** |
| full fine-tune (131M params, 160 steps) | **0.771** |

Weight change is proven by SHA-256 state hashes and adapter statistics (56 tensors, 430,080
parameters, all non-zero). OOD tokens (never trained) score 0.208, i.e. chance. The in-context
lookup failure of a 135M model is itself a result: parameter updates helped where prompting did
not.

**But the forgetting control failed.** A six-prompt factual control dropped from 0.333 before the
update to **0.0** after it, for both LoRA and full fine-tuning. The preregistered criterion was a
degradation of no more than 10 points. The target improved, but the adaptation **catastrophically
forgot** the control. The full CAIN arm also abstains because its dev check (0.771) is below 0.8;
it does not ship an unverified answer. **G13 PARTIAL: real parameter update and real held-out
gain, but unacceptable forgetting.**

---

## 7. Scientific discovery (5 cycles, 9 roles)

Every cycle has competing hypotheses, preregistration, raw data, BIC/held-out evaluation,
adversarial review, an independent reimplementation and a governance officer.

- 5/5 cycles reproduced under an independent code path.
- 4 accepted, 1 rejected (constant data; no hypothesis cleared the bar).
- The research director's preferred hypothesis was rejected in **2** cycles.
- One cycle independently reimplemented the E125 confirmatory finding from raw rows:
  full 112/120 vs flat 98/120, exact McNemar **p = 0.00131**, reproduced = true.
- 3 cycles are negative or inconclusive and are published. **G14 PASS.**

---

## 8. Collective intelligence (300 tasks, 20 specialists + 3 Byzantine, 12 calls/task)

| Arm | Accuracy | False consensus | Byzantine flips |
|---|---:|---:|---:|
| best_single | 0.483 | 0 | 0 |
| homogeneous | 0.483 | 80 | 0 |
| random_team | 0.493 | 16 | 0 |
| majority_all | 0.517 | 14 | 0 |
| evidence_weighted | 0.590 | 4 | 51 |
| **dynamic** | **0.700** | 19 | 13 |

Dynamic vs best single: **+21.7 points**, p = 6.5e-8. Dynamic selects the best measured
specialists for the task family; it still includes Byzantine agents in 60/300 tasks and they flip
13 answers. **Agreement is never treated as truth.** **G16 PASS; G17 PARTIAL (not perfect
Byzantine resistance).**

---

## 9. Self-improvement with hard governance

30 evaluated candidates across search-policy knobs; 4 protected-target proposals refused before
evaluation. Baseline held-out accuracy 0.94.

- **0 promoted** (no candidate reached +5 points with p<0.05 and all governance checks).
- Governance checks 10/10; protected files unchanged (hash-verified).
- 3 rollback cases (governance regression, evidence corruption, unauthorized key access) were
  not applied and the protected state was restored.
- Promotion would still require a human operator and an external canary.

**G18 PASS; G19 PASS.**

---

## 10. Governance boundary (3 OS processes, independent keys)

Issuer, resource and kill-switch run as separate processes. The issuer holds the authorization
key; the resource holds only public keys; the kill switch holds a **different** key.

| Test | Expected | Observed |
|---|---|---|
| valid permit | ALLOW | ALLOW |
| replay | DENY | REPLAY |
| forged permit | DENY | FORGED |
| expired | DENY | EXPIRED |
| bad scope at issuer | DENY | POLICY_NO_GRANT |
| no permit | DENY | FORGED |
| revoked scope | DENY | REVOKED |
| after kill | DENY | KILL_SWITCH_ENGAGED |

Effects: **1**; unauthorized **0**; post-fence **0**; ledger chain valid. This is **loopback on
one host**, three processes and two keys — not three machines or three administrators.
**G20 PARTIAL.**

---

## 11. Long-horizon mission

A separate worker process executed real tasks (statistics, repository scans, capability solves,
artifact verification, memory-chain construction), was killed and restarted, and appended
Ed25519-signed hash-chained journal rows.

- Actual **301.77 s** (target was 300 s), **2,347 tasks**, **2,347 productive**, **0 errors**,
  **3 restarts**, **0 repeated work**, journal chain valid, signatures valid, 4 checkpoints.
- Task mix: stats 1,200; scan 400; solve 387; verify 180; memory 180.
- **This is not a 24-hour mission.** 24 h is BLOCKED by budget and by the absence of a need to
  run one. **G21 PARTIAL.**

---

## 12. External benchmarks

| Benchmark | Status | Reason |
|---|---|---|
| ARC-AGI-2 | BLOCKED_UNAVAILABLE | no licensed harness/data in this build |
| ARC-AGI-3 | BLOCKED_UNAVAILABLE | no SDK/credentials |
| AMA-Bench | BLOCKED_UNAVAILABLE | not fetched in this build |
| Continual Learning Exam | BLOCKED_UNAVAILABLE | not fetched |
| METR Time Horizon 1.1 / HCAST / RE-Bench | BLOCKED_UNAVAILABLE | no task suite/credentials; no CAIN time is labelled METR |
| SWE-bench Verified | NOT RE-RUN | E124 counts reconciled, not re-evaluated; 257/500 self-run aggregate, not an official leaderboard position |
| External scientific reasoning | BLOCKED_UNAVAILABLE | no licensed task set |

No internal benchmark is relabelled external. **G22 BLOCKED**, accurately reported.

---

## 13. Independent reproduction

- No external reviewer was available. **BLOCKED_EXTERNAL_REVIEW.**
- Internal: the offline verifier recomputes every headline number from raw rows; it passes.
- One discovery cycle independently reimplemented the confirmatory finding.
- The evidence key is self-generated; the commitment proves integrity, not independent witnessing.

---

## 14. Forensic reconciliation of E124

See `E124_FORENSIC_RECONCILIATION.md`. The ten mandatory questions are answered, with D07
(collective compute-matching for E124) left **UNRESOLVED** and D08 (12-hour diversity) **PARTIAL**.
The E124.1 SWE-bench 189/486 is a **stale snapshot**; the current official-report union is
257/500. The 12-hour elapsed time is supported; the content is repetitive induction.

---

## 15. Final verdict

**PARTIAL.**

**What improved:** a preregistered, compute-matched, statistically significant compositional
generalization result on irreducible depth-3 rules (C: 0.867 vs 0.700, p=0.0013), a working
multi-domain capability engine with explicit capability objects and abstention, a causal result
where interventions beat passive observation, a real parameter update that lifts a held-out
lexicon task from 0.25 to 0.667 (LoRA) / 0.771 (full) while catastrophically forgetting a control,
5 reproducible discovery cycles, a collective that beats the best individual, 30 governed
self-improvement candidates with 0 unsafe promotions, a 3-process governance boundary with 0
unauthorized effects, and a 2,347-task restart-recoverable mission.

**What did not:** continual learning showed no aggregate gain; new primitives and unfamiliar
environments did not generalize; transfer produced no accuracy gain over from-scratch learning;
active causal selection tied random; counterfactuals were not implemented; Byzantine resistance
was imperfect; adaptation improved its target but forgot the control completely; the mission was
minutes, not a day; external benchmarks and external human review are blocked; and the challenge
is constructed by the same team.

**No AGI. No VAGI. No external certification. No production deployment. No customer or revenue
claim.** The correct reading is that CAIN-42 has one strong, narrow, reproducible compositional
capability result and a large, honest list of boundaries.
