# Evolution 80 — Limitations and Unverified Claims

This file exists so that every claim in Evolution 80 is paired with what was actually measured and
what was not. If a statement here conflicts with a more enthusiastic sentence elsewhere, this file
wins.

## Execution classes used in E80

| Label | Meaning in this evolution |
|---|---|
| `REAL MODEL INFERENCE` | not used; no remote model was called |
| `LOCAL MODEL INFERENCE` | a local Hugging Face checkpoint ran on CPU (Qwen2.5-0.5B/1.5B-Instruct, SmolLM2-135M-Instruct) |
| `LOCAL TRAINING (CPU)` | a hand-written LoRA adapter was trained on SmolLM2-135M on CPU |
| `DETERMINISTIC EXECUTION` | rules, SQL, hashing, sandboxed arithmetic and the Agentic 75 engines on bundled samples |
| `MOCK EXECUTION` | **0** |
| `SIMULATED EXECUTION` | **0** in the headline results; the only simulated items are explicitly labelled (see below) |
| `BLOCKED` | a dependency is missing and nothing was substituted |

## Upstream baseline defects found and reported, not hidden

1. **Two public Evolution 79 blocks on all three homepages.** v1 (1,186 calls, 22/22, B-) and v2
   (986 calls, 144/144, C-) are both labelled "Evolution 79"; the older block still says LIVE. The
   branch fixes only the label; the live sites were not changed (deployment not authorized).
2. **E79 replication overlap.** The E79 replication split shared 14 questions with the main set
   (the cybersecurity generator can emit only 18 distinct questions). E79's "independent
   replication" was overstated for that domain. E80 measures overlap with E79 explicitly and does
   not claim E80's task templates are new.
3. **E79 scorecard memory value is stale.** The signed v2 scorecard records memory 0.0 → 0.0 while
   its own raw rows record 0.10 → 0.70, so the 5.78 composite is internally inconsistent. The
   signed bundle was not edited; E80 recomputes its own memory result from its own rows.
4. **E79 tool arm was misapplied.** A calculator protocol was forced onto label questions
   (14/104 correct). E80's program-of-thought path is marked not applicable on label tasks.

## Claims that are NOT made

- **No AGI, superintelligence, consciousness, human-level reasoning or universal reasoning.** E80
  runs three small CPU models on constructed closed-form tasks.
- **No general software-engineering or open-world benchmark result.** SWE-bench, OSWorld,
  WebArena, GAIA, AgentBench, MultiAgentBench, MemoryAgentBench, LongMemEval, LoCoMo and AMA-Bench
  were **not run**. The task generators are the E79 templates with new seeds; this measures
  in-distribution generalization to unseen instances, not transfer to new task types.
- **No collaboration advantage unless the paired test says so.** If no multi-agent arm is
  significantly better than one agent, that is the reported result. A negative result is preserved.
- **No remote model provider.** Anthropic, OpenAI-compatible remote, Gemini and Hugging Face
  Inference API are `BLOCKED` (no credential, no funding). Zero cloud calls, zero cloud spend. A
  blocked model is never replaced by a mock.
- **No real cross-host intelligence execution.** The second host (2 vCPU, 3.4 GB) is used only to
  re-verify a bundle. It is the same administrative domain; it is not an independent witness.
- **No network partition, regional scheduling, checkpoint replication or host reboot test in E80.**
  Those are `NOT EVALUABLE` here; earlier evolution evidence is preserved and not re-run.
- **No 6 h / 24 h / 72 h / 168 h endurance campaign.** A 1 h campaign, if present, is reported with
  its actual elapsed time and is still a single-host run.
- **No hardware attestation, TEE, SGX, SEV or Nitro claim.**
- **No gVisor, Kata, Firecracker or Kubernetes Agent Sandbox result.** Not installed.
- **No transparency-log entry and no independent witness.** No Sigstore Rekor entry exists for E80;
  no witness has cosigned anything.
- **No third-party audit, no SOC 2, no ISO 27001/42001, no certification, no "production-ready".**
- **No customer, revenue, partnership, deployment or market validation claim.** Historical
  founder-attested revenue ($193 / 3 customers) is not evidence produced by E80.
- **No patent claim.** Nothing was filed by E80.
- **No claim that a skill or policy was promoted.** Promotion is withheld pending human approval.
- **No claim that the Agentic 75 engines are model-backed or independently evaluated.** They are
  deterministic engines executed on bundled samples; a sample run is not a production assessment.
- **No claim that the reference gate protects the production MCP proxy.** The production proxy was
  not modified or re-tested.

## Known weaknesses of E80 itself

1. **Small samples.** 48 test tasks and 24 replication tasks detect only large effects. Confidence
   intervals are wide; a non-significant result does not prove equality.
2. **One model family.** Qwen2.5-0.5B and 1.5B plus SmolLM2-135M. Another family could behave
   differently; no cross-family conclusion is drawn.
3. **Prompt formats were fixed after a smoke run.** The smoke run touched a few test-split
   questions for format repair only; the frozen policy and preregistration were written afterwards,
   before the scored test calls. This is a real limitation and is recorded in the research report.
4. **The verifier shares the producer's model family** for model-backed mission nodes. Agreement is
   weak evidence of correctness; deterministic nodes are recomputed in a fresh sandbox instead.
5. **Memory evaluation is constructed** (30 questions, lexical BM25). Retrieval hit-rate is reported
   separately from answer accuracy; neither is a public-benchmark score.
6. **Injection screening is regex-based** and misses paraphrases. The architectural defence is that
   model text never carries authority.
7. **The durable queue is single-host.** SQLite WAL, leases and fencing are real, but cross-host
   shared state and network partitions are not exercised in E80.
8. **The kill switch is in-process.**
9. **The evidence key is self-generated**, stored outside the repository with mode 0600. Its
   fingerprint is not published out of band. Without an out-of-band pin, the signature proves
   internal consistency, not authorship.
10. **The second-environment check is not independence.** Same operator, same admin domain.
11. **The build host is the production host.** E80 changed no production file, restarted no
    service, changed no DNS and spent no money. During the run the public homepages were observed
    timing out under unrelated host load; this was reported, not acted on.
12. **Concurrent sessions.** Another session wrote the Phase 0 reports and the initial E80 package;
    this session took over, completed the runs, added the acceptance tests and wrote the final
    reports. The evidence is the artifacts, not the authorship.

## Reproduction caveat

Re-running the benchmark re-runs local CPU inference and will produce different latency numbers and
possibly different sampled outputs at non-zero temperature. The **raw rows of the reported run** are
preserved; the offline verifier recomputes the reported numbers from those rows. Replication is a
new-run check, not a bit-for-bit replay.
