# Evolution 80 — Collaboration Benchmark

> Generated by `scripts/evolution80/docs_e80.py` from `artifacts/e80/` (run 3). FIRME AI CORPORATION — a three-person team (founder: Vincent Noriega).
> Development branch `evolution-80`. Not deployed. No cloud model was called. No third-party audit.

Evolution 79 found that its multi-agent configurations did not beat one agent. Evolution 80 tested ten
coordination architectures on the same 48 test tasks with the same 0.5B model.

| arm | what it is | correct | accuracy | Wilson 95% | model calls | tokens |
|---|---|---|---|---|---|---|
| c01_single | one agent | 11/48 | 22.9% | [13.3%, 36.5%] | 48 | 5282 |
| c02_parallel_vote | parallel agents, majority vote | 11/48 | 22.9% | [13.3%, 36.5%] | 144 | 16122 |
| c03_pairwise_debate | pairwise debate | 11/48 | 22.9% | [13.3%, 36.5%] | 139 | 15564 |
| c04_critic_reviser | critic and reviser | 11/48 | 22.9% | [13.3%, 36.5%] | 96 | 11512 |
| c05_planner_executor | planner and executor | 13/48 | 27.1% | [16.6%, 41.0%] | 96 | 14904 |
| c06_hierarchical | hierarchical delegation | 15/48 | 31.2% | [20.0%, 45.3%] | 181 | 27529 |
| c07_specialist_committee | specialist committee | 10/48 | 20.8% | [11.7%, 34.3%] | 144 | 16538 |
| c08_evidence_first | evidence-first collaboration | 16/48 | 33.3% | [21.7%, 47.5%] | 96 | 13525 |
| c09_parallel_synthesis | parallel research then synthesis | 13/48 | 27.1% | [16.6%, 41.0%] | 192 | 22942 |
| c10_adaptive_graph | adaptive graph (talks only on disagreement) | 22/48 | 45.8% | [32.6%, 59.7%] | 60 | 8742 |

## Each architecture against one agent (paired, exact McNemar)

| arm | delta vs one agent | p (two-sided) | Holm-adjusted p | calls per task | delta per extra call | communication tokens |
|---|---|---|---|---|---|---|
| c02_parallel_vote | +0.0000 | 1.0 | 1.0 | 3.0 | 0.0 | 10840 |
| c03_pairwise_debate | +0.0000 | 1.0 | 1.0 | 2.9 | 0.0 | 10282 |
| c04_critic_reviser | +0.0000 | 1.0 | 1.0 | 2.0 | 0.0 | 6230 |
| c05_planner_executor | +0.0417 | 0.6875 | 1.0 | 2.0 | 0.0417 | 9622 |
| c06_hierarchical | +0.0833 | 0.34375 | 1.0 | 3.77 | 0.0301 | 22247 |
| c07_specialist_committee | -0.0208 | 1.0 | 1.0 | 3.0 | -0.0104 | 11256 |
| c08_evidence_first | +0.1042 | 0.125 | 1.0 | 2.0 | 0.1042 | 8243 |
| c09_parallel_synthesis | +0.0417 | 0.5 | 1.0 | 4.0 | 0.0139 | 17660 |
| c10_adaptive_graph | +0.2292 | 0.000977 | n/a | 1.25 | 0.9168 | 3460 |

## Result

- **Collaboration advantage: NOT demonstrated.** Significant arms after testing: none.
- Best multi-agent arm: `c08_evidence_first` at +0.1042. Arms measured worse than one agent: ['c07_specialist_committee'].
- The adaptive graph communicates only when its first two agents disagree, which is why its communication cost is the lowest of the multi-agent arms (3460 tokens).
- Vote-agreement calibration: expected calibration error 0.4722 (confidence = fraction of voters agreeing with the majority).

## Why collaboration does not help here (hypotheses, not findings)

No causal ablation was run. The plausible explanations are that every participant shares the same weights, so
errors are correlated; that a 0.5B model is a weak critic of its own family; and that extra context costs a
small model more than a second opinion gives it. These are labelled HYPOTHESIS in `research_report.json`.

The system is permitted to conclude that one agent is the better choice, and on this evidence that is the conclusion.
