# Evolution 18 — test results

Environment: this host, Python 3.14.4, pytest 9.0.2, hypothesis 6.169.0, Linux 7.0.0-38, run as root, every command
under `ulimit -v 4000000`, HEAD `ad40473b` plus the uncommitted Evolution 18 files.

| Command | Result |
|---|---|
| `python3 -m pytest -q -p no:cacheprovider tests/test_evolution18_execution_truth.py` | **100 passed** (82 corpus scenarios, 3 corpus-level checks, 4 property tests, 11 unit/end-to-end tests) |
| same plus `tests/test_reality.py tests/test_evolution17_interoperability.py tests/test_evolution15_governed_runtime.py tests/test_evolution15_self_anchored_signatures.py tests/sovereign/evolution_07/test_e7_enforcement.py tests/test_evolution16_durable_runtime.py` | **316 passed, 0 failed, 0 skipped** |
| `InteropBenchmarkRunner().run_all()` (Evolution 17) | 20 / 20 |
| `python3 scripts/evolution18/run_evidence.py` | corpus 82 defined, 82 executed, 82 held, 0 not held, 0 errors, 0 skipped; 7 fixtures; performance (see `performance_results.md`) |
| `python3 scripts/evolution18/mutation_run.py` | 26 mutants, 26 killed (M18 after the TT-05 change; re-run with `mutation_run.py M18`) |

## Corpus (`benchmark_results.json`)

| Counted | Number |
|---|---|
| scenarios generated / defined | 82 |
| executed | 82 |
| unique attack cases (excluding the 3 LEGITIMATE controls and 2 LIMIT demonstrations) | 77 |
| property held | 82 |
| failures | 0 |
| skipped / unsupported (not run) | 0 |
| outcomes | PREVENTED 39 · DETECTED 38 · LEGITIMATE 3 · LIMIT 2 |

15 categories: authorization binding 10, effect canonicalization 7, observer compromise 7, evidence omission 7,
causal ordering 5, TOCTOU 9, concurrent revocation 6, partial execution 3, receipt replay 3, contradictory
observations 3, incomplete evidence 6, recovery 4, unsupported semantics 4, adapter bypass 4, privilege escalation 4.

Measures the specification asks for:

- **False allows:** 0 — no PREVENTED scenario let the effect happen (checked on disk) and no DETECTED scenario was
  VERIFIED. This counts only these 77 attacks.
- **False blocks:** 0 of 3 honest controls. Known false block not in the count: an honest tool that writes through
  mmap is FAILED (CX-02, limit L-7).
- **Detection latency:** detection happens when the receipt is sealed, i.e. within the same action (p50 ≈ 11 ms for
  the full path). There is no asynchronous detection.
- **Recovery outcomes:** RC-01 full, RC-02 partial, RC-04 not compensated, PE-03 ambiguous — each reported as such.
- **Evidence completeness:** honest runs with both sensors: complete. One sensor: INCOMPLETE by design.
- **Assertions:** each scenario asserts its property on disk state and on the verdict; the counts above are scenarios,
  not individual asserts.

Scenarios marked LIMIT pass because the system *says* it does not protect: EO-07 (a write outside the root happens and
is not detected; the receipt lists it as not observed) and TT-09 (a path-string executor follows the swapped link and
modifies the victim; the receipt is labelled `PATH_CHECK_ONLY` and the run FAILED afterwards). They are not counted as
blocked attacks.

## Defects found and fixed during this evolution (in Evolution 18 code)

1. OC-07: a same-process observer was graded INSTRUMENTED but its snapshot was still used for postconditions → only
   separate-process tree sensors feed postconditions (bridge and verifier).
2. IE-06: a verifier without sensor anchors returned FAILED ("claims VERIFIED without support") for an honest receipt
   → the reader's missing anchors now cap the verdict at INCOMPLETE without blaming the receipt.
3. M18 survived → TT-05 strengthened (above).
4. Corpus CR-01 expected the wrong refusal reason (a revoked tool's call has no known effects at all); corrected.