# PROOFLOOP — FRESH evaluation (signed)

**Don't trust this number. Verify it.**

    pip install cryptography
    curl -fsSLO https://cainstudio.online/verify_proofloop_eval.py
    curl -fsSLO https://cainstudio.online/evidence/proofloop-fresh-eval/record.json
    python3 -I verify_proofloop_eval.py record.json

The verifier imports no CAIN code. It checks the pinned Ed25519 key's fingerprint, the signature, the record hash,
and recomputes every headline count from the per-finding table (it does not trust the summary).

## What this measures
The deterministic PROOFLOOP sweep ran over 17 serious ("BIG") findings across the FRESH
corpus of real public agent/MCP repositories. FRESH was **not** used to tune any rule. Each finding was labelled by
a human reviewer **before** the AI layer ran (the labels' SHA-256 was recorded first — see `inputs`). Then the
shipped AI triage layer ran, and this record scores its decisions against those blind labels.

| | Count |
|---|---|
| BIG findings (rules only) | 17 |
| Real bugs | 8 |
| Plausible | 1 |
| False alarms | 8 |
| AI removed false alarms | 0 of 8 |
| AI wrongly demoted a real bug | 0 of 8 |
| Precision, rules only | 0.529 |
| Precision, after AI triage | 0.529 |
| Real-bug recall, after AI | 1.0 |

record_sha256: `cf08bdab536b814d6d9ed937823d0f2d2b160408e0641246f86bb048fdf3b6e2`
key: `SHA256:2232b4add9b35d4016e5be270df4ed56e0dc9b39a0530ac69891073253aae90d`

Heuristic detectors; a human labelled the ground truth; no third-party audit. Reproduce the sweep with
`python3 -m keystone.sweep <repo>` and the AI layer with `--ai`.