CAIN-42 evidence library
PROOFLOOP — FRESH evaluation (signed)
Evidence bundle: 0 claims, 0 of 0 invariants held.
Last reviewed 2026-11-08
The bundle's own README
PROOFLOOP — FRESH evaluation (signed)#
Don't trust this number. Verify it.
pip install cryptography curl -fsSLO https://cainstudio.online/verify_proofloop_eval.py curl -fsSLO https://cainstudio.online/evidence/proofloop-fresh-eval/record.json python3 -I verify_proofloop_eval.py record.json
The verifier imports no CAIN code. It checks the pinned Ed25519 key's fingerprint, the signature, the record hash, and recomputes every headline count from the per-finding table (it does not trust the summary).
What this measures#
The deterministic PROOFLOOP sweep ran over 17 serious ("BIG") findings across the FRESH corpus of real public agent/MCP repositories. FRESH was not used to tune any rule. Each finding was labelled by a human reviewer before the AI layer ran (the labels' SHA-256 was recorded first — see inputs). Then the shipped AI triage layer ran, and this record scores its decisions against those blind labels.
| Count | |
| BIG findings (rules only) | 17 |
| Real bugs | 8 |
| Plausible | 1 |
| False alarms | 8 |
| AI removed false alarms | 0 of 8 |
| AI wrongly demoted a real bug | 0 of 8 |
| Precision, rules only | 0.529 |
| Precision, after AI triage | 0.529 |
| Real-bug recall, after AI | 1.0 |
record_sha256: cf08bdab536b814d6d9ed937823d0f2d2b160408e0641246f86bb048fdf3b6e2 key: SHA256:2232b4add9b35d4016e5be270df4ed56e0dc9b39a0530ac69891073253aae90d
Heuristic detectors; a human labelled the ground truth; no third-party audit. Reproduce the sweep with python3 -m keystone.sweep <repo> and the AI layer with --ai.
Try CAIN-42 on your own agents
Create a free account and every new account starts with a 30-day trial of the full platform. Or try the sandbox first, with no account at all.
Create a free account → · Try the sandbox · See the whole ecosystem