CAIN-42 CAIN StudioEvidence
A blind set is written before the change it measures and run once. Once we fix the engine using a set, that set is for tuning and its new score proves nothing. Here is every run of 2026-11-02, misses included.
| Set | Written | Engine | Caught | False alarms | Blind? | Note |
|---|---|---|---|---|---|---|
| A (60 danger / 60 safe) | before any change on 2026-11-02 | 1.0.0 live | 55/60 | 0/60 | yes | |
| A | after commit 1506f111 | 60/60 | 0/60 | no (tuning) | ||
| C (30 / 40) | after 1506f111, run once | after 1506f111 | 26/30 | 2/40 | yes | the 1.0.0 engine also scored 26/30 with 0 false alarms: those rules did not generalise |
| D (30 / 40) | after the set-C fixes, run once | after set-C fixes | 26/30 | 2/40 | yes | |
| A + C + D | 1.1.0 | 120/120 | 1/140 | no (tuning) | the 1 false alarm (git push to an SSH address) is kept on purpose: pushing to an address instead of a configured remote copies the code anywhere |
The honest headline is the last blind run: set D, 26/30 caught, 2/40 false alarms. Its 4 misses and 2 false alarms were then fixed, so set D is now a tuning set. The next blind set will be written before the next change.
Raw data: codeguard-evolution-2026-11-02.json · codeguard-blind-d-2026-11-02.json (the set-D run as it happened).
All numbers are from our own runs. No independent tester has checked them.
FIRME AI CORPORATION · CAIN-42 is a Byzantine trust fabric. Numbers marked "our own check" were run by us, not by an independent party. Nothing on these pages is a certification.