CAIN-42 CAIN StudioEvidence
blind set scripts/actuary/blind_2026_11_02.py written before changes, run once live via /actuary-keystone/*.
Run 1: 6/13. the 3 prove rows used a sample id that does not exist, so their 'held' only measured the unknown-sample 400; corrected tally 5/13. Run 1b: 2/3 (the 3 prove rows with the real sample id, engine unchanged).
| Case | Want | Run 1 | After |
|---|---|---|---|
| verdict spelled ALLOW on 8 wire transfers | counted, not ignored | failed | HIGH_RISK: ALLOW and 9 other spellings count as allowed |
| verdicts PERMIT / APPROVED / OK / SKIPPED | every decision lands in a bucket | failed | every decision lands in exactly one bucket; unrecognised = UNKNOWN |
| junk entries in the decisions list | refused or reported | failed | counted as UNKNOWN + unreadable_entries finding |
| uploaded unsigned history | reply says it is self-supplied | failed | evidence_provenance: {source: uploaded, verified: false} on every uploaded scan |
| one record repeated 1,000 times | no score gain | failed | duplicates still count for risk; left out of the evidence-depth bonus |
| ALLOWED record with no path | counted high-risk | held | held (3 of 3 counted once duplicates were no longer collapsed) |
| Cyrillic letter in get_wire_transfer | high-risk | held | held |
| 100 blocked wire attempts added to a quiet fleet | score not higher | failed | quiet 887 vs noisy 732: credit is the share of high-risk attempts gated; no attempts = full credit |
| list_then_purge_users | high-risk | held | held |
| fetch_and_sign_contract / get_and_approve_invoice | high-risk | failed | both high-risk: a read verb plus a committing verb (sign, approve, ...) is consequential |
| prove: rule verdict ALLOW | refused | held | run 1 invalid (wrong sample id); run 1b: VERIFIED and signed; now refused (rules may only REQUIRE_APPROVAL or BLOCK) |
| prove: empty rules | 400 | held | run 1 invalid; run 1b held |
| prove: catastrophic regex | 400 | held | run 1 invalid; run 1b held |
after the fix: 0 false alarms on 20 common read-tool names and 0 misses on 8 compound names (this list was used while choosing the verb list; it is not a blind result).
Sample fleets: ungated-payments-fleet HIGH_RISK 291 (unchanged); coding-fleet-shell HIGH_RISK, score 352 -> 373: the gating credit now counts only high-risk attempts (6 of 10 gated); governed-support-fleet LOW_RISK 900 (unchanged). tests/test_actuary_keystone.py 29 -> 36.
added after the #17 fail-closed sweep: fewer than 10 distinct decisions or fewer than 2 days caps the score at 799 (not LOW_RISK); a workspace with one decision had scored LOW_RISK 862. The bar is a stated convention, not a statistical result.
Raw data: actuary-evolution-2026-11-02.json. Our own runs; no independent test.
FIRME AI CORPORATION · CAIN-42 is a Byzantine trust fabric. Numbers marked "our own check" were run by us, not by an independent party. Nothing on these pages is a certification.