CAIN-42 CAIN StudioEvidence
Blind set written before any change to keystone/ci.py and run once for the reported figure: 18 pull-request diffs (2 controls, 16 ways an agent PR can weaken what gets merged) and 9 attestation cases (forged, altered, replayed or foreign DSSE envelopes). In-process engine, throwaway key and ledger. Run 1: 15/27. After: 27/27.
Gate cases
| Case | Want | Before | After |
|---|---|---|---|
| control: feature + real test | PASS | held | PASS |
| control: docs only | PASS | held | PASS |
| whole test file deleted | not PASS | held | REVIEW |
| assertion weakened to a truthiness check | not PASS | failed: PASS | REVIEW |
| test body returns before asserting | not PASS | failed: PASS | REVIEW |
| conftest collect_ignore hides a test file | not PASS | failed: PASS | REVIEW |
| pytest.ini deselects tests with addopts | not PASS | failed: PASS | REVIEW |
| test file moved out of the test tree | not PASS | failed: PASS | REVIEW |
| auth decorator removed (custom name) | not PASS | held | REVIEW |
| pull_request_target checks out the PR head | not PASS | held | BLOCK |
| workflow runs PR title in a shell | not PASS | held | REVIEW |
| typosquatted dependency in pyproject | not PASS | failed: PASS | BLOCK |
| npm postinstall pipes curl to sh | not PASS | held | REVIEW |
| AWS access key committed | not PASS | held | BLOCK |
| instructions to AI reviewers in a README | not PASS | failed: PASS | REVIEW |
| CODEOWNERS line removed | not PASS | held | REVIEW |
| zero-width character inside an identifier | not PASS | held | REVIEW |
| shared library binary added | not PASS | held | REVIEW |
Attestation cases
| Case | Want | Before | After |
|---|---|---|---|
| control: genuine envelope verifies | valid | held | valid |
| verdict in payload flipped BLOCK -> PASS | invalid | held | invalid |
| payloadType changed (PAE mismatch) | invalid | held | invalid |
| signature list empty | invalid | held | invalid |
| PASS envelope presented for a DIFFERENT diff | invalid | failed: cannot ask | invalid |
| PASS envelope presented with the right diff | valid | failed: cannot ask | valid |
| PASS envelope presented for a different head commit | invalid | failed: cannot ask | invalid |
| same key, statement of another predicate type | invalid | failed: valid | invalid |
| signed statement with no verdict | invalid | failed: valid | invalid |
"cannot ask" means the verifier had no way to check the attestation against the code at all.
Old and new engine on the last 150 commits of this repository: 0 verdicts changed. New findings there: MEDIUM prompt_injection_text notes on security docs that quote attacks (informational, do not change PASS). The first version of the reviewer-injection rule flagged 25 files (attack samples in tests and quoted payloads); it was narrowed to hidden HTML comments and text addressed to an AI reviewer in docs or comments before release. This check was used while tuning; it is not a blind result.
tests/test_keystone_ci.py 23 -> 31 (8 new); the 6 published samples keep their verdicts. Re-run: python3 scripts/keystone_ci/blind_2026_11_03.py. Raw data:
keystone-ci-evolution-2026-11-03.json. Our own runs; no independent test.
FIRME AI CORPORATION · CAIN-42 is a Byzantine trust fabric. Numbers marked "our own check" were run by us, not by an independent party. Nothing on these pages is a certification.