CAIN-42 CAIN StudioKEYSTONE: figures & benchmarks
Evidence for KEYSTONE: 1. Evidence pack · 2. Answers to buyer questions · 3. Figures & benchmarks · 4. Selling tools · IHVRK kit · Learn it · why these pages exist
Every number on this page is read from the signed transcript inside the IHVRK kit; none is typed in by hand.
8/8
claims matched when recorded
6
attack cases refused or flagged as claimed
67 ms
median round trip (min 50, max 107), incl. network
| # | Kind | Claim | Recorded value | HTTP | ms |
|---|---|---|---|---|---|
| C01 | attack | Sample: A signature check that never checks the signature -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "PARTIAL" | 200 | 67 |
| C02 | attack | Sample: A metric that reports a random number as accuracy -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "STUB" | 200 | 50 |
| C03 | attack | Sample: A webhook signature compared with == (timing leak) -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "PARTIAL" | 200 | 59 |
| C04 | attack | Sample: An alert function that says 'sent' and sends nothing -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "PARTIAL" | 200 | 56 |
| C05 | control | Your code: honest constant-time compare -> Real code must grade REAL or nobody trusts the alarms. | "REAL" | 200 | 60 |
| C06 | attack | Your code: verify() that always says yes -> A security check that cannot fail is fabrication. | "PARTIAL" | 200 | 67 |
| C07 | control | Ledger: latest record re-verifies -> The signed fix record verifies against the pinned key. | true | 200 | 107 |
| C08 | attack | Ledger: latest record with its verdict edited to NOT_VERIFIED -> Changing any signed field breaks the record hash and signature. | false | 200 | 89 |
Copy the table (paste into a spreadsheet)
id kind claim recorded_value expected http ms recorded_at C01 attack Sample: A signature check that never checks the signature -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "PARTIAL" ["PARTIAL", "STUB"] 200 67 2026-11-03T22:46:54+00:00 C02 attack Sample: A metric that reports a random number as accuracy -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "STUB" ["PARTIAL", "STUB"] 200 50 2026-11-03T22:46:55+00:00 C03 attack Sample: A webhook signature compared with == (timing leak) -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "PARTIAL" ["PARTIAL", "STUB"] 200 59 2026-11-03T22:46:55+00:00 C04 attack Sample: An alert function that says 'sent' and sends nothing -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "PARTIAL" ["PARTIAL", "STUB"] 200 56 2026-11-03T22:46:56+00:00 C05 control Your code: honest constant-time compare -> Real code must grade REAL or nobody trusts the alarms. "REAL" ["REAL"] 200 60 2026-11-03T22:46:56+00:00 C06 attack Your code: verify() that always says yes -> A security check that cannot fail is fabrication. "PARTIAL" ["STUB", "PARTIAL"] 200 67 2026-11-03T22:46:57+00:00 C07 control Ledger: latest record re-verifies -> The signed fix record verifies against the pinned key. true [true] 200 107 2026-11-03T22:46:57+00:00 C08 attack Ledger: latest record with its verdict edited to NOT_VERIFIED -> Changing any signed field breaks the record hash and signature. false [false] 200 89 2026-11-03T22:46:58+00:00
Download: figures.tsv · transcript.json (full answers) · whole kit
How to read these numbers
- Recorded 2026-11-03T22:46:59+00:00 against cainstudio.online from our build host; times include the internet round trip.
- They show these inputs, not every possible input. Known misses are on the limits page.
- This is not a load test, an SLA or a third-party benchmark. Re-run it yourself:
python3 verify.py --live.