CAIN-42 CAIN StudioFALSEWORK: figures & benchmarks
Evidence for FALSEWORK: 1. Evidence pack · 2. Answers to buyer questions · 3. Figures & benchmarks · 4. Selling tools · IHVRK kit · Learn it · why these pages exist
Every number on this page is read from the signed transcript inside the IHVRK kit; none is typed in by hand.
7/7
claims matched when recorded
6
attack cases refused or flagged as claimed
79 ms
median round trip (min 52, max 102), incl. network
| # | Kind | Claim | Recorded value | HTTP | ms |
|---|---|---|---|---|---|
| C01 | attack | Sample: A signature check that never checks the signature -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "PARTIAL" | 200 | 95 |
| C02 | attack | Sample: A metric that reports a random number as accuracy -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "STUB" | 200 | 85 |
| C03 | attack | Sample: A webhook signature compared with == (timing leak) -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "PARTIAL" | 200 | 102 |
| C04 | attack | Sample: An alert function that says 'sent' and sends nothing -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. | "PARTIAL" | 200 | 69 |
| C05 | control | Your code: honest constant-time compare -> Real code must grade REAL or nobody trusts the alarms. | "REAL" | 200 | 63 |
| C06 | attack | Your code: verify() that always says yes -> A security check that cannot fail is fabrication. | "PARTIAL" | 200 | 79 |
| C07 | attack | Your code: encryption that is base64 -> Looks like encryption; is encoding. | "PARTIAL" | 200 | 52 |
Copy the table (paste into a spreadsheet)
id kind claim recorded_value expected http ms recorded_at C01 attack Sample: A signature check that never checks the signature -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "PARTIAL" ["PARTIAL", "STUB"] 200 95 2026-11-03T22:46:42+00:00 C02 attack Sample: A metric that reports a random number as accuracy -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "STUB" ["PARTIAL", "STUB"] 200 85 2026-11-03T22:46:43+00:00 C03 attack Sample: A webhook signature compared with == (timing leak) -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "PARTIAL" ["PARTIAL", "STUB"] 200 102 2026-11-03T22:46:43+00:00 C04 attack Sample: An alert function that says 'sent' and sends nothing -> A deterministic detector, not the model, finds the defect the AI is then asked to fix. "PARTIAL" ["PARTIAL", "STUB"] 200 69 2026-11-03T22:46:44+00:00 C05 control Your code: honest constant-time compare -> Real code must grade REAL or nobody trusts the alarms. "REAL" ["REAL"] 200 63 2026-11-03T22:46:44+00:00 C06 attack Your code: verify() that always says yes -> A security check that cannot fail is fabrication. "PARTIAL" ["STUB", "PARTIAL"] 200 79 2026-11-03T22:46:45+00:00 C07 attack Your code: encryption that is base64 -> Looks like encryption; is encoding. "PARTIAL" ["STUB", "PARTIAL"] 200 52 2026-11-03T22:46:45+00:00
Download: figures.tsv · transcript.json (full answers) · whole kit
How to read these numbers
- Recorded 2026-11-03T22:46:46+00:00 against cainstudio.online from our build host; times include the internet round trip.
- They show these inputs, not every possible input. Known misses are on the limits page.
- This is not a load test, an SLA or a third-party benchmark. Re-run it yourself:
python3 verify.py --live.