CAIN-42 CAIN StudioEvidence
blind set scripts/math_gates/blind_2026_11_02.py written before changes, run once against production through /suite/*; references computed independently in the script.
Run 1: 9/19. 5 rows could not reach the engine (one 413 size limit, four 401 keyed routes); one 401 was counted as held by the harness and is corrected in 'after'.
Addenda
- same question as the 3,000-step case at a size under the limit: a 1,200-step chain: HTTP 500 live for one of six id spellings (RecursionError depends on set order); locally 2 of 6 hash seeds
- every count pair up to 59 per group whose exact ratio is 4/5: 1,053 pairs computed below 0.8 in floating point and would be flagged
| Group | Case | Want | Run 1 | After |
|---|---|---|---|---|
| FlowGate | duplicate task id with conflicting dependencies | refused or reported | held | held |
| FlowGate | 3,000-step chain | valid, ordered, no crash | failed | blocked by the 64 KB request limit (no crash); addendum below found the real bug at 1,200 steps |
| FlowGate | task depends on itself | cycle [a] | held | held |
| FlowGate | returned cycle is made of real edges (12 random graphs) | all real | held | held |
| FlowGate | empty id / empty pipeline | 422 / handled | failed | empty pipeline refused (400); empty id now makes the pipeline invalid (invalid_task_ids) |
| FlowGate | topological order respects every edge (10 random DAGs) | all correct | held | held |
| LiftGate | experiments route reachable | 200 | failed | needs a paid key through the proxy: not measurable live, measured on the engine |
| LiftGate | alpha = 5 with identical rates | refused, never ship | held | live reply was 401 (not a real hold); engine gave keep_control + significant; now refused_invalid_alpha |
| LiftGate | alpha 0, -0.5, 1.0, 1.5 | refused | failed | engine interpreted them; now refused_invalid_alpha |
| LiftGate | conversions above total | refused | failed | engine already refuses (400); live route needs a key |
| LiftGate | p-value vs independent reference (8 random) | |diff| < 1e-5 | failed | not measurable live (key); engine formula matches the closed form used here |
| LiftGate | sample size vs independent reference (4) | exact | held | held |
| LiftGate | alpha 0.10 / power 0.95 | clear answer or clear refusal, no 500 | held | held (clear refusal) |
| FairGate | records missing the outcome field | refused or reported | failed | left out of the rates and reported (was counted as unfavorable) |
| FairGate | outcome 'true' string vs True | type mismatch reported | failed | left out and reported; a group with no usable records is a 400 |
| FairGate | privileged group has the lower rate | flagged or reported | failed | flagged, impacted_group named, four_fifths_ratio added |
| FairGate | 1 record per group | small-sample warning | failed | small-sample warning under 30 usable records per group |
| FairGate | ratio exactly 0.8 | not flagged | held | held on this pair; but exact-0.8 scan found 1,053 count pairs falsely flagged by float rounding: now exact fractions |
| FairGate | group value absent from data | 4xx | held | held |
Fixes
| Product | Commit | What | Tests |
|---|---|---|---|
| FlowGate | e7c02bd | iterative cycle search, sorted roots (seed-independent), heap topological sort, empty ids invalid | 15 -> 17 |
| LiftGate | 7a1ae84 | alpha outside (0, 0.5] refused, never interpreted | 18 -> 19 |
| FairGate | 716f9dd (after ef79810) | missing/mistyped outcomes left out + reported; reverse disparity flagged; small-sample warning; exact fractions; undefined ratio UNDETERMINED | 12 -> 18 |
Raw data: math-gates-evolution-2026-11-02.json. Our own runs; no independent test.
FIRME AI CORPORATION · CAIN-42 is a Byzantine trust fabric. Numbers marked "our own check" were run by us, not by an independent party. Nothing on these pages is a certification.