AGENT-CI KEYSTONE: a release gate that grades the path, not only the answer

You changed a prompt, a model or a tool. The answers still match. But did the new version skip the policy check before a refund, call a forbidden tool, loop on retries at three times the cost, or quietly drop the case it fails? AGENT-CI diffs a baseline run of your suite against the candidate, case by case, and blocks the release on any of those. No language model decides. An independent clean-room implementation re-derives every verdict in a sandbox before the result is signed.

#21 of the CAIN-42 Top 25 8 gaps found and fixed today 1.1.0 LIVE since 2026-11-02 18:56Z (gateway deploy), checked on both instances

4/11

held on today's blind set, before any change

11/11

hold on 1.1.0 (engine and clean-room verifier agree on each)

3,000

random suites on which the two implementations agree

models in the verdict

Try it

The homepage section AGENT-CI KEYSTONE runs the six bundled suites live. Or:

curl -s https://cainstudio.online/agent-ci-keystone/samples
curl -s -O https://cainstudio.online/agent-ci-keystone/client.py   # CI client: exit 0 PASS (WARN too unless --strict), 1 BLOCK, 2 the gate could not decide

The rules · Pinning your suite · Evidence

FIRME AI CORPORATION · CAIN-42 is a Byzantine trust fabric. Numbers marked "our own check" were run by us, not by an independent party. Nothing on these pages is a certification.