CAIN-42 CAIN StudioAGENTIC AI 75 · #10
Failure-driven agent improvement
Third Moment Research described identifying recurring agent weaknesses and generating targeted scenarios to address them.
Last reviewed 2026-11-11
Agentic AI 75 / #10
2026 status: Reported research architecture · Category: Evaluation
The 2026 development
Third Moment Research described identifying recurring agent weaknesses and generating targeted scenarios to address them.
The governance gap
A system that generates its own test scenarios can also generate easy ones. Improvement is only measurable against a held-out distribution the improver did not author.
Where CAIN-42 fits: Moat 6 — Adversarial Trust Engine
Core question: Can CAIN break its own guarantees? Continuous attack, mutation and red-team testing of the controls themselves.
Separate the attacker from the improver and keep a held-out suite the improver cannot edit. CAIN's red-team and eval-run paths already produce signed failure records and re-run them after a change.
How the wiring works
- Harvest failures from signed production records, not from the agent's own summary.
- Generate targeted scenarios and freeze them before the fix.
- Re-run the frozen suite plus a held-out set after the change.
- Publish the delta and refuse to call an unmeasured change an improvement.
What is real today
LIVE A public route in this deployment exercises this mechanism today, within its stated scope.
The hosted red-team demo attacks and patches fixed tool policies. It is not a general agent-improvement benchmark.
| Component in this repository | What the module says it is |
|---|---|
keystone/redteam.py | REDTEAM KEYSTONE: attack an agent's tool policy, write the smallest patch, prove the patch closes every breach. |
keystone/evalrun.py | Signed evaluation runs: "these scores came from this code, this data, these model providers, unmodified". |
cain/runtime/mutation.py | CAIN Sovereign — Governance Mutation Testing (Prompt 7, Phase 30). |
platform-gateway/adversarial_engine.py | CAIN Adversarial Trust Engine - Continuous Attack Testing |
Descriptions are the first line of each module's own docstring, read without importing it. A module path is a pointer into this repository, not a hosted endpoint. Its existence is checked by the test suite; its status for this use is the label above.
Check it yourself
- Run the live red-team loop
/redteam-keystone - Fire a live decision through the CAIN-42 pipeline
/fabric/try— the same enforcement path every product sits behind.
Commercial package
| Offer | Failure-Driven Improvement Audit |
|---|---|
| Buyer | Agent-platform teams and evaluation vendors |
| Price (proposal) | $3,500–$7,500 per audit |
| Bundle | Code & Proof — Prove the fix, not just the diff. |
| Operator stream | Stream 9 of the 10-stream operator playbook |
Platform engineering, AI coding-tool vendors, security engineering is the bundle buyer. The entry offer for this bundle is Fabrication audit or long-run engineering attestation; the range is $1,500–$6,000 per audit; $2,500–$4,000/month retainer. A price on this page is a proposal, not a validated rate, and no customer is claimed.
Buy this as a product or service
CAIN-A75-010 — Failure-Driven Improvement Audit DELIVERABLE
A bounded engine run on the buyer's sample: findings, a written report, and a SHA-256 artifact manifest. The manifest is unsigned: it is not a signature and not an attestation.
Kind: Assessment · Price (proposal): $3,500–$7,500 per audit · Lead time: 5 business days.
Open the product page · all 75 products
What this page does not claim
What this page is not. CAIN-42 is not affiliated with, endorsed by, integrated into, or shipped with the organisation named in the 2026 item. The item is public context; the page describes how CAIN-42 would govern that class of capability. No page here claims a customer, a partnership, a certification, a patent, or a production integration. Prices are proposals. Built by a three-person team.