CAIN-42 CAIN StudioAGENTIC AI 75 · #56
Visually complex knowledge-work evaluation
Google and Artificial Analysis introduced the Multimodal Knowledge Agent benchmark.
Last reviewed 2026-11-11
Agentic AI 75 / #56
2026 status: Released benchmark · Category: Evaluation
The 2026 development
Google and Artificial Analysis introduced the Multimodal Knowledge Agent benchmark.
The governance gap
A benchmark score is a statement about a benchmark. It becomes trustworthy for a buyer only when the task distribution, the grading method and the failure cases are inspectable.
Where CAIN-42 fits: Moat 4 — Execution Provenance & Attestation
Core question: Can we prove what actually happened? Hash-chained, signed records binding decision to execution to observable effect.
Treat benchmark artifacts like any other evidence: record the task, the model, the inputs, the grader and the result, and publish the failures. A score without its grading record is marketing.
How the wiring works
- Record the exact task and input files.
- Record the model, version and decoding settings.
- Record the grader and its rubric.
- Publish failures and near-misses alongside the score.
What is real today
LIBRARY Production code and tests exist in this repository; there is no hosted route for this exact use.
Evaluation and model-card code exist. CAIN does not publish a multimodal knowledge-work score.
| Component in this repository | What the module says it is |
|---|---|
keystone/evalrun.py | Signed evaluation runs: "these scores came from this code, this data, these model providers, unmodified". |
keystone/modelcard.py | Model / agent cards made only of measured, signed evidence (proof factory #35). |
keystone/heldout.py | Held-out evaluation vault + contamination certificate, run in the CAIN agent jail, signed. |
cain/worm_export.py | CAIN Trust Fabric - tamper-evident WORM Causal Evidence Fabric (Phase 4) |
Descriptions are the first line of each module's own docstring, read without importing it. A module path is a pointer into this repository, not a hosted endpoint. Its existence is checked by the test suite; its status for this use is the label above.
Check it yourself
- Inspect a live signed evaluation run
/keystone - Fire a live decision through the CAIN-42 pipeline
/fabric/try— the same enforcement path every product sits behind.
Commercial package
| Offer | Benchmark Evidence Review |
|---|---|
| Buyer | Benchmark authors and procurement teams |
| Price (proposal) | $2,500–$5,000 per review |
| Bundle | Runtime Containment & Evidence — Contain first, then prove what happened. |
| Operator stream | Stream 10 of the 10-stream operator playbook |
SOC, incident response, compliance, enterprise IT is the bundle buyer. The entry offer for this bundle is Runtime containment review; signed evidence pack; the range is $2,500–$5,000 per review; $2,000–$10,000 per evidence pack. A price on this page is a proposal, not a validated rate, and no customer is claimed.
Buy this as a product or service
CAIN-A75-056 — Benchmark Evidence Review PILOT
A bounded pilot report: up to five scenarios, the relevant library engine or analysis, a written result and the stated limits. Engineering time is quoted per pilot.
Kind: Evaluation · Price (proposal): $2,500–$5,000 per review · Lead time: 10 business days.
Open the product page · all 75 products
What this page does not claim
What this page is not. CAIN-42 is not affiliated with, endorsed by, integrated into, or shipped with the organisation named in the 2026 item. The item is public context; the page describes how CAIN-42 would govern that class of capability. No page here claims a customer, a partnership, a certification, a patent, or a production integration. Prices are proposals. Built by a three-person team.