CAIN-42 CAIN StudioCAIN-42 · measured, not self-assessed
Catalog and KEYSTONE grades
Each of the 105 catalog services is graded by a script that calls the running service: its homepage scenario end to end, malformed requests that must be refused cleanly, its published API contract, how long it takes, and whether it has tests. Nothing is typed in by hand. Checks that couldn't run for a service are left out of its score, not counted as passes.
Distribution
| Language model |
|---|
KEYSTONE family
The 15 KEYSTONE products, graded by scripts/grade/grade_keystone.py against the live gateway. Each run must flag the product’s planted defect, pass a clean input and verify a signed record. Each product’s auditor bundle is also downloaded and checked offline by its own standalone verifier, against the key pinned from the live server.
| Product | Grade | Live scenario | Malformed input | Offline proof | Time | Tests |
|---|
Raw results: keystone-grades.json.
How the score is computed
Reproduce it on the gateway host with python3 scripts/grade/grade_catalog.py. Raw results: catalog-grades.json.