# Framework Evaluation Review

**SKU:** CAIN-A75-058  
**Development:** #58 Repeated-run comparisons of agent frameworks  
**Product status:** DELIVERABLE  
**Engine:** `framework_eval` (`cain_a75_engines/frameworkeval/engine.py`)  
**Profile:** `three_framework_study`  
**Generated:** 2026-11-13T03:32:40Z UTC

## Result

VERIFIED: frozen independent harness, pinned model and tools on every run, coverage, mean and variance, ranking, raw-results hash and leader separation recomputed; 22/22 framework-study failure cases as expected

- **three_framework_study**: VERIFIED
- **honest_framework_study**: SOUND
- **close_race_reported_as_close**: SOUND
- **winner_within_noise**: UNSOUND
- **harness_changed_mid_study**: UNSOUND
- **task_edited_after_freeze**: UNSOUND
- **model_version_drift**: UNSOUND
- **tool_version_drift**: UNSOUND
- **temperature_drift**: UNSOUND
- **too_few_repetitions**: UNSOUND
- **cherry_picked_runs**: UNSOUND
- **best_run_counted_twice**: UNSOUND
- **run_before_freeze**: UNSOUND
- **mean_inflated**: UNSOUND
- **variance_hidden**: UNSOUND
- **ranking_misordered**: UNSOUND
- **raw_results_not_published**: UNSOUND
- **harness_frozen_by_vendor**: UNSOUND
- **run_for_unlisted_framework**: UNSOUND
- **framework_version_unpinned**: UNSOUND
- **unknown_claim**: UNSOUND
- **wrong_schema**: UNSOUND
- **garbage**: UNSOUND

## Limits of this run

Reviews the study record. It does not re-run any framework or judge whether the tasks are representative. The run is Ed25519-signed; pin the issuer key out of band.

This is a bounded run on a sample. It is not an assessment of any production system, not a
certification, and not a signed attestation. The accompanying `manifest.json` records SHA-256
hashes so a recipient can detect a changed file. The manifest is not a signature and not an
attestation; it does not prove who produced the artifact.

## What this is not

- Not a vendor integration, partnership or endorsement.
- Not legal, tax, clinical, insurance or safety advice.
- Not a third-party audit, SOC 2 report, ISO certification or patent filing.
- Not a production write action: no downstream tool was called.

Prices and packaging are proposals. Built by a three-person team.
