# Evolution 80 — Model Evaluation

> Generated by `scripts/evolution80/docs_e80.py` from `artifacts/e80/` (run 3). FIRME AI CORPORATION — a three-person team (founder: Vincent Noriega).
> Development branch `evolution-80`. Not deployed. No cloud model was called. No third-party audit.

## Measured models (all local CPU inference)

| model | Hugging Face id | parameters | license | context | calls | tokens | mean latency ms | train direct accuracy | tool-expression validity (train) | weight file SHA-256 |
|---|---|---|---|---|---|---|---|---|---|---|
| qwen2.5-0.5b-instruct | Qwen/Qwen2.5-0.5B-Instruct | 494M | Apache-2.0 | 32768 | 933 | 124320 | 2882.1 | 25.0% | 70.4% | fdf756fa7fcbe740… |
| qwen2.5-1.5b-instruct | Qwen/Qwen2.5-1.5B-Instruct | 1.54B | Apache-2.0 | 32768 | 181 | 23590 | 9365.0 | 32.5% | 66.7% | dd924a11b4c220f3… |
| smollm2-135m-instruct | HuggingFaceTB/SmolLM2-135M-Instruct | 135M | Apache-2.0 | 8192 | 40 | 5222 | 1360.4 | 2.5% | n/a | 5af571cbf074e6d2… |

Model versions are pinned by the SHA-256 of the weight file. Quantization: none (fp32).

## Inference accounting

1154 model calls, 153132 tokens (134710 in, 18422 out), 4438.5 s summed latency,
0 cloud calls, $0.00. Execution class: LOCAL_MODEL_INFERENCE. Host time is not priced.

## Not evaluated

| model or family | status |
|---|---|
| DeepSeek family | NOT_EVALUATED (weights not on this host; not downloaded: CPU-only shared host budget) |
| Gemma (gated) | NOT_EVALUATED (weights not on this host; not downloaded: CPU-only shared host budget) |
| Llama family | NOT_EVALUATED (weights not on this host; not downloaded: CPU-only shared host budget) |
| Mistral family | NOT_EVALUATED (weights not on this host; not downloaded: CPU-only shared host budget) |
| Phi family | NOT_EVALUATED (weights not on this host; not downloaded: CPU-only shared host budget) |
| Qwen >= 3B | NOT_EVALUATED (weights not on this host; not downloaded: CPU-only shared host budget) |

## Blocked providers

| provider | status |
|---|---|
| anthropic | BLOCKED_BILLING (no credit purchased; billing not enabled) |
| gemini | BLOCKED (no credential) |
| huggingface_inference_api | BLOCKED (no credential) |
| llama_cpp | BLOCKED (binary not installed on this host) |
| ollama | AVAILABLE_NOT_USED (production catalog container cain-cat-ollama, capped at 2 CPU / 4 GiB; qwen2.5-coder:3b Q4_K_M measured at 10-40 s per call in 4 probe calls; not loaded with benchmark traffic without owner approval) |
| openai_compatible_remote | BLOCKED (no credential) |
| vllm | BLOCKED (no GPU; not installed) |

Fallback policy: a path whose model is unavailable is refused with MODEL_BLOCKED_NOT_AVAILABLE; the router falls back to the next measured path for that family; a blocked model is never replaced by a mock

No credits were purchased, no billing was enabled and no cloud result was invented. A blocked model is never replaced by a mock.

## Specialist capability profiles

10 of 20 specialist profiles are measured; 10 are NOT EVALUATED.
All specialists share model weights; they differ by identity, instructions, tool grants and sampling. A claimed expertise is not evidence of expertise.
