Contamination Scanner, explained for beginners

Contamination Scanner checks whether a model memorized the test it is scored on, so you don't buy a score instead of a model.

What it is

Companies choose AI models by their scores on tests (benchmarks). If a model saw the test's answers during training, it can recite them, and its score says nothing about how it will do on your real work. This is called contamination.

The scanner gives the model the start of each test item and checks whether it completes it word for word. It does the same with control items the model cannot have seen. A model that recites the test but not the controls is flagged; a clean model is cleared.

An everyday comparison

Checking whether a student memorized last year's exam answers instead of learning the subject.

How it works, in 4 steps

1. Give the model the start of each test item.
2. Measure how much of the true answer it reproduces word for word.
3. Do the same with unseen control items.
4. Big difference: contaminated. No difference: clean.

Try it now

Open the three live demo tools → Every button sends a real request to the live product and shows the real answer.

See it on the homepage: Contamination Scanner section →

Who it is for

Teams choosing or fine-tuning models based on benchmark scores, and anyone publishing model results.

FIRME AI CORPORATION · CAIN-42 is a Byzantine trust fabric. Numbers marked "our own check" were run by us, not by an independent party. Nothing on these pages is a certification.