FALSEWORK · PROVE IT'S REAL

Sample report. The system audited here is a small demonstration project written for this purpose. It is not a customer, and its defects are deliberate. The engine output, the auditor notes and the grades were produced by the same tool and the same steps as a real engagement.

Audit report: Acme Support Bot

Is this AI implementation real? A mechanical audit of the source and the running endpoints, read and graded by an auditor.

Prepared for Acme Corp (demonstration)

Report FW-SAMPLE-0001

Auditor FIRME AI

Status FINAL

Scanned 2026-10-13

Basis static and live

1. Verdict

STUB

At least one result it presents is not computed at all, or a route calls code that does not exist.

Overall grade is the lowest grade of any component, and never better than PARTIAL when a tested endpoint gives the same answer to different inputs.

The summarizer and the billing arithmetic are real. The confidence figure shown to customers is a random number, the sentiment route calls a method that was never written, and the sentiment endpoint returns the same score whatever it is sent. The CRM integration is unfinished.

Component	Grade	Files	Lines	Critical	High	Medium	Low
`app`	STUB	2	28	2	0	0	0
`billing`	REAL	2	20	0	0	0	0
`integrations`	PARTIAL	2	24	0	1	1	1

Counts are findings that count toward the grade: confirmed, or still an open question. 2 finding(s) the auditor judged not an issue are listed in section 4 and do not count.

Grade	What it means
REAL	No finding that counts against it. The code computes what it returns.
PARTIAL	Works in part. At least one confirmed finding means some of what it presents is incomplete or is not computed.
STUB	At least one result it presents is not computed at all, or a route calls code that does not exist.
UNKNOWN	Could not be determined from what was supplied.

2. Findings that count

CRITICALapp/engine.py:12rng_fabricationCONFIRMED

Random number returned as the result of `confidence` -- the value it presents as a computation or measurement is built from a random draw and constants only.

return random.uniform(0.82, 0.99)

CRITICALapp/main.py:14route_engine_mismatchCONFIRMED

Calls engine.sentiment() but class Engine has no method named 'sentiment' -- this call will raise AttributeError at request time.

return engine.sentiment(body["text"])

HIGHintegrations/crm.py:4swallowed_import_errorCONFIRMED

ImportError silently caught (symbol set to None, or ignored with pass) -- later use will fail or silently no-op rather than fail loudly at startup.

except ImportError:

Auditor: crmlib is used on line 10 without checking for None.

MEDIUMintegrations/crm.py:9placeholder_markerOPEN QUESTION

Explicit placeholder/incomplete-work marker left in shipped code.

# TODO: placeholder until the CRM contract is signed

Auditor: Is the CRM sync sold to customers today, or switched off?

LOWintegrations/crm.py:11hardcoded_successCONFIRMED

Hardcoded success response with no visible computation preceding it (needs human review, not conclusive alone).

return {"status": "ok"}

Auditor: Returns ok whether or not the ticket reached the CRM.

3. Live differential test

Each endpoint at https://support-bot.acme.example (demonstration server) was called twice with two different payloads. A real computation gives different answers to different inputs; byte-identical answers mean the response does not depend on the input.

Endpoint	Result	SHA-256 of the two answers (first 16)
`POST /summarize`	Answers differ with the input	`64e722cf93f7e6a2` `b69862068f99e8e3`
`POST /score`	Same answer to two different inputs — CONFIRMED Auditor: Two different support messages received the identical score and label.	`7a3f1e65c962d1cd` `7a3f1e65c962d1cd`

4. Findings judged not an issue

Reported by the engine, read by the auditor, and set aside for the reason given. They are kept in the report so the reader can disagree.

LOWbilling/retry.py:7rng_fabricationNOT AN ISSUE

`backoff` returns a random draw and is named as a sampler -- listed for review, not counted as fabrication.

delay = min(30.0, 0.5 * 2 ** attempt) + random.uniform(0, 0.25)

Auditor: Jitter on a retry delay. The random value is a wait time, not a result shown to anyone.

LOWintegrations/cache.py:4swallowed_import_errorNOT AN ISSUE

ImportError caught; the file checks the name for None before use, so the missing dependency is handled (review that every use is behind that check).

except ImportError:

Auditor: Redis is optional and get() checks for None before use.

5. What was examined

6 Python file(s), 72 lines, in 3 component(s) (one per top-level directory).
Digest of the examined tree: 29c11bb88b51e05899fe2f4aaa83192766b851d38ef0c77e3b0a36160179ccc9. It is the SHA-256 of the list of every file's path and SHA-256, so you can confirm the report is about the code you supplied.
Engine: CAINWitness, engine.py SHA-256 1063628a86484b7b….

6. Method and limits

Static pass. A fixed list of source patterns (random numbers returned as results, import errors that are swallowed, placeholder markers, hardcoded success answers, data named as mock or simulated) and a cross-check of each route's calls against the members its engine class defines. It reads Python only.
Live pass. Each endpoint is called twice with different payloads and the answers are compared byte for byte.
Auditor. Every finding is read by a person, who confirms it, sets it aside with a reason, or leaves it as a question for you. The engine's findings are never edited or removed.
What a REAL grade does not mean. It does not mean the code is correct, secure, accurate or fit for a purpose. It means none of the patterns above was found and the tested endpoints answered differently to different inputs. Fabrication written outside those patterns is not detected by the static pass.
What this report is not. It is not a certification, a penetration test or a legal opinion, and it has not been reviewed by a third party.

7. Check this report yourself

This report is unsigned. Its integrity rests on digests you can recompute:

verdict.json, delivered with this report, holds every grade, finding and determination above in machine-readable form. Its SHA-256 is 17f3a18000c4f89ffc024d1b5c27d43ebed18da758f66e484566aa430fcc9e10.
verdict.json names the SHA-256 of the engine's raw output (audit.json) and the digest of the examined tree, so a changed finding or a different code base shows up as a different digest.

FALSEWORK is a service of FIRME AI, built on the CAINWitness engine. Report FW-SAMPLE-0001, generated 2026-10-13. Grades describe the code and endpoints supplied on the scan date and nothing else.