# Evolution 16 — failure mode and effects analysis

Severity (S) and detectability (D) are 1-5 (5 = worst / hardest to detect). Likelihood (L) is an **assumption**, not a
measurement: there is no field data. Owner is the role that would have to act; no person has been assigned.

| # | Failure mode | Effect | S | L (assumed) | D | Mitigation in Evolution 16 | Owner | Remaining risk |
|---|---|---|---|---|---|---|---|---|
| 1 | Crash after the target applied the effect, before the result is written | effect exists, record says only "intent" | 4 | 3 | 1 | intent is written first; recovery marks AMBIGUOUS; no re-dispatch | runtime | action waits for an observer; mission stalls |
| 2 | Acknowledgement lost (timeout after success) | caller retries, duplicate effect | 4 | 4 | 1 | AMBIGUOUS, retries refused (crash sweep: 0 duplicate writes at 71 points) | runtime | same as 1 |
| 3 | Observer reports "absent" wrongly | second dispatch, duplicate effect | 4 | 2 | **5** | none; the target's own idempotency is the only defence | operator | **open**: require k observers for effects |
| 4 | Observer reports "present" wrongly | mission recorded complete without the effect | 4 | 2 | **5** | a later contradicting pinned observation reopens it and fails verification | operator | **open** |
| 5 | Journal tail deleted | history silently shortened | 5 | 2 | 2 (with head) / 5 (without) | verifier returns INCOMPLETE unless a witness-held head is supplied | operator | nobody is running a witness |
| 6 | Journal file lost entirely | mission state gone; in-flight effects unknown | 5 | 2 | 1 | none (single file, single host) | operator | **open**: replicate the journal (Evolution 14 state service) |
| 7 | Replay diverges after a code upgrade | runtime refuses to start | 3 | 3 | 1 | fail closed (`RecoveryDivergence`) | engineering | **open**: no versioned replay or snapshot; an upgrade that changes any decision strands the mission |
| 8 | Journal grows without bound | recovery time grows linearly (measured in performance_results.json) | 2 | 4 | 1 | none | engineering | **open**: no snapshot or compaction |
| 9 | Two runtimes on one journal | forked history | 5 | 2 | 1 | `flock`; second opener gets `JournalLocked` | runtime | `flock` is advisory and does not hold across NFS reliably |
| 10 | Contract narrowed while an action is in flight | action completes under the old contract | 3 | 3 | 2 | started actions are kept and recorded under the old digest; unstarted admissions are void | principal | by design; visible in the journal |
| 11 | Budget lowered below what is already used | usage above the limit | 2 | 2 | 1 | usage carried over; further reservations fail | principal | none |
| 12 | Compensation executor succeeds but does not undo | false sense of rollback | 4 | 3 | 1 | counts only on observer confirmation; else COMPENSATION_FAILED, completion blocked | runtime | incident handling is manual |
| 13 | Compensation dispatched, outcome unknown | unknown residual effect | 3 | 3 | 1 | COMP_AMBIGUOUS; no blind retry | runtime | waits for an observer |
| 14 | Irreversible action wrongly labelled reversible in the plan | compensation attempted on something that cannot be undone | 4 | 2 | 4 | the label comes from the signed plan; unknown is treated as irreversible; mislabelling is not detectable here | plan authority, approver | **open** |
| 15 | Operation declared idempotent but the target does not deduplicate | duplicate effect on retry | 4 | 2 | 4 | none; it is operator configuration | operator | **open** |
| 16 | Clock set back or forward | expired authority accepted, or valid authority refused | 4 | 1 | 4 | none beyond using one clock per entry | operator | **open** |
| 17 | Executor blocks forever | runtime lock held; mission and all other inputs stall | 3 | 3 | 2 | none; the executor runs under the lock | engineering | **open**: no dispatch timeout, no concurrent dispatch |
| 18 | Verifier and runtime disagree | false FAILED or false VERIFIED | 4 | 2 | 2 | exploration invariant I7 over 16,104 sequences; differential test of the two classifiers | engineering | bounded; same author wrote both |
| 19 | fsync does not reach stable storage (tmpfs, lying disk) | acknowledged entries lost on power loss | 4 | 2 | 5 | none | operator | measured on tmpfs and ext4 only; no power-loss test |
| 20 | Key compromise of the runtime | arbitrary journal | 5 | 1 | 3 | other keys still required for every admission; witnessed head limits rewriting | operator | no key rotation path for the journal key |