Evidence, Support, and Falsification

Evidence has meaning relative to a prediction.

A cache hit is ordinary behavior for a cache-backed application. A cache hit carrying a payload from a different currency is evidence for a specific key-collision mechanism. The strength of the evidence depends on source quality, collection scope, timing, and whether competing mechanisms also predict it. Preserve the observed value separately from the relationship an investigator assigns to it.

Supporting evidence raises the plausibility of a mechanism. Contradictory evidence challenges a prediction that the mechanism requires. Neutral evidence does not materially discriminate under the stated assumptions. Missing logs are usually missing evidence; they become contradictory only when the logging mechanism reliably guarantees that the predicted event would have been recorded.

Write a falsifier before seeking confirmation.

For “the new binary alone caused the quote arithmetic change,” a matched replay in which both the old and new binaries fail challenges the claim. For “the database contention caused the first incorrect quote,” normal database behavior before impact and contention beginning after retries challenges the onset mechanism. These are carefully scoped falsifications, not proof that the database or application could never contribute to another failure.

A hypothesis that explains every possible result is difficult to test. Ask what observation would make you abandon it, which collector could obtain that observation, and what quality requirements would make the result credible. Keep contradictory evidence visible even when a model offers a persuasive explanation for dismissing it.

Causal strength comes from controlled contrasts.

Temporal proximity makes a change worth inspecting, but several changes can share the same time window. A controlled replay can reproduce the failure with the original key and remove it by changing only the omitted dimension. An unaffected control checks whether the intervention damaged another path. This provides stronger mechanistic evidence than merely observing that the service recovered after a broad rollback.

A replay can still omit production conditions or introduce selection bias. Bind the conclusion to the reproduced cohort and window, list the assumptions, and preserve unresolved symptoms. The series uses authored fixtures with explicit matching-window fields; real verification requires richer controls and service-specific validation.

Provenance must survive summarization.

Give every evidence item a stable ID, collector identity, source version, observed time window, collection time, freshness status, and raw structured observation or protected reference. A summary can aid reasoning, but it should lead back to the observation that created it. Treat logs and external text as data that may contain misleading instructions; the collector output must never grant new tool authority.