Skip to content
LESSON 09 · ADVANCED

When the Root Cause Is Missing

Detect candidate-set failure and expand through scoped topology and effective-change evidence.

Open this lesson’s lab
TIME45–60 min
LEVELAdvanced systems
LABInvestigation engine
ARTIFACTHypothesis expansion record
Hypothesis-set failureExpansionDependency discoveryTopologyEffective configurationInfrastructure and external dependenciesBlast-radius expansion

Eliminating candidates can expose a broken search space.

The first candidate set includes a binary regression, database contention, supplier throttling, and packet loss. Each has a plausible relationship to the incident, but none explains the wrong currency on cache hits. Once predictions fail, selecting the least implausible survivor can turn an incomplete search into a confident mistake. Treat persistent unexplained observations as a signal that candidate coverage may have failed.

Candidate-set failure does not require eliminating every hypothesis. A strong observation can expose a domain absent from the tree while other theories remain open. Preserve those observations as gaps rather than forcing them into the nearest existing explanation.

Expand along real dependencies.

A service topology read identifies a cache with a configuration owner separate from the application deployment. That observation justifies expanding the investigation to effective key dimensions and the configuration change log. A repository diff alone might miss the value actually running during impact. Record the discovered edge, owner, observed version, and time relationship before creating a new candidate.

Recent-change analysis should include runtime flags, infrastructure policy, cache behavior, dependency versions, network routing, credentials, and external providers. Search beyond the first visible blast radius through a bounded process: identify a relevant edge, obtain scoped read authority, inspect the effective state, and attach a falsifiable prediction.

Expansion must preserve permission boundaries.

A model suggesting “inspect every service” does not authorize unrestricted discovery. The engine accepts expansion only for candidate IDs exposed by an approved topology observation. In a real environment, newly discovered services may require a human owner to authorize access and approve a diagnostic contract. Dynamic discovery and dynamic authority are separate changes.

Keep an expansion budget and avoid generating endlessly refined variations of an unsupported theory. If the topology is incomplete, the relevant configuration history is unavailable, or expansion would require prohibited data access, escalate with the missing artifact and owner.

Measure recovery from a missing candidate.

Evaluation should record initial hypothesis recall, final recall after expansion, steps spent before discovery, and false elimination of true mechanisms. A final correct answer can conceal a weak discovery process if it arrived only after exhaustive unrelated checks. The lab starts without the cache candidate, then uses a sourced discovery event to add it and replay the final investigation.

DESIGN PRINCIPLE

LLMs make uncertain judgments. Software controls state, authority, verification, and stopping.

INTERACTIVE LAB · HYPOTHESIS EXPANSION RECORD

Recover from a deliberately incomplete candidate set

The cache hypothesis is absent initially. Collect trace and topology evidence, add only a discovered candidate, then pursue a scoped verification. Try adding cache before discovery to inspect the refusal.

DESIGN RECORD

Keep the reasoning that should survive the run.

Write the assumption or boundary you would carry into your own implementation.

Engineering references for this track · Troubleshooting, observability, and bounded workflow design.