60–120 minNative JavaScript modulesOptional server-side model adapterNo production mutation capability
A small distributed system with a difficult diagnostic story.
A synthetic client calls a quote service, which uses a cache, a database, and an external supplier. An application deployment and database latency spike compete for attention while some requests return incorrect data with HTTP 200. Choose the standard, missing-cause, two-fault, or insufficient-evidence fixture. The latter must end with a usable unresolved handoff.
The browser uses an authored fixture judge. It makes the same bounded proposals a model adapter would submit, without an API key. The Node starter includes a model interface and an opt-in JSON chat-completions server adapter; model credentials stay outside the browser. Source observations, permissions, budgets, and verification remain owned by software.
System responsibilities
INCIDENT + SIGNALSScope user impact
Define the broken semantic contract and retain the affected time window.
CANDIDATES + POLICYChoose bounded checks
Maintain multiple hypotheses and expand through scoped discovery.
COLLECTOR + LEDGERPreserve observations
Record source versions, raw results, freshness, and stable evidence IDs.
INDEPENDENT VERIFIERTest the mechanism
Check baseline, treatment, negative control, and every impact scope.
ORCHESTRATORControl state and budgets
Validate judgments, enforce deadlines, and record accepted and refused actions.
OPERATOR + REPORTVerify or hand off
Export a replayable account and identify the owner of unresolved gaps.
Completion contract
Generate or expand multiple testable hypotheses.
Maintain explicit incident and per-hypothesis state.
Select only permitted diagnostic actions with typed envelopes.
Record structured evidence, source versions, scope, and time windows.
Falsify candidates when scoped evidence contradicts their predictions.
Keep unrestricted execution and production mutation outside the model interface.
Enforce decision, tool, model-call, cost, and elapsed-time budgets.
Distinguish a supported theory, a verified mechanism, and verified incident coverage.
Abstain when matching evidence is unavailable and escalate to an owner.
Export a replayable JSON ledger and a readable investigation report.
SYNTHETIC INVESTIGATION WORKBENCH
Make each check earn its place.
Step through bounded decisions or run the fixture judge. Request verification early, submit an invalid command field, lower a budget, or replay an exported report. Export a terminal report to complete the capstone artifact.
CENTRAL PRINCIPLE
LLMs make uncertain judgments. Software controls state, authority, verification, and stopping.
Take the pattern into your own service.
Replace the synthetic collectors with typed, authenticated, read-only service adapters. Supply a protected evidence store, service-specific verification contracts, a policy version, and an authorized server-side judge. Keep production mitigation in a separate approval workflow. Run held-out incidents, incomplete observations, misleading metrics, and adversarial tool output through the evaluator before expanding authority.