Bounded Decisions Instead of Open-Ended Agents

Define the decision before writing the prompt.

“Investigate this incident” lets a model choose the goal interpretation, queries, tools, memory updates, and ending condition together. Each of those choices creates a separate failure surface. A bounded decision supplies a limited question, the relevant evidence, and an allowed output space. The model can choose a permitted diagnostic check; ordinary software validates the choice and resolves that ID to a trusted collector.

Five useful judgments are candidate proposal, evidence classification, check ranking, next-check choice, and evidence-gap assessment. Keep their inputs narrow. A classifier needs an evidence item and a hypothesis prediction; a ranker needs permitted checks, expected outcomes, and cost. Neither needs production credentials or a generic shell tool.

Structured output exposes the proposed action.

A JSON envelope lets software reject unknown fields, unavailable IDs, duplicate rankings, and unsupported judgment kinds. It does not make the reasoning correct. Keep the returned rationale bounded, bind evidence references to the current ledger, and validate the permitted action again immediately before collection. A model can propose a check that was available in an old snapshot but has already been completed.

The teaching engine accepts choose_check, rank_checks, classify_evidence, and discovered-candidate expansion. Initial hypotheses come from the authored scenario. Invalid proposals consume decision budget, so repeated schema failures cannot create an infinite free retry loop. Rankings influence selection among available checks; they cannot introduce a new tool or override prerequisites.

{
  "kind": "choose_check",
  "checkId": "cache-keys",
  "reason": "Different currencies may collide under the same effective key."
}

Reserve authority for deterministic code.

A classification proposal belongs in a judgment log. It must not directly set a hypothesis to FALSIFIED or VERIFIED. In this series, authored predicates interpret trusted synthetic observations; the independent gate reads those observations rather than trusting a model-written status. A production system would need validated parsers and service-specific diagnostic contracts at the same boundary.

A yes/no model assessment such as “does this observation support this claim?” remains uncertain. The authorization decision “may this tool run?” uses explicit permissions and budgets. The transition “may this incident close?” uses named verification requirements. This separation allows model errors to remain visible without becoming execution authority.

Keep transitions small enough to replay.

Record the proposed judgment, whether validation accepted it, and the observation produced by the collector. The next view is reconstructed from that ledger. Replay should not ask a model to reason again; it should consume the recorded decisions and reproduce the deterministic control behavior. Version the fixture, policy, schema, and tool contracts so that a later software change cannot silently reinterpret an old report.