The article describes a determinism layer for LLM‑based agents that isolates the model to a single node so the surrounding system remains ordinary, testable code. An agent is defined as a pure function `propose(context) → Proposal` that never mutates business state; a separate, heavily‑tested substrate applies proposals only after approval. The core execution graph consists of twelve shared nodes (entry, pre_check, context_load, llm_decision, output_guardrail, verification, judge, confidence_compose, routing, prepare_proposal, memory_write, exit) plus a few capability‑specific nodes, with llm_decision as the sole nondeterministic step and verification as the deterministic, capability‑specific check that validates the model’s structured output. Structured output is enforced via schema‑guided JSON, tool/function calls, or grammar‑constrained decoding, with validation and up to two retries before failing closed. Confidence is composed from model output, verification score, and a sampled judge, then routed by a threshold T (default 0.85) into auto, hitl_recommended, hitl_required, or reject paths. Every decision is written as an immutable row in an append‑only ledger that hashes inputs, records model/prompt versions, and stores the final JSON decision, confidence, routing, and outcome. For open‑ended tasks, a bounded ReAct loop is allowed, limited by a hard step cap (e.g., MAX_STEPS = 6), a per‑capability tool allow‑list, full per‑iteration trace, and the same guardrails, verification, confidence, and routing as the fixed graph. This layer yields testable pure proposals, safety through proposal‑only effects, composability via explicit state, and auditable, uniform decision records.
Read original
stackoverflow/blog