The article details the engineering required to harden a LangGraph-based analytics agent for Mattrx from a naive demo into a production system with p95 latency reduced from 4.2s to 1.8s and per-question cost cut from ~$0.90 to ~$0.12. The core agent loop implements a bounded ReAct pattern with a hard step cap (max_steps=8) to prevent infinite loops, strict tool schema validation to reject hallucinated calls, and safe invocation with error messages designed for model recovery. Planning strategy distinguishes ReAct for open-ended exploration from plan-and-execute for structured multi-step tasks. Tools are defined as typed, intent-level functions (~10) with docstrings serving as selection prompts, ISO date validation, tenant-scoped authorization, and least-privilege allowlists excluding destructive actions. Memory combines a LangGraph checkpointer (PostgresSaver) for durable working memory and vector retrieval for cross-session facts. A reflection step critiques high-stakes outputs before return. Guardrails enforce token and step budgets, human-in-the-loop approval for write actions, input/output validation including PII scanning, tool-result fencing against indirect prompt injection, and per-call timeouts. Full tracing via Langfuse provides observability. The author emphasizes that reliability stems from structural constraints—budgets, validation, and explicit graphs—not model upgrades, and identifies evals, observability, and cost engineering (caching, model routing) as the next production skills.

Read original