Article · 2026-07-22 · 6 moments

Best Practices for Building AI Agents That Work in Production

In this article, we try to explore the collective thinking into a smaller set of practices and explain the reasoning behind each one, rather than asking anyone to memorize a numbered list. ✦ AI generated

Read original ↗

Claim◆

A production agent is mostly deterministic software that calls a language model at a few deliberate points, with the design decisions lying in choosing those points and limiting how much the model decides on its own.

The article defines a production-grade AI agent as primarily deterministic software that invokes a language model only at carefully chosen decision points, contrasting this with demo-quality agents that rely heavily on the model.

transcript

Best Practices for Building AI Agents That Work in Production (author): A production agent, reduced to its essentials, is mostly deterministic software that calls a language model at a few deliberate points. The design decisions lie in choosing those points and limiting how much the model decides on its own. The path to that definition ran through the simplest possible agent and its predictable failures, with each failure answered by one practice.

Data

Compounding error multiplies across chained steps: if each model call is correct 95% of the time, running twenty steps in sequence succeeds only about one in three times.

The article explains that even high per-step accuracy (95%) compounds into low overall reliability over a chain of twenty steps (roughly 33%), which is why production teams add many guardrails around model calls.

transcript

Best Practices for Building AI Agents That Work in Production (author): Suppose each step is correct 95 percent of the time, which sounds reliable. Run twenty such steps in sequence, and the odds that all of them succeed fall to roughly one in three. The math is multiplicative, so reliability that looks fine in isolation degrades quickly across a long chain. This fact explains why production teams add so many guardrails.

explains mechanism · 3

Mechanism◆

Controlling what the model sees on every call — owning prompts, pruning the context window deliberately, and writing precise tool descriptions — is the first and largest lever on reliability.

The article identifies context engineering — owning prompts like source code, actively pruning the context window to keep it focused, and designing precise tool schemas — as the most impactful reliability practice for AI agents.

transcript

Best Practices for Building AI Agents That Work in Production (author): A model given a focused, relevant context performs better than the same model handed a large pile of loosely related history. Quality degrades as the window fills with marginal material. Therefore, deliberate pruning by actively removing content beyond what the current step requires preserves the model's accuracy. In other words, relevance beats volume on almost every call.

explains mechanism · 2

Mechanism

State should live in serializable storage controlled by the application, keeping the model stateless and allowing agents to pause, resume, recover from crashes, and scale horizontally behind a load balancer.

The article explains that storing agent state in the application's own software rather than in the model's context enables crash recovery, resumability, and horizontal scaling, since any instance can pick up any request from the saved state.

transcript

Best Practices for Building AI Agents That Work in Production (author): The application stores the real state of the work, the conversation so far, the plan, the progress, and reconstructs the context from it on every call. Since the state lives in serializable storage, an agent can pause partway through a task and resume later, or recover cleanly after a crash, by loading the saved state and continuing.

Mechanism

The control flow in a dependable agent belongs to deterministic code around the model, with the model consulted only at a few chosen points where open-ended reasoning is genuinely needed.

The article advises that ordinary code — sequences, conditionals, limits — should own the loop and stopping conditions, while the model is reserved for judgment calls that require reasoning, with hard caps on iterations as an escape hatch.

transcript

Best Practices for Building AI Agents That Work in Production (author): Picture the overall flow as ordinary code, a sequence of steps, conditionals, and calls to external systems. At two or three points in that flow, where a decision genuinely calls for judgment, the system invokes the model. Everywhere else, plain deterministic logic does the work because it is cheaper to run, predictable in its output, and straightforward to test.

extends · 1supports · 1

Claim

The main debate in agent architecture is single versus multiple agents, and the emerging resolution is that a single orchestrator owning the full context and spawning isolated, short-lived sub-agents works best, while sub-agents that communicate directly tend to produce conflicting results.

The article describes the 2025 debate between Cognition (arguing against multi-agent designs) and Anthropic (showing multi-agent scoring 90% higher on research tasks), and distills the resolution: one orchestrator spawns isolated sub-agents, avoiding direct sub-agent communication that causes conflicts.

transcript

Best Practices for Building AI Agents That Work in Production (author): A single orchestrator owns the full context and spawns isolated, short-lived sub-agents, each of which completes one task and returns a summary. The lesson to understand is that small, focused agents work best under firm orchestration, while sub-agents that communicate directly with one another tend to produce conflicting results.

provides context · 1

Highlight slides

A Production Agent is Mostly Deterministic✦ from: A production agent is mostly deterministic software that calls a language model at a few deliberate points, with the design decisions lying in choosing those points and limiting how much the model decides on its own.Context Engineering: The #1 Reliability Lever✦ from: Controlling what the model sees on every call — owning prompts, pruning the context window deliberately, and writing precise tool descriptions — is the first and largest lever on reliability.Why Pruning Works✦ from: Controlling what the model sees on every call — owning prompts, pruning the context window deliberately, and writing precise tool descriptions — is the first and largest lever on reliability.

Related episodes

Article · The Pragmatic Engineer

A production agent is mostly deterministic software that calls a language model at a few deliberate points, with the design decisions lying in choosing those points and limiting how much the model decides on its own.

Compounding error multiplies across chained steps: if each model call is correct 95% of the time, running twenty steps in sequence succeeds only about one in three times.

Controlling what the model sees on every call — owning prompts, pruning the context window deliberately, and writing precise tool descriptions — is the first and largest lever on reliability.

State should live in serializable storage controlled by the application, keeping the model stateless and allowing agents to pause, resume, recover from crashes, and scale horizontally behind a load balancer.

The control flow in a dependable agent belongs to deterministic code around the model, with the model consulted only at a few chosen points where open-ended reasoning is genuinely needed.

The main debate in agent architecture is single versus multiple agents, and the emerging resolution is that a single orchestrator owning the full context and spawning isolated, short-lived sub-agents works best, while sub-agents that communicate directly tend to produce conflicting results.

Context engineering with Dex Horthy

From Code Generation to Full System Autonomy: Blitzy's Approach

How Capital One Delivers Multi-Agent Systems [Rashmi Shetty] - 765