Agents depend on slow, unreliable services, including the model itself. A few resilience patterns keep one failure from becoming a pile-up.
Timeouts
Every external call needs a deadline. Without one, a hung request can freeze the whole agent. Set timeouts based on observed latency, and apply an overall budget for the full task, not just each call.
Retries with backoff
Transient errors often clear up on a second attempt. Retry only errors that are likely temporary, increase the delay between attempts, and add random jitter so many clients do not retry at the same moment. Limit the number of attempts.
Retry only what is safe
Retrying a read is usually safe. Retrying an action with side effects requires idempotency protection.
Circuit breakers
A circuit breaker tracks recent failures for a dependency. After too many, it opens and fails fast for a period instead of sending more requests. It then allows a few trial requests to test recovery. This protects the dependency and keeps the agent responsive.
Fallbacks
Decide what the agent should do when something is unavailable: use a cached result, switch to a simpler method, ask a human, or stop cleanly with a clear message.
Observe it
Log retries, timeouts, and breaker state so you can tell whether failures are rare blips or a developing outage.
Continue with the book
Explore the full treatment, worked examples, and exercises in Building Production-Grade AI Agents.
Adapted from Building Production-Grade AI Agents, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.