Not every LLM application should be an agent. A key design decision is how much freedom the model gets over the control flow.
Workflows
In a workflow, developers define the sequence of steps in code. The model may fill in each step, such as classifying a message or drafting a reply, but the order is fixed. Workflows are predictable, easier to test, and cheaper.
Agents
In an agent, the model chooses the next action dynamically, based on what it has observed. This handles open-ended tasks where the steps cannot be known in advance, but it adds variability and risk.
A spectrum, not a switch
Many systems sit in between: a mostly fixed workflow with a small agentic loop for one uncertain step, or an agent restricted to a narrow set of tools.
How to choose
- If the steps are known, use a workflow.
- If the task needs exploration or branching you cannot enumerate, consider an agent.
- Prefer the least autonomy that solves the problem.
Define explicit boundaries
For any agent, write down what it may do on its own, what needs human approval, and what is forbidden. Add limits on the number of steps, the spend, and the tools available.
Why this matters
Autonomy is a cost as much as a capability. Each added degree of freedom needs matching tests, monitoring, and safeguards.
Continue with the book
Explore the full treatment, worked examples, and exercises in Building Production-Grade AI Agents.
Adapted from Building Production-Grade AI Agents, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.