Agents rely on the model producing well-formed tool calls. Smaller local models make more mistakes here, so it pays to know the common ones.
How tool calling works
The runtime describes available tools to the model with names and parameter schemas. The model replies with a structured request. The runtime validates it, runs the tool, and returns the result.
Typical failures
- Malformed output: invalid JSON or missing required fields.
- Wrong tool or arguments: choosing a tool that does not fit, or inventing file paths.
- Repeated calls: looping on the same failing action.
- Ignoring results: continuing as if a failed command had succeeded.
Defenses in the harness
Validate every call against its schema and return a precise error message. Limit the number of tools shown at once. Constrain output formats where the runtime supports it. Cap the number of steps.
Simplify the task
Smaller models perform better with short, specific instructions and focused tools. Break large tasks into stages.
Read the transcripts
Review logs of failed runs like you would application logs. They show whether the problem was the prompt, the tool design, or the model.
Takeaway
Do not blame the model before checking the harness. Strict validation and simpler tools make small models far more dependable.
Continue with the book
Explore the full treatment, worked examples, and exercises in Local AI Agents for Angular Teams.
Adapted from Local AI Agents for Angular Teams, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.