Running a coding model on your own hardware keeps source code in-house, but the model must fit your machine and be good enough for the job.
The five layers of a stack
A local agent setup has several parts: the hardware, an inference runtime, a model, an agent harness that provides tools, and the iOS toolchain the agent drives.
Memory is the first constraint
A model's weights occupy memory in proportion to the number of parameters and the precision used. Quantization stores weights with fewer bits, reducing memory with some quality loss. Context length also consumes memory through the key-value cache, so long sessions need headroom.
Throughput and latency
Agents make many calls, so speed matters. Measure tokens per second for both reading the prompt and generating output, using realistic prompts from your own projects.
Choose and pin versions
Select a model by testing it on your real tasks. Then pin the exact model, quantization, runtime, and settings so that results are reproducible and everyone on the team uses the same stack.
Remember the toolchain boundary
The model runs locally, but Xcode builds and simulators have their own requirements. Plan the machine to handle both.
Benchmark the whole stack
Evaluate the complete pipeline on tasks you care about, not only a general benchmark score.
Takeaway
The best model is the one that fits your hardware, runs at a usable speed, and completes your tasks reliably.
Continue with the book
Explore the full treatment, worked examples, and exercises in Agentic iOS App Development.
Adapted from Agentic iOS App Development, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.