Local Models for Coding Agents

Local models are worth considering when code cannot leave the machine, or when thousands of small edits run every day. This page lists candidates by GPU memory (8, 12, 16 and 24 GB) and by Apple Silicon unified memory (16, 32 and 64 GB). Whether a given model fits, and how much room is left for context, is worked out in single-device deployment.

The first test for a candidate is format, not score: a high benchmark score cannot compensate for output the harness cannot parse. gpt-oss-20b, for example, was trained on Harmony, the chat and tool-call format OpenAI defined for gpt-oss, and its model card says it will not work correctly with any other format. If the runtime does not support Harmony, the harness cannot parse the model's tool calls, however high its scores.

A Mac's unified memory also has to hold the operating system, the display and other programs, so less is left for the model than the nominal size. Since Ollama 0.40.0, architectures that MLX supports run on MLX automatically on Apple Silicon; Gemma 4 is among them.

Candidate snapshot

Verified from the providers' official model cards on 2026-10-29. No candidate below has yet earned a universal “best local coder” label. Exact quantization, runtime, template, context, and harness define the tested system.

Candidates by memory tier

Memory tier	Try first	Also worth trying	Check first
8 GB GPU	Gemma 4 E4B: 4.5B effective parameters, 128K context; official QAT q4_0 GGUF of 5.15 GB	MiniCPM5-2B: 2.5B dense (1.98B without embeddings), 131,072 context, Apache-2.0, official GGUF, MLX and GPTQ builds	This tier suits triage, query generation and small one-file edits only. LiveCodeBench v6 is 52.0 for E4B and 69.1 for MiniCPM5, both vendor figures; MiniCPM5 needs the minicpm5 tool-call parser in SGLang
12 GB GPU	Gemma 4 12B: 11.95B dense, 256K context, LiveCodeBench v6 72.0; official QAT q4_0 GGUF of 6.98 GB	The 8 GB models with a longer context	The file takes about 7 GB, leaving about 5 GB for the KV cache and the runtime
16 GB GPU	gpt-oss-20b: MoE with 21B total / 3.6B active parameters, MXFP4, which the model card says runs within 16 GB of memory; Apache-2.0	Gemma 4 26B-A4B: MoE with 25.2B total / 3.8B active; official QAT q4_0 GGUF of 14.4 GB	gpt-oss needs a runtime that supports Harmony; once the Gemma 26B file is loaded, less than 2 GB is left for context
24 GB GPU	Devstral Small 2 24B: Apache-2.0, 256K context, SWE-bench Verified 68.0 (vendor figure); the card says it runs on a single RTX 4090	Gemma 4 31B: 30.7B dense, LiveCodeBench v6 80.0, official QAT q4_0 GGUF of 17.7 GB. Qwen3.8-27B: 27B dense, Apache-2.0, 262,144 native context, about 13.5 GB of weights at 4-bit	Devstral in vLLM needs --tool-call-parser mistral --enable-auto-tool-choice. Qwen3.8-27B has no official GGUF, only about 1,600 community quantizations; thinking is on by default and can be turned off per request with enable_thinking: False
Mac 16 GB	The QAT GGUF of Gemma 4 E4B or 12B	The official MLX 4-bit build of MiniCPM5-2B	mlx-lm 0.31.2 added the Gemma 4 tool-call parser and 0.32.0 the MiniCPM5 one. gpt-oss-20b is rated for 16 GB, which is too tight when the system shares that memory
Mac 32 GB	Devstral Small 2 24B: the card says it runs on a Mac with 32 GB of RAM	The QAT GGUF of Gemma 4 26B-A4B or 31B (14.4 / 17.7 GB); gpt-oss-20b	Confirm that the runtime parses Devstral's Mistral tool-call format and gpt-oss's Harmony correctly
Mac 64 GB	An 8-bit community quantization of Qwen3.8-27B or Qwen3-Coder-30B-A3B, about 27–31 GB of weights	Flint 40B-A2.5B: Recursive's coding-agent model, 39.8B total / about 2.5B active. 262,144 context, Apache-2.0, SWE-bench Verified 74.2 (vendor figure); the Ollama build is 23 GB	Qwen3-Coder-30B-A3B has a non-thinking mode only and 262,144 native context. Flint's official card deploys on a single B300 or H200 and ships no official quantized files; the Ollama build is packaged separately

Note: “Try first” and “Also worth trying” are inferences from file size and model positioning, not official model-card recommendations. MoE “active parameters” describe per-token compute; they do not shrink the total weights that must be stored. “About N GB of weights” is parameter count times bytes per parameter (0.5 byte at 4-bit, 1 byte at 8-bit), without the KV cache.

Candidates that enter a real selection are retested with the local model evaluation protocol.

Selection Axes

Before downloading, choose the task and score candidates on:

• exact patch and tool-call validity;
• repository-language coverage, not only Python;
• accepted-result rate over repeated runs;
• latency, peak memory, CPU offload, and usable context;
• license and downstream redistribution obligations for the exact artifact;
• chat/reasoning format support in the chosen runtime;
• multilingual behavior and security/refusal behavior relevant to the workflow.

Role-Based Routing

• A compact model takes low-consequence, externally checked steps: file triage, query generation, log reduction, or a one-file mechanical edit; a larger quantized model takes bounded implementation or review.
• Every change happens in a copy of the repository with tests run there; on failure, discard the copy without touching the working directory.
• Architecture, security-sensitive work, or a repeated error signature goes to a stronger model or a person; when a task counts as done and when to stop are covered in Bounded Agent Loops.

My reading