How much VRAM do you need for a local coding model?

24GB is the practical entry point and 32GB is where it becomes comfortable. Qwen3-Coder-30B-A3B-Instruct at Q4_K_M is 18.6GB of weights alone, which rules out every 16GB card. On 24GB you have roughly 5GB left for KV cache, which is about 55,000 tokens of context at fp16 or about 110,000 with an 8-bit KV cache. On 32GB you can hold the model's full 262,144-token native window with an 8-bit KV cache.

Does quantization hurt coding models more than chat models?

The published evidence says no, which contradicts the common advice. Red Hat's study of over half a million evaluations across Llama 3.1 8B, 70B and 405B found HumanEval accuracy recovery of 99.9% for 8-bit and 98.9% for 4-bit quantization. A 2025 empirical study of quantized code models concluded that quantization not only preserves functional correctness but also retains qualitative code attributes. A separate 2025 robustness study found quantized models were often more resilient to adversarial prompts than full-precision ones. Q4 is not the thing costing you code quality.

How much VRAM does the KV cache use for a coding model?

For Qwen3-Coder-30B-A3B-Instruct, roughly 9.8GB per 100,000 tokens at fp16, or about 4.9GB at 8-bit. That figure is derived from the model's published configuration: 48 layers, 4 key-value heads and a head dimension of 128, which works out to about 98KB per token at fp16. The full 262,144-token native window costs about 25.8GB at fp16 — more than the weights themselves.

Is a 16GB GPU enough for local coding?

It is enough for a smaller model and short context, not for a 30B-class coding model. A 16GB card cannot hold Qwen3-Coder-30B at Q4_K_M, which needs 18.6GB before any context. On 16GB you run a 20B-class model such as gpt-oss 20B at Q4 with a modest window, which is genuinely useful for single-file edits and much less useful for agentic work across a repository.

What is the cheapest GPU that replaces a paid coding assistant?

A used RTX 3090 at roughly $1,000-1,300 as of August 2026. It is the cheapest card with 24GB, which is the cheapest capacity that runs a 30B-class coding model at a useful context length. Below that the honest answer changes: a 16GB card runs a smaller model well, but you are no longer comparing like for like against a cloud assistant on repository-scale tasks.

← All guides

Hardware August 18, 2026

Best GPU for Running a Local Coding Model (August 2026)

Most GPU buying guides for coding models give you the wrong reason to spend money. They tell you to buy VRAM so you can run a higher quantization. The published research says quantization costs you almost nothing on code. What actually eats your card is the context window — and a modern coding model ships with 262,144 tokens of it.

Setting up a local coding assistant?

See our AI training options. We'll pick the card, the model, and wire it into your editor.

Bottom Line (August 2026)

24GB is the entry. A used RTX 3090 ($1,000-1,300) or an RTX 4090 is the cheapest card that holds a 30B-class coding model with real context.
32GB is where it stops hurting. An RTX 5090 ($4,300-5,000) or an AMD R9700 32GB ($1,299 MSRP) holds the full 262K native window with an 8-bit KV cache.
16GB cannot do this job. Qwen3-Coder-30B at Q4_K_M is 18.6GB of weights before a single token of context.
Stop buying VRAM for a higher quant. Red Hat measured 98.9% HumanEval recovery at 4-bit across half a million evaluations. Q4 is fine. Buy VRAM for context.
Context is the real consumer: about 9.8GB per 100K tokens at fp16 on this model. The full window costs more than the weights.

Prices are US street as of August 2026 and have risen through the year — see our price context.

The Advice Everyone Gives, and Why It Is Wrong

Search for a coding-model GPU guide and you will be told, in some form: coding needs precision, so buy enough VRAM to run a high quantization, because Q4 will make your model write bad code.

The published research does not support this.

Red Hat ran over half a million evaluations across Llama 3.1 8B, 70B and 405B at W8A8-INT, W8A8-FP and W4A16-INT. On HumanEval — the code benchmark — 8-bit models recovered 99.9% of full-precision accuracy and 4-bit models recovered 98.9%. HumanEval+ held similarly.

A 2025 empirical study of quantized large code models (CodeLlama and DeepSeekCoder under AWQ, evaluated on Java and Python) concluded that quantization “not only preserves functional correctness, but also retains key qualitative code attributes” — measured with static analysis on cyclomatic complexity and cognitive complexity, not just pass rates.

A separate 2025 study on robustness went further in the same direction: across LLaMA, DeepSeek, CodeGen and StarCoder from 350M to 33B, quantized models showed better resilience to adversarial prompts than their full-precision versions in 51.59% of experiments against 42.86%.

Three independent bodies of work, one direction. Q4 is not what is degrading your local coding assistant.

So what is? Almost always the context window. A coding assistant that has read four files instead of forty is not a precision problem, it is a capacity problem — and capacity is exactly what people spend their VRAM budget failing to buy.

The Arithmetic That Should Drive Your Purchase

Take the model most people actually run for local coding in 2026: Qwen3-Coder-30B-A3B-Instruct. Its published configuration is 30.5B total parameters with 3.3B active (128 experts, 8 per token), 48 layers, 4 key-value heads, head dimension 128, and a native context length of 262,144 tokens, extendable toward 1M with YaRN.

Weights, from the published GGUF file sizes:

Quant	File size
Q4_K_M	18.6 GB
Q5_K_M	21.7 GB
Q6_K	25.1 GB
Q8_0	32.5 GB
BF16	61.1 GB

KV cache, derived from that configuration. Per token, the cache stores keys and values for 4 KV heads of 128 dimensions across 48 layers: 2 × 4 × 128 × 48 = 49,152 values, which at fp16 is about 98KB per token.

Context	KV cache at fp16	KV cache at Q8
32K tokens	~3.1 GB	~1.6 GB
100K tokens	~9.8 GB	~4.9 GB
262K (full native)	~25.8 GB	~12.9 GB

Those KV figures are arithmetic from the model’s own config file, not benchmarks — runtime overhead adds a little on top.

Read the two tables together and the buying decision falls out. At full context, the cache costs more than the model. No amount of quantizing the weights fixes that; quantizing the cache does, which is why Q8 KV cache is the single highest-value setting in a local coding setup.

What Each Card Actually Gives You

Assuming Qwen3-Coder-30B-A3B at Q4_K_M (18.6GB) and leaving ~1GB of runtime overhead:

VRAM	Cards	Free for KV	Context at fp16	Context at Q8 KV
12GB	RTX 3060, Arc B580	—	Does not fit	Does not fit
16GB	RTX 4060 Ti, 5060 Ti, 5080	—	Does not fit	Does not fit
24GB	RTX 3090, 4090, RX 7900 XTX	~4.5 GB	~45K	~90K
32GB	RTX 5090, R9700, Arc Pro B70	~12.5 GB	~125K	~250K
48GB	2x 3090, RTX A6000	~28 GB	~280K	full window + headroom

That table is the post. 24GB gets you a coding assistant that sees a few files. 32GB gets you one that sees a repository.

If your card is 16GB or smaller, you are not buying a smaller context — you are buying a different model. gpt-oss 20B at Q4 lands around 12-13GB and is a real local coding model at that tier; see our 16GB picks and the 20B-35B band overview.

What to Buy

Best value — used RTX 3090 24GB

The cheapest route into the 24GB tier, at roughly $1,000-1,300 used as of August 2026. CUDA, 936 GB/s, and every local coding toolchain works on it without argument. Its weakness for this workload is exactly the context ceiling above — around 90K tokens with an 8-bit KV cache, which covers a working set of files but not a whole codebase.

Buy used carefully: our used 3090 checklist exists because the price has nearly doubled since the “$650 3090” advice still circulating online.

24GBEVGA RTX 3090 24GB ↗

Faster 24GB — RTX 4090

Same capacity, meaningfully faster at 1008 GB/s, and better prefill throughput — which matters more for coding than for chat, because agentic sessions re-read large prompts constantly. New stock is limited and prices run $2,500-3,700, with used cards around $2,270-2,600. It does not change what fits; it changes how long you wait.

24GBGIGABYTE RTX 4090 GAMING 24GB ↗

The right card, at a price — RTX 5090 32GB

32GB at 1792 GB/s. This is the card that actually matches the workload: full native context with an 8-bit KV cache, and the bandwidth to prefill a long prompt quickly. It is also $4,300-5,000 as of August 2026, several times its nominal MSRP, because 32GB of GDDR7 is exactly what the memory shortage is squeezing.

If the machine is a business expense and you code with it daily, this is the buy. If it is not, the 3090 with an 8-bit KV cache is the honest recommendation and we would rather you kept the money.

32GBGIGABYTE RTX 5090 WINDFORCE 32GB ↗

32GB without the NVIDIA tax — AMD Radeon AI PRO R9700

32GB at $1,299 MSRP when in stock, roughly a third of a 5090 for the same capacity. It is on AMD’s supported list in current ROCm, so llama.cpp and Ollama work — but the ecosystem around them is thinner, and stock is intermittent with resale listings running well above MSRP. Read R9700 vs 3090 before committing.

32GBXFX Radeon AI PRO R9700 32GB ↗

Under $800 — accept a smaller model

If the budget is a 16GB card, buy one and run a 20B-class model rather than stretching for a 30B you cannot hold. The RTX 5060 Ti 16GB is the current entry, at $589-805 street.

16GBASUS Dual RTX 5060 Ti 16GB ↗

Before You Buy Anything: Turn On the 8-Bit KV Cache

This is free and it doubles your usable context on any card in the table above.

Most local runtimes default to an fp16 KV cache. Switching to 8-bit halves the per-token cost — 9.8GB per 100K tokens becomes 4.9GB — for a quality impact far smaller than the context you gain. On a 24GB card it is the difference between a coding assistant that has read your file and one that has read your module.

The general principle holds across the whole quantization discussion: quantizing the cache is cheaper than quantizing the model, and both are cheaper than running out of memory.

The Honest Caveat

Everything above assumes a 30B-class local model is the goal. It is worth naming what you give up against a cloud assistant.

A local 30B coder at 24GB with 90K of context is genuinely useful, and it is not Claude or GPT on a large refactor. Our own comparison of Qwen as a Claude Code alternative is direct about where it holds and where it does not. The cases where local wins outright are privacy, cost predictability, offline work, and not being rate-limited mid-task — which is why we also wrote the cheapest hardware that cancels a coding subscription.

Buy the card for those reasons. Do not buy it expecting the frontier.