What is the best model to run on an RTX 3090 in 2026?

Gemma 4 26B-A4B at Q4_K_M is the best all-round pick for a 24GB RTX 3090 — about 16GB of weights and roughly 71 tok/s, because only ~4B parameters are active per token. For OpenClaw agent loops, gpt-oss 20B at MXFP4 uses about 13.8GB and runs near 73 tok/s with the cleanest tool-call JSON of any open-weight model. Dense options like Qwen 3.6 27B at Q4_K_M fit in ~18GB but decode at only ~31 tok/s on the 3090's 936 GB/s bandwidth.

Is the RTX 4090 still worth it for local LLMs in August 2026?

Yes, if you already own one. The 4090 runs the same 24GB model set as the 3090 but roughly 20-25% faster: Gemma 4 26B-A4B at ~85 tok/s, gpt-oss 20B at ~87 tok/s, Laguna XS 2.1 (33B MoE, 3B active) at ~86 tok/s, and Qwen 3.5/3.6 35B-A3B at ~84 tok/s. What it does not give you is a higher ceiling — it is the same 24GB wall as a used 3090 at two to three times the price. Buy a 4090 for speed, buy a 5090 for headroom.

What is the best model for an RTX 5090 32GB?

Qwen 3.6 35B-A3B at Q4_K_M — about 22GB of weights and roughly 118 tok/s on the 5090's 1,792 GB/s GDDR7. Laguna XS 2.1 at Q4_K_M is the agentic-coding pick at ~20.3GB and ~121 tok/s. The extra 8GB over a 4090 matters most for context: at 32GB you can hold 64K KV cache alongside a 20GB MoE instead of capping at 8-16K.

Can one RTX PRO 6000 Blackwell run gpt-oss 120B?

Yes. gpt-oss 120B at MXFP4 needs about 65GB, so it fits in the card's 96GB ECC GDDR7 with room for a full 128K context — roughly 51 tok/s in typical single-stream llama.cpp runs, and far higher under batched serving on optimized builds. Other 96GB picks: Qwen3-Next 80B-A3B at Q4_K_M (~50GB, ~83 tok/s), Qwen 3.5 122B-A10B at Q4_K_M (~72GB, ~41 tok/s), and Qwen 3.6 35B-A3B at Q8_0 (~39GB, ~73 tok/s) when you want near-FP16 quality on a mid-size model.

← All guides

Hardware August 5, 2026

Best Models to Run on Popular RTX GPUs (August 2026): 3090, 4090, 5090 & RTX PRO 6000

Four RTX cards cover almost every local-LLM build in August 2026: the used 3090 at 24GB, the 4090 at 24GB, the 5090 at 32GB, and the RTX PRO 6000 Blackwell at 96GB. The model that wins on each is different, and the reason is always the same — bandwidth sets your speed, VRAM sets your ceiling. On 24GB cards a small-active MoE like Gemma 4 26B-A4B beats every dense model. On the 5090, Qwen 3.6 35B-A3B hits ~118 tok/s. On 96GB, gpt-oss 120B finally fits in one card at full context.

Building an RTX rig for OpenClaw?

See our AI training options. We'll pick the model, quant, and context window for the exact card you own — and wire it into OpenClaw.

🎮 THE FOUR CARDS IN THIS GUIDE

24 GB gets you 27B-35B MoE models. 32 GB adds real context headroom. 96 GB runs 120B in one card.

3090EVGA RTX 3090 24 GB ↗ 4090GIGABYTE RTX 4090 24 GB ↗ 5090GIGABYTE RTX 5090 32 GB ↗ 96GBRTX PRO 6000 Blackwell 96 GB ↗

Bottom Line

RTX 3090 (24GB, 936 GB/s) — Gemma 4 26B-A4B Q4_K_M, ~16GB, ~71 tok/s. Best value in local AI, full stop.
RTX 4090 (24GB, 1008 GB/s) — Gemma 4 26B-A4B Q4_K_M at ~85 tok/s, or Laguna XS 2.1 Q4_K_M at ~86 tok/s for agentic coding.
RTX 5090 (32GB, 1792 GB/s) — Qwen 3.6 35B-A3B Q4_K_M, ~22GB, ~118 tok/s. The 8GB over a 4090 buys context, not just speed.
RTX PRO 6000 Blackwell (96GB, 1792 GB/s) — gpt-oss 120B MXFP4, ~65GB, ~51 tok/s at full 128K context. One card, frontier-class open weights.

One rule explains all four rows: MoE models with small active-parameter counts win on every RTX card. A 26B model with 4B active decodes at 70-85 tok/s. A dense 27B on the same card decodes at 31-37. Same VRAM, half the speed.

RTX 3090 — 24GB, the value floor

The used 3090 is still the cheapest card that runs a 32B-class model entirely in VRAM with no CPU offload. Its 936 GB/s is the slowest of the four, which is exactly why MoE matters most here.

Model	Quant	VRAM	Speed	Why
Gemma 4 26B-A4B	Q4_K_M	~16 GB	~71 tok/s	Best generalist; 4B active keeps it fast on slow bandwidth
gpt-oss 20B	MXFP4	~13.8 GB	~73 tok/s	Cleanest tool calls — the OpenClaw agent pick
Qwen 3.5 35B-A3B	Q4_K_M	~20 GB	~71 tok/s	Bigger total params, same speed class
LFM2 24B-A2B	Q4_K_M	~14 GB	~98 tok/s	Fastest option; 2B active, lighter reasoning
Qwen 3.6 27B (dense)	Q4_K_M	~18 GB	~31 tok/s	Strong quality, but you feel the 936 GB/s

Our pick: Gemma 4 26B-A4B at Q4_K_M. It leaves ~8GB for KV cache, so 32K context is comfortable. Run gpt-oss 20B alongside it as your agent driver.

EVGA RTX 3090 24GB on Amazon →

RTX 4090 — 24GB, the same ceiling, 20% more speed

Identical VRAM to the 3090, 1008 GB/s instead of 936, and much better prompt-processing throughput. Every model above runs here, faster.

Model	Quant	VRAM	Speed	Why
Gemma 4 26B-A4B	Q4_K_M	~16 GB	~85 tok/s	Best all-round pick at 24GB
Laguna XS 2.1 (33B/3B MoE)	Q4_K_M	~20.3 GB	~86 tok/s	Agentic coding; 70.9% SWE-Bench Verified
gpt-oss 20B	MXFP4	~13.8 GB	~87 tok/s	Production OpenClaw agent loops
Qwen 3.5 35B-A3B	Q4_K_M	~20 GB	~84 tok/s	Broadest knowledge in the MoE class
Qwen 3.6 27B (dense)	Q4_K_M	~18 GB	~37 tok/s	Quality per GB, if you can accept the speed

Our pick depends on the job. Chat and general work: Gemma 4 26B-A4B. Coding agents: Laguna XS 2.1. The catch on 24GB is context — Laguna at 20.3GB leaves under 4GB for KV cache, so cap it around 8-16K. That single limit is the argument for a 5090.

GIGABYTE RTX 4090 24GB on Amazon →

RTX 5090 — 32GB, the context card

The 5090 pairs 32GB of GDDR7 with 1,792 GB/s, 77% more bandwidth than a 4090. Speeds roughly double on MoE models, and the extra 8GB turns 16K-context builds into 64K-context builds.

Model	Quant	VRAM	Speed	Why
Qwen 3.6 35B-A3B	Q4_K_M	~22 GB	~118 tok/s	Best overall; fastest strong model on the card
Laguna XS 2.1 (33B/3B MoE)	Q4_K_M	~20.3 GB	~121 tok/s	Coding agent with ~64K context headroom
Gemma 4 26B-A4B	Q4_K_M	~16 GB	~118 tok/s	Leaves 16GB for very long context
Qwen 3.6 27B (dense)	Q4_K_M	~18 GB	~52 tok/s	Dense quality at usable speed
Gemma 4 31B (dense)	Q4_K_M	~20 GB	~46 tok/s	Largest dense Gemma that fits well

Our pick: Qwen 3.6 35B-A3B at Q4_K_M. At ~118 tok/s it reads faster than most cloud APIs feel, and the 10GB of slack holds a large KV cache. Buy the 5090 for the ceiling and the context, not the raw tok/s number.

GIGABYTE RTX 5090 32GB on Amazon →

RTX PRO 6000 Blackwell — 96GB, the 120B threshold

Three times the 5090’s VRAM at the same 1,792 GB/s, with ECC GDDR7 and native NVFP4 support. This is the first single card that holds a 120B-class model plus a full 128K context.

Model	Quant	VRAM	Speed	Why
gpt-oss 120B	MXFP4	~65 GB	~51 tok/s	Frontier-class open weights, full 128K context, one card
Qwen3-Next 80B-A3B	Q4_K_M	~50 GB	~83 tok/s	Best speed-per-quality on the card
Qwen 3.5 122B-A10B	Q4_K_M	~72 GB	~41 tok/s	Broadest knowledge that fits
Llama 4 Scout (109B)	Q4_K_M	~67 GB	~35 tok/s	Long-context Llama option
Qwen 3.6 35B-A3B	Q8_0	~39 GB	~73 tok/s	Near-FP16 quality, huge context budget

Our pick: gpt-oss 120B at MXFP4. It is the reason to buy this card — 65GB of weights leaves 30GB for context and concurrency, and MXFP4 was co-designed for Blackwell’s 4-bit tensor cores. Move to NVFP4 builds where your serving stack supports them; quality tracks FP8 at half the memory.

The honest caveat: street price sits near $13,000, well above the $8,565 launch MSRP. Two 5090s cost far less and give you 64GB — but split across cards, which rules out single-model 120B inference at full context.

RTX PRO 6000 Blackwell 96GB on Amazon →

Which Card Should You Buy?

Under $1,000 — used RTX 3090. Runs 90% of what a 4090 runs, 20% slower.
You already own a 4090 — keep it. The upgrade buys context, not capability.
$2,000-2,500 — RTX 5090. The 32GB ceiling is the real product.
You need 120B in one card — RTX PRO 6000 Blackwell, and nothing else in the RTX line.

OpenClaw Setup (any card)

ollama pull gemma4:26b-a4b-q4_K_M      # 3090 / 4090
ollama pull qwen3.6:35b-a3b-q4_K_M     # 5090
ollama pull gpt-oss:120b               # RTX PRO 6000
openclaw config set agents.defaults.models.chat ollama/gemma4:26b-a4b-q4_K_M
openclaw config set agents.defaults.models.agent ollama/gpt-oss:20b
openclaw config set agents.defaults.keep_alive 30m

Keep gpt-oss 20B as the agent model on 24GB and 32GB cards. Its tool-call JSON is the most reliable of any open-weight model, and reliability beats raw benchmark score inside an agent loop.