What is the best local LLM for 128GB of VRAM?

Qwen3-235B-A22B is the model this tier exists for. Its IQ4_XS build is 125GB and its Q4_K_S build is 134GB, so 128GB is almost exactly the line between running it at 4-bit and not. Below that, Llama 4 Maverick at Q4 lands around 95-100GB and gpt-oss 120B at MXFP4 is 63.4GB with enormous context headroom. 128GB is the first tier where a frontier-class open-weight mixture-of-experts model fits at a quantization worth using.

How do you actually get 128GB of VRAM?

Four 32GB cards, such as four RTX 5090s, or an RTX PRO 6000 Blackwell 96GB paired with a 32GB card. At August 2026 street prices four 5090s run roughly $17,000-20,000 and draw 2,300W of GPU power alone, and the memory is still four separate pools rather than one. There is no single consumer-reachable card with 128GB of VRAM.

Is 128GB of unified memory as good as 128GB of VRAM?

At this tier it is much closer than at any tier below. A DGX Spark or ASUS Ascent GX10 has 128GB at 273 GB/s against a stack of 5090s at 1,792 GB/s each. That gap looks fatal until you notice that the models which need 128GB are mixture-of-experts designs. Qwen3-235B-A22B activates 22B of its 235B parameters per token, so each token reads a small fraction of the weights, and the bandwidth penalty applies to that fraction rather than to the whole 125GB file.

Is four RTX 5090s worth it for local LLMs?

Only for specific workloads. It is the fastest way to serve many concurrent users or to fine-tune, because you get four full compute units and 7,168 GB/s of aggregate bandwidth. For one person running one model it is poor value: about $18,000, 2,300W, a motherboard with enough CPU-connected PCIe lanes, and a model still sharded across four pools with cross-card traffic on every token.

← All guides

Hardware August 19, 2026

Best Local LLM for 128GB of VRAM (August 2026)

There is a trap inside this question. 128GB of actual VRAM is four 32GB cards, roughly $18,000 and 2,300W of GPU power. 128GB of unified memory is a box that costs $3,999. And at this specific tier — unlike every tier below it — the cheap box is usually the right call, for a reason that comes out of the models rather than the hardware.

Sizing a serious local AI machine?

See our AI training options. We will tell you which of these three routes fits your workload before you spend.

Bottom Line (August 2026)

No single card has 128GB of VRAM. Real 128GB is four 32GB cards — about $17,000-20,000 and 2,300W of GPU power — or a 96GB RTX PRO 6000 plus a 32GB card.
128GB of unified memory costs $3,999. An ASUS Ascent GX10 or a DGX Spark at $4,699 holds the same weights.
The model that defines this tier is Qwen3-235B-A22B. IQ4_XS is 125GB; Q4_K_S is 134GB. 128GB sits exactly on that line.
Here is the part nobody says: everything at 128GB is mixture-of-experts, and MoE is the architecture that forgives slow memory. The cheap box loses far less here than it does at 70B.
For one person running one model, buy the unified box. For serving many users or fine-tuning, buy the cards.

If you meant 128GB of system RAM, that is a different machine — see best local LLMs for 128GB RAM.

The Trap in the Query

“128GB VRAM” gets typed as if it were a tier you can shop for. It is not a product. Here is what it actually costs to reach, at August 2026 US street prices:

Route	Total memory	Rough cost	GPU power	Memory pools
4x RTX 5090 32GB	128GB	$17,000-20,000	2,300W	4
RTX PRO 6000 96GB + RTX 5090	128GB	~$17,500	1,175W	2
5x used RTX 3090 24GB	120GB	$5,000-6,500	1,750W	5
DGX Spark	128GB unified	$4,699	~240W	1
ASUS Ascent GX10	128GB unified	$3,999	~240W	1
Ryzen AI Max+ 395 box	128GB unified	~$2,000-4,300	~140W	1

The spread between the top row and the bottom is more than 4x, for the same number on the box. That alone is worth pausing on before spending.

And the pool count matters as much as the total. Four 32GB pools are not one 128GB pool. A 125GB model split across four cards moves activations over PCIe at every layer boundary, which is exactly the setup our multi-GPU motherboard guide found most consumer boards cannot feed — mainstream B650 boards have a single CPU-connected x16 slot, and even a creator-class X870E board drops its second slot to x4 when you populate the second M.2.

What Actually Fits at 128GB

Published GGUF sizes. Weights only.

Model	Quant	Size	128GB verdict
Qwen3-235B-A22B	UD-Q3_K_XL	104 GB	Fits with context room
Qwen3-235B-A22B	IQ4_XS	125 GB	Just fits — the tier-defining model
Qwen3-235B-A22B	Q4_K_S / UD-Q4_K_XL	134 GB	No — 6GB over
Qwen3-235B-A22B	Q4_K_M	142 GB	No
Llama 4 Maverick (400B/17B)	Q4	~95-100 GB	Fits, tight
gpt-oss 120B	MXFP4 (native)	63.4 GB	Trivial, huge context budget
Llama 3.3 70B	Q8_0	75 GB	Trivial
Qwen3-235B-A22B	Q8_0	250 GB	No

Note what that table is made of. Every model at this tier except the 70B is mixture-of-experts: 235B total with 22B active, 400B total with 17B active, 117B total with a small active set. That is not a coincidence. Dense models stopped scaling into this size class for local use, and it changes the hardware answer completely.

Why MoE Rescues the Cheap Box

Here is the arithmetic that decides this page, derived rather than benchmarked.

Token generation is memory-bandwidth-bound. For each token, the runtime reads the weights it needs out of memory. On a dense model that means the whole file. On a mixture-of-experts model it means the shared layers plus the handful of experts the router selected.

Take Qwen3-235B-A22B at IQ4_XS — 125GB on disk, 22B of 235B parameters active per token. Roughly 9.4% of the weights move per token, so on the order of 12GB read per token rather than 125GB.

Now put that against the two machines:

Machine	Bandwidth	Dense 70B Q8 (75GB/token)	Qwen3-235B-A22B (~12GB/token)
DGX Spark / Ascent GX10	273 GB/s	~3.6 tok/s	~23 tok/s
RTX 5090 (single)	1,792 GB/s	model does not fit	model does not fit
4x RTX 5090 (aggregate)	7,168 GB/s	~95 tok/s	~150+ tok/s (sharding losses apply)

Those are theoretical upper bounds from bandwidth divided by bytes-per-token, not measurements. Real throughput lands below them — routing overhead, attention, the KV cache and cross-card traffic all take a cut, and the MoE figure is an approximation because shared attention weights are read every token regardless of routing.

But the shape is what matters, and the shape is solid. The same slow box that is nearly unusable on a dense 70B is perfectly usable on a 235B MoE. Six times the effective speed, from the model architecture, on identical hardware.

This inverts the advice we give everywhere else on this site. At 24GB and 48GB, VRAM bandwidth is the thing you are buying and unified memory is the compromise — see Strix Halo vs Mac Studio and dual 3090 vs 5090. At 128GB the models flipped to sparse, and the compromise mostly stopped costing you anything.

What to Buy

Best value at this tier — ASUS Ascent GX10

128GB of unified memory, GB10 Grace Blackwell, CUDA out of the box, $3,999 for the 1TB SKU as of August 2026. It undercuts the DGX Spark by $700 for the same GB10 silicon and the same 273 GB/s. For running a 235B MoE at home, this is the recommendation.

128GBASUS Ascent GX10 128 GB ↗

The NVIDIA-branded version — DGX Spark

Same 128GB, same 273 GB/s, $4,699 after NVIDIA’s own memory-driven price increase from $3,999. You are paying the difference for NVIDIA’s software bundle and support path, which is a real thing to want if your deployment target is DGX-class infrastructure. We priced the whole decision in is the DGX Spark worth it.

128GBNVIDIA DGX Spark 128 GB ↗

Cheapest 128GB unified — a Ryzen AI Max+ 395 box

The AMD route to 128GB, at roughly half the price of the GB10 boxes when in stock. The tradeoffs are software rather than memory: ROCm rather than CUDA, and genuinely volatile availability — we found every advertised SKU sold out or repriced when we checked the vendors directly.

128GBRyzen AI Max+ 395 128 GB Box ↗

If you genuinely need real VRAM — start at 96GB on one card

Before you buy four cards, check whether one will do. The RTX PRO 6000 Blackwell holds 96GB in a single coherent pool at 1,792 GB/s, which covers everything in the table above except Qwen3-235B at 4-bit. One card, one pool, 600W — or 300W for the Max-Q. We wrote up that tier in best local LLM for 96GB of VRAM.

96GBRTX PRO 6000 Blackwell 96 GB ↗

The four-card build — and who it is actually for

Four RTX 5090s is the right answer for exactly two workloads: serving many concurrent users, where four independent compute units beat one large memory pool, and fine-tuning, where you need FP16/BF16 throughput and optimizer state rather than just capacity. We separated those cases in buying a GPU for fine-tuning, not just inference.

Budget for the rest of the build honestly: a board with real CPU-connected lanes, 1,600W or more of PSU for 2,300W of transient-prone GPUs across two units, and a room that can take the heat. Our PSU guide covers why sustained inference trips supplies that survive gaming fine.

32GBGIGABYTE RTX 5090 WINDFORCE 32 GB ↗

The Honest Caveat

Two things this page is not claiming.

The MoE argument does not make the cheap box fast. It makes it usable. About 23 tokens per second as a derived ceiling on a 235B is a good local experience for chat and a frustrating one for an agent that runs for an hour — and prefill, which reads far more of the model, stays slow no matter how the router behaves. Long prompts hurt on a 273 GB/s machine in a way the generation numbers hide.

And the model list at 128GB is thinner than the marketing suggests. Exactly one open-weight model really justifies this tier right now, and it sits 3GB inside your budget at IQ4_XS and 6GB outside it at Q4_K_S. If Qwen’s next release lands at 140GB, a $4,000 box drops a tier overnight and an $18,000 card stack does too. Sizing a machine to one model is the risk nobody prices, and it applies to both routes equally.

If you can be happy at 96GB, you are on much more stable ground.