Best Local LLM for 128GB of VRAM

Bottom Line (August 2026)

• No single card has 128GB of VRAM. Real 128GB is four 32GB cards — about $17,000-20,000 and 2,300W of GPU power — or a 96GB RTX PRO 6000 plus a 32GB card.
• 128GB of unified memory costs $4,699. A DGX Spark holds the same weights. The ASUS Ascent GX10 listed $3,999 in August 2026 but $6,999 on the ASUS US store in September (every GB10 box price).
• The model that defines this tier is Qwen3-235B-A22B. IQ4_XS is 125GB; Q4_K_S is 134GB. 128GB sits exactly on that line.
• Here is the part nobody says: everything at 128GB is mixture-of-experts, and MoE is the architecture that forgives slow memory. The cheap box loses far less here than it does at 70B.
• For one person running one model, buy the unified box. For serving many users or fine-tuning, buy the cards.

Ready to buy? See the tested hardware list with current prices.

If you meant 128GB of system RAM, that is a different machine — see best local LLMs for 128GB RAM.

The Trap in the Query

“128GB VRAM” gets typed as if it were a tier you can shop for. It is not a product. Here is what it actually costs to reach, at August 2026 US street prices:

The spread between the top row and the bottom is more than 4x, for the same number on the box. That alone is worth pausing on before spending.

And the pool count matters as much as the total. Four 32GB pools are not one 128GB pool. A 125GB model split across four cards moves activations over PCIe at every layer boundary, which is exactly the setup our multi-GPU motherboard guide found most consumer boards cannot feed — mainstream B650 boards have a single CPU-connected x16 slot, and even a creator-class X870E board drops its second slot to x4 when you populate the second M.2.

What Actually Fits at 128GB

Published GGUF sizes. Weights only.

Note what that table is made of. Every model at this tier except the 70B is mixture-of-experts: 235B total with 22B active, 400B total with 17B active, 117B total with a small active set. That is not a coincidence. Dense models stopped scaling into this size class for local use, and it changes the hardware answer completely.

Why MoE Rescues the Cheap Box

Here is the arithmetic that decides this page, derived rather than benchmarked.

Token generation is memory-bandwidth-bound. For each token, the runtime reads the weights it needs out of memory. On a dense model that means the whole file. On a mixture-of-experts model it means the shared layers plus the handful of experts the router selected.

Take Qwen3-235B-A22B at IQ4_XS — 125GB on disk, 22B of 235B parameters active per token. Roughly 9.4% of the weights move per token, so on the order of 12GB read per token rather than 125GB.

Now put that against the two machines:

Those are theoretical upper bounds from bandwidth divided by bytes-per-token, not measurements. Real throughput lands below them — routing overhead, attention, the KV cache and cross-card traffic all take a cut, and the MoE figure is an approximation because shared attention weights are read every token regardless of routing.

But the shape is what matters, and the shape is solid. The same slow box that is nearly unusable on a dense 70B is perfectly usable on a 235B MoE. Six times the effective speed, from the model architecture, on identical hardware.

This inverts the advice we give everywhere else on this site. At 24GB and 48GB, VRAM bandwidth is the thing you are buying and unified memory is the compromise — see Strix Halo vs Mac Studio and dual 3090 vs 5090. At 128GB the models flipped to sparse, and the compromise mostly stopped costing you anything.

What to Buy

Best value at this tier — ASUS Ascent GX10

128GB of unified memory, GB10 Grace Blackwell, CUDA out of the box, $3,999 for the 1TB SKU in August 2026, but the ASUS US store listed it at $6,999 in September 2026 (every GB10 box price). It no longer undercuts the $4,699 DGX Spark, which has the same GB10 silicon and the same 273 GB/s. For running a 235B MoE at home, this is the recommendation.

128GBASUS Ascent GX10 128 GB ↗

The NVIDIA-branded version — DGX Spark

Same 128GB, same 273 GB/s, $4,699 after NVIDIA’s own memory-driven price increase from $3,999. You are paying the difference for NVIDIA’s software bundle and support path, which is a real thing to want if your deployment target is DGX-class infrastructure. We priced the whole decision in is the DGX Spark worth it.

128GBNVIDIA DGX Spark 128 GB ↗

Cheapest 128GB unified — a Ryzen AI Max+ 395 box

The AMD route to 128GB, and still the cheapest one when in stock. In September 2026 the cheapest in-stock 128GB box is $3,499.99 against the $4,699 DGX Spark, so you save about $1,200. The memory shortage closed the old half-price gap. The tradeoffs are software rather than memory: ROCm rather than CUDA, and genuinely volatile availability — we found every advertised SKU sold out or repriced when we checked the vendors directly.

128GBRyzen AI Max+ 395 128 GB Box ↗

If you genuinely need real VRAM — start at 96GB on one card

Before you buy four cards, check whether one will do. The RTX PRO 6000 Blackwell holds 96GB in a single coherent pool at 1,792 GB/s, which covers everything in the table above except Qwen3-235B at 4-bit. One card, one pool, 600W — or 300W for the Max-Q variant. We wrote up that tier in best local LLM for 96GB of VRAM.

96GBRTX PRO 6000 Blackwell 96 GB ↗

The four-card build — and who it is actually for

Four RTX 5090s is the right answer for exactly two workloads: serving many concurrent users, where four independent compute units beat one large memory pool, and fine-tuning, where you need FP16/BF16 throughput and optimizer state rather than just capacity. We separated those cases in buying a GPU for fine-tuning, not just inference.

Budget for the rest of the build honestly: a board with real CPU-connected lanes, 1,600W or more of PSU for 2,300W of transient-prone GPUs across two units, and a room that can take the heat. Our PSU guide covers why sustained inference trips supplies that survive gaming fine.

32GBGIGABYTE RTX 5090 WINDFORCE 32 GB ↗

The Honest Caveat

Two things this page is not claiming.

The MoE argument does not make the cheap box fast. It makes it usable. About 23 tokens per second as a derived ceiling on a 235B is a good local experience for chat and a frustrating one for an agent that runs for an hour — and prefill, which reads far more of the model, stays slow no matter how the router behaves. Long prompts hurt on a 273 GB/s machine in a way the generation numbers hide.

And the model list at 128GB is thinner than the marketing suggests. Exactly one open-weight model really justifies this tier right now, and it sits 3GB inside your budget at IQ4_XS and 6GB outside it at Q4_K_S. If Qwen’s next release lands at 140GB, a $4,000 box drops a tier overnight and an $18,000 card stack does too. Sizing a machine to one model is the risk nobody prices, and it applies to both routes equally.

If you can be happy at 96GB, you are on much more stable ground.

Before you order parts, check the tested hardware list for current prices by tier.

See Also

• Can I run Qwen3.8-Flash-Next locally? — 6B active, but even the 1-bit GGUF needs a 128GB box
• Best Local LLM for 96GB of VRAM — one card, one pool, the tier below
• Best Local LLMs for 128GB RAM — the system-memory version of this question
• Is the DGX Spark Worth It? — the $4,699 box, priced honestly
• Which Strix Halo Mini PC to Buy — the cheapest 128GB unified route
• Motherboard and CPU for a Multi-GPU Rig — what four cards actually needs
• Buying a GPU for Fine-Tuning — the case where the card stack wins
• What Hardware for 1M Context Locally — when the KV cache is the constraint