What Hardware Do You Need for 1M Context Locally? (August 2026)

Bottom Line

• 1M context is a KV cache problem. The cache, not the weights, sets the hardware requirement.
• For Qwen3.6 27B, full 1M context costs roughly 69GB at fp16 or 34GB at 8-bit KV, on top of 18GB of Q4 weights. (Derived below from the published config — the method is shown so you can check it.)
• No consumer GPU reaches 1M. A 24GB RTX 4090 gets to roughly 96K tokens at fp16; a 32GB RTX 5090 to roughly 220K.
• The realistic 1M tiers are a 96GB workstation card, or a 128GB unified-memory box, with an 8-bit KV cache doing most of the work.
• 1M is not native. Qwen3.6’s config sets 262,144 as the trained context; 1M is YaRN extrapolation on top.
• The reason this is possible at all is hybrid attention — only 16 of 64 layers accumulate a KV cache. All-full-attention would need ~275GB.

Everyone Sizes This Wrong

The standard advice for long context is to buy more VRAM until the model fits. That advice is answering the wrong question, because at long context the model is the small part.

Model weights are fixed. A 27B model at Q4 is 18GB whether you run it at 4K context or 1M. The KV cache is what grows, linearly, with every token you put in the window. Past a few hundred thousand tokens it dominates the memory budget completely.

So “what hardware do I need for 1M context” has almost nothing to do with parameter count. It is a question about cache.

The Actual Math, Shown

Let us derive it rather than quote it, because the numbers floating around for this are inconsistent and most of them are too high.

Qwen3.6 27B’s published config.json gives:

That last row is the one that changes everything. The 4-layer cycle repeats 16 times across 64 layers, so only 16 layers are full-attention layers. Linear-attention layers carry fixed-size state rather than a cache that grows per token, so they contribute effectively nothing to this calculation.

Per token, per full-attention layer, at fp16:

K + V  ×  kv_heads  ×  head_dim  ×  bytes
  2    ×      4     ×    256     ×    2     =  4,096 bytes  (4 KiB)

Across the 16 full-attention layers:

4 KiB × 16 layers = 64 KiB per token

At 1,048,576 tokens:

64 KiB × 1,048,576 = 64 GiB  ≈  69 GB  (fp16)

With an 8-bit KV cache, halve it: ~34GB. At the native 262,144 context instead of 1M, it is ~17GB at fp16 or ~9GB at 8-bit.

Two caveats on this figure, stated plainly. It is a derived architectural lower bound, not a measured allocation — real runtimes add padding, block-allocation overhead, and per-sequence bookkeeping, so budget 10-20% above it. And it is specific to this model’s architecture; do not carry the 64 KiB-per-token number over to a different model without redoing the arithmetic.

Why this is not the number you usually see

Guides commonly quote well over 200GB for 1M context, or vaguely say “48GB+.” The high figures come from assuming all 64 layers are full attention. That assumption would give:

4 KiB × 64 layers × 1,048,576 = 256 GiB ≈ 275 GB

That is roughly 4x too high for this model, and it is why long local context looks impossible in most write-ups. Hybrid attention is the entire reason a million-token local context is on the table. It is an architecture win. No amount of buying hardware would have gotten a dense all-attention 27B there on a desk.

What Each Tier Actually Reaches

Using 64 KiB/token at fp16 and 32 KiB/token at 8-bit KV, with an 18GB Q4 27B model resident:

The honest reading of that table: 48GB gets you to 1M with an 8-bit KV cache, and nothing below 48GB gets close. The 8-bit cache is doing as much work as the hardware upgrade is — switching from fp16 to 8-bit KV roughly doubles every number in the table for a quality cost most people cannot detect at long context.

If your runtime supports it, set the cache type before you buy anything.

The used RTX A6000 48GB is the cheapest single card that reaches 1M with an 8-bit cache, at roughly $2,600-3,800 in August 2026. For a new card with a warranty, the RTX PRO 5000 Blackwell 48GB is $5,600-6,250 — confirm the listing is the 48GB variant, since a 72GB version also exists. Our 48GB VRAM routes guide compares all four ways to reach this tier.

The Unified-Memory Route Is Better Here Than Usual

Long context is the one workload where the big unified-memory boxes look genuinely strong.

Normally we warn against them for dense models, because ~256 GB/s of bandwidth decodes a dense 70B at about 5 tokens per second. That criticism still stands. But KV cache is capacity-hungry rather than bandwidth-hungry per token, and a 128GB box has capacity that no consumer GPU can match at any price.

A 128GB Ryzen AI Max+ 395 box or an ASUS Ascent GX10 at $3,999 holds a 27B model plus a full 1M-token fp16 cache with room left over. Neither will be fast, but “slow and possible” beats “fast and out of memory.”

Before You Buy Anything: You Probably Do Not Need This

The uncomfortable part of this article.

1M context is not native. Qwen3.6 is trained to 262,144 tokens; the 1M figure comes from YaRN rope scaling applied on top. Extrapolated context degrades — the model attends less reliably over the extrapolated range than the trained one. You are paying full cache cost for tokens the model handles worst.

Prefill is the hidden tax. Memory determines whether a 1M-token prompt fits. Compute determines how long you wait before the first token. Filling a million-token window means processing a million tokens of prompt, and on the same low-bandwidth boxes that make the capacity affordable, that wait is measured in many minutes.

And most long-context failures happen far below 1M. In our experience the common breakage is agents dying around 10 prompts in, because the default context window silently truncates tool schemas — a configuration bug, not a capacity limit. We wrote up those failure modes in context window traps for local agents.

The order of operations that saves money: fix your context configuration, then enable an 8-bit KV cache, then adopt retrieval so you are not stuffing a million tokens in the first place, and only then buy memory. Most people stop being blocked at step one.

See Also

• Context Window Traps for Local Agents — why agents break at 10 prompts, and the per-runtime fixes
• How Much Context Fits in 128GB RAM? — the full memory budget for a 128GB machine
• Four Routes to 48GB of VRAM — the tier that reaches 1M with an 8-bit cache
• Which Strix Halo Mini PC Should You Buy? — the 128GB unified-memory option
• Mac mini M4 Pro 48GB vs 64GB — why Apple’s memory ceiling dropped in 2026