How Much VRAM Does gpt-oss 120B Need at Full 128K Context? About 4.5 GiB of Cache, Not 9

Bottom Line

• The full 131,072-token window costs about 4.5 GiB of KV cache at fp16. Not 9 GiB, which is what the naive calculation gives.
• The reason is sliding-window attention. 18 of the 36 layers only ever attend to 128 tokens and cost about 4.5 MiB in total.
• The weights dominate. Hugging Face safetensors shards total 61.48 GB; the Ollama gpt-oss:120b tag is listed at 65GB, MXFP4, 117B parameters.
• Budget roughly 70GB for the whole job at full context, then add runtime headroom on top.
• 96GB VRAM fits it with about 25GB spare. A 128GB unified box fits it comfortably. 64GB does not fit at all — the weights alone exceed it.
• A 4-bit cache saves you almost nothing here. Halving 4.5 GiB frees 2.25 GiB on a ~70GB job. Spend the quality elsewhere.
• Verify what your runtime allocated. A runtime that ignores the sliding window reserves 9.0 GiB instead, and that gap is bigger than any cache-quantisation setting you can change.

The Arithmetic, From the Model’s Own Config

Every number below comes from the config.json published with openai/gpt-oss-120b, read on 25 August 2026. Nothing here is from memory.

The KV cache holds one key and one value per key-value head, per layer, per token. So the per-layer, per-token cost at fp16 is:

2 (K and V) x 8 (kv heads) x 64 (head_dim) x 2 bytes = 2,048 bytes = 2 KiB

Now split the layers, because they do not behave the same way.

The sliding half is not a rounding error you should keep track of. It is a thousand times smaller than the other half. At the full window it is 0.1% of the cache.

Compare that with the number you get if you assume all 36 layers cache the whole window:

36 x 131,072 x 2,048 = 9,663,676,416 B = 9.0 GiB

Exactly double. That 9.0 GiB figure is what a generic VRAM calculator will hand you, and it is what most sizing pages quote for a model of this shape.

What the Whole Job Costs

We are quoting the two weight figures separately and not averaging them. They are different distributions of the same model and they are measured differently. Use the 65GB Ollama figure if you pull it with Ollama, and the 61.48GB figure if you pull the safetensors.

Which Machines Hold It

Capacity is not speed. All three hold the model. The 96GB card runs it several times faster than either 128GB box, because memory bandwidth governs tokens per second. Choose on which side of that trade you sit, and see is 96GB of VRAM enough in 2026 for the wider tier argument.

The Thing Nobody Else Says About This Model

The usual rule is that a bigger model buys you less context, because the cache scales with layer count and hidden size. gpt-oss 120B breaks that rule. A 117B-parameter model here carries a full 128K window for 4.5 GiB, which is less than many 30B dense models charge for the same window.

The tradeoff is real and worth naming, because the config makes it explicit. A 128-token sliding window is very short. In half of this model’s layers, a token can attend no further back than 127 positions. Long-range recall in those layers is gone by construction. The model compensates through the 18 full-attention layers, and the design is a deliberate exchange of long-range attention breadth for cache size.

So the honest framing is not “this model is more efficient.” It is: this model spends its long-range attention budget in half its layers and charges you half the cache. If your work depends on precise recall of something 100,000 tokens back, test that specifically rather than trusting the window number. Our context window traps for local agents page covers how that failure looks in practice.

Check What Your Runtime Actually Reserved

The 4.5 GiB figure assumes the runtime implements the short cache for the sliding layers. Not every configuration does. If your loader reports about 9 GiB of KV cache for this model at 128K, it is allocating a full-length cache for all 36 layers, and you are paying double for memory you cannot use.

That is a larger saving than any cache-quantisation option. Fix the allocation before you reach for KV cache quantisation, which on this model saves only about 2.25 GiB on a 70GB job and costs quality to get it.

See Also

• DeepSeek V4 Flash vs gpt-oss 120B — the head-to-head on the same memory tier
• Best Local LLM for 96GB of VRAM — the tier picks, with this model in them
• Best Models for the NVIDIA DGX Spark — the 128GB unified view
• How Much VRAM for 128K Context? — the general version of this arithmetic
• KV Cache Quantization: Q8 vs Q4 — what halving the cache costs you
• Is 96GB of VRAM Enough in 2026? — the tier question this model sits inside
• Why Is My Local LLM So Slow? — why fitting and running fast are different problems

Sources

• openai/gpt-oss-120b config.json (Hugging Face) — all layer, head, window and quantisation fields, read 2026-08-25
• openai/gpt-oss-120b file listing (Hugging Face) — 15 safetensors shards totalling 61.48 GB as displayed, read 2026-08-25
• gpt-oss:120b (Ollama library) — 65GB download, MXFP4, 117B parameters, read 2026-08-25

Every cache figure on this page is arithmetic derived from the published config, not a measured allocation. Runtimes add their own buffers and some ignore the sliding window entirely. Check your own loader’s reported cache size against these numbers.