Qwen3 128K Context: YaRN Is Off by Default

Bottom Line

• Qwen3 32B and Qwen3 30B-A3B are 32K models. The model card says “32,768 natively and 131,072 tokens with YaRN.”
• YaRN is off unless you turn it on. Both config files ship "rope_scaling": null.
• Past 32K without YaRN, nothing errors. llama.cpp prints one warning line. Quality drops quietly, and it looks like “the model is dumb.”
• Do not leave YaRN on all the time. Qwen warns that static YaRN can hurt short-text quality.
• Qwen3.6 27B and Qwen3.8 27B are different. They are 262,144 tokens natively. The Qwen3 YaRN advice does not apply to them below 262K.
• Ollama may give you far less than any of this. Its default context depends on VRAM: 4K under 24 GiB.

Check Your Model in Thirty Seconds

Open the model’s config.json on Hugging Face and read two fields.

Three patterns are hiding in that table:

1. Scaling in the config (gpt-oss). The model ships with YaRN on. Runtimes read it. You do nothing.
2. Scaling in the model card only (Qwen3). The long-context number is real, but you must add it yourself.
3. Long native context (Qwen3.6, Qwen3.8). No scaling is needed until you go past 262K.

The trap is pattern 2, and it is the pattern most “128K local model” guides were written about.

One detail worth knowing: the Qwen3 card says 32,768 native, but the config says 40,960. The loader reads the config. Treat 32,768 as the trained length and the config number as the loader’s ceiling.

What Goes Wrong Without YaRN

Two failures, and neither one stops the model:

• The runtime caps you. It reads 40,960 from the config and never lets the context grow past it. You think you have 128K. You have 40K, and your long document is cut.
• The runtime lets you overrun. You set a large context by hand. The model reads positions it never saw in training. Recall and reasoning over the far end of the prompt get worse, with no error.

The second one is why this gets misdiagnosed. The model still answers. It answers badly about page 90 of a 100-page document, and the user decides Qwen3 is weak at long context. It is not. It was never told to scale.

How to Turn YaRN On, per Runtime

These are the exact settings from the Qwen3-32B model card.

llama.cpp (llama-server or llama-cli):

llama-server -m Qwen3-32B-Q4_K_M.gguf -c 131072 \
  --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768

vLLM:

vllm serve Qwen/Qwen3-32B --max-model-len 131072 \
  --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}'

Transformers or any config-reading loader — add this to config.json:

"rope_scaling": {
  "rope_type": "yarn",
  "factor": 4.0,
  "original_max_position_embeddings": 32768
}

Ollama — raise the context first, because the default is small. Ollama’s docs set it by VRAM: 4K under 24 GiB, 32K from 24 to 48 GiB, 256K at 48 GiB and up. Set it with OLLAMA_CONTEXT_LENGTH=131072 ollama serve, and confirm with ollama ps. We could not verify a documented Ollama switch that enables YaRN for a model whose GGUF does not already carry it, so for Qwen3 past 32K we recommend llama.cpp or vLLM, where the flags are documented.

How to Tell Which Case You Are In

llama.cpp tells you in the load log. It compares your context to the model’s training context and prints one of two lines:

n_ctx_seq (131072) > n_ctx_train (40960) -- possible training context overflow

That is a warning. The run continues. If you see it, you are in the silent-overrun case. Add the YaRN flags.

n_ctx_seq (8192) < n_ctx_train (40960) -- the full capacity of the model will not be utilized

That one is info. You asked for less context than the model supports. It is fine if intended.

When YaRN is active, llama.cpp logs custom YaRN scaling detected, re-adjusting n_ctx_train. If you added the flags and do not see that line, the flags did not reach the loader.

The Cost of Leaving YaRN On

Qwen’s card is direct about this: “All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length, potentially impacting performance on shorter texts.”

Static means the ×4 stretch applies to a 500-token chat exactly as it does to a 120K document. So the practical rule:

• Chat and short agent turns: YaRN off. 32K is plenty.
• Long-document sessions: start a separate server with YaRN on.

Two servers on the same model is a normal setup, not a hack.

Memory: YaRN Does Not Make Context Free

Turning YaRN on lets the model use 131,072 positions. It does not make them fit. KV cache still grows with every token. For Qwen3 32B (64 layers, 8 KV heads, head dim 128) at f16, the cache is about 256 KB per token, so roughly 32 GiB at 128K — our arithmetic from the config, the same formula as our 128K VRAM guide. That is more than the Q4 weights.

If you want long context on modest hardware, the model choice matters more than the flag. Qwen3.6 27B is native 262K and uses 4 KV heads. Our lowest KV cache model ranking compares the cache cost per 128K across current models.

Hardware for Long Context

This is a configuration page, so it carries no product links. The buying decision it feeds is memory:

• 16GB cards: see best local LLM for 16GB VRAM. Long context competes with weights for the same 16GB.
• 48GB and up: see the best 48GB VRAM setup, where 128K on a 32B model becomes realistic.
• Beyond 262K: see hardware for 1M context locally.

See Also

• How Much VRAM for 128K Context? — the KV cache formula, per model
• Context Window Traps for Local Agents — the other ways a local agent loses its context
• KV Cache Quantization: q8_0 vs q4_0 — halve the cache cost once YaRN is on
• Lowest KV Cache Local LLM — models that make long context cheap
• gpt-oss 120B VRAM at Full Context — the model that ships with YaRN already in its config

Sources

• Hugging Face, Qwen/Qwen3-32B model card — “32,768 natively and 131,072 tokens with YaRN”; rope_scaling JSON; vLLM, SGLang and llama.cpp flags; static-YaRN warning — read 2026-09-10
• Hugging Face config.json files read 2026-09-10: Qwen3-32B (40,960, rope_scaling null, 64 heads / 8 KV), Qwen3-30B-A3B (40,960, null), gpt-oss-120b (131,072, YaRN factor 32, original 4,096), Qwen3.6-27B (262,144, rope_type default)
• Hugging Face, Qwen/Qwen3.8-27B model card — 262,144 native, extensible to 1,000,000 with YaRN
• llama.cpp server README (--rope-scaling, --rope-scale, --yarn-orig-ctx, -c) and src/llama-context.cpp log strings, read 2026-09-10
• Ollama docs, context length — VRAM-based defaults, OLLAMA_CONTEXT_LENGTH, ollama ps