IQ4_XS vs Q4_K_M vs Q4_K_XL (2026): Pick by VRAM

Bottom Line

• Q4_K_XL (Unsloth UD-Q4_K_XL): lowest error, largest file. Use it when it fits with the context you need.
• Q4_K_M: the safe default. Every tool supports it and every uploader publishes it.
• IQ4_XS: smallest file, highest error of the three. Use it only when the extra 1-2GB decides whether the model fits.
• The numbers: in Unsloth’s Qwen3.5-35B-A3B benchmark, mean KL divergence was 0.0137 for Q4_K_XL, 0.0182-0.0192 for Q4_K_M and 0.0234-0.0235 for IQ4_XS.
• The fact that decides it on a 16GB card: Qwen3.6-27B is 15.44GB at IQ4_XS and 16.82GB at Q4_K_M. Only IQ4_XS leaves real room on 16GB.

The Three Formats in One Table

The llama.cpp descriptions come from the quant option table in tools/quantize/quantize.cpp. File sizes come from the Hugging Face file listing for unsloth/Qwen3.6-27B-GGUF. KL divergence comes from Unsloth’s benchmark page, which compares its files against bartowski and AesSedai uploads.

KL divergence measures how far the quantized model’s next-token probabilities drift from the full model. Lower is better. The mean shows the typical drift. The 99.9% figure shows the worst tokens, which is where wrong answers come from.

Read the Ranges Carefully

Two caveats matter before you apply these numbers.

The benchmark is a different model. Unsloth published KL divergence for Qwen3.5-35B-A3B, a mixture-of-experts model. We have no equivalent table for Qwen3.6-27B. We use the 35B-A3B numbers to rank the formats, not to predict exact 27B quality. That is an inference, not a measurement.

Same label, different file. The Q4_K_M range is wide because uploaders build it differently. AesSedai’s Q4_K_M was 20.62GB with a mean KLD of 0.0096. Unsloth’s was 18.49GB at 0.0192. The AesSedai file is 2.1GB larger and has half the drift. A quant name tells you the recipe family, not the exact file. Compare file sizes as well as names.

What Each Format Costs, in Relative Terms

Using the Unsloth rows only, so the builder is the same:

Q4_K_M to Q4_K_XL is the cheapest quality gain on this page. About 0.7GB buys a 29% drop in mean drift. Dropping to 3-bit saves 3GB and more than doubles the drift.

The benchmark page also notes that Unsloth retired MXFP4 layers from its Q2_K_XL, Q3_K_XL and Q4_K_XL files. If you downloaded a Q4_K_XL file earlier in 2026, check the upload date on the repo and download again if needed.

Pick by VRAM: Qwen3.6-27B Example

A card sold as 16GB holds 16GiB, about 17.18GB. A 24GB card holds about 25.77GB. Headroom below is card memory minus the model file. It must cover the KV cache, the compute buffers and the display if the card drives one. These are arithmetic estimates, not measured runs.

On 16GB, the choice is really IQ4_XS or a different model. A 1.7GB margin holds only a modest KV cache, so long-context work on a 27B needs more card. Our 16GB VRAM guide covers the models that fit with room to spare, and Can I Run Qwen 3.5 27B on 16GB VRAM? walks through the same squeeze on the previous release.

On 24GB, IQ4_XS is the wrong choice for this model. You have the room, so spend it on lower drift with Q4_K_XL, or on 5-bit when context is short.

To size the KV cache for your context length, use How Much VRAM for 128K Context?. To see what your runtime actually allocated, use Check KV Cache Size in llama.cpp and Ollama.

When IQ4_XS Is the Right Call

• The model misses your VRAM by 1-2GB at Q4_K_M. IQ4_XS is the least-bad way to close that gap. It is better than dropping to 3-bit, which more than doubled drift in the table above.
• You offload a large mixture-of-experts model. The smaller file puts more layers on the GPU. See llama.cpp MoE offload flags.
• Disk space is tight. A smaller quant is smaller on disk too.

When none of those apply, the smaller file only saves space you do not need.

See Also

• Quantization in Plain English — what Q4, Q5 and Q8 mean before you compare variants
• MXFP4 vs Q4_K_M — the other 4-bit format you will see on model pages
• Best Local LLM for 16GB VRAM — where the IQ4_XS size saving matters most
• Best Local LLM for RTX 5060 Ti 16GB — per-model fit on the cheapest new 16GB card
• KV Cache Quantization: Q8 vs Q4 — the other way to win back VRAM

Sources

• Unsloth Qwen3.5 GGUF benchmarks — disk size, perplexity, mean and 99.9% KL divergence for Unsloth, bartowski and AesSedai quants of Qwen3.5-35B-A3B, and the MXFP4 retirement note; read 2026-09-14
• llama.cpp tools/quantize/quantize.cpp — IQ4_XS “4.25 bpw”, Q4_K_M “+0.1754 ppl @ Llama-3-8B”; read 2026-09-14
• Hugging Face file listing, unsloth/Qwen3.6-27B-GGUF — file sizes via the Hub API; read 2026-09-14
• Headroom figures are arithmetic: 16GiB = 17.18GB and 24GiB = 25.77GB, minus the file size. They exclude runtime overhead, which varies by backend.