GGUF Quant Names Explained (2026): Q, K, M, IQ, UD

The Decoder Table

Read a filename from left to right. For example, Qwen3.6-27B-UD-Q4_K_XL.gguf is: model name, then UD, then Q4, then K, then XL.

Two rules make the table work.

1. The name tells you the main type. It does not tell you every type. A Q4_K_M file holds many tensors at Q6_K and Q5_K, and the small 1-D tensors stay at F32.
2. Only the llama.cpp types have a fixed bits-per-weight value. UD, _XL, GSQ-RCO and similar labels are recipes. Their size changes from model to model.

How the Parts Fit Together

The Q number

The number after Q is the bit count of the base type. It is not the average for the file. A Q4_K_M file has an average above 4.5 bits per weight, because the recipe keeps some tensors at 6 bits.

Legacy, K and I: three generations

• Legacy (Q4_0, Q8_0): one scale for each block of 32 weights. They are simple and fast. Q8_0 is still the normal 8-bit choice.
• K-quants (_K): blocks of 32 inside super-blocks of 256. The scales are also quantized. This gives better quality at the same size.
• I-quants (IQ): the weights map to a non-linear grid. This gives better quality at 2 and 3 bits. At 4 bits, IQ4_XS is the smallest useful option.

The tier letter

After _K, the tier letter is a recipe, not a type. The quantizer applies the base type to most tensors. Then it applies a higher type to some sensitive tensors, such as parts of the attention and feed-forward layers. _S promotes fewer tensors. _L promotes more.

The llama.cpp quantize help table gives the size effect on Llama-3-8B: Q4_K_S 4.37G, Q4_K_M 4.58G. Q3_K_S 3.41G, Q3_K_M 3.74G, Q3_K_L 4.03G.

IQ and imatrix are two different things

Many people read IQ as “imatrix quant”. That is not correct. IQ names a storage type. The imatrix is calibration data that the quantizer uses to pick the weights to protect. Uploaders use an imatrix for most IQ files, because low-bit i-quants lose too much quality without one. Uploaders also use an imatrix for K-quants. A file named Q4_K_M can be an imatrix quant.

UD and _XL

Unsloth Dynamic files have a UD- prefix. Unsloth’s docs say it now changes “the quantization type of every possible layer” and uses its own imatrix calibration set [source: Unsloth Dynamic 2.0 GGUFs docs]. For the pick between UD-Q4_K_XL, Q4_K_M and IQ4_XS on a given card, see IQ4_XS vs Q4_K_M vs Q4_K_XL. This page does not repeat that pick.

MXFP4 and NVFP4

These are float formats with 4-bit values and a shared scale. They are not k-quants. gpt-oss ships with MXFP4 weights, so every gpt-oss GGUF is close to the same size. The details are in MXFP4 vs Q4_K_M and NVFP4 vs MXFP4.

The Same Label Is Not the Same File

We read the file sizes from the Hugging Face API and the tensor types from the GGUF headers on 2026-09-20.

F32 tensors are not in the counts. Sizes are bytes divided by 10^9.

What this shows:

• The two Q4_K_M files of Qwen3.6-27B differ by 1.17 GB. bartowski keeps 32 tensors at Q8_0. unsloth keeps none at Q8_0.
• The header does not tell the full story. The unsloth Qwen3.6-27B Q4_K_M and UD-Q4_K_XL files both carry file_type = 15. In llama.h, 15 is LLAMA_FTYPE_MOSTLY_Q4_K_M. The label _XL exists only in the filename.
• A 2026 Unsloth “Q4_K_M” has more IQ4_XS tensors than Q4_K tensors. The label names the family, not the content.
• IQ4_NL and Q4_K_M can be the same size to the byte. Both base types are 4.5 bits per weight. bartowski uses one recipe and changes only the base type.
• A label can name a type that is not the main type. The byteshape “IQ4_XS” file has more IQ3_S tensors than IQ4_XS tensors. The uploader put the real average, 3.84 bpw, in the filename.

The file size decides the fit. A 1 GB difference is the difference between a model that fits a 16 GB or 24 GB card with context, and a model that spills. If you are near a VRAM limit, see the cards this matters for before you download.

Method: Read the Real Size and Tensor Mix Yourself

Do not trust the label. Read the file. Each step below takes less than a minute. The numbers for your model will be different from the numbers above.

1. Get the exact file sizes

Use the Hugging Face API with ?blobs=true. Without that parameter, the API does not return sizes.

curl -s "https://huggingface.co/api/models/unsloth/Qwen3.6-27B-GGUF?blobs=true" \
  | python3 -c "import json,sys
for s in json.load(sys.stdin)['siblings']:
    if s['rfilename'].endswith('.gguf'):
        print(s['rfilename'], round(s['size']/1e9, 2), 'GB')"

Do this for two uploaders of the same model. Compare the same label.

2. See the tensor types in the browser

On a Hugging Face model page, click a .gguf file. The GGUF viewer shows the metadata and a list of tensors with the type of each tensor. Look at token_embd, output.weight, attn_v and ffn_down. Uploaders change these tensors most often.

3. Dump the header locally

The gguf Python package from llama.cpp includes gguf-dump.

pip install gguf
gguf-dump ./Qwen3.6-27B-Q4_K_M.gguf | less

The output shows general.file_type and each tensor with its type. Count the types. That count is the real recipe.

4. Get the true bits per weight

Divide the file size in bits by the parameter count.

bits per weight = file size in bytes x 8 / number of parameters

This value includes the embeddings and the higher-type tensors. It is the value that decides VRAM use. Then add memory for the KV cache. The llama.cpp MoE offload flags page shows how to fit a file that is too large for the card.

Common Mistakes

1. Comparing labels across uploaders. Compare sizes. The same label from two uploaders can differ by more than 1 GB.
2. Reading IQ as “imatrix”. IQ is a storage type. Any type can use an imatrix.
3. Treating the Q number as the average. Q4_K_M is more than 4.5 bits per weight on average.
4. Looking for _XL in llama.cpp. It is not there. It is an Unsloth tier.
5. Assuming a third-party type runs everywhere. PQ2_0 and PTQ1_0 need the PrismML fork. Mainline llama.cpp does not load them.

See Also

• IQ4_XS vs Q4_K_M vs Q4_K_XL — which 4-bit file to pick for your VRAM
• Quantization in Plain English — what quantization does, before you read the names
• MXFP4 vs Q4_K_M — the float format on gpt-oss
• NVFP4 vs MXFP4 — the two 4-bit float formats and the hardware for each
• —n-cpu-moe Explained: llama.cpp MoE Offload Flags — run a file that is larger than your VRAM
• llama.cpp Flags Explained — the other flags that change memory use

Sources

• llama.cpp tools/quantize/quantize.cpp — quantize help table: IQ bpw values, TQ1_0 1.69, TQ2_0 2.06, Q1_0 1.125, Q2_0 2.25, Llama-3-8B sizes for K-quant tiers, MXFP4_MOE; read 2026-09-20
• llama.cpp ggml/include/ggml.h — the full type list, including MXFP4 (39) and NVFP4 (40); read 2026-09-20
• llama.cpp ggml/src/ggml-common.h — block layouts and “effectively N bits per weight” comments for K-quants; MXFP4 and NVFP4 bpw are arithmetic from the block sizes; read 2026-09-20
• llama.cpp include/llama.h — file_type 15 = MOSTLY_Q4_K_M; read 2026-09-20
• llama.cpp PRs: #1684 k-quants, #4773 SOTA 2-bit quants, #4861 importance matrix, #5590 IQ4_NL, #5747 IQ4_XS, #15091 gpt-oss / MXFP4, #19769 NVFP4
• Unsloth Dynamic 2.0 GGUFs docs — per-layer type selection and imatrix calibration; read 2026-09-20
• Hugging Face API (?blobs=true) and GGUF headers for unsloth/Qwen3.6-27B-GGUF, bartowski/Qwen_Qwen3.6-27B-GGUF, unsloth/Qwen3.8-27B-GGUF, bartowski/Qwen3.8-27B-GGUF, byteshape/Qwen3.8-27B-GGUF; read 2026-09-20
• Model cards: ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF and davetha/Ternary-Bonsai-2-27B-Abliterated-PQ2_0-GGUF (PQ2_0 and PTQ1_0 are PrismML fork types, community-reported); read 2026-09-20