Guide September 11, 2026

MXFP4 vs Q4_K_M (2026): MXFP4 Wins on gpt-oss

MXFP4 stores 32 weights in 17 bytes, which is 4.25 bits per weight. llama.cpp's own quantize tool measures Q4_K_M at 4.8944 bits per weight. So MXFP4 is the smaller format, and on gpt-oss it is also the more accurate one, because OpenAI post-trained the model in MXFP4 rather than converting to it afterwards. The strange consequence is visible in the Hugging Face file list: every gpt-oss 120B quant from Q2_K up to Q8_0 lands within 1.3% of the same file size. This page explains why, and when Q4_K_M is still the right choice.

Bottom Line

MXFP4 is the smaller format. 32 weights in 17 bytes is 4.25 bits per weight. llama.cpp measures Q4_K_M at 4.8944 bits per weight.
On gpt-oss, MXFP4 is also the more accurate one. OpenAI post-trained the MoE weights in MXFP4, so the MXFP4 file is the reference copy, not a lossy conversion of it.
The quant label barely changes a gpt-oss file. gpt-oss 120B spans 62.57 GB at Q2_K to 63.39 GB at Q8_0 — a 1.3% spread across the whole ladder.
So “Q2_K gpt-oss” is not a 2-bit model, and “Q8_0 gpt-oss” is not an 8-bit model. Both hold the same MXFP4 experts.
For every other model, Q4_K_M is still the safer default. llama.cpp’s MXFP4_MOE option can convert MoE experts to MXFP4, but a conversion lacks the training that makes gpt-oss accurate in it.
Action: download the official MXFP4 build of gpt-oss. Stop shopping its quant ladder.

The Two Formats, Side by Side

	MXFP4	Q4_K_M
Origin	OCP Microscaling spec, ratified 2023	llama.cpp K-quant family
Block size	32 weights	256-weight superblock
Element format	E2M1: 1 sign, 2 exponent, 1 mantissa bit	4-bit integer
Scale	one shared 8-bit E8M0 power-of-two per block	two half-precision values plus 12 scale bytes
Bytes per block	17 (1 scale + 16 packed)	144
Bits per weight	4.25	4.5 for plain Q4_K; 4.8944 measured for Q4_K_M
Produced by llama.cpp quantize?	MoE experts only (`MXFP4_MOE`, rest Q8_0)	Yes
Read and run by llama.cpp?	Yes — CUDA, Vulkan, Metal, CPU	Yes

The 4.25 figure comes from the block structure: block_mxfp4 in ggml is one uint8_t scale plus 16 bytes of packed 4-bit values for 32 weights. The 4.5 figure for block_q4_K comes the same way: two half scales, 12 scale bytes and 128 bytes of nibbles for 256 weights, so 144 bytes. Q4_K_M then lands higher than 4.5 because the “M” mix promotes some tensors to Q6_K, and llama.cpp’s own quantize documentation puts the result at 4.8944 bits per weight on Llama-3.1-8B.

MXFP4 is roughly 13% smaller than Q4_K_M at the tensor level. Most comparisons call the two formats “about the same size.” They are not.

Why One Mantissa Bit Decides Everything

E2M1 has a single mantissa bit. Within any power-of-two interval it can express exactly two values. Between 1 and 2 you get 1.0 and 1.5, and nothing else.

That is brutal for a weight that was trained in bfloat16 and then rounded down. It is fine for a weight that was trained knowing this is where it would live. OpenAI’s model card states the models “were post-trained with MXFP4 quantization of the MoE weights.” The training process placed the weights on the MXFP4 grid.

This is the whole reason the verdict is format-specific rather than universal:

gpt-oss in MXFP4 is the original. Everything else is a copy of a copy.
Qwen or Llama in MXFP4 would be a 1-mantissa-bit rounding of weights that never expected it. llama.cpp builds that only for MoE experts, through MXFP4_MOE.

Our plain-English quantization guide covers the K-quant ladder that applies to those models. This page is the exception to it.

The Proof: the gpt-oss File List

This is the part that surprises people, and it is checkable in a browser. Read the file sizes in Unsloth’s gpt-oss GGUF repositories.

gpt-oss 120B, two-part files, totals:

Quant	Total size	Difference from Q2_K
Q2_K	62.57 GB	—
Q3_K_M	62.62 GB	+0.1%
Q4_K_M	62.77 GB	+0.3%
Q5_K_M	62.89 GB	+0.5%
Q6_K	63.28 GB	+1.1%
Q8_0	63.39 GB	+1.3%

gpt-oss 20B, single files:

Quant	Size
Q2_K	11.47 GB
Q4_K_M	11.62 GB
Q6_K	12.04 GB
Q8_0	12.11 GB
Official MXFP4 build (ggml-org)	12.11 GB
F16	13.79 GB

On the 120B the entire ladder from 2-bit to 8-bit moves the file by 0.8 GB. On a normal model, Q2_K to Q8_0 is a factor of three.

Our reading of the cause, stated as a reading rather than a measurement: the MoE expert tensors are the overwhelming bulk of gpt-oss, they arrive in MXFP4, and the published quant ladder leaves them in MXFP4. So the quant setting only touches the small remainder — attention projections, embeddings, the output head. Unsloth’s card supports the premise directly, describing the F32 file as “MXFP4 upcasted to BF16 for every single layer.”

One detail worth naming: at 11.62 GB, the 20B Q4_K_M is smaller than the 12.11 GB official MXFP4 build. That is not a better compression of the experts. It is the non-expert tensors being cut down while the experts stay put. You save half a gigabyte by degrading the parts that were full precision.

What This Means When You Pick a File

Your model	Download
gpt-oss 20B or 120B	The official MXFP4 build. Ignore the quant ladder.
gpt-oss, and you are short on disk by under 1 GB	A K-quant saves a rounding error, not a gigabyte. Free space elsewhere.
Qwen, Llama, Gemma, Mistral	Q4_K_M, or Q5/Q6 if VRAM allows
A MoE model, and you want to convert to MXFP4	`llama-quantize ... MXFP4_MOE` (experts only; expect conversion loss)

The practical failure this prevents: somebody with a 16GB card sees “gpt-oss 20B Q2_K, 11.47 GB” and expects a small, degraded, fast build, then sees “Q8_0, 12.11 GB” and expects a heavy, high-quality one. Neither expectation is right. The two files run at nearly the same speed and nearly the same quality, because the experts are the same bytes.

For the memory side of the same decision, see how much VRAM gpt-oss 120B needs at full context and gpt-oss 120B vs 20B.

Runtime Support, Verified

llama.cpp added first-class MXFP4 support as part of the gpt-oss release, across CUDA, Vulkan, Metal and CPU backends. So the format is not a niche you have to work around; it is the fast path.

Correction, 2026-09-12: this section first said llama.cpp cannot emit MXFP4. That was wrong. The quantize README omits it, but the tool’s option table in tools/quantize/quantize.cpp lists MXFP4_MOE, and src/llama-quant.cpp sets MoE expert tensors to MXFP4 and all other tensors to Q8_0 under it. A dense model has no MXFP4 output. For NVIDIA’s competing 4-bit float, see NVFP4 vs MXFP4.

Community quality testing of MXFP4 against Q4_K_M on the same model is still thin, and we are not going to invent a benchmark number to fill the gap. The size arithmetic above is solid. A head-to-head perplexity table is not available, so this page does not print one.

FAQ

Is MXFP4 better than Q4_K_M?

For gpt-oss, yes, and it is smaller too. MXFP4 packs 32 weights into 17 bytes, which is 4.25 bits per weight. llama.cpp's quantize tool reports Q4_K_M at 4.8944 bits per weight. MXFP4 also wins on quality for gpt-oss specifically, because OpenAI post-trained the MoE weights in MXFP4, so the MXFP4 file is the reference copy rather than a lossy conversion of it. For any model that was not trained in MXFP4, the comparison flips. llama.cpp's MXFP4_MOE quantize option can round a MoE model's expert tensors into MXFP4, but that is a conversion without the training behind gpt-oss, so Q4_K_M stays the safer 4-bit default.

Why are all the gpt-oss quant files the same size?

Because the part that dominates the file is already 4-bit and stays that way. Reading Unsloth's gpt-oss-120b-GGUF file list, Q2_K totals about 62.57 GB and Q8_0 totals about 63.39 GB, a spread of 1.3% across the whole ladder. On the 20B the spread is wider but still small: Q2_K 11.47 GB against Q8_0 12.11 GB, about 5.6%. The MoE expert weights are the bulk of the model and they were trained in MXFP4, so the quant label only changes the small non-expert tensors such as attention and embeddings.

Should I download the Q8 version of gpt-oss for better quality?

No. On gpt-oss 120B, Q8_0 is about 0.8 GB larger than Q2_K and the experts are identical MXFP4 in both. You are paying disk space to widen the container around numbers that were never more precise than 4 bits. Download the official MXFP4 build. The one exception is Unsloth's F32 file, which the model card describes as MXFP4 upcasted to BF16 for every layer and unquantized — that is a different artefact for a different purpose, not a quality upgrade for inference.

What exactly is MXFP4?

MXFP4 is the Open Compute Project's microscaling 4-bit float, ratified in 2023 by a consortium including AMD, Intel, Microsoft, NVIDIA, Arm, Meta and Qualcomm. Blocks of 32 elements share one 8-bit power-of-two scale in E8M0 format, and each element is a 4-bit E2M1 float: 1 sign bit, 2 exponent bits, 1 mantissa bit. One mantissa bit means each power-of-two interval holds exactly two values, so between 1 and 2 the only representable numbers are 1.0 and 1.5. That coarseness is why MXFP4 works well for a model trained into it and badly for one converted into it.

Can I convert any model to MXFP4 with llama.cpp?

Only a MoE model, and only partly. llama.cpp's quantize tool has an MXFP4_MOE option: its source sets MoE expert tensors to MXFP4 and every other tensor to Q8_0. A dense model gets no MXFP4 option. The quantize README does not list MXFP4_MOE, which is why this page said otherwise until 2026-09-12; the tool's own option table does. A converted model is a rounding of weights that never trained on the MXFP4 grid, so for a Qwen or Llama model Q4_K_M remains the safer 4-bit answer.