Guide September 12, 2026

NVFP4 vs MXFP4 (2026): NVFP4 Is Finer, MXFP4 Runs on Mac

NVFP4 and MXFP4 store the same 4-bit float for each weight. The difference is the scale. NVFP4 gives every 16 weights an 8-bit float scale and adds a per-tensor scale. MXFP4 gives every 32 weights one power-of-two scale. So NVFP4 costs 4.5 bits per weight and fits the numbers more closely; MXFP4 costs 4.25 and runs on more hardware. This page shows which file to download, and the runtime facts we read from llama.cpp's source on 2026-09-12.

Bottom Line

Same 4-bit element, different scale. Both formats store each weight as an E2M1 float. The scale decides quality.
NVFP4 is the finer format: 16 weights per FP8 scale plus a per-tensor scale, 4.5 bits per weight.
MXFP4 is the smaller format: 32 weights per power-of-two scale, 4.25 bits per weight.
NVFP4 does not force a Blackwell upgrade in llama.cpp. Its CUDA backend runs NVFP4 weights on older NVIDIA cards too. Blackwell adds FP4 activation math.
On a Mac, skip NVFP4. llama.cpp’s Metal backend has no NVFP4 matrix kernel. MXFP4 runs there.
Action: gpt-oss → MXFP4. A model published as NVFP4, on an NVIDIA card → try the NVFP4 GGUF. Anything on Apple Silicon → MXFP4 or Q4_K_M.

The Two Formats, Side by Side

	NVFP4	MXFP4
Owner	NVIDIA	Open Compute Project (2023)
Element	E2M1 4-bit float	E2M1 4-bit float
Block size	16 weights	32 weights
Block scale	8-bit E4M3 float	8-bit E8M0 power of two
Second scale	FP32, one per tensor	None
Bits per weight	4.5	4.25
llama.cpp block	64 weights in 36 bytes	32 weights in 17 bytes
Made by `llama-quantize`?	No	MoE experts only (`MXFP4_MOE`)
Runs on llama.cpp Metal?	No	Yes

The llama.cpp rows come from ggml-common.h. block_nvfp4 holds 4 UE4M3 scale bytes and 32 bytes of packed 4-bit values for 64 weights. That is one scale per 16 weights and 4.5 bits per weight, the same figure NVIDIA publishes. The converter writes the per-tensor scale as a separate small tensor.

Why the Scale Matters More Than the Bits

A 4-bit E2M1 float has only 15 distinct values. The block scale stretches those values over the real weights in the block. A bad scale wastes most of the 15 values.

MXFP4’s scale must be a power of two. If a block’s largest weight falls between two powers of two, the scale rounds, and every weight in the block rounds with it. NVFP4’s E4M3 scale has mantissa bits, so it lands close to the real range.

NVIDIA measured the effect on the scale alone: MSE 0.08 for E4M3 against 0.72 for E8M0. NVIDIA also reports 1% or less accuracy loss against FP8 on DeepSeek-R1-0528. Both numbers are NVIDIA’s own, from its NVFP4 introduction post. We did not reproduce them.

Block size adds to the gap. One outlier weight sets the scale for its whole block. In NVFP4 it distorts 15 neighbours; in MXFP4 it distorts 31.

The exception is a model trained in MXFP4. OpenAI post-trained gpt-oss’s MoE weights in MXFP4. That file is the original. Converting it to NVFP4 re-rounds weights that already sit on the MXFP4 grid, so it cannot add accuracy. See MXFP4 vs Q4_K_M for that case.

What Runs Where (Read From Source, 2026-09-12)

Most NVFP4 articles describe a format that “needs Blackwell.” That is true for one engine path and false for another. We read llama.cpp’s master source on 2026-09-12 and vLLM’s LLM Compressor docs the same day.

Runtime and hardware	NVFP4	MXFP4
llama.cpp, CUDA, RTX 50 / RTX PRO Blackwell	Runs, plus the Blackwell FP4 activation path	Runs
llama.cpp, CUDA, RTX 30 / 40	Runs (kernel settings exist for Ampere and older)	Runs
llama.cpp, AMD (ROCm)	Kernel settings exist for RDNA2 to RDNA4	Runs
llama.cpp, Vulkan / CPU	Code present	Runs
llama.cpp, Apple Metal	No — matrix multiply and row lookup return unsupported	Runs
vLLM, Blackwell (compute capability 10.0+)	W4A4: 4-bit weights and activations	Runs
vLLM, older NVIDIA	Weight-only (W4A16)	Runs (Marlin on Ampere, per the gpt-oss recipe)

The Blackwell dependence in llama.cpp is narrow. common.cuh wraps the FP4 activation scale conversion in BLACKWELL_MMA_AVAILABLE, with the comment “only for Blackwell”. The weight kernels have no such guard.

One honest limit: we confirmed the kernels exist. We did not measure NVFP4 tokens per second on an RTX 3090 or 4090, and we found no published number. Code that exists is not the same as code that is fast. If speed on an older card matters, test a K-quant of the same model next to it.

The Download Decision

NVFP4 GGUFs are not a niche. On 2026-09-12 the Hugging Face API listed 2,788,978 downloads for cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF alone.

Sizes for Qwen3.8-27B, read from the Hugging Face file lists the same day:

File	Size
NVFP4 GGUF, lowest tier (esatapedico COMPACT-LOW)	15.16 GB
Unsloth UD-Q4_K_M	16.46 GB
NVFP4 GGUF, iMatrix (cdiamond)	17.13 GB
NVFP4 GGUF, HIGHEST tier (NVFP4 MLP, Q8_0 attention)	23.19 GB
Unsloth Q8_0	29.05 GB

The NVFP4 files and Q4_K_M occupy the same size band. So the choice is not about fitting a card. It is about which 4-bit rounding you trust, and which kernel your hardware runs.

You have	Download
gpt-oss 20B or 120B, any hardware	MXFP4 (the original)
A model released as NVFP4, RTX 50 or RTX PRO Blackwell	NVFP4 GGUF
A model released as NVFP4, RTX 30 or 40	NVFP4 runs; benchmark it against Q4_K_M before you commit
Any model, Mac or other Apple Silicon	MXFP4 for gpt-oss, Q4_K_M for the rest
A BF16 model you want to quantize yourself	Q4_K_M — `llama-quantize` cannot make NVFP4

Where This Changes a GPU Purchase

NVFP4 is a reason to prefer Blackwell only if you serve with vLLM or TensorRT-LLM. Those engines run 4-bit activations on Blackwell and fall back to weight-only on older cards. For a single user in llama.cpp or Ollama, the format alone does not justify an upgrade.

If you serve several users from vLLM, the Blackwell path is real. The 32GB card is the entry point: the GIGABYTE RTX 5090 WINDFORCE 32GB. Street prices are far above MSRP in 2026; check current listings. Which RTX 5090 to buy explains why the cheapest board is the right one.

For 96GB on one Blackwell card, the NVIDIA RTX PRO 6000 Blackwell 96GB is the step up. As of September 2026, NVIDIA’s own marketplace lists the Workstation Edition at $16,000, and Newegg listings start near $15,600.

Converting Your Own Model

llama-quantize has no NVFP4 option. We checked its option table in tools/quantize/quantize.cpp. It does have MXFP4_MOE, which puts MoE expert tensors in MXFP4 and every other tensor in Q8_0.

NVFP4 GGUFs start from a checkpoint that is already NVFP4, such as an NVIDIA ModelOpt export. convert_hf_to_gguf then repacks its weight_scale, weight_scale_2 and input_scale tensors. It does not create them. Nemotron 3.5 Lightning is an example of a model NVIDIA publishes as NVFP4.

FAQ

What is the difference between NVFP4 and MXFP4?

Both store each weight as a 4-bit E2M1 float. MXFP4, the Open Compute Project format, shares one 8-bit power-of-two (E8M0) scale across 32 weights, which is 4.25 bits per weight. NVFP4, NVIDIA's format, shares one 8-bit E4M3 float scale across 16 weights and adds a second FP32 scale per tensor, which NVIDIA puts at 4.5 bits per weight. Smaller blocks and a finer scale let NVFP4 follow the real weight values more closely.

Is NVFP4 more accurate than MXFP4?

On the scale itself, yes. NVIDIA's published analysis measured a mean squared error of 0.08 for E4M3 scales against 0.72 for E8M0 scales, and reports 1% or less accuracy loss against FP8 on DeepSeek-R1-0528. Those are NVIDIA's numbers, not ours. The exception is a model trained in MXFP4, such as gpt-oss: its MXFP4 file is the original, so a conversion to NVFP4 cannot improve it.

Does NVFP4 need a Blackwell GPU?

Not in llama.cpp. Its CUDA backend carries NVFP4 kernel settings for Pascal, Ampere, Blackwell and AMD RDNA cards, so NVFP4 weights load and run on an RTX 3090. The Blackwell-only part is FP4 activation math. vLLM states the same split: on GPUs below compute capability 10.0 it runs NVFP4 as weight-only. Some NVFP4 GGUF model cards still say Blackwell is required; read that as the tested setup.

Can I run NVFP4 on a Mac?

Not well today. In llama.cpp's Metal backend, the support check returns false for NVFP4 matrix multiplication and row lookup, so those operations do not run on the Apple GPU. MXFP4 has Metal kernels. On a Mac, download MXFP4 for gpt-oss and a K-quant such as Q4_K_M for other models.

Can llama.cpp quantize a model to NVFP4?

No. llama-quantize has no NVFP4 option; it has MXFP4_MOE for MoE experts only. NVFP4 GGUF files come from converting a checkpoint that was already quantized to NVFP4. The converter repacks its weight_scale, weight_scale_2 and input_scale tensors into GGUF.