Guide September 12, 2026

MXFP4 vs NVFP4 (2026): NVFP4 Wins, Except on a Mac

NVFP4 and MXFP4 store the same 4-bit float for each weight. The difference is the scale. NVFP4 gives every 16 weights an 8-bit float scale and adds a per-tensor scale. MXFP4 gives every 32 weights one power-of-two scale. So NVFP4 costs 4.5 bits per weight and fits the numbers more closely; MXFP4 costs 4.25 and runs on more hardware. This page shows which file to download, and the runtime facts we read from llama.cpp's source (re-read on 2026-11-03).

Bottom Line

MXFP4 vs NVFP4: NVFP4 is the more accurate format. Both store each weight as the same 4-bit E2M1 float. The scale decides quality, and NVFP4’s scale is finer.

Question	Answer
Which is more accurate?	NVFP4. 16 weights per FP8 scale, 4.5 bits per weight. MXFP4: 32 weights per power-of-two scale, 4.25 bits.
Which is smaller?	MXFP4, by 0.25 bits per weight. For a 27B model that is under 1 GB.
Does an RTX 3090 run NVFP4?	Yes, in llama.cpp. Weights run on the normal CUDA kernels. Only FP4 activation math needs Blackwell.
Does a Mac run NVFP4?	No. llama.cpp’s Metal backend still rejects NVFP4 (re-read 2026-11-03). Use MXFP4 or Q4_K_M.
gpt-oss?	MXFP4. gpt-oss was trained in MXFP4, so that file is the original.

Action: gpt-oss → MXFP4. A model published as NVFP4, on an NVIDIA card → try the NVFP4 GGUF. Anything on Apple Silicon → MXFP4 or Q4_K_M.

The Two Formats, Side by Side

	NVFP4	MXFP4
Owner	NVIDIA	Open Compute Project (2023)
Element	E2M1 4-bit float	E2M1 4-bit float
Block size	16 weights	32 weights
Block scale	8-bit E4M3 float	8-bit E8M0 power of two
Second scale	FP32, one per tensor	None
Bits per weight	4.5	4.25
llama.cpp block	64 weights in 36 bytes	32 weights in 17 bytes
Made by `llama-quantize`?	No	MoE experts only (`MXFP4_MOE`)
Runs on llama.cpp Metal?	No	Yes

The llama.cpp rows come from ggml-common.h. block_nvfp4 holds 4 UE4M3 scale bytes and 32 bytes of packed 4-bit values for 64 weights. That is one scale per 16 weights and 4.5 bits per weight, the same figure NVIDIA publishes. The converter writes the per-tensor scale as a separate small tensor.

Why the Scale Matters More Than the Bits

A 4-bit E2M1 float has only 15 distinct values. The block scale stretches those values over the real weights in the block. A bad scale wastes most of the 15 values.

MXFP4’s scale must be a power of two. If a block’s largest weight falls between two powers of two, the scale rounds, and every weight in the block rounds with it. NVFP4’s E4M3 scale has mantissa bits, so it lands close to the real range.

NVIDIA measured the effect on the scale alone: MSE 0.08 for E4M3 against 0.72 for E8M0. NVIDIA also reports 1% or less accuracy loss against FP8 on DeepSeek-R1-0528. Both numbers are NVIDIA’s own, from its NVFP4 introduction post. We did not reproduce them.

Block size adds to the gap. One outlier weight sets the scale for its whole block. In NVFP4 it distorts 15 neighbours; in MXFP4 it distorts 31.

The exception is a model trained in MXFP4. OpenAI post-trained gpt-oss’s MoE weights in MXFP4. That file is the original. Converting it to NVFP4 re-rounds weights that already sit on the MXFP4 grid, so it cannot add accuracy. See MXFP4 vs Q4_K_M for that case.

What Runs Where (Read From Source, Re-Read 2026-11-03)

Most NVFP4 articles describe a format that “needs Blackwell.” That is true for one engine path and false for another. We read llama.cpp’s master source on 2026-09-12 and vLLM’s LLM Compressor docs the same day. On 2026-11-03 we re-read the Metal, CUDA and quantize files at commit b74298c; the rows below still hold.

Runtime and hardware	NVFP4	MXFP4
llama.cpp, CUDA, RTX 50 / RTX PRO Blackwell	Runs, plus the Blackwell FP4 activation path	Runs
llama.cpp, CUDA, RTX 30 / 40	Runs (kernel settings exist for Ampere and older)	Runs
llama.cpp, AMD (ROCm)	Kernel settings exist for RDNA2 to RDNA4	Runs
llama.cpp, Vulkan / CPU	Code present; Vulkan scale-table bug fixed 2026-11-03 (PR #31079)	Runs
llama.cpp, Apple Metal	No — matrix multiply and row lookup return unsupported	Runs
vLLM, Blackwell (compute capability 10.0+)	W4A4: 4-bit weights and activations	Runs
vLLM, older NVIDIA	Weight-only (W4A16)	Runs (Marlin on Ampere, per the gpt-oss recipe)

The Blackwell dependence in llama.cpp is narrow. common.cuh wraps the FP4 activation scale conversion in BLACKWELL_MMA_AVAILABLE, with the comment “only for Blackwell”. The weight kernels have no such guard.

One honest limit: we confirmed the kernels exist. We did not measure NVFP4 tokens per second on an RTX 3090 or 4090, and we found no published number. Code that exists is not the same as code that is fast. If speed on an older card matters, test a K-quant of the same model next to it.

Can an RTX 3090 Run NVFP4?

Yes. This is the question behind most “nvfp4 3090” and “nvfp4 on ampere” searches, so here is the full answer by card.

Your hardware	NVFP4 GGUF in llama.cpp	What to expect
RTX 3090 / 3090 Ti (Ampere)	Runs	Weights dequantize inside the normal CUDA matrix kernels. No published speed figure exists.
RTX 4090 (Ada)	Runs	Same path as the 3090. No published speed figure exists.
RTX 5090, RTX PRO Blackwell, DGX Spark	Runs, with FP4 activations	The only measured gain: prefill +24% to +40% over Q4_K_M (NVFP4 vs Q4_K_M).
AMD Radeon (ROCm)	Kernel settings for RDNA2 to RDNA4	Untested by us.
AMD / Intel on Vulkan	Runs	Use a build from 2026-11-03 or later. PR #31079 fixed an out-of-bounds scale lookup in the Vulkan NVFP4 shader.
Apple Silicon (Metal)	No	Matrix multiply and row lookup return unsupported.

The thing most NVFP4 model cards get wrong: many say “Blackwell required.” That describes the setup the uploader tested. In llama.cpp source, the only Blackwell guard on the NVFP4 path is the activation scale conversion in common.cuh, commented “only for Blackwell”.

What a 3090 owner should do: download both the NVFP4 GGUF and Q4_K_M of the same model, then run one command on each:

llama-bench -m model-NVFP4.gguf -m model-Q4_K_M.gguf -p 512 -n 128

Keep the file with the higher tokens per second, unless you see quality loss in your own prompts. The two files are about the same size, so VRAM does not decide it.

The Download Decision

NVFP4 GGUFs are not a niche. On 2026-11-03 the Hugging Face API listed 3,192,548 downloads in the last 30 days for cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF alone (2,788,978 on 2026-09-12). The same search found NVFP4 GGUFs of Qwen3.6 35B-A3B, Qwen3.6 27B and Flint above 70,000 downloads each.

Sizes for Qwen3.8-27B, read from the Hugging Face file lists the same day:

File	Size
NVFP4 GGUF, lowest tier (esatapedico COMPACT-LOW)	15.16 GB
Unsloth UD-Q4_K_M	16.46 GB
NVFP4 GGUF, iMatrix (cdiamond)	17.13 GB
NVFP4 GGUF, HIGHEST tier (NVFP4 MLP, Q8_0 attention)	23.19 GB
Unsloth Q8_0	29.05 GB

The NVFP4 files and Q4_K_M occupy the same size band. So the choice is not about fitting a card. It is about which 4-bit rounding you trust, and which kernel your hardware runs.

You have	Download
gpt-oss 20B or 120B, any hardware	MXFP4 (the original)
A model released as NVFP4, RTX 50 or RTX PRO Blackwell	NVFP4 GGUF
A model released as NVFP4, RTX 30 or 40	NVFP4 runs; benchmark it against Q4_K_M before you commit
Any model, Mac or other Apple Silicon	MXFP4 for gpt-oss, Q4_K_M for the rest
A BF16 model you want to quantize yourself	Q4_K_M — `llama-quantize` cannot make NVFP4

Where This Changes a GPU Purchase

NVFP4 is a reason to prefer Blackwell only if you serve with vLLM or TensorRT-LLM. Those engines run 4-bit activations on Blackwell and fall back to weight-only on older cards. For a single user in llama.cpp or Ollama, the format alone does not justify an upgrade.

If you serve several users from vLLM, the Blackwell path is real. The 32GB card is the entry point: the GIGABYTE RTX 5090 WINDFORCE 32GB. Street prices are far above MSRP in 2026; check current listings. Which RTX 5090 to buy explains why the cheapest board is the right one.

For 96GB on one Blackwell card, the NVIDIA RTX PRO 6000 Blackwell 96GB is the step up. Its price moves by retailer and by week in 2026; check current listings.

Converting Your Own Model

llama-quantize has no NVFP4 option. We checked its option table in tools/quantize/quantize.cpp. It does have MXFP4_MOE, which puts MoE expert tensors in MXFP4 and every other tensor in Q8_0.

NVFP4 GGUFs start from a checkpoint that is already NVFP4, such as an NVIDIA ModelOpt export. convert_hf_to_gguf then repacks its weight_scale, weight_scale_2 and input_scale tensors. It does not create them. Nemotron 3.5 Lightning is an example of a model NVIDIA publishes as NVFP4.

FAQ

What is the difference between NVFP4 and MXFP4?

Both store each weight as a 4-bit E2M1 float. MXFP4, the Open Compute Project format, shares one 8-bit power-of-two (E8M0) scale across 32 weights, which is 4.25 bits per weight. NVFP4, NVIDIA's format, shares one 8-bit E4M3 float scale across 16 weights and adds a second FP32 scale per tensor, which NVIDIA puts at 4.5 bits per weight. Smaller blocks and a finer scale let NVFP4 follow the real weight values more closely.

Is NVFP4 more accurate than MXFP4?

On the scale itself, yes. NVIDIA's published analysis measured a mean squared error of 0.08 for E4M3 scales against 0.72 for E8M0 scales, and reports 1% or less accuracy loss against FP8 on DeepSeek-R1-0528. Those are NVIDIA's numbers, not ours. The exception is a model trained in MXFP4, such as gpt-oss: its MXFP4 file is the original, so a conversion to NVFP4 cannot improve it.

Does NVFP4 need a Blackwell GPU?

Not in llama.cpp. Its CUDA backend carries NVFP4 kernel settings for Pascal, Ampere, Blackwell and AMD RDNA cards, so NVFP4 weights load and run on an RTX 3090. The Blackwell-only part is FP4 activation math. vLLM states the same split: on GPUs below compute capability 10.0 it runs NVFP4 as weight-only. Some NVFP4 GGUF model cards still say Blackwell is required; read that as the tested setup.

Can an RTX 3090 run NVFP4?

Yes, in llama.cpp. NVFP4 GGUF weights load and run on an RTX 3090, 4090 or other pre-Blackwell NVIDIA card through the normal CUDA kernels. Only the FP4 activation path needs Blackwell. No one has published an NVFP4 speed figure for an RTX 3090, so benchmark the NVFP4 file against Q4_K_M of the same model before you keep it. In vLLM, a 3090 runs NVFP4 as weight-only.

Can I run NVFP4 on a Mac?

Not well today. In llama.cpp's Metal backend, the support check returns false for NVFP4 matrix multiplication and row lookup, so those operations do not run on the Apple GPU. MXFP4 has Metal kernels. On a Mac, download MXFP4 for gpt-oss and a K-quant such as Q4_K_M for other models.

Can llama.cpp quantize a model to NVFP4?

No. llama-quantize has no NVFP4 option; it has MXFP4_MOE for MoE experts only. NVFP4 GGUF files come from converting a checkpoint that was already quantized to NVFP4. The converter repacks its weight_scale, weight_scale_2 and input_scale tensors into GGUF.

Sources

llama.cpp master at commit b74298c (2026-11-03): ggml/src/ggml-metal/ggml-metal-device.m (NVFP4 rejected for mul_mat and get_rows), ggml/src/ggml-cuda/common.cuh (Blackwell guard on activation scales only), tools/quantize/quantize.cpp (no NVFP4 type; MXFP4_MOE only).
llama.cpp PR #31079, “vulkan : fix out-of-bounds NVFP4 UE4M3 scale lookup”, merged 2026-11-03.
llama.cpp ggml-common.h (block layouts), read 2026-09-12.
Hugging Face API, model cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF and search “nvfp4” filtered to GGUF, read 2026-11-03.
NVIDIA, “Introducing NVFP4” technical blog (scale MSE and DeepSeek-R1-0528 accuracy figures).
vLLM LLM Compressor docs (W4A4 on compute capability 10.0+, weight-only below), read 2026-09-12.