Guide October 30, 2026

NVFP4 vs Q4_K_M (2026): Faster Only on Blackwell

NVFP4 and Q4_K_M are both 4-bit formats, and for the same model the files are almost the same size. So the choice is not about fit. It is about speed on your card and about which rounding you trust. On a Blackwell GPU, llama.cpp reads NVFP4 prompts 24-40% faster. On every other machine, Q4_K_M is the safer download. Here are the measured numbers, from llama.cpp pull requests and Hugging Face file lists read on 2026-10-30.

Bottom Line

Same size class. For Qwen3.8 27B, Unsloth UD-Q4_K_M is 15.33 GiB and the most-downloaded NVFP4 GGUF is 15.95 GiB. Both fit the same cards.
NVFP4 is faster only on Blackwell. In llama.cpp, a 16K prompt ran 40% faster on an RTX 5090 and 24% faster on a DGX Spark. Same model, same benchmark command.
No proven quality win. No published test compares the two files on the same model. One community tester measured a ready-made FP4 file worse than plain Q4_0.
On a Mac, NVFP4 does not run. llama.cpp’s Metal backend rejects it.
Action: RTX 50, RTX PRO Blackwell or DGX Spark → try the NVFP4 GGUF. RTX 30 or 40, AMD, or a Mac → download Q4_K_M.

	NVFP4 GGUF	Q4_K_M
Bits per weight	4.5 (16 weights per FP8 scale)	~4.5 base, higher for the Q6_K tensors it mixes in
Qwen3.8 27B file	15.95 GiB (cdiamond iMatrix)	15.33 GiB (Unsloth UD-Q4_K_M)
Prefill, RTX 5090, 16K prompt	5,314 tok/s	3,791 tok/s
Prefill, DGX Spark, 16K prompt	1,115 tok/s	900 tok/s
Runs on RTX 30 / 40	Yes, 8-bit activations	Yes
Runs on Apple Metal	No	Yes
Made by `llama-quantize`	No	Yes

Picking a card for this? See the tested hardware list with current prices by tier.

The Speed Numbers, Same Command

One llama.cpp contributor benchmarked both formats on the same cards with the same command: llama-bench -ngl 99 -fa on -p 16384 -n 0 -b 2048 -ub 2048. The Q4_K_M rows come from PR #29948 (merged 2026-10-25). The NVFP4 rows come from PR #31296 (open on 2026-10-30). Both are Qwen3.8 27B.

Card	Q4_K_M	NVFP4 (master)	NVFP4 (PR #31296)	NVFP4 lead
RTX 5090	3,791 tok/s	5,314 tok/s	5,833 tok/s	+40% (+54% with the PR)
DGX Spark (GB10)	900 tok/s	1,115 tok/s	1,317 tok/s	+24% (+46% with the PR)
RTX 4090	not published	2,786 tok/s	2,893 tok/s	unknown
RTX 3090 Ti	not published	1,281 tok/s	1,306 tok/s	unknown

The Q4_K_M figures already include the fused gate/up kernel from #29948. The NVFP4 master figures do not include the NVFP4 version of that kernel. So the 40% lead is the lead you get today, and #31296 widens it if it merges.

Why Blackwell wins. RTX 50, RTX PRO Blackwell and GB10 have FP4 matrix units. llama.cpp runs NVFP4 there as W4A4: 4-bit weights and 4-bit activations. Cards from Turing to Hopper have no FP4 units. They run NVFP4 as W4A8, with 8-bit activations, as PR #31296 states.

Prefill is not decode. These numbers measure prompt processing, which is compute-bound. Decode, one token at a time, reads every weight once per token. It depends on memory bandwidth and file size. The two files differ by 4% in size, so we estimate decode speed within a few percent of each other. That is our estimate, not a measurement. For reference, PR #30788 reports Qwen3.8 27B NVFP4 decode at 68.65 tok/s on an RTX PRO 6000 (community-reported).

This is the number most comparisons miss. NVFP4 does not make chat replies much faster. It makes long prompts faster: agent loops, RAG context and large code files. If you paste 20K tokens into every request, the 40% matters. If you chat with short prompts, it barely shows.

Quality: No Proven Winner

Three data points exist. None of them is a same-test NVFP4 vs Q4_K_M comparison.

Source	What it measured	Result
NVIDIA model card, `nvidia/Qwen3.8-27B-NVFP4`	NVFP4 MLP + FP8 attention vs BF16	GPQA Diamond 88.01 vs 88.92; Terminal-Bench 74.02 vs 75.56; IFBench 78.93 vs 80.07
cdiamond model card (community)	Short WikiText-2 perplexity, lower is better	Ready-made FP4 GGUF 6.4949; plain Q4_0 6.3798
cdiamond model card (community)	Their iMatrix NVFP4 hybrid vs Q4_1	6.1197 vs 6.1127, called a tie

Read the first row with care. NVIDIA’s checkpoint is not NVFP4 everywhere. Its card says NVFP4 covers the MLP layers and the LM head, and FP8 covers attention. The scores belong to that mix.

The second row is the useful warning. Q4_0 is the oldest and plainest 4-bit GGUF, and Q4_K_M normally beats it. A ready-made FP4 file that loses to Q4_0 is not a quality upgrade over Q4_K_M. The tester called it a short check, so treat it as one data point.

The honest summary: NVFP4 and Q4_K_M are the same quality class. Pick on speed and hardware, not on a quality claim.

”NVFP4” on a File Name Does Not Mean 4-Bit Everywhere

The label tells you the format of the large MLP tensors. It does not tell you the rest. The Hugging Face file lists for Qwen3.8 27B, read 2026-10-30:

File	Size	What the other tensors are
esatapedico NVFP4 VERY-LOW	13.84 GiB	NVFP4 attention, Q3_K LM head, Q2_K embeddings
esatapedico NVFP4 COMPACT-LOW	14.12 GiB	NVFP4 attention, Q4_K LM head
esatapedico NVFP4 MEDIUM	15.25 GiB	NVFP4 attention, Q8_0 LM head
Unsloth UD-Q4_K_M	15.33 GiB	K-quant mix
cdiamond iMatrix NVFP4 + MTP	15.95 GiB	iMatrix hybrid, MTP head
Unsloth UD-Q4_K_XL	16.35 GiB	K-quant mix, larger
esatapedico NVFP4 HIGHEST	21.59 GiB	Q8_0 attention, BF16 embeddings
Unsloth Q8_0	27.05 GiB	8-bit
esatapedico NVFP4 ORIG	30.86 GiB	BF16 attention and last 8 MLP layers

One “NVFP4” file is larger than Q8_0. Check the size before you download, not the name.

Which File to Download

You have	Download	Why
RTX 5090, RTX PRO Blackwell, DGX Spark	NVFP4 GGUF	24-40% faster prefill, same size
RTX 5060 Ti or 5070 Ti 16GB (Blackwell)	Compact NVFP4 tier (~14 GiB)	Leaves room for context on 16GB
RTX 3090, 4090 or older NVIDIA	Q4_K_M	NVFP4 runs, but no proven speed or quality gain
AMD GPU	Q4_K_M	The NVFP4 speed-up needs NVIDIA FP4 units
Mac, any Apple Silicon	Q4_K_M or 4-bit MLX	Metal does not run NVFP4
A BF16 model you quantize yourself	Q4_K_M	`llama-quantize` has no NVFP4 option
vLLM serving on Blackwell	NVFP4 safetensors (`nvidia/Qwen3.8-27B-NVFP4`, 20.42 GiB)	The checkpoint vLLM loads natively

The 16GB case. The esatapedico card reports its COMPACT-LOW file (14.12 GiB) running on one 16GB Blackwell card at 32K context, with a quantized KV cache. Qwen3.8 27B needs 64 KiB of f16 KV cache per token, so 32K costs 2 GiB at f16 and about 1.06 GiB at q8_0. A 15.33 GiB Q4_K_M leaves no room for that. See KV cache quantization for the q8_0 flags.

NVFP4 vs Q5 and Q8

NVFP4 is a 4-bit format. It does not compete with Q5_K_M or Q8_0. Qwen3.8 27B Q8_0 is 27.05 GiB, 70% larger than the NVFP4 GGUF. If Q8_0 fits your card with your context, it beats every 4-bit file on quality. NVFP4 is the choice only when you need 4-bit to fit.

Where This Changes a GPU Purchase

NVFP4 is a real reason to prefer Blackwell only for long prompts. If your work is agents, RAG or large code context, the 40% prefill gain on the RTX 5090 32GB is measurable. As of October 2026 the 5090 trades far above MSRP: GPU Poet showed a current lowest average of $6,332 on 2026-10-30.

The cheapest Blackwell card that runs Qwen3.8 27B NVFP4 is a 16GB card such as the RTX 5060 Ti 16GB, with the compact tier above. Prices move weekly; check current listings.

If your prompts are short, the format does not justify the price gap. A used RTX 3090 24GB runs Q4_K_M with room for context, at a $1,563 average used price as of October 2026 (resaleprices.com, 2026-10-30). Qwen3.8 27B on an RTX 3090 has the settings.

FAQ

Is NVFP4 better than Q4_K_M?

Faster on Blackwell, not proven better. In llama.cpp on an RTX 5090, Qwen3.8 27B NVFP4 processed a 16K prompt at 5,314 tok/s against 3,791 tok/s for Q4_K_M with the same benchmark command, about 40% faster. On a DGX Spark the gap was about 24%. No published test compares the two on quality for the same model, and one community tester found a ready-made FP4 file scored worse perplexity than plain Q4_0. Treat them as the same quality class.

NVFP4 vs Q4_K_XL: which should I download?

For Qwen3.8 27B, Unsloth's UD-Q4_K_XL is 16.35 GiB and the iMatrix NVFP4 GGUF is 15.95 GiB, so they fit the same cards. On an RTX 50 series or RTX PRO Blackwell card, try the NVFP4 GGUF for faster prompt processing. On an RTX 30 or 40 card or a Mac, download UD-Q4_K_XL.

Does NVFP4 work on an RTX 3090 or 4090?

Yes in llama.cpp, but without the Blackwell speed-up. Cards without native FP4 matrix units run NVFP4 with 8-bit activations. On an RTX 4090, Qwen3.8 27B NVFP4 processed a 16K prompt at 2,786 tok/s. No same-command Q4_K_M number exists for that card, so benchmark both files before you keep one. Q4_K_M is the default on these cards.

Can I run NVFP4 on a Mac?

No. In llama.cpp master on 2026-10-30, the Metal backend returns unsupported for NVFP4 matrix multiplication and row lookup, so NVFP4 does not run on the Apple GPU. Use Q4_K_M in llama.cpp or a 4-bit MLX file on Apple Silicon.

Is NVFP4 the same as Q8 or Q5 quality?

No. NVFP4 is a 4-bit format at 4.5 bits per weight, the same class as Q4_K_M. Q5_K_M and Q8_0 keep more bits and need more memory: Qwen3.8 27B Q8_0 is 27.05 GiB against 15.95 GiB for the NVFP4 GGUF. If Q8_0 fits your card, it beats any 4-bit file on quality.

Sources

llama.cpp PR #29948 (merged 2026-10-25): Q4_K_M prefill, RTX 5090 and DGX Spark, -p 16384 -ub 2048 (read 2026-10-30).
llama.cpp PR #31296 (open): NVFP4 prefill, master and PR, RTX 5090, DGX Spark, RTX 4090, RTX 3090 Ti; W4A4 vs W4A8 (read 2026-10-30).
llama.cpp PR #30788 (open): Qwen3.8 27B NVFP4 tg128 on RTX PRO 6000 (read 2026-10-30).
llama.cpp master ggml-metal-device.m: NVFP4 unsupported for mul_mat and get_rows; tools/quantize/quantize.cpp: no NVFP4 type (read 2026-10-30).
Hugging Face API file lists: unsloth/Qwen3.8-27B-GGUF, cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF, esatapedico/Qwen3.8-27B-NVFP4-MTP-GGUF, nvidia/Qwen3.8-27B-NVFP4 (read 2026-10-30).
nvidia/Qwen3.8-27B-NVFP4 model card: recipe and accuracy table. cdiamond model card: perplexity checks (community-reported). esatapedico model card: 16GB single-card run (community-reported).
Qwen/Qwen3.8-27B config.json: 16 full-attention layers, 4 KV heads, head_dim 256 (read 2026-10-30).
GPU Poet RTX 5090 October 2026 page and resaleprices.com RTX 3090, via price-reference.md (read 2026-10-30).

Before you order parts, check the tested hardware list for current prices by tier.