Qwen 3.8 27B on RTX 3090 (2026): 41 tok/s, 66 With MTP

Bottom Line

• Yes, one RTX 3090 runs Qwen3.8-27B fully on the GPU at 4-bit. Expect about 40 tok/s in llama.cpp, and 34 tok/s at 64K context.
• Turn on the MTP head. One community test measured 41.55 to 66.39 tok/s (+59.8%) at 8K context. The flag is --spec-type draft-mtp.
• The MTP gain shrinks with context. A second test at 131K measured only +33% on the same card. Quote the gain with its context length, or it misleads.
• MTP output is not byte-identical to normal greedy decoding in llama.cpp. Fine for chat. Turn it off for text-diff regression tests.
• A 4090 buys prompt speed, not reply speed. It reads prompts 2.3x faster and writes only 14% faster.

All speed numbers on this page are community-reported, with the source named. We did not run them. Your numbers will differ with your build, driver, quant and power limit. The method to measure your own is below.

Measured Speed on One RTX 3090

The thc1006 test used a stock 3090 at its 420W default, llama.cpp built from source with CUDA 13.3, and --spec-draft-n-max 2. More drafted tokens made it slower: n-max 3 gave 63.29 tok/s and n-max 5 gave 54.89 tok/s.

The two MTP results do not conflict. They measure different context lengths. Speculative decoding helps most when each step is cheap. At 131K each step reads a much larger cache, so the same trick returns less. If you work at long context, plan on the +33% figure.

The Catch Nobody Puts in the Headline

The same HackMD test checked the output text. MTP decoding did not reproduce serial greedy decoding byte for byte. At 1,600 generated tokens, 92 to 100 percent of requests diverged from the normal output. The author found the divergence deterministic, and attributed it to floating-point paths that change with batch shape.

This is not a quality loss. The target model still verifies every drafted token. But “lossless” is the wrong word if your test suite compares strings. Turn MTP off for those runs.

What Fits in 24GB

File sizes are from the Hugging Face API for unsloth/Qwen3.8-27B-GGUF, read 2026-09-16.

Vision needs the mmproj-F16 file as well, another 0.86 GiB.

KV Cache: Why Long Context Is Cheap on This Model

Qwen3.8-27B is a dense model with hybrid attention. Its config.json lists 64 layers, and only every fourth layer is full attention. The other 48 are Gated DeltaNet layers with a fixed-size state. So only 16 layers keep a cache that grows.

2 (K and V) x 16 layers x 4 KV heads x 256 head dim x 2 bytes = 65,536 bytes per token

That is 64 KiB per token in f16. This is our arithmetic from the published config.

These sums exclude compute buffers and the DeltaNet state, which we did not measure. Treat each row as a lower bound. A normal 64-layer model would need four times this cache.

The practical rule: Q4_K_XL for work up to about 64K. IQ4_XS when you need 128K. For how the two quants differ in quality, see IQ4_XS vs Q4_K_M vs Q4_K_XL.

Find Your Own Number

Numbers do not transfer between machines. Run this sweep on yours.

1. Build a current llama.cpp. MTP support merged in PR #22673 (2026-05-16). A CUDA-graph change for the MTP draft merged on 2026-09-16, so older builds can understate MTP speed.
2. Start the server without MTP and record tokens per second at your real context:

llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf -ngl all -fa on -c 32768 --parallel 1

3. Add the MTP flags and repeat the same prompt:

llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf -ngl all -fa on -c 32768 --parallel 1 \
  --spec-type draft-mtp --spec-draft-n-max 2

4. Try --spec-draft-n-max at 2, 3 and 4. The default is 3. In the public test, 2 was fastest.
5. Repeat at the longest context you use. Expect the gain to drop.

The MTP head ships inside the GGUF. You do not download a separate draft model. For what each of the other flags does, see llama.cpp flags explained.

What About the 114 tok/s vLLM Claim?

It is real, and it is a different setup. The syv-ai/qwen38-27b-rtx3090 GitHub repo reports about 114 tok/s single-user on one 3090 at a 250W power limit. It uses a patched vLLM 0.28.0, W4A16 AutoRound weights, and MTP. The repo notes a 5-8% spread between runs.

Do not compare that to 40 tok/s from a GGUF file. The weights, the engine and the decoding method all differ. It does show how much headroom the card has if you accept a custom stack.

RTX 3090 vs RTX 4090 for This Model

Hardware Corner tested both cards with the same llama.cpp build (153d324bc), the same Q4_K_S file, and Flash Attention on.

The 4090 reads a prompt about 2.3x faster and writes a reply about 14% faster. Reply speed follows memory bandwidth, and the two cards are close there. Prompt speed follows compute, and the 4090 has much more.

So the choice depends on your work:

• Chat and short prompts: buy the 3090. You will not feel 40 against 46 tok/s.
• Coding agents that re-read 30K-token files every turn: the 4090 cuts each wait by more than half. That adds up over a day.

Two RTX 5060 Ti 16GB cards measured 22.47 tok/s at 4K in the same test. That is about half the speed of one 3090. See dual RTX 5060 Ti vs RTX 3090.

What to Buy

The used RTX 3090 is still the value card for this model. It holds the 4-bit file with room for 64K of context, and it writes at 40 tok/s before MTP.

Used prices have moved up in 2026. Our August reference range was $1,000 to $1,300. On 2026-09-16 the gpudojo tracker showed used listings from $1,399. That is one scraped tracker, so check current listings before you decide. Read how to buy a used RTX 3090 safely first.

Check RTX 3090 24GB prices on Amazon

Buy the 4090 only if long prompts are your daily work. It gives the same 24GB and the same model choices.

Check RTX 4090 24GB prices on Amazon

Sources

• Hugging Face API and README, Qwen/Qwen3.8-27B (27,781,427,952 parameters, Apache 2.0, created 2026-08-05, 262,144 native context, mtp_num_hidden_layers: 1), read 2026-09-16
• Qwen/Qwen3.8-27B config.json (64 layers, full attention every 4th layer, 4 KV heads, head dim 256), read 2026-09-16
• Hugging Face API file tree, unsloth/Qwen3.8-27B-GGUF, read 2026-09-16
• thc1006 on HackMD: “Qwen3.8-27B on One RTX 3090: 59.8% Faster with MTP — But Not Bit-Exact” (community-reported)
• sudoingX/qwen38-mtp on GitHub: paired MTP benchmarks at 131K context (community-reported)
• Hardware Corner: “We Tested Qwen3.8 27B” (build 153d324bc, driver 595.58.03, CUDA 12.8)
• syv-ai/qwen38-27b-rtx3090 on GitHub (patched vLLM, 250W, community-reported)
• llama.cpp docs/speculative.md and GitHub API for PRs #22673 and #28549, read 2026-09-16
• gpudojo RTX 3090 tracker (936 GB/s, used from $1,399), prices updated 2026-09-17 02:21 UTC

See Also

• Can Qwen 3.8 replace Claude for coding?
• Best local LLM for the RTX 3090
• Xing4.0 29B A4B on RTX 3090 — a 4B-active MoE that fits 24GB with 128K context, via a llama.cpp fork
• RTX 3090 vs RTX 4090 for local LLMs
• Qwen 3.5 27B on one RTX 3090
• Qwen3.8-27B vs Muse Glimmer 30B
• Best uncensored local LLM (2026) — the abliterated builds of this same model, with the measured quality cost
• Lowest KV cache models for long context
• llama.cpp flags explained
• Bonsai 2 27B on RTX 3060 12GB — the same model at 1.72 bits on a 12GB card
• Qwen 3.8 27B GSQ-RCO quants — the 2.50 to 3.50 bpw mixed-precision files, and the 12GB and 16GB fit tables