Open-Source Text Diffusion Models You Can Run Locally (2026)

Bottom Line (August 2026)

• Run this one: DiffusionGemma-26B-A4B. Apache 2.0, ~3.8B active of 25.2B parameters, roughly 18GB in community GGUF quants — a 24GB card runs it. It had day-zero vLLM support on 10 June 2026, which no other dLLM had.
• Biggest open dLLM: LLaDA2.0-flash, 100B total / 6.1B active, from Ant Group’s InclusionAI. First diffusion LM at the 100B scale, weights and training code on Hugging Face. It needs multi-GPU — this is not a laptop model.
• Best small experiments: Dream 7B, LLaDA-8B, RND1-Base (30B/3B). All open weights. Dream and LLaDA run through llama-diffusion-cli in llama.cpp.
• Cannot run locally: Mercury 2 and Gemini Diffusion. Both closed-weight, API-only. Mercury 2 scored 90% on AIME 2026 versus DiffusionGemma’s 69.1%, so the quality leader is still the one you cannot download.
• The real blocker is tooling, not weights. Diffusion inference is a separate code path in every runtime. Ollama and LM Studio load the GGUFs, but the mature, boring reliability of autoregressive llama.cpp is not here yet.

What a Text Diffusion Model Actually Does

An autoregressive model writes left to right. Each forward pass produces exactly one token, so your ceiling is memory bandwidth divided by model size. That is why local LLMs feel slow on bandwidth-starved hardware.

A diffusion LM works differently. It starts from a fully masked sequence and runs a fixed number of denoising steps, filling in many positions per step. LLaDA trains a Transformer to predict masked tokens under a forward masking process and a reverse generation process. DiffusionGemma uses block diffusion, denoising a block at a time so it can keep some cache behavior.

Two consequences matter for practical use:

1. Throughput can be much higher, because one forward pass emits many tokens.
2. Infilling is native. Dream 7B supports arbitrary-order generation and infilling — you edit the middle of a sequence, not just append. For code editing that is a genuine structural advantage over autoregressive models.

The Open-Weight Models

Everything above the Mercury row you can download today. Everything below it you rent.

Our Pick: DiffusionGemma

DiffusionGemma is the only text diffusion model in 2026 with a normal local-AI on-ramp. Google released it under Apache 2.0 with quantized checkpoints published by RedHat AI (FP8-dynamic and NVFP4), and 18+ community GGUF builds landed at launch. vLLM shipped native support on release day — the first diffusion LLM to get that treatment.

The MoE design does the heavy lifting: only about 3.8B of 25.2B parameters activate per step. Quantized builds land near 18GB of VRAM, which puts an RTX 3090 or 4090 comfortably in range. FP8 needs roughly 28GB; full BF16 is 50GB-plus and targets H100/H200.

Published throughput on datacenter cards: 1,008 tokens/second on H100 and 1,288 on H200 at FP8. Consumer numbers are messier. Community RTX 4090 reports span a few hundred tokens/second to over 1,000, depending on runtime, quantization and step count. Even the low end of that range beats the ~40 tok/s an autoregressive Gemma 4 27B does on the same card.

Quality is the honest weak spot. DiffusionGemma scored 69.1% on AIME 2026, against 88.3% for the non-diffusion Gemma 4 and 90% for closed-weight Mercury 2. You are buying speed and infilling, not frontier reasoning.

LLaDA2.0: The Scale Proof

Ant Group’s InclusionAI team released LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B total, 6.1B active) — the first diffusion language models at the 100B scale, with weights and training code open on Hugging Face. The LLaDA2.0-flash-CAP variant reports up to 535 tokens/second.

They shipped a custom inference engine built on dInfer and SGLang with KV-cache reuse and block-level parallel decoding. That is the right signal: the team treated deployment as part of the release rather than a follow-up. But it also tells you the state of the field — they needed a bespoke engine because the standard ones did not fit.

Practical read: LLaDA2.0-mini is the version most people can actually host. The 100B flash model is a multi-GPU proposition and belongs on rented datacenter hardware, not a desk. If you have that budget, our 20B–35B local model guide covers autoregressive options that will likely serve you better today.

The Honest Caveats

This is where the ecosystem is, not where the marketing is.

llama.cpp support is real but partial. Dream, LLaDA and RND1 are merged, and there is a dedicated llama-diffusion-cli runner. DiffusionGemma’s block-diffusion architecture needs that runner too — the standard llama-cli and llama-server cannot generate from it yet. If your workflow depends on an OpenAI-compatible local server, check that path before you commit.

Most dLLMs have no usable KV cache. LLaDA-8B-Instruct performs best when the number of sampling steps equals the output length, which means each step effectively produces about one token — erasing the parallel-decode advantage entirely at that setting. The speed you read about depends on aggressive step-count reduction, and quality degrades as you cut steps.

Treat speed claims skeptically. A 2025 analysis (arXiv 2510.18480) argues that dLLM efficiency evaluations routinely compare optimized diffusion inference against unoptimized autoregressive baselines. A properly served vLLM autoregressive model is much faster than a naive Transformers loop.

Apple Silicon does not benefit much. The parallel-decode win needs compute headroom. Macs are bandwidth-rich and compute-modest, which is exactly backwards for diffusion decoding. If you are on a Mac, check what fits your machine and stay autoregressive for now.

Some releases are base models. RND1-Base is an experimental research artifact converted from Qwen3-30B-A3B via 500B tokens of continual pretraining. It is a strong result for the autoregressive-to-diffusion conversion technique, not a polished assistant.

Compared against llama.cpp’s autoregressive path — years of quantization formats, grammar-constrained sampling, a stable server, GPU offload that just works — text diffusion in August 2026 is roughly where local LLMs were in early 2023. Interesting, usable, and rough.

Who Should Actually Try This

Run a diffusion LLM locally if you want very high token throughput on an NVIDIA GPU for bulk generation, or if you need native infilling for code editing. DiffusionGemma on a 24GB card with vLLM is the setup to start with.

Skip it if you want the best local reasoning per gigabyte, if you are on Apple Silicon, or if you need a drop-in OpenAI-compatible local server. Autoregressive models still win all three.

See Also

• Best 20B–35B Local LLMs — the autoregressive models at this same size tier
• What Local LLM Fits My Machine? — match models to your actual VRAM
• Why Is My Local LLM So Slow? — the bandwidth math diffusion is trying to escape