Can I Run MiMo V2.6 Flash Locally (2026): Needs 256GB

Bottom Line

• Flash needs a 256GB machine. The checkpoint is 165.53 GiB and it already ships at about 4 bits. The mlx-community build says loading takes about 170 GB of unified memory.
• “15B active” does not mean 15B of memory. The model stores 309B parameters across 256 experts. Every expert must sit in memory.
• There is no 4x quantization win left. The safetensors are packed U8 (two 4-bit weights per byte) plus FP8 attention. 165.53 GiB is the floor, not a BF16 number.
• No GGUF exists yet. As of 21 September 2026, Hugging Face has no GGUF of Flash. llama.cpp has the MIMO2 architecture from V2.5, so one may follow.
• The one measured speed: Vontra reports 59.4 tok/s decode on a 256GB M3 Ultra Mac Studio, at a peak of 164.3-166.8 GB.
• The fact most guides miss: the 1M context is cheap. Our arithmetic on config.json puts a full 1M-token KV cache at about 22.5 GiB. The weights lock this model to 256GB, not the context.
• Everyone else: run MiMo-V2.6-Distill-Qwen-9B. Q8_0 is 8.87 GiB and fits a 12GB card.

What the Model Is

All figures below come from the MiMo-V2.6-Flash-RL model card and the Hugging Face API, read on 21 September 2026.

Spec	Value
Parameters	309B total, 15B active
Experts	256 routed, 8 active, no shared experts
Layers	48: 39 sliding-window (window 128) + 9 global attention
Hidden size	4096
Attention heads	SWA 64 Q / 8 KV; global 64 Q / 4 KV; head dims QK 192, V 128
Context	1,000,000 tokens
Modalities	Text, image, video, audio (681M vision encoder; 308M + 127M audio encoders)
Draft head	5-layer MTP (DFlash-style)
Checkpoint	159,358,725,504 stored parameters, 165.53 GiB, 90 safetensors files
Storage dtypes	U8 151.40B, F8_E4M3 3.86B, BF16 4.10B, F32 12,032
License	MIT, not gated
Created on Hugging Face	21 September 2026

The dtype split is the key row. Our reading: the U8 count is packed 4-bit expert weights (MXFP4, two parameters per byte), and the F8 tensors are attention. Two 4-bit weights per U8 slot turns 151.40B stored bytes into about 302.8B expert parameters, which lines up with the vendor’s 309B total. So the file on Hugging Face is already the “4-bit quant”. Nobody can cut it 4x from here.

Xiaomi’s vendor recipes are SGLang with --tp 8 and vLLM with --tensor-parallel-size 4. That is 4 to 8 datacenter GPUs. Recommended sampling is temperature 1.0 and top_p 0.95.

The same day, Xiaomi released MiMo-V2.6-Pro-RL at 1.02T total / 42B active, 534.07 GiB across 155 files. That is not a local model, and we do not cover it further.

Vendor benchmarks

These are Xiaomi’s self-reported numbers from the model card. We did not run them.

Benchmark	Flash	Pro	Claude Opus 5
DeepSWE v1.1	67.9	71.9	74.0
Terminal Bench 2.1	87.6	89.9
Toolathlon-Verified	73.6	76.9
CyberGym	95.1	94.0
ExploitBench	25.3	47.9
OSWorld-Verified	80.8

Flash leads Pro on CyberGym and trails it by 22.6 points on ExploitBench. Take the whole table as the vendor’s own runs.

Every Published Build, Measured

Sizes come from the Hugging Face API, read on 21 September 2026. GiB to GB uses 1.0737.

Build	Format	Size (GiB)	Size (GB)	Memory to run	Smallest machine
XiaomiMiMo/MiMo-V2.6-Flash-RL	Safetensors, MXFP4 + FP8	165.53	177.74	4-8 datacenter GPUs (vendor recipe)	Server
mlx-community/MiMo-V2.6-Flash-RL-mxfp4-q8	MLX, 4.334 bpw, text-only	155.78	167.26	”about 170 GB” per README	256GB Mac
Vontra/MiMo-V2.6-Flash-RL-MLX-4bit-MTP	MLX, 4.257 bpw + MTP payload	159.85 (whole repo)	171.63	164.3-166.8 GB measured peak	256GB Mac
GGUF	none published

The mlx-community README describes the recipe: experts stay in native MXFP4 (4-bit, group 32), while attention, embeddings and lm_head go to 8-bit affine. That gives 4.334 bits per weight overall and “156 GB on disk. Loading takes about 170 GB of unified memory.” It requires mlx-lm from a branch: pip install git+https://github.com/kernelpool/mlx-lm.git@add-mimo-v2.

A 24GB, 32GB, 96GB or 128GB machine cannot hold any row in this table.

Expected GGUF ladder (inference from V2.5)

llama.cpp shipped unsloth/MiMo-V2.5-GGUF for the predecessor, MiMo-V2.5 at 310.8B parameters. V2.6-Flash has the same parameter count, so a V2.6 GGUF would likely land near these sizes. This is our inference, not a published file list.

V2.5 quant	Size (GiB)	Would fit
UD-IQ1_M	86.17	128GB, tight
UD-IQ2_M	89.93	128GB, tight
UD-Q2_K_XL	95.93	128GB
UD-IQ3_XXS	117.27	128GB, little context room
UD-IQ4_XS	139.18	192GB+
UD-Q4_K_S	166.56	256GB
MXFP4_MOE	172.70	256GB

Note what the ladder says. A 1-bit or 2-bit GGUF would bring Flash to 128GB, at a quality cost nobody has measured for V2.6. Any 4-bit file stays at 192GB or above.

Why 15B Active Still Needs 256GB

Active parameters set speed. Total parameters set memory. Most people read only the first number.

Per token, the model reads 8 of 256 experts plus the attention stack, about 15B parameters. That is why decode is fast once loaded. But all 256 experts in all 48 layers must sit somewhere the runtime can reach, and they are already stored at 4 bits.

The 1M context is not the problem

We derived this from config.json. Only the 9 global-attention layers keep a full-length KV cache. The 39 sliding-window layers keep a 128-token window each.

Per token, global-attention KV = 9 layers x 4 KV heads x (192 + 128) dims x 2 bytes = 23,040 bytes, about 22.5 KB per token. At 1,000,000 tokens that is 23.04 GB, about 22.5 GiB at BF16, plus the small SWA windows.

So a full 1M context adds about 22.5 GiB on top of the weights. On a 256GB Mac, 166.8 GB of weights and runtime plus 22.5 GiB of cache is about 191 GB, and it fits. The weights alone are what push this model past 128GB and 192GB.

Runtime Support, Checked Today

We checked GitHub and Hugging Face on 21 September 2026.

Runtime	Status
SGLang, vLLM	Vendor recipes on the model card (--tp 8, --tensor-parallel-size 4)
mlx-lm	Needs the add-mimo-v2 branch from PR #1219 “Add MiMo V2.5”, open, not merged
oMLX	Vontra’s measurements used oMLX 0.7.0.dev2 with MLX 0.32.2 and mlx-lm 0.31.3
llama.cpp	MIMO2 architecture exists in the gguf constants from V2.5. No V2.6 GGUF published. A GitHub issue search for “MiMo-V2.6” in ggml-org/llama.cpp returned zero results
Ollama, LM Studio	Not checked; do not assume support

The V2.5 track record in llama.cpp: issue #28831 (13 September 2026, MiMo-V2.5-Pro hparam load error) is closed, and the MTP feature request #23924 is closed. Expect a V2.6 GGUF to need a small loader fix first, as V2.5-Pro did.

Third parties are also early. An orcarouter.ai post from 21 September 2026 notes no per-token price is published for V2.6 and no major provider serves it yet. Local is the only way to run it today outside Xiaomi’s own API.

Community Speed Numbers

We did not measure these. One report exists, from Vontra, who packaged the MLX build.

Machine	Build	Setup	Number	Source
Mac Studio M3 Ultra 256GB	Vontra/MiMo-V2.6-Flash-RL-MLX-4bit-MTP	oMLX 0.7.0.dev2, 128-token decode	59.4 tok/s	Vontra model card
same	same	Prompt processing, 512 tokens	477.1 tok/s	same
same	same	Prompt processing, 2,048 tokens	562.8 tok/s	same
same	same	Peak unified memory, short context	164.3 GB	same
same	same	Peak unified memory, 2,048-token prompt	166.8 GB	same

Vontra packaged the MTP/DFlash draft head but reports it did not beat serial decode in their test. Treat 59.4 tok/s as the plain number.

M5 Ultra estimate (derived, not measured). The M3 Ultra has 819 GB/s of memory bandwidth. Apple’s configurator lists the M5 Ultra at 1.2 TB/s. Decode on a loaded MoE is bandwidth-bound, so the ceiling scales with bandwidth: 59.4 tok/s x (1,200 / 819) = about 87 tok/s. Real numbers will land below that once the runtime overhead shows up.

To measure your own box, load the Vontra or mlx-community build in oMLX or the branch mlx-lm, then test a real prompt at the context length you plan to use.

What to Buy

If you own a 128GB Mac, a Strix Halo box or a DGX Spark: Flash does not run. The smallest build loads about 170 GB. Do not try to make it fit. Run the 9B distill below.

If you are looking at a 192GB Ryzen AI Max+ PRO 495 box: no price is public, and our wait-or-buy page puts GPU-usable memory near 160 GB. That is below the 164.3-166.8 GB measured peak. A 2-bit GGUF could change this once one exists. Today it does not.

If you are buying for Flash: the answer is a 256GB Mac Studio M5 Ultra. Prices below are from Apple’s configurator, read 21 September 2026, with deliveries from 22 September.

Machine	Price (as of 21 September 2026)	Memory bandwidth	Why pick it
Mac Studio M5 Ultra 256GB, 64-core GPU	$9,499	1.2 TB/s	Cheapest 256GB unified-memory box; runs both MLX builds with room for context
Mac Studio M5 Ultra 256GB, 80-core GPU	$10,799	1.2 TB/s	Faster prompt processing for long inputs

No affiliate link exists for either. See which Mac Studio to buy for local LLMs for the tier comparison and best local LLMs for an M5 Ultra 256GB for what else runs at this size.

Everyone else: run the 9B distill. XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B was created on 21 September 2026. It is an SFT of Qwen3.5-9B on 77.4B tokens of MiMo-generated data (27.2B loss-bearing), with 9,409,813,744 BF16 parameters. The architecture is stock Qwen3_5ForConditionalGeneration, so current llama.cpp loads it.

Xiaomi’s own table against Qwen3.5-9B (vendor runs; rows marked with † use Xiaomi’s internal evaluation sets):

Benchmark	Distill 9B	Qwen3.5-9B
SWE Verified	61.1	60.0
SWE Pro	44.6	32.0
Terminal Bench 2.1	37.1	27.0
Toolathlon-Verified	35.2	25.9
AutomationBench	30.3	5.0

The card also reports MiMo Code, Cyber, General and Visual mini sets marked †; those are internal evaluation sets, so we leave them out.

GGUF sizes, read from Hugging Face on 21 September 2026:

Repo	Quant	Size (GiB)	Fits
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF	IQ4_XS	4.87	8GB card, short context
same	Q4_K_M	5.44	8GB card, tight
same	Q5_K_M	6.40	12GB card
same	Q6_K	7.26	12GB card
ggml-org/MiMo-V2.6-Distill-Qwen-9B-GGUF	Q8_0 + mmproj	8.87 + 0.58	12GB card, short context; 16GB comfortable
bartowski	Q8_0	8.89	same
bartowski	bf16	16.69	24GB card

Prices below are as of August 2026 from our price reference and move weekly.

• 12GB path: the RTX 3060 12GB ($329-460 new) holds Q8_0 at 8.87 GiB with a short context, or Q6_K with more room.
• 16GB path: the RTX 5060 Ti 16GB ($589-805) holds Q8_0 plus the 0.58 GiB mmproj for image input plus a real context. This is the card to buy for the distill.

For the wider tier question, see the tested hardware list.

Honest Caveats

1. We did not measure tok/s. The only speed number is Vontra’s single report on one M3 Ultra. The M5 Ultra figure is a bandwidth ratio, not a measurement.
2. The MXFP4 reading is ours. The Hugging Face API reports U8 and F8_E4M3 counts; the vendor card does not label them. The arithmetic matches 309B, but confirm against config.json before you plan a build around it.
3. The GGUF ladder is an inference. It is the V2.5 file list at the same parameter count. No V2.6 GGUF existed on 21 September 2026.
4. mlx-lm support is a branch. PR #1219 is open. The add-mimo-v2 branch may change or break before merge.
5. Benchmarks are vendor-run. Both tables come from Xiaomi’s model cards. We found no third-party runs on release day.
6. The MLX builds are text-only. Image, video and audio input need the vision and audio encoders, which the MLX conversions drop.
7. Sizes are as of 21 September 2026. The model is hours old. Expect re-uploads.

Sources

• XiaomiMiMo/MiMo-V2.6-Flash-RL model card — parameters, architecture, vendor benchmarks, deployment recipes, license
• Hugging Face API: XiaomiMiMo/MiMo-V2.6-Flash-RL — creation date, stored parameter count, dtype split, file sizes
• XiaomiMiMo/MiMo-V2.6-Pro-RL — 1.02T sibling, 534.07 GiB
• mlx-community/MiMo-V2.6-Flash-RL-mxfp4-q8 — MXFP4 recipe, 4.334 bpw, 170 GB load note, branch install command
• Vontra/MiMo-V2.6-Flash-RL-MLX-4bit-MTP — measured decode, prompt processing and peak memory on a 256GB M3 Ultra
• mlx-lm PR #1219 — MiMo V2 support, open
• unsloth/MiMo-V2.5-GGUF — the V2.5 quant ladder used for the size inference
• llama.cpp issue #28831 — MiMo-V2.5-Pro hparam load error, closed
• llama.cpp issue #23924 — MTP feature request, closed
• XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B — distill training data, parameter count, benchmarks vs Qwen3.5-9B
• ggml-org/MiMo-V2.6-Distill-Qwen-9B-GGUF — Q8_0 and mmproj sizes
• bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF — IQ4_XS through bf16 sizes
• Apple Mac Studio configurator — M5 Ultra 256GB prices and bandwidth, read 21 September 2026
• orcarouter.ai blog — no published per-token price, no major provider on release day

See Also

• Which Mac Studio to buy for local LLMs — the 256GB M5 Ultra against the smaller tiers
• Best local LLMs for an M5 Ultra 256GB — what else runs at this size
• DeepSeek V4.1 Flash local requirements — the other “Flash” MoE and where it lands
• Can I run Qwen3.8 Flash Next on 128GB? — a Flash model that does fit a 128GB box
• Ryzen AI Max+ PRO 495 192GB: wait or buy? — why 192GB is still short for this model
• Best local LLM for 16GB VRAM — the tier where the 9B distill lives
• Best local LLM for an RTX 3060 12GB — the 12GB path for the distill