Can I Run GLM-5.3-Flash Locally (2026): 128GB Minimum

Bottom Line

• 128GB is the floor. The smallest published GGUF, UD-IQ1_S, is 86.69 GiB. The smallest one worth running, UD-IQ2_XXS, is 94.85 GiB.
• No single consumer GPU runs this. 86.69 GiB is nearly 3x an RTX 5090’s 32GB.
• Stock llama.cpp and Ollama cannot load it today. The support PR, ggml-org/llama.cpp#27754, is open and unmerged as of 15 September 2026.
• 18B active parameters means it decodes fast once loaded. The constraint is capacity, not bandwidth — the opposite of a dense 70B.
• The fact nobody else prints: the Q8_0 GGUF (317.56 GiB) is larger than the original FP8 checkpoint (305.8 GiB). The model ships natively in FP8, so 8-bit GGUF adds scale data and recovers nothing.

Every Published Quant, Measured

Sizes read from the Hugging Face file listing for unsloth/GLM-5.3-Flash-GGUF on 15 September 2026. Multi-part files are summed.

Two non-GGUF checkpoints matter for the same decision:

Read the Q8_0 Row Again

This is the part that catches people who quantize by habit.

The Hugging Face API reports GLM-5.3-Flash as 321,323,031,390 parameters, of which 314,396,639,232 are stored as F8_E4M3 and only 6,926,096,640 as BF16. The model was released in FP8. It was not converted down to it.

So the usual ladder inverts at the top. Q8_0 is 317.56 GiB against a 305.8 GiB original. You pay about 12 GiB for per-block scale metadata and receive no quality back, because there was no 16-bit precision there to preserve. The BF16 GGUF at 597.57 GiB is worse still: it is an upcast that nearly doubles the file while carrying exactly the information the FP8 weights already had.

If you want maximum quality, run the native FP8 checkpoint. Quantizing is for getting smaller, and on this model that means starting at Q6_K_XL and below.

The Runtime Problem, Checked Tonight

The model card lists its supported frameworks explicitly: SGLang, vLLM, TokenSpeed, Transformers, KTransformers and Unsloth. llama.cpp and Ollama are not on that list.

Unsloth publishes the GGUFs anyway, and its own repository README says to use its llama.cpp pull request or the Unsloth Desktop app. We checked that PR through the GitHub API on 15 September 2026:

• Title: model: add GLM-5-Next (GLM-5.3-Flash)
• Number: 27754, opened 26 August 2026
• State: open, merged: false
• Last updated 11 September 2026, 43 commits, 42 changed files

That is an active branch, not an abandoned one, so this paragraph has a shelf life. Check the PR before you assume either way. But today, downloading UD-IQ2_XXS and pointing stock llama-server or ollama run at it will not work. This is the single most common way to waste 95 GiB of bandwidth this month.

What This Means For Buying

Be careful here, because the honest answer is not “buy a box”.

If you already own 128GB of unified memory, you can run UD-IQ2_XXS through a PR build or Unsloth Desktop, and the 18B active parameter count means it will feel quick. That is a real reason to try it this week.

If you do not, wait. Two things should land before money moves: the llama.cpp merge, and community quality reports on the 1-bit and 2-bit files. A 2-bit quant of a 321B model is a genuinely different product from the hosted version, and nobody has published a KL-divergence table for these files yet. We will not guess at one.

A 24GB or 32GB card is not a path here at any quant. If that is your hardware, the right move is a model sized for it. See Best 20B-35B local LLMs instead.

For how memory capacity maps to model size in general, the local LLM estimator and what local LLM fits my machine answer the tier question directly, and the gear page lists the machines at each memory tier.

Why 18B Active Changes the Shape of the Problem

A dense 70B reads roughly 40GB of weights for every token, so memory bandwidth sets its speed. GLM-5.3-Flash reads only the 18B of active parameters per token. At a 2-bit quant that is a few gigabytes per token rather than tens.

The practical consequence: on a 128GB box with about 250-270 GB/s of bandwidth, this model is not bandwidth-starved the way a dense 70B is on the same hardware. What limits you is fitting it at all, and then how much memory is left for the KV cache. GLM-5.3-Flash advertises a 1M-token context, and a context that long will not fit alongside a 95 GiB model on a 128GB machine. Plan for a short context and read how to check what your runtime allocated before you assume a number.

This is the same shape as the DeepSeek V4 Flash and gpt-oss 120B decision, one tier up in capacity.

Honest Caveats

1. We did not measure tokens per second. No primary benchmark for GLM-5.3-Flash on a 128GB box has been published that we could read tonight, and we will not repeat an unsourced figure.
2. We did not quote an API price. The Z.ai documentation page we read lists a 1M context and 128K max output but no per-million rate we could verify. Check the vendor’s pricing page directly.
3. Quant quality is unmeasured. The sizes above are facts. Which of UD-IQ1_S, UD-IQ2_XXS and UD-Q2_K_XL stays coherent for real work is not yet documented.
4. Sizes are as of 15 September 2026. Unsloth re-uploads quant families when recipes improve.

See Also

• Can you run Kimi K3 locally? — the same arithmetic on a 2.8T-parameter model, and the answer is no at every home tier
• Kimi K3 vs DeepSeek V4 vs GLM-5.2 — where the previous GLM generation landed
• DeepSeek V4 Flash vs gpt-oss 120B — the big-MoE pick from 64GB to 192GB
• Best local LLM for 128GB VRAM — what else fits at this tier
• IQ4_XS vs Q4_K_M vs Q4_K_XL — how Unsloth’s UD quants differ from the built-ins
• NVFP4 vs MXFP4 — what the NVIDIA NVFP4 checkpoint above actually is
• Quantization in plain English — why a 2-bit 321B is not simply “a smaller model”