Is 16GB VRAM Enough for Edge0 145B (2026)? Yes, on NVIDIA
Edge0 145B is a 145B mixture-of-experts model that its makers say runs on one NVIDIA GPU with 16 GiB of memory. We read the release notes, both Hugging Face model repos and the speed table. The 16GB answer is yes, with one cost the headline numbers hide: prompt processing.
Bottom Line
- Yes, 16GB of VRAM runs Edge0 145B, on an NVIDIA card. The v0.2.0 release (2026-11-11) asks for compute capability 7.5 or newer, at least 16 GiB of GPU memory, 20 GiB of system memory and CUDA Toolkit 13.
- Decode survives the squeeze; prompts do not. At the 16 GiB budget, chat decode is 36.8 tok/s, 2.7x slower than the same card unconstrained. Prefill on a 16K prompt drops 10.6x, from 3,554.9 to 335 tok/s.
- The 16 GiB row is not a 16GB card. Edge0 measured an RTX 5090 D 24GB held to 16 GiB VRAM and 20 GiB RAM by a cgroup. A real 16GB card is slower; nobody has published one yet.
- No Mac, no Linux binary. The release packages are Windows only. Linux users build from source. The M2 Ultra row on the model card is not a macOS release.
| Your GPU | Edge0 145B? | Evidence |
|---|---|---|
| 16GB NVIDIA (RTX 5060 Ti, 4060 Ti, 5070 Ti) | Yes, slow prompts | Simulated 16 GiB row: 36.8 tok/s decode, 335 tok/s 16K prefill |
| 24GB (RTX 3090, 4090) | Yes, comfortable | RTX 4090 Q3_K: 69.9 tok/s decode, 3,179.4 tok/s 16K prefill |
| 32GB (RTX 5090) | Yes | Measured on the 5090 D 24GB only |
| Mac, AMD, Intel | No release | CUDA only in v0.2.0 |
What Edge0 145B Is
Edge0 145B is a sparse mixture-of-experts model built on Qwen3.8-Flash-Next (the base model named on the Hugging Face card). It is made to keep only part of itself on the GPU.
| Spec | Value (model card) |
|---|---|
| Parameters | 145B total, 10.12B active per token |
| Layers | 48 |
| Experts | 512 per layer, 10 routed + 1 shared active |
| Expert pool | 84.6B parameters |
| PLE table | 51.2B n-gram embedding table, fp8, memory-mapped |
| Draft head (MTP) | 1.26B, for speculative decode |
| Context | 256K, KV cache int8 with 32K resident |
| Builds | Q3_K 40.81 GiB, NVFP4 47.79 GiB (expert weights) |
| Engine | edge0-strata (models), edge0-infer (CLI, server, Edge0 Chat app) |
The trick is that the GPU holds the hot experts and the rest stays in system RAM. The card says the cost “tracks how many experts are resident and not how big the model is”. That is why a 145B model starts on 16 GiB.
Speeds From Edge0’s Own Table
All rows are vendor measurements from the Edge0-145B-NVFP4 card, read 2026-11-12. In tok/s:
| Machine | Build | 4K prefill | 16K prefill | Chat decode | Agent decode |
|---|---|---|---|---|---|
| RTX 5090 D 24GB | Q3_K | 2,169.9 | 3,554.9 | 99.7 | 121.2 |
| RTX 5090 D 24GB | NVFP4 | 1,839.6 | 3,165.7 | 79.6 | 98.3 |
| RTX 4090 24GB | Q3_K | 2,042.0 | 3,179.4 | 69.9 | 91.7 |
| RTX Spark N1X 60 GiB | NVFP4 | 821.2 | 922.4 | 40.1 | 46.0 |
| Apple M2 Ultra (Metal) | Q3_K | 755.6 | 800.3 | 60.9 | 57.0 |
| 16 GiB VRAM + 20 GiB RAM | Q3_K | 249 | 335 | 36.8 | 44.9 |
What the 16 GiB row means for an agent. A coding agent sends a long prompt every turn. A 16K-token prompt at 335 tok/s takes about 49 seconds before the first output token. On the unconstrained 5090 D it takes about 4.6 seconds (arithmetic from the table). Chat feels fine on 16GB; an agent loop feels slow.
Pick Q3_K, not NVFP4, on GeForce cards. Q3_K was faster on the 5090 D in every column, and its expert weights are 7 GiB smaller.
Disk and RAM: the Parts People Miss
The download is much bigger than the GPU budget. File sizes from the Hugging Face API, read 2026-11-12:
| Repo | Total files | Largest file |
|---|---|---|
| Edge0-145B-q3k | 105.37 GB | ple-fp8.gguf 51.20 GB |
| Edge0-145B-NVFP4 | 112.87 GB | ple-fp8.gguf 51.20 GB |
In the q3k repo, experts.bin is 40.08 GB and model-skeleton.gguf 5.99 GB. Half the download is the memory-mapped PLE table, so it reads from your SSD while you run. Put the model on an NVMe drive with about 110 GB free.
System RAM. The release asks for 20 GiB. With residency on, the 24GB card held the Q3_K build as 23.14 GiB VRAM plus 31.28 GiB host RAM, and NVFP4 as 23.25 + 38.27 GiB. Our read: 20 GiB is the floor, and 32GB of RAM is the sensible minimum for a desktop that also runs other programs.
What to Buy for Edge0 145B
Prices are US street prices from our price tracker, as of November 2026. They moved a lot this year.
- Already own a 16GB NVIDIA card: run it. Start with the Q3_K build and short prompts.
- Buying new on a budget: the RTX 5060 Ti 16GB meets the minimum. GPUPrix listed it from $800 on 4 November 2026. Expect it to be slower than Edge0’s simulated 16 GiB row, which ran on a 5090 D’s memory bus.
- For agent use: a 24GB card. A used RTX 3090 averages about $1,607 on eBay (resaleprices.com, 10 November 2026; the Amazon link opens a Renewed card) (compute capability 8.6, so it qualifies, but Edge0 has not published a 3090 row). The RTX 4090 has the measured 3,179 tok/s prefill; used ones average about $3,426.
- RTX 5090: listings sit around $6,400-6,600. Edge0 measured the 24GB RTX 5090 D, not the 32GB RTX 5090, so the extra 8GB is untested.
Limits Before You Download
- Vendor numbers only. On 2026-11-12, every speed and score came from Edge0. The Hacker News thread (116 points) questioned the benchmarks. Measure your own card with
edge0-infer bench. - Windows only as a package. Linux users follow the build steps in
edge0-strata/EDGE0.md. - License. Personal, research and internal use are allowed under the Edge0 Community Model License 1.0. Selling the model or running a commercial model service needs a written Edge0 license, and the Qwen Community License applies too.
- New engine. It does not load in llama.cpp, Ollama or LM Studio. edge0-infer serves an OpenAI-compatible API and an Anthropic Messages API (
/v1/messages), so Claude Code-style tools can point at it.
Sources
- Edge0 v0.2.0 release notes, GitHub, published 2026-11-11 (requirements, packages, APIs).
- Hugging Face model cards and API file lists: Edge0/Edge0-145B-NVFP4 and Edge0/Edge0-145B-q3k (created 2026-11-11), read 2026-11-12.
- Hacker News item 50475713, “Edge0-145B: Frontier AI Capabilities on Your Own Computer”, 2026-11-12.
- OpenClaw DC price reference: RTX 5060 Ti 16GB (GPUPrix 11-04), used RTX 3090 and 4090 (resaleprices, Nov 10), RTX 5090 (GPU Poet, November).
See Also
- Is 16GB of VRAM still enough in 2026?: the tier-level verdict
- Edge0 35B-A3B on a 16GB Mac mini: the earlier, Mac-only Edge0 release
- Can a 24GB Mac run Qwen 3.8 Flash Next?: the base model on Apple silicon
- Best local LLM for 16GB VRAM: the model picks for this tier
- Best local LLM for the RTX 5060 Ti 16GB: per-card picks
- Best local LLM for an RTX 4090: the 24GB tier
Before you buy a card for this, check the tested hardware list for current prices by tier.