← All guides

Is 16GB VRAM Enough for Edge0 145B (2026)? Yes, on NVIDIA

Edge0 145B is a 145B mixture-of-experts model that its makers say runs on one NVIDIA GPU with 16 GiB of memory. We read the release notes, both Hugging Face model repos and the speed table. The 16GB answer is yes, with one cost the headline numbers hide: prompt processing.

Bottom Line

  • Yes, 16GB of VRAM runs Edge0 145B, on an NVIDIA card. The v0.2.0 release (2026-11-11) asks for compute capability 7.5 or newer, at least 16 GiB of GPU memory, 20 GiB of system memory and CUDA Toolkit 13.
  • Decode survives the squeeze; prompts do not. At the 16 GiB budget, chat decode is 36.8 tok/s, 2.7x slower than the same card unconstrained. Prefill on a 16K prompt drops 10.6x, from 3,554.9 to 335 tok/s.
  • The 16 GiB row is not a 16GB card. Edge0 measured an RTX 5090 D 24GB held to 16 GiB VRAM and 20 GiB RAM by a cgroup. A real 16GB card is slower; nobody has published one yet.
  • No Mac, no Linux binary. The release packages are Windows only. Linux users build from source. The M2 Ultra row on the model card is not a macOS release.
Your GPUEdge0 145B?Evidence
16GB NVIDIA (RTX 5060 Ti, 4060 Ti, 5070 Ti)Yes, slow promptsSimulated 16 GiB row: 36.8 tok/s decode, 335 tok/s 16K prefill
24GB (RTX 3090, 4090)Yes, comfortableRTX 4090 Q3_K: 69.9 tok/s decode, 3,179.4 tok/s 16K prefill
32GB (RTX 5090)YesMeasured on the 5090 D 24GB only
Mac, AMD, IntelNo releaseCUDA only in v0.2.0

What Edge0 145B Is

Edge0 145B is a sparse mixture-of-experts model built on Qwen3.8-Flash-Next (the base model named on the Hugging Face card). It is made to keep only part of itself on the GPU.

SpecValue (model card)
Parameters145B total, 10.12B active per token
Layers48
Experts512 per layer, 10 routed + 1 shared active
Expert pool84.6B parameters
PLE table51.2B n-gram embedding table, fp8, memory-mapped
Draft head (MTP)1.26B, for speculative decode
Context256K, KV cache int8 with 32K resident
BuildsQ3_K 40.81 GiB, NVFP4 47.79 GiB (expert weights)
Engineedge0-strata (models), edge0-infer (CLI, server, Edge0 Chat app)

The trick is that the GPU holds the hot experts and the rest stays in system RAM. The card says the cost “tracks how many experts are resident and not how big the model is”. That is why a 145B model starts on 16 GiB.

Speeds From Edge0’s Own Table

All rows are vendor measurements from the Edge0-145B-NVFP4 card, read 2026-11-12. In tok/s:

MachineBuild4K prefill16K prefillChat decodeAgent decode
RTX 5090 D 24GBQ3_K2,169.93,554.999.7121.2
RTX 5090 D 24GBNVFP41,839.63,165.779.698.3
RTX 4090 24GBQ3_K2,042.03,179.469.991.7
RTX Spark N1X 60 GiBNVFP4821.2922.440.146.0
Apple M2 Ultra (Metal)Q3_K755.6800.360.957.0
16 GiB VRAM + 20 GiB RAMQ3_K24933536.844.9

What the 16 GiB row means for an agent. A coding agent sends a long prompt every turn. A 16K-token prompt at 335 tok/s takes about 49 seconds before the first output token. On the unconstrained 5090 D it takes about 4.6 seconds (arithmetic from the table). Chat feels fine on 16GB; an agent loop feels slow.

Pick Q3_K, not NVFP4, on GeForce cards. Q3_K was faster on the 5090 D in every column, and its expert weights are 7 GiB smaller.

Disk and RAM: the Parts People Miss

The download is much bigger than the GPU budget. File sizes from the Hugging Face API, read 2026-11-12:

RepoTotal filesLargest file
Edge0-145B-q3k105.37 GBple-fp8.gguf 51.20 GB
Edge0-145B-NVFP4112.87 GBple-fp8.gguf 51.20 GB

In the q3k repo, experts.bin is 40.08 GB and model-skeleton.gguf 5.99 GB. Half the download is the memory-mapped PLE table, so it reads from your SSD while you run. Put the model on an NVMe drive with about 110 GB free.

System RAM. The release asks for 20 GiB. With residency on, the 24GB card held the Q3_K build as 23.14 GiB VRAM plus 31.28 GiB host RAM, and NVFP4 as 23.25 + 38.27 GiB. Our read: 20 GiB is the floor, and 32GB of RAM is the sensible minimum for a desktop that also runs other programs.

What to Buy for Edge0 145B

Prices are US street prices from our price tracker, as of November 2026. They moved a lot this year.

  • Already own a 16GB NVIDIA card: run it. Start with the Q3_K build and short prompts.
  • Buying new on a budget: the RTX 5060 Ti 16GB meets the minimum. GPUPrix listed it from $800 on 4 November 2026. Expect it to be slower than Edge0’s simulated 16 GiB row, which ran on a 5090 D’s memory bus.
  • For agent use: a 24GB card. A used RTX 3090 averages about $1,607 on eBay (resaleprices.com, 10 November 2026; the Amazon link opens a Renewed card) (compute capability 8.6, so it qualifies, but Edge0 has not published a 3090 row). The RTX 4090 has the measured 3,179 tok/s prefill; used ones average about $3,426.
  • RTX 5090: listings sit around $6,400-6,600. Edge0 measured the 24GB RTX 5090 D, not the 32GB RTX 5090, so the extra 8GB is untested.

Limits Before You Download

  1. Vendor numbers only. On 2026-11-12, every speed and score came from Edge0. The Hacker News thread (116 points) questioned the benchmarks. Measure your own card with edge0-infer bench.
  2. Windows only as a package. Linux users follow the build steps in edge0-strata/EDGE0.md.
  3. License. Personal, research and internal use are allowed under the Edge0 Community Model License 1.0. Selling the model or running a commercial model service needs a written Edge0 license, and the Qwen Community License applies too.
  4. New engine. It does not load in llama.cpp, Ollama or LM Studio. edge0-infer serves an OpenAI-compatible API and an Anthropic Messages API (/v1/messages), so Claude Code-style tools can point at it.

Sources

  • Edge0 v0.2.0 release notes, GitHub, published 2026-11-11 (requirements, packages, APIs).
  • Hugging Face model cards and API file lists: Edge0/Edge0-145B-NVFP4 and Edge0/Edge0-145B-q3k (created 2026-11-11), read 2026-11-12.
  • Hacker News item 50475713, “Edge0-145B: Frontier AI Capabilities on Your Own Computer”, 2026-11-12.
  • OpenClaw DC price reference: RTX 5060 Ti 16GB (GPUPrix 11-04), used RTX 3090 and 4090 (resaleprices, Nov 10), RTX 5090 (GPU Poet, November).

See Also

Before you buy a card for this, check the tested hardware list for current prices by tier.

Read next

MiniMax H3 Local Requirements: Yes, 16GB VRAM Works (2026)
MiniMax H3 is a 33B audio-video model plus a Qwen3-VL-32B text encoder. A 16GB card runs ComfyUI's pruned INT8 files with offload. 24GB holds the video model. System RAM and the license territory matter more than most guides say.
Can I Run LTX-2.5 Locally? VRAM by GPU (2026)
LTX-2.5 is a 22B video model plus a 12B Gemma 4 text encoder. The official INT8 files need a 24GB card. 16GB works with NVFP4 or GGUF and weight streaming. The '16GB minimum' leans on cloud text encoding. Ollama does not run it.
Is 16GB of VRAM Still Enough for Local AI in 2026?
16GB is the most-sold VRAM tier of 2026 — and the best agentic models just moved to 20-27B. Where 16GB still works, the exact lines where it stops, and when to pay for 24GB.
Best Local LLM for RTX 4080 & 4080 Super: 16GB VRAM Picks
The best local LLM for the RTX 4080 and 4080 Super (16GB). 2026 picks with quants, tokens/sec, what fits in 16GB, and OpenClaw setup.