Run an LLM Locally Without a GPU (2026): RAM Channels Win

Bottom Line (September 2026)

• Yes, it works. llama.cpp, Ollama, and LM Studio all run models on the CPU with no graphics card.
• Memory bandwidth sets your speed, not CPU cores. The CPU reads the active weights from RAM for every token.
• Channels set bandwidth, not RAM speed. Eight channels of DDR4-2400 (~154 GB/s) beats two channels of DDR5-6000 (~96 GB/s).
• A typical desktop is dual-channel and caps near 90–100 GB/s, no matter how much RAM you add.
• Use a mixture-of-experts model. gpt-oss 20B activates only 3.6B of 20.9B parameters per token, so it runs faster than a dense 7B.
• Quantize to Q4_K_M. It cuts the bytes read per token, which raises the speed ceiling directly.
• Realistic speeds: 7B–8B dense at Q4 → 10–15 tok/s on a desktop, 15–25 tok/s on server CPUs. Small models (2B–4B) → 12–15 tok/s.

Ready to buy? See the tested hardware list with current prices.

The One Formula That Answers Everything

Rough ceiling in tokens/sec = memory bandwidth ÷ bytes read per token.

To write one token, the machine reads every active weight out of RAM. That read is the work. Everything else — cores, clock speed, cache — arrives after this ceiling is already set.

Expect 60% to 70% of the ceiling in real use, because attention, the KV cache, and sampling all cost time on top.

This formula is why more cores do not help. A 16-core CPU and a 6-core CPU on the same dual-channel board read from RAM at the same speed. Cores help with prompt processing, which is compute-bound, and that is a different problem covered in why CPU prompt processing is slow.

Channels, Not Capacity, Not Speed

This is the part most guides get wrong. Memory bandwidth scales with channel count.

Read the first and third rows together. The eight-channel DDR4 machine from years ago is about 60% faster at this job than a new DDR5-6000 desktop, because it reads from eight lanes instead of two. The DDR5 sticks are faster individually and it does not matter.

That also explains the honest gap to a GPU: 10x to 30x. A CPU is not going to feel like a graphics card. It is going to feel like a usable assistant, which is a different and achievable goal.

The practical consequence for a normal buyer: adding more RAM to a dual-channel desktop does not make your model faster. It lets you load a bigger model, which then runs slower, because bigger models read more bytes per token. Capacity and speed pull against each other on CPU.

The MoE Trick That Changes the Answer

Here is the thing almost no CPU-inference guide says plainly.

A mixture-of-experts model does not read all of its weights for each token. It routes each token to a small subset of experts and reads only those.

gpt-oss 20B has 20.9B total parameters and activates 3.6B per token — about 17% of the model, using top-4 routing across 32 experts. Its checkpoint is 12.8 GiB.

Now put both facts into the formula, on an ordinary dual-channel desktop at ~96 GB/s:

The 20B model is about twice as fast as the 7B model on the same CPU. It is three times the size on disk and it generates quicker, because it reads less per token.

The check that this is sound: the dense 7B row predicts 13–15 tok/s, and measured desktop results for dense 7B at Q4 land at 10–15 tok/s. The formula agrees with the measurements, so the MoE row is trustworthy for the same reason.

The trade you are making is capacity for speed. gpt-oss 20B needs about 13GB resident — the RAM footprint of a 20B model — while delivering the token rate of a small one. On a CPU box, where RAM is cheap and bandwidth is not, that is the right trade almost every time.

What to Actually Run

Always use Q4_K_M, or Q4_K_S when memory is tight. On CPU, quantization is not only a memory saving — it cuts the bytes read per token, so it raises the speed ceiling in direct proportion.

What to Buy, If You Are Buying

You do not need a graphics card. You do need enough RAM to hold the model you want plus its context.

32GB is the target. It holds gpt-oss 20B at ~13GB with a long context window and leaves your desktop usable. 16GB works if you stay at 8B and below. Start here: Crucial DDR5 32GB kit (2x16) — and note that installing two sticks rather than one is what gives you both channels. A single stick halves your bandwidth and therefore halves your token rate.

64GB only if you plan to run larger MoE models or keep several loaded at once: Crucial DDR5 64GB kit. It will not make any single model faster.

The CPU matters least. A current mid-range chip is fine. If you are building the machine anyway, the AMD Ryzen 9 9950X gives you strong prompt-processing speed — but understand that it is still a dual-channel platform, so your ~96 GB/s ceiling does not move.

One warning before you spend. DRAM prices roughly doubled through the 2026 shortage. Check current listings rather than any price you remember, and buy the kit rather than two separate sticks so the pair is matched.

If you want to break the dual-channel ceiling, you have to leave the desktop platform. That means a used many-channel server, which is a genuinely different build with different tradeoffs — see used EPYC servers for CPU-only MoE inference.

See Also

• Can I Run a Local LLM With 128GB RAM and No GPU? — the same question at a much larger memory tier
• Used EPYC Servers for CPU-Only MoE Inference — how to get eight memory channels
• Why Is My Local LLM So Slow? — the bandwidth math in full
• What Local LLM Fits My Machine? — pick a model for the hardware you already own
• Best Local LLM for Mac mini — the no-GPU answer on Apple silicon, where bandwidth is much higher
• Best Local LLM for 16GB VRAM — what changes once you do add a graphics card