Dual-Socket vs Single-Socket EPYC for LLM Inference: The Second CPU Pays 1.83x on a Dense Model and 1.02x on DeepSeek R1

Bottom Line

• A controlled A/B says the second socket helps. Same machine, one socket then two: Llama-3.1 70B F16 generation went 2.35 to 4.30 tok/s (182.9%) on a 2x EPYC 9175F.
• On DeepSeek R1 it does almost nothing. Same test method, 2x EPYC 9654: 8.48 to 8.67 tok/s — a 2.2% gain.
• The rule is matrix size. Mixture-of-experts models activate few parameters per token, so each matrix multiply is small and synchronisation eats the gain.
• Prompt processing always scales. It gained +45.7% to +82.6% in every case, because it is compute-bound rather than bandwidth-bound.
• “Dual socket is slower” is usually a placement bug, not a hardware verdict. A documented three-step fix took one machine from 2.4 to 4.31 tok/s, about +80%.
• We are correcting our own page. Our used EPYC server guide cited cross-machine user reports to say dual socket is slower. That comparison was not controlled. This page supersedes it.
• The buying answer for most readers is still one socket — because most large 2026 open-weight models are MoE, which is the case where the second CPU pays least.

Ready to buy? See the tested hardware list with current prices.

The Claim We Got Wrong

On 24 August 2026 we published a table of self-reported CPU inference speeds. Two rows sat next to each other. A dual EPYC 7K62 with 96 cores and 16 DIMMs ran DeepSeek-R1 Q5_K_S at 2.9 tok/s. A single EPYC 7K62 with 48 cores and 8 DIMMs ran the same file at 4.2 tok/s. We concluded that dual socket is slower and told readers to buy one socket.

The conclusion was right for the wrong reason, and the reasoning does not survive a controlled test.

Those two rows are different machines, owned by different people, configured differently. They share a CPU model and nothing else. The dual machine could have had a NUMA setting wrong, a different BIOS, a different thread count, or a cold page cache. A cross-machine comparison cannot isolate the socket count, and we treated it as if it could.

The controlled data below changes one variable at a time. Read it instead.

The Controlled Test

These numbers come from llama.cpp discussion #11733, run by fairydreaming. The method matters: the same physical machine was measured with one socket and then with both, so socket count is the only variable that moved.

Three platforms were used.

And here is what the second socket bought.

Read the last column downwards. The second CPU is worth 83% more tokens per second on a dense model, 46% more on Mixtral, and 2% more on DeepSeek R1. Same hardware. Same method. The model decides.

Why the Gain Collapses

fairydreaming states the mechanism in one line: “The smaller is multiplied matrix, the higher (relatively) is synchronization and communication overhead resulting from dual CPU usage.”

Unpack that for a mixture-of-experts model.

A dense 70B model multiplies all 70B parameters for every token. That is a large matrix, and splitting it across two CPUs gives each one plenty of work before they have to talk to each other. A MoE model routes each token to a few experts, so only a small fraction of the weights participate. The matrix per token is small. The work splits fine, but the fixed cost of synchronising two sockets does not shrink with it, so it becomes the dominant term.

The reported overhead figures line up with this. Synchronisation consumed 15.4% of computation time on dual-socket systems against 9.1% on single-socket, while memory bandwidth efficiency ranged from 73.6% to 92.3% across the platforms.

This is the part that matters for a 2026 buying decision, and no other page we have found states it plainly: the models that make a big-RAM CPU server attractive in the first place are exactly the models that waste a second socket. Kimi, DeepSeek and GLM-class releases are MoE. They are the reason you want 512GB of system memory. They are also the reason the second CPU returns 2%.

The 80% Fix for a Genuinely Slow Dual-Socket Machine

If your dual-socket box really is crawling, there is a documented recovery. It is a memory-placement problem, not a silicon problem.

The point is where the model file lands in physical memory. If the pages get placed while the machine is doing prompt processing, they end up on the wrong nodes for generation. Placing them during generation instead fixes the layout.

The procedure reported by fairydreaming on 8 February 2025, for a dual EPYC 9175F with 16 modules of DDR5-6400:

# 1. Drop the page cache, as root
echo 3 > /proc/sys/vm/drop_caches

# 2. Preload the model with the SAME numa policy and thread count you will run
llama-bench --numa distribute -t <threads> -m <model.gguf> -r 1 -p 0

# 3. Run inference with the SAME flags
llama-server --numa distribute -t <threads> -m <model.gguf>

Result on Llama-3.1-70B-Instruct at F16: 2.4 tok/s before, 4.31 tok/s after — about +80%.

Two things to keep straight. The --numa distribute flag and the thread count must match between the preload and the run, or the placement you just paid for is discarded. And step 1 needs root, because dropping the page cache is a system-wide action.

Check your BIOS as well. Both controlled dual-socket platforms above ran NPS1. If your board defaults to a higher NPS value, you are subdividing memory into more NUMA nodes than the tested configuration used.

So: One Socket or Two?

For most readers of this site the answer is one socket, because the reason to build a CPU inference box in 2026 is to run a model that fits nowhere else, and those models are MoE. That is the same recommendation our used EPYC server guide gave. The evidence behind it is now different, and better.

We do not stock an affiliate link for a single-socket EPYC board and we are not going to invent one. These are used and refurbished parts, and the listings turn over weekly. Buy from a seller who states the BIOS version and the exact DIMM population.

Honest Limits of This Data

• The controlled figures are one contributor’s measurements on three specific platforms. They are far better than the cross-machine reports they replace, and they are still not an independent benchmark suite.
• P1 and P2 use different CPUs, memory speeds and channel counts, so do not compare rows across platforms. Only compare one socket against two within a row.
• The 80% recovery figure was measured on a dense F16 model on P1. We have not seen the same procedure measured on an MoE model, so treat the size of the gain as platform-specific and the technique as general.
• All figures here are from 2025 llama.cpp reports. The code has moved since. Re-measure on your own build before you spend money on the strength of a percentage.

Before you order parts, check the tested hardware list for current prices by tier.

See Also

• Used EPYC Servers for CPU-Only MoE Inference — the build this page tunes, and the page this one corrects
• Can I Run a Local LLM on 128GB of RAM With No GPU? — the small end of CPU inference
• Dense 70B vs MoE 120B Locally — why the model architecture decides your hardware
• Open Weights Are Not Local Any More — why anyone considers a 512GB server at all
• Why Is My Local LLM So Slow? — the bandwidth argument underneath all of this
• Ollama vs llama.cpp — --numa distribute is a llama.cpp flag, and this is the difference

Sources

• Dual Epyc Genoa/Turin token generation performance bottlenecks (llama.cpp discussion #11733) — all one-socket vs two-socket figures, platform definitions, the matrix-size quotation, and the 15.4% vs 9.1% synchronisation overhead; read 2026-08-25
• Possible solution for poor token generation performance on dual Epyc Genoa/Turin systems (llama.cpp issue #11744) — the drop_caches plus --numa distribute preload procedure and the 2.4 to 4.31 tok/s result, fairydreaming, 8 February 2025; read 2026-08-25
• Inference LLM DeepSeek-V3 671B on CPU only (llama.cpp discussion #11765) — the self-reported cross-machine figures our earlier page relied on, retained here only to explain why they are not conclusive
• LLaMA NUMA could be better (llama.cpp issue #1437) — background on NUMA handling in llama.cpp

All percentages above are quoted as reported in the source discussion. We did not re-run any benchmark.