Is a Used V100 32GB Worth It? (2026): Yes, on CUDA 12

Bottom Line

Yes, if you run llama.cpp on CUDA 12 and need 32GB per card on a budget. No, if you want vLLM, current CUDA, or a quiet desktop card.

• What you get: 8GB more than a 3090 for about $590 less, at almost the same bandwidth.
• What you give up: current software. The V100 is compute capability 7.0. CUDA 13 and vLLM have both left it behind.
• The price is rising fast: the lowest ask went from $624.95 on 31 July to $909.95 on 3 November. That is +46% in three months.

Why It Is Trending

On 2 November 2026, Hacker News carried “V100 SXM2 pinout and NVLink reference” (43 points). The same day, in “Ask HN: What hardware are you using to run local LLMs?” (47 points), one reader described 2x V100 32GB in a Dell T5820 with 64GB of RAM, “around $1800 USD” all in. Another commenter in the pinout thread runs four V100S cards. The interest is real: the V100 is the last cheap 32GB NVIDIA card.

Specs That Matter

From NVIDIA’s V100 datasheet:

Bandwidth is why it still works. Token generation reads every active weight once per token. At 900 GB/s, the V100 is within 4% of an RTX 3090.

Precision is why it is aging. Volta’s tensor cores do FP16. The datasheet lists no BF16, FP8 or FP4 mode. So NVFP4 and FP8 checkpoints get no hardware help on it. GGUF K-quants do not need those modes, which is why llama.cpp is the right engine for this card. See MXFP4 vs NVFP4 for the 4-bit float formats.

What It Runs (Community Numbers)

None of these numbers is ours. They do not transfer between machines, builds or settings.

How to read them:

• A dense 27B at 4 bits is the sweet spot. About 33 tok/s is a comfortable chat speed, and the file fits in 32GB with room for context.
• MoE models fly. With about 3B active parameters, a 35B MoE decodes above 100 tok/s.
• The 55 tok/s HN figure is higher than the 33 tok/s run. The HN reader did not state the settings. A draft model or MTP would explain the gap; we do not know which.

Where the Software Stops

This is the part most “cheap V100” posts leave out.

What to do: install a CUDA 12.x toolkit and keep it. Build llama.cpp against it. Do not let a distro update move you to CUDA 13 without a plan. One community build page states that NVIDIA’s R580 driver branch is the last for Volta; we could not confirm that on an NVIDIA page.

PCIe or SXM2?

The SXM2 route saves money but costs work. The module plus a basic board can come in under a PCIe card. But a 300W module on a cooler built for about 140W will throttle. Budget for a real heatsink and fans. Our quiet cooling guide covers 24/7 airflow.

Both versions are passive. A desktop case with no front-to-back airflow will overheat a V100. Plan a blower shroud or a server chassis before you buy.

What Else the Same Money Buys

For a first local AI card, a used RTX 3090 is still the safer buy: it runs every current engine. The V100 is the better buy only when 32GB per card matters more than software support. That is true for Qwen3.8 27B at Q8_0 (27.05 GiB), which does not fit a 24GB card. It is also true for two cards: 64GB for about $1,820 at the 3 November lowest asks, against $3,000 for 48GB from two 3090s. For every tier, see the cards this matters for.

Before You Buy a Used V100

1. Ask for an nvidia-smi screenshot that shows the 32GB memory size and the serial. Many listings mix 16GB and 32GB cards.
2. Check the form factor. “SXM2” needs an adapter board. “PCIe” fits a slot.
3. Plan the airflow before the card arrives.
4. Pin CUDA 12.x on the machine.
5. Run one benchmark in the return window:

llama-bench -m model-Q4_K_M.gguf -ngl 99 -fa 1 -p 512 -n 128

Compare your decode number with the table above. A card far below it may throttle or have a fault.

Sources

• NVIDIA V100 datasheet (memory, bandwidth, power, interconnect, tensor TFLOPS), read 2026-11-03; RTX 3090 bandwidth and power as on our RTX 3090 page
• Prices: GPU Dojo V100 and RTX 3090 eBay trackers (asking prices, read 2026-11-03); AMCompute appraisals for V100 PCIe 32GB, V100 SXM2 32GB, V100 PCIe 16GB and MI50 32GB (sold prices, data to August 2026); resaleprices.com RTX 3090 average (2026-10-30); adapter board prices from eBay search results (undated asks)
• llama.cpp ggml/src/ggml-cuda/CMakeLists.txt (70-virtual only when CUDA < 13), read 2026-11-03; llama.cpp discussion #15013 (V100 rows)
• NVIDIA CUDA Toolkit release notes (cuFFT 13.0, cuSPARSE 13.0 Volta removal); vLLM GPU installation docs (compute capability 7.5+)
• Atlas3D/llama.cpp-sm70-cuda12.9 on Hugging Face (llama-bench rows, flash-attention patch note), released 2026-10-20
• Strata community benchmark list, V100-SXM2-32GB report, 2026-10-08
• Hacker News items 50344912 and 50337431, both 2026-11-02

See Also

• Is a used RTX 3090 worth it? — the mainstream 24GB alternative
• Is the AMD MI50 32GB worth it? — the cheaper 32GB HBM2 card on ROCm
• Is NVLink worth it for local LLMs? — what the SXM2 link does and does not speed up
• Best local LLM for 32GB VRAM — what to run once the card works
• Best hardware for Strata — the engine with a Volta community build
• Cheapest 32GB VRAM GPU — the new 32GB cards compared