Do I need a different GPU for fine-tuning than for inference?

Often yes. Inference stores one thing in VRAM: the weights, plus a KV cache. Speed is set by memory bandwidth. Full fine-tuning stores four things: weights, gradients, optimizer states and activations, which is roughly 16 bytes per parameter with Adam versus 2 bytes for FP16 inference. That makes fine-tuning capacity-bound rather than bandwidth-bound. A card that is excellent at one job can be mediocre at the other.

How much VRAM do I need to fine-tune a 7B model?

It depends entirely on the method. Full fine-tuning a 7B model with the Adam optimizer needs roughly 16 bytes per parameter, about 112GB, so it does not fit on any single consumer card. LoRA freezes the base model in FP16 (about 14GB for 7B) and trains small adapters, which fits comfortably on 24GB. QLoRA loads the base in 4-bit (about 3.5-4GB for 7B) and fits on 12GB or less. The QLoRA paper demonstrated fine-tuning a 65B model on a single 48GB GPU while preserving 16-bit fine-tuning task performance.

Is the RTX 3090 still worth buying for fine-tuning in 2026?

For multi-GPU fine-tuning specifically, yes, because of one feature no newer consumer card has. The RTX 3090 is the last GeForce card with NVLink. NVIDIA dropped it on the RTX 4090 and did not bring it back for the RTX 5090. Fine-tuning across two cards moves gradients and optimizer state between them constantly, which is exactly the traffic NVLink was built for. Be aware the price premise has changed: used 3090s traded around $1,000-1,300 as of August 2026, not the $650-750 still quoted across older guides.

Is the RTX 5090 good for fine-tuning?

It is an excellent single-card fine-tuning GPU and a poor scaling one. Its 32GB of GDDR7 is the most VRAM on any current GeForce card, so single-card QLoRA on larger models works well. But it has no NVLink, so two 5090s must exchange gradients over PCIe. If your plan is to add a second card later to train bigger models, that plan is weaker on the 5090 than it was on the 3090. As of August 2026 the 5090 also trades at roughly $4,300-5,000, far above its $1,999 MSRP.

Should I just rent a cloud GPU for fine-tuning instead?

For most people, yes, and this guide will say so plainly. Fine-tuning is bursty: you do it for a few hours and then stop. Inference is continuous. Buying a card sized for your occasional fine-tuning job means paying 2026 shortage prices for capacity that sits idle most of the time. The economically sane split for most local-AI users is to buy for your inference workload and rent capacity for the rare full fine-tune.

← All guides

Hardware August 17, 2026

Best GPU for Fine-Tuning vs Inference (August 2026): Why the Answer Flips

Most GPU guides give you one ranking and assume it covers both jobs. It does not. Inference is bandwidth-bound and single-card friendly, so the RTX 5090's 1.8 TB/s wins. Fine-tuning is capacity-bound and interconnect-sensitive, and NVIDIA removed NVLink from consumer cards after the RTX 3090 — so a pair of used 3090s with an NVLink bridge does something no pair of 5090s can. Here is the memory math for each job, and what to actually buy in August 2026.

Bottom Line (August 2026)

If you only run models: buy for bandwidth. The RTX 5090 32GB is the fastest consumer inference card, and its 32GB is the largest GeForce frame buffer.
If you fine-tune across two cards: NVLink is the whole argument. The RTX 3090 24GB is the last GeForce card with it. Two of them behave differently from two of anything newer.
If you fine-tune on one card: capacity beats everything. 24GB runs LoRA on 7B–13B comfortably and QLoRA far above that.
Most people should not buy a fine-tuning GPU at all. Fine-tuning is bursty; inference is continuous. Buy for the continuous job and rent for the bursty one.
The honest caveat: at August 2026 prices, almost every “buy a second card to train bigger models” plan loses to renting. The shortage changed this math, and most guides have not updated.

The Two Jobs Are Not the Same Shape

Inference puts one large thing in VRAM: the weights. Add a KV cache that grows with context. Once the model fits, speed is set almost entirely by memory bandwidth, because generating each token requires reading every active weight.

Training puts four things in VRAM:

the weights,
the gradients,
the optimizer states (Adam keeps two running moments per trainable parameter),
the activations saved for the backward pass.

That is the entire reason the buying advice diverges. One job is a bandwidth problem. The other is a capacity problem.

The memory math, derived

The figures below are arithmetic, not benchmarks — bytes per parameter multiplied out. Real usage runs higher because of activations, batch size, sequence length and framework overhead.

Method	What is stored per parameter	Bytes/param	7B model	13B model
Inference, FP16	weights	2	~14GB	~26GB
Inference, Q4	weights	~0.5	~3.5–4GB	~7GB
Full fine-tune, Adam	FP16 weights + FP32 master + 2 FP32 moments + gradients	~16	~112GB	~208GB
LoRA	frozen FP16 base + tiny adapters	~2 + ε	~14GB + overhead	~26GB + overhead
QLoRA	frozen 4-bit base + FP16 adapters	~0.5 + ε	~4GB + overhead	~7GB + overhead

Two things fall out of that table immediately.

Full fine-tuning is off the consumer table. A 7B full fine-tune wants roughly 112GB. That is not a 5090 problem or a 3090 problem — it does not fit on any single card you can buy at a desk, including the 96GB workstation part. This is why essentially all local fine-tuning is LoRA or QLoRA.

QLoRA collapses the requirement to inference-scale. The QLoRA paper (May 2023) demonstrated fine-tuning a 65B model on a single 48GB GPU while preserving 16-bit fine-tuning task performance. That result is why a 24GB card is a genuinely capable fine-tuning machine in 2026, and why “you need a datacenter GPU to fine-tune” is out of date.

The NVLink Fact That Reverses the Ranking

Here is the thing almost no buying guide tells you, because it requires knowing what the cards do together rather than what they score alone.

The RTX 3090 is the last consumer GPU NVIDIA ever shipped with NVLink. NVIDIA restricted NVLink to the 3090 within the 30-series, then removed it on the RTX 4090 and did not restore it on the RTX 5090 or anything else in Ada or Blackwell GeForce.

For inference, this barely matters. Splitting a model across two cards layer-by-layer sends only a thin activation tensor across the boundary once per token. PCIe handles that fine.

For fine-tuning, it matters a great deal. Data-parallel training synchronises gradients across cards on every step, and the volume scales with the number of trainable parameters. That is precisely the traffic NVLink exists to carry, and it is why the two-card story inverts:

Setup	Total VRAM	Inference	Fine-tuning at 2×
1× RTX 5090	32GB	Best single-card	Good, capacity-limited
2× RTX 5090	64GB	Excellent	Gradient sync over PCIe only
1× RTX 3090	24GB	Fine, older bandwidth	Good — the QLoRA workhorse
2× RTX 3090 + NVLink bridge	48GB	Fine	The only consumer NVLink option
1× RTX PRO 6000 96GB	96GB	Excellent	Best single-card, no bridge needed

The line nobody else prints: NVIDIA’s consumer multi-GPU training story got worse over three generations, not better. If you are comparing a 3090 pair against a 5090 pair for training, you are comparing across a feature removal — not a normal generational upgrade. Note also that an NVLink bridge is a separate purchase and we do not stock one; buy it from a reputable seller and confirm the 3-slot/4-slot spacing matches your board.

What Each Card Actually Costs Right Now

Prices below are US street, as of August 2026, during the DRAM/GDDR7 shortage. Treat them as ranges, not quotes, and re-check before buying.

Card	VRAM	Street price (Aug 2026)	Commonly quoted (wrong)
RTX 3090 (used)	24GB	$1,000–1,300	”$650–750”
RTX 4090	24GB	~$2,500–3,700	”$1,599 MSRP”
RTX 5090	32GB	$4,300–5,000+	“$1,999 MSRP”
RTX PRO 6000 Blackwell	96GB	$13,250 (NVIDIA list)	“$8,435–8,565”

The RTX PRO 6000 deserves a warning: NVIDIA’s list price rose more than 50% from its $8,565 launch. A Max-Q variant has been seen near $8,300 — check which variant a listing means before you buy, because they are not the same card.

The used 3090 number is the most-wrong figure on the local-AI web. Nearly every “budget fine-tuning rig” guide still runs its cost math on $650–750 cards. At $1,000–1,300, a dual-3090 NVLink build is no longer the bargain those guides describe — it is a considered purchase.

So What Should You Buy?

You mostly run models, occasionally tinker. Buy one card sized to the models you want to run. 24GB handles most 24B-class models at Q4 and does QLoRA on the side. Skip the second card.

You fine-tune regularly on 7B–13B. One 24GB card is enough. QLoRA on a 3090 is the well-trodden path, with the largest volume of working community recipes behind it.

You fine-tune 30B+ locally and it is core to your work. This is the only case where the dual-3090 NVLink build still earns its keep — 48GB of pooled VRAM with the one interconnect consumer NVIDIA no longer sells. Budget for a 1200W PSU and a board with genuine dual-x8 spacing.

You want one card to do everything and budget is not the constraint. The RTX PRO 6000 Blackwell 96GB removes every capacity argument at once. At $13,250 list it should be a business expense with a payback calculation attached.

Honestly: most of you should rent. Fine-tuning is bursty. If you fine-tune for six hours a month, buying a second $1,200 card to serve those six hours is poor allocation at 2026 prices. Buy for the workload that runs every day, and rent for the one that does not.