AMD ROCm Setup for Local LLMs (2026): llama.cpp and Ollama

Bottom Line

• Use Ollama if you want it working in ten minutes. Install Ollama, then extract its separate ROCm package. That is two commands on Linux.
• Do not compile llama.cpp first. Every llama.cpp release ships a prebuilt ubuntu-rocm-10.0 build and a prebuilt ubuntu-vulkan build. The ROCm build already carries kernels for the Radeon targets from gfx1010 to gfx1201, including gfx1031 and gfx1151.
• The Ryzen AI Max+ 395 is now on AMD’s own list. ROCm 10.1 data names it as gfx1151, on Ubuntu 26.04 or Ubuntu 24.04.4 with the 6.17 HWE kernel. The old HSA_OVERRIDE_GFX_VERSION=11.5.1 advice for Strix Halo is out of date.
• Vulkan is the fallback, and on some machines it is the default. It needs no ROCm at all, only the graphics driver. It is the answer for Windows, for cards ROCm skips, and for Strix Halo owners who want the most stable stack.
• Your card still has to be worth setting up. If you have not bought yet, read which AMD GPUs actually work before this page.

Which ROCm Exists in October 2026

AMD now publishes ROCm in two lines, and the version numbers jumped.

Line	Latest	Released	How AMD ships it
ROCm Core SDK (built with TheRock)	10.1.0	2026-10-05	pip wheels, tarballs, runfile installer
ROCm Core SDK (built with TheRock)	10.0.0	2026-08-26	same
ROCm Core SDK (built with TheRock)	7.14.0	2026-07-15	same
ROCm 7.2 (classic packages)	7.2.4	2026-05-29	amdgpu-install and distro packages

Dates are from the TheRock and ROCm release pages. The 7.14.0 notes say that release “transitions ROCm to TheRock”, a modular build and release system. The release lists go from 7.14 straight to 10.0. Nothing was skipped; the number jumped.

What this means for you. llama.cpp builds its own ROCm binaries against ROCm 10.0.0 (its release workflow installs rocm[libraries,devel]==10.0.0 from AMD’s pip index). If you plan to use llama.cpp’s prebuilt ROCm build, match it with a 10.x runtime. Ollama only asks for “the AMD ROCm v7 driver” on Linux (Ollama GPU docs).

The cards AMD lists for ROCm 10.1

From AMD’s own ROCm 10.1 GPU data, Radeon and Ryzen only:

LLVM target	Listed products
gfx1201	RX 9070 XT, RX 9070 GRE, RX 9070, Radeon AI PRO R9700, R9700S, R9600D
gfx1200	RX 9060 XT, RX 9060 XT LP, RX 9060, RX 9050
gfx1100	RX 7900 XTX, RX 7900 XT, RX 7900 GRE, PRO W7900, PRO W7800
gfx1101	RX 7800 XT, RX 7700 XT, RX 7700, PRO W7700, PRO V710
gfx1102	RX 7600
gfx1030	PRO W6800, PRO V620 only
gfx1151	Ryzen AI Max+ 395, Max+ 392, Max+ 388, Max 390, Max 385, and the PRO 300/400 parts

Not listed: the consumer RX 6000 cards (RX 6800, 6800 XT, 6900 XT, 6700 XT) and the RX 7600 XT. That does not mean they fail. The next section shows why most of them still run.

This list is newer than the one in our AMD support comparison and our unsupported-GPU guide, which were written against ROCm 7.14. The main change is that gfx1102 and gfx1151 are now on AMD’s list.

Step 1: Driver, Groups, and a Visible GPU

Do this once, whichever runtime you pick.

1. Install a supported Linux. For Radeon cards, AMD’s ROCm 10.1 OS data lists Ubuntu 26.04, 24.04.4 and 22.04.5. For the Ryzen AI Max+ 395 it lists Ubuntu 26.04, or 24.04.4 with the HWE 6.17 kernel. The GA 6.8 kernel is not on the list for that chip.
2. Install the ROCm driver stack. AMD’s install selector offers amdgpu-install for Radeon and Ryzen on Linux (install-methods data). Ollama’s GPU docs point to the same amdgpu-install utility.
3. Give your user GPU access. Add yourself to the render and video groups, then log out and back in:

   sudo usermod -aG render,video $USER

4. Confirm ROCm sees the GPU and note your target:

   rocminfo | grep gfx | head -1

   This is the same check the llama.cpp build guide uses. No output means ROCm cannot see the card. Fix that before you install anything else.

Step 2a: Ollama on ROCm (the Short Path)

From the Ollama Linux docs:

curl -fsSL https://ollama.com/install.sh | sh
curl -fsSL https://ollama.com/download/ollama-linux-amd64-rocm.tar.zst | sudo tar x -C /usr

Then run a model and check that it lands on the GPU with ollama ps.

Docker (Ollama Docker docs):

docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama \
  -p 11434:11434 --name ollama ollama/ollama:rocm

Three settings from the Ollama GPU docs solve most problems:

• HSA_OVERRIDE_GFX_VERSION=x.y.z makes ROCm treat your card as a nearby supported target. It is Linux only. Our override guide has the values per card.
• ROCR_VISIBLE_DEVICES picks which AMD GPUs Ollama uses. Use the Uuid from rocminfo when it has one. -1 forces CPU.
• On Fedora and RHEL, SELinux can block containers from the GPU. Run sudo setsebool container_use_devices=1 on the host.

Ollama also has a Vulkan backend. Its docs say Vulkan is on by default when the backend is installed, on Windows and Linux. OLLAMA_VULKAN=0 turns it off. On a machine with an integrated and a discrete GPU, set GGML_VK_VISIBLE_DEVICES to the discrete card’s index.

Step 2b: llama.cpp From a Prebuilt Release

Each llama.cpp release carries these AMD-relevant files (names from build b11852, 2026-10-24):

File	Backend	Needs
llama-b11852-bin-ubuntu-rocm-10.0-x64.tar.gz	HIP (ROCm)	A ROCm 10 runtime on the host
llama-b11852-bin-ubuntu-vulkan-x64.tar.gz	Vulkan	Mesa or vendor Vulkan driver
llama-b11852-bin-win-rocm-10.0-x64.zip	HIP (ROCm)	Windows AMD driver
llama-b11852-bin-win-vulkan-x64.zip	Vulkan	Windows AMD driver

The true thing most guides miss. The release workflow compiles the Linux ROCm build for gfx908;gfx90a;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1200;gfx1201. That list includes gfx1031, the RX 6700 XT. AMD’s own TheRock support table marks gfx1031 and the other RDNA2 targets “Release Ready” on Linux. So on a current build, an RX 6700 XT has native kernels and may not need HSA_OVERRIDE_GFX_VERSION at all. Try it without the override first.

The Linux tarball holds only the llama.cpp binaries. The workflow packs the build/bin folder and nothing else, so the ROCm libraries must come from your system.

Run it:

tar xzf llama-b11852-bin-ubuntu-rocm-10.0-x64.tar.gz
cd llama-b11852
./llama-server -m /path/to/model.gguf -c 16384

-ngl defaults to auto on current builds, so you do not need -ngl 99. If a copied command fails with error: invalid argument, the flag was probably removed in 2026. See every llama.cpp invalid argument error and its fix.

Step 2c: Build llama.cpp Yourself (Only When You Need To)

Build from source when you want a commit newer than the last release, or a binary for one target only. The llama.cpp build guide gives this HIP command for Linux:

HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
    cmake -S . -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1030 -DCMAKE_BUILD_TYPE=Release \
    && cmake --build build --config Release -- -j 16

Replace gfx1030 with your target from rocminfo. The guide says GPU_TARGETS is optional: without it, the build covers every GPU in the machine.

The Vulkan build is two lines once vulkaninfo works. On Debian or Ubuntu, install libvulkan-dev glslc spirv-headers first:

cmake -B build -DGGML_VULKAN=1
cmake --build build --config Release

Step 2d: LM Studio

LM Studio runs models through llama.cpp. Its docs say to open the runtime manager with Ctrl+Shift+R on Windows and Linux. That is where you install and manage the llama.cpp runtimes LM Studio uses. If it offers both a ROCm and a Vulkan runtime for your card and ROCm does not detect the GPU, try Vulkan before you change anything else. LM Studio’s docs do not publish an AMD support list, so test your own card.

Vulkan: the Fallback That Usually Just Works

Vulkan skips ROCm entirely. It talks to the GPU through the normal graphics driver: Mesa RADV on Linux, the Adrenalin driver on Windows.

Choose Vulkan when:

• Your card is not on any ROCm list, and the override does not hold.
• You are on Windows. The llama.cpp build guide says HSA_OVERRIDE_GFX_VERSION is not supported on Windows. Ollama’s Windows ROCm list is only the RX 7000 series (7900 XTX to 7600) and PRO W7500 to W7900.
• You want the most stable Strix Halo stack. The kyuz0 Strix Halo toolboxes describe their vulkan-radv image as “Most stable and compatible. Recommended for most users and all models.” They label the ROCm image as the one “Recommended for Performance”.

The honest caveat: neither backend wins everywhere. Speed depends on the card, the model, the quant and the build. Install both prebuilt binaries, run the same prompt through each, and keep the faster one. Our Ryzen AI Max+ 395 model guide has community tokens-per-second numbers for that chip.

Ryzen AI Max+ 395 (Strix Halo) Notes

The integrated Radeon 8060S is gfx1151. Four things are specific to it.

1. Use the supported kernel. AMD’s ROCm 10.1 OS data lists Ubuntu 26.04 or Ubuntu 24.04.4 with HWE 6.17 for this chip.
2. Raise the GPU memory limit. The Strix Halo Toolboxes host guide uses two kernel parameters to give the GPU about 124 GB: amdgpu.gttsize=126976 and ttm.pages_limit=32505856. It also lists amd_iommu=off for a 5-12% speedup, and warns that it stops the NPU working and removes DMA attack protection. Leave that one out unless you accept both costs.
3. Load models with -lm none. The kyuz0 README refers to “the usual Strix Halo --load-mode none guidance”. That is the current spelling of the old --no-mmap, which no longer parses. Why the flag changed.
4. vLLM has a known slowdown. The ROCm 7.14.0 release notes list lower-than-expected LLM inference on RX 7900 series, RX 7800 XT and Ryzen AI MAX chips with PyTorch older than 2.14. The workaround is TORCH_BLAS_PREFER_HIPBLASLT=1. This affects vLLM and PyTorch, not llama.cpp or Ollama.

Choosing a box rather than setting one up? See which Strix Halo mini PC to buy.

When It Fails

Symptom	Likely cause	Fix
rocminfo lists no gfx agent	No access to /dev/kfd	Join render and video, log in again
Works on the host, not in Docker	Devices not passed	Add --device /dev/kfd --device /dev/dri
Container cannot see the GPU on Fedora/RHEL	SELinux	sudo setsebool container_use_devices=1
Ollama loads the model on CPU	Card outside Ollama’s ROCm list	Override (Linux) or use Vulkan
The integrated GPU is used, not the card	Device order	ROCR_VISIBLE_DEVICES or GGML_VK_VISIBLE_DEVICES
Override set on Windows, no change	Not supported on Windows	Use the Vulkan build
Build error: cannot find ROCm device library	Device libs not found	Set HIP_DEVICE_LIB_PATH, as the build guide shows
error: invalid argument: --no-mmap	Flag removed in 2026	-lm none

Still stuck after this table? Check the card against the lists first. Most failures that survive these fixes are a card that is genuinely outside every list, and then Vulkan is the answer.

See Also

• Which AMD GPUs Actually Work for Local LLMs? — the support lists, and what to buy
• Run a Local LLM on an Unsupported AMD GPU — HSA_OVERRIDE_GFX_VERSION values per card
• Best Models for AMD Ryzen AI Max+ 395 — what to run once Strix Halo works
• AMD R9700 vs RTX 3090 for Local LLMs — 32GB of ROCm or 24GB of CUDA
• RX 9070 XT for Local LLM — the 16GB RDNA4 card
• Best Local LLM for RX 7900 XTX — the 24GB AMD card
• llama.cpp Flags Explained — every flag, and every invalid argument error
• Ollama vs llama.cpp — which runtime to keep

Sources

• ROCm 10.1 GPU list, OS support data and install methods. AMD’s source files for its compatibility pages, docs/10.1.0 branch. Read 2026-10-24.
• TheRock releases and TheRock supported GPUs. Release dates for 7.14.0, 10.0.0 and 10.1.0, and per-target readiness. Read 2026-10-24.
• ROCm 7.14.0 release notes. The TheRock transition and the hipBLASLt workaround. Read 2026-10-24.
• llama.cpp build guide and release workflow, at commit 6b9ba4f. HIP and Vulkan build commands, the Windows override limit, and the prebuilt GPU target list.
• Ollama GPU, Linux and Docker docs. Read 2026-10-24.
• LM Studio docs. The runtime manager shortcut. Read 2026-10-24.
• kyuz0/amd-strix-halo-toolboxes and the Strix Halo Toolboxes host guide. Backend labels, load mode, and kernel memory parameters. Read 2026-10-24.

This page carries no affiliate link by design. It is for hardware you already own.