Who this is for, and what actually works
A Radeon card on Linux, running GGUF through llama.cpp. Two things to settle before you start. ROCm on consumer Radeon is Linux-first: llama.cpp does document a Windows HIP build, but the HSA_OVERRIDE_GFX_VERSION workaround below does not work on Windows, so a card outside AMD’s official list is usually better served there by the Vulkan backend. And format choice is narrower than on NVIDIA — GGUF works, and vLLM ships official ROCm builds for AWQ/GPTQ serving (RX 7700 XT and up, RX 9000 — not RX 6000 or the 7600 XT), but EXL2 is CUDA-only and no amount of ROCm setup will change that.
Best reports: RX 7900 XTX / XT (gfx1100), W7900
Works: RX 9070 XT / 9070 / 9070 GRE (gfx1201, ROCm 7.x)
RX 9060 XT / 9060 (gfx1200, on Ollama's ROCm list)
RX 7800 XT / 7700 XT (gfx1101), RX 6800 XT / 6900 XT (gfx1030)
Patchy: RX 6700 XT (gfx1031), older Polaris
Integrated: Ryzen AI APUs (Radeon 8060S = gfx1151) are on Ollama's ROCm list;
other iGPUs only via the override + unified memory (see below)Prerequisites
Install ROCm 6.1 or newer from AMD’s repository for your distribution — llama.cpp’s HIP build refuses anything older; an RX 9000 card needs a current ROCm 7.x release, which is also what Ollama now requires — then add your user to the render and video groups and log out and back in. Missing group membership is the classic first failure: rocminfo reports no agents, and every later step looks like a build problem when it is a permissions problem. Find your card’s gfx target now — you need it for the build.
sudo usermod -aG render,video $USER # then log out and back in
rocminfo | grep -i gfx # e.g. gfx1100 RX 7900 XTX, gfx1201 RX 9070 XT, gfx1200 RX 9060 XT
rocm-smi # card, VRAM, temperatureBuild llama.cpp with the HIP backend
The flag is GGML_HIP=ON. The older LLAMA_HIPBLAS name is gone and, unlike LLAMA_CUDA, is not forwarded to the new one — pass it and you get a build that compiles cleanly, runs, and is CPU-only. CMake’s only complaint is a line near the end of configure listing it under “Manually-specified variables were not used by the project”, which is easy to scroll past. Set GPU_TARGETS to your gfx target so the kernels are compiled for your card (the older AMDGPU_TARGETS name is still forwarded; leaving both out builds for every GPU in the machine). Point HIPCXX at ROCm’s own clang, as llama.cpp’s build docs do — passing hipcc as the compiler still works, but CMake now warns that it is legacy.
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build \
-DGGML_HIP=ON \
-DGPU_TARGETS=gfx1100 \
-DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j$(nproc)If your card is not on the official list
ROCm refuses to initialise on gfx targets it does not officially support, even when the card is architecturally close to one that is. HSA_OVERRIDE_GFX_VERSION tells the runtime to treat it as the nearest supported target: 11.0.0 for RDNA3, 10.3.0 for RDNA2. This is a workaround, not a supported configuration — it is widely used and it can also produce wrong results or hangs on some kernels, so validate output before trusting it. An integrated Radeon outside AMD’s list (a gfx1035 laptop chip, say) can take the same override — llama.cpp’s build docs map gfx1035 to 10.3.0 — and also needs GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 so the model can live in system RAM; the docs warn that this setting slows a discrete card, so set it only for the iGPU.
# RDNA2 card reporting gfx1031, treated as gfx1030
export HSA_OVERRIDE_GFX_VERSION=10.3.0
# RDNA3
# export HSA_OVERRIDE_GFX_VERSION=11.0.0
# Integrated Radeon (e.g. gfx1035) only:
# export HSA_OVERRIDE_GFX_VERSION=10.3.0 GGML_CUDA_ENABLE_UNIFIED_MEMORY=1Start the server and confirm the GPU is used
Bind to 127.0.0.1 unless you deliberately want the server on your network — llama-server has no authentication of its own. On startup the log names the backend and the number of layers offloaded; that line, not the fact that it answers, is the confirmation. A HIP build that fell back to CPU still serves tokens, slowly. rocm-smi should show VRAM held while the model is loaded.
./build/bin/llama-server \
-m ./models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
-ngl 99 -c 8192 --host 127.0.0.1 --port 8080
# In the startup log, look for the ROCm device line and:
# load_tensors: offloaded 33/33 layers to GPU
rocm-smi --showmemuseSizing a model for your card
A 24GB RX 7900 XTX sits in the same band as an RTX 3090 or 4090 for what fits: memory arithmetic does not care which vendor made the card, only how much VRAM it has and how large the KV cache is. Use the VRAM calculator with your Radeon selected — it is in the GPU list — rather than reading NVIDIA guidance and hoping it transfers. What does not transfer is throughput: tok/s figures on this site are measured on NVIDIA hardware and are not a prediction for ROCm.
Common problems
rocminfo finds no agents: group membership, or the kernel module did not load — check dmesg for amdgpu. hipErrorNoBinaryForGpu at load: the build did not include your gfx target; rebuild with the right GPU_TARGETS, or set HSA_OVERRIDE_GFX_VERSION. Builds fine but runs on CPU: almost always the old LLAMA_HIPBLAS flag, which CMake lists as unused and otherwise ignores — check the startup log for the backend line. Hangs or garbage output after an override: the override is not a supported path; drop back to a lower context or a different quant before assuming the model is at fault.
Next steps
The models linked below are the ones this guide is written around. Open any of them with your Radeon selected in the calculator to see the context length it can actually hold.
Common questions
Should I use ROCm or Vulkan for llama.cpp on a Radeon?
On Linux with a card AMD officially supports, ROCm (the HIP build in this guide) is the documented path. Vulkan is the fallback that needs no ROCm install at all: build with -DGGML_VULKAN=ON instead of GGML_HIP. It is the better choice on Windows for a card outside AMD’s list, because llama.cpp’s build docs note that HSA_OVERRIDE_GFX_VERSION is not supported on Windows. If you only want to run models on Windows rather than build llama.cpp, Ollama’s Windows app handles Radeon cards itself: ROCm on the RX 7600 to 7900 series, and Vulkan on the others. This site has not measured either backend on a Radeon, so it does not say which is faster — run both on your card with the same model and context and compare.
Can a Radeon run AWQ, GPTQ or EXL2 models?
AWQ and GPTQ, yes, but only through vLLM’s ROCm builds, and only on cards vLLM lists: the RX 7700 XT, 7800 XT and 7900 series, and the RX 9000 series. The RX 6000 series and the RX 7600 XT are not on that list, so on those cards GGUF through llama.cpp is the route. EXL2 needs CUDA and does not run on any Radeon, and its runtime, ExLlamaV2, is now archived anyway. The GPU pages and the VRAM calculator apply the same rule, so they only recommend formats your card can load.
What fits on a 24GB RX 7900 XTX compared with a 16GB Radeon?
At 4K context, 86 of the 112 models in this index fit the RX 7900 XTX comfortably, against 64 on a 16GB RX 7800 XT or RX 9070 XT. The difference is the 30B class. Qwen3 30B-A3B at Q4_K_M is about 20.2 GB at 8K context — 84% of a 7900 XTX, comfortable — and does not fit a 16GB card. GPT-OSS 20B at MXFP4 is about 12.9 GB at 8K context, so it runs on either. These are memory figures only; this site has no Radeon speed measurements.
What this guide uses
- Hardware
- Radeon RX 7900 XTX24GB
- Format
- GGUF
- Picks for it
- Best local LLM for 24GB
Next steps
See everything that fits 🔴 Radeon RX 7900 XTXReverse lookup — 24GB at 8192 context, ranked by qualityModels covered in this guide
Did this actually run?
Copying a command is not the same as it working, so this is the only place the site asks. Nothing is collected beyond the answer itself.
Related guides
Quantize Your Own Model to GGUF
Use llama.cpp's quantize tool to convert any HF model to GGUF Q4_K_M for local inference.
llama.cpp on Windows with CUDA
Build llama.cpp with NVIDIA GPU support on Windows 11 — the path of least resistance for PC gamers.
Ollama on Windows (Native, No WSL)
Install the Windows Ollama app for the simplest path — NVIDIA and AMD Radeon GPUs both work, with no WSL and no toolkit to install.