Runtimes & backends: ROCm vs Vulkan vs CPU, in practice

What the lanes mean and when each wins. Reference docs live in Docs; this page is the community’s field guide.

  • ROCm — usually the throughput leader on Strix Halo when the kernel/driver pairing is right; also the most version-sensitive. Note your ROCm version in every report — point releases have swung prefill 3-4x.
  • Vulkan (RADV) — the resilient fallback: fewer version traps, broadly works, often close on decode, behind on prefill.
  • CPU — for embeddings/rerankers and tiny models; leaves the GPU to the big slots.
  • NPU / XDNA — see the Docs hardware matrix for what’s actually supported today before planning around it.

Rule of thumb: benchmark your own box across lanes with the same model + quant before trusting anyone’s table — including ours. The software moves weekly.

Wiki — keep the state current as runtimes update.