Hardware matrix reference

[system]

Note

hal0 does not hard-block installs below a minimum spec. Backend selection degrades
gracefully to CPU and the smallest curated model rather than refusing to install.
There is no single “minimum RAM/VRAM” gate — see the decision tree below.

Hardware probe

HardwareProbe.probe() (hal0.hardware.probe) writes a snapshot to
/etc/hal0/hardware.json on install, on hal0 config hardware --refresh, and from the dashboard’s re-probe button. All probing is
subprocess/sysfs-based and never raises.

Signal	Source
CPU model/cores/threads	/proc/cpuinfo, with an ARM fallback chain (Hardware/Model/CPU part fields), last-resort platform.machine().
RAM total/available	/proc/meminfo (MemTotal/MemAvailable).
GPU (AMD)	DRM sysfs scan of /sys/class/drm/card*/device/mem_info_vram_total.
Hostname	/proc/sys/kernel/hostname.

The result is validated against the HardwareInfo schema — see the
hardware.json section of Config schema for the
full field list, including unified_memory_mb (the true UMA pool — use this, not
ram_mb + vram_mb, on unified-memory systems like Strix Halo) and platform
(strix-halo, wsl2, proxmox-kvm, kvm, lxc, bare-metal-*, unknown).

Canonical device taxonomy

hal0.model_meta is the single source of truth for device ids (previously duplicated
across 5 call sites, now unified). DEFAULT_DEVICE = "gpu-rocm".

Device id	Label	Recommended	Default profile	Notes
gpu-rocm	GPU (ROCm)		chat	Best throughput on Strix Halo — the recommended default.
gpu-vulkan	GPU (Vulkan)	—	chat	Runs anywhere Mesa Vulkan does, and needs only a render node — so it is the lane for a box with no /dev/kfd. It is not the seeded default for maturity reasons, not speed ones. LLM slots on this device are gated on a runner image whose digest is Vulkan-gated in manifest.json (#1888/#1948). Also the device the non-llama.cpp Vulkan runtimes use (Kokoro, ComfyUI).
gpu-cuda	GPU (CUDA)	—	chat	NVIDIA GPUs via llama.cpp CUDA — experimental on hal0.
cpu	CPU	—	cpu-chat	Always available, slow but correct.
npu	NPU (FLM)	—	flm	AMD XDNA NPU via FastFlowLM — chat/embed/ASR trio on one process.

VALID_DEVICES is the frozenset of these 5 ids. A legacy v0.1 backend enum
(rocm|vulkan|cuda|cpu|flm|moonshine|kokoro) is deprecated and round-trips for one
release; map_backend_to_device() maps unknown legacy values to "cpu" as a safe
fallback rather than crashing config load.

Backend-selection decision tree

hardware.recommend._backend_for() — this is the actual logic the installer and
dashboard use to pick a device for a new slot, given a probed HardwareInfo:

1. AMD, compute_capable (rocm-smi reachable) or /dev/kfd present →
   ROCm. The rationale string still names the “AMD UMA / Strix Halo class”
   when unified memory ≥ 32 GB and vram_mb <= 4096 (a small dedicated VRAM
   number is a UMA signal, not a real cap). Note this is a maturity
   preference, not a throughput one — see the lane comparison below.
2. AMD, no ROCm compute node but a Vulkan-capable GPU → Vulkan. This
   is the unprivileged-LXC shape: a working iGPU with no /dev/kfd forwarded.
   The lane is real, but an LLM slot on it must resolve a runner image whose
   digest has "vulkan" in its lanes_gated pin in manifest.json, or slot
   load refuses it by name (#1888/#1948) — an ungated Vulkan backend can emit
   invalid tokens for every model.
3. AMD, neither ROCm compute nor a usable Vulkan device → CPU.
4. NVIDIA, NVIDIA Container Toolkit present (nvidia-ctk on PATH or a CDI spec
   under /etc/cdi//var/run/cdi) → CUDA.
5. NVIDIA, no CDI → Vulkan fallback.
6. Intel iGPU / unknown-but-vulkan_capable → Vulkan.
7. No GPU detected → CPU (“no GPU detected — CPU inference only, slow but
   correct”).

ROCm vs Vulkan on AMD

Both lanes run the same Standard runner image. Throughput figures for a pinned
digest come from the runner-images hardware gate that admits it; measure your
own box with hal0 bench.

hal0 still seeds gpu-rocm on a box that can run it. The reasons are
maturity, not speed:

• the ROCm lane carries hal0’s whole validation history — every release
  candidate has been exercised on it;
• MTP / speculative-decode profiles are tuned against the ROCm lane, and those
  gains are not represented in a plain pp512/tg128 comparison;
• the Vulkan lane is admitted per digest by the hardware gate.

The seeded default is an open question, revisited at GA against a broader
matrix. If your workload is plain prefill/decode on Strix Halo, switching is
a defensible thing to do today — see the “Switching a slot to the Vulkan
lane” section of Manage slots.

VRAM budget (_vram_budget_gb()): AMD UMA → half of the unified pool; discrete GPU →
vram_mb; CPU-only → half of ram_available_mb.

Platform support

HardwareInfo.platform enumerates every platform hal0 recognizes during probing:
strix-halo, wsl2, proxmox-kvm, kvm, lxc, bare-metal-amd-gpu,
bare-metal-nvidia-gpu, bare-metal-intel-igpu, bare-metal-cpu-only, unknown.

For install-target-specific setup (Proxmox LXC, WSL2, bare-metal Ubuntu), see the
Getting Started guides — this page covers only the hardware-detection and
backend-selection logic, not per-platform install steps.

See also

• Providers, profiles & devices — how a
  device selection flows into a slot’s device field.
• Model roster & benchmarks — how hardware
  tier drives the curated primary-slot model recommendation.