hal0’s inference containers need the host kernel to already expose the
GPU (and, on Strix Halo, the NPU) as device nodes. hal0 ships the
userspace runtime — ROCm, XRT, the FastFlowLM binary — inside its
container images, but it cannot load a kernel driver from inside a
container. This page covers what the host needs before you install, and
how container/LXC device passthrough must be wired so hal0 can see it.
Kernel and firmware
The iGPU uses the in-tree amdgpu driver. On AMD Strix Halo (Ryzen AI
Max+ 395/385/390), get onto a kernel ≥ 6.14 with current
linux-firmware — earlier kernels are missing GTT and NPU fixes that
Strix Halo depends on. If you also want the XDNA NPU, the kernel needs
the amdxdna module available (modprobe amdxdna) and NPU firmware
current enough to match it.
hal0 does not install or manage this layer — it is host-side operator
setup done before hal0’s installer runs.
Verify amdgpu
Confirm the kernel actually bound amdgpu to the GPU before you
install:
lspci -nnk | grep -A3 VGA
Look for Kernel driver in use: amdgpu in the output. If it instead
shows amdgpu under “Kernel modules” but not “Kernel driver in use”, the
module is available but not bound — check dmesg for why the driver
probe failed (firmware mismatch is the usual cause).
Device nodes
Once amdgpu (and, for NPU hosts, amdxdna) is bound, the kernel
exposes the accelerators as device nodes:
| Node | Role |
|---|---|
/dev/dri/renderD* |
Render node — Vulkan / ROCm render access |
/dev/kfd |
ROCm compute queues |
/dev/accel/accel0 |
XDNA NPU (FastFlowLM) |
hal0’s hardware probe reads these directly; if a node is missing, the
corresponding capability drops out of the probe rather than failing the
install.
Note: Host ROCm is not required
The host only needs amdgpu bound and the nodes above present — it does
not need a host ROCm install. Every hal0 slot container carries its
own ROCm userspace. In particular, hal0 does not need rocm-smi on the
host to decide the ROCm lane: it checks whether /dev/kfd is present and
openable by the identity that runs your slot containers, not whether a
ROCm CLI happens to be installed. rocm-smi, when present, is read only
as telemetry (GPU name, free VRAM) — never as the gate for whether ROCm
runs.
Re-probing after a driver or kernel change
hal0 caches what it found in /etc/hal0/hardware.json, written by hal0 probe. A kernel upgrade, a newly forwarded device, or a driver change
can make that cache stale — the ROCm/Vulkan lane derivation and the
dashboard’s VRAM/RAM split both read it. hal0 detects the common staleness
shapes itself (the file is missing, it predates the running kernel, or it
was written in a previous boot cycle) and falls back to a live re-probe
automatically rather than trusting a stale answer — hal0 doctor surfaces
this as the hardware_freshness row so you know when that is happening.
Still, re-run hal0 config hardware --refresh yourself after any
host-level hardware or driver change so the cache catches up and every
read goes back to being the cheap cached one.
Render-group gids
Containers (podman slots, or an LXC passing devices through to an inner
container runtime) need group access to these nodes, and the group id
that matters is the device node’s owner gid, not whatever name it
resolves to on the host. Check it directly:
stat -c %g /dev/dri/renderD128
Use that numeric gid with --group-add (or the container runtime’s
equivalent) when granting the container access. Don’t rely on
getent group render — on a given host that name can resolve to a
different gid than the one that actually owns the render node, which
silently leaves the container unable to open it.
Size the GTT pool
On Strix Halo the GPU’s usable memory is the amdgpu GTT pool, carved
from system RAM (unified memory). A model can only allocate up to the
GTT cap, not the full RAM pool.
On kernel ≥ 6.14, amdgpu grows the GTT/GART limit dynamically — hal0
reads the live pool size from mem_info_gtt_total on every request
rather than trusting a boot-time snapshot, precisely because that limit
can move (see src/hal0/api/routes/hardware.py). You generally don’t
need to pin anything; hal0 measures whatever the kernel currently
reports and sizes its model recommendations against it.
Pinning GTT/TTM parameters on the kernel command line is an operator
choice, not a requirement — it reserves a large fixed pool up front
instead of letting the kernel size it dynamically, which is useful if
you want a guaranteed floor regardless of what else the host is doing.
Here’s a worked example from a 128 GB Strix Halo host that pins one:
# /etc/default/grub — GRUB_CMDLINE_LINUX_DEFAULT on a 128 GB Strix Halo host
iommu=pt amd_iommu=on amd_pstate=active amdgpu.gttsize=118784 \
amdgpu.cwsr_enable=1 ttm.pages_limit=30408704 ttm.page_pool_size=30408704 \
amdttm.pages_limit=30408704 amdttm.page_pool_size=30408704
| Parameter | Value | Meaning |
|---|---|---|
amdgpu.gttsize |
118784 |
GTT window in MiB (116 GiB) |
ttm.pages_limit / ttm.page_pool_size |
30408704 |
TTM page cap in 4 KiB pages (116 GiB — keep equal to gttsize) |
amdttm.pages_limit / amdttm.page_pool_size |
30408704 |
Same cap, duplicated for the amdttm module name used on some kernels |
iommu=pt amd_iommu=on |
— | IOMMU passthrough mode, left on here (this host does not disable the IOMMU) |
amd_pstate=active |
— | Active P-state driver for CPU frequency scaling |
amdgpu.cwsr_enable=1 |
— | Compute wave save/restore, needed for compute-queue preemption |
On this host the pinned values leave 116 GiB of a 128 GB machine as the
GTT pool; the observed live pool
(/sys/class/drm/renderD*/device/mem_info_gtt_total) matched the pinned
gttsize/pages_limit exactly. Treat these as a starting point, not a
universal constant — scale gttsize and the pages_limit/
page_pool_size pairs down together (they must stay equal) to reserve
more host RAM for other tenants. After rebooting, cat /proc/cmdline
should show the params.
NVIDIA
NVIDIA support is experimental. hal0 targets AMD Strix Halo first,
and RTX 30/40/50-series GPUs are supported through
CDI and the
NVIDIA Container Toolkit rather than through the AMD device-node path
above. Install the NVIDIA Container Toolkit and generate a CDI spec on
the host so podman/docker can request the GPU by CDI device name; hal0’s
hardware probe picks it up from there. Expect rougher edges than the
AMD path — this tier gets less field validation.