Reviewed 2026-08-09.
Every slot hal0 loads competes for the same GTT pool (see Unified memory
is not VRAM). The
memory map is the one command that shows what’s actually resident right
now, rather than what you think you configured.
hal0 slots --memory
primary qwen3.5-9b-q4kxl 6.0 GB serving
embed nomic-embed-text-v2 2.3 GB ready
stt whisper-v3.turbo 1.6 GB idle
────────────────────────────────────────────
resident 9.9 GB / 96 GB GTT
What each state means for memory
serving/ready— fully resident, holding its full memory
footprint.idle— resident but not the most-recently-used; the first
candidate for eviction if a new slot needs room.warming/starting— mid-load; its footprint is climbing toward
the steady-state number, not there yet.offline— not resident at all; doesn’t appear in the memory map.
Eviction is LRU, not size-based
When a new slot needs memory the pool doesn’t have, hal0 evicts the
least-recently-used idle slot first — not the largest one. A small
embed slot that hasn’t been hit in an hour goes before a large agent slot
that was just used, even if evicting the large one would free more room
in one step.
Reading drift over time
The number in the memory map right after a slot reaches serving is
usually its floor, not its ceiling — KV cache grows with conversation
length up to the configured context window. A slot that read 6.0 GB on
its first request can read several GB higher after a long session. If a
box that fit fine at boot starts evicting mid-day, this is usually why —
check hal0 slots --memory before assuming it’s a leak.