Validation matrix

[system]

hal0 validates a change at two widening tiers before it reaches a
release, plus one exploratory layer that runs only against a release
candidate. This page names what each layer proves, where it runs, and what
blocks a release — see CONTRIBUTING.md
for the day-to-day commands. There is no β (integration) tier: it was
retired in v0.2 and not replaced, so nothing automated sits between α and γ.

The three layers

Tier	What it proves	Where it runs	Blocks what
α — Unit	Pure logic, mocked systemd and HTTP clients. make test is unfiltered, so host-dependent tests (e.g. the OpenWebUI prewire smoke’s real docker pull/run) also run when their daemon is reachable.	Any host — make test	Every commit / PR.
γ — Release-gate script	The 8-row matrix in scripts/release-test.sh — vulkan, rocm, flm (NPU), moonshine (STT), kokoro (TTS), an unload row (every slot the run loaded must unload to offline), openwebui, and a check-only updater row — on one designated box.	hal0-test LXC over SSH — make release-test	Per release candidate, not per commit. Row status pass|fail|skip|deferred; only fail blocks.
γ+ — Release-validation fleet (rc-validate kit)	The product as an operator would actually use it — CLI, API, MCP, Hermes, UI, then serialized stateful lanes (slots, routing, brain, memory, capabilities, Hermes e2e), then an in-place upgrade — on real hardware, per tests/release-validation/.	A private real-hardware fleet (Proxmox-hosted; see Support matrix)	Release-candidate confidence. Agent-driven and exploratory, not a fixed pytest suite — findings that prove mechanical get folded back into pytest/release-test.sh so they stop needing an agent (tests/release-validation/README.md).

Note

The γ script and the rc-validate kit are complementary, not redundant. make release-test
answers “does each backend row still work” with a fixed, scriptable matrix. The rc-validate kit
answers “does the product still work the way an operator experiences it” — model registry, the
memory engine, the agent board, upgrade behavior — surfaces a fixed matrix can’t reach because
they depend on multi-step, stateful operator workflows.

The release-validation kit, in more detail

tests/release-validation/ (orchestrated by the rc-validate Claude Code workflow) runs nine
numbered phases, 0–8 (the kit’s README.md carries the authoritative table): 0 Preflight
(per-box readiness — a version/reachability mismatch aborts the run) → 1 Read-only sweep
(api, cli, mcp, hermes, ui lanes, parallel, no mutations) → 2 Stateful lane
(slots → routing → brain → memory → capabilities → hermes-e2e, strictly serialized on
one box — two agents mutating the same install would produce unreproducible findings) with 2b
Update lane (upgrade + post-upgrade on a separate box, concurrent with phase 2) → 3
Regression probes (every previously filed finding in regressions.yaml, re-checked every
release) → 4 Triage + dedup → 5 Adversarial verify (one skeptic agent per candidate
finding) → 6 Synthesis + report, committed under tests/release-validation/reports/<version>.md
→ 7 File issues (only in mode: file) → 8 Kit curation, the standing phase that folds
newly discovered checks back into the lane briefs and promotes confirmed findings into
regressions.yaml, so the kit gets sharper release over release instead of starting from zero.

“User Green” for hal0

ODS’s release gate names a combined User Green result: every enabled release surface either
passed or was explicitly accounted for. hal0 doesn’t yet have a single named gate that plays that
role end to end, but the equivalent, stated in hal0’s own vocabulary, is:

User Green, for hal0 — a fresh box reaches first chat unattended: install.sh completes
non-interactively, hal0-api comes up healthy (/api/health — Verify the install
step 1), at least one model is registered (step 2), a /v1/chat/completions request against the
brain slot (hal0/brain, the one slot a default install gives a model) returns real text (step 3), the dashboard serves (step 4), and
hal0 doctor verify reports no critical failure (step 6) — with nothing but the one-line
installer command and, optionally, a HuggingFace token.

Composed from the same layers as the table above:

Component	What must be true	Proven by
Install Green	The public install path completes on a real fresh box without operator intervention.	rc-validate’s Preflight + fresh-install evidence; the fresh-install box in the fleet.
Product Green	Core API, dashboard, and OpenWebUI serve; doctor’s aggregate check has no critical row.	hal0 doctor verify (or GET /api/doctor); the read-only lane sweep (api, ui lanes).
Capability Green	A chat completion against the brain slot returns real generated text, not an error or a timeout.	The first-chat step of Verify the install; the brain/slots stateful lanes.
Lifecycle Green	An in-place hal0 update converges cleanly and the box remains healthy after.	The update lane (upgrade, post-upgrade) on the fleet’s update box.
User Green	All four of the above, for the candidate under test, with any skip or deferral named and justified rather than silently dropped.	The synthesized rc-validate report for that release.

Caution

This is a documentation mapping, not a new CI gate — hal0 has no automated job today that
computes a single “User Green” boolean and fails a release on it. Release readiness is currently
a human judgment call informed by the α/γ results plus the rc-validate report; this table
exists so that judgment has a name and a checklist, not a formula.

See also

• Support matrix — the hardware/platform coverage these tiers actually exercise.
• Verify the install — the six-step checklist an operator runs by hand; steps 1, 2, 3, 4, and 6 are exactly the checks “Product Green” and “Capability Green” above cite.
• tests/release-validation/README.md — the kit’s own layout, phase table, and process lessons.
• CONTRIBUTING.md — α/γ commands and CI wiring.