# Speech to text and text to speech

**URL:** <https://forum.hal0.dev/t/speech-to-text-and-text-to-speech/42>\
**Category:** Guides\
**Created:** [August 22, 2026, 6:50am UTC](https://forum.hal0.dev/t/speech-to-text-and-text-to-speech/42 "2026-08-22T06:50:01Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![system](https://forum.hal0.dev/user_avatar/forum.hal0.dev/system/32/37_2.png) [@system](https://forum.hal0.dev/u/system)\
**Post date:** [August 22, 2026, 6:50am UTC](https://forum.hal0.dev/t/speech-to-text-and-text-to-speech/42/1 "2026-08-22T06:50:01Z")

</div>

hal0 exposes two voice directions through OpenAI-compatible endpoints:

- **Speech-to-text** (`POST /v1/audio/transcriptions`) — served from the  
single canonical `stt` slot, which can run **either** engine: Moonshine  
(ONNX) on CPU, or whisper-v3:turbo on the XDNA NPU, co-loaded with the  
chat model in one FLM process (the “NPU trio”: chat + STT + embed).  
Choosing a device swaps which engine the `stt` slot runs — there’s no  
separate slot to configure.
- **Text-to-speech** (`POST /v1/audio/speech`) — served from the single  
canonical `tts` slot, which can run **either** engine: Kokoro-82M ONNX  
on CPU, or Qwen3-TTS on GPU (ROCm, native gfx1151). Choosing a device  
swaps which provider the `tts` slot runs — there’s no separate slot to  
configure.

Both are children of the **`voice`** capability: `voice.stt` maps to the  
`stt` slot and `voice.tts` maps to the `tts` slot. Both children are  
now device-keyed engine switches with the same shape. Before this  
redesign, `voice.stt` could only ever run `backend=npu` — any host  
without the XDNA NPU had zero speech-to-text. Reinstating Moonshine as a  
CPU-side STT engine and keying `stt` on device the same way `tts`  
already was closed that gap (see ADR-0001, “Reinstate Moonshine as the  
CPU STT engine”, `docs/adr/0001-moonshine-cpu-stt-reinstatement.md` in  
the repository — an internal design record, not published to these  
docs).

## Enable the voice capability

```sh
hal0 capabilities set voice stt --model <your-stt-model>
hal0 capabilities set voice tts --model <your-tts-model>

```

Or over the API, which accepts any subset of `{ backend, provider, model, enabled }`:

### Speech-to-text (CPU, Moonshine)

```sh
curl -X POST http://localhost:8080/api/capabilities/voice/stt \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "backend": "cpu", "provider": "moonshine"}'

```

### Speech-to-text (NPU, FLM)

```sh
curl -X POST http://localhost:8080/api/capabilities/voice/stt \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "backend": "npu", "provider": "flm"}'

```

### Text-to-speech (CPU, Kokoro)

```sh
curl -X POST http://localhost:8080/api/capabilities/voice/tts \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "backend": "cpu", "provider": "kokoro"}'

```

### Text-to-speech (GPU, Qwen3-TTS)

```sh
curl -X POST http://localhost:8080/api/capabilities/voice/tts \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "backend": "gpu-rocm", "provider": "qwen3tts"}'

```

> [@](#):
>
> **Note**
> 
> STT runs from the single `stt` slot on either engine: Moonshine (CPU,  
> ONNX) or whisper-v3:turbo via the XDNA NPU’s FLM trio — picking a device in  
> the `voice.stt` picker swaps the engine in place, it does not add a second  
> slot. GPU devices carry **no** STT engine: hal0 ships no GPU STT backend,  
> so selecting a GPU device for `voice.stt` returns a typed error instead of  
> silently loading the wrong profile. TTS runs from the single `tts` slot on  
> either engine: Kokoro (CPU, ONNX) or Qwen3-TTS (GPU, ROCm) — picking a  
> device in the `voice.tts` picker swaps the engine in place the same way.

### The `voice.stt` engine switch

| Device | Provider | Runs on |
| --- | --- | --- |
| `cpu` | `moonshine` | CPU (ONNX) |
| `npu` | `flm` | XDNA NPU, coresident with chat + embed in the FLM trio |

There is deliberately no GPU row — hal0 ships no GPU STT engine. This is a  
two-engine, two-device special case, not a general `provider` selector: if  
a second CPU (or GPU) STT engine ever lands, `device` alone stops being  
enough to disambiguate and `provider` must become a first-class part of  
capability selection — this future-work trigger is recorded in ADR-0001  
(`docs/adr/0001-moonshine-cpu-stt-reinstatement.md`, repo-only).

### The `voice.tts` engine switch

| Device | Provider | Runs on |
| --- | --- | --- |
| `cpu` | `kokoro` | CPU (ONNX) |
| `gpu-rocm` | `qwen3tts` | Strix Halo iGPU (ROCm, native gfx1151) |

Selecting a device rewrites which provider the `tts` (or `stt`) slot  
loads — the slot itself doesn’t move. Qwen3-TTS and Moonshine weights are  
operator-staged (not pulled through the hal0 registry; whisper-v3:turbo on  
the FLM/NPU path resolves through FastFlowLM’s own model handling instead).  
Moonshine’s weights preflight at slot spawn — a missing or empty bundle  
fails loudly, by name (`slot.weights_missing`), rather than starting a  
container that 500s on the first request.

### Kokoro weights

Kokoro’s weights come from the registry: `hal0 model pull kokoro-v1`  
installs the ONNX model and the voices pack into the model store, at  
`<store>/tts/kokoro-v1/kokoro-v1.0.onnx` and  
`<store>/tts/kokoro-v1/voices-v1.0.bin`. `<store>` is the configured  
`[models].store`, `/var/lib/hal0/models` on a fresh install. The `tts` slot  
mounts the store read-only and hal0 passes that directory as the runner’s  
`--model_path`, so the `kokoro` profile carries no path.

If either file is missing, the slot refuses to start with  
`slot.weights_missing` and names the command to run  
(`hal0 model pull kokoro-v1`). It never falls back to downloading the  
weights at start-up, so an offline box runs Kokoro once the bundle is  
pulled.

On upgrade to 1.4.0, `hal0 update` pulls the bundle once if a slot or  
capability selection uses Kokoro and the store does not have it yet. This  
also replaces the single `model.gguf` an earlier pull left there (the ONNX  
file under the wrong name, with no voices pack). On a box without internet  
access the update still completes; run `hal0 model pull kokoro-v1` once it  
is online. Until then the `tts` slot’s error names the same command.

### Staging the Moonshine weights

The bundle is a directory the loader reads directly — it resolves  
`encoder_model.ort` (or `.onnx`) plus `decoder_model_merged.*` and the  
tokenizer inside whatever directory it is handed. Two consequences:

- **Stage a non-streaming variant.** The `base-en` bundle has the file  
names the loader wants. The _streaming_ bundles (`small-streaming-en`,  
`medium-streaming-en`) ship a different file set (`encoder.ort`,  
`frontend.ort`, `adapter.ort`) belonging to the Moonshine streaming SDK,  
which this image does not consume. Pointing the slot at one is rejected  
at spawn with `slot.weights_missing` naming the streaming case.
- **Point `--model_path` anywhere sensible.** hal0 walks the tree to find  
the directory holding the encoder, so both `…/base-en` and  
`…/base-en/quantized/base-en` work. If the resolved path is outside the  
model store root, the slot mounts it read-only at the same absolute path  
(plus the symlink target, if any) so the container can read it.

Verify staging before enabling the capability:

```sh
hal0 doctor all --json | jq '.[] | select(.key == "stt-weights")'

```

A `pass` names the staged path; a `fail` names exactly what is missing.

### How the CPU STT path works (Moonshine)

Enabling `voice.stt` with `backend=cpu` spawns the `stt` slot’s own  
container from the `moonshine` runner’s image (`ghcr.io/hal0ai/hal0-runner-moonshine`, `MoonshineProvider`),  
distinct from the NPU path below — it’s a standalone process, not a  
co-loaded FLM role. The container wraps an OpenAI-compatible FastAPI  
server (`POST /v1/audio/transcriptions`, `GET /v1/models`, `GET /health`).  
It also exposes `WS /v1/audio/stream` for live PCM16 (16 kHz mono)  
streaming transcription — this endpoint is **in-container only** and is  
**not** routed by the hal0 dispatcher; it isn’t part of the public `/v1/*`  
surface. hal0 binds `0.0.0.0:8080` with no built-in auth by design  
(LAN-trust posture); voice endpoints accept file uploads on that  
unauthenticated bind.

### How the NPU STT path works

Enabling `voice.stt` with `backend=npu` does **not** spawn a standalone  
process — it drives the FLM trio, one `flm serve` anchor process serving  
chat, transcription, and embeddings together. The orchestrator toggles  
the anchor’s `[npu].asr` flag and writes a `type=transcription` slot  
record for dispatch gating. The anchor is **not** auto-restarted — the  
response carries `pending_reload: true`, and the change takes effect once  
you reload the FLM anchor yourself (`hal0 slot restart <anchor-slot>`).  
NPU transcription requires the FLM chat anchor to already be loaded.

## Transcribe audio (speech to text)

```sh
curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -F 'file=@recording.wav' \
  -F 'model=<your-stt-model>'

```

The `model` form field is **required** ; omitting it returns **400**  
(`request.missing_model`).

## Synthesize speech (text to speech)

```sh
curl -X POST http://localhost:8080/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "<your-tts-model>",
    "input": "Hello from hal0.",
    "voice": "<voice-id>"
  }' \
  --output speech.wav

```

`model` is required. `voice`, `speed`, and `response_format` are  
optional — see TTS request defaults below.

### TTS request defaults

The `tts` slot can carry persisted request defaults that  
`/v1/audio/speech` seeds into a request whenever the body omits the  
matching field — an explicit value in the request always wins:

- `default_voice` — voice id injected when the request omits `voice`.
- `default_speed` — playback speed (`0.25`–`4.0`) injected when the  
request omits `speed`.
- `default_response_format` — `mp3` / `wav` / `opus` / `flac` / `pcm`,  
injected when the request omits `response_format`.

These are set from the dashboard’s Settings → Voice panel and take effect  
immediately — no container restart. The dashboard’s voice picker reads a  
live voice list from `GET /api/slots/tts/voices`; if the `tts` slot is  
cold or unreachable, it falls back to a built-in seed list.

## See also

- [Manage slots](https://forum.hal0.dev/t/manage-slots-hal0-docs/38) — the `stt` and `tts` slots.
- [Choose models](https://forum.hal0.dev/t/choose-models-hal0-docs/31) — what a model row records, and  
how profiles carry the tune for voice models the same as chat models.