v0.7: pre-synth buffering, session IDs, segment timeout playback

This commit is contained in:
2026-08-12 10:30:11 +00:00
parent e1739059d9
commit ecf00c1bc4
6 changed files with 285 additions and 372 deletions
+43 -18
View File
@@ -34,33 +34,46 @@ with the 6-byte magic `HKMSTR` to distinguish our traffic from real DHCP.
**NOP (heartbeat while PTT held):**
```
HKMSTR <nonce>\n
HKMSTR <session> <nonce>\n
```
Sent every 50ms while PTT is held. The nonce is an incrementing unsigned
integer that makes each datagram unique. The server discards it — it's
purely for packet uniqueness, not for any protocol logic.
Sent every 50ms while PTT is held. The session is an incrementing unsigned
integer that identifies the current PTT utterance (incremented on each PTT
press). The nonce is an incrementing unsigned integer that makes each
datagram unique. The server should echo the session back in replies. Both
are discarded by the server for protocol logic — the server tracks liveness
via "did anything arrive recently."
**OFF (PTT released):**
```
HKMSTR:OFF <nonce>\n
HKMSTR:OFF <session> <nonce>\n
```
Sent once when PTT is released. This is the fast-stop signal. If lost, the
150ms timeout acts as a backstop.
### Server → Client
**Partial transcript:**
```
HKMSTR:P <text>\n
```
Intermediate recognition result. Fire-and-forget. Client logs it but does
not act on it.
The server echoes the session ID from the NOPs in all replies. The client
drops any reply with a stale session ID.
**Final transcript:**
**Partial segment (completed VAD segment):**
```
HKMSTR:F <text>\n
HKMSTR:P <session> <text>\n
```
Complete utterance. Client feeds this to the TTS engine.
A completed, VAD-separated utterance segment. The client starts TTS
synthesis immediately and buffers the audio output, but does **not** play
it yet. Playback starts when `:F` arrives (or timeout).
**Final (all done):**
```
HKMSTR:F <session> <text>\n
```
Signals that all segments have been sent. May be empty
(`HKMSTR:F <session>\n`). Triggers playback of all buffered audio on the
client. If `<text>` is non-empty, the client synthesizes it before playing.
The purpose of this design is to minimize latency: TTS synthesis runs in
parallel with recording, so by the time `:F` arrives, audio is already
buffered and playback starts immediately.
## Server state machine
@@ -79,15 +92,27 @@ Complete utterance. Client feeds this to the TTS engine.
│ PROCESSING │ │
└─────────────┘ │
│ │
STT │ │
done │ │
send │ │
:P/:F │ │
▼ │
send HKMSTR:F ─────────┘
back to IDLE ──────────┘
```
- **IDLE → RECORDING:** first NOP received, start mic capture
- **RECORDING:** VAD detects completed segments → send `HKMSTR:P <text>`
- **RECORDING → PROCESSING:** OFF received, OR 150ms since last NOP
- **PROCESSING → IDLE:** STT done, send `HKMSTR:F <text>`
- **PROCESSING → IDLE:** send remaining segments as `:P`, then `HKMSTR:F`
## Client playback model
1. `:P` arrives → start TTS synthesis immediately, buffer audio (don't play).
Reset 250ms segment timer.
2. More `:P` arrive → keep synthesizing and buffering, reset timer each time.
3. `:F` arrives → play all buffered audio immediately, cancel timer.
4. If `:F` doesn't arrive within 250ms of the last `:P` → play buffered audio
early. If more `:P` arrive after early playback, synthesis continues and
new audio is appended to the output — not a failure.
5. `:F` may be empty — it just signals "all segments sent, start/confirm playback."
## Timing