v0.7: pre-synth buffering, session IDs, segment timeout playback
This commit is contained in:
+43
-18
@@ -34,33 +34,46 @@ with the 6-byte magic `HKMSTR` to distinguish our traffic from real DHCP.
|
||||
|
||||
**NOP (heartbeat while PTT held):**
|
||||
```
|
||||
HKMSTR <nonce>\n
|
||||
HKMSTR <session> <nonce>\n
|
||||
```
|
||||
Sent every 50ms while PTT is held. The nonce is an incrementing unsigned
|
||||
integer that makes each datagram unique. The server discards it — it's
|
||||
purely for packet uniqueness, not for any protocol logic.
|
||||
Sent every 50ms while PTT is held. The session is an incrementing unsigned
|
||||
integer that identifies the current PTT utterance (incremented on each PTT
|
||||
press). The nonce is an incrementing unsigned integer that makes each
|
||||
datagram unique. The server should echo the session back in replies. Both
|
||||
are discarded by the server for protocol logic — the server tracks liveness
|
||||
via "did anything arrive recently."
|
||||
|
||||
**OFF (PTT released):**
|
||||
```
|
||||
HKMSTR:OFF <nonce>\n
|
||||
HKMSTR:OFF <session> <nonce>\n
|
||||
```
|
||||
Sent once when PTT is released. This is the fast-stop signal. If lost, the
|
||||
150ms timeout acts as a backstop.
|
||||
|
||||
### Server → Client
|
||||
|
||||
**Partial transcript:**
|
||||
```
|
||||
HKMSTR:P <text>\n
|
||||
```
|
||||
Intermediate recognition result. Fire-and-forget. Client logs it but does
|
||||
not act on it.
|
||||
The server echoes the session ID from the NOPs in all replies. The client
|
||||
drops any reply with a stale session ID.
|
||||
|
||||
**Final transcript:**
|
||||
**Partial segment (completed VAD segment):**
|
||||
```
|
||||
HKMSTR:F <text>\n
|
||||
HKMSTR:P <session> <text>\n
|
||||
```
|
||||
Complete utterance. Client feeds this to the TTS engine.
|
||||
A completed, VAD-separated utterance segment. The client starts TTS
|
||||
synthesis immediately and buffers the audio output, but does **not** play
|
||||
it yet. Playback starts when `:F` arrives (or timeout).
|
||||
|
||||
**Final (all done):**
|
||||
```
|
||||
HKMSTR:F <session> <text>\n
|
||||
```
|
||||
Signals that all segments have been sent. May be empty
|
||||
(`HKMSTR:F <session>\n`). Triggers playback of all buffered audio on the
|
||||
client. If `<text>` is non-empty, the client synthesizes it before playing.
|
||||
|
||||
The purpose of this design is to minimize latency: TTS synthesis runs in
|
||||
parallel with recording, so by the time `:F` arrives, audio is already
|
||||
buffered and playback starts immediately.
|
||||
|
||||
## Server state machine
|
||||
|
||||
@@ -79,15 +92,27 @@ Complete utterance. Client feeds this to the TTS engine.
|
||||
│ PROCESSING │ │
|
||||
└─────────────┘ │
|
||||
│ │
|
||||
STT │ │
|
||||
done │ │
|
||||
send │ │
|
||||
:P/:F │ │
|
||||
▼ │
|
||||
send HKMSTR:F ─────────┘
|
||||
back to IDLE ──────────┘
|
||||
```
|
||||
|
||||
- **IDLE → RECORDING:** first NOP received, start mic capture
|
||||
- **RECORDING:** VAD detects completed segments → send `HKMSTR:P <text>`
|
||||
- **RECORDING → PROCESSING:** OFF received, OR 150ms since last NOP
|
||||
- **PROCESSING → IDLE:** STT done, send `HKMSTR:F <text>`
|
||||
- **PROCESSING → IDLE:** send remaining segments as `:P`, then `HKMSTR:F`
|
||||
|
||||
## Client playback model
|
||||
|
||||
1. `:P` arrives → start TTS synthesis immediately, buffer audio (don't play).
|
||||
Reset 250ms segment timer.
|
||||
2. More `:P` arrive → keep synthesizing and buffering, reset timer each time.
|
||||
3. `:F` arrives → play all buffered audio immediately, cancel timer.
|
||||
4. If `:F` doesn't arrive within 250ms of the last `:P` → play buffered audio
|
||||
early. If more `:P` arrive after early playback, synthesis continues and
|
||||
new audio is appended to the output — not a failure.
|
||||
5. `:F` may be empty — it just signals "all segments sent, start/confirm playback."
|
||||
|
||||
## Timing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user