hackmaster/Robovoice
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v0.8: TCP transport via gatuna tunnel, session IDs, pre-synth playback, server Rust rewrite

Browse Source
This commit is contained in:
Mute
2026-08-13 09:58:33 +00:00
parent ecf00c1bc4
commit facbfe6a5c
16 changed files with 955 additions and 638 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Server/PROTOCOL.md
+68 -98
View File
@@ -1,136 +1,106 @@
# Robovoice DHCP Tunnel Protocol
# Robovoice STT Protocol
## Overview
Robovoice communicates with a remote STT server by tunneling through the
DHCP UDP ports (68→67). This exploits a common killswitch exception: VPN
software (e.g. WireGuard) blocks all traffic except DHCP, which is allowed
for network connectivity maintenance.
Robovoice connects to the STT server over TCP (typically through a gatuna
L2 tunnel). The server captures audio, runs Moonshine STT, and sends
transcript segments back. The client pre-synthesizes TTS on segments and
plays audio on final.
```
[Robovoice client] --broadcast UDP :68→:67--> [STT server]
[Robovoice client] <--unicast UDP :67→:68-- [STT server]
[Robovoice client] --TCP--> [STT server 127.0.0.1:6996]
│ │
├── ON <session>\n ────────►│ (abort old, start new session)
├── OFF <session>\n ────────►│ (stop, final STT pass)
│◄── P <session> <text>\n ──┤ (completed segment)
│◄── F <session> <text>\n ──┤ (all done; text may be empty)
```
The client broadcasts NOP heartbeats while PTT is held. The server starts
recording on the first NOP and stops when it receives OFF or when 150ms
pass with no NOPs.
## Transport
- **Protocol:** UDP (connectionless, unreliable)
- **Client → Server:** broadcast, source port 68, dest port 67
- **Server → Client:** unicast, source port 67, dest port 68
- **Client binds:** to a specific LAN interface IP on port 68 (with
`SO_REUSEADDR` to coexist with the Windows DHCP service)
- **No connection state** — purely fire-and-forget datagrams
- **Protocol:** TCP (reliable, ordered, connection-oriented)
- **Server:** `127.0.0.1:6996` (hardcoded loopback)
- **Framing:** newline-delimited text (`\n`), UTF-8
- **Auto-reconnect:** client retries every 3s if connection drops
## Wire format
All messages are plain text, newline-terminated (`\n`). Every message starts
with the 6-byte magic `HKMSTR` to distinguish our traffic from real DHCP.
### Client → Server
**NOP (heartbeat while PTT held):**
**ON (PTT pressed):**
```
HKMSTR <session> <nonce>\n
ON <session>\n
```
Sent every 50ms while PTT is held. The session is an incrementing unsigned
integer that identifies the current PTT utterance (incremented on each PTT
press). The nonce is an incrementing unsigned integer that makes each
datagram unique. The server should echo the session back in replies. Both
are discarded by the server for protocol logic — the server tracks liveness
via "did anything arrive recently."
Starts a new STT session. The server aborts any active session and starts
recording. `<session>` is an incrementing unsigned integer chosen by the
client. Replies from the server echo this session ID.
**OFF (PTT released):**
```
HKMSTR:OFF <session> <nonce>\n
OFF <session>\n
```
Sent once when PTT is released. This is the fast-stop signal. If lost, the
150ms timeout acts as a backstop.
Stops the session. The server does a final STT pass on remaining audio and
sends any new segments followed by `F`.
### Server → Client
The server echoes the session ID from the NOPs in all replies. The client
drops any reply with a stale session ID.
**Partial segment (completed VAD segment):**
**Segment (completed VAD segment):**
```
HKMSTR:P <session> <text>\n
P <session> <text>\n
```
A completed, VAD-separated utterance segment. The client starts TTS
synthesis immediately and buffers the audio output, but does **not** play
it yet. Playback starts when `:F` arrives (or timeout).
synthesis immediately and buffers the audio (does not play yet).
**Final (all done):**
```
HKMSTR:F <session> <text>\n
F <session> <text>\n
```
Signals that all segments have been sent. May be empty
(`HKMSTR:F <session>\n`). Triggers playback of all buffered audio on the
client. If `<text>` is non-empty, the client synthesizes it before playing.
Signals all segments have been sent. `<text>` may be empty (`F <session>\n`).
Triggers playback of all buffered audio on the client. If text is non-empty,
the client synthesizes it before playing.
The purpose of this design is to minimize latency: TTS synthesis runs in
parallel with recording, so by the time `:F` arrives, audio is already
buffered and playback starts immediately.
## Session IDs
- Client increments session ID on each PTT press
- Server echoes the session ID in all replies for that session
- Client drops any reply with a stale session ID (handles the race where
stale segments from an aborted session are still in the TCP buffer)
- Server aborts old session on receiving `ON` with a new session ID
## Client playback model
1. `P` arrives → start TTS synthesis immediately, buffer audio (don't play)
2. More `P` arrive → keep synthesizing and buffering
3. `F` arrives → play all buffered audio immediately
4. PTT pressed → flush: stop playback, cancel synthesis, clear buffers
The purpose of pre-synthesis is to minimize latency between PTT release
and audio playback. By the time `F` arrives, audio is already buffered.
## Server state machine
```
┌──────────────────────────────────────────┐
│ │
▼ │
┌──────────┐ first NOP ┌──────────────┐ │
│ IDLE │ ──────────► │ RECORDING │ │
└──────────┘ └──────────────┘ │
│ │ │
OFF │ │ 150ms │
recv'd │ │ silence │
▼ ▼ │
┌─────────────┐ │
│ PROCESSING │ │
└─────────────┘ │
│ │
send │ │
:P/:F │ │
▼ │
back to IDLE ──────────┘
┌──────────┐ ON <session> ┌──────────────┐
│ IDLE │ ──────────────► │ RECORDING │
└──────────┘ └──────────────┘
│ │
OFF │ │
recv'd │ │
▼ │
┌─────────────┐
│ PROCESSING │
└─────────────┘
│
send │
P/F │
▼
back to IDLE
```
- **IDLE → RECORDING:** first NOP received, start mic capture
- **RECORDING:** VAD detects completed segments → send `HKMSTR:P <text>`
- **RECORDING → PROCESSING:** OFF received, OR 150ms since last NOP
- **PROCESSING → IDLE:** send remaining segments as `:P`, then `HKMSTR:F`
- **IDLE → RECORDING:** `ON <session>` received, start mic capture
- **RECORDING:** Moonshine streaming produces completed segments → send `P`
- **RECORDING → PROCESSING:** `OFF <session>` received
- **PROCESSING → IDLE:** final STT pass, send remaining `P` + `F`
## Client playback model
1. `:P` arrives → start TTS synthesis immediately, buffer audio (don't play).
Reset 250ms segment timer.
2. More `:P` arrive → keep synthesizing and buffering, reset timer each time.
3. `:F` arrives → play all buffered audio immediately, cancel timer.
4. If `:F` doesn't arrive within 250ms of the last `:P` → play buffered audio
early. If more `:P` arrive after early playback, synthesis continues and
new audio is appended to the output — not a failure.
5. `:F` may be empty — it just signals "all segments sent, start/confirm playback."
## Timing
| Parameter | Value | Purpose |
|-----------|-------|---------|
| NOP interval | 50ms | Heartbeat frequency while PTT held |
| Silence timeout | 150ms | Stop recording if no NOPs (3 missed = lost OFF) |
| NOP bandwidth | ~20 msg/s × ~20 bytes | ~400 bytes/s — negligible |
## Why this works
1. **Outbound broadcast `:68→:67` to `255.255.255.255`** passes the
WireGuard WFP killswitch (DHCP exception matches this exact pattern)
2. **Inbound `:67→:68`** has no address restriction in the WFP rule, so
unicast replies pass through
3. **Binding to a specific interface IP** (not `0.0.0.0`) wins unicast
delivery over the Windows DHCP client service
4. **NOP spam** ensures the ON message gets through even at 5% packet loss
(3 consecutive NOPs = ~0.01% drop probability)
5. **150ms timeout** is the backstop for lost OFF — at 50ms intervals, 3
consecutive NOPs must all be lost to false-stop
If `ON` arrives while recording, the current session is aborted (no final
flush) and a new session starts immediately.