move PROTOCOL.md to repo root

2026-08-13 10:57:56 +00:00
parent b034ed15df
commit fbbb9968b1
1 changed files with 0 additions and 0 deletions
@@ -0,0 +1,106 @@
+# Robovoice STT Protocol
+
+## Overview
+
+Robovoice connects to the STT server over TCP (typically through a gatuna
+L2 tunnel). The server captures audio, runs Moonshine STT, and sends
+transcript segments back. The client pre-synthesizes TTS on segments and
+plays audio on final.
+
+```
+[Robovoice client] --TCP--> [STT server 127.0.0.1:6996]
+     │                            │
+     ├── ON <session>\n  ────────►│  (abort old, start new session)
+     ├── OFF <session>\n ────────►│  (stop, final STT pass)
+     │◄── P <session> <text>\n ──┤  (completed segment)
+     │◄── F <session> <text>\n ──┤  (all done; text may be empty)
+```
+
+## Transport
+
+- **Protocol:** TCP (reliable, ordered, connection-oriented)
+- **Server:** `127.0.0.1:6996` (hardcoded loopback)
+- **Framing:** newline-delimited text (`\n`), UTF-8
+- **Auto-reconnect:** client retries every 3s if connection drops
+
+## Wire format
+
+### Client → Server
+
+**ON (PTT pressed):**
+```
+ON <session>\n
+```
+Starts a new STT session. The server aborts any active session and starts
+recording. `<session>` is an incrementing unsigned integer chosen by the
+client. Replies from the server echo this session ID.
+
+**OFF (PTT released):**
+```
+OFF <session>\n
+```
+Stops the session. The server does a final STT pass on remaining audio and
+sends any new segments followed by `F`.
+
+### Server → Client
+
+**Segment (completed VAD segment):**
+```
+P <session> <text>\n
+```
+A completed, VAD-separated utterance segment. The client starts TTS
+synthesis immediately and buffers the audio (does not play yet).
+
+**Final (all done):**
+```
+F <session> <text>\n
+```
+Signals all segments have been sent. `<text>` may be empty (`F <session>\n`).
+Triggers playback of all buffered audio on the client. If text is non-empty,
+the client synthesizes it before playing.
+
+## Session IDs
+
+- Client increments session ID on each PTT press
+- Server echoes the session ID in all replies for that session
+- Client drops any reply with a stale session ID (handles the race where
+  stale segments from an aborted session are still in the TCP buffer)
+- Server aborts old session on receiving `ON` with a new session ID
+
+## Client playback model
+
+1. `P` arrives → start TTS synthesis immediately, buffer audio (don't play)
+2. More `P` arrive → keep synthesizing and buffering
+3. `F` arrives → play all buffered audio immediately
+4. PTT pressed → flush: stop playback, cancel synthesis, clear buffers
+
+The purpose of pre-synthesis is to minimize latency between PTT release
+and audio playback. By the time `F` arrives, audio is already buffered.
+
+## Server state machine
+
+```
+              ┌──────────┐  ON <session>   ┌──────────────┐
+              │  IDLE    │ ──────────────► │  RECORDING   │
+              └──────────┘                 └──────────────┘
+                                                │      │
+                                        OFF     │      │
+                                        recv'd  │      │
+                                                ▼      │
+                                            ┌─────────────┐
+                                            │  PROCESSING │
+                                            └─────────────┘
+                                                 │
+                                          send   │
+                                          P/F    │
+                                                 ▼
+                                           back to IDLE
+```
+
+- **IDLE → RECORDING:** `ON <session>` received, start mic capture
+- **RECORDING:** Moonshine streaming produces completed segments → send `P`
+- **RECORDING → PROCESSING:** `OFF <session>` received
+- **PROCESSING → IDLE:** final STT pass, send remaining `P` + `F`
+
+If `ON` arrives while recording, the current session is aborted (no final
+flush) and a new session starts immediately.