Robovoice STT Protocol

Overview

Robovoice connects to the STT server over TCP (typically through a gatuna L2 tunnel). The server captures audio, runs Moonshine STT, and sends transcript segments back. The client pre-synthesizes TTS on segments and plays audio on final.

[Robovoice client] --TCP--> [STT server 127.0.0.1:6996]
     │                            │
     ├── ON <session>\n  ────────►│  (abort old, start new session)
     ├── OFF <session>\n ────────►│  (stop, final STT pass)
     │◄── P <session> <text>\n ──┤  (completed segment)
     │◄── F <session> <text>\n ──┤  (all done; text may be empty)

Transport

Protocol: TCP (reliable, ordered, connection-oriented)
Server: 127.0.0.1:6996 (hardcoded loopback)
Framing: newline-delimited text (\n), UTF-8
Auto-reconnect: client retries every 3s if connection drops

Wire format

Client → Server

ON (PTT pressed):

ON <session>\n

Starts a new STT session. The server aborts any active session and starts recording. <session> is an incrementing unsigned integer chosen by the client. Replies from the server echo this session ID.

OFF (PTT released):

OFF <session>\n

Stops the session. The server does a final STT pass on remaining audio and sends any new segments followed by F.

Server → Client

Segment (completed VAD segment):

P <session> <text>\n

A completed, VAD-separated utterance segment. The client starts TTS synthesis immediately and buffers the audio (does not play yet).

Final (all done):

F <session> <text>\n

Signals all segments have been sent. <text> may be empty (F <session>\n). Triggers playback of all buffered audio on the client. If text is non-empty, the client synthesizes it before playing.

Session IDs

Client increments session ID on each PTT press
Server echoes the session ID in all replies for that session
Client drops any reply with a stale session ID (handles the race where stale segments from an aborted session are still in the TCP buffer)
Server aborts old session on receiving ON with a new session ID

Client playback model

P arrives → start TTS synthesis immediately, buffer audio (don't play)
More P arrive → keep synthesizing and buffering
F arrives → play all buffered audio immediately
PTT pressed → flush: stop playback, cancel synthesis, clear buffers

The purpose of pre-synthesis is to minimize latency between PTT release and audio playback. By the time F arrives, audio is already buffered.

Server state machine

              ┌──────────┐  ON <session>   ┌──────────────┐
              │  IDLE    │ ──────────────► │  RECORDING   │
              └──────────┘                 └──────────────┘
                                                │      │
                                        OFF     │      │
                                        recv'd  │      │
                                                ▼      │
                                            ┌─────────────┐
                                            │  PROCESSING │
                                            └─────────────┘
                                                 │
                                          send   │
                                          P/F    │
                                                 ▼
                                           back to IDLE

IDLE → RECORDING: ON <session> received, start mic capture
RECORDING: Moonshine streaming produces completed segments → send P
RECORDING → PROCESSING: OFF <session> received
PROCESSING → IDLE: final STT pass, send remaining P + F

If ON arrives while recording, the current session is aborted (no final flush) and a new session starts immediately.

4.1 KiB Raw Blame History