Robovoice/PROTOCOL.md

# Robovoice STT Protocol

## Overview

Robovoice connects to the STT server over TCP (typically through a gatuna
L2 tunnel). The server captures audio, runs Moonshine STT, and sends
transcript segments back. The client pre-synthesizes TTS on segments and
plays audio on final.

```
[Robovoice client] --TCP--> [STT server 127.0.0.1:6996]
     │                            │
     ├── ON <session>\n  ────────►│  (abort old, start new session)
     ├── OFF <session>\n ────────►│  (stop, final STT pass)
     │◄── P <session> <text>\n ──┤  (completed segment)
     │◄── F <session> <text>\n ──┤  (all done; text may be empty)
```

## Transport

- **Protocol:** TCP (reliable, ordered, connection-oriented)
- **Server:** `127.0.0.1:6996` (hardcoded loopback)
- **Framing:** newline-delimited text (`\n`), UTF-8
- **Auto-reconnect:** client retries every 3s if connection drops

## Wire format

### Client → Server

**ON (PTT pressed):**
```
ON <session>\n
```
Starts a new STT session. The server aborts any active session and starts
recording. `<session>` is an incrementing unsigned integer chosen by the
client. Replies from the server echo this session ID.

**OFF (PTT released):**
```
OFF <session>\n
```
Stops the session. The server does a final STT pass on remaining audio and
sends any new segments followed by `F`.

### Server → Client

**Segment (completed VAD segment):**
```
P <session> <text>\n
```
A completed, VAD-separated utterance segment. The client starts TTS
synthesis immediately and buffers the audio (does not play yet).

**Final (all done):**
```
F <session> <text>\n
```
Signals all segments have been sent. `<text>` may be empty (`F <session>\n`).
Triggers playback of all buffered audio on the client. If text is non-empty,
the client synthesizes it before playing.

## Session IDs

- Client increments session ID on each PTT press
- Server echoes the session ID in all replies for that session
- Client drops any reply with a stale session ID (handles the race where
  stale segments from an aborted session are still in the TCP buffer)
- Server aborts old session on receiving `ON` with a new session ID

## Client playback model

1. `P` arrives → start TTS synthesis immediately, buffer audio (don't play)
2. More `P` arrive → keep synthesizing and buffering
3. `F` arrives → play all buffered audio immediately
4. PTT pressed → flush: stop playback, cancel synthesis, clear buffers

The purpose of pre-synthesis is to minimize latency between PTT release
and audio playback. By the time `F` arrives, audio is already buffered.

## Server state machine

```
              ┌──────────┐  ON <session>   ┌──────────────┐
              │  IDLE    │ ──────────────► │  RECORDING   │
              └──────────┘                 └──────────────┘
                                                │      │
                                        OFF     │      │
                                        recv'd  │      │
                                                ▼      │
                                            ┌─────────────┐
                                            │  PROCESSING │
                                            └─────────────┘
                                                 │
                                          send   │
                                          P/F    │
                                                 ▼
                                           back to IDLE
```

- **IDLE → RECORDING:** `ON <session>` received, start mic capture
- **RECORDING:** Moonshine streaming produces completed segments → send `P`
- **RECORDING → PROCESSING:** `OFF <session>` received
- **PROCESSING → IDLE:** final STT pass, send remaining `P` + `F`

If `ON` arrives while recording, the current session is aborted (no final
flush) and a new session starts immediately.