gatuna/PROTOCOL.md

# gatuna wire protocol

All frames ride Ethernet with ethertype `0x6969`. The ethertype is the sole
discriminator; there is no magic number inside the payload.

## Frame layout (non-DATA frames)

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+---------------+---------------+-------------------------------+
| version       | type          | session_id                    |
+---------------+---------------+-------------------------------+
| payload_len (big-endian)      | payload ...                   |
+-------------------------------+                               +
|                                                               |
+---------------------------------------------------------------+
```

- **version** (u8): protocol version. Currently `2`.
- **type** (u8): frame type, see table below.
- **session_id** (u32, big-endian): `0` for non-session frames; the
  server-assigned ID for session-scoped frames.
- **payload_len** (u16, big-endian): number of payload bytes that follow.
  The receiver reads exactly this many bytes and ignores any trailing
  Ethernet padding (frames under 60 bytes are zero-padded by the NIC to
  meet the minimum Ethernet frame size).
- **payload** (`payload_len` bytes): type-dependent.

## DATA frame layout

DATA frames carry two additional fields after the common header for L2-level
reliability:

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+---------------+---------------+-------------------------------+
| version=2     | type=0x06     | session_id                    |
+---------------+---------------+-------------------------------+
| payload_len (big-endian)      | seq (big-endian)              |
+-------------------------------+-------------------------------+
| ack_seq (big-endian)          | payload ...                   |
+-------------------------------+                               +
|                                                               |
+---------------------------------------------------------------+
```

- **seq** (u32, big-endian): monotonically increasing per-session sequence
  number. Wraps at 2^32 (same as TCP). Identifies this DATA frame's position
  in the byte stream.
- **ack_seq** (u32, big-endian): cumulative acknowledgment — the highest
  contiguous `seq` that the sender of this frame has delivered to its local
  TCP socket. The receiver uses this to advance its retransmit window.
- **payload** (`payload_len` bytes): raw application bytes (0–1480). A
  `payload_len = 0` DATA frame is a **pure ACK** — it carries no data, just
  an acknowledgment. This mirrors TCP's empty-segment ACK.

Maximum payload: 1500 (Ethernet MTU) − 8 (common header) − 8 (seq + ack_seq)
= **1484 bytes**. In practice we cap at **1480** to stay conservative.

## Frame types

| Type | Name        | Direction     | session_id | Payload                          |
|------|-------------|---------------|------------|----------------------------------|
| 0x01 | DISCOVER    | C → broadcast | 0          | empty                            |
| 0x02 | MANIFEST    | S → C         | 0          | `hostname_len:1, hostname:N, entries...` |
| 0x03 | OPEN        | C → S         | 0          | `upstream_id:1, proto:1`        |
| 0x04 | OPEN_ACK    | S → C         | assigned   | `upstream_id:1, proto:1`        |
| 0x05 | OPEN_NAK    | S → C         | 0          | `upstream_id:1, reason:1`        |
| 0x06 | DATA        | both          | session    | `seq:4, ack_seq:4, raw bytes`    |
| 0x07 | CLOSE       | both          | session    | optional `reason:1`              |
| 0x0B | PING        | C → S         | 0          | `nonce:8`                        |
| 0x0C | PONG        | S → C         | 0          | `nonce:8` (echoed)               |

Reserved (unimplemented; parse returns Err, encode unimplemented):

| Type | Name        |
|------|-------------|
| 0x08 | UDP_OPEN    |
| 0x09 | UDP_DATA    |
| 0x0A | UDP_CLOSE   |

## L2 reliability

The reliability semantics of a session depend on its transport proto,
which is carried end-to-end through OPEN/OPEN_ACK:

- **TCP (`proto = 1`):** DATA frames are delivered reliably and in-order via
  a TCP-like sequence + cumulative ACK + retransmit mechanism. This ensures
  that a dropped Ethernet frame does not permanently corrupt a TCP session
  (which would otherwise happen because the local kernel TCP stack ACKs data
  before it is chunked into DATA frames).
- **UDP (`proto = 2`, reserved):** DATA frames are delivered best-effort. The
  `seq` and `ack_seq` fields are present in the frame layout for uniformity
  but are ignored — no retransmit, no in-order buffering, no pure ACKs. Drops
  are tolerated because stateless protocols either don't care or handle
  recovery at the application layer.

The following sections describe the TCP reliability mechanism.

### Sender state (per session)

- `send_seq`: next seq to assign (starts at 0, increments per DATA frame).
- `retransmit_buffer`: map of `seq → (payload, timestamp)`, holding all sent
  but unacked frames.
- `acked_seq`: highest seq acknowledged by the peer (initially `None`).

On sending DATA:
1. Assign `seq = send_seq; send_seq += 1`.
2. Set `ack_seq` to the highest contiguous seq we have received from the peer
   (our receive side's `deliver_seq`).
3. Store `(payload, now)` in `retransmit_buffer[seq]`.
4. Transmit the frame.

On receiving an `ack_seq` in any DATA frame (including pure ACKs):
1. Advance `acked_seq` to `max(acked_seq, ack_seq)`.
2. Remove all entries from `retransmit_buffer` with `seq <= ack_seq`.

Retransmit timer (per session, checked periodically):
1. For each entry in `retransmit_buffer` older than `RETRANSMIT_TIMEOUT`
   (default 5 ms), retransmit the frame and reset its timestamp.
2. If any entry has been retransmitted more than `MAX_RETRIES` times (default
   10), send `CLOSE` and tear down the session.

### Receiver state (per session)

- `expected_seq`: next seq expected (starts at 0).
- `receive_buffer`: map of `seq → payload`, holding out-of-order frames.
- `deliver_seq`: highest seq delivered to the local TCP socket (starts at
  `None`; reported as `ack_seq` in outgoing DATA frames).

On receiving DATA with `seq`:
1. If `seq < expected_seq`: duplicate (already delivered). Discard the payload,
   but still process the `ack_seq` field to advance the send window. Send a
   pure ACK so the sender can converge.
2. If `seq == expected_seq`: deliver payload to the TCP socket. Increment
   `expected_seq`. Then check `receive_buffer` for the next contiguous seq and
   deliver those too (drain the buffer in order). Update `deliver_seq`.
3. If `seq > expected_seq`: store in `receive_buffer[seq]`. Do not deliver yet.
   Send a pure ACK (re-ACKing `deliver_seq`) to trigger retransmit of the gap.

### Pure ACK frames

When a side needs to ACK but has no data to send, it sends a DATA frame with
`payload_len = 0`. The `seq` field is set to `send_seq` (consuming a seq
number, same as TCP's empty segment) and `ack_seq` carries the cumulative
acknowledgment. The receiver processes the `ack_seq` and discards the empty
payload without delivering to the TCP socket.

### Acknowledgment timing

- When data is flowing in both directions, each DATA frame carries the latest
  `ack_seq` — no separate ACK frames needed.
- When data is one-directional, the receiver sends a pure ACK after each DATA
  frame (or after a small batch, implementation-defined).
- On receiving a duplicate or out-of-order frame, the receiver immediately
  sends a pure ACK to help the sender converge.

### Why this is not the Two Generals Problem

We do not need mutual consensus. We need one-sided reliable delivery: the
sender retransmits until it gets an ACK. If the ACK is lost, the sender
retransmits the data; the receiver sees a duplicate, discards it, and re-ACKs.
This converges in O(1) round trips. The only unsolvable case — the last frame
before a permanent link death — is handled by `MAX_RETRIES` → `CLOSE`, which is
correct: a dead link should kill the session.

## Payload field encodings

### MANIFEST payload

A hostname prefix followed by variable-length entries, parsed sequentially
until the payload is consumed.

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+---------------+-----------------------------------------------+
| hostname_len  | hostname (UTF-8, hostname_len bytes) ...       |
+---------------+-----------------------------------------------+
| id            | proto         | port (big-endian)             |
+---------------+---------------+-------------------------------+
| label_len     | label (UTF-8, label_len bytes) ...           |
+---------------+-----------------------------------------------+
| ... repeated ...                                             |
+---------------------------------------------------------------+
```

- **hostname_len** (u8): length in bytes of the server's hostname. `0` is valid
  (unknown hostname).
- **hostname** (`hostname_len` bytes, UTF-8): the server's hostname, read via
  `gethostname(2)` at startup. Maximum 255 bytes.
- **id** (u8): upstream identifier (1-based positional index from `gatunad`
  cmdline).
- **proto** (u8): `1 = TCP`, `2 = UDP` (reserved; not emitted in v1).
- **port** (u16, big-endian): the real port on the server's `127.0.0.1`.
- **label_len** (u8): length in bytes of the label that follows. `0` means no
  label.
- **label** (`label_len` bytes, UTF-8): human-readable name for the upstream,
  taken from the `PORT[:label]` cmdline argument. Maximum 255 bytes.

To parse: read `hostname_len`, then `hostname_len` bytes of hostname, then read
5-byte fixed entry prefixes + `label_len` bytes of label each, repeating until
the payload is exhausted. The number of entries is not carried explicitly.

### OPEN payload

```
+---------------+---------------+
| upstream_id   | proto         |
+---------------+---------------+
```

- **upstream_id** (u8): which MANIFEST entry to open.
- **proto** (u8): the transport protocol of the upstream (`1 = TCP`,
  `2 = UDP`). Carried end-to-end so both sides know which reliability
  semantics apply to the session. Must match the proto advertised in the
  MANIFEST for that upstream.

### OPEN_ACK payload

```
+---------------+---------------+
| upstream_id   | proto         |
+---------------+---------------+
```

- **upstream_id** (u8): echoes the requested upstream.
- **proto** (u8): echoes the requested proto. The session is identified by
  the `session_id` field in the header, not the payload.

### OPEN_NAK payload

```
+---------------+---------------+
| upstream_id   | reason        |
+---------------+---------------+
```

- **upstream_id** (u8): echoes the requested upstream.
- **reason** (u8): see reason codes.

### DATA payload

`[ seq:4 ][ ack_seq:4 ][ raw bytes ]` — the `seq` and `ack_seq` fields are
part of the DATA frame's extended header (between `payload_len` and the
payload). The `payload_len` field counts only the raw bytes, not the seq/ack
fields. Up to 1480 bytes of application data per frame.

### CLOSE payload

```
+---------------+
| reason?       |
+---------------+
```

- **reason** (u8, optional): present iff payload length ≥ 1. See reason codes.

### PING / PONG payload

```
+                                                               +
| nonce (big-endian, 8 bytes)                                   |
+                                                               +
```

- **nonce** (u64, big-endian): arbitrary value chosen by the client. The
  server echoes it verbatim in the PONG reply. Used to correlate RTT
  measurements.

## Reason codes

| Value | Meaning           |
|-------|-------------------|
| 0     | unspecified       |
| 1     | unknown_upstream  |
| 2     | connect_failed    |
| 3     | oversize          |
| 4     | unknown_session   |
| 5     | max_retries       |

## Discovery flow

```
client                            server
  |                                 |
  | DISCOVER (dst = broadcast)      |
  |-------------------------------->|
  |                                 |
  |        MANIFEST (unicast)       |
  |<--------------------------------|
  |                                 |
```

The server learns the client's MAC from the DISCOVER frame's source address and
unicasts the MANIFEST back. The server never speaks unsolicited.

## TCP session lifecycle

```
client                            server
  |                                 |
  | OPEN { upstream_id }            |
  |-------------------------------->|
  |                                 | TcpStream::connect(127.0.0.1:port)
  |                                 |
  |   OPEN_ACK { session_id }       |
  |<--------------------------------|   (on success)
  |        OR                       |
  |   OPEN_NAK { reason }           |
  |<--------------------------------|   (on failure)
  |                                 |
  | DATA { seq, ack_seq, bytes }    |
  |<------------------------------->| DATA { seq, ack_seq, bytes }
  |                                 |
  | CLOSE { session_id, reason? }   |
  |<------------------------------->|   (on EOF, RST, or max_retries)
  |                                 |
```

- `session_id` is allocated by the server as a monotonically increasing u32
  (starting at 1) from an atomic counter. Collision by wraparound is ignored.
- `seq` starts at 0 on both sides of each session and increments per DATA
  frame (including pure ACKs).
- Either side may send `CLOSE`. The side receiving `CLOSE` tears down its half
  and stops emitting frames for that session.
- The server's socket→tunnel pump reads `TcpStream` in 1480-byte chunks and
  emits one DATA frame per chunk. On `read` returning 0 (FIN) or an error,
  it emits `CLOSE` and exits.
- The server's tunnel→socket path delivers DATA payloads in seq order to the
  `TcpStream` with `write_all`. On error it emits `CLOSE` and drops the session.
- A DATA frame is retransmitted if no ACK is received within 5 ms. After 10
  failed retransmits, the session is closed with `reason = max_retries`.

## Network test (PING/PONG)

```
client                            server
  |                                 |
  | PING { nonce }                  |
  |-------------------------------->|
  |                                 |
  |        PONG { nonce }           |
  |<--------------------------------|
  |                                 |
  | (repeated at random intervals)  |
  |                                 |
```

- The client sends `PING` frames at random 10–100 ms intervals, each with a
  unique `nonce`.
- The server echoes the nonce verbatim in a `PONG` frame.
- The client correlates `PONG` nonces with outstanding `PING` timestamps to
  compute RTT, average latency, jitter (mean absolute delta of consecutive
  RTTs), and drop rate (unanswered PINGs).
- PING/PONG frames use `session_id = 0`; they are independent of TCP sessions.

## Version compatibility

- Version 1: no `seq`/`ack_seq` in DATA frames, no L2 reliability. Dropped
  DATA frames kill the TCP session.
- Version 2: DATA frames carry `seq` + `ack_seq`, L2 retransmit. Dropped
  frames are recovered.

Version 2 is the current version. A receiver that sees `version != 2` rejects
the frame. Both sides of a session must speak the same version; there is no
negotiation.