1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
|
# Server ↔ client transport ideas
Status: design notes, written 2026-11-08. Items 1–5 were adopted (with the
differences recorded in `docs/adr/0004-protocol-v5.md`): correlation ids,
per-connection idempotency, clock sync + ticks, commit–reveal crash, and
length-prefixed framing. Items 6–7 remain future work.
These came out of reviewing zagan-bet's transport, but they were prompted by a
*different* app. Nothing here is a backlog item for this repository. The
zagan-bet paths are included because that is where the patterns were worked
out; treat the ideas as portable.
zagan-bet's current shape, for context:
- TLS with TOFU over the certificate's SPKI fingerprint, trust decision after
the handshake and before credentials (`src/client/transport.rs`,
`src/tofu.rs`).
- Newline-delimited text lines over the TLS stream (`src/protocol.rs`).
- `HELLO MAJOR.MINOR` lockstep before auth (`src/version.rs`).
- Request/reply plus exactly one server-initiated line, `CRASH_BUSTED`
(`src/bin/server/handler.rs`, `ServerCore::poll`).
- `ClientCore` is I/O-free; frontends send `Command` and render `View`
(`src/client/core.rs`). The GUI owns the stream in `gui/src/net.rs`.
Each idea below lists the wire shape, where it would attach, and the cost.
## 1. Correlation IDs, so a single `Pending` can go away
One request in flight, and pushes distinguished by string prefix
(`expects_push`, `CRASH_BUSTED`, `src/client/core.rs:622`). A late reply can
race a pushed bust; the code already has a special case for it
(`src/client/core.rs:211`). An envelope removes the whole class:
```
REQ 42 idem=f3a1... INFO
RES 42 OK balance=1200 account=...
EVT seq=913 crash.tick {"session_id":"...","server_ms":837412,"multiplier":1.42}
```
`REQ`/`RES` correlate by id; `EVT` is connection-scoped and never answers
anything. The core keeps a map of in-flight requests instead of one `Option`,
and "a stale reply must not clobber the result" becomes a generic rule rather
than a crash-specific branch. Also allows pipelining, e.g. clock probes while a
`HISTORY` is in flight, without app-layer head-of-line stalls. Keep line
framing for now; just prepend tokens.
## 2. Idempotency keys on every balance mutation
A request can be applied server-side and the reply lost. Today that is safe
only because nothing retries automatically; the moment a client retries or
resumes, it can double-debit.
```
REQ 43 idem=9c2e... PLAY_START crash 50
```
Server keeps `seen_requests (idem PRIMARY KEY, user, reply, created_at)`. The
replay check, the debit, and the row insert sit in the same transaction as the
existing reservation (`database::reserve_play`). A duplicate returns the stored
reply and never re-executes. TTL-sweep at startup next to
`recover_orphaned_reservations`. Scope by user, not connection, so a resumed
session cannot replay someone else's key.
## 3. Clock sync and a real tick stream
zagan-bet advances timed games on read timeouts (`POLL_INTERVAL`,
`src/bin/server/handler.rs:7`) and the client draws crash locally from an
`Instant`. Payouts are correctly server-clock authoritative, but the client
does not know its offset. Two parts:
**Clock sync**, Cristian/NTP style over the existing connection:
```
REQ 7 idem=... TIME t0=837412
RES 7 OK t0=837412 t1=837498 # t1 = server monotonic ms
```
`offset = t1 - (t0 + t2) / 2`; keep the minimum-RTT sample over a sliding
window. Render timed games from `server_now`.
**Ticks** at 10–20 Hz, monotonic `seq`, no reply semantics:
```
EVT seq=914 crash.tick {"session_id":"...","server_ms":837520,"multiplier":1.44}
```
Coalesce in the writer: replace an unqueued tick, never drop a settlement.
That also enables lag compensation — render at `server_now - presentation_delay`
(e.g. 150 ms) so the displayed number never exceeds what the server accepts at
the moment of a click. A cleaner fairness story than trusting the client clock.
## 4. Commit–reveal instead of a hidden `Option<f64>`
zagan-bet already withholds the crash point (`CrashState::crash_point`,
`src/crash.rs`). Make it verifiable:
```
RES 51 OK COMMIT 7f3a... 4b9c... # sha256(server_seed), round id
...round runs...
EVT seq=980 crash.busted {"seed":"<server_seed>","crash_point":2.37,...}
```
Player supplies a client seed; the server echoes it. Crash point derives from
`HMAC-SHA256(server_seed, client_seed)` → 52 bits → `[0, 1)` → the existing
`crash_point_from_uniform`. Verification is a pure function and belongs in a
unit test, not a network test. A hash chain lets the server pre-commit every
future round at session start; that is stronger than needed in most cases.
## 5. Framing: length prefix, then maybe QUIC
Weaknesses in newline framing:
- `LineBuffer` grows without bound; a client that never sends `\n` is a memory
DoS. Reject oversized frames *before* allocating.
- `from_utf8_lossy` silently rewrites corrupt input. Newline framing is only
safe while `serde_json` keeps escaping newlines — an implicit dependency.
```
[u32 LE len][u8 proto_version][u8 kind][payload]
```
This allows a move to `postcard`/CBOR later without touching the read loop.
The larger swing is QUIC (`quinn`): separate streams so a slow `HISTORY`
cannot block ticks, 0-RTT resumption, connection migration for NAT rebinds
mid-round, and unreliable datagrams for ticks where latest-wins beats
retransmission. zagan-bet already uses rustls on the client, so this would also
drop OpenSSL from the server build. Cost: UDP reachability, TLS 1.3 only, a
rustls bump.
## 6. Resume tokens and a blocking reader thread
`AUTH` returns an opaque HMAC-signed resume token plus a session id;
`RESUME <token> since=<seq>` replays missed `EVT`s from a bounded per-session
buffer. Turns disconnect from "forfeit everything" into a policy choice; a
server-clocked round is trivially resumable. In zagan-bet this contradicts the
AGENTS rule that disconnect = forfeit, so it would need an explicit carve-out
and a grace window.
Separately, both ends poll with timers: the server's 50 ms read timeout, and
the GUI flipping `SO_RCVTIMEO` to 1 ms every 8 ms in
`TlsStream::read_available_line`. A dedicated blocking reader thread feeding an
`mpsc` removes the per-call timeout mutation, `expects_push`, and
`POLL_INTERVAL`; the handler becomes a `select` over inbound lines, a tick
deadline, and shutdown. Still no async runtime.
## 7. Small ones worth batching
- `HELLO 5.0 caps=idem,tick,commit` with server-side intersection instead of
strict MAJOR.MINOR lockstep for additive changes. Keep MAJOR for framing.
- Structured errors: `ERR code=INSUFFICIENT_FUNDS retryable=0 msg=...` so the
frontend can tell "reconnect" from "show text".
- `set_nodelay(true)` plus an app-level `PING` deadline; a half-open TCP
connection is otherwise noticed only on the next write.
- Per-connection token-bucket rate limiting. Every inbound line is logged, so a
chatty client is also a log-flood/disk problem.
- A protocol fuzzer (`proptest`/`cargo-fuzz`) asserting the server never panics
and `balance >= 0` for arbitrary input, plus transcript-replay tests.
## If only three
1. Correlation IDs + idempotency keys — small, kills two bug classes.
2. Clock sync + coalesced ticks — turns a special-case push into a generic
event channel.
3. Commit–reveal — cheap, and the strongest fairness story for a betting
server.
The envelope plus idempotency is the least invasive pair: it touches
`src/protocol.rs`, `ServerCore::handle_message`, and
`ClientCore::handle_reply`, and can sit behind a `caps=` flag so version
lockstep stays intact.
|
