A Protocol of Her Own

One mind, one process, and the wall it just hit

Everything she does today happens inside a single Python process. The event bus from chapter 15 feels like modules talking to each other, but every message on it is a function call: publisher and subscriber share one interpreter, one memory space, one crash. That was the right first architecture, and it has quietly become the ceiling. The language model wants the machine with the GPU. The body she is getting runs on a microcontroller bolted to an arm. Two programs on two machines cannot share a Python dict. They can only share a wire.

The wire is nearly free: a TCP socket, the reliable pipe every higher protocol is built on. TCP promises that the bytes you send arrive, in order, without gaps or duplicates. What it does not promise (and this trips nearly everyone exactly once) is anything about messages. TCP is a byte stream. Send {"command": "status"} and the receiver's recv(4096) may hand back the whole thing, half of it, or all of it plus the front of the next message; the operating system batches and splits traffic however it likes. Call json.loads() on half an object and the error reads like corrupt data, but nothing is corrupt. You assumed a boundary the protocol never drew.

Drawing that boundary yourself is called framing, and the volume closes by doing it in the simplest form that works: one message is one line — serialize the dict to JSON, append a newline, and on the other side refuse to parse until that newline has arrived. Held on both ends, that one-sentence contract turns a stream of bytes into a sequence of dicts her modules can trust.

An echo first, then a protocol

# labs/wire.py
import json
import socket
import threading
import time

HOST = "localhost"
PORT = 9999

def serve_raw() -> None:
    server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server.bind((HOST, PORT))
    server.listen(1)
    conn, _addr = server.accept()
    with conn:
        raw = conn.recv(4096)   # one read
        conn.sendall(raw)       # echo the exact bytes back
    server.close()

if __name__ == "__main__":
    threading.Thread(target=serve_raw, daemon=True).start()
    time.sleep(0.1)
    with socket.create_connection((HOST, PORT)) as sock:
        sock.sendall(json.dumps({"command": "status"}).encode() + b"\n")
        reply = sock.recv(4096)
    print(f"Raw reply: {reply!r}")

$ uv run python labs/wire.py
Raw reply: b'{"command": "status"}\n'

def recv_message(sock: socket.socket) -> dict:
    data = b""
    while True:
        chunk = sock.recv(4096)
        if not chunk:               # empty bytes: the peer closed
            break
        data += chunk
        if data.endswith(b"\n"):    # the frame is complete
            break
    return json.loads(data.decode().strip())

def serve_once() -> None:
    server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server.bind((HOST, PORT))
    server.listen(1)
    conn, _addr = server.accept()
    with conn:
        msg = recv_message(conn)
        response = {"status": "ok", "echo": msg}
        conn.sendall((json.dumps(response) + "\n").encode())
    server.close()

$ uv run python labs/wire.py
Parsed reply: {'status': 'ok', 'echo': {'command': 'status', 'source': 'glados'}}

# labs/wire.py — full file
import json
import socket
import threading
import time

HOST = "localhost"
PORT = 9999

def recv_message(sock: socket.socket) -> dict:
    data = b""
    while True:
        chunk = sock.recv(4096)
        if not chunk:
            break
        data += chunk
        if data.endswith(b"\n"):
            break
    return json.loads(data.decode().strip())

def send_message(host: str, port: int, message: dict) -> dict:
    with socket.create_connection((host, port), timeout=5) as sock:
        sock.sendall((json.dumps(message) + "\n").encode())
        return recv_message(sock)

def handle_client(conn: socket.socket) -> None:
    with conn:
        msg = recv_message(conn)
        response = {"status": "ok", "echo": msg}
        conn.sendall((json.dumps(response) + "\n").encode())

def serve_once(host: str, port: int) -> None:
    server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    server.bind((host, port))
    server.listen(1)
    conn, _addr = server.accept()
    handle_client(conn)
    server.close()

def main() -> None:
    threading.Thread(target=serve_once, args=(HOST, PORT), daemon=True).start()
    time.sleep(0.1)
    response = send_message(HOST, PORT, {"command": "status", "source": "glados"})
    print(f"Response: {response}")

if __name__ == "__main__":
    main()

$ uv run python labs/wire.py
Response: {'status': 'ok', 'echo': {'command': 'status', 'source': 'glados'}}

Why this works: a contract about endings

A TCP connection gives you ordering and delivery but no record boundaries; imposing structure on the stream is always the receiver's job. The delimiter makes that job finite, and given it, recv_message is a three-case machine. Buffer ends with \n: the frame is complete, parse it. Chunk is non-empty but no delimiter yet: a partial frame, loop and append. Chunk is empty bytes: EOF, the peer closed, stop and parse what arrived. Only one combination hangs it: a peer that sends no newline and holds the connection open.

One worry to retire early: what if a message contains a newline, a transcript with line breaks, say? It cannot break the frame, because json.dumps never emits a raw newline inside a string value; it escapes it to a backslash followed by an n, two ordinary characters. The first actual 0x0A byte on the wire is provably the one we appended: the delimiter is safe because the serializer promises never to use it.

Newlines are one answer to the question every protocol must answer: where does a message end? The other classic answer is a length prefix, where a few bytes announce the payload size and the receiver reads exactly that many. You have met it without the name: the WAV files of chapter 3 open every chunk with a declared size, length-prefix framing on disk. Prefixes win for binary payloads; newlines win for JSON, being trivial to implement and readable in a terminal. What never works is the third option people ship by accident, hoping one recv equals one message — the failure below runs it so you never have to.

def serve_once() -> None:
    ...
    conn, _addr = server.accept()
    with conn:
        raw = conn.recv(4096)            # one read, assumed complete
        msg = json.loads(raw.decode())   # BUG: raw may be half a frame
        conn.sendall((json.dumps({"echo": msg}) + "\n").encode())

# and the client sends a payload bigger than one segment:
big = {"command": "log", "text": "x" * 8000}

$ uv run python labs/wire.py
Exception in thread Thread-1 (serve_once):
Traceback (most recent call last):
  ...
    msg = json.loads(raw.decode())   # BUG: raw may be half a frame
  File ".../json/decoder.py", line 361, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 28 (char 27)

Checkpoint at the edge of the volume

Stand back and look at what this volume added. She curates her own few-shot examples from the exchanges you rated well. Her preferences overlay your choices onto safe defaults. Her speech is normalized before the voice reads it, and she reads your mood back, letting it fade honestly with time. The lab notebook records every experiment with the hypothesis written before the result. Her plans are data you can query. Her body already exists as a simulated interface that clamps what you ask of it. The pipeline reports where its milliseconds go, her sensor logs bound their own growth, and as of today her processes own a wire protocol with framing they can trust. Volume 2 gave her a mind; this volume gave her a craft: habits of measurement, honesty, and repair that a growing system needs more than any single ability.

Volume 4 is called One System, and it earns the name: the pile of working scripts becomes an architecture drawn on purpose, modules composed behind explicit seams, with a startup sequence that validates every dependency before she says a word. Then the protocol you built today gets its first real job: a wire out of the computer entirely, to an ESP32 microcontroller listening on the far end. Her body's nervous system starts there, and it speaks in framed messages because you decided, this chapter, where a message ends.