Watching Every Subsystem at Once
The browser can only ask
Chapter 92 gave her a monitor that keeps watching after everyone has gone to bed. It knows the disk is filling, it knows the second the model server stops answering, and it writes all of that into a journal on a machine with no screen attached. Chapter 89 put her behind HTTP, so anything on the network can ask her a question. Neither one gives you what you actually want on a phone propped against the monitor: a page that is simply correct, all the time, without being touched.
The obvious build is four lines of JavaScript. Fetch /health, draw the
cards, and do it again in a second. It works on the first try, and then you count what
it costs. One open tab is 3,600 requests an hour, 86,400 a day, and every one of them
re-runs the checks: a socket opened to Ollama, /proc/meminfo read and
parsed, the filesystem stat'd. Leave the page open on a laptop, a phone and a tablet
and the model server is now being probed three times a second by a screen nobody is
looking at.
The readings behind all that traffic change once a minute at most, because that is how often the monitor polls. So the page asks sixty times more often than the answer can possibly move, and is still up to a second late when it finally does. Tightening the interval to feel live makes both numbers worse at once.
The direction of HTTP is what makes this feel unavoidable. The browser asks and the
server answers, and a server with news has no way to speak first. But nothing in the
protocol says the answer has to end. That gap is where Server-Sent Events live, and
here is the whole idea before any code: an SSE endpoint is an ordinary HTTP
response with no declared length that never finishes, and the server writes one
data: line into it whenever there is something to say.
One limit belongs up front, because it decides the design. The stream runs one way.
Only the server writes to it, and the browser never sends anything back through it.
Commands keep going the way chapter 89 built them, as separate POST requests to
/speak and /ask. Two channels doing two jobs: a stream that
reports, and requests that ask for things.
One message, and the lines it is made of
# labs/dashboard.py
import json
import pathlib
import time
from labs.health_monitor import HealthStatus, run_checks
PAGE = pathlib.Path(__file__).with_name("dashboard.html")
def as_dict(status: HealthStatus) -> dict:
"""One check, written out as the four fields the wire carries."""
return {"name": status.name, "ok": status.ok,
"detail": status.detail, "value": status.value}
def snapshot(checks: list[HealthStatus], at: float) -> dict:
return {"at": time.strftime("%H:%M:%S", time.localtime(at)),
"overall": "ok" if all(c.ok for c in checks) else "degraded",
"checks": [as_dict(c) for c in checks]}
if __name__ == "__main__":
print(json.dumps(snapshot(run_checks(), time.time()), indent=2))
$ uv run python -m labs.dashboard # measured on the bench — yours will vary
{
"at": "09:53:02",
"overall": "ok",
"checks": [
{
"name": "ollama",
"ok": true,
"detail": "6 model(s) available",
"value": 6.0
},
{
"name": "disk",
"ok": true,
"detail": "71.2% used, 67.1 GiB free",
"value": 71.2
},
{
"name": "memory",
"ok": true,
"detail": "41.8% used, 4.3 GiB available",
"value": 41.8
}
]
}
No Flask yet, and no browser. The hard decision in a dashboard is what to report, and
once that answer is a plain dict every later layer only moves it around.
run_checks already returns a list of frozen
HealthStatus records, so the only work here is turning each one into
JSON-native values: strings, numbers, booleans, None, lists and dicts.
Nothing else survives the trip.
as_dict names its four fields by hand where
dataclasses.asdict would have done it in one line, and that is deliberate.
This dict is a published interface: a page you are not going to redeploy in step with
the server reads these key names. Generating them from the attributes means the day
you rename a field on the dataclass, the wire quietly changes and the page quietly
stops finding detail. Typing them out makes that rename a visible edit in
one file.
overall exists so the page never has to know the rule for combining
verdicts. One flag, computed once by the server, and every reader agrees about whether
the house is fine.
def sse_event(payload: dict) -> str:
"""Frame one dict as a single Server-Sent Events message."""
return f"data: {json.dumps(payload)}\n\n"
if __name__ == "__main__":
print(repr(sse_event({"at": "09:53:02", "overall": "ok"})))
$ uv run python -m labs.dashboard
'data: {"at": "09:53:02", "overall": "ok"}\n\n'
That is the entire transport layer of this chapter, and repr is there so
the two newlines are visible instead of being rendered as empty space. The format is
line based. Every line is a field: value pair, and a blank line ends the
current message and tells the browser to hand it to your code. One
\n instead of two leaves the browser holding a half-finished message
forever, waiting for a terminator that never arrives, with no error anywhere.
There are only four fields, plus one line that is not a field at all:
data:is the payload. It fires the browser'sonmessagehandler, and everything after the space arrives asevent.data, a string.event:names a channel, soaddEventListener("alert", ...)receives it instead ofonmessage.id:tags the message. The browser remembers the last one it saw and sends it back as aLast-Event-IDheader when it reconnects.retry:sets how many milliseconds the browser waits before reconnecting after a drop.- A line starting with
:is a comment. The parser reads it and throws it away, which makes it the cheapest possible way to write something into a connection to prove it still exists.
That list is the whole protocol. There is nothing to install on the server, because the server is writing text, and nothing to install in the browser, because the parser for those five line types has been built into browsers for over a decade.
A response that never ends
# labs/glados_api.py — added beside the routes from chapter 89
from flask import Response
from labs.dashboard import sse_event, snapshot
from labs.health_monitor import run_checks
@app.route("/events", methods=["GET"])
def events():
def stream():
while True:
yield sse_event(snapshot(run_checks(), time.time()))
time.sleep(5)
return Response(stream(), mimetype="text/event-stream",
headers={"Cache-Control": "no-cache",
"X-Accel-Buffering": "no"})
$ curl -iN http://127.0.0.1:5000/events # ctrl-C to stop it; measured on the bench — yours will vary
HTTP/1.1 200 OK
Server: Werkzeug/3.1.8 Python/3.11.15
Date: Sun, 23 Aug 2026 14:53:02 GMT
Content-Type: text/event-stream; charset=utf-8
Cache-Control: no-cache
X-Accel-Buffering: no
Connection: close
data: {"at": "09:53:02", "overall": "ok", "checks": [{"name": "ollama", "ok": true, "detail": "6 model(s) available", "value": 6.0}, {"name": "disk", "ok": true, "detail": "71.2% used, 67.1 GiB free", "value": 71.2}, {"name": "memory", "ok": true, "detail": "41.8% used, 4.3 GiB available", "value": 41.8}]}
data: {"at": "09:53:07", "overall": "ok", "checks": [{"name": "ollama", "ok": true, "detail": "6 model(s) available", "value": 6.0}, {"name": "disk", "ok": true, "detail": "71.2% used, 67.1 GiB free", "value": 71.2}, {"name": "memory", "ok": true, "detail": "41.8% used, 4.3 GiB available", "value": 41.8}]}
The -N turns off curl's output buffering, so the lines land as they are
written and the terminal sits there afterwards, waiting for more. That is the whole
trick made visible: an ordinary GET, a 200, a normal header block, and then a body
that is still being written. Every byte is text you can read without a decoder.
Two headers in that block deserve a look. Content-Type: text/event-stream
is the switch that puts a browser into SSE mode; get it wrong and the browser treats
the reply as a document and gives up. And there is no Content-Length at
all, which is what makes the rest possible. A length is a promise about how many bytes
follow, and this response cannot make one. Chapter 89's captures ended with the
receiver counting bytes and stopping; this one ends when someone closes the socket.
Cache-Control: no-cache and X-Accel-Buffering: no exist for
the machines in between. A cache that stores a response it thinks is finished, or a
reverse proxy that holds bytes back until it has a few kilobytes to send, breaks
this in a way that looks exactly like a dead server. The second header is nginx's own
switch, read per response, and it costs nothing to send when no proxy is there.
What this stage still gets wrong is the same thing polling got wrong. It runs the checks on a timer, once per connection, so two tabs mean two independent probe loops hammering Ollama out of step with each other.
# labs/dashboard.py
import threading
from labs.health_monitor import POLL_SECONDS, Monitor
KEEPALIVE_SECONDS = 15.0
class Board:
"""The latest snapshot, and a version number readers can wait on."""
def __init__(self) -> None:
self._cond = threading.Condition()
self._latest: dict | None = None
self._version = 0
def publish(self, payload: dict) -> None:
with self._cond:
self._latest = payload
self._version += 1
self._cond.notify_all()
def wait(self, seen: int, timeout: float) -> tuple[int, dict | None]:
"""Block until the version moves past `seen`, or the timeout expires."""
with self._cond:
self._cond.wait_for(lambda: self._version != seen, timeout)
return self._version, self._latest
BOARD = Board()
def publish_forever(interval: float = POLL_SECONDS) -> None:
"""Chapter 92's monitor, with every poll put on the board."""
monitor = Monitor()
while True:
at = time.time()
checks, _ = monitor.poll(at)
BOARD.publish(snapshot(checks, at))
time.sleep(max(0.0, interval - (time.time() - at)))
# labs/glados_api.py
from labs.dashboard import BOARD, KEEPALIVE_SECONDS, PAGE, publish_forever, sse_event
@app.route("/events", methods=["GET"])
def events():
def stream():
seen = 0
while True:
version, payload = BOARD.wait(seen, KEEPALIVE_SECONDS)
if version == seen or payload is None:
yield ": waiting\n\n"
continue
seen = version
yield sse_event(payload)
return Response(stream(), mimetype="text/event-stream",
headers={"Cache-Control": "no-cache",
"X-Accel-Buffering": "no"})
if __name__ == "__main__":
threading.Thread(target=publish_forever, args=(5.0,), daemon=True).start()
app.run(host="0.0.0.0", port=5000)
$ curl -N localhost:5000/events # measured on the bench; in a third terminal: sudo systemctl stop ollama
data: {"at": "09:56:12", "overall": "ok", "checks": [{"name": "ollama", "ok": true, "detail": "6 model(s) available", "value": 6.0}, {"name": "disk", "ok": true, "detail": "71.2% used, 67.1 GiB free", "value": 71.2}, {"name": "memory", "ok": true, "detail": "41.8% used, 4.3 GiB available", "value": 41.8}]}
data: {"at": "09:56:17", "overall": "degraded", "checks": [{"name": "ollama", "ok": false, "detail": "URLError: <urlopen error [Errno 111] Connection refused>", "value": null}, {"name": "disk", "ok": true, "detail": "71.2% used, 67.1 GiB free", "value": 71.2}, {"name": "memory", "ok": true, "detail": "41.8% used, 4.3 GiB available", "value": 41.8}]}
$ curl -N localhost:5000/events # the same route with the interval left at POLL_SECONDS
data: {"at": "10:02:00", "overall": "degraded", "checks": [{"name": "ollama", "ok": false, "detail": "URLError: <urlopen error [Errno 111] Connection refused>", "value": null}, {"name": "disk", "ok": true, "detail": "71.3% used, 67.0 GiB free", "value": 71.3}, {"name": "memory", "ok": true, "detail": "42.0% used, 4.3 GiB available", "value": 42.0}]}
: waiting
: waiting
: waiting
data: {"at": "10:03:00", "overall": "degraded", "checks": [{"name": "ollama", "ok": false, "detail": "URLError: <urlopen error [Errno 111] Connection refused>", "value": null}, {"name": "disk", "ok": true, "detail": "71.3% used, 67.0 GiB free", "value": 71.3}, {"name": "memory", "ok": true, "detail": "42.1% used, 4.3 GiB available", "value": 42.1}]}
The checks now run in exactly one place, on the schedule chapter 92 chose, and the
alerts it sends still go out from the same poll. Ten tabs cost ten sleeping threads
and no extra probes at all. A reader arriving mid-flight passes seen = 0,
the version is already past that, and wait_for returns without blocking,
so a new tab paints itself immediately instead of staring at an empty page until the
next poll.
The board keeps the latest snapshot and not a queue, and that is the right choice for status. A reader that was slow, or asleep in a background tab, wants the truth now, not a backlog of the six intermediate versions it missed. Anything where every event matters, like her alert history, needs ids and a buffer instead, which is exercise 1.
The three comment lines are the reason this design survives a real house. With the
interval at sixty seconds a connection can sit silent long enough for a phone's radio,
a NAT table or an impatient proxy to decide it is dead. More importantly, a generator
blocked in wait_for has no idea its reader closed the tab; the server
only finds out when it next tries to write and the socket refuses. Writing eleven
bytes every fifteen seconds is what turns a closed tab into a freed thread.
One judgement call is baked in here: every poll counts as news, so the numbers on the
cards stay fresh even when nothing interesting happened. If you would rather stream
only true changes, compare the name and ok pairs against the
previous payload inside publish and return early when they match. The
cost is a page whose timestamp can be hours old, which reads as a dead dashboard.
A page that repaints itself
<!-- labs/dashboard.html -->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>GLaDOS status</title>
<style>
body { background: #101014; color: #d8d8dc; font: 15px/1.6 monospace; margin: 2rem; }
h1 { font-size: 1rem; letter-spacing: 0.15em; text-transform: uppercase; }
#line { color: #8a8a92; margin-bottom: 1rem; }
.card { border: 1px solid #2a2a32; border-left-width: 5px; border-radius: 4px;
padding: 0.6rem 0.9rem; margin: 0.4rem 0; }
.ok { border-left-color: #3ddc84; }
.bad { border-left-color: #ff5f56; }
.name { display: inline-block; width: 7rem; }
.detail{ color: #a0a0a8; }
</style>
</head>
<body>
<h1>GLaDOS subsystems</h1>
<div id="line">connecting</div>
<div id="board"></div>
<form id="say"><input id="text" size="40" placeholder="make her say something"></form>
<script>
const board = document.getElementById("board");
const line = document.getElementById("line");
const text = document.getElementById("text");
function card(check) {
const row = document.createElement("div");
row.className = "card " + (check.ok ? "ok" : "bad");
const name = document.createElement("span");
name.className = "name";
name.textContent = check.name;
const detail = document.createElement("span");
detail.className = "detail";
detail.textContent = check.detail;
row.append(name, detail);
return row;
}
const stream = new EventSource("/events");
stream.onopen = () => { line.textContent = "live"; };
stream.onerror = () => { line.textContent = "connection dropped, retrying"; };
stream.onmessage = (message) => {
const snap = JSON.parse(message.data);
line.textContent = snap.overall + " at " + snap.at;
board.replaceChildren(...snap.checks.map(card));
};
document.getElementById("say").addEventListener("submit", (submit) => {
submit.preventDefault(); // the other channel: an ordinary POST
fetch("/speak", {method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({text: text.value})});
text.value = "";
});
</script>
</body>
</html>
# labs/glados_api.py
@app.route("/", methods=["GET"])
def dashboard():
return PAGE.read_text(encoding="utf-8"), 200, {"Content-Type": "text/html; charset=utf-8"}
# the server terminal, measured on the bench: the page open in a browser, one sentence typed into it
127.0.0.1 - - [23/Aug/2026 10:11:38] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [23/Aug/2026 10:11:38] "GET /events HTTP/1.1" 200 -
127.0.0.1 - - [23/Aug/2026 10:12:04] "POST /speak HTTP/1.1" 202 -
Three requests, and then the log goes quiet while the cards keep repainting. That is the entire argument of the chapter in one capture. The polling version puts a line in that terminal every second the tab is open, 86,400 of them by this time tomorrow, and it would still have been late to the one thing that mattered.
Reading the page off disk on every request costs a few microseconds and buys you an edit-and-refresh loop: change a colour, press F5, see it, with the server and her models left running. Nothing on the page is fetched from anywhere else. No framework, no CDN, no build step, no font server that turns her dashboard into something that stops working when your internet does.
textContent is doing quiet safety work in card. The
detail field carries whatever a failing subsystem said about itself,
including exception text from a library nobody here wrote. Building the row with
innerHTML and a template string would hand that text to the HTML parser,
and an error message containing angle brackets would rearrange the page instead of
being displayed on it. Elements built by hand and filled with
textContent cannot do that.
The single input at the bottom is the other direction, and it earns its place on the
page precisely because it is so plainly separate. Typing a sentence and pressing enter
sends a POST to chapter 89's /speak, which answers 202 on its own
short-lived connection while the stream carries on untouched. The stream reports; the
POST asks. If you ever find yourself wanting to send something up the stream, that is
the signal you have outgrown SSE and want a WebSocket.
app.run already sets threaded=True, so this works out of the
box, but the arithmetic is worth knowing: each held-open stream occupies one thread of
the development server for its entire life. Half a dozen tabs is nothing; a hundred
would need an async server or a worker model built for long-lived connections. That is
the standing cost of push, and it is the same cost a WebSocket would have.
Reconnection is handled entirely by the browser. Stop the server with the page open
and onerror fires, the readout says so, and a few seconds later the
access log of the restarted server shows the page finding its way back with no code of
yours involved:
# the server, restarted at 10:19:28 with the tab still open
127.0.0.1 - - [23/Aug/2026 10:19:31] "GET /events HTTP/1.1" 200 -
There is one way to lose that for good, and it is the first thing to check when a page
never recovers. If the server answers the reconnect with anything other than a 200
carrying text/event-stream, the browser stops trying permanently. A typo
in the constructor produces a single 404 in the log, never repeated, and a page that
sits at "connecting" until you reload it yourself.
Why this works: a body that is still being written
Nothing new was invented anywhere in this chapter. The route returns a generator, and a WSGI server has always been allowed to write an iterable out one chunk at a time; Flask just does not build the whole body first. Because the body has no length, the server keeps the socket open, and because it stays open, anything written into it later arrives at the client later. Server-Sent Events is the four-line text convention that makes a stream like that parseable: fields, then a blank line, then the next message.
That is why both ends needed no new software. curl read the stream with a
flag that only turned off its own buffering. Werkzeug wrote it with no knowledge of SSE
whatsoever. The browser parsed it with an object browsers have shipped since 2011. The same endpoint can be consumed by ten lines of Python, a Go program, or a
shell pipeline, because from the outside it is a slow file arriving over HTTP.
The generalization is worth carrying past this chapter. Any time you catch yourself writing a timer that asks a question, check whether the thing being asked could tell you instead. The monitor already knew. The only thing missing was somewhere to put the news, and a socket held open is the cheapest such place available over plain HTTP.
Stage 1 printed its snapshot with indent=2 because a wall of one-line
JSON is hard on the eyes, and the same instinct arrives an hour later while staring at
the curl -N output:
def sse_event(payload: dict) -> str:
return f"data: {json.dumps(payload, indent=2)}\n\n" # BUG: newlines inside the payload
$ curl -N localhost:5000/events # trimmed to one event
data: {
"at": "10:24:41",
"overall": "ok",
"checks": [
{
"name": "ollama",
"ok": true,
"detail": "6 model(s) available",
"value": 6.0
}
]
}
Readable, correct JSON, and a dashboard that has gone blank. The browser console says the page broke inside the handler, on the first line that touches the data:
# Chromium's console; Firefox words this differently
Uncaught SyntaxError: Unexpected end of JSON input
at JSON.parse (<anonymous>)
at EventSource.stream.onmessage (dashboard.html:47:26)
End of input, on a payload the terminal just showed to be complete. When a parser disagrees with you about what it was handed, print what it was handed. One line inside the handler settles it:
console.log(JSON.stringify(message.data)); // what actually arrived
# Chromium's console, on the next event
"{"
One character. The event fired, so the connection is fine and the framing is fine; the
browser simply took the first line as the data and dropped everything after it. Read
that back against the format from stage 2 and the cause is exact. Lines are the unit.
data: { ends at the first newline, so the payload is the single brace,
and the lines that follow are read as fields named "at" and
"overall", none of which the parser knows, so it discards
them in silence. The closing brace on its own line has no colon at all and is ignored
too. Then the blank line arrives and dispatches everything it collected, which is
{.
The fix is to delete indent=2 and keep the payload on one line, and the
rule underneath it is the one to remember: a value crossing this wire may contain no
newline. If you ever must send genuinely multi-line text, the format has an answer for
that too, which is to write several data: lines in a row; the browser
joins them with a newline and hands you the whole thing. What it will never do is
guess that your pretty-printing was meant to be one value.
Checkpoint, and the thing a dashboard cannot show
- I can read a raw
curl -Nstream and say which lines dispatch an event, which are discarded, and which byte does the dispatching. - I can explain why the
/eventsresponse carries noContent-Length, and what the header would have promised if it did. - I can trace one repainted card backwards to the monitor poll that caused it, and say how many probes ten open tabs add.
- I can say what the keep-alive comment does for a proxy and what it does for a tab that was closed while the generator was blocked.
- I can name the two ways a browser reacts to a broken stream, and which server answer makes it stop reconnecting for good.
- I can point at the part of the page that sends a command and say why it is a POST and not something written back up the stream.
Exercise 1 — hand back the events a reader missed. Give
every message an id:, keep the last twenty snapshots in a deque, and
use the Last-Event-ID header on reconnect to replay what the browser
did not see.
The header has to be read before the generator starts, since the generator outlives the request. Frame the id alongside the data, and replay from the deque on the way in:
def sse_event(payload: dict, event_id: int | None = None) -> str:
head = "" if event_id is None else f"id: {event_id}\n"
return f"{head}data: {json.dumps(payload)}\n\n"
@app.route("/events", methods=["GET"])
def events():
since = int(request.headers.get("Last-Event-ID", 0)) # read it out here
def stream():
for version, payload in BOARD.since(since):
yield sse_event(payload, version)
...
Give Board a deque(maxlen=20) of
(version, payload) pairs and a since method that filters
it. To watch it work, stop the server for half a minute, restart it, and look at
the reconnect in the access log: the browser sends its last id back and the page
fills in the gap on its own. Twenty entries is a deliberate cap, in the spirit of
chapter 49's ring buffer. A reader gone longer than that gets the current state and
a hole, which beats a queue that grows forever.
Exercise 2 — a second channel for the rare thing. Send
chapter 92's alerts as event: alert messages, and have the page show
them in a list that keeps its history while the status cards keep replacing
themselves.
Monitor.observe already returns a level and a message on the polls
where something changed. Publish those on a second board and frame them with a
named event:
def sse_alert(level: str, message: str) -> str:
return f"event: alert\ndata: {json.dumps({'level': level, 'text': message})}\n\n"
stream.addEventListener("alert", (message) => {
const alert = JSON.parse(message.data);
const line = document.createElement("div");
line.textContent = alert.level + " " + alert.text;
document.getElementById("alerts").prepend(line);
});
Stop Ollama and watch two different things happen on one connection: the cards flip
to degraded on the next poll, and a CRITICAL line appears in the alert list two
polls later, when the debouncer is finally convinced. One socket, two channels, and
onmessage never sees the alerts at all.
Exercise 3 — count what polling costs. Write the four-line polling page from the opening of this chapter, leave it open for one minute, and count the lines it puts in the access log. Then do the same with the SSE page.
Start the server with its output going to a file, so the counting is a shell command instead of a scroll:
uv run python -m labs.glados_api 2> run.log
grep -c '"GET /health' run.log # after a minute of the polling page
grep -c '"GET /events' run.log # after a minute of the dashboard
Around sixty against exactly one. Then open the polling page in a second tab and
run the count again to watch the number double, while the SSE page's count stays
where it is however many tabs you open. Put a print inside
check_ollama before you start, and you get to watch the model server
being asked whether it is alive twice a second by two screens nobody is reading.
A screen in the hallway now tells you what she knows about herself, the moment she knows it. It is all machine state: disks, memory, a model server answering or not. None of it says anything about the person standing in front of her. Chapter 97 reads the other channel in an utterance, the loudness and pitch that decide whether "I'm fine" means what the words say.