Documentation as Data
The reference that goes stale by itself
She answers to eight things now, and they arrive from two different places. Three are rows in the automation table from chapter 17, matched by substring and dispatched through the action registry. Five more are behaviors from chapter 46, matched against a context dictionary and sorted by priority. Both dispatchers work. Neither of them can tell a person standing in your kitchen what to say.
So you write a reference. A Markdown file, one row per command, the phrase on the left and the effect on the right, with an example anyone can imitate. It takes twenty minutes and it is accurate for about a week. Then you add a ninth command and forget the file. You rename an action during a refactor and the file keeps the old name. You switch the night snark behavior off and the manual goes on advertising it. None of those edits break anything you would notice; the first person to find out is a guest who reads the manual, says the sentence it promised, and gets silence.
The trouble is that the file is a second copy of facts the program already holds. So this chapter takes the copy away: the manual is generated from the system's own data, and the only thing you write by hand is what no part of the system knows. Read what she is made of and most of a manual is already there. The trigger strings live in the rule rows and the behavior table. The action names are the registry keys. Whether a behavior is on is a field. Whether the servo an emergency halt would stop is actually attached to the board is a line in the wiring manifest you built last chapter. What is missing is small: which category a command belongs in, and one sentence a human would really say. That is the table you type, and it is eight lines long.
The obvious Python answer to "keep the docs next to the code" is a docstring on
each handler, harvested with inspect.getdoc. It fails here for a
structural reason. A docstring can describe the function it sits on and nothing
else, and the facts a user needs are not on the function: the trigger phrase lives
in a rule row, the dispatch key is a string in a registry, the on/off flag lives in
the behavior table, and the hardware fact lives in a different file entirely. The
handler does not know its own name as the dispatcher spells it. Documentation has
to be generated at the level where those pieces meet.
What she already knows about herself
# labs/user_manual.py
from dataclasses import dataclass
@dataclass
class Command:
action: str
trigger: str
source: str
category: str
example: str
enabled: bool = True
requires: str = ""
if __name__ == "__main__":
cmd = Command("turn_off_lights", "lights off", "rules", "home",
"GLaDOS, lights off please.")
print(f'{cmd.action:<16} matched on "{cmd.trigger}" by the {cmd.source} engine')
print(f'{"":<16} category={cmd.category} enabled={cmd.enabled}')
$ uv run python -m labs.user_manual
turn_off_lights matched on "lights off" by the rules engine
category=home enabled=True
Seven fields, and the split between them is the design. action,
trigger, source and enabled are facts the
running system owns; nobody types those into a manual. category and
example are editorial, invented by a person who knows what a household
would actually say. requires is the odd one: a person decides that
halting everything is meaningless without a servo, and the wiring manifest decides
whether that servo exists. Command is not a record you write, it is
the record the generator produces after joining those sources.
from labs.automation import ACTION_REGISTRY, RULES
from labs.behavior_engine import BEHAVIORS
def discover() -> list[tuple[str, str, str, bool]]:
found = []
for rule in RULES:
if rule["action"] in ACTION_REGISTRY:
found.append((rule["action"], rule["trigger"], "rules", True))
for b in BEHAVIORS:
found.append((b.action, b.trigger, "behaviors", b.enabled))
return found
if __name__ == "__main__":
for action, trigger, source, enabled in discover():
flag = " " if enabled else "-"
print(f"{flag} {action:<16} {trigger:<12} {source}")
$ uv run python -m labs.user_manual
turn_off_lights lights off rules
set_timer set timer rules
play_music play music rules
halt.all emergency behaviors
warn.power battery_low behaviors
greet.user wake_word behaviors
status.report status behaviors
- snark.reply wake_word behaviors
Eight lines, produced by importing two modules and reading their data. The
membership test on the rules side matters: a rule whose action names a handler
nobody registered is a row that warns at dispatch time and does nothing, so it is
not a command and does not belong in a manual. On the behavior side there is no
such test, because a behavior carries its own enabled flag, and the
single dash on the last line is that flag surviving the trip into documentation.
Night snark is off today. Any manual that quietly promised it would be lying, and
this one cannot, because the flag came from the table the engine consults.
One table by hand, everything else looked up
@dataclass
class Doc:
category: str
example: str
requires: str = ""
DOCS: dict[str, Doc] = {
"turn_off_lights": Doc("home", "GLaDOS, lights off please."),
"set_timer": Doc("home", "GLaDOS, set a timer for five minutes."),
"play_music": Doc("home", "GLaDOS, play music."),
"greet.user": Doc("social", "GLaDOS, wake up."),
"snark.reply": Doc("social", "GLaDOS, wake up."),
"status.report": Doc("system", "Give me a status report."),
"halt.all": Doc("system", "GLaDOS, emergency stop.", requires="servo_sg90"),
"warn.power": Doc("system", ""),
}
def build_commands() -> list[Command]:
found: list[Command] = []
for rule in RULES:
doc = DOCS.get(rule["action"])
if doc is None or rule["action"] not in ACTION_REGISTRY:
continue
found.append(Command(rule["action"], rule["trigger"], "rules",
doc.category, doc.example, requires=doc.requires))
for b in BEHAVIORS:
doc = DOCS.get(b.action)
if doc is None:
continue
found.append(Command(b.action, b.trigger, "behaviors", doc.category,
doc.example, enabled=b.enabled, requires=doc.requires))
return found
def print_manual(commands: list[Command], wired: set[str]) -> None:
by_category: dict[str, list[Command]] = {}
for cmd in commands:
by_category.setdefault(cmd.category, []).append(cmd)
print(f"=== GLaDOS voice commands ({len(commands)} known) ===\n")
for category, cmds in sorted(by_category.items()):
print(f"[{category.upper()}]")
for cmd in cmds:
say = f'"{cmd.example}"' if cmd.example else "(nothing: she starts this one)"
print(f" Say: {say}")
print(f" Matches: {cmd.trigger} -> {cmd.action} ({cmd.source})")
if cmd.requires:
state = "wired" if cmd.requires in wired else "NOT WIRED"
print(f" Hardware: {cmd.requires} ({state})")
if not cmd.enabled:
print(" Status: switched off in the behavior table")
print()
$ uv run python -m labs.user_manual
=== GLaDOS voice commands (8 known) ===
[HOME]
Say: "GLaDOS, lights off please."
Matches: lights off -> turn_off_lights (rules)
Say: "GLaDOS, set a timer for five minutes."
Matches: set timer -> set_timer (rules)
Say: "GLaDOS, play music."
Matches: play music -> play_music (rules)
[SOCIAL]
Say: "GLaDOS, wake up."
Matches: wake_word -> greet.user (behaviors)
Say: "GLaDOS, wake up."
Matches: wake_word -> snark.reply (behaviors)
Status: switched off in the behavior table
[SYSTEM]
Say: "GLaDOS, emergency stop."
Matches: emergency -> halt.all (behaviors)
Hardware: servo_sg90 (wired)
Say: (nothing: she starts this one)
Matches: battery_low -> warn.power (behaviors)
Say: "Give me a status report."
Matches: status -> status.report (behaviors)
DOCS is keyed by action name because the action name is the one string
both dispatchers agree on. Everything else about a command is reachable from it.
setdefault builds each category bucket the first time that category is
seen, and sorted on the buckets fixes the printing order at HOME,
SOCIAL, SYSTEM, so two runs a month apart differ only where the system differs. Two
entries are doing quiet work. The empty example on warn.power renders
as "she starts this one", which is the truth: a battery warning is not something
you can ask for. And the hardware line reads servo_sg90 (wired)
only because the manifest's parts list declares a component under exactly that
name. Pull the servo off the board, take it out of the rig, and the manual stops
claiming she can halt a limb she no longer has.
# labs/user_manual.py — the parts that turn the list into artifacts
import json
import sys
from dataclasses import asdict
from pathlib import Path
from labs.wiring_manifest import bench_manifest
MANUAL_PATH = Path("docs/voice_commands.json")
def wired_components() -> set[str]:
return {c.name for c in bench_manifest().components}
def to_help_text(commands: list[Command]) -> str:
usable = [c for c in commands if c.enabled and c.example]
names = ", ".join(f'"{c.trigger}"' for c in usable[:3])
return f"I answer to {len(usable)} phrases, including {names}. The rest are in the manual."
def save_manual(commands: list[Command], path: Path) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
payload = {"count": len(commands), "commands": [asdict(c) for c in commands]}
path.write_text(json.dumps(payload, indent=2))
def audit() -> list[str]:
system = set(ACTION_REGISTRY) | {b.action for b in BEHAVIORS}
problems = []
for action in sorted(system - set(DOCS)):
problems.append(f"UNDOCUMENTED {action}: she can run it, the manual never mentions it")
for action in sorted(set(DOCS) - system):
problems.append(f"UNKNOWN {action}: documented, but nothing registers that name")
return problems
def main() -> None:
commands = build_commands()
print_manual(commands, wired_components())
save_manual(commands, MANUAL_PATH)
print(f"Wrote {MANUAL_PATH} ({len(commands)} commands)")
print(to_help_text(commands))
problems = audit()
for line in problems:
print(line)
print(f"audit: {len(problems)} problem(s)")
sys.exit(1 if problems else 0)
if __name__ == "__main__":
main()
$ uv run python -m labs.user_manual # last lines only
Wrote docs/voice_commands.json (8 commands)
I answer to 6 phrases, including "lights off", "set timer", "play music". The rest are in the manual.
audit: 0 problem(s)
$ head -12 docs/voice_commands.json
{
"count": 8,
"commands": [
{
"action": "turn_off_lights",
"trigger": "lights off",
"source": "rules",
"category": "home",
"example": "GLaDOS, lights off please.",
"enabled": true,
"requires": ""
},
Three consumers, one list. The printed reference is for a person at a terminal, the
JSON is for anything that is not a person, and to_help_text is the
sentence she speaks when someone asks what she can do: six phrases, not eight,
because the disabled behavior and the automatic one are filtered by fields rather
than by memory. asdict converts each Command into a plain
dict, since json.dumps handles built-in types only and raises
TypeError: Object of type Command is not JSON serializable otherwise.
The last line is the one that changes how the project behaves:
sys.exit(1) on any problem makes stale documentation a failing command
instead of an observation. Run it from the systemd pre-start check you generated in
chapter 54, or from CI, and a command that nobody documented stops the build.
Why this works: a join, and a set difference
Two ideas are carrying the whole design, and both are older than this project. The
first is the join. build_commands is a lookup of one table against
another on a shared key, exactly what a database does with a foreign key: the system
tables supply behavior, DOCS supplies prose, and the action name is the
key they share. Neither side can be edited into disagreement without the key going
missing on one side of the join, and a missing key is detectable. That is what buys
the freshness guarantee. It is not discipline, it is a structure where drift shows up
as an absent row.
The second idea is the audit's arithmetic. system - set(DOCS) and
set(DOCS) - system are set differences in both directions, and running
both is what makes the check complete: the first finds capabilities with no
documentation, the second finds documentation with no capability. One direction alone
leaves half the drift invisible. This pairing shows up everywhere once you have seen
it here: an API spec checked against its routes, a translation file checked against
its message keys, a test suite checked against its modules. Two sets, two
differences, one exit code.
You write DOCS from memory of what she can do, and you spell the light
and timer entries the way the behavior actions are spelled, with dots:
DOCS: dict[str, Doc] = {
"lights.off": Doc("home", "GLaDOS, lights off please."), # BUG: registry key is turn_off_lights
"timer.set": Doc("home", "GLaDOS, set a timer for five minutes."), # BUG: registry key is set_timer
"play_music": Doc("home", "GLaDOS, play music."),
...
}
$ uv run python -m labs.user_manual # last lines only
Wrote docs/voice_commands.json (6 commands)
I answer to 4 phrases, including "play music", "emergency", "wake_word". The rest are in the manual.
UNDOCUMENTED set_timer: she can run it, the manual never mentions it
UNDOCUMENTED turn_off_lights: she can run it, the manual never mentions it
UNKNOWN lights.off: documented, but nothing registers that name
UNKNOWN timer.set: documented, but nothing registers that name
audit: 4 problem(s)
Look at what happened without the audit: the count fell from 8 to 6 and the manual
printed cleanly. DOCS.get returned None for two real
commands, the loop skipped them, and the document you would have handed to a guest
simply omits the lights and the timer with no sign anything went wrong. The cause
is a naming split the project grew into honestly: chapter 17's handlers are named
like Python functions, chapter 46's behaviors are named like events, and a person
writing documentation from memory picks whichever convention comes to mind. The fix
is never to rename in the annotation table until the code agrees. Copy the exact
keys the dispatchers use, or better, converge on one convention across both
registries and let the audit prove you finished the rename: every key you miss
comes back as a matched pair of UNKNOWN and UNDOCUMENTED lines naming the old and
new spelling.
Checkpoint, and a manual that fails the build
- I can sort any field of a command record into "the system owns this" or "a person owns this", and say why only the second kind gets typed.
- I can explain why the rules side tests membership in
ACTION_REGISTRYwhile the behavior side reads anenabledfield instead. - I can name the join key between the annotation table and the two dispatchers, and describe what breaks on each side when it is misspelled.
- I know why the audit computes two set differences instead of one, and what staleness each direction catches.
- I can point at the line that turns out-of-date documentation into a non-zero exit status, and name a place in this project that would run it.
Exercise 1 — add a command and skip the docs on purpose.
Append a rule with the trigger "fan on", register a
turn_on_fan handler for it, then run the generator without touching
DOCS. Predict the exit status before you check it.
The printed manual still shows 8 commands, the audit prints
UNDOCUMENTED turn_on_fan, and echo $? answers
1. Adding the Doc("home", "GLaDOS, turn on the fan.")
line takes the count to 9 and the exit status back to 0. That round trip is the
whole argument in ninety seconds: the documentation debt existed for the length
of time it took you to notice a red run, instead of for a month.
Exercise 2 — unwire the servo. Take
servo_sg90 out of bench_manifest(), its parts-list entry
and its three wires together, and re-run. Then make an unwired requirement an
audit problem too, and decide whether it should fail the build or only
warn.
The emergency halt entry flips to Hardware: servo_sg90 (NOT WIRED)
without any edit to the command data, because availability was always a lookup.
For the audit, add a third loop over the built commands collecting
c.requires and c.requires not in wired. A warning fits better than
a failure here: an unwired servo is a true statement about a machine mid-build,
while an undocumented action is a mistake. Return the two kinds separately and
let main decide which one sets the exit status.
Exercise 3 — the README table. Write
to_markdown(commands) that renders the same list as a Markdown table
sorted by category, and pipe it into the project README between two marker
comments.
A header row, a separator row, then one line per command:
f"| {c.category} | {c.example} | `{c.action}` | {c.source} |" over
sorted(commands, key=lambda c: c.category). Splicing it into the
README between <!-- commands:start --> and
<!-- commands:end --> markers is a few lines of string
slicing, and it makes the fourth renderer of the same list. The test that it
worked is a rendered table on your project page listing exactly the commands
that ran this morning.
The manual now describes her honestly because it asks her. Next comes a question no amount of description answers: the components you have been registering depend on each other, and one of those dependency chains can quietly close into a loop that deadlocks at boot. A diagram will not catch a three-hop cycle. A depth-first search will, and the next chapter writes one.