Action parsing
The registry has been waiting for a better key
Chapter 17 left her with an engine that never needs editing again. Rules are rows of
data, handlers live in ACTION_REGISTRY under their names, and the
twenty-first automation costs one dict. Chapter 20 put a router in front of the same
idea, classifying a request by keyword and handing it to the specialist who owns it.
Both are sound, and both decide what you meant by looking for substrings.
Substrings are literal-minded. "Lights off" trips the rule. "It's too bright in here" trips nothing, because it shares no character sequence with any trigger you wrote, and neither does "could you do something about the glare." You can keep appending triggers, and people do, until the rule file is an attempt to enumerate English one fragment at a time. It never finishes.
The thing that reads meaning has been running on your machine since chapter 8. Let it pick the action. The catch is what it hands back: text written for a human, capitalized and punctuated and sometimes apologetic, and occasionally the name of an action you never registered, invented because it sounded like something a house would have. The registry accepts exact keys and nothing else. So the rule for this chapter: the model chooses only from a list your registry generated, and no name becomes an action until the complete reply names one registered action. Registration only establishes that a handler exists. It does not establish the user's intent or permission to run it; the permission gate is a separate check.
Be clear about the trade before you take it. The model buys paraphrase: any sentence meaning "too bright" reaches the lights, and the keyword list stops growing. It charges a model call, a second or two on a 3b model on a laptop CPU; it charges determinism, since the same sentence can come back worded differently on two runs; and it opens a failure keyword matching never had, a confident well-formatted action that does not exist. The first two you pay. The third you defend against, with a gate that fits in five lines.
A list, a reply, a key
# labs/command_loop.py
from labs.automation import ACTION_REGISTRY, RULES, match_rules, register_action
COMMAND_SYSTEM_PROMPT = """You control a house. Reply with one line: the name of the
action that fits the request, chosen from this list, or NONE if none of them fit.
{catalog}
No punctuation, no explanation, no other words."""
def action_catalog() -> str:
return "\n".join(f"- {name}" for name in sorted(ACTION_REGISTRY))
if __name__ == "__main__":
print(COMMAND_SYSTEM_PROMPT.format(catalog=action_catalog()))
$ uv run python -m labs.command_loop
You control a house. Reply with one line: the name of the
action that fits the request, chosen from this list, or NONE if none of them fit.
- play_music
- set_timer
- turn_off_lights
No punctuation, no explanation, no other words.
The catalog is generated, never typed. Write those three names into the prompt by
hand and you have made a second copy of the registry that nothing compares against the
first: register a fourth handler six months from now and the model will never choose
it, silently, because it was never told the action exists. Reading the list out of
ACTION_REGISTRY makes that bug unrepresentable. The sort keeps the prompt
text stable between runs, so two prompts can be diffed to explain a behavior change.
import ollama
MODEL = "llama3.2:3b"
def propose_action(text: str) -> str:
response = ollama.chat(
model=MODEL,
messages=[
{"role": "system", "content": COMMAND_SYSTEM_PROMPT.format(catalog=action_catalog())},
{"role": "user", "content": text},
],
options={"temperature": 0},
)
return response["message"]["content"]
if __name__ == "__main__":
for line in ["It's too bright in here.", "Put something on.", "How was your day?"]:
print(f"{line!r} -> {propose_action(line)!r}")
$ uv run python -m labs.command_loop
"It's too bright in here." -> 'turn_off_lights'
'Put something on.' -> 'play_music.'
'How was your day?' -> "NONE\n\nI don't have days."
Your three replies will read differently; the pattern behind them will not. Print with
!r, because the interesting damage is invisible otherwise: the trailing
period on the second, the newlines and the editorial comment on the third. That is a
model doing its job. It was trained to write for people, and no amount of "no other
words" in a system prompt makes a 3b model a parser. Line one is the argument for this
chapter in one string: not a single trigger from chapter 17 appears in "It's too
bright in here," and she found the lights anyway.
def extract_action(reply: str) -> str | None:
"""Accept one complete registered name, ignoring only case and outer whitespace."""
name = reply.strip().lower()
return name if name in ACTION_REGISTRY else None
if __name__ == "__main__":
for reply in [
"turn_off_lights",
"Turn_Off_Lights",
"play_music.",
"NONE",
"do not turn_off_lights",
"turn_off_lights or play_music",
"turn_off_lights\nActually, do not.",
"",
"I could turn_off_lights, but where would the fun be in that?",
]:
print(f"{extract_action(reply)!s:<16} <- {reply!r}")
$ uv run python -m labs.command_loop
turn_off_lights <- 'turn_off_lights'
turn_off_lights <- 'Turn_Off_Lights'
None <- 'play_music.'
None <- 'NONE'
None <- 'do not turn_off_lights'
None <- 'turn_off_lights or play_music'
None <- 'turn_off_lights\nActually, do not.'
None <- ''
None <- 'I could turn_off_lights, but where would the fun be in that?'
Fixed strings, so this run is identical on your machine and mine: the gate is testable
apart from the model that feeds it. Only outer whitespace and case can change.
Punctuation, extra lines and every other word cause rejection; a model reply of
play_music. now does nothing. That costs a missed command, which you can
log and ask the user to repeat. A three-token scan would accept
do not turn_off_lights, so short replies need the same complete-name
check as long ones. The regression cases run offline with
python -m unittest labs.test_command_loop in the reference source.
This parser checks representation, not meaning. A model can still return a perfectly spelled action for a negated request, and the earlier substring rules have that weakness too. Keep this lab's handlers as print statements. A real action needs a caller permission check and confirmation where a mistaken selection could matter; never treat registry membership as evidence of consent.
Running it, and paying for it
def execute_action(name: str, text: str) -> bool:
handler = ACTION_REGISTRY.get(name)
if handler is None:
print(f"[WARN] no handler registered for {name!r}")
return False
try:
handler(text)
except Exception as err:
print(f"[ERROR] {name} failed: {type(err).__name__}: {err}")
return False
return True
if __name__ == "__main__":
@register_action("open_pod_bay_doors")
def open_pod_bay_doors(text: str) -> None:
raise TimeoutError("servo controller did not answer in 2.0 s")
print(execute_action("open_pod_bay_doors", "open the doors"))
print(execute_action("dim_lights", "dim the lights"))
$ uv run python -m labs.command_loop
[ERROR] open_pod_bay_doors failed: TimeoutError: servo controller did not answer in 2.0 s
False
[WARN] no handler registered for 'dim_lights'
False
Two gates, one function, one boolean, so a caller can branch without catching
anything. The .get() guard is redundant from this call site, since
extract_action already refused any name the registry does not hold, and
dim_lights is the invention a model offers when the house has dimmers and
the registry does not. Keep the guard anyway: chapter 17's rule engine and chapter
19's scheduler both call actions by name without passing through the extractor. The
try matters more with every volume. Today a handler prints; in volume 5
it drives a servo over a serial link that can time out, and one hardware fault must
not take the voice loop down with it. Catching bare Exception is usually
a smell, but a dispatch boundary exists to run code it does not control and keep
going, and printing the exception type stops it from being where errors die quietly.
def handle(text: str) -> bool:
matched = match_rules(text, RULES)
if matched:
print(f"[RULE] {matched[0]['trigger']}")
return execute_action(matched[0]["action"], text)
reply = propose_action(text)
name = extract_action(reply)
if name is None:
print(f"[MODEL] no action (she said {reply.strip()!r})")
return False
print(f"[MODEL] {name}")
return execute_action(name, text)
if __name__ == "__main__":
for line in ["Lights off, please.", "It's too bright in here.", "How was your day?"]:
print(f"> {line}")
handle(line)
$ uv run python -m labs.command_loop
> Lights off, please.
[RULE] lights off
[ACTION] Turning off lights.
> It's too bright in here.
[MODEL] turn_off_lights
[ACTION] Turning off lights.
> How was your day?
[MODEL] no action (she said 'NONE')
Keyword rules did not become obsolete; they became the fast path. A phrase you say every night hits a substring test, dispatches in microseconds, and never reaches the model. Everything else falls through to the interpreter that reads meaning, at the price named in the first section. Timed on my laptop over ten requests each, the rule path stayed under a tenth of a millisecond and the model path ran between 0.9 and 2.1 seconds; your second figure depends on your CPU and how warm the model is. The two labels are not decoration either: when she does the wrong thing, the first word of the log says whether to go argue with a rule row or with a prompt.
Her conversational replies want randomness; a machine that answers the same insult
identically every time stops being funny by Thursday. This call wants the opposite.
Choosing an action is a classification with one right answer, so
options={"temperature": 0} asks for the most likely token at every step,
making repeated runs on the same sentence far more consistent and much easier to
debug. Consistent, not guaranteed: batching and floating-point order can still shift a
token, so the validation gate stays load-bearing even at zero.
Why this works: a list the model cannot leave
Three moving parts, each removing a different way for a free-text answer to hurt you. The candidate list comes from the registry, so the model is never offered a name with no handler behind it. The reply is checked against that same registry before it means anything, so an invented name is a lookup that fails instead of an action that fires. And the effect stays behind the dictionary chapter 17 built, so the worst outcome of a wrong answer is the wrong registered action, never arbitrary code. Generalized: give a model a closed set, validate its answer against that set, and keep the effects behind a table you wrote.
Notice what did not change. Chapter 20 split classification from dispatch so the
classifier could be replaced later; this chapter replaced it, and
execute_action is still the dispatcher it always was. A seam designed in
volume 2 absorbed a component swap in volume 4 without edits on either side of it, and
the return on all those small dispatch tables is paid in a currency you can count: the
diff.
You already have a working chat function, so reusing it for the command call looks like thrift:
from labs.glados_llm import chat # chapter 8: persona prompt plus conversation history
history: list[dict] = []
reply, history = chat("It's too bright in here.", history)
print("reply: ", repr(reply))
print("action:", extract_action(reply))
reply, history = chat("Thanks.", history)
print("next: ", repr(reply))
$ uv run python -m labs.broken_command_loop
reply: "Oh, is it? I could turn_off_lights for you. I could also not."
action: None
next: "You're welcome. play_music? set_timer? I have a list now, apparently."
Two symptoms, one cause. Chapter 8's build_messages puts her persona in
the system slot, so the command instructions arrive as a user message and lose the
argument to a standing order to be sardonic; she answers in character, the extractor
refuses to pull a key out of a sentence, and nothing dispatches. Meanwhile
chat() appends both halves of every exchange to the history, so the
action names are now part of what she has said, and she imitates herself in the next
reply. That is why stage 2 looks the way it does: propose_action builds a
fresh two-message list on every call and appends nothing to anything. One model, two
conversations, no shared state.
Checkpoint, and the cost you cannot feel
- I can state what a language model buys over keyword triggers and the three things it charges for, including the failure mode keyword matching never had.
- I can explain why the action list in the prompt is generated from the registry, and name the bug a hand-typed copy creates.
- I can trace one sentence from microphone text through rule check, model call, normalizer, registry lookup and handler, and say what happens at each refusal.
- I can show why a three-token scan accepts a negation, and why the complete-name check rejects it without claiming to understand intent.
- I can argue for catching bare
Exceptionat a dispatch boundary and say what makes that different from catching it anywhere else. - I can say why the command call keeps no history and why her conversation call must.
Exercise 1 — let the two lists drift. Replace
action_catalog() with the three names typed by hand, then register a
set_thermostat handler and ask her to warm the room up. What does she
answer, and how long would that bug survive in a house?
NONE, usually with an apology, because the action exists in your code
and not in the only list she was shown. Nothing errors, no log line accuses
anyone, and a handler you wrote and tested never runs. Put
action_catalog() back and the same request finds the thermostat on
the first try.
Exercise 2 — two actions in one breath. Send "kill the
lights and put some music on" through handle(). What fires, what does
not, and what would you change?
The model may pick one action or return extra text that the gate rejects. The current contract cannot represent two actions. Try a separate, explicit list format as an exercise, keeping the handlers as print statements. Parse the whole reply and validate every element before dispatching anything; reject the entire proposal if it contains an unknown name, negation or ambiguous syntax. Do not scavenge valid survivors from a failed parse. Print the proposed sequence for confirmation, then compare its output with the current one-name parser.
Exercise 3 — measure both paths. Wrap the rule branch and
the model branch in time.perf_counter(), run twenty mixed requests, and
print the two averages. Which number would a person in the room actually notice?
The model one, by a factor of roughly ten thousand: a rule dispatch is a substring test over three rows, a model call is a forward pass per token. The number that matters is the slowest few, though, not the average. A two-second pause before the lights go off reads as her ignoring you, and one slow turn in twenty is what people remember. Keep the figures you printed; the next chapter turns this measurement into something logged on every turn instead of taken once by hand.
She interprets now, not just matches, and every interpretation costs a model call you added to a loop that used to be free. That cost is invisible from the inside: nothing crashes, nothing logs, and "she feels slower this week" is not a diagnosis anyone can act on. So the next build makes the system watch itself, sampling CPU, memory and per-component latency into a rolling log that answers the question with a number.