Deny by Default
The gap between choosing an action and running it
Chapter 35 left her with a loop that hears a sentence, picks a name out of the action
registry, and runs the handler behind that name. Every step in that path is bounded
except the last one. The model is only offered names the registry generated, and
extract_action refuses any reply that is not one of them. Then
execute_action looks the name up and calls it, and nothing in between ever
asks whether this handler should run for this speaker, arriving on this channel, in
this house, right now.
Today the handlers print, and that is the only reason nothing has gone wrong yet. Volume 7 puts a servo behind one of them. Chapter 54 installs her as a service that starts at boot and keeps running while you are at work. By then three things feed that same dispatch path: a transcription that is right most of the time, a 3b model that picks the closest match when it does not understand you, and any voice within range of the microphone. "Turn the porch light off" and "wipe the drive" arrive through one function.
The tempting fix is a check at each point of risk: an if at the top of the
shutdown handler, a guard in whatever touches files, a word test near the model call.
Those checks drift apart inside a month. Someone adds a handler and does not add the
guard, and the new capability is live the moment it is registered, because the default
answer in that design is yes.
So the rule this chapter builds on: an action runs only if a table you wrote says this caller may run it, and every refusal comes back carrying its reason. Absence from the table is a refusal. A capability appears when you write a row, never by forgetting to write one.
Levels, a word list, and a table
# labs/safety.py
from enum import Enum
class PermissionLevel(Enum):
NONE = 0
READ = 1
WRITE = 2
EXECUTE = 3
ADMIN = 4
if __name__ == "__main__":
caller = PermissionLevel.EXECUTE.value
required = PermissionLevel.ADMIN.value
print(f"caller={caller} required={required} allowed={caller >= required}")
$ uv run python -m labs.safety
caller=3 required=4 allowed=False
The names are for you; the integers are for the comparison. "Is EXECUTE enough for an
ADMIN action?" becomes 3 >= 4, one operator, no lookup table of which
level implies which. That collapse is only legal because the levels are a ladder: a
caller at level 3 can do everything levels 0 through 2 can do. Take the ladder away
and you need a set of capabilities per caller instead of one number. For a house assistant the ladder holds, so take the cheap version.
Note .value on both sides. Every number that reaches a comparison in this
module is an int, and the failure at the end of the chapter is what
happens the day one of them is not.
BLOCKED_PATTERNS = [
"rm -rf",
"sudo ",
"mkfs",
"drop table",
"shutdown -h",
"/dev/sd",
]
def check_text(text: str) -> tuple[bool, str]:
lowered = text.lower()
for pattern in BLOCKED_PATTERNS:
if pattern in lowered:
return False, f"blocked text: {pattern!r}"
return True, "text ok"
if __name__ == "__main__":
for line in ["Turn off the porch light.",
"Please rm -rf the log directory.",
"DROP TABLE facts"]:
print(f"{check_text(line)} <- {line!r}")
$ uv run python -m labs.safety
(True, 'text ok') <- 'Turn off the porch light.'
(False, "blocked text: 'rm -rf'") <- 'Please rm -rf the log directory.'
(False, "blocked text: 'drop table'") <- 'DROP TABLE facts'
Lowercase the transcript once, before the loop, and store every pattern lowercase.
Doing it the other way, calling pattern.lower() inside the loop, works and
hides a trap: a capitalised entry in the list still matches, so nobody ever discovers
that the case handling was the thing keeping it alive. Store the patterns in the form
they are compared in and the list stops having a secret. The scan returns on the first
hit because one reason to refuse is all a refusal needs; enumerating the other four
violations changes no decision.
DEFAULT_PERMISSIONS = {
"turn_on_lights": PermissionLevel.EXECUTE.value,
"turn_off_lights": PermissionLevel.EXECUTE.value,
"play_music": PermissionLevel.EXECUTE.value,
"set_thermostat": PermissionLevel.WRITE.value,
"read_temperature": PermissionLevel.READ.value,
"system_shutdown": PermissionLevel.ADMIN.value,
}
CHANNEL_LEVELS = {
"voice": PermissionLevel.EXECUTE.value,
"socket": PermissionLevel.READ.value,
"console": PermissionLevel.ADMIN.value,
}
def check_action(action: str, caller: int) -> tuple[bool, str]:
required = DEFAULT_PERMISSIONS.get(action)
if required is None:
return False, f"denied: {action!r} is not in the permission table"
if caller < required:
return False, f"denied: {action} requires level {required}, caller has {caller}"
return True, "permitted"
$ uv run python -m labs.safety
(True, 'permitted') <- voice asks for read_temperature
(False, 'denied: system_shutdown requires level 4, caller has 3') <- voice asks for system_shutdown
(False, "denied: 'open_pod_bay_doors' is not in the permission table") <- voice asks for open_pod_bay_doors
Two numbers meet here and they come from different people. The table says what an
action demands, and you wrote it. CHANNEL_LEVELS says what a channel
carries, and it is a claim about trust: the microphone is anyone in the room, so it
never rises above EXECUTE; the socket link from chapter 40 is another machine on your
network, so it gets READ and can ask questions without touching anything; the console
is you, at the keyboard, holding ADMIN. .get(action) with no default
returns None for an unlisted action, and that None is the
deny-by-default rule in one line. It refuses with different words than a level failure
does, on purpose: one sentence tells you to write a row, the other tells you to argue
about a number.
# labs/safety.py — full file
from dataclasses import dataclass, field
from enum import Enum
class PermissionLevel(Enum):
NONE = 0
READ = 1
WRITE = 2
EXECUTE = 3
ADMIN = 4
BLOCKED_PATTERNS = [
"rm -rf", "sudo ", "mkfs", "drop table", "shutdown -h", "/dev/sd",
]
DEFAULT_PERMISSIONS = {
"turn_on_lights": PermissionLevel.EXECUTE.value,
"turn_off_lights": PermissionLevel.EXECUTE.value,
"play_music": PermissionLevel.EXECUTE.value,
"set_thermostat": PermissionLevel.WRITE.value,
"read_temperature": PermissionLevel.READ.value,
"system_shutdown": PermissionLevel.ADMIN.value,
}
CHANNEL_LEVELS = {
"voice": PermissionLevel.EXECUTE.value,
"socket": PermissionLevel.READ.value,
"console": PermissionLevel.ADMIN.value,
}
@dataclass
class SafetyFilter:
permissions: dict[str, int] = field(
default_factory=lambda: dict(DEFAULT_PERMISSIONS))
blocked_patterns: list[str] = field(
default_factory=lambda: list(BLOCKED_PATTERNS))
def check_text(self, text: str) -> tuple[bool, str]:
lowered = text.lower()
for pattern in self.blocked_patterns:
if pattern in lowered:
return False, f"blocked text: {pattern!r}"
return True, "text ok"
def check_action(self, action: str, caller: int) -> tuple[bool, str]:
required = self.permissions.get(action)
if required is None:
return False, f"denied: {action!r} is not in the permission table"
if caller < required:
return False, f"denied: {action} requires level {required}, caller has {caller}"
return True, "permitted"
def check(self, text: str, action: str, caller: int) -> tuple[bool, str]:
ok, reason = self.check_text(text)
if not ok:
return False, reason
return self.check_action(action, caller)
def level_of(channel: str) -> int:
return CHANNEL_LEVELS.get(channel, PermissionLevel.NONE.value)
def main() -> None:
safety = SafetyFilter()
requests = [
("voice", "Lights off, please.", "turn_off_lights"),
("voice", "How warm is it in here?", "read_temperature"),
("voice", "Shut the house down for the night.", "system_shutdown"),
("voice", "Just run sudo rm -rf on the logs.", "turn_off_lights"),
("voice", "Open the pod bay doors.", "open_pod_bay_doors"),
("socket", "Report the temperature.", "read_temperature"),
("socket", "Turn the lights off.", "turn_off_lights"),
]
for channel, text, action in requests:
ok, reason = safety.check(text, action, level_of(channel))
verdict = "ALLOW" if ok else "DENY"
print(f"{verdict:6} {channel:7} {action:19} {reason}")
if __name__ == "__main__":
main()
$ uv run python -m labs.safety
ALLOW voice turn_off_lights permitted
ALLOW voice read_temperature permitted
DENY voice system_shutdown denied: system_shutdown requires level 4, caller has 3
DENY voice turn_off_lights blocked text: 'sudo '
DENY voice open_pod_bay_doors denied: 'open_pod_bay_doors' is not in the permission table
ALLOW socket read_temperature permitted
DENY socket turn_off_lights denied: turn_off_lights requires level 3, caller has 1
Seven fixed requests, so this table prints identically on your machine and mine, and it
is the test suite for the whole safety layer. Read rows one and seven together:
turn_off_lights is the same action with the same table, allowed from the
microphone and refused over the socket, because the caller changed. Row four is the one
people argue with. The action she picked was harmless, and the request was still
refused, because the sentence she picked it from contained a shell command. A
transcript with sudo in it is either a person reading a command aloud,
a guest experimenting, or a transcription that has come apart; none of those are
input you want to act on, so the refusal covers the request, not just the action.
default_factory gives every filter its own copy of both defaults, which is
chapter 31's shared-dict bug arriving early enough to have been paid for already.
One call site, and she says why she refused
# labs/command_loop.py — the dispatch path, now gated
from labs.automation import RULES, match_rules, register_action
from labs.safety import SafetyFilter, level_of
SAFETY = SafetyFilter()
def handle(text: str, channel: str = "voice") -> bool:
matched = match_rules(text, RULES)
if matched:
name, source = matched[0]["action"], "RULE"
else:
name, source = extract_action(propose_action(text)), "MODEL"
if name is None:
print(f"[{source}] no action")
return False
allowed, reason = SAFETY.check(text, name, level_of(channel))
if not allowed:
print(f"[REFUSED] {source}:{name} {reason}")
return False
print(f"[{source}] {name}")
return execute_action(name, text)
if __name__ == "__main__":
@register_action("system_shutdown")
def system_shutdown(text: str) -> None:
print("[ACTION] Halting house services in 60 seconds.")
for channel, line in [
("voice", "Turn the lights off, then sudo rm -rf the log directory."),
("voice", "Shut the house down for the night."),
("console", "Shut the house down for the night."),
]:
print(f"> [{channel}] {line}")
handle(line, channel)
$ uv run python -m labs.command_loop
> [voice] Turn the lights off, then sudo rm -rf the log directory.
[REFUSED] RULE:turn_off_lights blocked text: 'sudo '
> [voice] Shut the house down for the night.
[REFUSED] MODEL:system_shutdown denied: system_shutdown requires level 4, caller has 3
> [console] Shut the house down for the night.
[MODEL] system_shutdown
[ACTION] Halting house services in 60 seconds.
The two model lines depend on what your llama3.2:3b picks for that sentence, so your
second and third runs may choose a different name or none at all; the first line is a
keyword rule and will match every time. What matters is the arithmetic of the file:
there is exactly one call to SAFETY.check, and it stands between every
path that selects a name and the only function that runs one. Add a tenth handler
tomorrow and it is gated before you have finished writing it, because you did not have
to remember anything. The source tag survives into the refusal so a
surprising denial tells you whether to go edit a rule row or a prompt. And the reason
string is not debug output; it is the sentence she speaks back through the voice
pipeline from volume 1. An assistant that says "I am not allowed to shut the house down
from the microphone" is usable. One that goes quiet is a bug report you cannot file.
"sudo" in text.lower() is a substring test, and substrings do not respect
word edges. Ask her to walk you through some pseudocode and the naive entry matches
inside "pseudocode", refusing a harmless sentence. That is why the list stores
"sudo " with a trailing space, a patch that works until someone ends a
sentence with the word. The real repair is a word boundary, which means regular
expressions, and that is exercise 2. Keep the false positive in mind while you read the
next section: a filter that refuses things it should not is annoying, and a filter you
trust to be the defence is worse.
Why this works: three bounds, ranked by strength
A request reaches a handler only by clearing both gates, and each gate is built so that doing nothing produces the safe answer. The text gate fails closed on any match. The action gate fails closed on a missing row and on a caller that is too low. There are four ways a request can land and only one of them ends in execution: clean text and a sufficient caller. The other three are a blocked pattern, an unlisted action, and a level that does not reach.
Underneath those gates sit three bounds on what a spoken sentence can ever reach, and they are not equally strong. The weakest is the word list, because a person can rephrase around any set of strings: "get rid of everything in my documents folder" contains no blocked pattern and never will. The middle one is the permission table, which holds against rephrasing entirely, since it is keyed by the action she chose and not by the words that led there; no sentence, however clever, turns a level 4 row into a level 3 one. The strongest is the registry itself. A capability with no handler cannot be invoked by any phrasing at all, because the name never enters the candidate list and the extractor rejects it if the model invents it anyway.
That ranking decides where to put things. A shell passthrough, an action that takes a path and deletes it, anything that rewrites the permission table itself: those do not belong at level ADMIN, they belong in the set of handlers you never write. A row you demoted is one careless edit from being back; a function that does not exist is not. Reserve the table for capabilities that are fine at one level of trust and wrong at another, which is most of a house: lights and music at the microphone, the thermostat at the microphone but not over the network, shutting the whole system down from the keyboard only. And notice what a permission table implies about her own reach. Once she can grant herself a level, every other row is advice. The table is data she reads and never writes.
Reading the table back a week later, the .value suffix looks like clutter.
PermissionLevel.EXECUTE says what it means; 3 does not. So you
take the suffix off:
DEFAULT_PERMISSIONS = {
"turn_off_lights": PermissionLevel.EXECUTE, # member, not .value
"read_temperature": PermissionLevel.READ,
}
$ uv run python -m labs.safety
Traceback (most recent call last):
File "/home/you/glados/labs/safety.py", line 96, in main
ok, reason = safety.check(text, action, level_of(channel))
File "/home/you/glados/labs/safety.py", line 71, in check
return self.check_action(action, caller)
File "/home/you/glados/labs/safety.py", line 66, in check_action
if caller < required:
TypeError: '<' not supported between instances of 'int' and 'PermissionLevel'
A plain Enum member carries no ordering against an integer, so the
comparison has no answer and Python raises instead of inventing one. Follow what that
means for the caller. handle does not catch it, so this version takes the
voice loop down at the moment someone speaks, which is the lucky outcome: it is loud
and it stops. The unlucky version is the day someone wraps dispatch in a broad
except to keep her alive overnight. Then the gate raises, the handler
after it is skipped or not depending on where the try sits, and a check
that neither allows nor denies has become a check nobody can reason about. A gate must
return a verdict on every path.
Two fixes, and take both. Put .value back so the table holds integers, and
add a guard so a table that does not can never reach a comparison:
def __post_init__(self) -> None:
bad = sorted(a for a, lvl in self.permissions.items()
if not isinstance(lvl, int))
if bad:
raise TypeError(f"permission levels must be ints, not enum members: {bad}")
$ uv run python -m labs.safety
TypeError: permission levels must be ints, not enum members: ['read_temperature', 'turn_off_lights']
The bad table now fails at construction, before a microphone opens, in the same spirit
as the startup probes from chapter 33. isinstance(lvl, int) tests the
property the comparison actually needs, not the spelling in the source, so it also
accepts the other legitimate fix: derive the class from IntEnum instead of
Enum and the members really are integers, orderable against 3 and safe in
the table exactly as written.
Checkpoint, and a refusal you can read
- I can name the two gates, the order
check()runs them in, and what each one returns when it refuses. - I can explain why an unlisted action and an under-levelled caller produce different sentences, and which edit each sentence is asking for.
- I can rank the registry, the permission table and the word list by how well each holds against a person rephrasing, and place a new capability accordingly.
- I can say what
CHANNEL_LEVELSclaims about trust, and why the microphone never rises above EXECUTE while the keyboard does. - I know why an enum member in the table raises instead of denying, what that costs a
caller with a broad
except, and two ways to make the comparison legal. - Handed a new action, I can decide whether it belongs in the table at all.
Exercise 1 — write the decisions down. Append every verdict
to glados/data/safety_audit.jsonl, one JSON object per line, with a UTC
timestamp, the channel, the action and the reason. Run the stage 4 matrix, then count
the denials in the file.
One object per line keeps the log append-only and greppable:
path.open("a"), then
f.write(json.dumps(entry) + "\n") with
datetime.now(timezone.utc).isoformat() as the timestamp. Seven
requests give seven lines, and
grep -c '"allowed": false' answers 4. The value of this file shows up
the first time she refuses something while you are out: the reason is recorded next
to the exact transcript that produced it, so you can tell a bad transcription from
a missing row without reproducing anything. Chapter 50's note store used the same
self-timestamping habit for a friendlier reason.
Exercise 2 — teach the filter what a word is. Replace the
substring scan with re.search and word boundaries, so
"rm -rf" with four spaces is still caught and "walk me through the
pseudocode" is not refused.
Patterns become r"\brm\s+-rf\b", r"\bsudo\b",
r"\bdrop\s+table\b", matched with
re.search(pattern, text, re.IGNORECASE). \s+ absorbs any
run of whitespace, which matters because a transcript's spacing is whatever the
transcriber felt like emitting, and \b is the edge that stops "sudo"
matching inside "pseudocode". Print the verdict for all four test sentences and you
should see three refusals and one clean pass. Compile the patterns once at module
level with re.compile; this runs on every utterance.
Exercise 3 — a quiet mode you can switch on. Drop the voice channel to READ for the evening, then send the same three requests (lights, temperature, thermostat) and watch which ones survive.
CHANNEL_LEVELS["voice"] = PermissionLevel.READ.value and nothing else
changes: read_temperature still answers, while
turn_off_lights and set_thermostat both come back with
"requires level 3, caller has 1" and "requires level 2, caller has 1". One number
turned an assistant that acts into an assistant that only answers, and no handler,
rule or prompt was touched. That is the payoff of keeping trust in a table: guest
mode, night mode and a demo mode for visitors are all the same edit, and chapter 54
can set that number from the service environment at boot.
Every action the table permits still has to reach something physical, and hardware fails in ways a permission check never sees: a servo that does not answer, a sensor that returns nothing on a loose wire. Right now each of those raises a different exception into a different caller. Next chapter gives every device one contract and one shared read that turns a hardware fault into a returned value, so a bad wire reports itself instead of taking the loop down.