What She Brings Into the Room
Stored is not remembered
Chapter 11 gave her a database and chapter 8 explained the model's amnesia, and between them sits an embarrassing gap: the facts are on disk, and she never mentions them. Tell her your name on Monday and on Tuesday the row exists, unread, while she greets a stranger. A language model is judged purely on the text it receives this call. Memory that stays in the database might as well be someone else's.
The naive bridge is appending the entire transcript to every prompt, and chapter 8's exercise 2 already measured why that fails: the context window fills, old turns get dropped silently, and every call re-processes a wall of "ok" and "thanks" to find the three facts that matter. The real design inverts it. Store distilled memory; inject only the relevant slice, at call time, at one boundary. Memory is not something the model has. It is a string you assemble right before the model speaks.
Distill, budget, inject
# labs/memory_context.py
import sqlite3
from datetime import datetime, timezone
def init_db(conn: sqlite3.Connection) -> None:
conn.executescript("""
CREATE TABLE IF NOT EXISTS facts (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS conversations (
id INTEGER PRIMARY KEY,
summary TEXT NOT NULL,
created_at TEXT NOT NULL
);
""")
conn.commit()
def save_fact(conn, key: str, value: str) -> None:
conn.execute(
"INSERT OR REPLACE INTO facts (key, value, created_at) VALUES (?, ?, ?)",
(key, value, datetime.now(timezone.utc).isoformat()),
)
conn.commit()
$ uv run python labs/memory_context.py
Tables: ['conversations', 'facts']
After two saves of user_name: [('user_name', 'Kaleb')]
One schema change from chapter 11, and it carries the chapter's philosophy:
key is now the PRIMARY KEY, and
INSERT OR REPLACE makes save_fact an upsert. A fact like
user_name has exactly one current value, so telling her your name
twice updates one row instead of accumulating contradictions. Summaries keep their
append-only table, because a log of what happened should accumulate. State
deduplicates; history appends. You met this distinction as an exercise; now it is
load-bearing.
def load_context(conn, max_facts: int = 10, max_summaries: int = 3) -> str:
facts = conn.execute(
"SELECT key, value FROM facts ORDER BY created_at DESC LIMIT ?",
(max_facts,),
).fetchall()
summaries = conn.execute(
"SELECT summary FROM conversations ORDER BY created_at DESC LIMIT ?",
(max_summaries,),
).fetchall()
lines = []
if facts:
lines.append("Known facts about the user:")
for key, value in facts:
lines.append(f" - {key}: {value}")
if summaries:
lines.append("Recent conversation summaries:")
for (summary,) in summaries:
lines.append(f" - {summary}")
return "\n".join(lines)
$ uv run python labs/memory_context.py
Known facts about the user:
- favorite_language: Python
- user_name: Kaleb
Recent conversation summaries:
- User switched the wake word to 'Hey GLaDOS'.
The limits are a budget, and the budget is the design. Context windows are finite,
so the engine caps how much memory rides along, and ORDER BY created_at
DESC decides what survives the cap: the newest. When she someday holds five
hundred facts, the ten most recent make the cut and the stale ones wait on disk.
Note the unpacking, for (summary,) in summaries: single-column rows
are still tuples, and forgetting that produces one of Python's more baffling
beginner tracebacks.
def build_system_prompt_with_memory(base_prompt: str, conn) -> str:
context = load_context(conn)
if context:
return f"{base_prompt}\n\nMemory context:\n{context}"
return base_prompt
$ uv run python labs/memory_context.py
You are GLaDOS. Be sardonic and brief.
Memory context:
Known facts about the user:
- user_name: Kaleb
This ten-line function is the entire bridge between her past and her mouth. The
if context: guard matters more than it looks: on a fresh install there
is no memory, and without the guard every prompt carries an empty "Memory context:"
header, wasted tokens that can also confuse a small model. When there is nothing to
say, the base prompt passes through untouched. Chapter 12's engine slots in here
directly; its assembled personality is the base_prompt argument, and
the two systems compose without either knowing the other exists.
import re
FACT_PATTERNS = [
(re.compile(r"\bmy name is (\w+)", re.I), "user_name"),
(re.compile(r"\bi (?:use|prefer|write) (\w+) for", re.I), "favorite_language"),
(re.compile(r"\bi live in ([\w\s]+?)[.,!]", re.I), "location"),
]
def extract_facts(text: str) -> list[tuple[str, str]]:
found = []
for pattern, key in FACT_PATTERNS:
m = pattern.search(text)
if m:
found.append((key, m.group(1).strip()))
return found
print(extract_facts("Hey, my name is Kaleb and I use Python for everything."))
$ uv run python labs/memory_context.py
[('user_name', 'Kaleb'), ('favorite_language', 'Python')]
The last piece closes the loop: facts get into the database by listening.
A handful of regex patterns over the user's lines catches the explicit
declarations, and each hit becomes a save_fact upsert. It is
deliberately humble. Regexes catch "my name is Kaleb" and miss "call me Kaleb,"
and that is fine for now: a memory that reliably catches plain statements beats
one that cleverly misunderstands subtle ones. Volume 9's retrieval work upgrades
the mining; the plumbing it feeds is finished today.
Why this works: memory is retrieval, not recall
Nothing in the model changed. What feels like memory is a loop running entirely outside it: distill facts to disk, retrieve the relevant few at call time, inject them into the prompt. The model cannot distinguish a fact it was "told" yesterday from one you pasted in this second, because from where it sits there is no difference. Once that clicks, every "AI with memory" product you have used becomes legible: they are all running some version of this chapter, with bigger retrieval.
The engineering payoff is the single boundary.
build_system_prompt_with_memory takes a string and returns a string, and
nothing downstream knows where the extra lines came from. Swap SQLite for a vector
store next year and only load_context changes; the contract holds. When
people say "design to interfaces," this is the size of thing they mean, and it is
rarely bigger.
The formatting loop, written the way everyone writes it first, treating rows as strings:
for summary in summaries: # rows, not strings
lines.append(f" - {summary}")
$ uv run python labs/memory_context.py
You are GLaDOS. Be sardonic and brief.
Memory context:
Recent conversation summaries:
- ("User switched the wake word to 'Hey GLaDOS'.",)
No crash this time, which makes it worse: the prompt ships with tuple syntax inside it, parentheses, quotes, trailing comma and all, and the model reads that noise on every call. You find it days later while debugging why her replies sometimes quote punctuation back at you. The fix is stage 2's unpacking. The lesson generalizes past SQLite: look at the actual string you send the model. Print the assembled prompt whenever behavior is odd; half of prompt bugs are visible in one glance at what the model actually received.
Checkpoint, and she knows you now
- I can explain the three-step loop (persist, retrieve, inject) and say which step the model participates in: none of them.
- I can say why facts upsert while summaries append, in schema terms and in meaning.
- I can defend the memory budget and predict which facts survive when the cap overflows.
- I know what the empty-context guard prevents, and what single-column rows actually are.
- When her prompts misbehave, I print the assembled string before I touch any code.
Exercise 1 — the Tuesday test. Wire the engine into the voice loop: extract facts from each user line, save them, and build every prompt through the memory boundary. Tell her your name, kill the process, restart, and ask who you are.
She answers with your name, and the whole volume so far is in the answer: the regex mined it, the upsert stored it, the loader budgeted it, the boundary injected it, and the model read it fresh off the prompt like it had known all along. That is the trick, and now it is your trick.
Exercise 2 — starve the budget. Set
max_facts=2, store five facts, and print the assembled prompt.
Which two ride along, and what would you change if one of the dropped three was
the user's name?
The two newest, by timestamp. If the name matters more than recency, recency is the wrong ranking, and you have discovered relevance: facts could carry a priority column, or be selected by similarity to the current question. The second idea has a name (retrieval) and a chapter (85). The budget forced the question, which is what budgets are for.
Exercise 3 — teach the miner one more pattern. Add a pattern for coffee orders ("I take my coffee black"), then find a phrasing that defeats all your patterns. How would a smarter miner catch it?
"Black coffee for me, always" sails past a regex built for "I take my coffee." The smarter miner is the model itself, asked after each exchange: "Does this turn state a durable fact about the user? Answer as key: value or NONE." That costs one extra model call per exchange, and the trade (regex is free and literal; the model is slow and general) is one you can now price with real numbers.
Her mind now has four organs: memory, personality, a name, and a context engine binding them into her sentences. What it lacks is a nervous system. Every module so far calls the next one directly, and the web of imports is already forming. Next chapter cuts the web before it hardens.