GLaDOS Vol 4 · One System
ch 37 / 99
Chapter 37

A Memory She Can Search

She stores everything and finds the newest thing

Chapter 11 gave her a file that survives the power button. Chapter 14 taught her to read from it before she speaks: the ten newest facts, the three newest summaries, formatted into lines and pasted above her persona. It works, and it has exactly one idea in it. Newest.

Recency is a fine ranking for a week-old database. Live with her for a few months and the ten newest facts are the ten things you happened to mention on Tuesday, while the answer to "what did I say the printer's address was" sits at row 340, on disk, unread, as she assembles a prompt out of your dinner plans. The store grew. The retrieval did not.

A second problem hides under the first. Every row weighs the same. The address you typed carefully, the string chapter 14's regex miner scraped out of a half-finished sentence, and anything the model itself proposed are all just rows, so when she is wrong you cannot tell whether she remembered badly or was told badly.

Hence this chapter's rule: a stored fact is only as useful as your ability to find it by what it is about and to say where it came from. Two columns and one query, and her memory stops being a log she reads the end of.

A table that knows where its rows came from

▣ Build · stage 1 — provenance decides the number
# glados/knowledge.py
import sqlite3
from datetime import datetime, timezone
from pathlib import Path

DB_PATH = Path("glados/data/memory.db")

CONFIDENCE = {"stated": 1.0, "mined": 0.6, "inferred": 0.4}

def init_knowledge(conn: sqlite3.Connection) -> None:
    conn.execute("""
        CREATE TABLE IF NOT EXISTS knowledge (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            topic TEXT NOT NULL,
            content TEXT NOT NULL,
            source TEXT NOT NULL,
            confidence REAL NOT NULL,
            created_at TEXT NOT NULL
        )
    """)
    conn.commit()

def remember(conn: sqlite3.Connection, topic: str, content: str,
             source: str = "stated") -> None:
    conn.execute(
        "INSERT INTO knowledge (topic, content, source, confidence, created_at)"
        " VALUES (?, ?, ?, ?, ?)",
        (topic, content, source, CONFIDENCE[source],
         datetime.now(timezone.utc).isoformat()),
    )
    conn.commit()

if __name__ == "__main__":
    conn = sqlite3.connect(":memory:")
    init_knowledge(conn)
    remember(conn, "network", "The office printer is on 192.168.1.40.")
    remember(conn, "network", "The printer might be on the guest network.",
             source="inferred")
    for row in conn.execute("SELECT topic, source, confidence FROM knowledge"):
        print(row)
$ uv run python glados/knowledge.py
('network', 'stated', 1.0)
('network', 'inferred', 0.4)

This is a new table beside chapter 11's, not a replacement for it, because the two hold different kinds of thing. facts is keyed state: one current value for user_name, upserted. Knowledge is a growing pile of statements about topics, and three true sentences about the printer can coexist without contradicting each other. State deduplicates, history appends, and knowledge accumulates.

The CONFIDENCE dict is the load-bearing part. Confidence here is not measured, it is assigned, and the thing it is assigned from is provenance: you said it, a regex mined it, or the model guessed it. That keeps the number defensible in one sentence, and it means the caller never picks a float out of the air. Add a fourth source someday and you have to decide what it is worth before you can store anything through it, which is the point.

▣ Build · stage 2 — find by substring, order by trust
def search(conn: sqlite3.Connection, terms: list[str],
           limit: int = 5) -> list[dict]:
    if not terms:
        return []
    clause = " OR ".join(["topic LIKE ? OR content LIKE ?"] * len(terms))
    params: list = []
    for term in terms:
        params += [f"%{term}%", f"%{term}%"]
    params.append(limit)
    rows = conn.execute(
        "SELECT topic, content, confidence FROM knowledge"
        f" WHERE {clause}"
        " ORDER BY confidence DESC, created_at DESC LIMIT ?",
        params,
    ).fetchall()
    return [{"topic": r[0], "content": r[1], "confidence": r[2]} for r in rows]

if __name__ == "__main__":
    remember(conn, "network", "The printer jams on thick paper.", source="mined")
    for hit in search(conn, ["printer"]):
        print(f"{hit['confidence']:.1f}  [{hit['topic']}] {hit['content']}")
$ uv run python glados/knowledge.py
1.0  [network] The office printer is on 192.168.1.40.
0.6  [network] The printer jams on thick paper.
0.4  [network] The printer might be on the guest network.

Three sentences about one printer, returned in the order you would want a person to read them: the one you told her, then the one a regex found, then the guess. That ordering is the entire difference between a search box and a knowledge store. created_at DESC breaks ties so two rows at the same confidence come back newest-first and never in whatever order SQLite feels like, which matters the day you diff two prompts to explain a behavior change.

Look hard at the f-string in the middle of that SQL, because chapter 11 told you never to write one. Read what it splices: len(terms) copies of a fixed clause containing question marks. The only thing formatted into the statement is punctuation generated from an integer, and every value still travels as a bound parameter. The rule was never "no f-strings near SQL", it was "no data in the SQL text", and knowing which one you are obeying is what lets you build a variable-length WHERE without reopening the injection you closed in volume 2.

The question people actually ask

▣ Build · stage 3 — a sentence becomes search terms
import re

STOPWORDS = {
    "the", "and", "but", "for", "with", "you", "your", "was", "were", "are",
    "did", "does", "what", "when", "where", "who", "why", "how", "that",
    "this", "about", "have", "has", "had", "say", "said", "tell", "know",
    "can", "could", "would", "there", "then", "from", "into", "not", "get",
    "got", "please", "again", "any", "all", "its", "her", "his", "our",
}

def keywords(line: str, limit: int = 4) -> list[str]:
    words = re.findall(r"[a-z0-9]+", line.lower())
    out: list[str] = []
    for word in words:
        if len(word) < 3 or word in STOPWORDS or word in out:
            continue
        out.append(word)
    return out[:limit]

if __name__ == "__main__":
    print(keywords("What did I say the printer's address was?"))
    print(keywords("Tell me about Whisper."))
$ uv run python glados/knowledge.py
['printer', 'address']
['whisper']

Nobody types a search term at a house assistant. They ask a question, and a question is mostly grammar: seven of the nine words in the first line carry no information about which row you want. Three filters do the work. The character class splits on everything that is not a letter or digit, so printer's becomes printer plus a stray s; the length floor of three drops that s and every me and it; the stopword set removes the question words that appear in half your sentences and would therefore match half your rows. The cap of four keeps one rambling sentence from turning into a query with twenty clauses.

▣ Build · stage 4 — results become a block of prompt
def knowledge_context(conn: sqlite3.Connection, line: str,
                      limit: int = 3) -> str:
    hits = search(conn, keywords(line), limit=limit)
    if not hits:
        return ""
    lines = ["Relevant knowledge (confidence in brackets, 1.0 means the user said so):"]
    for hit in hits:
        lines.append(f"  - [{hit['confidence']:.1f}] {hit['content']}")
    return "\n".join(lines)

if __name__ == "__main__":
    print(knowledge_context(conn, "What did I say the printer's address was?"))
    print(repr(knowledge_context(conn, "How was your day?")))
$ uv run python glados/knowledge.py
Relevant knowledge (confidence in brackets, 1.0 means the user said so):
  - [1.0] The office printer is on 192.168.1.40.
  - [0.6] The printer jams on thick paper.
  - [0.4] The printer might be on the guest network.
''

The empty string on the second call is chapter 14's guard, kept deliberately: a query with no hits contributes nothing, so a small model never reads a "Relevant knowledge" header with nothing under it and never has to work out what that means. Printing the confidence into the prompt is a judgment call you can argue with. A 3b model will not do arithmetic on those numbers, but it does treat a bracketed 0.4 as a hedge, and her answers get noticeably more careful about the low rows. Delete the brackets and she states the guess as flatly as the fact.

▣ Build · stage 5 — one prompt, two kinds of memory
# glados/knowledge.py -- the boundary, and the real file
from labs.memory_context import load_context   # chapter 14's budgeted loader

def build_prompt(base_prompt: str, conn: sqlite3.Connection,
                 user_text: str) -> str:
    parts = [base_prompt]
    memory = load_context(conn)
    if memory:
        parts.append(f"Memory context:\n{memory}")
    knowledge = knowledge_context(conn, user_text)
    if knowledge:
        parts.append(knowledge)
    return "\n\n".join(parts)

if __name__ == "__main__":
    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
    conn = sqlite3.connect(DB_PATH)
    init_knowledge(conn)
    remember(conn, "network", "The office printer is on 192.168.1.40.")
    remember(conn, "network", "The printer jams on thick paper.", source="mined")
    print(build_prompt("You are GLaDOS. Be sardonic and brief.", conn,
                       "What did I say the printer's address was?"))
    conn.close()
$ uv run python glados/knowledge.py
You are GLaDOS. Be sardonic and brief.

Memory context:
Known facts about the user:
  - favorite_language: Python
  - user_name: Kaleb

Relevant knowledge (confidence in brackets, 1.0 means the user said so):
  - [1.0] The office printer is on 192.168.1.40.
  - [0.6] The printer jams on thick paper.

Two retrieval strategies, one prompt, and each block earns its place a different way. The memory block is unconditional and recency-ranked: who you are travels with every call, because she needs your name whatever you asked about. The knowledge block is conditional and relevance-ranked: it appears only when the question shares vocabulary with something stored. The whole chapter reduces to the third argument on build_prompt. Chapter 14's boundary took a database and a persona; this one also takes the question, and a retriever that never sees the question can only ever guess with a clock.

One wart is deliberate. Run this file twice and the printer rows appear twice, since knowledge has no unique key. A UNIQUE(topic, content) constraint plus INSERT OR IGNORE ends that in one line, and you should add it only once you have decided that two identical sentences are never two separate observations.

Why this works: candidates, a ranking, and a cap

Every retrieval system ever built is the same three moves. Generate candidates, score them, keep the top few. Here the generator is LIKE, which compares a pattern against the whole stored value where % stands for any run of characters and _ for exactly one, so '%printer%' matches anything containing those seven letters while a bare 'printer' matches only a value that is precisely that word. The scorer is the confidence column. The cap is LIMIT, and it is what makes retrieval safe to wire into a prompt at all: however many rows match, a bounded number reach the model, so the budget chapter 14 set cannot be blown by a lucky query on a large database.

Swap any one of the three and the other two keep working, which is the payoff for keeping them in separate functions. That is also the honest limit of what you built today. The generator matches characters, and characters are not meaning. Ask about the printer and she finds the printer. Ask "what is the thing in the hall that eats paper" and she finds nothing at all, because not one of those words appears in the row that answers the question. Keyword retrieval buys you every question phrased with the same words as the answer, which in a house is most of them, and it fails completely on the rest with no warning that it failed.

◆ Note — FTS5, and the ceiling it does not raise

SQLite ships a full-text engine. A virtual table declared USING fts5(topic, content) tokenizes text into words, so MATCH 'printer' hits the word printer without hitting sprinter, stems plurals, handles a phrase query, and gives you a rank column computed from term rarity instead of a confidence number you assigned. It is a real upgrade and exercise 3 builds it. It is also the same kind of search: better candidate generation over the same words. The question about the thing in the hall still returns nothing. Getting past that needs retrieval over meaning rather than spelling, where every sentence becomes a list of numbers and closeness is arithmetic, and that machinery is volume 9's subject.

⚠ Worked failure — the sentence that matched nothing

Skip stage 3 and hand the user's line straight to the search, which is the obvious thing to write once search takes a list:

def knowledge_context(conn, line: str, limit: int = 3) -> str:
    hits = search(conn, [line], limit=limit)   # BUG: the whole sentence is one term
    ...
$ uv run python glados/knowledge.py
You are GLaDOS. Be sardonic and brief.

Memory context:
Known facts about the user:
  - user_name: Kaleb
$ uv run python labs/wire_core.py   # her reply; yours will differ
GLaDOS: I have no idea. Perhaps try asking the printer.

No traceback, no empty-result warning, no clue in the prompt that anything was attempted. LIKE '%What did I say the printer's address was?%' asks whether that entire question appears inside a stored value, and a forty-character pattern cannot be contained in a thirty-eight-character sentence, so the match count is zero for reasons of length before it is zero for reasons of wording. Then the guard from stage 4 does its job perfectly: no hits, no block, prompt unchanged. She answers from her own weights with the fact sitting one table away, and she sounds exactly as confident as when she is right.

The move that ends this in twenty seconds is the one from chapter 14: print what you actually sent. Here that means the terms, not the prompt. print(keywords(line)) above the search shows ['printer', 'address'] in the fixed version and ["What did I say the printer's address was?"] in the broken one, and the square brackets around a whole sentence say everything. Retrieval that returns nothing is the failure mode to fear in this chapter, because it is indistinguishable from a database that never knew.

Checkpoint, and a pile of numbers you guessed

✓ Checkpoint — what you can now do
  • I can say why knowledge gets its own table instead of joining chapter 11's keyed facts, in terms of what each one promises about duplicate rows.
  • I can defend every number in CONFIDENCE by naming the source it stands for, and I know why the caller is not allowed to pass a float.
  • I can explain the f-string in the WHERE clause and state the rule it obeys, which is not "never format SQL".
  • I can name the three filters that turn a spoken question into search terms and say what each one would let through if removed.
  • I can trace one question through keywords, LIKE, confidence ordering, LIMIT and the prompt boundary, and say what an empty result does at every step.
  • I can state a question my search will fail on, explain why FTS5 would not save it, and name what would.
⚡ Exercises — try first, then reveal
Exercise 1 — confirm a guess. Write confirm(conn, row_id) that promotes a mined or inferred row to stated with confidence 1.0, returns the number of rows changed, and prove it by re-running the printer search before and after.

One UPDATE knowledge SET source = 'stated', confidence = ? WHERE id = ? with CONFIDENCE["stated"] bound in, then return cursor.rowcount so a caller can tell a real promotion from a typo in the id. Run the search again and the row you confirmed has climbed to the top of the printed list. That is the first half of a memory that corrects itself: she proposes, you confirm, and the ranking remembers your verdict. Deciding when she should ask you is a policy question volume 6 takes up properly.

Exercise 2 — rank by how much matched. A row matching one term outranks a row matching three whenever its confidence is higher. Fetch a wider candidate set, score each row by how many terms it contains, and sort by that score before confidence. Which questions change their answer?

Raise the SQL LIMIT to something like 20, count matches in Python with sum(term in row["content"].lower() for term in terms), then sorted(rows, key=lambda r: (r["matches"], r["confidence"]), reverse=True) and slice to three. Single-term questions print exactly what they printed before; the ones that move are the specific questions, where "printer address" now beats a confident row that only says printer. You have separated candidate generation from scoring, in code, which is the structure every serious retrieval system has and the reason volume 9 can replace one half without touching the other.

Exercise 3 — build the FTS5 version, then break it. Create CREATE VIRTUAL TABLE knowledge_fts USING fts5(topic, content), copy your rows in, search with WHERE knowledge_fts MATCH ? ORDER BY rank, and then find a question that both searches miss.

MATCH 'printer' returns the printer rows ranked by term rarity, with no wildcards written by you, no sprinter false positive, and an inverted index instead of a scan. Then ask "what is the thing in the hall that eats paper" and watch both versions return an empty list. Neither engine knows that a printer eats paper. Keep the question written down: it is the exact demonstration you will want on the day volume 9 turns sentences into vectors and the same query finally lands.

Count the numbers you invented in the last hour: three confidence weights, a term cap of four, a length floor of three, a result limit of three, a stopword list assembled by intuition. Every one of them changes her answers, and right now you would tune them by trying a value, reading a few replies, and remembering the outcome wrong by Thursday. The next build gives a tuning session a structure, sweeping one parameter across several values and writing every trial to a file that still means something next month.