Retrieval by Meaning

# glados/embeddings.py
import json
import urllib.request

EMBED_URL = "http://localhost:11434/api/embeddings"
EMBED_MODEL = "nomic-embed-text"

def embed(text: str, model: str = EMBED_MODEL) -> list[float]:
    payload = json.dumps({"model": model, "prompt": text}).encode()
    req = urllib.request.Request(
        EMBED_URL, data=payload, headers={"Content-Type": "application/json"}
    )
    with urllib.request.urlopen(req, timeout=30) as resp:
        return json.loads(resp.read())["embedding"]

if __name__ == "__main__":
    line = "The office printer is on 192.168.1.40."
    vec = embed(line)
    print(f"dimensions: {len(vec)}")
    print("first four:", [round(x, 4) for x in vec[:4]])
    print("same text, same vector:", vec == embed(line))

$ uv run python glados/embeddings.py
dimensions: 768
first four: [0.0271, -0.6194, -3.4283, -0.2117]
same text, same vector: True

The four numbers on your screen will differ from the four printed here, and that is fine: nothing in this chapter reads an individual coordinate. Two facts about that run do matter. The dimension is 768 for this model and it never varies, whether you embed one word or four hundred, because the model always folds its input into the same number of slots. And the second call returns a list identical to the first, so embed is a function of text and model with no clock and no randomness in it. That is what makes storing the result on disk sane.

The endpoint is /api/embeddings, not the /api/generate she has been talking to. Generation samples the next token repeatedly and streams words back. Embedding runs the input through the model once and hands you the internal numbers it computed, no sampling and no reply. Same server, same HTTP habits you already have, entirely different job.

def cosine(a: list[float], b: list[float]) -> float:
    if len(a) != len(b):
        raise ValueError(f"vector length mismatch: {len(a)} vs {len(b)}")
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = sum(x * x for x in a) ** 0.5
    norm_b = sum(y * y for y in b) ** 0.5
    if norm_a == 0.0 or norm_b == 0.0:
        return 0.0
    return dot / (norm_a * norm_b)

if __name__ == "__main__":
    a, b, c, d = [2, 1, 0], [4, 2, 0], [0, 0, 3], [1, 3, 0]
    print(f"a vs b (same direction, twice as long): {cosine(a, b):.3f}")
    print(f"a vs d (leaning apart):                 {cosine(a, d):.3f}")
    print(f"a vs c (right angles):                  {cosine(a, c):.3f}")

$ uv run python glados/embeddings.py
a vs b (same direction, twice as long): 1.000
a vs d (leaning apart):                 0.707
a vs c (right angles):                  0.000

Those three numbers are exact and you can check them on paper, which is the reason to test the function on toy vectors before pointing it at 768 dimensions you cannot picture. 0.707 is the cosine of 45 degrees. Nothing in the code knows or cares that the lists are three long; swap in two embeddings and the same twelve lines run 768 multiplications and return a number in the same range. No library, no index, no magic, and the guard on the first line is the one that saves an afternoon later in this chapter.

import sqlite3
from array import array
from pathlib import Path

DB_PATH = Path("glados/data/memory.db")

def add_vector_columns(conn: sqlite3.Connection) -> None:
    cols = {row[1] for row in conn.execute("PRAGMA table_info(knowledge)")}
    if "embedding" not in cols:
        conn.execute("ALTER TABLE knowledge ADD COLUMN embedding BLOB")
    if "embed_model" not in cols:
        conn.execute("ALTER TABLE knowledge ADD COLUMN embed_model TEXT")
    conn.commit()

def pack(vec: list[float]) -> bytes:
    return array("f", vec).tobytes()

def unpack(blob: bytes) -> list[float]:
    arr = array("f")
    arr.frombytes(blob)
    return list(arr)

def backfill(conn: sqlite3.Connection, model: str = EMBED_MODEL) -> int:
    rows = conn.execute(
        "SELECT id, content FROM knowledge"
        " WHERE embedding IS NULL OR embed_model IS NULL OR embed_model != ?",
        (model,),
    ).fetchall()
    for row_id, content in rows:
        conn.execute(
            "UPDATE knowledge SET embedding = ?, embed_model = ? WHERE id = ?",
            (pack(embed(content, model=model)), model, row_id),
        )
    conn.commit()
    return len(rows)

if __name__ == "__main__":
    conn = sqlite3.connect(DB_PATH)
    add_vector_columns(conn)
    n = backfill(conn)
    blob = conn.execute(
        "SELECT embedding FROM knowledge WHERE embedding IS NOT NULL LIMIT 1"
    ).fetchone()[0]
    print(f"embedded {n} row(s) with {EMBED_MODEL}")
    print(f"bytes per vector: {len(blob)}")
    print(f"round trip similarity: {cosine(unpack(blob), unpack(blob)):.3f}")

$ uv run python glados/embeddings.py
embedded 3 row(s) with nomic-embed-text
bytes per vector: 3072
round trip similarity: 1.000

768 numbers at four bytes each is 3,072 bytes per row, exactly, and that number is the honest price of this chapter. A thousand stored facts cost about three megabytes of vectors on top of the sentences themselves. array("f", vec) writes single-precision floats, half the width of Python's own, and the round-trip check proves the truncation costs nothing that matters: the recovered vector still scores 1.000 against itself at three decimals.

Two design decisions are hiding in backfill. It selects only rows that lack a vector or carry one from a different model, so running it on every startup costs one query when nothing changed and re-embeds exactly the stale rows when you switch models. And it stores embed_model beside each vector, because a list of 768 floats carries no record of where it came from. That column looks like bookkeeping right up until stage 5, where it becomes the difference between an answer and a crash.

The other cost is time. Every fact costs one model call before it can ever be found, so a knowledge base of two thousand rows means two thousand calls the first time you run this. On a laptop that is a couple of minutes, on the Jetson a little more. After that it is one call per new fact and one per question, which is why the vectors live in the database instead of being recomputed at query time.

def semantic_search(conn: sqlite3.Connection, question: str, limit: int = 3,
                    floor: float = 0.0, model: str = EMBED_MODEL) -> list[dict]:
    q = embed(question, model=model)
    hits: list[dict] = []
    for content, confidence, blob in conn.execute(
        "SELECT content, confidence, embedding FROM knowledge"
        " WHERE embedding IS NOT NULL"
    ):
        score = cosine(q, unpack(blob))
        if score >= floor:
            hits.append({"content": content, "confidence": confidence,
                         "score": score})
    hits.sort(key=lambda h: (h["score"], h["confidence"]), reverse=True)
    return hits[:limit]

if __name__ == "__main__":
    question = "what is the thing in the hall that eats paper"
    for hit in semantic_search(conn, question, limit=5):
        print(f"  {hit['score']:.2f}  {hit['content']}")

$ uv run python glados/embeddings.py   # scores will differ on your machine
  0.71  The printer jams on thick paper.
  0.63  The office printer is on 192.168.1.40.
  0.29  The morning routine starts the coffee maker at 07:00.

There it is: the question chapter 37 could not answer, answered, with a top hit that shares exactly one word with the question and that word is "paper". The model has read enough English to place "printer" and "eats paper" near each other in its 768-dimensional space, and the arithmetic in stage 2 reports the result as 0.71.

Look at the third line before you celebrate. The coffee maker has nothing to do with the hall, and it still scored 0.29, because cosine similarity always returns a number for every pair of vectors. A keyword search that finds nothing returns an empty list and tells you so. A vector search that finds nothing returns your whole database, sorted, with the least irrelevant row at the top. That is what floor is for, and picking its value is a measurement you make on your own data, not a constant to copy.

# glados/knowledge.py -- the retriever swaps; the boundary does not
from glados.embeddings import semantic_search

def knowledge_context(conn: sqlite3.Connection, line: str,
                      limit: int = 3, floor: float = 0.45) -> str:
    try:
        hits = semantic_search(conn, line, limit=limit, floor=floor)
    except OSError:                       # Ollama down: fall back to spelling
        hits = search(conn, keywords(line), limit=limit)
    if not hits:
        return ""
    lines = ["Relevant knowledge (confidence in brackets, 1.0 means the user said so):"]
    for hit in hits:
        lines.append(f"  - [{hit['confidence']:.1f}] {hit['content']}")
    return "\n".join(lines)

if __name__ == "__main__":
    print(build_prompt("You are GLaDOS. Be sardonic and brief.", conn,
                       "what is the thing in the hall that eats paper"))

$ uv run python glados/knowledge.py
You are GLaDOS. Be sardonic and brief.

Memory context:
Known facts about the user:
  - user_name: Kaleb

Relevant knowledge (confidence in brackets, 1.0 means the user said so):
  - [0.6] The printer jams on thick paper.
  - [1.0] The office printer is on 192.168.1.40.

$ uv run python labs/wire_core.py   # her reply; yours will differ
GLaDOS: The printer. It is in the hall, it eats paper, and it is at 192.168.1.40.
        You are welcome.

build_prompt did not change. Neither did the persona engine, the memory loader, or the voice loop. One function body swapped its candidate generator and the whole assistant got better at being asked things, which is the payoff chapter 14 predicted when it drew a single boundary between memory and mouth and refused to let anything downstream know where the lines came from.

The except OSError is not decoration. Embedding is the first retrieval step in this book that needs a service to be up, so a stopped Ollama would otherwise turn every question into a traceback. Falling back to the keyword path degrades her answers without silencing her, and the old code earns its keep as the thing that still works when the new code cannot run.