GLaDOS Vol 3 · Her Craft
ch 25 / 99
Chapter 25

The Lab Notebook

Did that change actually help?

Volume 3 has been one tuning decision after another: a normalization rule added in chapter 23, a mood threshold nudged in chapter 24, and behind all of it the standing questions every reader of this book eventually asks. Would the small Whisper model beat base on your microphone? Does a longer personality prompt cut the out-of-character replies, or just slow her down? Each of these is an experiment: a hypothesis, a few parameters you change, a number you measure. After a dozen of them, memory blurs. You re-run tests you already ran, you misremember which configuration won, and, worst of all, you "remember" results that flatter the change you were hoping would work.

The usual fixes rot. Terminal scrollback gets cleared. A scratch file keeps the numbers but loses which parameters produced them. And your head quietly rewrites the hypothesis after it has seen the result, which is the cardinal sin of experimentation: "I always suspected base was good enough" is a sentence memory manufactures on demand. Six months from now, "I think it was around ninety-something percent" defends no decision at all.

The fix is to make each experiment a first-class object with a life cycle. This chapter's rule: create the experiment, with its hypothesis, before you run anything; record the result afterward, in a separate step; and let a file on disk, never your memory, be the record. Chapter 2 gave her configuration a dataclass. This chapter gives your decisions one.

Stamp, save, record

▣ Build · stage 1 — a record that stamps its own id and time
# labs/experiments.py
import uuid
from dataclasses import dataclass
from datetime import datetime


@dataclass
class Experiment:
    name: str
    hypothesis: str
    parameters: dict
    results: dict | None = None
    status: str = "pending"
    id: str = ""
    created_at: str = ""

    def __post_init__(self) -> None:
        if not self.id:
            self.id = str(uuid.uuid4())[:8]
        if not self.created_at:
            self.created_at = datetime.now().isoformat()


if __name__ == "__main__":
    exp = Experiment(
        name="whisper model size",
        hypothesis="small beats base on word accuracy without pushing latency past 2s",
        parameters={"model": "small", "compute_type": "int8", "beam_size": 5},
    )
    print(f"id: {exp.id}  status: {exp.status}")
    print(f"created_at: {exp.created_at}")
$ uv run python labs/experiments.py
id: 7c1f22a9  status: pending
created_at: 2026-08-21T09:14:03.512847

Your id and timestamp will differ; that is the point of them. The interesting move is __post_init__. The dataclass machinery writes __init__ for you from the field annotations, assigns the plain defaults, and then hands control to __post_init__, where real code can run once per instance. A fresh UUID and the current time have to be computed there, at construction, because a field default is evaluated exactly once, when the class is defined. Also note results: dict | None = None: no result exists yet, and the record says so honestly instead of holding an empty dict that looks like a measurement of nothing.

▣ Build · stage 2 — one JSON file per experiment
import json
from dataclasses import asdict
from pathlib import Path

EXPERIMENTS_DIR = Path("glados/data/experiments")


def save_experiment(exp: Experiment) -> Path:
    EXPERIMENTS_DIR.mkdir(parents=True, exist_ok=True)
    path = EXPERIMENTS_DIR / f"{exp.id}.json"
    path.write_text(json.dumps(asdict(exp), indent=2))
    return path
$ cat glados/data/experiments/7c1f22a9.json
{
  "name": "whisper model size",
  "hypothesis": "small beats base on word accuracy without pushing latency past 2s",
  "parameters": {
    "model": "small",
    "compute_type": "int8",
    "beam_size": 5
  },
  "results": null,
  "status": "pending",
  "id": "7c1f22a9",
  "created_at": "2026-08-21T09:14:03.512847"
}

asdict() walks the dataclass recursively (nested dicts included) and returns plain Python that json.dumps understands, so keep the contents of parameters and results JSON-native: strings, numbers, booleans, lists, dicts. Naming the file by id gives every experiment a stable address; two experiments can never clobber each other, and the next function can find any record with nothing but its id. Python's None lands in the file as JSON null, exactly the "no result yet" marker a pending record wants.

▣ Build · stage 3 — record the result as a separate step
def record_result(exp_id: str, results: dict) -> Experiment:
    path = EXPERIMENTS_DIR / f"{exp_id}.json"
    data = json.loads(path.read_text())
    exp = Experiment(**data)
    exp.results = results
    exp.status = "complete"
    save_experiment(exp)
    return exp


def main() -> None:
    exp = Experiment(
        name="whisper model size",
        hypothesis="small beats base on word accuracy without pushing latency past 2s",
        parameters={"model": "small", "compute_type": "int8", "beam_size": 5},
    )
    save_experiment(exp)
    print(f"created  {exp.id}  (status: {exp.status})")

    done = record_result(exp.id, {"word_accuracy": 0.94, "latency_ms": 1840})
    print(f"recorded {done.id}  (status: {done.status})")
    print(f"results: {done.results}")


if __name__ == "__main__":
    main()
$ uv run python labs/experiments.py
created  4a70f0cd  (status: pending)
recorded 4a70f0cd  (status: complete)
results: {'word_accuracy': 0.94, 'latency_ms': 1840}

The accuracy and latency here are one bench run on one machine; when you run the real comparison, your numbers will differ. What holds everywhere is the signature: record_result takes an id, not the live object, because in real use the run happens later, sometimes in a different process, with only the id written on a sticky note. Loading with Experiment(**data) proves the round-trip works: the JSON keys match the field names, so the dict unpacks straight into the constructor, and __post_init__ leaves the loaded id and timestamp alone because they are already truthy. The function can change results and status and nothing else. That asymmetry is deliberate: no code path exists for editing a hypothesis after the fact.

◆ Note — why JSON files, and why they belong in git

Python could pickle the object in one line, but a pickle is opaque bytes tied to today's class definition. JSON stays readable in any editor six months from now, diffs cleanly, and survives the class growing new fields. Commit glados/data/experiments/ along with your code: each record is a few hundred bytes, and the day you need to defend a model choice, the audit trail is already sitting in your history.

Why this works: defaults run once, __post_init__ runs every time

The mechanism under this chapter generalizes to every dataclass you will ever write. When Python executes the class body, it evaluates each default value exactly once and stores it on the class; every future instance that omits the field receives that same stored object. For an immutable value like "pending", sharing is harmless: nobody can change a string in place. For anything that must be fresh per instance (a new UUID, the current time, an empty list) sharing would be a quiet disaster, so the rule is to compute those in __post_init__, which the generated __init__ calls once for every object constructed.

The sentinel pattern in stage 1 follows from that rule: default the field to something falsy (""), and have __post_init__ fill it only when it is empty. Fresh objects get stamped; objects rebuilt from disk keep their original id and timestamp. One method serves both construction paths, and it is the same guard that made Experiment(**data) in stage 3 safe. Dataclasses go one step further for mutable types: a shared dict default is dangerous enough that the decorator refuses to build the class at all.

⚠ Worked failure — the class that would not even load

Defaulting results to an empty dict feels natural, since it spares every reader a None check:

@dataclass
class Experiment:
    name: str
    hypothesis: str
    parameters: dict
    results: dict = {}        # looks harmless
    status: str = "pending"
$ uv run python labs/experiments.py
Traceback (most recent call last):
  File "labs/experiments.py", line 7, in <module>
    @dataclass
     ^^^^^^^^^
  File "/usr/lib/python3.11/dataclasses.py", line 1230, in dataclass
    return wrap(cls)
ValueError: mutable default <class 'dict'> for field results is not allowed: use default_factory

Read the traceback's top frame: the error points at the @dataclass line itself, not at any call site, and main() never ran. That location is the diagnosis. The failure happens while the decorator is building the class, because one shared dict as a default would mean every experiment silently writing into every other experiment's results, so dataclasses ban mutable defaults (dict, list, set) outright. The error even names both exits: field(default_factory=dict) when you truly want a fresh empty dict per instance, or, as this chapter chose, None, because "no result yet" and "empty result" deserve different spellings.

Checkpoint, hypothesis first

✓ Checkpoint — what you can now do
  • I can explain when a dataclass field default is evaluated, why that is safe for "pending", and why it forces the UUID and timestamp into __post_init__.
  • I can trace the two-step life cycle (create as pending, record to complete) and point to the line where the hypothesis becomes unchangeable.
  • I know why record_result takes an id instead of the object, and what the Experiment(**data) round-trip proves about the file.
  • I can predict what a fresh experiment's results field serializes to in JSON, and why None beats {} as its default.
  • Handed the mutable-default ValueError, I can read from the traceback that the class itself failed to build, and name both fixes.
⚡ Exercises — try first, then reveal
Exercise 1 — the index you will actually use. Write list_experiments(): scan glados/data/experiments/*.json and print one line per record with id, status, and name. Run it after the stage 3 demo.

Glob, load, format: for path in sorted(EXPERIMENTS_DIR.glob("*.json")), then data = json.loads(path.read_text()) and print f"{data['id']} {data['status']:<9} {data['name']}". The :<9 pads pending and complete to the same width so the names line up in a column. With the demo's record on disk you get one complete line; create a second experiment without recording a result and a pending line joins it.

Exercise 2 — the unfinished-business report. Add list_pending() returning the ids of every experiment still awaiting results. What does a long pending list tell you about your own habits?

Same scan with a filter: keep records where data["status"] == "pending", collect the ids, return the list. The report is a mirror. A few pending entries mean experiments queued honestly; a dozen mean you have been creating hypotheses and never running them, or running them and recording the outcome nowhere, which is the exact failure this chapter exists to end. Either way the file system now tells you, instead of you having to notice.

Exercise 3 — settle the model question. Create two experiments, base versus small, with the same hypothesis and your own machine's numbers from re-running chapter 5's transcription on the same WAV. Record both, then write compare_experiments(ids, metric) printing a table of model, status, and one metric.

Load each id, pull data["parameters"].get("model", "?") and (data["results"] or {}).get(metric, "-"); the or {} guard keeps a still-pending record from crashing the table. The measurements are yours, so the winner is too: on many machines small transcribes noticeably better and takes two to three times as long, and now the trade-off sits in two files you can cite instead of one impression you would have had to trust.

Decisions about her are now records you can query. The same move applies one level up: the project itself is a pile of goals (finish the prosody pass, order the servos, wire the permission system) living in your head, exactly where hypotheses used to live. Next chapter turns the roadmap into dataclasses too: goals with priorities, deadlines, and subtasks, and a planner that can answer "what should I work on next?" from data instead of mood.