GLaDOS Vol 7 · The Body
ch 74 / 99
Chapter 74

Full System Test

Every subsystem has been tested alone, and only alone

The eye behaved on an afternoon when nothing else was moving. The ring lit up while the arm sat unpowered. The microphones were measured in a quiet room, the speaker checked before the pack was even in the shell, and the arm homed on a bench supply that never sagged. Eleven chapters of hardware, eleven separate good days.

A machine is not the sum of its good days. Servos pull current in bursts that reach the light ring through a shared pack. A microphone that heard you at arm's length may be sitting next to a speaker cone now. Calibration numbers measured last week describe a bracket that has been unbolted and refitted twice since. None of that is visible from any single subsystem's own test, because each of those tests carefully removed the others.

So before the software stack goes back on top of this, one run has to answer one question with evidence: does every part of her work, right now, at the same time? The obvious way to write that run is six calls in a row, and it fails on contact with hardware. The first call raises, the program dies, and you have learned the state of exactly one subsystem out of six. Six faults become six round trips through a machine you have to power down between attempts.

Chapter 45 solved half of this for software tests and chapter 58 for the seams between them: wrap each check so a raised exception becomes a recorded result. This chapter puts that runner in front of real hardware and adds the two things hardware needs. Every row carries the number it measured next to the limit it was measured against, the way chapter 42 insisted. And the run ends in a word, the way chapter 59's acceptance report does. One rule holds it together: each check owns its own failure and returns a row instead of raising, every row starts as not-yet-run and can only be moved to PASS by a probe that measured something, and the verdict is GO only when all six rows say PASS.

◆ Note — ninety seconds, and why a person is standing there

Ninety seconds is a budget, chosen so the test is short enough to run before every session and long enough to contain a three second capture, a real arm motion and ten seconds of loaded rail sampling. Three of the six checks end by asking you a question, because a servo reports nothing back and a pixel has no way to tell you what colour it is. Those three are the reason this is a test you watch rather than a job you schedule. Every number quoted below came off one assembled body on one evening. Yours will differ, and the thresholds need review for your actual components before you use them. GO records this supervised functional check only. It does not certify fault-current protection, unseen voltage transients or an independent motion-power stop; establish those separately as chapter 70 requires before energizing the body.

The ninety-second budget, and which checks depend on which Six bars laid end to end across a ninety second axis: eye twelve seconds, light ring ten, microphones ten, speaker twelve, arm twenty four, power twenty two. Two of the bars are marked as dependent. An arrow runs from the microphones bar to the speaker bar, because the speaker is checked by playing a tone and recording it on the array. Two more arrows run from the light ring bar and the arm bar to the power bar, because the power check loads the rails with the ring at full and the arm in motion. SIX CHECKS · 90 SECONDS · TWO OF THEM WAIT THEIR TURN eye 12 s light ring 10 s microphones 10 s speaker 12 s arm 24 s power 22 s an arrow means: this check uses that subsystem, so it runs after it 0 30 60 90
Figure 74.1 — The order is not alphabetical and not arbitrary. A check may only lean on a subsystem an earlier check already cleared, so the speaker can be measured through the microphones and the rails can be loaded by the ring and the arm.

One row per subsystem, and a default of not yet

▣ Build · stage 1 — the row, and the runner that fills it in
# labs/hardware_check.py
import time
from dataclasses import dataclass
from typing import Callable

PASS, FAIL, SKIP = "PASS", "FAIL", "SKIP"


@dataclass
class Check:
    """One subsystem's row in the report. SKIP until a probe reports otherwise."""
    name: str
    budget_s: float
    status: str = SKIP
    detail: str = ""
    elapsed_s: float = 0.0


def show(check: Check) -> Check:
    print(f"  [{check.status}] {check.name:<14}{check.elapsed_s:5.1f}s  {check.detail}")
    return check


def run_check(check: Check, probe: Callable[[], tuple[bool, str]]) -> Check:
    """Run one probe. Whatever it raises becomes this row's detail, not a traceback."""
    start = time.perf_counter()
    try:
        ok, detail = probe()
        check.status, check.detail = (PASS if ok else FAIL), detail
    except Exception as exc:
        check.status, check.detail = FAIL, f"{type(exc).__name__}: {exc}"
    check.elapsed_s = time.perf_counter() - start
    return show(check)


def skip(check: Check, why: str) -> Check:
    check.status, check.detail = SKIP, why
    return show(check)


if __name__ == "__main__":
    def eye() -> tuple[bool, str]:
        time.sleep(0.2)
        return True, "eye_pan 1250-1700 us, eye_tilt 1350-1650 us"

    def microphones() -> tuple[bool, str]:
        time.sleep(3.1)
        return False, "rms 0.00002, gate 0.00500"

    def power() -> tuple[bool, str]:
        raise FileNotFoundError(2, "No such file or directory", "/dev/spidev4.0")

    for check, probe in ((Check("eye", 12.0), eye),
                         (Check("microphones", 10.0), microphones),
                         (Check("power", 22.0), power)):
        run_check(check, probe)
    print("the runner is still here")
$ uv run python -m labs.hardware_check
  [PASS] eye             0.2s  eye_pan 1250-1700 us, eye_tilt 1350-1650 us
  [FAIL] microphones     3.1s  rms 0.00002, gate 0.00500
  [FAIL] power           0.0s  FileNotFoundError: [Errno 2] No such file or directory: '/dev/spidev4.0'
the runner is still here

Three probes and three different endings, and the runner treats all of them as data. The middle one is the interesting case. It did not raise: it ran to completion, took its three seconds, and reported that what it heard was two hundred and fifty times quieter than the gate. A crash and a measurement that missed are different events for the person reading the report, and identical for the code that has to keep going.

Two smaller decisions carry weight later. The status field starts at SKIP, which is chapter 58's habit of defaulting a result to not-passed and making something prove otherwise. And run_check stamps elapsed_s after the except as well as after the happy path, so a check that dies at 19 seconds is distinguishable from one that dies instantly. The first is a subsystem that half worked. The second is usually a missing file.

▣ Build · stage 2 — count what passed, never what failed
# labs/hardware_check.py — continued
BUDGET_S = 90.0


def tally(checks: list[Check]) -> dict[str, int]:
    return {status: sum(1 for c in checks if c.status == status)
            for status in (PASS, FAIL, SKIP)}


def verdict(checks: list[Check]) -> str:
    """GO needs every row to have passed. A check nobody ran is not a check that passed."""
    return "GO" if all(c.status == PASS for c in checks) else "NO-GO"


def summarise(checks: list[Check]) -> str:
    counts = tally(checks)
    elapsed = sum(c.elapsed_s for c in checks)
    unresolved = [c.name for c in checks if c.status != PASS]
    call = verdict(checks)
    print(f"\n{counts[PASS]} passed, {counts[FAIL]} failed, {counts[SKIP]} not run"
          f"   {elapsed:.1f}s of a {BUDGET_S:.0f}s budget")
    print(f"{call} - " + ("every subsystem answered for itself" if not unresolved
                          else "unresolved: " + ", ".join(unresolved)))
    return call

verdict asks whether everything passed. Written the other way round, as a search for failures, it reads better and it is wrong, for a reason the worked failure at the end of this chapter pays for in a committed report. Three states exist, and only two of them are opinions about the hardware. SKIP is an opinion about the test.

▣ Build · stage 3 — a missing file is one row, not the end of the run
# labs/signoff_config.py
import json
from pathlib import Path
from typing import Any

HW_PATH = Path("configs/hardware.json")
CAL_PATH = Path("configs/calibration.json")


def load(path: Path) -> dict:
    """An absent or unreadable config is an empty dict."""
    try:
        return json.loads(path.read_text())
    except FileNotFoundError:
        return {}
    except (OSError, json.JSONDecodeError) as exc:
        print(f"  {path} cannot be read as JSON and will be treated as empty: {exc}")
        return {}


def need(config: dict, path: Path, *keys: str) -> Any:
    """Walk into a config, or raise naming both the key and the file it was missing from."""
    node: Any = config
    for depth, key in enumerate(keys, start=1):
        if not isinstance(node, dict) or key not in node:
            raise LookupError(f"{'.'.join(keys[:depth])} missing from {path}")
        node = node[key]
    return node


if __name__ == "__main__":
    hw, cal = load(HW_PATH), load(CAL_PATH)
    print(f"{len(hw)} sections in {HW_PATH}, {len(cal)} servos in {CAL_PATH}")
    for keys in (("audio", "mic_name_substr"), ("arm", "channels"), ("audio", "speaker")):
        try:
            print(f"  {'.'.join(keys):<24} -> {need(hw, HW_PATH, *keys)}")
        except LookupError as exc:
            print(f"  {'.'.join(keys):<24} -> {exc}")
$ uv run python -m labs.signoff_config
6 sections in configs/hardware.json, 3 servos in configs/calibration.json
  audio.mic_name_substr    -> ReSpeaker
  arm.channels             -> {'shoulder': 0, 'elbow': 1, 'wrist': 2, 'gripper': 3}
  audio.speaker            -> audio.speaker missing from configs/hardware.json
$ mv configs configs.away && uv run python -m labs.signoff_config
0 sections in configs/hardware.json, 0 servos in configs/calibration.json
  audio.mic_name_substr    -> audio missing from configs/hardware.json
  arm.channels             -> arm missing from configs/hardware.json
  audio.speaker            -> audio missing from configs/hardware.json

The two halves degrade in opposite directions on purpose. load never raises, because a diagnostic tool that cannot start on a machine with no config is useless on precisely the machine you most need to diagnose. need always raises, because the alternative is a default, and defaults here are how hardware gets broken. Chapter 73 measured this eye at 1250 to 1700 microseconds; the library's fallback of 1000 to 2000 would drive the pan servo hard into a printed yoke and hold it there. A missing calibration file must stop the eye check, and it must stop nothing else.

Notice what the error string carries: the key path and the file. That message ends up in a JSON report someone reads on another machine in six months, so "missing from configs/hardware.json" is the difference between a fix and an investigation. The keys[:depth] slice is why the second run says audio instead of audio.mic_name_substr. The whole section is gone, not just the field, and the message says which.

The four checks that go first

▣ Build · stage 4 — the two that make light and motion
# labs/signoff_checks.py
import time

from labs.calibrated import axis_limit_deg, calibrated_eye
from labs.led_ring import EyeRing
from labs.mood_state import MoodState
from labs.signoff_config import CAL_PATH, HW_PATH, need

EYE_DWELL_S = 1.2
RING_DWELL_S = 1.5
RING_MOODS = ("hostile", "satisfied", "neutral")   # red, green, blue: all three dice


def watched(question: str) -> bool:
    """The part of a hardware test that no sensor on this machine can perform."""
    return input(f"       {question} [y/N]: ").strip().lower() == "y"


def probe_eye(hw: dict, cal: dict) -> tuple[bool, str]:
    """Both axes out to their measured stops and back to centre, slowly enough to follow."""
    entries = {name: need(cal, CAL_PATH, name) for name in ("eye_pan", "eye_tilt")}
    for name, entry in entries.items():
        if entry.get("travel_deg") is None:
            return False, f"{name} has no measured travel; calibrate before testing"
    pan_limit = axis_limit_deg(entries["eye_pan"])
    tilt_limit = axis_limit_deg(entries["eye_tilt"])
    eye = calibrated_eye()
    for pan, tilt in ((-pan_limit, 0.0), (pan_limit, 0.0), (0.0, -tilt_limit),
                      (0.0, tilt_limit), (0.0, 0.0)):
        eye.look_at(pan, tilt)
        time.sleep(EYE_DWELL_S)
    detail = f"pan +-{pan_limit:.1f} deg, tilt +-{tilt_limit:.1f} deg, four stops and centre"
    return watched("did the eye reach all four stops and return to centre?"), detail


def probe_ring(hw: dict, cal: dict) -> tuple[bool, str]:
    """Twenty four pixels through three moods and then dark."""
    spi0 = need(hw, HW_PATH, "buses", "spi0")
    if not spi0.get("exclusive"):
        return False, "spi0 is not declared exclusive; the ring cannot share that line"
    ring = EyeRing()
    for mood in RING_MOODS:
        ring.show_mood(MoodState(mood, 1.0))
        time.sleep(RING_DWELL_S)
    ring.off()
    detail = f"24 pixels through {', '.join(RING_MOODS)}, dark"
    return watched("did all 24 pixels light red, then green, then blue, then go dark?"), detail

Both probes take the same two config dictionaries whether they need them or not, so the runner can call any of them the same way, and both return a pair: a boolean and the sentence that will be in the report. Neither prints a verdict of its own, because run_check owns that line.

The eye probe refuses in two distinct ways before it ever moves. A missing calibration entry raises out of need and lands in the row as a LookupError. An entry that exists but carries a null travel returns False with a sentence about it, because that entry is real: chapter 73 writes exactly that record for a servo you declined to calibrate. The first is a broken setup, the second is unfinished work, and the report should not call them the same thing.

The three ring moods are chosen for their hex values, not their meanings. Hostile is #ff4444, satisfied is #39ff14 and neutral is #4a9eff, so the sequence drives every red, green and blue die in all twenty four packages. A pixel with a dead green channel looks perfectly healthy showing hostile and gives itself away one and a half seconds later.

▣ Build · stage 5 — the microphones, then the speaker through them
# labs/signoff_checks.py — continued
import numpy as np
import sounddevice as sd

from labs.audio_out import SAMPLE_RATE, apply_fades, make_sine_tone, scale_volume
from labs.mic_distance import SILENCE_GATE
from labs.mic_setup import SAMPLE_RATE as MIC_RATE, find_input_device, record, signal_report

LOOPBACK_RMS = 0.020       # a tone at 0.70 across a desk; a dead amplifier is nowhere near it
TONE_SECONDS = 2.0


def input_index(hw: dict) -> int:
    """Resolve the array quietly: this program's console output is the report."""
    name = need(hw, HW_PATH, "audio", "mic_name_substr")
    index = find_input_device(sd.query_devices(), name)
    if index is None:
        raise LookupError(f"no input device whose name contains {name!r}")
    return index


def output_index(hw: dict) -> int:
    name = need(hw, HW_PATH, "audio", "output_name_substr")
    for index, dev in enumerate(sd.query_devices()):
        if dev["max_output_channels"] > 0 and name.lower() in dev["name"].lower():
            return index
    raise LookupError(f"no output device whose name contains {name!r}")


def probe_microphones(hw: dict, cal: dict) -> tuple[bool, str]:
    """Three seconds of the room, judged by the gate the wake word already uses."""
    index = input_index(hw)
    print("       say something for three seconds")
    peak, rms = signal_report(record(index, seconds=3))
    detail = f"index {index}, peak {peak:.4f}, rms {rms:.4f}, gate {SILENCE_GATE:.4f}"
    return rms >= SILENCE_GATE, detail


def probe_speaker(hw: dict, cal: dict) -> tuple[bool, str]:
    """Play a tone on the amplifier and listen for it on the array that just passed."""
    heard: list[np.ndarray] = []

    def keep(data: np.ndarray, frames: int, timing: object, status: object) -> None:
        heard.append(data.copy())          # the driver reuses this buffer on the next call

    tone = scale_volume(apply_fades(make_sine_tone(duration=TONE_SECONDS)), 0.7)
    with sd.InputStream(device=input_index(hw), samplerate=MIC_RATE, channels=1,
                        dtype="float32", callback=keep):
        sd.play(tone, samplerate=SAMPLE_RATE, device=output_index(hw), blocking=True)
        time.sleep(0.2)
    _, rms = signal_report(np.concatenate(heard) if heard else np.zeros(1, dtype=np.float32))
    detail = f"440 Hz at 0.70, array heard rms {rms:.4f}, floor {LOOPBACK_RMS:.4f}"
    return rms >= LOOPBACK_RMS, detail

The microphone gate is imported, not invented. SILENCE_GATE is the number chapter 67 calibrated for deciding whether anyone is speaking, so a machine that passes this check passes it at the level the pipeline will actually demand. A test with its own private threshold can be green while the system it is testing is deaf.

The speaker check is the first place two subsystems are used at once, and it is legal here for one reason: the microphones ran first and passed. If the array had failed, this row would be measuring an unknown against an unknown, and the runner would not have started it. What the dependency buys is a speaker check with a number in it. Every other way of testing an amplifier ends in "did you hear that", and the room answers this one.

sd.play and sd.rec both write into one module-level stream slot inside sounddevice, so calling them back to back stops the recording the instant playback starts. An explicit InputStream is a separate object that the playback call cannot touch, and the with block closes it even if the tone fails to reach the amplifier. Inside the callback, data.copy() is not caution: the driver hands the same buffer back on the next call, so keeping a reference gives you a list whose entries all quietly become the last block recorded.

The arm, the rails under it, and one command

▣ Build · stage 6 — a motion you can watch, and a load you can measure
# labs/signoff_checks.py — continued
import threading

import spidev

from labs.arm_control import RoboticArm
from labs.arm_driver import ChannelDriver
from labs.power_monitor import SAMPLE_HZ, SPI_BUS, SPI_DEVICE, SPI_HZ, read_channel
from labs.power_sense import SENSES, adc_to_volts, capacity_percent
from labs.wire_drop import SERVO_MIN_V

ARM_DWELL_S = 2.0
MIN_CAPACITY_PCT = 20
REST_S, LOAD_S = 3.0, 10.0


def probe_arm(hw: dict, cal: dict) -> tuple[bool, str]:
    """Home, reach, grab, reach, release, home, relax, with time to see each one."""
    channels = need(hw, HW_PATH, "arm", "channels")
    arm = RoboticArm(ChannelDriver())
    arm.home()
    for move in (lambda: arm.reach(45.0, 90.0, 90.0), arm.grab,
                 lambda: arm.reach(120.0, 60.0, 90.0), arm.release, arm.home):
        time.sleep(ARM_DWELL_S)
        move()
    arm.relax()
    detail = f"{len(channels)} joints: home, reach, grab, reach, release, home, relax"
    return watched("did all four joints move, and is the arm folded and limp now?"), detail


def sample(spi: spidev.SpiDev, seconds: float) -> dict[str, list[int]]:
    """Every sense channel, SAMPLE_HZ times a second, for this long."""
    raws: dict[str, list[int]] = {name: [] for name in SENSES}
    end = time.monotonic() + seconds
    while time.monotonic() < end:
        for name, sense in SENSES.items():
            raws[name].append(read_channel(spi, sense.channel))
        time.sleep(1 / SAMPLE_HZ)
    return raws


def floor_volts(raws: dict[str, list[int]], rail: str) -> float:
    """The worst sample in a window: what the rail fell to, not what it averaged."""
    return adc_to_volts(min(raws[rail]), SENSES[rail])


def top_volts(raws: dict[str, list[int]], rail: str) -> float:
    return adc_to_volts(max(raws[rail]), SENSES[rail])
# labs/signoff_checks.py — continued
def probe_power(hw: dict, cal: dict) -> tuple[bool, str]:
    """Both rails at rest, then both rails with the ring full and the arm working."""
    ring, arm = EyeRing(), RoboticArm(ChannelDriver())   # neither holds anything yet
    spi = spidev.SpiDev()
    spi.open(SPI_BUS, SPI_DEVICE)
    spi.max_speed_hz = SPI_HZ
    try:
        rest = sample(spi, REST_S)
        ring.show_mood(MoodState("neutral", 1.0))
        stir = threading.Thread(target=arm.reach, args=(120.0, 60.0, 90.0), daemon=True)
        stir.start()
        loaded = sample(spi, LOAD_S)
        stir.join()
    finally:
        arm.relax()
        ring.off()
        spi.close()

    pack_rest, pack_low = top_volts(rest, "pack"), floor_volts(loaded, "pack")
    rail_rest, rail_low = top_volts(rest, "motion"), floor_volts(loaded, "motion")
    percent = capacity_percent(pack_rest)
    detail = (f"pack {pack_rest:.2f}/{pack_low:.2f} V {percent}%, "
              f"motion {rail_rest:.2f}/{rail_low:.2f} V, floor {SERVO_MIN_V:.2f} V")
    return rail_low >= SERVO_MIN_V and percent >= MIN_CAPACITY_PCT, detail

The power check is the only one that tests the machine as a machine. A voltage read while nothing is happening says almost nothing: the rails are supposed to be fine at rest. What matters is the ten seconds when twenty four pixels are at full and three joints are accelerating together, and what matters inside those ten seconds is the worst observed sample, not the average. Fifty samples a second nominally gives about five hundred samples in ten seconds; reads and scheduling add delay. The roughly twenty millisecond gaps can miss a dip lasting only a few milliseconds, even over a longer window. A passing row means none of the recorded samples crossed the limit. Chapter 70's warning still applies: capture short transients with an appropriately configured oscilloscope or faster acquisition system before making claims about them.

The arm runs on a thread so the sampling loop keeps its rhythm while the joints move. The finally is not decoration. If the ADC stops answering halfway through the load window, the exception propagates to the runner and becomes a FAIL row, and without that block it would leave the ring at full brightness and four servos holding torque against a pose nobody chose. A check that fails must still put the hardware down.

▣ Build · stage 7 — the run, the gate between checks, and the file it leaves

Keep four files: hardware_check.py owns result states, signoff_config.py owns configuration helpers, signoff_checks.py owns probes, and the complete signoff.py below owns orchestration. The probes and runner both import the helper module, which imports neither of them. That removes the circular import in either module entry path. Import every status used by the report, including FAIL and SKIP. Probe imports stay inside main, so python -m unittest labs.test_signoff can exercise report construction without importing device libraries or running a probe.

# labs/signoff.py — complete file; helpers stay in signoff_config.py
import argparse
import json
from datetime import datetime
from functools import partial
from pathlib import Path

from labs.hardware_check import (BUDGET_S, FAIL, PASS, SKIP, Check, run_check, skip,
                                 summarise, tally, verdict)
from labs.signoff_config import CAL_PATH, HW_PATH, load

REPORT_PATH = Path("glados/data/hardware_signoff.json")

BUDGETS = {"eye": 12.0, "light ring": 10.0, "microphones": 10.0,
           "speaker": 12.0, "arm": 24.0, "power": 22.0}
NEEDS = {"speaker": ("microphones",), "power": ("light ring", "arm")}


def last_calibration(cal: dict) -> str:
    dates = [entry["measured_at"] for entry in cal.values() if "measured_at" in entry]
    return max(dates) if dates else "never"


def write_report(checks: list[Check], hw: dict, cal: dict, started: datetime) -> None:
    counts = tally(checks)
    REPORT_PATH.parent.mkdir(parents=True, exist_ok=True)
    REPORT_PATH.write_text(json.dumps({
        "started_at": started.isoformat(timespec="seconds"),
        "body_revision": hw.get("revision", "unknown"),
        "calibrated_at": last_calibration(cal),
        "verdict": verdict(checks),
        "budget_s": BUDGET_S,
        "elapsed_s": round(sum(c.elapsed_s for c in checks), 1),
        "passed": counts[PASS], "failed": counts[FAIL], "not_run": counts[SKIP],
        "checks": [{"name": c.name, "status": c.status, "detail": c.detail,
                    "budget_s": c.budget_s, "elapsed_s": round(c.elapsed_s, 1)}
                   for c in checks],
    }, indent=2) + "\n")


def main() -> None:
    from labs.signoff_checks import (probe_arm, probe_eye, probe_microphones,
                                     probe_power, probe_ring, probe_speaker)

    probes = {"eye": probe_eye, "light ring": probe_ring, "microphones": probe_microphones,
              "speaker": probe_speaker, "arm": probe_arm, "power": probe_power}
    parser = argparse.ArgumentParser(description="GLaDOS full system test")
    parser.add_argument("--skip", nargs="*", default=[], choices=list(probes),
                        help="subsystems that are off the machine right now")
    args = parser.parse_args()

    hw, cal = load(HW_PATH), load(CAL_PATH)
    checks = [Check(name, BUDGETS[name]) for name in probes]
    by_name = {c.name: c for c in checks}
    started = datetime.now().astimezone()
    print(f"GLaDOS body rev {hw.get('revision', 'unknown')}, full system test, "
          f"{len(checks)} checks, {BUDGET_S:.0f}s budget")

    for check in checks:
        blocked = [n for n in NEEDS.get(check.name, ()) if by_name[n].status != PASS]
        if check.name in args.skip:
            skip(check, f"not run: --skip {check.name}")
        elif blocked:
            skip(check, f"not run: {' and '.join(blocked)} did not pass")
        else:
            run_check(check, partial(probes[check.name], hw, cal))

    summarise(checks)
    write_report(checks, hw, cal, started)
    print(f"wrote {REPORT_PATH}")


if __name__ == "__main__":
    main()

The following bench record is retained from the earlier build. The import and report corrections have offline checks, but no new hardware run accompanies this revision; these readings are not evidence that the revised program passed on your body. Run a new supervised sign-off after validating your hardware configuration.

$ uv run python -m labs.signoff   # measured on the bench — yours will vary
GLaDOS body rev 1.0.0, full system test, 6 checks, 90s budget
       did the eye reach all four stops and return to centre? [y/N]: y
  [PASS] eye            11.4s  pan +-25.5 deg, tilt +-17.0 deg, four stops and centre
       did all 24 pixels light red, then green, then blue, then go dark? [y/N]: y
  [PASS] light ring      9.6s  24 pixels through hostile, satisfied, neutral, dark
       say something for three seconds
  [PASS] microphones     6.7s  index 1, peak 0.3120, rms 0.0912, gate 0.0050
  [PASS] speaker         8.8s  440 Hz at 0.70, array heard rms 0.1642, floor 0.0200
       did all four joints move, and is the arm folded and limp now? [y/N]: y
  [PASS] arm            19.6s  4 joints: home, reach, grab, reach, release, home, relax
  [PASS] power          17.3s  pack 7.92/7.68 V 72%, motion 5.02/4.86 V, floor 4.60 V

6 passed, 0 failed, 0 not run   73.4s of a 90s budget
GO - every subsystem answered for itself
wrote glados/data/hardware_signoff.json

That is the whole body reporting for itself in seventy three seconds. The motion rail fell to 4.86 volts with the ring at full and three joints accelerating, which clears the 4.60 volt floor chapter 70 set for these servos by 0.26 volts, on a pack still holding seventy two percent. The array heard the tone at an RMS of 0.1642 against a floor of 0.0200, so the amplifier, the speaker, the microphones and the two metres of air between them all work in the same room at the same time.

$ mv configs configs.away && uv run python -m labs.signoff
GLaDOS body rev unknown, full system test, 6 checks, 90s budget
  [FAIL] eye             0.0s  LookupError: eye_pan missing from configs/calibration.json
  [FAIL] light ring      0.0s  LookupError: buses missing from configs/hardware.json
  [FAIL] microphones     0.0s  LookupError: audio missing from configs/hardware.json
  [SKIP] speaker         0.0s  not run: microphones did not pass
  [FAIL] arm             0.0s  LookupError: arm missing from configs/hardware.json
  [SKIP] power           0.0s  not run: light ring and arm did not pass

0 passed, 4 failed, 2 not run   0.1s of a 90s budget
NO-GO - unresolved: eye, light ring, microphones, speaker, arm, power
wrote glados/data/hardware_signoff.json

Same command, same six rows, one tenth of a second. Four rows name the section of config they went looking for, and two say plainly that nobody attempted them, because their dependencies had not passed. No traceback, no partial run, and a report on disk either way. A NO-GO report is the more useful of the two: it is the document you open tomorrow to remember what you were in the middle of.

// glados/data/hardware_signoff.json — the good run
{
  "started_at": "2026-08-22T19:41:07-05:00",
  "body_revision": "1.0.0",
  "calibrated_at": "2026-08-22T09:31:55",
  "verdict": "GO",
  "budget_s": 90.0,
  "elapsed_s": 73.4,
  "passed": 6,
  "failed": 0,
  "not_run": 0,
  "checks": [
    {"name": "eye", "status": "PASS",
     "detail": "pan +-25.5 deg, tilt +-17.0 deg, four stops and centre",
     "budget_s": 12.0, "elapsed_s": 11.4},
    {"name": "power", "status": "PASS",
     "detail": "pack 7.92/7.68 V 72%, motion 5.02/4.86 V, floor 4.60 V",
     "budget_s": 22.0, "elapsed_s": 17.3}
  ]
}

Two fields at the top do most of the long-term work. body_revision comes from the pin map, so the report says which wiring arrangement it describes, and a report taken against rev 1.0.0 makes no claim about the rev 1.1.0 you build next month. calibrated_at is the newest measurement date in the calibration file, because a sign-off is only as true as the numbers the checks ran against. A GO from a body last calibrated three brackets ago is a GO about a machine that no longer exists. Commit this file. The diff between two of them is the cheapest regression test this build will ever have.

Why this works: three states, and an order that earns something

The whole design rests on refusing to let an exception mean anything. A raised exception is a control flow event: it unwinds until something catches it, and if nothing does it takes the process with it. That behaviour is correct for a program doing work and exactly backwards for a program whose only job is to find out what is broken. Catching at the check boundary converts the event into a row, one line down from where it happened, with the message intact. The runner above it never learns that anything went wrong. It counts.

The third state is what makes the count honest. With two states, every question has to be answered yes or no, so a check that could not run has to lie in one direction or the other. Call it a failure and you cannot tell a broken subsystem from an absent one, and people start ignoring red rows. Call it a pass and the report claims something nobody measured. SKIP lets the run say "I did not look", which is a true statement about the test, and verdict then refuses to say GO because GO is a statement about the hardware.

The ordering rule falls out of the same idea. Six checks that assume nothing about each other must each stand alone, and standing alone is expensive: the speaker can only be tested by asking a person, and the rails can only be read at rest. Order them so each check may lean on subsystems already marked PASS, and two of the six get better. When the dependency is not satisfied the check does not guess and does not fail: it skips, and the skip is visible in the verdict. That is how integration suites are ordered in general, and it is chapter 58's discipline applied to seams made of wire.

⚠ Worked failure — the GO that was printed next to four passes

The arm is off the machine for an evening: a shoulder bracket is back on the printer, so the four joints are in a box on the desk. Testing the head alone is reasonable, that is what --skip is for, and the run looks fine. Here is verdict as it was first written, which is how most people write it:

# labs/hardware_check.py — the version that shipped a false sign-off
def verdict(checks: list[Check]) -> str:
    """BUG: 'no failures' is not 'all passed' once a third status exists."""
    return "NO-GO" if any(c.status == FAIL for c in checks) else "GO"
$ uv run python -m labs.signoff --skip arm
GLaDOS body rev 1.0.0, full system test, 6 checks, 90s budget
  [PASS] eye            11.2s  pan +-25.5 deg, tilt +-17.0 deg, four stops and centre
  [PASS] light ring      9.4s  24 pixels through hostile, satisfied, neutral, dark
  [PASS] microphones     6.6s  index 1, peak 0.2884, rms 0.0771, gate 0.0050
  [PASS] speaker         8.9s  440 Hz at 0.70, array heard rms 0.1590, floor 0.0200
  [SKIP] arm             0.0s  not run: --skip arm
  [SKIP] power           0.0s  not run: arm did not pass

4 passed, 0 failed, 2 not run   36.1s of a 90s budget
GO - every subsystem answered for itself
wrote glados/data/hardware_signoff.json

Read the last three lines together. Four passed. Two were never run. Verdict: GO. The counts are right there, one line above a word that contradicts them, and the file on disk records "verdict": "GO" next to "not_run": 2. Nothing raised, nothing was measured wrongly, and every individual number in that report is accurate. Only the conclusion is false.

The path from symptom to cause is short once you look at the right line. The verdict disagrees with a count computed from the same list, so the bug is in whichever of the two consults fewer states. any(c.status == FAIL ...) asks about one status out of three and treats the other two as equivalent, which was true on the day it was written, when only PASS and FAIL existed. Adding SKIP silently changed what that line means, and it changed it in the direction that flatters the build.

The fix is all(c.status == PASS ...), and the general form is the part to keep: derive an approval from the evidence that supports it, never from the absence of evidence against it. Any function that decides yes or no by searching for a negative has this bug waiting in it, and it arrives on the day somebody adds a third possibility. The --skip flag survives the fix unharmed. It still lets you test the head with the arm in a box; it just can no longer call that a GO.

Checkpoint, and a body that has run out of parts to add

✓ Checkpoint — what you can now do
  • I can explain why load is forbidden to raise while need is required to, and what a defaulted calibration would do to the pan servo.
  • I can name the three statuses, say which two are claims about the hardware, and predict the verdict for any mix of them.
  • Given the six checks, I can say which two depend on which others, and what each dependency buys the check that has it.
  • I know why the power check keeps the lowest observed sample, and why a passing row cannot exclude millisecond dips between samples.
  • I can explain why the speaker probe opens an InputStream by hand instead of calling sd.rec, and why its callback copies the buffer.
  • Handed a report whose verdict disagrees with its own counts, I can find the line that asked about one status out of three.
⚡ Exercises — try first, then reveal
Exercise 1 — an unattended mode that can never say GO. Add --unattended so the three watched checks skip instead of prompting, then run it and read the verdict.

Give watched a module-level switch and have it return a skip rather than a boolean. The cleanest version raises a small exception the runner already knows how to turn into a row, but a flag on the probe list is enough:

WATCHED = ("eye", "light ring", "arm")

if args.unattended and check.name in WATCHED:
    skip(check, "not run: unattended, nobody watching")
$ uv run python -m labs.signoff --unattended
  [SKIP] eye             0.0s  not run: unattended, nobody watching
  [SKIP] light ring      0.0s  not run: unattended, nobody watching
  [FAIL] microphones     3.2s  index 1, peak 0.0041, rms 0.0009, gate 0.0050
  [SKIP] speaker         0.0s  not run: microphones did not pass
  [SKIP] arm             0.0s  not run: unattended, nobody watching
  [SKIP] power           0.0s  not run: light ring and arm did not pass

0 passed, 1 failed, 5 not run   3.2s of a 90s budget
NO-GO - unresolved: eye, light ring, microphones, speaker, arm, power

An unattended run is now structurally incapable of printing GO, and that is the right outcome: three of these subsystems produce light or motion, and no sensor on this machine observes either. Watch the cascade in the middle as well. Nobody spoke, so the array failed its own gate, and the speaker row skipped behind it without being asked to decide anything. An empty room told the truth six times over.

Exercise 2 — trend two reports. Read the last two sign-off files and print, per check, whether the status changed and how the measured numbers moved.

Keep each run as hardware_signoff_<timestamp>.json, load the two newest, and compare by name:

for name in {c["name"] for c in old["checks"]} & {c["name"] for c in new["checks"]}:
    a = next(c for c in old["checks"] if c["name"] == name)
    b = next(c for c in new["checks"] if c["name"] == name)
    if a["status"] != b["status"] or a["detail"] != b["detail"]:
        print(f"{name:<12} {a['status']} -> {b['status']}\n  was: {a['detail']}\n  now: {b['detail']}")
$ uv run python -m labs.signoff_trend
power        PASS -> PASS
  was: pack 7.92/7.68 V 72%, motion 5.02/4.86 V, floor 4.60 V
  now: pack 7.61/7.19 V 50%, motion 5.01/4.71 V, floor 4.60 V

Both runs passed, and the second one is a warning anyway. The motion rail lost 0.15 volts of its loaded floor while the pack dropped from seventy two percent to fifty, so the sag is tracking the state of charge and the margin above 4.60 has halved. A pass or fail hides that. The numbers behind it do not, which is the argument for storing the measurement in the row and not only the verdict.

Exercise 3 (stretch) — make her announce her own result. Send the verdict through the voice path, so the last thing volume 7 does is spoken by the body that was just tested.

Compose the sentence from the report, never from a literal, and use the speaker the check just cleared:

report = json.loads(REPORT_PATH.read_text())
line = (f"{report['passed']} of {len(report['checks'])} subsystems nominal. "
        f"Verdict: {report['verdict']}.")
speak(line, device=output_index(hw))     # the path volume 1 built

It is the whole volume in one demonstration. The numbers came from probes that touched real hardware, the hardware was calibrated by hand, the voice came from volume 1, and the sound arrives through an amplifier that was measured by her own microphones ninety seconds ago. Record it. That file is the first thing this machine has ever said about itself.

Look at what volume 7 added. A printed shell with a material spec that a validator checks before the slicer runs. A servo you can reason about, down to the pulse width and the deadband around it. An eye that eases through a clamped choke point, and a ring of twenty four pixels driven over one wire without corrupting a colour. A microphone array that resolves by name so a reboot cannot point the pipeline at a speaker, and an amplifier whose array dimensions and sample rate decide whether a sound exists at all. Four arm joints, each with its own measured band, behind one method that clamps and writes. A power budget with two converters, a wire gauge chosen from Ohm's law, and an ADC that turns the rail into a number. One pin map that outranks every constant in the build. A checklist that survived the evening you closed the laptop halfway through assembly. Calibration anchors measured against real stops. And now a run that exercises all of it at once and leaves a dated file saying so.

She physically exists. Six volumes of her lived in files and processes, and the only evidence of any of it was text on your screen. Now there is an object on the desk with a shell, an eye that tracks, a ring that changes colour with her mood, an arm that reaches and holds, ears that hear you from across the room and a voice loud enough to answer over a kitchen. It has a serial number of sorts, a revision, a calibration date and a sign-off. When something in this build says she moves, something in the room moves.

What she does not have yet is a brain that fits her body. Every model in this book has been running on a laptop, on the other end of a cable, or on a Pi that manages a small language model and a whisper transcription only by keeping both of them tiny. The Pi is the right controller for servos, pixels and an ADC, and it is the wrong computer for a mind. Volume 8 moves her onto a Jetson Orin Nano: flashing JetPack and proving CUDA is real, benchmarking the GPU against the numbers she runs at today, running Ollama on hardware that can hold a model several times larger, driving these same pins from a different board, migrating the whole stack across, and setting her up to start on boot and stay up. Same code, same configs, same sign-off run. A bigger mind in the body you just finished testing.