Chapter 60 — Tuning by Feedback

Two people, one manner of speaking

She has one voice and one set of instructions about how to use it. That was fine while you were the only person in the room. Put her in a house and the arrangement breaks within a day: someone wants three dry sentences, someone else wants one word and no jokes, and the person who asked for brevity on Monday wants the long version on Thursday because they are actually debugging something. Every one of those is a reasonable request. None of them can be satisfied by a prompt written once.

The obvious answer is to open the file and edit the instruction, and it costs more than it looks. The edit lives in a string, so nothing records that it happened or why; the next edit overwrites the last one; and the person who wanted the change has to know where the file is. Tuning by hand puts the maintainer in the loop for something the person she lives with should be able to say to her face.

So the rule for this chapter: feedback moves numbers in a profile, and the prompt is rendered from that profile every time she speaks. Nobody edits an instruction. Somebody says "too long," a field drops by one, and the next request she receives carries a different sentence because the renderer read a different number. Four small pieces make that work: the profile, a nudge with limits, a reader that recognises feedback in ordinary speech, and a render.

◆ Note — three files in configs/, three different owners

personality.json is who she is, and you wrote it. preferences.json holds the settings you chose on purpose, by name and by value. The file this chapter adds, configs/interaction_tuning.json, is the only one of the three that gets written by somebody talking. Keeping it separate matters: a stream of nudges is allowed to drift, and you never want a bad afternoon of feedback quietly rewriting her identity. If feedback ever needs to change who she is, that is a decision you make in an editor, not a side effect of a complaint.

A profile, a nudge, and a reader for plain speech

▣ Build · stage 1 — the fields, and the sentences they produce

# labs/interaction_tuning.py
from dataclasses import dataclass, field


@dataclass
class InteractionProfile:
    max_sentences: int = 3
    sarcasm: int = 3                 # 0 = straight, 5 = relentless
    portal_references: bool = True
    topics: list[str] = field(default_factory=list)

    def to_addendum(self) -> str:
        parts = [f"Keep replies to {self.max_sentences} sentences or fewer."]
        if self.sarcasm == 0:
            parts.append("Answer straight, with no mockery.")
        elif self.sarcasm >= 4:
            parts.append("Be openly, relentlessly sarcastic.")
        if self.portal_references:
            parts.append("Reference Aperture Science testing occasionally.")
        if self.topics:
            parts.append(f"This person cares about: {', '.join(self.topics)}.")
        return " ".join(parts)


if __name__ == "__main__":
    profile = InteractionProfile(topics=["robotics", "sourdough"])
    print(profile.to_addendum())

$ uv run python labs/interaction_tuning.py
Keep replies to 3 sentences or fewer. Reference Aperture Science testing occasionally. This person cares about: robotics, sourdough.

Each condition contributes at most one sentence, and every sentence traces back to exactly one field. That is the property to protect: read any line of the addendum and you can name the number that produced it. Notice what the default sarcasm of 3 emits, which is nothing at all. The middle of the range is her ordinary manner, already described in her base prompt, so only a deliberate move away from centre earns an instruction, and the addendum stays short enough to leave room for what she must be told every turn.

▣ Build · stage 2 — nudges that cannot leave the profile broken

SIGNALS = {
    "shorter":      ("max_sentences", -1),
    "longer":       ("max_sentences", +1),
    "less_sarcasm": ("sarcasm", -1),
    "more_sarcasm": ("sarcasm", +1),
}

LIMITS = {"max_sentences": (1, 8), "sarcasm": (0, 5)}


    def nudge(self, signal: str) -> str:
        if signal not in SIGNALS:
            return f"no rule for {signal!r}"
        name, delta = SIGNALS[signal]
        low, high = LIMITS[name]
        before = getattr(self, name)
        after = min(high, max(low, before + delta))
        setattr(self, name, after)
        if after == before:
            return f"{name} stays at {before} (limit)"
        return f"{name} {before} -> {after}"


if __name__ == "__main__":
    profile = InteractionProfile()
    for signal in ["shorter", "shorter", "shorter", "less_sarcasm"]:
        print(profile.nudge(signal))

$ uv run python labs/interaction_tuning.py
max_sentences 3 -> 2
max_sentences 2 -> 1
max_sentences stays at 1 (limit)
sarcasm 3 -> 2

Feedback arrives as a stream, and streams repeat. Someone irritated at eleven at night will say "too long" four times, and without min and max that fourth complaint asks her for zero sentences, then minus one. Clamping means every reachable state is a usable state, so no sequence of nudges can produce a prompt she cannot obey. The other decision here is the table: SIGNALS and LIMITS are data, so a fifth knob is two new rows and no new branch, and getattr/setattr look up the attribute by the name the table supplies. The return value is a sentence, not None, because a nudge that hit its limit and a nudge that moved look identical from outside otherwise.

▣ Build · stage 3 — hearing feedback inside an ordinary sentence

PHRASES = {
    "shorter":      ("too long", "shorter", "get to the point", "stop rambling"),
    "longer":       ("too short", "more detail", "explain more"),
    "less_sarcasm": ("less sarcasm", "not funny", "be nice"),
    "more_sarcasm": ("more sarcasm", "be meaner"),
}


def read_feedback(said: str) -> str | None:
    text = said.lower()
    for signal, phrases in PHRASES.items():
        if any(p in text for p in phrases):
            return signal
    return None


if __name__ == "__main__":
    profile = InteractionProfile()
    for said in ["that was too long", "less sarcasm, please", "what is the arm doing"]:
        signal = read_feedback(said)
        if signal is None:
            print(f"you> {said}\n      (not feedback; goes to the model)")
        else:
            print(f"you> {said}\n      {profile.nudge(signal)}")

$ uv run python labs/interaction_tuning.py
you> that was too long
      max_sentences 3 -> 2
you> less sarcasm, please
      sarcasm 3 -> 2
you> what is the arm doing
      (not feedback; goes to the model)

This is the piece that closes the loop, and its position in the turn is the whole point. read_feedback runs before the model call, so "that was too long" is treated as an instruction about her manner and never reaches the model as a question. Send it through instead and you get what you deserve: a sardonic reply about length, and a profile that never moved. Substring matching is crude on purpose, since the transcript from her ears is already approximate and a small table of real phrases beats a clever parser you cannot inspect. Returning None for everything else keeps ordinary conversation as the default path.

▣ Build · stage 4 — the trail, the file, and the render at call time

# labs/interaction_tuning.py — full file
import json
from dataclasses import asdict, dataclass, field, fields
from datetime import datetime
from pathlib import Path

TUNING_PATH = Path("configs/interaction_tuning.json")
MAX_TRAIL = 20

SIGNALS = {
    "shorter":      ("max_sentences", -1),
    "longer":       ("max_sentences", +1),
    "less_sarcasm": ("sarcasm", -1),
    "more_sarcasm": ("sarcasm", +1),
}

LIMITS = {"max_sentences": (1, 8), "sarcasm": (0, 5)}

PHRASES = {
    "shorter":      ("too long", "shorter", "get to the point", "stop rambling"),
    "longer":       ("too short", "more detail", "explain more"),
    "less_sarcasm": ("less sarcasm", "not funny", "be nice"),
    "more_sarcasm": ("more sarcasm", "be meaner"),
}


@dataclass
class InteractionProfile:
    max_sentences: int = 3
    sarcasm: int = 3
    portal_references: bool = True
    topics: list[str] = field(default_factory=list)
    trail: list[str] = field(default_factory=list)

    def nudge(self, signal: str) -> str:
        if signal not in SIGNALS:
            return f"no rule for {signal!r}"
        name, delta = SIGNALS[signal]
        low, high = LIMITS[name]
        before = getattr(self, name)
        after = min(high, max(low, before + delta))
        setattr(self, name, after)
        note = (f"{name} stays at {before} (limit)" if after == before
                else f"{name} {before} -> {after}")
        stamp = datetime.now().isoformat(timespec="seconds")
        self.trail.append(f"{stamp} {signal}: {note}")
        self.trail[:] = self.trail[-MAX_TRAIL:]
        return note

    def to_addendum(self) -> str:
        parts = [f"Keep replies to {self.max_sentences} sentences or fewer."]
        if self.sarcasm == 0:
            parts.append("Answer straight, with no mockery.")
        elif self.sarcasm >= 4:
            parts.append("Be openly, relentlessly sarcastic.")
        if self.portal_references:
            parts.append("Reference Aperture Science testing occasionally.")
        if self.topics:
            parts.append(f"This person cares about: {', '.join(self.topics)}.")
        return " ".join(parts)


def read_feedback(said: str) -> str | None:
    text = said.lower()
    for signal, phrases in PHRASES.items():
        if any(p in text for p in phrases):
            return signal
    return None


def load_profile(path: Path) -> InteractionProfile:
    if not path.exists():
        return InteractionProfile()
    data = json.loads(path.read_text())
    known = {f.name for f in fields(InteractionProfile)}
    return InteractionProfile(**{k: v for k, v in data.items() if k in known})


def save_profile(profile: InteractionProfile, path: Path) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(json.dumps(asdict(profile), indent=2) + "\n")


def main() -> None:
    base = "You are GLaDOS, the rogue AI from Aperture Science."
    profile = load_profile(TUNING_PATH)
    print("prompt:", base, profile.to_addendum())
    for said in ["that was too long", "less sarcasm, please"]:
        signal = read_feedback(said)
        if signal is not None:
            print(f"you> {said}\n      {profile.nudge(signal)}")
    print("prompt:", base, profile.to_addendum())
    save_profile(profile, TUNING_PATH)
    print(f"saved {TUNING_PATH} ({len(profile.trail)} nudges on record)")


if __name__ == "__main__":
    main()

$ uv run python labs/interaction_tuning.py
prompt: You are GLaDOS, the rogue AI from Aperture Science. Keep replies to 3 sentences or fewer. Reference Aperture Science testing occasionally.
you> that was too long
      max_sentences 3 -> 2
you> less sarcasm, please
      sarcasm 3 -> 2
prompt: You are GLaDOS, the rogue AI from Aperture Science. Keep replies to 2 sentences or fewer. Reference Aperture Science testing occasionally.
saved configs/interaction_tuning.json (2 nudges on record)

$ head -8 configs/interaction_tuning.json
{
  "max_sentences": 2,
  "sarcasm": 2,
  "portal_references": true,
  "topics": [],
  "trail": [
    "2026-03-04T21:14:08 shorter: max_sentences 3 -> 2",
    "2026-03-04T21:14:08 less_sarcasm: sarcasm 3 -> 2"

Two prompt lines, printed from the same function, differing because two numbers differ. The timestamps in your file will be your own, and the first run creates the file from defaults since load_profile treats a missing path as a fresh start. self.trail[:] = self.trail[-MAX_TRAIL:] keeps the record bounded by assigning into the existing list, so anything else holding a reference to it sees the trim. The trail is the difference between a profile and a mystery. When she starts answering in one clipped line and you cannot remember asking for that, the file tells you which evening you did.

Why this works: one direction, and nothing cached

The design is a one-way flow. Speech becomes a signal, a signal moves a field, and the instruction text is computed from the fields at the moment it is needed. Nothing writes to the prompt, ever. Because to_addendum reads fields and returns a string with no side effects, the same profile always renders the same instruction, so you can test it offline in milliseconds with no model running. That one small file also holds the entire adaptable state of her manner: copy it and everything she learned about you moves with it.

Be precise about what "she learns" means here, because the word invites a larger claim than the code makes. The model on your machine is unchanged. Its weights are the same bytes they were when you pulled it, and no gradient was computed anywhere in this chapter. What changed is the input she is given, and that is enough for manner, tone and length, which live in the instructions and not in the parameters. Nudging data is cheap, instant, reversible, and auditable; training is none of those.

⚠ Worked failure — she agrees to be shorter and stays long

The module works. Wire it into the conversation loop and the nudges stop having any effect, with no error to explain why:

profile = load_profile(TUNING_PATH)
system_prompt = f"{base} {profile.to_addendum()}"   # BUG: rendered once, before the loop

while True:
    said = listen()
    signal = read_feedback(said)
    if signal is not None:
        profile.nudge(signal)
        continue
    speak(ask_model(system_prompt, said))

$ uv run python labs/voice_loop.py   # after saying "too long" three times
profile.max_sentences: 1
prompt in use:         Keep replies to 3 sentences or fewer. Reference Aperture Science testing occasionally.

The profile moved exactly as designed, all the way down to its limit, and she kept answering at her original length. The f-string ran once and produced a plain string; it copied the characters that the numbers implied at that instant and then had no further connection to the object. Later writes to profile.max_sentences update the dataclass, and the string on the other variable has never heard of it. The fix is to move one line inside the loop and call to_addendum() per turn, which costs a list join against a model call measured in hundreds of milliseconds. Any value derived from data that changes is either recomputed at use or invalidated when the data moves, and recomputing is the version with no bookkeeping to get wrong.

Checkpoint, and a manner that answers to the room

✓ Checkpoint — what you can now do

I can trace one spoken phrase through read_feedback, SIGNALS, LIMITS and to_addendum, and name the sentence it changed.
I can say why read_feedback has to run before the model call, and what she does with "too long" if it does not.
I can explain what the clamp protects against, and give the sequence of feedback that would break an unclamped profile.
I know why a default sarcasm of 3 contributes no sentence, and what that buys in the prompt she actually receives.
Shown an assistant whose profile changes and whose behaviour does not, I can name the cached render as the first suspect and confirm it by printing both.
I can state what did not change in this chapter: the model, its weights, and anything requiring training data.

⚡ Exercises — try first, then reveal

Exercise 1 — a way back to normal. Add the signal "reset", triggered by "go back to normal", that restores the defaults in place while keeping the trail. Print the addendum before and after.

nudge is table-driven and a reset is not a delta, so give it its own branch: build InteractionProfile() as a throwaway and copy its values onto self field by field, skipping trail. Doing it in place, instead of rebinding the name to a new object, matters because the voice loop is holding a reference to this one. Append a trail entry for the reset too, then watch a tuned addendum render as the default one again with its history intact underneath.

Exercise 2 — let her confirm the change. Give each signal a spoken acknowledgement, keyed by signal name, and have the loop speak it through her voice instead of printing it.

A dict beside SIGNALS holding one line each is all the structure this needs: "shorter": "Noted. Fewer sentences. I will try to make them count." The loop already knows the signal, so it looks up the line and hands it to the same speech function every other reply goes through. Handle the limit case separately or she will promise to shorten a reply already pinned at one sentence, which is the kind of small lie that makes an assistant feel broken. Run it and the tuning loop becomes audible instead of textual.

Exercise 3 — one profile per person. Move the file to configs/tuning/<name>.json, load the profile for whoever is speaking, and diff two people's files after a few days of use.

load_profile already takes a path, so this is mostly a naming function: lowercase the name, strip anything that is not a letter, and refuse an empty result so a blank name cannot write to configs/tuning/.json. Save on every nudge, since two people alternating turns will otherwise overwrite each other from stale objects in memory. The diff is the payoff: diff -u configs/tuning/chell.json configs/tuning/doug.json prints a handful of lines that describe, in numbers, how differently two people want to be spoken to by the same machine.

Her manner now answers to the people around her, and the record of every adjustment survives a reboot. What nobody has measured is the wait. Between a finished sentence and her first syllable sits a chain of components, and one of them is slower than the rest in a way an average will hide from you. Next chapter puts a clock on each link and reads the slow tail instead of the middle.