The World Vol 9 · The Village
ch 92 / 105
Chapter 92

A Memory Is a Row

Eight hundred and thirty-two parcels that no row records

Halla asked to hand a parcel eight hundred and thirty-two times in the run that closed the last chapter, and there is no row anywhere in this world that says so. Every one of those ticks was computed from the valley as it stood and thrown away. The roster knows what she is carrying now. The chronicle knows that a world was named and, if a daemon is running, how many bodies were walking at each midsummer. Nothing knows what happened to Halla on tick 1015, and nothing can be asked.

A memory is one row, weighed once at the tick it happened and never touched again. The table takes no UPDATE and no DELETE, and that is the chronicle's law a second time.

The database has existed since the seventh volume, so the problem is the table's contract. Everything a villager knows about one moment of its own life sits in one row, and everything it knows about its whole life sits in one stream of those rows. The chronicle made the same promise first: append rows, never rewrite them. The embedding table in the last volume kept that promise by putting a model's numbers beside the chronicle instead of filling a new column later.

A weight that can never be revised has to be decided when the row is written, from facts known then, by arithmetic that comes out the same every time it is run. Importance becomes an integer read off a table small enough to check on paper, and the chapter shows one person's day scored by hand and then by the bench.

Freshness also has to come from something the row already holds. The obvious measure, time since last recall, needs a write on every read. That version gets built, run and removed, with its cost counted; the way out is the world's own clock.

The columns are settled before any code is written, because they are the one thing every later page of this volume leans on. A row holds an identity; the villager it belongs to; the tick it happened at, which is the world's clock and never the wall clock; a kind, of which there are exactly four, saw, did, heard and thought; one line of text; a weight from 1 to 10; the entity identities the line is about, so that whatever reads the row later can check what this villager could have known; who said it, for a heard memory and for nothing else; and the memory identities it was drawn from, for a thought and for nothing else. Nine columns. This chapter writes rows of two of the four kinds, because a world on its own produces things seen and things done, and a villager who has heard or thought anything is a villager somebody has spoken to or a villager who has reflected, and neither of those exists yet.

Nine columns, two of them tied to a kind

The schema of this world is a directory of numbered files, applied in the order of their numbers and never edited once one has run anywhere. Seven files make eight tables, and the seven are the last volume's to the byte. This is the eighth file, and it makes two tables, because a memory belongs to a villager and a villager with no stream is a row nothing will ever ask about.

▣ Build · stage 1 — the eighth migration, whole
-- migrations/0008_memory.sql

-- The people, and what each of them remembers. Two tables in one
-- file, because a memory belongs to a villager and a villager with no
-- stream is a row nothing will ever ask about.
--
-- A villager is keyed by name and not by body. The body is a roster
-- number handed out at a founding; the name is what the roll wrote
-- down before anything happened, and it is the name a memory is
-- filed under. Nothing here is ever updated.
CREATE TABLE villager (
    name    text   NOT NULL,
    body    bigint NOT NULL,
    persona text   NOT NULL,
    law     text   NOT NULL,
    home_x  int    NOT NULL,
    home_y  int    NOT NULL,
    founded bigint NOT NULL,

    CONSTRAINT villager_key         PRIMARY KEY (name),
    CONSTRAINT villager_has_one_body UNIQUE (body),
    CONSTRAINT villager_founded_on_a_real_tick CHECK (founded >= 0),
    CONSTRAINT villager_lives_by_something CHECK (persona <> '' AND law <> '')
);

-- One row a memory, written once and never touched again. It is the
-- chronicle's law a second time: id is GENERATED ALWAYS, so nothing
-- can write its own number or put a row into the middle of a life,
-- and there is no statement anywhere in this world that updates or
-- deletes one of these rows.
--
-- tick is the world's own clock and never now(). A memory is about a
-- moment in the world, and the wall clock says nothing about when
-- that was.
--
-- weight is written at append and can never be revised, because
-- revising it would be an UPDATE. What a memory was worth is decided
-- once, on the tick it happened, by arithmetic that replays.
--
-- about holds the entity identities the line names, so that a later
-- reader can check what this villager could have known. said_by is
-- who said it, and drawn_from is which memories it was made out of;
-- the two checks below tie each of those to exactly one kind, so a
-- row cannot claim a speaker it did not hear or a source it did not
-- think from.
CREATE TABLE memory (
    id         bigint   GENERATED ALWAYS AS IDENTITY,
    villager   text     NOT NULL,
    tick       bigint   NOT NULL,
    kind       text     NOT NULL,
    text       text     NOT NULL,
    weight     int      NOT NULL,
    about      bigint[] NOT NULL DEFAULT '{}',
    said_by    bigint,
    drawn_from bigint[],

    CONSTRAINT memory_key PRIMARY KEY (id),
    CONSTRAINT memory_of_a_villager FOREIGN KEY (villager) REFERENCES villager (name),
    CONSTRAINT memory_on_a_real_tick CHECK (tick >= 0),
    CONSTRAINT memory_is_one_of_four_kinds CHECK (kind IN ('saw', 'did', 'heard', 'thought')),
    CONSTRAINT memory_says_something CHECK (text <> ''),
    CONSTRAINT memory_weighs_one_to_ten CHECK (weight BETWEEN 1 AND 10),
    CONSTRAINT only_a_heard_memory_has_a_speaker CHECK ((kind = 'heard') = (said_by IS NOT NULL)),
    CONSTRAINT only_a_thought_is_drawn_from_memories CHECK ((kind = 'thought') = (drawn_from IS NOT NULL))
);

-- Every question asked of a stream is about one villager over a
-- stretch of ticks, and id on the end returns one tick's rows in the
-- order they were written.
CREATE INDEX memory_by_villager ON memory (villager, tick, id);
$ go run ./cmd/rows -mode plan
rows: the schema through file 0008, and not one statement of it run

  n    file                       sha256, first 16     bytes  what it makes
  0001 0001_world.sql             55b759d43aa929d4       641  world
  0002 0002_chronicle.sql         58878101d9384245       735  chronicle
  0003 0003_creature.sql          5f95f200d36d9d2e      1740  creature
  0004 0004_death.sql             43845ea86c947233       696  death
  0005 0005_species.sql           54029ed637c5b65d       858  species
  0006 0006_snapshot.sql          e80d9a5289d0f66c      1183  snapshot, snapshot_part
  0007 0007_chronicle_vector.sql  b3d9bebcf37a8003      2313  chronicle_vector
  0008 0008_memory.sql            ce5492205ec9bfdd      3133  villager, memory

  8 files, 11299 bytes, 10 tables, applied in the order of their numbers and no other
  the first seven files are volume 8's to the byte, and the eighth is this chapter's

  villager, 7 columns, as 0008_memory.sql declares them
    column       type                                 null
    name         text                                 no
    body         bigint                               no
    persona      text                                 no
    law          text                                 no
    home_x       int                                  no
    home_y       int                                  no
    founded      bigint                               no
    and 2 checks: villager_founded_on_a_real_tick, villager_lives_by_something

  memory, 9 columns, as 0008_memory.sql declares them
    column       type                                 null
    id           bigint, generated always             no
    villager     text                                 no
    tick         bigint                               no
    kind         text                                 no
    text         text                                 no
    weight       int                                  no
    about        bigint[], default '{}'               no
    said_by      bigint                               yes
    drawn_from   bigint[]                             yes
    and 6 checks: memory_on_a_real_tick, memory_is_one_of_four_kinds, memory_says_something, memory_weighs_one_to_ten, only_a_heard_memory_has_a_speaker, only_a_thought_is_drawn_from_memories

  nothing this world runs updates or deletes a row of either table. That is
  not a line the file can declare; it is the chronicle's law a second time,
  and a test over the store's own source is where it is held to

The two checks at the bottom of the memory table are the ones to read slowly, because each is an equality between two booleans. (kind = 'heard') = (said_by IS NOT NULL) is true when both sides are true and when both are false, and false otherwise, so it refuses a heard memory with no speaker and a seen memory with one. A column that is only meaningful for one kind is a column that can be filled in for the wrong kind, and a later reader who trusts said_by as provenance has to be able to trust that a row with a speaker on it was heard. The database is the cheapest place to make that true.

The last three lines of the run say the thing the file cannot. A schema can refuse a weight of eleven and a fifth kind, and it does; it cannot declare that nobody will ever send it an UPDATE. What holds that line is the same thing that holds it for the chronicle: there is no such statement in the one package that speaks SQL, and this chapter adds a test that reads that package's source and counts.

▣ Build · stage 2 — eight files against an empty database, twice
$ podman exec -w /bench world-go go run ./cmd/rows -mode up -twice
rows: bringing a database up to the schema this build holds

  the first run
    0001 0001_world.sql             applied
    0002 0002_chronicle.sql         applied
    0003 0003_creature.sql          applied
    0004 0004_death.sql             applied
    0005 0005_species.sql           applied
    0006 0006_snapshot.sql          applied
    0007 0007_chronicle_vector.sql  applied
    0008 0008_memory.sql            applied
    8 applied, 0 already there, and this run changed the database

  the second run
    0001 0001_world.sql             already applied, and its bytes still agree
    0002 0002_chronicle.sql         already applied, and its bytes still agree
    0003 0003_creature.sql          already applied, and its bytes still agree
    0004 0004_death.sql             already applied, and its bytes still agree
    0005 0005_species.sql           already applied, and its bytes still agree
    0006 0006_snapshot.sql          already applied, and its bytes still agree
    0007 0007_chronicle_vector.sql  already applied, and its bytes still agree
    0008 0008_memory.sql            already applied, and its bytes still agree
    0 applied, 8 already there, and this run changed nothing at all

  tables in this database, the ledger aside    10
  villager   7 columns, 0 rows
  memory     9 columns, 0 rows
$ podman exec world-db psql -U world -d world -c '\d memory'
                            Table "public.memory"
   Column   |   Type   | Collation | Nullable |           Default
------------+----------+-----------+----------+------------------------------
 id         | bigint   |           | not null | generated always as identity
 villager   | text     |           | not null |
 tick       | bigint   |           | not null |
 kind       | text     |           | not null |
 text       | text     |           | not null |
 weight     | integer  |           | not null |
 about      | bigint[] |           | not null | '{}'::bigint[]
 said_by    | bigint   |           |          |
 drawn_from | bigint[] |           |          |
Indexes:
    "memory_key" PRIMARY KEY, btree (id)
    "memory_by_villager" btree (villager, tick, id)
Check constraints:
    "memory_is_one_of_four_kinds" CHECK (kind = ANY (ARRAY['saw'::text, 'did'::text, 'heard'::text, 'thought'::text]))
    "memory_on_a_real_tick" CHECK (tick >= 0)
    "memory_says_something" CHECK (text <> ''::text)
    "memory_weighs_one_to_ten" CHECK (weight >= 1 AND weight <= 10)
    "only_a_heard_memory_has_a_speaker" CHECK ((kind = 'heard'::text) = (said_by IS NOT NULL))
    "only_a_thought_is_drawn_from_memories" CHECK ((kind = 'thought'::text) = (drawn_from IS NOT NULL))
Foreign-key constraints:
    "memory_of_a_villager" FOREIGN KEY (villager) REFERENCES villager(name)

The database is the same pinned image the last volume ran on, started the same way on the same closed bridge, with its data directory on a tmpfs so that every run of this page starts from nothing. The second run of the migrations applies none and skips eight, which is the whole of what idempotent means here, and the server's own description of the table is the file's: about defaults to an empty array and not to null, because a line about nobody is a fact and a missing array is a question.

⚙ Tool — psql, and which container each command runs in

Every podman exec on this page runs on the closed bridge the last volume built: the Go commands inside world-go, where the module is copied to /bench, and the psql commands inside world-db, which is the only container that has it. \d is the client's own way of describing a table back; its reference is at postgresql.org/docs under psql.

The Go side of the row is a struct with the same nine fields, in a package the valley has never heard of, and a check that refuses in Go what the schema would refuse at the server. The check exists because the server's refusal, correct as it is, arrives a long way from the thing that was wrong, on the far side of a socket, wrapped in a code; the schema is the floor and the check is the early word.

▣ Build · stage 3 — the row in Go, and the four kinds
// internal/village/memory.go

// Kind is what sort of memory a row is, and there are exactly four.
// A villager saw something, did something, heard something said, or
// thought something out of what it already had. Nothing else is a
// memory, and the schema refuses a fifth.
type Kind string

const (
	Saw     Kind = "saw"
	Did     Kind = "did"
	Heard   Kind = "heard"
	Thought Kind = "thought"
)
// ...
type Memory struct {
	ID        int64
	Villager  string
	Tick      int
	Kind      Kind
	Text      string
	Weight    int
	About     []sim.EntityID
	SaidBy    sim.EntityID
	DrawnFrom []int64
}

// Check refuses a row the table would refuse, here rather than at the
// server, because the server's refusal is correct and arrives a long
// way from the thing that was wrong. It tests the same seven things
// the schema tests, in the same words.
func (m Memory) Check() error {
	// ...
	if (m.Kind == Heard) != (m.SaidBy != 0) {
		return fmt.Errorf("village: %s: only a heard memory has a speaker, and every heard memory has one", m.Villager)
	}
	if (m.Kind == Thought) != (m.DrawnFrom != nil) {
		return fmt.Errorf("village: %s: only a thought is drawn from memories, and every thought is", m.Villager)
	}
	return nil
}
// internal/store/memory.go

// Remember appends rows to the memory table, all of them in one
// transaction, and fills in the id the database gave each one.
//
// It is called between ticks and never inside one, like everything
// else in this package. One transaction a boundary and not one a
// row, because the boundary is the unit: what one tick did to a
// village is either all written down or none of it is.
//
// The id comes back through RETURNING rather than being chosen here,
// because the column is GENERATED ALWAYS: the database hands the
// number out and nothing can write its own.
func (d *DB) Remember(ctx context.Context, rows []village.Memory) error {
	if len(rows) == 0 {
		return nil
	}
	for _, m := range rows {
		if err := m.Check(); err != nil {
			return fmt.Errorf("store: %w", err)
		}
	}
	tx, err := d.pool.Begin(ctx)
	if err != nil {
		return fmt.Errorf("store: remembering: %w", err)
	}
	defer tx.Rollback(ctx)
	const q = `INSERT INTO memory (villager, tick, kind, text, weight, about, said_by, drawn_from)
	           VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING id`
	// ...
	if err := tx.Commit(ctx); err != nil {
		return fmt.Errorf("store: remembering: %w", err)
	}
	return nil
}
$ go test ./internal/store/ -run 'TestNothingButTheWorkedFailureTouchesAMemory|TestAMemoryRowIsCheckedBeforeItReachesTheServer' -v
=== RUN   TestNothingButTheWorkedFailureTouchesAMemory
    memory_test.go:51: every file of the store read, and the one statement that touches a row is the worked failure's
--- PASS: TestNothingButTheWorkedFailureTouchesAMemory (0.01s)
=== RUN   TestAMemoryRowIsCheckedBeforeItReachesTheServer
--- PASS: TestAMemoryRowIsCheckedBeforeItReachesTheServer (0.00s)
PASS
ok  	theworld/internal/store	0.012s

The store imports the village and the village does not import the store, which is the direction every arrow in this module has run since the database arrived: the thing that writes a world down is allowed to know what a world is, and the world is not allowed to know that a database exists. So the row type lives with the people, the SQL lives with the store, and a row crosses from one to the other as a value. The first test in that run is the law made into something that can fail. It reads every source file in the store, finds each UPDATE or DELETE aimed at either of the two new tables, and requires that there is exactly one, in a file of its own, which is this chapter's worked failure. A second one anywhere is a failing test, whatever it is for.

The third test is the store's table list, which has to grow in the same change as the migration. It grew by two. That test is not pinned in the run above because its one log line counts every table the store knows, and the count moves each time a file is added to the directory; it runs in the module's full test pass, where a list that has not grown with a migration fails the build.

Seventy-four transactions, every one between two ticks

A row is written at the daemon's boundary, and the boundary is a line of a loop this book has named three times: the place where no phase is part way through, no view is open and no creature is half stepped, where the seventh volume took its snapshots, where the eighth hung its embedding job, and where the last chapter's worker handed its answers over. Nothing inside a tick opens a transaction. That rule has held since the database arrived, and this is the first chapter in which something inside a tick has a reason to want to break it, because the things a memory is about happen inside ticks.

So the design is two halves that run at two different times, joined by a slice in memory. The tick half notices. The boundary half writes down. Between them the world can move, and the whole of the care in the code is about what that movement can and cannot change.

▣ Build · stage 4 — a witness in front of the mind, and what it does inside a tick
// internal/village/witness.go

// Witness answers beast.Mind for a villager by asking the Mind
// underneath it, and writes down what the villager did, in memory,
// for the boundary to turn into rows.
// ...
type Witness struct {
	Mind *Mind
	Body *beast.Beast
	Name string

	// View is the reading of the world the current phase opened on,
	// handed over through Roster.Seen before the first creature is
	// stepped. It is what the world's own search reads, so it is what
	// this one reads.
	View *beast.View
	// ...
}

// Pick is the tick side. It answers with whatever the Mind answers
// and, for a transfer, notes which body it would cross to.
func (w *Witness) Pick(row *beast.Senses) beast.Act {
	a := w.Mind.Pick(row)
	w.asked = a
	if w.Body.Owed > 0 || w.View == nil {
		// The choice is dropped: the action under way takes the tick.
		return a
	}
	if a == beast.Hand || a == beast.Take {
		w.partner = 0
		if o, _ := w.Body.Hands(w.View, a); o != nil {
			w.partner = o.ID
		}
	}
	return a
}

The witness is a beast.Mind that holds a Mind, so the mailbox and the habit from the last chapter are untouched and the body never finds out there is one more layer. What the tick half does is small on purpose. It notes the entry the mind answered with, and if that entry is a hand or a take it asks the same search the world is about to ask, Hands, against the same reading of the world the world is about to use, so that the body a parcel crosses to is known exactly. That is a walk over nine cells, it allocates nothing, and it runs on the same view because the roster hands that view out once a tick through the seam it has carried since the fifth volume for exactly this purpose.

Why not work the partner out afterwards, at the boundary, where there is time? Because the answer can be wrong there. Hands ranks the bodies inside reach by how much room they have, and between the moment the parcel crosses and the end of the tick the recipient has been handed half a gram and every other candidate has been charged rent and may have eaten. Two neighbours whose room differed by less than that swap places, and a memory names the wrong person. A row that names the wrong person is the one thing a table built for grounding cannot hold, so the identity is taken at the only moment it is certain.

▣ Build · stage 5 — the boundary confirms against the body and never against the choice
// internal/village/witness.go

// Boundary is the other side. tick is the tick that has just run, and
// view is the world as it stands after it.
func (w *Witness) Boundary(tick int, view *beast.View) {
	b := w.Body
	// ...
	// What the tick did, read off the body and not off the choice. A
	// fresh action leaves Owed at one less than the ticks it takes;
	// an action still under way leaves it lower, and a refused choice
	// leaves Busy at a rest.
	done := b.Busy == w.asked && b.Owed == beast.Table[w.asked].Ticks-1
	cell := b.Cell()
	step := run{first: tick, last: tick}
	switch {
	case done && w.asked == beast.Bite:
		step.act, step.at, step.count, step.grams = beast.Bite, cell, 1, b.Ate-w.ate
	case done && (w.asked == beast.Hand || w.asked == beast.Take) && w.partner != 0:
		step.act, step.who, step.count, step.grams = w.asked, w.partner, 1, b.Kind.Parcel()
	case cell != w.cell:
		step.act, step.at, step.count = beast.Walk, w.cell, 1
	}
	w.ate, w.cell = b.Ate, cell
	// ...
}
// internal/village/watch.go

// Boundary runs every witness's boundary step against the world as it
// stands, then tells the other end of every transfer that closed.
// tick is the tick that has just run.
func (w *Watch) Boundary(v *terra.Valley, r *beast.Roster) {
	tick := v.Now - 1
	view := beast.NewView(v, r.Live)
	for _, wit := range w.Wits {
		wit.Boundary(tick, view)
	}
	w.tell()
}

The mind chose; the world may have refused. A choice the check refuses costs the tick and is spent resting, a choice made while an action is still under way is dropped, and the witness knows about neither, because it was written to know nothing the mind does not. So the boundary asks the body. A fresh two-tick action leaves Owed at one; a held second tick leaves it at nought; a refusal leaves Busy at a rest. The body's own bookkeeping says whether the choice happened, and grams come off the body too: Ate is the roster's count of what a mouth has taken off the valley, so a meal's grams are the difference between two readings of it and never a multiplication somebody did in a comment.

A run is the unit a row records. Fifty mouthfuls off the same ground are one meal, fifty parcels to the same neighbour are one gift, and the steps from one cell to another are one walk. A run closes when something else happens, when it pauses longer than the gap, or when it has gone on for the span, and both of those numbers are flags the bench prints. The row carries the tick of the last thing in the run, so a gift that closes because the giver turned away is filed under its last parcel. Who is in sight is cast fresh at the boundary with the villager's own fan over the world as it now stands, and a body counts as having come into sight only if it was out of sight for longer than the gap; three people on adjoining cells would otherwise write each other down ten times a second.

A day, in this volume's loose sense: two thousand ticks, which is two hundred seconds of world at ten a second, over the same ground the last two chapters founded, with the same three people on the same habit. Nothing here opens a socket. The rows are made, weighed and printed out of memory, which is exactly the state they are in at the boundary before the store is handed them.

▣ Build · stage 6 — two thousand ticks, and what one person remembers of them
$ go run ./cmd/rows -mode day
rows: a day in the village, and what one person remembers of it

  the world      seed 5, 12x8 cells, tick 901, 12 plants at 1517.5 grams
  the run        2000 ticks, from tick 901 to tick 2900
  the village    3 people out of configs/village.json, on a mailbox and a habit each
  the habit      feed below 25%, trade past 40%, top up below 90%
  the witness    one a person, in front of the mind; a run of one thing may
                 pause 10 ticks and go on for 100 before it is one memory
  the weights    off configs/importance.json, 1 to 10

  what the day produced, one row a memory
  name       rows   by kind and event
  Halla        24   did gift 5, did meal 12, did walk 2, saw body 4, saw taken 1
  Ander        39   did meal 24, did take 1, did walk 4, saw body 5, saw gift 5
  Mose         25   did meal 19, did walk 2, saw body 4

  Halla's stream, every row, in the order it was written
     #   tick  kind weight  text                                                       about
     1    901  saw       4  Ander came into sight, 0.8 cells off                       2
     2    901  saw       2  Mose came into sight, 0.8 cells off                        3
     3    999  did       7  handed Ander 50 parcels, 25.0 grams                        2
     4   1015  saw      10  Ander took 40 parcels out of Halla's hands, 20.0 grams     2,1
     5   1099  did       5  handed Ander 50 parcels, 25.0 grams                        2
     6   1173  did       4  handed Ander 37 parcels, 18.5 grams                        2
     7   1337  saw       2  Ander came into sight, 0.8 cells off                       2
     8   1348  did       3  went from 8,5 to 9,3                                       -
     9   1448  did       5  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    10   1548  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    11   1648  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    12   1748  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    13   1848  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    14   1948  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    15   2048  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    16   2148  did       3  ate 50 mouthfuls on 9,3, 23.5 grams                        -
    17   2203  saw       2  Ander came into sight, 0.2 cells off                       2
    18   2202  did       2  ate 27 mouthfuls on 9,3, 9.3 grams                         -
    19   2307  did       5  handed Ander 50 parcels, 25.0 grams                        2
    20   2341  did       4  handed Ander 17 parcels, 8.5 grams                         2
    21   2776  did       1  ate 2 mouthfuls on 9,3, 1.0 grams                          -
    22   2781  did       1  went from 9,3 to 9,2                                       -
    23   2883  did       3  ate 50 mouthfuls on 9,2, 25.0 grams                        -
    24   2899  did       1  ate 8 mouthfuls on 9,2, 4.0 grams                          -

  the 8 heaviest, weight first and then the newest, which is the order
  a reader of this stream with room for 8 rows would take them in
     #   tick  kind weight  text
     4   1015  saw      10  Ander took 40 parcels out of Halla's hands, 20.0 grams
     3    999  did       7  handed Ander 50 parcels, 25.0 grams
    19   2307  did       5  handed Ander 50 parcels, 25.0 grams
     9   1448  did       5  ate 50 mouthfuls on 9,3, 25.0 grams
     5   1099  did       5  handed Ander 50 parcels, 25.0 grams
    20   2341  did       4  handed Ander 17 parcels, 8.5 grams
     6   1173  did       4  handed Ander 37 parcels, 18.5 grams
     1    901  saw       4  Ander came into sight, 0.8 cells off

  88 rows over 3 people, every one at a tick the world ran, every weight
  between 1 and 10, and none with a speaker or a source on it: what a world
  writes is saw and did, and nothing else yet. 1 of Halla's rows were written
  down after a row about a later tick, never by more than the gap, because
  a run is written when it is over and carries the tick of its last parcel

  2000 ticks in 32ms, 61824 ticks a second (measured here; yours will differ)

Twenty-four rows is a life. Halla saw two neighbours on the tick she was founded, handed Ander fifty parcels over the next hundred ticks, had forty taken back out of her hands while she was doing it, handed over eighty-seven more, walked two cells north-east, and then ate for the rest of the afternoon in hundred-tick stretches, with one more gift in the middle of it. Every line of that is in the register of a chronicle entry: what happened, to whom, how much, and a cell. None of it was composed. Each line is a Go format string filled in from the run, so it replays, and so the about column can be trusted to hold exactly the bodies the line names.

Rows 17 and 18 are the pair to look at. A sighting at tick 2203 was written before a meal filed under tick 2202, because the meal's last mouthful was at 2202 and the run did not close until something else happened. The id says when a row was written and the tick says when the thing it records happened, and the two can disagree by up to the gap. A stream is read back ORDER BY tick, id, so a reader sees the day in the order it was lived; the bench prints it in the order it was written so that the disagreement is on the page and not hidden by the sort.

The eighty-eight rows are every event three people had in two hundred seconds of world, and the last block is the audit that lets the number stand: every row at a tick the world ran, every weight on the scale, and no speaker and no source on any of them. This world produces things seen and things done. A heard row has exactly one way in and a thought has exactly one, and neither door is built on this page, so a day with either kind in it would be a day something was wrong.

Now the same day with the store on the far side of the boundary. The bench takes the pool's own count of connections handed out before every v.Tick() and after it, and the two numbers have to agree two thousand times.

▣ Build · stage 7 — the same day, written down as it happens
// cmd/rows/db.go
	for i := 0; i < c.ticks; i++ {
		before := pool.Stat().AcquireCount()
		w.v.Tick()
		inTick += pool.Stat().AcquireCount() - before
		// ...
		rows, err := w.boundary()
		if err != nil {
			return err
		}
		if err := write(rows); err != nil {
			return err
		}
		all = append(all, rows...)
	}
$ podman exec -w /bench world-go go run ./cmd/rows -mode keep
rows: a day in the village, written down as it happens

  the world      seed 5, 12x8 cells, tick 901, 12 plants at 1517.5 grams
  the run        2000 ticks, from tick 901 to tick 2900
  the village    3 people out of configs/village.json, on a mailbox and a habit each
  the habit      feed below 25%, trade past 40%, top up below 90%
  the witness    one a person, in front of the mind; a run of one thing may
                 pause 10 ticks and go on for 100 before it is one memory
  the weights    off configs/importance.json, 1 to 10
  the writes     at the boundary, after v.Tick() returns and before the next is asked for

  the villager table, ORDER BY name, one transaction
  name      body law        home  founded  persona
  Ander        2 settled    9,5       901  Ander lives one cell east of Halla an...
  Halla        1 settled    8,5       901  Halla came to Firstlight before there...
  Mose         3 settled    8,6       901  Mose walks the edge of the standing p...

  what the day wrote
  ticks run                                        2000
  rows appended                                      88
  transactions, one a boundary that closed something       74
  connections taken at the boundary                  74
  connections taken inside a tick                     0

  the memory table, one stream a person, ORDER BY villager
  Ander        39 rows
  Halla        24 rows
  Mose         25 rows

  Halla's stream read back, ORDER BY tick, id
    id   tick  kind weight  text                                                       about
     1    901  saw       4  Ander came into sight, 0.8 cells off                       2
     2    901  saw       2  Mose came into sight, 0.8 cells off                        3
     8    999  did       7  handed Ander 50 parcels, 25.0 grams                        2
    11   1015  saw      10  Ander took 40 parcels out of Halla's hands, 20.0 grams     2,1
    13   1099  did       5  handed Ander 50 parcels, 25.0 grams                        2
    17   1173  did       4  handed Ander 37 parcels, 18.5 grams                        2
    25   1337  saw       2  Ander came into sight, 0.8 cells off                       2
    28   1348  did       3  went from 8,5 to 9,3                                       -
    35   1448  did       5  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    38   1548  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    41   1648  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    44   1748  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    47   1848  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    50   1948  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    53   2048  did       3  ate 50 mouthfuls on 9,3, 25.0 grams                        -
    56   2148  did       3  ate 50 mouthfuls on 9,3, 23.5 grams                        -
    63   2202  did       2  ate 27 mouthfuls on 9,3, 9.3 grams                         -
    62   2203  saw       2  Ander came into sight, 0.2 cells off                       2
    69   2307  did       5  handed Ander 50 parcels, 25.0 grams                        2
    73   2341  did       4  handed Ander 17 parcels, 8.5 grams                         2
    83   2776  did       1  ate 2 mouthfuls on 9,3, 1.0 grams                          -
    84   2781  did       1  went from 9,3 to 9,2                                       -
    86   2883  did       3  ate 50 mouthfuls on 9,2, 25.0 grams                        -
    87   2899  did       1  ate 8 mouthfuls on 9,2, 4.0 grams                          -

  24 rows read back, every one the row the village made, tick, kind, text,
  weight and about alike; 0 connections taken inside 2000 ticks, and every
  one of the 74 transactions opened between two of them

  2000 ticks in 53ms, 38016 ticks a second (measured here; yours will differ)
$ podman exec world-db psql -U world -d world -c "SELECT id, villager, tick, kind, weight, text FROM memory WHERE tick BETWEEN 990 AND 1020 ORDER BY tick, id;"
 id | villager | tick | kind | weight |                          text
----+----------+------+------+--------+--------------------------------------------------------
  8 | Halla    |  999 | did  |      7 | handed Ander 50 parcels, 25.0 grams
  9 | Ander    |  999 | saw  |      9 | Halla put 50 parcels in Ander's hands, 25.0 grams
 10 | Mose     |  999 | did  |      5 | ate 50 mouthfuls on 8,6, 25.0 grams
 11 | Halla    | 1015 | saw  |     10 | Ander took 40 parcels out of Halla's hands, 20.0 grams
 12 | Ander    | 1015 | did  |      7 | took 40 parcels from Halla, 20.0 grams
(5 rows)

Seventy-four transactions, one at every boundary that closed something, and every one of them opened between two ticks. The count inside ticks is nought, and the bench stops with a non-zero status if it is anything else, which the second exercise at the end of this chapter makes it do. The ids in Halla's stream have gaps in them because the three streams share one sequence; the database hands out one number a row across the whole village, in the order rows arrived, and nobody can write their own.

The query is both ends of a transfer in one place. Halla's row 8 and Ander's row 9 are one event written into two lives: she handed, he was handed, the same fifty parcels at the same tick, and the weights differ because the same thing is not worth the same to the two people it happened to. Row 9 names Ander in his own stream, and that is the reason the about column on it holds two identities and not one. The witness at the giving end knew the recipient exactly, and the boundary told the recipient's witness, so neither row was worked out from anything but the transfer itself.

The schema's refusals are worth one run, because a check that has never been watched refusing anything is a check nobody has tested.

▣ Build · stage 8 — eight rows the table has to refuse, and one it takes
$ podman exec -w /bench world-go go run ./cmd/rows -mode refuse
rows: eight rows the memory table has to refuse, and one it takes

  the row                              state  what the server said
  a fifth kind                         23514  new row for relation "memory" violates check constraint "memory_is_one_of_four_kinds"
  a weight of 11                       23514  new row for relation "memory" violates check constraint "memory_weighs_one_to_ten"
  a weight of 0                        23514  new row for relation "memory" violates check constraint "memory_weighs_one_to_ten"
  a did row with a speaker             23514  new row for relation "memory" violates check constraint "only_a_heard_memory_has_a_speaker"
  a heard row with no speaker          23514  new row for relation "memory" violates check constraint "only_a_heard_memory_has_a_speaker"
  a saw row drawn from memories        23514  new row for relation "memory" violates check constraint "only_a_thought_is_drawn_from_memories"
  a thought drawn from nothing         23514  new row for relation "memory" violates check constraint "only_a_thought_is_drawn_from_memories"
  a memory of nobody the village has   23503  insert or update on table "memory" violates foreign key constraint "memory_of_a_villager"
  a well-formed did row                -      taken, and rolled back: this was a question and not a write

  23514 is a CHECK refusing a value and 23503 is a foreign key refusing a
  name the villager table does not hold. Seven of the eight are refused by
  village.Memory.Check before a row is sent; the eighth is a question only
  the villager table can answer

Each of those eight is put to the server inside a transaction that is rolled back whatever the answer, so the run leaves the table as it found it. The two equality checks refuse in both directions, as they were written to: a seen row with a speaker and a heard row without one get the same refusal, naming the same constraint. The one row that is not a check is a name the villager table does not hold, and that is the foreign key doing what the Go check cannot, because whether Nobody is a villager is a fact about a table and not about a row.

A base, three adjustments and a clamp: the sum that weighs a row

The weight column is filled at append and never changed, so whatever fills it has to be right the first time, from what is known at the boundary, in arithmetic that gives the same answer on every machine. That rules out asking a model, and the reason is not a preference; it is measured below. It also rules out floats, because a weight that comes out 6.999999 on one processor and 7 on another is a weight that does not replay, and a reader cannot check a float against a table with a pencil.

What is left is a table of small integers and addition. Every kind and event pair has a base. Three things adjust it: the line names the villager it belongs to, the line is the first of its kind and event in that villager's life, and the grams that moved in it fall into a band. The sum is held between 1 and 10. The table is a file, so a reader who thinks a meal is worth more than a sighting changes one number and rebuilds.

▣ Build · stage 9 — the importance table, and the sum
{
  "base": {
    "saw": {"body": 2, "death": 7, "gift": 3, "taken": 4},
    "did": {"meal": 1, "walk": 1, "gift": 3, "take": 3}
  },
  "names_me": 2,
  "first": 2,
  "grams": [
    {"over": 5, "add": 1},
    {"over": 20, "add": 2},
    {"over": 50, "add": 3}
  ],
  "least": 1,
  "most": 10
}
// internal/village/memory.go

// Weigh scores one event. first is whether this villager has had a
// memory of this kind and event before; the stream knows and the
// table does not.
//
// It is integer arithmetic from the first line to the last. The band
// is chosen by comparing tenths of a gram against a threshold in
// whole grams, which is one integer against another, and everything
// after that is a sum and a clamp.
func (t Importance) Weigh(e Event, first bool) (Working, error) {
	base, ok := t.Base[e.Kind][e.What]
	if !ok {
		return Working{}, fmt.Errorf("village: the importance table has no row for a %s/%s memory", e.Kind, e.What)
	}
	w := Working{Base: base}
	if e.Named {
		w.Named = t.Named
	}
	if first {
		w.First = t.First
	}
	for _, b := range t.Grams {
		if e.Tenths >= b.Over*10 {
			w.Band = b.Add
		}
	}
	w.Sum = w.Base + w.Named + w.First + w.Band
	w.Weight = w.Sum
	if w.Weight < t.Least {
		w.Weight = t.Least
	}
	if w.Weight > t.Most {
		w.Weight = t.Most
	}
	return w, nil
}
$ go test ./internal/village/ -run 'TestAWeightIsABaseAndThreeAdjustmentsAddedUp|TestAWeightIsHeldBetweenTheEndsOfTheScale|TestTheGramsBandIsChosenOnWholeGramsAndNothingElse|TestAnEventTheTableHasNoRowForIsRefused|TestTheFirstOfItsKindIsWeighedOnceInALife|TestAFlatScorerWeighsEverythingAlike|TestTheShippedTableIsIntegersOnAScaleOfOneToTen' -v
=== RUN   TestAWeightIsABaseAndThreeAdjustmentsAddedUp
--- PASS: TestAWeightIsABaseAndThreeAdjustmentsAddedUp (0.00s)
=== RUN   TestAWeightIsHeldBetweenTheEndsOfTheScale
--- PASS: TestAWeightIsHeldBetweenTheEndsOfTheScale (0.00s)
=== RUN   TestTheGramsBandIsChosenOnWholeGramsAndNothingElse
--- PASS: TestTheGramsBandIsChosenOnWholeGramsAndNothingElse (0.00s)
=== RUN   TestAnEventTheTableHasNoRowForIsRefused
--- PASS: TestAnEventTheTableHasNoRowForIsRefused (0.00s)
=== RUN   TestTheFirstOfItsKindIsWeighedOnceInALife
--- PASS: TestTheFirstOfItsKindIsWeighedOnceInALife (0.00s)
=== RUN   TestAFlatScorerWeighsEverythingAlike
--- PASS: TestAFlatScorerWeighsEverythingAlike (0.00s)
=== RUN   TestTheShippedTableIsIntegersOnAScaleOfOneToTen
--- PASS: TestTheShippedTableIsIntegersOnAScaleOfOneToTen (0.00s)
PASS
ok  	theworld/internal/village	0.004s

Grams arrive at the scorer as tenths, already rounded once when the line was rendered, so the number the text says moved and the number the band is chosen on are the same number. e.Tenths >= b.Over*10 is an integer against an integer: 250 tenths against 200 for the twenty-gram band. The bands are walked in rising order and the last one that fits wins, so a table whose bands were written out of order would score wrong, and the reader refuses one. The two adjustments the table cannot decide on its own are passed in: whether the line names this villager is a fact the witness knew when it wrote the event, and whether it is the first of its kind is a fact only the stream knows, because the stream is the only thing that has seen every earlier row.

An event the table has no row for is an error and never a nought. A nought would be a weight of one after the clamp, and a row quietly weighed one because somebody added a kind of event and forgot the table is exactly the kind of wrong that shows up as a villager who never brings something up. The rows this chapter writes are the ones a world produces, saw and did, and the test pins that those two kinds have rows and that no kind outside the four does.

∑ Math Interlude — Halla's day, scored by hand

Take six rows of the stream above and the table, and add. The first row is Ander coming into sight on tick 901. A seen body has a base of 2. The line does not name Halla, so nothing is added for that. It is the first seen body in her life, so 2 more. No grams moved. 2 + 0 + 2 + 0 is 4, which is inside the scale, so the weight is 4. The second row is Mose coming into sight on the same tick, and the only difference is that it is no longer the first: 2 + 0 + 0 + 0, weight 2.

Row 3 is the first gift. A gift given has a base of 3, it does not name her, it is the first gift of her life, so 2, and 25.0 grams moved. Twenty-five grams is 250 tenths, and 250 is at least 200, the twenty-gram band, and less than 500, so the band adds 2. 3 + 0 + 2 + 2 is 7. Row 5 is the second gift of fifty parcels: 3 + 0 + 0 + 2, weight 5. Row 9 is the first meal, 50 mouthfuls and 25.0 grams: a meal has a base of 1, it is her first, and 250 tenths is the twenty-gram band again: 1 + 0 + 2 + 2, weight 5. Every meal after it with 25.0 grams in it is 1 + 0 + 0 + 2, weight 3, and the meal of 9.3 grams is 93 tenths, which clears 50 and not 200, so 1 + 0 + 0 + 1, weight 2.

Row 4 is the one that reaches the top. Ander took forty parcels out of her hands. Being taken from has a base of 4. The line names Halla, so 2. It is the first time she was taken from, so 2. Twenty grams is 200 tenths, the twenty-gram band, so 2. 4 + 2 + 2 + 2 is 10, and 10 is the top of the scale, so the weight is 10 with nothing held back. Had he taken fifty grams instead, the band would be 3 and the sum 11, and the weight would still be 10: the clamp is the one line in the arithmetic that is not addition, and it is there because the column holds one to ten and nothing else.

The same sums in symbols:

sum = base + named + first + band

weight = least, if sum < least; most, if sum > most; sum otherwise

Every letter is an integer, and the band is the largest add in the table whose over, in tenths, the tenths of the row are not below. That is all of it. A reader with the table and the stream can score the day on the back of the stream's own printout, and the bench prints its own working beside every row so that the two can be held together:

basethe table's number for this kind and event
namedthe table's names_me if the line names this villager, else 0
firstthe table's first if this is the first of its kind and event in this life, else 0
bandthe add of the largest grams band the row reaches, else 0
tenthsthe grams that moved, times ten, rounded to a whole number
least, mostthe two ends of the scale; 1 and 10 here
a < ba is less than b
▣ Build · stage 10 — the same day scored by the bench, and priced against a model
$ go run ./cmd/rows -mode score | tail -39 | head -29
  Halla's day, every row, and the sum that weighed it
     #   tick  kind event    base named first band  sum  weight
     1    901  saw  body        2     0     2    0    4       4
     2    901  saw  body        2     0     0    0    2       2
     3    999  did  gift        3     0     2    2    7       7
     4   1015  saw  taken       4     2     2    2   10      10
     5   1099  did  gift        3     0     0    2    5       5
     6   1173  did  gift        3     0     0    1    4       4
     7   1337  saw  body        2     0     0    0    2       2
     8   1348  did  walk        1     0     2    0    3       3
     9   1448  did  meal        1     0     2    2    5       5
    10   1548  did  meal        1     0     0    2    3       3
    11   1648  did  meal        1     0     0    2    3       3
    12   1748  did  meal        1     0     0    2    3       3
    13   1848  did  meal        1     0     0    2    3       3
    14   1948  did  meal        1     0     0    2    3       3
    15   2048  did  meal        1     0     0    2    3       3
    16   2148  did  meal        1     0     0    2    3       3
    17   2203  saw  body        2     0     0    0    2       2
    18   2202  did  meal        1     0     0    1    2       2
    19   2307  did  gift        3     0     0    2    5       5
    20   2341  did  gift        3     0     0    1    4       4
    21   2776  did  meal        1     0     0    0    1       1
    22   2781  did  walk        1     0     0    0    1       1
    23   2883  did  meal        1     0     0    2    3       3
    24   2899  did  meal        1     0     0    0    1       1

  88 rows over the village, every one added up again here and agreeing,
  0 of them held to an end of the scale
$ go run ./cmd/rows -mode score | tail -9
  the same day weighed by a model instead, at 3.3 ticks a round trip
  rows this village laid down                        88
  ticks of round trips to weigh them                290
  ticks the day had                                2000
  rows 24 people would lay down at this rate        704
  ticks of round trips to weigh those              2323
  the second village spends 1.2 days of round trips weighing one day of
  memories, and every one of the weights would be a number that does not
  replay. The table costs an addition a row and every weight replays

The six rows scored by hand in the Interlude are rows 1, 2, 3, 4, 5 and 9 of that table, and the columns agree with the pencil in every place. The bench does not take the village's word for the sum: it adds the four columns up again itself, clamps, and stops with a non-zero status if the weight the row carries is anything else, and it checks the base of every row against the file. Nothing in this day reached the clamp, so the last line says nought; row 4 landed on ten exactly.

The second block is why the model is not asked, said with a number. The first chapter of this volume put a stopwatch on a small-model round trip at this world's own pace and got 3.3 ticks, on this machine, at ten ticks a second; the bench takes that figure as a flag and prints it, because it is a fact about one afternoon and the arithmetic is not. Three people laid down 88 rows in 2000 ticks. Weighed one round trip a row, that is 290 ticks of asking for a day of 2000, which is a seventh of the day spent on the question of what the day was worth. Scale the rate to a village of twenty-four and the asking is 2323 ticks for a day of 2000: the village would need more than a day to weigh a day, before one plan had been made or one question answered, with a single worker and the smallest model this book holds. And the answers would be text a model wrote, which is the one class of thing this volume does not compare. Addition is free and it replays.

An age is now minus tick and nothing else

A stream that is going to be read from needs a way to say which rows are fresh, and the usual answer is the one everybody writes first: keep, on every row, the tick it was last brought to mind, and let freshness decay from there. A memory that keeps coming up stays fresh. It sounds right, it is how a person's own recall feels from the inside, and it is a write on every read. Every time a stream is consulted, the rows that came back are stamped, and a stamp is an UPDATE on the one table this book has now said three times never takes one.

The measure this world uses instead is the tick. A row's age is the tick the question is asked at minus the tick the row carries, and nothing else goes into it. The read writes nothing, so the answer at tick 2900 is the same answer at tick 2900 tomorrow, and the same on any machine holding the same rows. What is lost is exact and the page will not soften it: a memory that keeps being recalled does not thereby stay fresh. What is gained is that the stream is a record of what happened and not of what has been read.

⚠ Worked failure — a last-recalled column, added, run for a day, and taken back out

Here is the obvious version, built so that what it costs can be run and not described. One column, recalled, null until a row is read; one read that ranks a villager's rows by their distance from now, counting from the last recall when there has been one; and the write that read has to make. All three are in a file of their own in the store, and nothing this world runs calls any of them.

// internal/store/recalled.go

// Freshest is the read the obvious design makes: the k rows of one
// villager's stream whose last recall, or whose tick if nobody has
// recalled them, is nearest to now. Ties break on weight, heavier
// first, and then on id.
func (d *DB) Freshest(ctx context.Context, name string, now, k int) ([]Fresh, error) {
	const q = `
SELECT id, tick, $2 - coalesce(recalled, tick) AS age
  FROM memory
 WHERE villager = $1
 ORDER BY age, weight DESC, id
 LIMIT $3`
	// ...
}

// Touch is the write every one of those reads has to make: the rows
// just retrieved are stamped with the tick they were retrieved at. It
// hands back how many rows the server rewrote.
func (d *DB) Touch(ctx context.Context, ids []int64, now int) (int64, error) {
	const q = `UPDATE memory SET recalled = $1 WHERE id = ANY ($2)`
	tag, err := d.pool.Exec(ctx, q, now, ids)
	if err != nil {
		return 0, fmt.Errorf("store: stamping %d rows as recalled: %w", len(ids), err)
	}
	return tag.RowsAffected(), nil
}

The run wants an empty database, because it founds the village a second time and the villager table refuses a name it already has. The pair of commands that empties one is the same pair the last two volumes used, and it is here and not assumed:

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go go run ./cmd/rows -mode recalled
rows: a last-recalled column, added, run for a day, and taken back out

  the world      seed 5, 12x8 cells, tick 901, 12 plants at 1517.5 grams
  the run        2000 ticks, from tick 901 to tick 2900
  the village    3 people out of configs/village.json, on a mailbox and a habit each
  the habit      feed below 25%, trade past 40%, top up below 90%
  the witness    one a person, in front of the mind; a run of one thing may
                 pause 10 ticks and go on for 100 before it is one memory
  the weights    off configs/importance.json, 1 to 10
  the recall     every 40 ticks, the 8 freshest of Halla's rows are read,
                 and freshness counts from the last recall if there was one

  ALTER TABLE memory ADD COLUMN recalled bigint
  the memory table now has 10 columns, and the tenth is null on every row

  what a day of reading did to the table
  rows the day wrote                                 88
  recalls, one every 40 ticks                        50
  rows recalled                                     363
  rows the server rewrote                           363
  different rows recalled all day                    22
  recalls that returned the recall before's rows       31

  a row that is recalled is stamped with the tick of the recall, so at the
  next recall it is fresher than every row written before it and older only
  than what was written since. What it was worth has no say in that

  the 8 freshest of Halla's rows at tick 2900, asked two ways
  from the last recall          from the tick it happened at
    id   tick    age     id   tick    age
    11   1015      0     87   2899      1  <- not the same row
     8    999      0     86   2883     17  <- not the same row
    13   1099      0     84   2781    119  <- not the same row
    35   1448      0     83   2776    124  <- not the same row
    69   2307      0     73   2341    559  <- not the same row
     1    901      0     69   2307    593  <- not the same row
    17   1173      0     62   2203    697  <- not the same row
    86   2883      0     63   2202    698  <- not the same row

  8 of the 8 places hold a different row. The left column is a function of
  every read anybody has ever made of this stream; the right is a function
  of the tick alone, and asking it again at tick 2900 gives it again

  ALTER TABLE memory DROP COLUMN recalled
  the memory table has 9 columns again, 88 rows, and the migrations run
  again apply 0 and skip 8: the ledger never knew the column was there,
  which is the other half of why it cannot stay

Start from the symptom, which is the left-hand column. Asked for the eight freshest of Halla's rows at tick 2900, the table with the column on it hands back eight rows at age nought, and not one of them is a row from the last hundred ticks. Row 1 is there, the sighting on the tick she was founded, two thousand ticks ago. Row 11 is the top: a weight of ten, taken from her at tick 1015. The right-hand column is the same question put to the same rows with the column ignored, and it is the eight newest things that happened to her, in the order they happened.

Work back. The recall at tick 2900 found seven of those eight at age forty, because the recall at tick 2860 had stamped them 2860, and found row 86 at age seventeen, because it was written since. Forty ticks old is fresher than every row written before tick 2860, and seventeen is fresher than that, so the eight came back and were stamped 2900, and the print above is the table as that last stamp left it. Thirty-one of the fifty reads in that day returned exactly what the read before them returned, and twenty-two different rows were ever recalled out of the twenty-four the day wrote. A read makes what it read fresh, so it reads it again, so it stays fresh. The stream has stopped being consulted about the day and is being consulted about its own consultations, and the weight a row carries has no say in any of it, because a nought in the first sort key beats a ten in the second.

And the cost the schema cares about is in the middle block. Fifty reads rewrote 363 rows of a table that was written 88 times. A stream read every forty ticks is rewritten four times more often than it is written to, every rewrite is an UPDATE on the table whose whole design is that it takes none, and every one of them is invisible to the ledger: the migrations run again after the column is dropped and report that nothing changed, because nothing they know about did. The cause is one sentence. A freshness that decays from the last recall is a function of who has read the stream, so the same question at the same tick has different answers on different days, and none of them replays. The column is dropped, the table has nine columns again, and the one UPDATE in this module stays behind the function the law's test allows it and nothing else calls.

A memory's road from a tick to a row, and the two clocks it never uses Three labelled boxes run left to right. The first, inside the tick, holds the witness noting the entry the mind answered and, for a transfer, the body it would cross to, with no socket and no write. The second, the boundary between two ticks, confirms against the body, closes a run into one event, weighs it as base plus named plus first plus band clamped to one to ten, and opens one transaction. The third is the table: one row a memory, nine columns, INSERT only, id generated always. A band along the bottom names what never happens: an UPDATE, a DELETE, now(), a write inside a tick, and a weight revised. FROM A TICK TO A ROW inside the tick the mind answers the witness notes the act who a parcel crosses to nine cells walked no socket, no write the boundary confirm against the body close a run into an event base+named+first+band held to 1..10 one transaction the table one row a memory nine columns INSERT, and only INSERT id generated always tick, never now() WHAT NEVER HAPPENS TO A ROW UPDATE DELETE now() a write inside a tick a weight revised after the tick it was decided on a freshness that counts from the last time anybody read the row
Figure 92.1 — the three places a memory passes through and what each is allowed to do. The tick half may read and note; the boundary may confirm, weigh and write; the table may take a row and hand it back. Everything in the bottom band is something one of the three could easily do, and the chapter's runs are the reasons none of them does.

Why records ignore their readers

Take the village away and what is left is a rule about any table that is going to be reasoned about later: a fact written down should not change because somebody looked at it. The last-recalled column breaks that in the most ordinary way there is. It is not malicious and it is not even wrong about people, who do forget what they never revisit. What it does is fold the history of the readers into the history of the world, so that two copies of the same stream, read on two different schedules, stop agreeing about what happened. Anything downstream of such a table inherits that: a ranking, a prompt, a decision, a test. The append-only law is what makes every one of those a function of the rows and the tick, and so a thing that can be run again.

The second thing that generalizes is where a fact is decided. A weight decided at append, from what was known at append, by arithmetic that replays, is a fact about the row; a weight that could be revised later is an opinion about the row that happens to be stored in it. The first can be checked by anybody with the table and a pencil, as the Interlude does, and the second can only be believed. Deciding at append costs nothing here because a table of integers is enough to decide with. Where a decision does need something expensive, the same rule says to pay for it once, at the moment of writing, and never to make the read side pay for it again.

The third is about where a write is allowed to be. Everything a memory records happens inside a tick, and the reflex is to write it there, at the point where the information is in hand. The reflex is what the count of connections inside ticks exists to refuse. What this chapter does instead is the pattern the whole module runs on: note the fact inside the budget, in memory, in a form that cannot be wrong later, and carry it across to the one place where a socket is allowed. The awkward part is never the carrying. It is deciding which facts have to be taken at the moment they are true, which is why the witness does one search inside the tick and everything else outside it.

✓ Checkpoint — nine columns, one sum, and no write on a read
  • Name the nine columns of a memory, say which two are tied to one kind each and how the schema ties them, and say what a seen row with a speaker on it gets from the server.
  • Given the table on this page, score a row by hand: a villager's first gift of twelve parcels, 6.0 grams, to a neighbour. Then score the same event as the neighbour saw it, and say why the two weights differ.
  • Explain why the body a parcel crosses to is taken inside the tick by the witness and not worked out at the boundary from the world as it stands, and name the two numbers that can change between those two moments.
  • Say what Owed and Busy tell the boundary about a choice the mind made, for a choice the world refused, a choice made while an action was under way, and a fresh two-tick action.
  • Work out how many rows a day of fifty reads of eight rows rewrites under the last-recalled design, and say why the migrations report nothing changed after the column is dropped.
  • Given one villager's 24 rows and the pace at which a small model answers, work out how many ticks weighing them one call each would cost, and say which of the two numbers in that product is a fact about the machine.
⚡ Exercises — try first, then reveal
Exercise 1 — take the table away. The scorer has a setting that weighs every memory alike. Run the day with every row weighed 5 and predict, before you look, which eight rows a reader with room for eight would take, and where the row weighed ten in the shipped run ends up.

The eight newest, in tick order, and the row that was worth ten is nowhere:

$ go run ./cmd/rows -mode day -flat 5 | tail -19 | head -11
  the 8 heaviest, weight first and then the newest, which is the order
  a reader of this stream with room for 8 rows would take them in
     #   tick  kind weight  text
    24   2899  did       5  ate 8 mouthfuls on 9,2, 4.0 grams
    23   2883  did       5  ate 50 mouthfuls on 9,2, 25.0 grams
    22   2781  did       5  went from 9,3 to 9,2
    21   2776  did       5  ate 2 mouthfuls on 9,3, 1.0 grams
    20   2341  did       5  handed Ander 17 parcels, 8.5 grams
    19   2307  did       5  handed Ander 50 parcels, 25.0 grams
    17   2203  saw       5  Ander came into sight, 0.2 cells off
    18   2202  did       5  ate 27 mouthfuls on 9,3, 9.3 grams

With every weight the same, the first sort key decides nothing and the second decides everything, so the ranking is recency wearing a weight column. Two mouthfuls at tick 2776 outrank forty parcels taken out of her hands at tick 1015, because the taking is older. Nothing in the run failed and the audit passed, because a flat five is a legal weight on every row; what went wrong is only visible in what a reader would be handed, which is the whole reason the column exists.

Exercise 2 — write from inside the tick. The bench has a flag that moves the write into the creature phase, where the information is freshest and where it is not allowed to be. Run it against an empty database and say what the audit prints and what the program exits with.

The day runs, the rows are written and read back the same, and the last check stops the run:

$ podman exec -w /bench world-go go run ./cmd/rows -mode keep -inside 2>&1 | tail -3
  NO: 73 connection(s) were taken inside a tick, and the rule is none
rows: 1 thing(s) did not add up
exit status 1

Seventy-three and not seventy-four, because the phase runs before the world's clock advances and the last boundary's rows are written after the loop by the same code either way. Every row is correct. Nothing about the table can tell the two runs apart, and that is the point of counting connections rather than rows: the rule this world keeps is about when a socket is touched, and only a measurement taken across the tick can see it.

Exercise 3 — put the UPDATE to the server yourself. Nothing in the module updates a memory, and the test walks the store's source to say so. The server was never told. Change the weight of the row weighed ten by hand, with psql, and say what the server answers and what that means about where the law lives.

The server takes it, because a schema declares what a row may hold and not who may change one:

$ podman exec world-db psql -U world -d world -c "UPDATE memory SET weight = 1 WHERE id = 11;"
UPDATE 1
$ podman exec world-db psql -U world -d world -c "SELECT id, tick, kind, weight, text FROM memory WHERE id = 11 ORDER BY id;"
 id | tick | kind | weight |                          text
----+------+------+--------+--------------------------------------------------------
 11 | 1015 | saw  |      1 | Ander took 40 parcels out of Halla's hands, 20.0 grams
(1 row)

The row now says one and the stream the village holds in memory still says ten, and nothing anywhere will ever reconcile them, because nothing anywhere expects them to differ. That is what a table that takes an UPDATE costs: two truths about one moment, and no way to say which is the world's. The law is in the same place the chronicle's is, in the program and in the test that reads the program, and the exercise shows the edge of it exactly: a person at a prompt can break it, and this world does not run with a person at a prompt. Restart the database before the next run on this page, because the row weighed one is not a row this chapter wrote.

So Halla has twenty-four rows, Ander thirty-nine and Mose twenty-five, every one of them weighed on the tick it happened and none of them ever to be touched again, in a table that holds two people's accounts of the same fifty parcels side by side. What none of them has is a way to bring any of it to mind. Twenty-four rows can be read whole. A stream after a week of ticks cannot, and a person about to decide something has room for a handful, so which handful is a question with three numbers in it: a weight, an age against the world's clock, and how near a row is to whatever is being asked.