Chapter 93 — What Comes to Mind

Twenty-four rows and room for eight

Halla's day is twenty-four rows and the space a question is answered from holds eight of them. Eight is a number in a file, k in configs/thought.json, and the reason it is small has been on the page since the first chapter of this volume: every line handed to a model is paid for in tokens and in the seconds a round trip takes, and a stream after a week of ticks is thousands of rows, most of them meals. So a villager about to be asked something needs a rule for which handful, and the rule has to be arithmetic over numbers the rows already hold, because it runs at the boundary of a tick, over every villager in the village, and a rule that asked a model which memories mattered would be paying for a round trip to decide what to pay for a round trip about.

A memory scores three terms, each put onto a scale from nought to one, each multiplied by a weight read out of a file, and added; ties break on the memory's identity; the top k come to mind; and the identities those k name, the identities in sight this tick and the villager itself are the whole of what a prompt rendered from them may name.

Three rules suggest themselves and each one is wrong on its own. The newest eight is the last chapter's first exercise: every weight flattened to five, and forty parcels taken out of her hands at tick 1015 outranked by two mouthfuls at tick 2776. The heaviest eight is what the last chapter printed instead, and it has the opposite fault: a meal at tick 1448 that happened to be her first outranks the meal she finished a moment ago, and will go on outranking it for the rest of her life, because a weight never changes. The most similar eight, by the cosine the last volume built, is the third rule, and it has a fault of its own kind. Measured on this stream, the cosine between a plain question and the twenty-four lines runs from 0.38 to 0.58: on a scale of nought to one, the question tells a fifth of the story, and a rule that listened to it alone would be sorting on a fifth of a scale.

Each term is one of the three rules above, put on the same scale as the others so that none of them can decide alone. Recency is a half-life decay on the memory's age in ticks, with the half-life in the same file. Importance is the stored weight over ten. Relevance is the cosine between the question's vector and the memory's, clamped at nought so that a line pointing away from the question scores nothing and never less than nothing. The three are added. The obvious alternative is to multiply them, and the reason not to is a run on this page and not a sentence: a product hands any one term a veto, and the term that uses it is the one that is nearly nought for every memory older than a few half-lives, which is most of a life.

Nothing in the arithmetic is a model's. The question's vector is the one thing on this page that a model produced, and it is read off a committed file, so that every score, every rank and every set printed here is the same on any machine that runs the code. The vectors of the memories themselves live in a table of their own, beside the memory table and welded to the model that filled it, for the reason the last volume gave when it put a vector beside the chronicle.

A ninth file, keyed on the memory and the digest

The schema is a directory of numbered files, applied in the order of their numbers and never edited once one has run anywhere. The eighth file made the villager and the memory tables, and this is the ninth. It makes one table, and the table is the eighth table's design with the chronicle's entry replaced by the memory's identity: a row of numbers, the digest of the weights that made it, the tag a person reads, and a primary key on the memory and the digest together.

▣ Build · stage 1 — the ninth migration, whole

-- migrations/0009_memory_vector.sql

-- What a model turned one memory into: one row a memory, a row a
-- model. It is the eighth table's design applied to the memory
-- table, for the eighth table's reason.
--
-- A memory is weighed once and never touched again, so a vector for
-- it cannot be a column on it: the column would be filled in after
-- the row was written, and that is an UPDATE of a table that takes
-- none. So the numbers go in a row of their own, keyed by the memory
-- they are about.
--
-- digest is half the key, exactly as it is beside the chronicle. A
-- row of numbers is welded to the weights that made it, two models'
-- rows are not comparable in any way at all, and a table that does
-- not say which model filled it is a table nobody can safely fill
-- again. model is the tag a person reads; digest is what decided the
-- numbers.
--
-- Everything here is derived. Drop the table and nothing is lost
-- that the memory table does not still hold, which is why a snapshot
-- does not take it and why there is no wall clock on it.
CREATE TABLE memory_vector (
    memory bigint      NOT NULL,
    digest text        NOT NULL,
    model  text        NOT NULL,
    v      vector(768) NOT NULL,

    CONSTRAINT memory_vector_key PRIMARY KEY (memory, digest),

    -- Which of the two tables is the derived one, written down where
    -- the server keeps it: there is no vector for a memory that was
    -- never written. Nothing in this world deletes a memory, so the
    -- cascade is a statement about the table and never a road
    -- anything takes.
    CONSTRAINT memory_vector_of FOREIGN KEY (memory)
        REFERENCES memory (id) ON DELETE CASCADE,

    -- A digest is a sha256 written out in lower case, and a pin that
    -- is not one pins nothing.
    CONSTRAINT memory_vector_is_pinned CHECK (digest ~ '^[0-9a-f]{64}$'),
    CONSTRAINT memory_vector_names_its_model CHECK (model <> '')
);

$ go run ./cmd/recall -mode plan
recall: the ninth migration, and not one statement of it run

  n    file                       sha256, first 16     bytes  what it makes
  0009 0009_memory_vector.sql     73593936d6b4e047      1890  memory_vector

  the first eight files are chapter 92's to the byte, and this is the ninth

  memory_vector, 4 columns, as 0009_memory_vector.sql declares them
    column       type                                 null
    memory       bigint                               no
    digest       text                                 no
    model        text                                 no
    v            vector(768)                          no
    and 2 checks: memory_vector_is_pinned, memory_vector_names_its_model

  the width, written in two places and held together by a test
    declared in     0009_memory_vector.sql
    held in Go as   store.Dim
    both of them    768

  every row in it can be made again from a memory and a model, so a
  snapshot does not take it and dropping it loses nothing the memory
  table does not still hold

Four columns, and the argument for each is the one the eighth table made, so it is made here in one line apiece. memory is what the row is about and the only way back to the words. digest is half the key, because the day the embedding model changes is a day this table has to survive, and a table keyed on the memory alone cannot be asked which rows are about the old weights. model is the tag a person reads. v is the row, at the width the model answers at, which is the last volume's 768 and not a setting. There is no timestamp and no column saying how wide a row is, for the same two reasons as before: the type says the width, and a derived row has no when.

The width lives in two places, this file and store.Dim, and the last volume's test that holds those two against each other now finds two vector columns declared and checks both. The bench prints the ninth file's own line and nothing about the schema as a set: a count over every file the build holds moves every time a later chapter adds one, and this volume adds three.

ON DELETE CASCADE needs a sentence, because the last chapter's law is that nothing deletes a memory. The clause is still here, and it is here for what it says rather than for what it does: written where the server keeps it, it records which of the two tables is the derived one. No statement in this world will ever make it run.

The database is killed and started so that it is properly empty, and the migrations are run twice in one go, so that the second block cannot have come from a different afternoon.

▣ Build · stage 2 — applied, then not applied, and described back

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go go run ./cmd/recall -mode up -twice
recall: bringing a database up to the schema this build holds

  the first run   0009 0009_memory_vector.sql  applied; the run changed the database
  the second run  0009 0009_memory_vector.sql  already applied, and its bytes still agree; the run changed nothing at all

  memory_vector  4 columns, 0 rows, and one of the columns is 768 numbers
  the vector extension, which 0007 asked for   0.8.6

$ podman exec world-db psql -U world -d world -c '\d memory_vector'
             Table "public.memory_vector"
 Column |    Type     | Collation | Nullable | Default
--------+-------------+-----------+----------+---------
 memory | bigint      |           | not null |
 digest | text        |           | not null |
 model  | text        |           | not null |
 v      | vector(768) |           | not null |
Indexes:
    "memory_vector_key" PRIMARY KEY, btree (memory, digest)
Check constraints:
    "memory_vector_is_pinned" CHECK (digest ~ '^[0-9a-f]{64}$'::text)
    "memory_vector_names_its_model" CHECK (model <> ''::text)
Foreign-key constraints:
    "memory_vector_of" FOREIGN KEY (memory) REFERENCES memory(id) ON DELETE CASCADE

The first run applies the ninth file, the second finds it applied and its checksum agreeing, and the extension the seventh file installed reports the version the last volume pinned, which is compared for the reason it was compared then: the image is pinned by digest, so a version that moved means the pin did. The server's own description of the table is the file's, constraint for constraint, and it is read back through psql and not through the program that made it, because the program will say what it meant to make.

⚙ Tool — psql, and which container each command runs in

Every podman exec on this page runs on the closed bridge the last volume built: the Go commands inside world-go, where the module is copied to /bench, the model calls to world-lm over that bridge, and the psql commands inside world-db, which is the only container that has it. Its reference is at postgresql.org/docs under psql.

The store gains the table's reads and writes and nothing else. Three of them matter to this page: the write, which is the eighth table's COPY in the text format with the memory's identity where the chronicle's entry was; the backfill's question, which is which memories have no row from this model yet; and the read of one villager's vectors, so that a ranking can be computed in Go over exactly the rows the table holds.

▣ Build · stage 3 — the eleventh table's three statements

// internal/store/recall.go

// MemoryVector is one row of the eleventh table: which memory it is
// about, which model produced it, and the numbers.
type MemoryVector struct {
	Memory int64
	Digest string
	Model  string
	V      []float32
}

// Embedded writes rows into the eleventh table, all of them in one
// COPY in the text format, which is the road the eighth table's rows
// take and for the same reason: the driver has never heard of the
// type, and this volume adds no dependency.
//
// It is called at the daemon's boundary and never inside a tick,
// like everything else in this package.
func (d *DB) Embedded(ctx context.Context, rows []MemoryVector) (int64, error) {
	// ...
	const q = `COPY memory_vector (memory, digest, model, v) FROM STDIN`
	tag, err := conn.Conn().PgConn().CopyFrom(ctx, strings.NewReader(b.String()), q)
	// ...
}

// Unembedded is the backfill's question asked of the memory table:
// which memories have no row from this model yet, oldest written
// first, at most limit of them.
//
// The model is named by its digest and not by its tag, for the
// reason the eighth table's Waiting names it so: the question is
// about the weights.
func (d *DB) Unembedded(ctx context.Context, digest string, limit int) ([]village.Memory, error) {
	// ...
	const q = `
SELECT m.id, m.villager, m.tick, m.kind, m.text, m.weight, m.about, m.said_by, m.drawn_from
  FROM memory m
 WHERE NOT EXISTS (SELECT 1 FROM memory_vector v
                    WHERE v.memory = m.id AND v.digest = $1)
 ORDER BY m.id
 LIMIT $2`
	// ...
}

// Vectors reads one villager's vectors from one model back, ORDER BY
// memory, so that a ranking can be computed in Go over exactly the
// rows the table holds. The numbers come back as the literal the
// server prints and are parsed here, because the driver cannot scan
// the type.
func (d *DB) Vectors(ctx context.Context, name, digest string) ([]MemoryVector, error) {
	const q = `
SELECT v.memory, v.digest, v.model, v.v::text
  FROM memory_vector v
  JOIN memory m ON m.id = v.memory
 WHERE m.villager = $1 AND v.digest = $2
 ORDER BY v.memory`
	// ...
}

$ go test ./internal/store/ -run 'TestTheEleventhTableIsInTheListAndMadeByTheNinthFile|TestAMemoryVectorOfTheWrongWidthOrNoModelNeverReachesTheServer|TestAVectorLiteralReadsBackAsTheSameBits' -v
=== RUN   TestTheEleventhTableIsInTheListAndMadeByTheNinthFile
    recall_test.go:46: 0009_memory_vector.sql makes memory_vector, keyed on the memory and the digest, and the list names it
--- PASS: TestTheEleventhTableIsInTheListAndMadeByTheNinthFile (0.00s)
=== RUN   TestAMemoryVectorOfTheWrongWidthOrNoModelNeverReachesTheServer
    recall_test.go:65: a row of the wrong width is refused naming both widths, and a row with no model behind it is refused too
--- PASS: TestAMemoryVectorOfTheWrongWidthOrNoModelNeverReachesTheServer (0.00s)
=== RUN   TestAVectorLiteralReadsBackAsTheSameBits
    recall_test.go:90: 8 numbers written as the literal the server takes and read back from it, the same bits
--- PASS: TestAVectorLiteralReadsBackAsTheSameBits (0.00s)
PASS
ok  	theworld/internal/store	0.004s

Vectors is the one statement of the three the eighth table did not need, and the reason is where the arithmetic runs. Beside the chronicle, the server computed the angle and the Go computed it again to check. Here the angle is one of three terms and the other two are a subtraction in ticks and a division by ten, so the whole score is computed in Go at the boundary over rows read back in a fixed order, and the server is asked for the rows and never for a ranking. The vector comes back as the literal the server prints and is parsed by the same rule that wrote it: the shortest string that is the same float32, so the third test's eight awkward numbers go out and come back as the same bits.

The first test is the store's table list growing in the same change as the file, which has been the rule since the seventh file, and it checks the four clauses that make this table the eighth table's twin: the key on the pair, the foreign key, the pattern the digest has to match and the model that has to be named. The second is the refusal before a socket is wanted, naming both widths.

Eighty-eight rows and five questions, written down once

Two files have to exist before any score can be printed, and both are the kind of file the last volume introduced: written down once, from something that ran, and carried by the build forever after. The first is the stream itself. The last chapter ran a day of two thousand ticks over three people and printed what one of them remembered of it; the same day, run again here, is dumped whole, one row a line, with the identity each row was given.

▣ Build · stage 4 — the day, dumped and held against the copy the build carries

// exemplars/exemplars.go
//go:embed chronicle.jsonl hero.header.json small.header.json vectors.json memories.jsonl memory-vectors.json
var Files embed.FS

// Memories is the name of the committed memory stream inside Files:
// one day of one village, every row the day laid down, dumped once
// from a run of it. MemoryVectors is the name of the rows a model
// turned those memories into, on the same terms as Vectors.
const (
	Memories      = "memories.jsonl"
	MemoryVectors = "memory-vectors.json"
)

// Memory is one row of the memory stream as the dump carries it: the
// nine columns of the table, with the two that belong to one kind
// each left out when they are empty.
//
// The identity is the number the database hands out when the same
// day is written to it, and the dump carries it because a ranking
// breaks its ties on it and a grounding set is checked against it.
type Memory struct {
	ID        int64    `json:"id"`
	Villager  string   `json:"villager"`
	Tick      int      `json:"tick"`
	Kind      string   `json:"kind"`
	Text      string   `json:"text"`
	Weight    int      `json:"weight"`
	About     []uint64 `json:"about"`
	SaidBy    uint64   `json:"said_by,omitempty"`
	DrawnFrom []int64  `json:"drawn_from,omitempty"`
}

$ go run ./cmd/recall -mode dump
recall: a day in the village, written out as the stream this chapter scores

  the world      seed 5, 12x8 cells, tick 901, 12 plants at 1517.5 grams
  the run        2000 ticks, from tick 901 to tick 2900
  the village    3 people out of configs/village.json, on a mailbox and a habit each
  the habit      feed below 25%, trade past 40%, top up below 90%
  the witness    one a person, in front of the mind; a run of one thing may
                 pause 10 ticks and go on for 100 before it is one memory
  the weights    off configs/importance.json, 1 to 10

  the day        88 rows over 3 people, the last at tick 2900
  numbered       1 to 88, in the order the boundary appended them
  written to     recall93-memories.jsonl, 10597 bytes, one row a line

  against the copy this build carries
    exemplars/memories.jsonl         9559076e342bbfff    10597 bytes
    recall93-memories.jsonl          9559076e342bbfff    10597 bytes

  row for row, on id, villager, tick, kind, text, weight and about
    rows in the committed stream       88
    rows this run laid down            88
    identical                          88
  every row the committed stream holds is a row this run made again, and
  every score in this chapter is arithmetic over those 88 rows

Eighty-eight rows, 10,597 bytes, the same bytes as the committed copy and the same row for row. The identities are the interesting column. The dump is made with no database anywhere, so nothing handed the numbers out; the bench counts from one in the order the boundary appended the rows, which is the order the boundary sends them, which is the order an identity column declared GENERATED ALWAYS numbers them in. That is a claim about a server and it is checked against one further down this page, where the same eighty-eight rows are put to the memory table one at a time and the identity each comes back with is held against the dump's.

The kind of file this is deserves one sentence. A stream dumped from a run is a different thing from the founding document, which an operator wrote before anything happened, and a different thing from the vector fixture below, which a model produced: it is the engine's own output, replayable to the byte, and it is committed so that the scores on this page are about a stream a reader can see whole rather than one that has to be run for.

The second file is the one that has to exist. A model turns each of those eighty-eight lines into a row of 768 numbers, and the row it turns out depends on the machine, the build and the batch it was asked in, so no score computed from a live answer is a score anybody else can get. The last volume's answer holds unchanged: ask once, write it down, and compute everything afterwards over the bytes that were written. The fixture also carries the questions this chapter puts to the stream, five of them, written by the engine's author in plain English and chosen to bring different parts of a day to mind.

▣ Build · stage 5 — every row and every question, turned into numbers once

// cmd/recall/fix.go

// asked is the questions this chapter puts to a stream, in plain
// English and in the order the fixture keeps them. They are written
// by the engine's author and not by a model, and each is chosen to
// bring a different part of a day to mind.
var asked = []string{
	"Who took something out of my hands?",
	"Where did I last eat, and how much?",
	"What have I handed to other people?",
	"Who is standing near me?",
	"What have I given Ander?",
}

$ podman exec -w /bench world-go go run ./cmd/recall -mode fix
recall: the stream turned into numbers, once, and written down

  the model          nomic-embed-text:v1.5
  pinned at          970aa74c0a90ef74
  the server         world-lm:11434, waiting up to 10m0s for an answer
  the corpus         every one of the stream's 88 rows, and 5 questions

  every answer was 768 numbers wide, which is the model's own and not a setting
  written to recall93-vectors.json, 797066 bytes

  against the copy this build carries
    exemplars/memory-vectors.json    b1fd66b539b85626     797066 bytes
    recall93-vectors.json            b1fd66b539b85626     797066 bytes

  the 71424 numbers this run was given, against the committed ones, at a tolerance of 1e-06
    not one of them differs at all

Ninety-three rows of 768 numbers, 797,066 bytes on disk, which is about seventy-five times the size of the stream they describe. The file records which model produced it and at which digest, and carries its own label so that a reader who finds it in ten years knows what it is without finding this page: Yours will differ: a different model, build, backend or machine answers with different numbers. The last two blocks are the check that keeps the fixture honest, and they are the check the last volume's fixture got: the mode does the work again against the live server, writes its own file under a name of its own, and holds it against the copy the build carries. Here the two came out byte for byte identical, which is a fact about this machine on this afternoon; the guarantee that travels is the tolerance line, that no number moved by more than a millionth. Not one of those 71,424 numbers is printed on this page, and none will be.

Every one of the eighty-eight rows is embedded and not a chosen few, because a retrieval over a stream half of which has vectors is a retrieval that is silently about half a stream; the scoring code refuses a row with no vector rather than score it at nought, and a test on this page holds it to that.

Three terms between nought and one

With both files carried, the arithmetic is a hundred lines in the village package, and nothing in it opens a connection, reads a clock or calls a model. The settings come out of configs/thought.json, which has a block for retrieval and room beside it; the block holds the half-life in ticks, the three weights and k, and the bench prints all five from the file on every run.

▣ Build · stage 6 — a decay, a division, an angle and a sum

{
  "retrieval": {
    "half_life": 250,
    "recency": 1.0,
    "importance": 1.0,
    "relevance": 3.0,
    "k": 8
  }
}

// internal/village/recall.go

// Retrieval is the block of configs/thought.json a memory is brought
// to mind by: the half-life in ticks, the three weights, and how many
// memories a question is answered with.
type Retrieval struct {
	HalfLife   int     `json:"half_life"`
	Recency    float64 `json:"recency"`
	Importance float64 `json:"importance"`
	Relevance  float64 `json:"relevance"`
	K          int     `json:"k"`

	// Cos is the angle between two rows, handed in by the caller;
	// it is lang.Cos everywhere this world runs. A Retrieval with no
	// cosine refuses to score rather than score them all at nought.
	Cos func(a, b []float32) (float64, error) `json:"-"`
}

// Decay is 0.5 raised to age over half: one for a memory of this
// tick, a half for one a half-life old, a quarter for two.
func Decay(age, half int) float64 {
	return math.Pow(0.5, float64(age)/float64(half))
}

// Score fills in the score from the three terms already on the row.
//
// The sum is the design. product is the counterfactual, kept behind
// a flag so that what a product costs can be run: with it, the three
// terms are multiplied, the weights play no part, and one term at
// nought takes the whole score to nought whatever the other two say.
func (r Retrieval) Score(s *Scored, product bool) {
	if product {
		s.Score = s.Recency * s.Importance * s.Relevance
		return
	}
	s.Score = r.Recency*s.Recency + r.Importance*s.Importance + r.Relevance*s.Relevance
}

// Retrieve scores every row against the question's vector at tick now
// and hands them all back, best first, ties on the memory's identity
// ascending so that the order is total and a ranking replays.
//
// vec is the memory's vector by its identity. A row with no vector is
// refused rather than scored at nought relevance: a ranking over a
// stream half of which has been embedded is a ranking that is
// silently about half a stream. A row from after now is refused too,
// because a memory of what has not happened yet is not a memory.
func (r Retrieval) Retrieve(now int, rows []Memory, vec map[int64][]float32, q []float32, product bool) ([]Scored, error) {
	if r.Cos == nil {
		return nil, fmt.Errorf("village: a retrieval with no cosine to score relevance by")
	}
	out := make([]Scored, 0, len(rows))
	for _, m := range rows {
		v, ok := vec[m.ID]
		if !ok {
			return nil, fmt.Errorf("village: %s: memory %d has no vector to be scored by", m.Villager, m.ID)
		}
		if m.Tick > now {
			return nil, fmt.Errorf("village: %s: memory %d is at tick %d, which is after tick %d", m.Villager, m.ID, m.Tick, now)
		}
		s := Scored{Memory: m, Age: now - m.Tick}
		s.Recency = Decay(s.Age, r.HalfLife)
		s.Importance = float64(m.Weight) / 10
		cos, err := r.Cos(q, v)
		if err != nil {
			return nil, fmt.Errorf("village: %s: memory %d: %w", m.Villager, m.ID, err)
		}
		s.Cos = cos
		s.Relevance = math.Max(0, cos)
		r.Score(&s, product)
		out = append(out, s)
	}
	sort.SliceStable(out, func(i, j int) bool {
		if out[i].Score != out[j].Score {
			return out[i].Score > out[j].Score
		}
		return out[i].ID < out[j].ID
	})
	return out, nil
}

$ go test ./internal/village/ -run 'TestRecencyHalvesEveryHalfLife|TestTheThreeTermsAreAddedEachTimesItsWeight|TestANegativeCosineIsClampedAndCannotSubtract|TestTiesBreakOnTheIdentityAscending|TestARowWithNoVectorOrFromTheFutureIsRefused|TestTheVillageImportsNeitherTheModelClientNorTheStore|TestTheGroundingSetIsThreeSourcesEachOnceAscending|TestTheShippedSettingsAreAHalfLifeThreeWeightsAndK' -v
=== RUN   TestRecencyHalvesEveryHalfLife
--- PASS: TestRecencyHalvesEveryHalfLife (0.00s)
=== RUN   TestTheThreeTermsAreAddedEachTimesItsWeight
--- PASS: TestTheThreeTermsAreAddedEachTimesItsWeight (0.00s)
=== RUN   TestANegativeCosineIsClampedAndCannotSubtract
--- PASS: TestANegativeCosineIsClampedAndCannotSubtract (0.00s)
=== RUN   TestTiesBreakOnTheIdentityAscending
--- PASS: TestTiesBreakOnTheIdentityAscending (0.00s)
=== RUN   TestARowWithNoVectorOrFromTheFutureIsRefused
--- PASS: TestARowWithNoVectorOrFromTheFutureIsRefused (0.00s)
=== RUN   TestTheVillageImportsNeitherTheModelClientNorTheStore
    recall_test.go:162: 7 files read, and none of them names the model client or the store
--- PASS: TestTheVillageImportsNeitherTheModelClientNorTheStore (0.00s)
=== RUN   TestTheGroundingSetIsThreeSourcesEachOnceAscending
--- PASS: TestTheGroundingSetIsThreeSourcesEachOnceAscending (0.00s)
=== RUN   TestTheShippedSettingsAreAHalfLifeThreeWeightsAndK
    recall_test.go:197: half-life 250 ticks, weights 1 1 3, k 8
--- PASS: TestTheShippedSettingsAreAHalfLifeThreeWeightsAndK (0.00s)
PASS
ok  	theworld/internal/village	0.004s

The cosine is lang.Cos, the last volume's, unchanged: the dot product over the product of the two lengths, summed in float64 in index order, so that two runs over the same rows produce the same digits. It is handed in by the caller as a field on the settings and not imported, and the reason was measured rather than argued. The first draft of this file imported lang for the one function, and the counterfactual two chapters back, a mailbox whose send waits, stopped printing fatal error: all goroutines are asleep - deadlock! and hung instead: the package that dials a model server imports net/http, whose initialisation leaves the runtime a goroutine it counts as able to wake, and a program that links it has no deadlock detector. A package that sits in front of a villager's mind inside a tick has no business linking a model client anyway, so the sixth test walks this package's imports and refuses both lang and store, with a floor under the walk. The clamp is one line and the third test is what it is for: a row pointing straight away from the question measures minus one and scores nought, so its recency and importance still count in full, and a row the model happens to dislike is never punished below a row the model has no opinion about. The sort is this volume's rule for every ranking it makes: ties break on the identity, ascending, so that two rows scoring alike come back the same way round on every machine, and the fourth test hands the sort three rows that agree on every term to make sure the identity is what decides.

math.Pow on a half is the one place a reader might expect a lookup table, and there is none, because the argument is a ratio of two integers and the function is the standard library's own, which is pure Go on every platform this book runs on. The half-life is a design choice and the file says two hundred and fifty ticks: at ten ticks a second that is twenty-five seconds of watched world, so what happened in the last minute still counts and what happened an hour ago counts for nothing in this term at all. That is on purpose. Recency is the term that says what happened a moment ago; what a memory was worth an hour ago is importance's business.

Now the terms themselves, over the stream, for the first question and at the last tick of the day. The mode prints Halla's twenty-four rows in the order she lived them, with the three terms and the score beside each, and then adds every score up again itself.

▣ Build · stage 7 — every row of the stream, and what each term says about it

$ go run ./cmd/recall -mode terms -q 1
recall: three terms for every memory, each between nought and one

  the stream     exemplars/memories.jsonl, 88 rows over the village, 24 of them Halla's
  the vectors    exemplars/memory-vectors.json, 88 rows and 5 questions, 768 numbers each
  the model      nomic-embed-text:v1.5
  pinned at      970aa74c0a90ef74
  the question   1, "Who took something out of my hands?"
  now            tick 2900
  the half-life  250 ticks, off configs/thought.json
  the weights    recency 1.00, importance 1.00, relevance 3.00, off the same file
  k              8
  combined       summed, each term times its weight

  Halla's stream at ticks up to 2900, in the order it was lived, and what each row scores
    id   tick   age weight  recency import      cos   relev   score  text
     1    901  1999      4   0.0039   0.40   0.4829  0.4829  1.8527  Ander came into sight, 0.8 cell...
     2    901  1999      2   0.0039   0.20   0.4875  0.4875  1.6663  Mose came into sight, 0.8 cells...
     8    999  1901      7   0.0051   0.70   0.4591  0.4591  2.0826  handed Ander 50 parcels, 25.0 g...
    11   1015  1885     10   0.0054   1.00   0.5822  0.5822  2.7520  Ander took 40 parcels out of Ha...
    13   1099  1801      5   0.0068   0.50   0.4591  0.4591  1.8842  handed Ander 50 parcels, 25.0 g...
    17   1173  1727      4   0.0083   0.40   0.4700  0.4700  1.8183  handed Ander 37 parcels, 18.5 g...
    25   1337  1563      2   0.0131   0.20   0.4829  0.4829  1.6620  Ander came into sight, 0.8 cell...
    28   1348  1552      3   0.0135   0.30   0.4401  0.4401  1.6339  went from 8,5 to 9,3
    35   1448  1452      5   0.0178   0.50   0.4016  0.4016  1.7228  ate 50 mouthfuls on 9,3, 25.0 g...
    38   1548  1352      3   0.0236   0.30   0.4016  0.4016  1.5285  ate 50 mouthfuls on 9,3, 25.0 g...
    41   1648  1252      3   0.0311   0.30   0.4016  0.4016  1.5360  ate 50 mouthfuls on 9,3, 25.0 g...
    44   1748  1152      3   0.0410   0.30   0.4016  0.4016  1.5459  ate 50 mouthfuls on 9,3, 25.0 g...
    47   1848  1052      3   0.0541   0.30   0.4016  0.4016  1.5590  ate 50 mouthfuls on 9,3, 25.0 g...
    50   1948   952      3   0.0714   0.30   0.4016  0.4016  1.5763  ate 50 mouthfuls on 9,3, 25.0 g...
    53   2048   852      3   0.0942   0.30   0.4016  0.4016  1.5991  ate 50 mouthfuls on 9,3, 25.0 g...
    56   2148   752      3   0.1243   0.30   0.4061  0.4061  1.6427  ate 50 mouthfuls on 9,3, 23.5 g...
    62   2203   697      2   0.1448   0.20   0.4759  0.4759  1.7726  Ander came into sight, 0.2 cell...
    63   2202   698      2   0.1444   0.20   0.3789  0.3789  1.4811  ate 27 mouthfuls on 9,3, 9.3 grams
    69   2307   593      5   0.1932   0.50   0.4591  0.4591  2.0706  handed Ander 50 parcels, 25.0 g...
    73   2341   559      4   0.2123   0.40   0.4769  0.4769  2.0430  handed Ander 17 parcels, 8.5 grams
    83   2776   124      1   0.7091   0.10   0.4066  0.4066  2.0289  ate 2 mouthfuls on 9,3, 1.0 grams
    84   2781   119      1   0.7190   0.10   0.4258  0.4258  2.0963  went from 9,3 to 9,2
    86   2883    17      3   0.9540   0.30   0.4021  0.4021  2.4602  ate 50 mouthfuls on 9,2, 25.0 g...
    87   2899     1      1   0.9972   0.10   0.3877  0.3877  2.2602  ate 8 mouthfuls on 9,2, 4.0 grams

  24 rows, every term between nought and one, every score added up again here
  and agreeing; 0 of the 24 cosines were below nought and clamped to it
  the cosine ran from 0.3789 to 0.5822 on this stream, a spread of 0.2033 of its scale

Read the three term columns down before reading anything across. Recency runs from 0.0039 at the top of the stream to 0.9972 at the bottom, which is nearly the whole scale, and it is below a tenth for everything before tick 2148. Importance runs from 0.10 to 1.00, and the 1.00 is row 11, the only row of her day that reached the top of the weight scale. Relevance runs from 0.3789 to 0.5822, and the last line of the run says so in one number: a spread of 0.2033. Every one of the twenty-four cosines is positive, so the clamp fired on none of them; the cosine of this model between a short question and a short annalistic line sits in a band a fifth of the scale wide, and the spread is what the ranking has to work with, not the scale.

That number is why the file weights relevance at three. Each term is on a scale from nought to one, and if the three were added as they stand, a question could move a score by 0.2 while age moves it by nearly 1.0 and weight by 0.9; the question would be a fifth of a vote. Three times 0.2033 is 0.61, which puts the question's say in the same range as the other two, and the weight is a number in a file so that a reader with a different model, whose cosines sit in a different band, changes one number and rebuilds. The two ones are ones because nothing measured says otherwise. The sum's largest possible value is five, and the highest score on this page is 3.0344.

∑ Math Interlude — one memory's three terms, by hand

Take row 11, the forty parcels taken out of her hands, and the question at tick 2900. Its age is 2900 minus 1015, which is 1885 ticks. The half-life is 250, so that is 1885 over 250, which is 7.54 half-lives. Halve one seven times: a half, a quarter, an eighth, a sixteenth, a thirty-second, a sixty-fourth, a hundred-and-twenty-eighth, which is 0.0078. The remaining 0.54 of a half-life is a little over half a halving, and a calculator says 0.5 to the power 0.54 is 0.688; 0.0078 times 0.688 is 0.0054. That is the recency term, and it says this memory is old.

Importance is the weight over ten, and the weight is 10, so the term is 1.00. Relevance is the cosine, 0.5822, which is above nought and so stays as it is. Now the weights: recency 1, importance 1, relevance 3. One times 0.0054 is 0.0054; one times 1.00 is 1.00; three times 0.5822 is 1.7466. Add: 0.0054 plus 1.00 plus 1.7466 is 2.7520, and the bench's score column for row 11 says 2.7520.

One more, for the halving. Row 56 is the meal at tick 2148, age 752, which is 3.008 half-lives: three halvings take one to an eighth, 0.125, and the eight ticks over shave it to 0.1243. Its weight is 3, so 0.30. Its cosine is 0.4061, times three is 1.2183. The pencil gets 1.6426 and the bench prints 1.6427, because the bench adds the terms before rounding them and the pencil adds them after; four decimal places of each term can disagree with the sum by one in the last place, and that is the only way the two columns ever differ.

The same in symbols:

recency = 0.5^{(now − tick) / half}

importance = weight / 10

relevance = cos, if cos ≥ 0; 0 otherwise

score = w_r × recency + w_i × importance + w_v × relevance

nowthe tick the question is asked at; 2900 above

tickthe tick the memory's row carries

halfthe half-life in ticks, from the file; 250 here

0.5^xa half raised to x: halved x times, with a fraction of a halving for the fraction of x

weightthe row's stored importance, 1 to 10, decided at append

costhe cosine between the question's vector and the memory's, as the last volume built it

w_r, w_i, w_vthe three weights from the file; 1, 1 and 3 here

a × ba multiplied by b

The ranking is the same table sorted, cut at k. The mode prints all twenty-four so that the line where a stream stops being brought to mind is visible, and it is the same mode that prints the grounding set.

▣ Build · stage 8 — the eight that come to mind, for two questions

$ go run ./cmd/recall -mode ask -q 1 | tail -36 | head -24
  Halla's 24 rows at ticks up to 2900, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1   11   1015     10   0.0054   1.00  0.5822  2.7520  Ander took 40 parcels out of Ha...
     2   86   2883      3   0.9540   0.30  0.4021  2.4602  ate 50 mouthfuls on 9,2, 25.0 g...
     3   87   2899      1   0.9972   0.10  0.3877  2.2602  ate 8 mouthfuls on 9,2, 4.0 grams
     4   84   2781      1   0.7190   0.10  0.4258  2.0963  went from 9,3 to 9,2
     5    8    999      7   0.0051   0.70  0.4591  2.0826  handed Ander 50 parcels, 25.0 g...
     6   69   2307      5   0.1932   0.50  0.4591  2.0706  handed Ander 50 parcels, 25.0 g...
     7   73   2341      4   0.2123   0.40  0.4769  2.0430  handed Ander 17 parcels, 8.5 grams
     8   83   2776      1   0.7091   0.10  0.4066  2.0289  ate 2 mouthfuls on 9,3, 1.0 grams
  ---- the line below which nothing comes to mind ----
     9   13   1099      5   0.0068   0.50  0.4591  1.8842  handed Ander 50 parcels, 25.0 g...
    10    1    901      4   0.0039   0.40  0.4829  1.8527  Ander came into sight, 0.8 cell...
    11   17   1173      4   0.0083   0.40  0.4700  1.8183  handed Ander 37 parcels, 18.5 g...
    12   62   2203      2   0.1448   0.20  0.4759  1.7726  Ander came into sight, 0.2 cell...
    13   35   1448      5   0.0178   0.50  0.4016  1.7228  ate 50 mouthfuls on 9,3, 25.0 g...
    14    2    901      2   0.0039   0.20  0.4875  1.6663  Mose came into sight, 0.8 cells...
    15   25   1337      2   0.0131   0.20  0.4829  1.6620  Ander came into sight, 0.8 cell...
    16   56   2148      3   0.1243   0.30  0.4061  1.6427  ate 50 mouthfuls on 9,3, 23.5 g...
    17   28   1348      3   0.0135   0.30  0.4401  1.6339  went from 8,5 to 9,3
    18   53   2048      3   0.0942   0.30  0.4016  1.5991  ate 50 mouthfuls on 9,3, 25.0 g...
    19   50   1948      3   0.0714   0.30  0.4016  1.5763  ate 50 mouthfuls on 9,3, 25.0 g...
    20   47   1848      3   0.0541   0.30  0.4016  1.5590  ate 50 mouthfuls on 9,3, 25.0 g...
    21   44   1748      3   0.0410   0.30  0.4016  1.5459  ate 50 mouthfuls on 9,3, 25.0 g...

$ go run ./cmd/recall -mode ask -q 2 | tail -36 | head -11
  Halla's 24 rows at ticks up to 2900, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1   86   2883      3   0.9540   0.30  0.5935  3.0344  ate 50 mouthfuls on 9,2, 25.0 g...
     2   87   2899      1   0.9972   0.10  0.5929  2.8760  ate 8 mouthfuls on 9,2, 4.0 grams
     3   83   2776      1   0.7091   0.10  0.5876  2.5719  ate 2 mouthfuls on 9,3, 1.0 grams
     4   11   1015     10   0.0054   1.00  0.4862  2.4638  Ander took 40 parcels out of Ha...
     5   84   2781      1   0.7190   0.10  0.5300  2.4089  went from 9,3 to 9,2
     6   35   1448      5   0.0178   0.50  0.5937  2.2989  ate 50 mouthfuls on 9,3, 25.0 g...
     7   56   2148      3   0.1243   0.30  0.6004  2.2255  ate 50 mouthfuls on 9,3, 23.5 g...
     8   53   2048      3   0.0942   0.30  0.5937  2.1752  ate 50 mouthfuls on 9,3, 25.0 g...
  ---- the line below which nothing comes to mind ----

Asked who took something out of her hands, the first thing that comes to Halla's mind at the end of the day is the one time somebody did, 1885 ticks ago, and it comes first with a recency of 0.0054. The weight carried it: a full point from importance and 1.75 from a cosine that is the highest on the stream. Beneath it the eight are three things from the last hundred and twenty ticks and three gifts to Ander, and the last row above the line is two mouthfuls at tick 2776, worth one, there because it is recent and for no other reason. Asked instead where she last ate, three meals come first, the newest at the top, the taking drops to fourth on its importance alone, and three meals from earlier in the day come in on their cosine: 0.6004 is the highest cosine on her stream for that question, on the one meal whose line carries 23.5 grams instead of 25.0.

Five of the eight rows are the same rows under both questions, in a different order, and that is what a sum does: a question moves what it can move and the other two terms keep what they can keep. The rows below the line are the argument for the line. Every meal from tick 1548 to tick 2048 scores within a tenth of the others, because they are the same line at a different age, and they sort by age because nothing else about them differs; rows 38 and 41 differ by 0.0075, all of it recency.

⚠ Worked failure — the three terms multiplied, and the memory that vanished

Here is the product, which is what a score written in a hurry looks like: three numbers between nought and one, so multiply them, and a memory has to be fresh and heavy and relevant to score well. The bench keeps it behind a flag, prints that the flag is set, and everything else about the run is the run above.

$ go run ./cmd/recall -mode ask -q 1 -product | tail -36 | head -19
  Halla's 24 rows at ticks up to 2900, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1   86   2883      3   0.9540   0.30  0.4021  0.1151  ate 50 mouthfuls on 9,2, 25.0 g...
     2   69   2307      5   0.1932   0.50  0.4591  0.0443  handed Ander 50 parcels, 25.0 g...
     3   73   2341      4   0.2123   0.40  0.4769  0.0405  handed Ander 17 parcels, 8.5 grams
     4   87   2899      1   0.9972   0.10  0.3877  0.0387  ate 8 mouthfuls on 9,2, 4.0 grams
     5   84   2781      1   0.7190   0.10  0.4258  0.0306  went from 9,3 to 9,2
     6   83   2776      1   0.7091   0.10  0.4066  0.0288  ate 2 mouthfuls on 9,3, 1.0 grams
     7   56   2148      3   0.1243   0.30  0.4061  0.0151  ate 50 mouthfuls on 9,3, 23.5 g...
     8   62   2203      2   0.1448   0.20  0.4759  0.0138  Ander came into sight, 0.2 cell...
  ---- the line below which nothing comes to mind ----
     9   53   2048      3   0.0942   0.30  0.4016  0.0114  ate 50 mouthfuls on 9,3, 25.0 g...
    10   63   2202      2   0.1444   0.20  0.3789  0.0109  ate 27 mouthfuls on 9,3, 9.3 grams
    11   50   1948      3   0.0714   0.30  0.4016  0.0086  ate 50 mouthfuls on 9,3, 25.0 g...
    12   47   1848      3   0.0541   0.30  0.4016  0.0065  ate 50 mouthfuls on 9,3, 25.0 g...
    13   44   1748      3   0.0410   0.30  0.4016  0.0049  ate 50 mouthfuls on 9,3, 25.0 g...
    14   41   1648      3   0.0311   0.30  0.4016  0.0037  ate 50 mouthfuls on 9,3, 25.0 g...
    15   35   1448      5   0.0178   0.50  0.4016  0.0036  ate 50 mouthfuls on 9,3, 25.0 g...
    16   11   1015     10   0.0054   1.00  0.5822  0.0031  Ander took 40 parcels out of Ha...

$ go run ./cmd/recall -mode ask -q 1 -product | tail -8 | head -5
  the grounding set at tick 2900, from three sources
    the about columns of the 8 retrieved       2 Ander
    Halla's eyes at the boundary of tick 2900  nobody
    Halla, whose stream it is                  1 Halla
    the set, each once, ascending              1 Halla, 2 Ander

Start from the symptom. Asked who took something out of her hands, the product answers with a meal, two gifts, another meal, a walk, two more meals and a sighting, and the forty parcels taken out of her hands are sixteenth of twenty-four, below eight meals of the same fifty mouthfuls each. Row 11 is the heaviest row of her life and the most relevant row to the question on the stream, and under the product neither fact can be seen.

Work back through its score. 0.0054 times 1.00 times 0.5822 is 0.0031. The importance term is a full one and multiplying by one changes nothing; the relevance term is the highest there is and multiplying by it keeps a little over half. What decided the score is the first factor, and the first factor is the age. Under the sum the age contributed 0.0054 to a score of 2.7520, which is nothing, and the other two terms carried the row to first place; under the product the age is not contributing, it is scaling, and a factor of 0.0054 takes the whole of what the other two terms had to say and keeps half a percent of it. The row at the top is the row whose worst term is best: a meal seventeen ticks ago, with a recency of 0.9540, whose other two terms are 0.30 and 0.4021.

The cause is one sentence. A product hands every term a veto, and the term that uses it here is recency, which is below a hundredth for every row older than seven half-lives, which at this half-life is everything before tick 1150: the first eighth of her day, and the two heaviest rows she has. The clamp makes the veto absolute in the other direction: a cosine below nought would be a relevance of exactly nought and a product of exactly nought, whatever the row weighed and whenever it happened. No cosine on this stream went below nought, so the erasure on this page was done by a term at 0.0054 and not by one at nothing, and the difference between the two is the difference between sixteenth place and last. The flag comes off, the sum goes back, and the second run says what the product costs a prompt as well as a ranking: the first source no longer names Halla, because the one memory in which somebody did something to her is gone from the eight, and she is in the set only because the third source puts her there.

Two identities a prompt may name

The ranking is half of what a retrieval produces and the other half is the reason this chapter is in a volume about a seam. A proposal is checked against what the villager could have known when it was asked, and could have known has to be a set the engine computed, on a tick, from stored numbers, so that the check is arithmetic and never an opinion. The set is three sources. The identities the retrieved memories are about, read off their about columns, which is why the last chapter put those columns on the row. The identities the villager's own eyes returned at the boundary of this tick, read off the same map the sighting memories are written from. And the villager itself. Each once, in ascending order, so that two sets built from the same retrieval are the same bytes.

▣ Build · stage 9 — the set, and where each identity in it came from

// internal/village/recall.go

// Ground is the grounding set: every identity a prompt rendered from
// this retrieval may name, and nothing else. It is the identities the
// retrieved memories are about, the identities the villager's own
// eyes returned this tick, and the villager itself, each once, in
// ascending order.
func Ground(me sim.EntityID, top []Scored, seen []sim.EntityID) []sim.EntityID {
	set := map[sim.EntityID]bool{me: true}
	for _, s := range top {
		for _, id := range s.About {
			set[id] = true
		}
	}
	for _, id := range seen {
		set[id] = true
	}
	out := make([]sim.EntityID, 0, len(set))
	for id := range set {
		out = append(out, id)
	}
	sort.Slice(out, func(i, j int) bool { return out[i] < out[j] })
	return out
}

// internal/village/witness.go

// InSight is every body this villager's eyes returned at the boundary
// of tick, in ascending order of identity. It is what a grounding set
// takes as the second of its three sources, and it is read off the
// same map the sighting memories are written from, so the two cannot
// disagree about who was there.
func (w *Witness) InSight(tick int) []sim.EntityID {
	var out []sim.EntityID
	for id, last := range w.seen {
		if last == tick {
			out = append(out, id)
		}
	}
	sort.Slice(out, func(i, j int) bool { return out[i] < out[j] })
	return out
}

$ go run ./cmd/recall -mode ask -q 4 -now 2400 | tail -32 | head -11
  Halla's 20 rows at ticks up to 2400, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1   69   2307      5   0.7727   0.50  0.3756  2.3995  handed Ander 50 parcels, 25.0 g...
     2   73   2341      4   0.8491   0.40  0.3765  2.3787  handed Ander 17 parcels, 8.5 grams
     3   62   2203      2   0.5791   0.20  0.4701  2.1896  Ander came into sight, 0.2 cell...
     4   11   1015     10   0.0215   1.00  0.3807  2.1637  Ander took 40 parcels out of Ha...
     5    8    999      7   0.0206   0.70  0.3756  1.8474  handed Ander 50 parcels, 25.0 g...
     6    1    901      4   0.0157   0.40  0.4693  1.8236  Ander came into sight, 0.8 cell...
     7   56   2148      3   0.4972   0.30  0.3255  1.7737  ate 50 mouthfuls on 9,3, 23.5 g...
     8   63   2202      2   0.5775   0.20  0.3094  1.7058  ate 27 mouthfuls on 9,3, 9.3 grams
  ---- the line below which nothing comes to mind ----

$ go run ./cmd/recall -mode ask -q 4 -now 2400 | tail -8
  the grounding set at tick 2400, from three sources
    the about columns of the 8 retrieved       2 Ander, 1 Halla
    Halla's eyes at the boundary of tick 2400  2 Ander
    Halla, whose stream it is                  1 Halla
    the set, each once, ascending              1 Halla, 2 Ander

  2 identities. A prompt rendered from this retrieval may name these and
  nothing else, and the set is stored with whatever the prompt proposes

$ go run ./cmd/recall -mode ask -q 1 | tail -8
  the grounding set at tick 2900, from three sources
    the about columns of the 8 retrieved       2 Ander, 1 Halla
    Halla's eyes at the boundary of tick 2900  nobody
    Halla, whose stream it is                  1 Halla
    the set, each once, ascending              1 Halla, 2 Ander

  2 identities. A prompt rendered from this retrieval may name these and
  nothing else, and the set is stored with whatever the prompt proposes

Two runs at two ticks, and the second source is what changes between them. At tick 2400 Halla handed Ander his last parcel sixty ticks ago and he is still standing on the next cell, so her eyes return him; at tick 2900 she has walked to 9,2 and eaten alone for a hundred and twenty ticks, and her eyes return nobody. The set is the same two identities both times, because in a village of three people whose day was mostly spent handing things to each other, the memories that come to mind already name everybody the eyes can see. The second source is there for the day that stops being true: a body that walks into sight for the first time has no row in the stream yet, since a sighting is written at the boundary and a question can be asked at the same boundary, and a stream that has never mentioned somebody is exactly the stream a prompt about them has to be grounded from.

The bench takes the eyes off a run of the day up to the tick in question and not one tick further, because the eyes are the one thing the fixtures cannot supply: a fixture is rows, and who was in sight is a fact about the world at a boundary. The run is in memory, opens no socket and takes a few tens of milliseconds, and the audit at the end of the mode walks the set both ways: everything in it came from one of the three sources, and everything in the three sources is in it.

What the set is for is stated here and built nowhere on this page. A prompt rendered from this retrieval may name these identities and no others, and the set is kept beside the proposal it produced, because a check of what a villager could have known has to be made against what it knew when it was asked and never against what is true when the answer lands. Those are two different clocks, and the engine keeps both. Nothing here renders a prompt, and nothing here proposes.

Figure 93.1 — what a retrieval is made from and what it makes. Three inputs, three terms each between nought and one, one sum with weights from a file, and a cut at k; then three sources into one set. Nothing in the diagram is a model except the vector at the top right, and that is read off a file.

Recall rows read back the same

Everything above was computed over two files with no database anywhere, which is what makes it replay. The table exists so that a running daemon does not have to hold every villager's vectors in memory, and so that a vector is produced once, at the boundary, and never again. Two runs put that on the page. The first puts both fixtures into the empty database from the start of this chapter and asks the same question of the rows read back. The second runs the day again with the writes and the embedding job both at the boundary, against a live model, and prints what it did and never what it received.

▣ Build · stage 10 — the fixtures into the tables, and the question asked of the tables

$ podman exec -w /bench world-go go run ./cmd/recall -mode load -q 1
recall: the committed stream and its vectors, put into a database that has neither

  the stream     exemplars/memories.jsonl, 88 rows over the village, 24 of them Halla's
  the vectors    exemplars/memory-vectors.json, 88 rows and 5 questions, 768 numbers each
  the model      nomic-embed-text:v1.5
  pinned at      970aa74c0a90ef74
  the question   1, "Who took something out of my hands?"
  now            tick 2900
  the half-life  250 ticks, off configs/thought.json
  the weights    recency 1.00, importance 1.00, relevance 3.00, off the same file
  k              8
  combined       summed, each term times its weight

  the stream, INSERT ... RETURNING id, one row at a time
    rows appended                                88
    given the identity the dump carries          88
    given some other identity                     0

  the vectors, one COPY, in the text format the server has always taken
    rows before                                   0
    rows the copy reported                       88
    rows after                                   88
    the literals it sent                     748298 bytes

  what the eleventh table holds, GROUP BY model, ORDER BY digest
    model                        pinned at            rows
    nomic-embed-text:v1.5        970aa74c0a90ef74       88
  0 memories with no row from this model

  Halla's rows read back, ORDER BY tick, id, and her vectors, ORDER BY memory
  the 8 that come to mind, scored over the rows the tables hold
  rank   id   tick weight  recency import   relev   score  text
     1   11   1015     10   0.0054   1.00  0.5822  2.7520  Ander took 40 parcels out of Ha...
     2   86   2883      3   0.9540   0.30  0.4021  2.4602  ate 50 mouthfuls on 9,2, 25.0 g...
     3   87   2899      1   0.9972   0.10  0.3877  2.2602  ate 8 mouthfuls on 9,2, 4.0 grams
     4   84   2781      1   0.7190   0.10  0.4258  2.0963  went from 9,3 to 9,2
     5    8    999      7   0.0051   0.70  0.4591  2.0826  handed Ander 50 parcels, 25.0 g...
     6   69   2307      5   0.1932   0.50  0.4591  2.0706  handed Ander 50 parcels, 25.0 g...
     7   73   2341      4   0.2123   0.40  0.4769  2.0430  handed Ander 17 parcels, 8.5 grams
     8   83   2776      1   0.7091   0.10  0.4066  2.0289  ate 2 mouthfuls on 9,3, 1.0 grams

  24 rows read back, 24 vectors read back, and the 8 that come to mind are the
  8 the fixtures give with no database anywhere, in the same order to the same digit

$ podman exec world-db psql -U world -d world -c "SELECT v.memory, m.villager, m.tick, m.weight, v.model, left(v.digest, 16) AS pinned_at FROM memory_vector v JOIN memory m ON m.id = v.memory WHERE m.villager = 'Halla' ORDER BY v.memory LIMIT 4;"
 memory | villager | tick | weight |         model         |    pinned_at
--------+----------+------+--------+-----------------------+------------------
      1 | Halla    |  901 |      4 | nomic-embed-text:v1.5 | 970aa74c0a90ef74
      2 | Halla    |  901 |      2 | nomic-embed-text:v1.5 | 970aa74c0a90ef74
      8 | Halla    |  999 |      7 | nomic-embed-text:v1.5 | 970aa74c0a90ef74
     11 | Halla    | 1015 |     10 | nomic-embed-text:v1.5 | 970aa74c0a90ef74
(4 rows)

Eighty-eight of eighty-eight identities agree, which settles the claim the dump made: the number a row gets by being counted in append order is the number the server hands it. The vectors go in as one COPY of 748,298 bytes of literals for eighty-eight rows of 768 numbers, over the socket as text, which is the road the eighth table's rows took, and the backfill's question afterwards is answered with nought: every memory has a row from this model. Then the eight that come to mind, computed over rows read back ORDER BY tick, id and vectors read back ORDER BY memory, are the eight the fixtures gave with no database anywhere, in the same order to the same digit, and the bench stops with a non-zero status if any place differs. The round trip through a text literal and back has lost nothing, and the table is holding what the file holds.

The join in the psql query is the reason the eleventh table has a foreign key and not a copy of the villager's name: a vector is about one memory, the memory knows whose it is, and the question which of Halla's memories have vectors is one join away. Every query on this page carries an ORDER BY, this one on the memory, so that four rows come back the same four rows on every server.

The last run is the one that puts a model in the loop, and the rule it is under is the rule this whole volume is under: nothing inside a tick opens a transaction, and nothing inside a tick calls a model. The database is emptied first, because the villager table refuses a village it already has.

▣ Build · stage 11 — the day written and embedded as it goes, with a live model at the boundary

// cmd/recall/db.go

	job := func() error {
		for {
			batch, err := db.Unembedded(ctx, m.Digest, c.batch)
			if err != nil {
				return err
			}
			if len(batch) == 0 {
				return nil
			}
			passes++
			var rows []store.MemoryVector
			for _, mem := range batch {
				v, err := cl.Embed(ctx, m.Name, mem.Text)
				if err != nil {
					return err
				}
				asked++
				rows = append(rows, store.MemoryVector{Memory: mem.ID, Digest: m.Digest, Model: m.Name, V: v})
			}
			n, err := db.Embedded(ctx, rows)
			if err != nil {
				return err
			}
			embedded += n
		}
	}
	// ...
	for i := 0; i < c.ticks; i++ {
		before := pool.Stat().AcquireCount()
		w.v.Tick()
		inTick += pool.Stat().AcquireCount() - before
		rows, err := w.boundary()
		if err != nil {
			return err
		}
		if err := write(rows); err != nil {
			return err
		}
		if (i+1)%c.every == 0 {
			t := time.Now()
			if err := job(); err != nil {
				return err
			}
			jobTime += time.Since(t)
		}
	}

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go go run ./cmd/recall -mode live
recall: a day written down as it happens, and embedded at the boundary as it goes

  the world      seed 5, 12x8 cells, tick 901, 12 plants at 1517.5 grams
  the run        2000 ticks, from tick 901 to tick 2900
  the village    3 people out of configs/village.json, on a mailbox and a habit each
  the habit      feed below 25%, trade past 40%, top up below 90%
  the witness    one a person, in front of the mind; a run of one thing may
                 pause 10 ticks and go on for 100 before it is one memory
  the weights    off configs/importance.json, 1 to 10
  the writes     at the boundary, after v.Tick() returns and before the next is asked for
  the job        at the same boundary, every 250 ticks and once at the end of the day,
                 32 waiting memories at a time
  the server     world-lm:11434, waiting up to 10m0s for an answer
  the model      nomic-embed-text:v1.5
  pinned at      970aa74c0a90ef74

  what the day wrote
  ticks run                                        2000
  rows appended to the memory table                  88
  transactions, one a boundary that closed something       74
  connections taken at the boundary                  74
  connections taken inside a tick                     0

  what the job did, at the boundary and nowhere else
  passes over what was waiting                        9
  memories sent to the model                         88
  rows the copies reported                           88
  memories with no row from this model                0

  what the eleventh table holds, GROUP BY model, ORDER BY digest
    model                        pinned at            rows
    nomic-embed-text:v1.5        970aa74c0a90ef74       88

  every row the day wrote has a vector from this model, 0 connections were
  taken inside 2000 ticks, and not one number the model answered is on this page
  2.693s inside the job, over 88 calls (measured here; yours will differ)

  2000 ticks in 2.749s, 728 ticks a second (measured here; yours will differ)

The job is the last volume's backfill with the memory table where the chronicle was: read what is waiting, ask for a row apiece, copy the rows in, go round again until nothing is waiting. It runs nine times, once at each of eight boundaries two hundred and fifty ticks apart and once after the day's last runs are closed, and between them it sends every one of the eighty-eight rows to the model and copies eighty-eight rows back. The pool's own count of connections is read before every v.Tick() and after it, two thousand times, and the two agree every time: the writes and the job are on the boundary side of that line and the tick touches nothing, and the bench stops with a non-zero status if the count is anything else.

Two durations are on that block and both are labelled. The job took 2.693 seconds over eighty-eight calls, on the eight-core AMD Ryzen 7 3700X with 30 GB of memory that every duration in this volume has been measured on, with the model answering on the processor and no graphics card in use; and the day ran at 728 ticks a second, which is an accelerated pace and not the watched one, with the job's seconds inside it. Neither number is compared by anything. What is compared is every count above them, and the fact that not one of the 88 rows of numbers the model answered with is printed here. The last volume's rule holds for memory text as it held for the chronicle: a vector a live model produced goes into the table and never onto a page.

Why recall scores read rows and ticks

Take the village away and the design is a rule about combining evidence of different kinds. Three measurements that answer three different questions, how long ago, how much it mattered, how close to what is being asked, cannot be compared until each is put on a scale the others share, and once they are on one scale the choice between adding them and multiplying them is the choice between a vote and a veto. A sum lets a memory that is old but heavy and apt come to mind; a product lets the smallest term decide alone, and the smallest term is nearly always the one that measures age, because age is the one of the three that grows without bound. Whenever a score is built out of terms and one of them can be nearly nought for most of the population, the sum is the safe default and the product needs an argument.

The second thing that generalizes is where the scale comes from. Recency and importance fill their scales by construction: a fresh row scores one and a full weight scores one. Relevance fills a fifth of its scale by measurement, because a cosine is a fact about a model and a corpus, and the fixture is what let that fact be measured before a weight was chosen. A term whose spread was assumed would have been given a weight of one, and the question would have decided nothing. Measure the spread of each term over the data it will actually run on, and set the weights from the measurement, and write the measurement on the page beside the weight so that the day the model changes, the reader knows which number to re-measure.

The third is the one the last chapter set up and this one finished. Every number on this page is a function of the rows, their vectors, the question's vector and the tick, and of nothing else: no last-access column, no wall clock, no live answer, no order the server happened to return rows in. That is what let a ranking computed over two files be held against a ranking computed over two tables and required to agree to the last digit, and it is what lets the same retrieval be run again, on any machine, in a year. The grounding set inherits every bit of that. A check against it is a check against a value the engine computed on a known tick from stored numbers, and a check like that can be replayed; a check that can be replayed is the only kind this volume puts a model's output through.

Checkpoint

✓ Checkpoint — three terms, one sum, and a set

Given a row's tick, weight and cosine, the tick of the question and the file's half-life and weights, score it by hand, and say which of the three terms can be worked without a calculator and which cannot.
Say why the file weights relevance at three and the other two at one, in terms of a number the terms run printed, and say what happens to the ranking for two different questions if all three weights are one.
Explain, with row 11's three terms, why the product put it sixteenth and the sum put it first, and say what a cosine below nought would have done to it under each.
Name the three sources of a grounding set, say which of them the fixtures cannot supply and why the bench runs the day to get it, and say why the set is stored with the proposal instead of being computed again when the answer lands.
Write down the ninth migration's four columns and its two-column key, and say what question the backfill could not ask if the digest were a note beside the vector instead of half the key.
Say where the embedding job runs, what the pool's connection count proves about it, and which of the numbers in the live run a reader on another machine should expect to get and which not.

⚡ Exercises — try first, then reveal

Exercise 1 — the name pulls harder than the verb. The fifth question in the fixture is What have I given Ander? Before you run it, predict which rows of Halla's stream the cosine alone would put first, then run the ranking and read the relev column.

The cosine puts the sightings of Ander above the gifts to him:

$ go run ./cmd/recall -mode ask -q 5 | tail -36 | head -11
  Halla's 24 rows at ticks up to 2900, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1   11   1015     10   0.0054   1.00  0.5935  2.7860  Ander took 40 parcels out of Ha...
     2   86   2883      3   0.9540   0.30  0.4080  2.4779  ate 50 mouthfuls on 9,2, 25.0 g...
     3    8    999      7   0.0051   0.70  0.5504  2.3564  handed Ander 50 parcels, 25.0 g...
     4   69   2307      5   0.1932   0.50  0.5504  2.3444  handed Ander 50 parcels, 25.0 g...
     5   73   2341      4   0.2123   0.40  0.5757  2.3393  handed Ander 17 parcels, 8.5 grams
     6    1    901      4   0.0039   0.40  0.6399  2.3235  Ander came into sight, 0.8 cell...
     7   87   2899      1   0.9972   0.10  0.4015  2.3018  ate 8 mouthfuls on 9,2, 4.0 grams
     8   62   2203      2   0.1448   0.20  0.6374  2.2571  Ander came into sight, 0.2 cell...
  ---- the line below which nothing comes to mind ----

The two highest cosines on the stream, 0.6399 and 0.6374, are on Ander came into sight, and the gifts to him sit at 0.5504 and 0.5757. The model has matched the name and not the verb, which is the last volume's finding about what an angle measures, met again on shorter lines: a question and a line that share a rare word point the same way. The sum is what keeps that from mattering much. The two sightings come sixth and eighth, on a weight of four and two, and the three gifts come third, fourth and fifth on their weights; the row at the top is the taking, which the question did not ask about, carried there by its weight. A prompt rendered from this retrieval would name Ander either way, because every one of the eight is about him.

Exercise 2 — give the question a fifth of a vote. The bench takes the relevance weight as a flag. Run the first two questions with it at one, and say what the first place is for each and how many of the eight the two questions share, before and after.

With the weight at one, both questions put the same meal first:

$ go run ./cmd/recall -mode ask -q 1 -relevance 1 | tail -36 | head -5
  Halla's 24 rows at ticks up to 2900, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1   86   2883      3   0.9540   0.30  0.4021  1.6561  ate 50 mouthfuls on 9,2, 25.0 g...
     2   11   1015     10   0.0054   1.00  0.5822  1.5876  Ander took 40 parcels out of Ha...
     3   87   2899      1   0.9972   0.10  0.3877  1.4849  ate 8 mouthfuls on 9,2, 4.0 grams

$ go run ./cmd/recall -mode ask -q 2 -relevance 1 | tail -36 | head -5
  Halla's 24 rows at ticks up to 2900, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1   86   2883      3   0.9540   0.30  0.5935  1.8474  ate 50 mouthfuls on 9,2, 25.0 g...
     2   87   2899      1   0.9972   0.10  0.5929  1.6902  ate 8 mouthfuls on 9,2, 4.0 grams
     3   11   1015     10   0.0054   1.00  0.4862  1.4915  Ander took 40 parcels out of Ha...

Asked who took something out of her hands, Halla's first thought at a weight of one is the meal she finished seventeen ticks ago, and asked where she last ate it is the same meal; the taking is second and third. The two questions share seven of their eight rows at a weight of one and five at a weight of three, so the weight moves two rows across the line and moves the first place from the newest memory to the apt one. The run prints that the weight came off the flag and not the file, so a reader of the listing cannot mistake it for the shipped setting.

Exercise 3 — the same question, from the other end of the parcels. Ander is the one who took. Ask the first question of his stream at the end of the day, predict what comes first, and read the grounding set.

What comes to Ander's mind is being given things, and the taking is seventh:

$ go run ./cmd/recall -mode ask -q 1 -who Ander | tail -51 | head -11
  Ander's 39 rows at ticks up to 2900, ranked, best first, ties on id ascending
  rank   id   tick weight  recency import   relev   score  text
     1    9    999      9   0.0051   0.90  0.5131  2.4445  Halla put 50 parcels in Ander's...
     2   70   2307      7   0.1932   0.70  0.5131  2.4325  Halla put 50 parcels in Ander's...
     3   88   2900      2   1.0000   0.20  0.4089  2.4267  ate 19 mouthfuls on 7,2, 9.5 grams
     4   74   2341      6   0.2123   0.60  0.5118  2.3476  Halla put 17 parcels in Ander's...
     5   14   1099      7   0.0068   0.70  0.5131  2.2462  Halla put 50 parcels in Ander's...
     6   85   2844      2   0.8562   0.20  0.3955  2.2426  ate 34 mouthfuls on 7,2, 17.0 g...
     7   12   1015      7   0.0054   0.70  0.4807  2.1476  took 40 parcels from Halla, 20....
     8   18   1173      6   0.0083   0.60  0.5080  2.1323  Halla put 37 parcels in Ander's...
  ---- the line below which nothing comes to mind ----

The same tick, the same forty parcels, and the row is worth seven to him and ten to her, because the last chapter's table adds two for a line that names the villager it belongs to and his does not. His stream is thirty-nine rows and the eight are mostly Halla putting things in his hands, which at a weight of nine for the first of them outranks his own meal of a moment ago at a recency of exactly one. Row 88 is the one row in the fixture at tick 2900 itself, so its age is nought and its recency is one with nothing shaved off. The set at the bottom is Halla and Ander, from the about columns and from himself; his eyes at that boundary return nobody, as hers did.

So a question put to Halla at the end of her day brings eight rows to mind out of twenty-four, the first of them the one thing anybody did to her, and hands back two identities she may speak of. Nothing has been asked of a model except a row of numbers, and nothing has been said. What the eight rows and the two identities are for is a prompt, and a prompt is text the engine assembles from them; what reads it is not the engine, and what it hands back is not a number.