Every Genome That Ever Lived

Archive survives the process

At the end of a two-century run, every genome the valley ever made is still in the pool: the fourteen hundred animals that starved as well as the three hundred still walking. When the process exits, that history vanishes. Every birth and every death goes to one append-only JSONL file, and a reader rebuilds ancestry from the file alone.

A born record carries the creature's identity, the species it was sorted into, the tick, both parents and every number it inherited. A died record carries the identity, the tick and what ended it. No record is rewritten.

The live pool is a cache, not an archive. It exists so the simulation can run, but it cannot answer a question after the program has gone away unless the same facts were written while they happened.

Append-only history also makes failure visible. If a child is born, dies, and leaves no trace because a map entry was overwritten, the archive reader cannot reconstruct it and the test fails from the bytes on disk.

The chapter writes born and died records, reads them back without a roster or valley, then uses the archive to ask what thirteen genes did over two hundred years. A gene with no pressure on it is allowed to drift, and the file has to show that drift without help from live objects.

Born and died records

Start with the writing, because how a record is written decides what can ever be asked of it later. The valley makes records at two moments and both are in the middle of a creature phase: a child goes on the end of the roster, and a carcass goes into the litter. Neither can afford anything clever. Three hundred creatures are standing and a tick is a tenth of a second, and a birth that stopped to reorganise a file would cost more than the animal it made.

So the file only ever grows at one end, and that rules out a great deal on purpose. There is no index at the top saying how many records follow, because the number is not known until the run ends. There is no field on a born record saying when that creature died, because filling it in would mean seeking back into a fifteen-megabyte file on the tick of every death. The death gets a record of its own instead, and the two are joined by the identity they share.

Which leaves the ordering. Lines come out as the run wrote them, and that is nearly but not exactly the order things happened: the burial hook fires when the phase hands back, after the breeding pass has run, so within one tick the births sit above the deaths that preceded them. Every record carries its own tick, so nothing reading the file has to care. A log where position carries the time cannot be written from two places at once, and this one is written from two places.

// internal/gene/roll.go

// The two kinds of record the archive holds, and the whole of what it
// holds. A creature is written down once when it is made and once when
// it stops, and neither line is ever touched again.
const (
	Born = "born"
	Died = "died"
)

// Empty is the only way to die in this valley: the store closed a tick
// at nothing. It is written into every died record as a word rather
// than left out, because a column with one value in it can be given a
// second one and a column that is not there cannot.
const Empty = "empty"

// Record is one line of the archive: a JSON object, one to a line, in
// the order the run wrote them.
//
// One struct carries both kinds because the file carries both kinds,
// and the first field says which this is. A born record holds an
// identity, a tick, a species, two parents and a whole genome. A died
// record holds an identity, a tick and a cause, and every other field
// is left out of the line rather than written as a zero: the seven
// characters of "of":0, on fourteen hundred deaths, are ten kilobytes
// of nothing.
//
// A founder's two parents are left out for the same reason. A key that
// is not in the line reads back as the zero of its type, and nought is
// exactly what the parent of a creature nobody made is.
type Record struct {
	R    string       `json:"r"`
	ID   sim.EntityID `json:"id"`
	Tick int          `json:"tick"`

	Kind int          `json:"kind,omitempty"` // the species it was sorted into
	Of   sim.EntityID `json:"of,omitempty"`   // the parent that paid for it
	With sim.EntityID `json:"with,omitempty"` // the second parent, if there was one

	Body   []float64 `json:"body,omitempty"`
	Mind   []float64 `json:"mind,omitempty"` // a flat controller's weights
	Temper []float64 `json:"temper,omitempty"`
	Look   []float64 `json:"look,omitempty"`

	// Wire is a grown controller: every node it holds and every link,
	// each link carrying the number it was minted under. Those numbers
	// are the whole reason a wiring can be written down and read back
	// as the same wiring: a link is the same link in two genomes when
	// it carries the same number, and a file that dropped them would
	// hand back a controller nobody could compare to anything.
	Wire *mind.Wiring `json:"wire,omitempty"`

	Why string `json:"why,omitempty"` // on a died record, what ended it
}

// internal/gene/roll.go

// OpenRoll opens the file one run writes its archive to.
//
// Three flags and each one is a decision. Create, because the usual
// case is a file nobody has made yet. Truncate, because a run owns its
// archive: appending to yesterday's file would put two creatures
// numbered 1 in it, and a pedigree with two number ones in it is not a
// pedigree. Append, because from the moment it is open nothing here
// ever goes back: every write lands at the end of the file, and no
// record already on disk can be moved by a later one.
func OpenRoll(path string) (*Roll, error) {
	f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC|os.O_APPEND, 0o644)
	if err != nil {
		return nil, fmt.Errorf("open archive: %w", err)
	}
	buf := bufio.NewWriterSize(f, 64*1024)
	r := NewRoll(buf)
	r.buf, r.f = buf, f
	return r, nil
}

// write puts one record on the end of the file and hands back the bytes
// it took, newline included.
func (r *Roll) write(rec Record) int {
	if r.Err != nil {
		return 0
	}
	line, err := json.Marshal(rec)
	if err != nil {
		r.Err = fmt.Errorf("archive: creature %d: %w", rec.ID, err)
		return 0
	}
	line = append(line, '\n')
	if _, err := r.out.Write(line); err != nil {
		r.Err = fmt.Errorf("archive: creature %d: %w", rec.ID, err)
		return 0
	}
	r.sum.Write(line)
	r.Bytes += len(line)
	return len(line)
}

$ go run ./cmd/roll -mode roll -years 2
roll: 16x12 valley, tick 901, year 1 summer, 90 plants standing at 4654.1 grams

  25 creatures founded on stream 12, each written down as it stood up
  every birth and every death appended to roll.jsonl, and nothing in the valley reads it
  the temperament and look genes go in the file: yes

  one born record as the file holds it: a founder, so no parents are in the
  line, and all but two of its links and two of its nodes are taken out here
{"r":"born","id":1,"tick":901,"kind":1,"body":[1,1,1,1,1,1,1,1,1,1],"temper":[0.5,0.5,0.5],"look":[0.5,0.5,0.5,0.5],"wire":{"id":1,"in":32,"out":6,"nodes":[{"ID":0,"B":0},{"ID":1,"B":0}],"links":[{"From":0,"To":32,"W":0.09849971850444732,"Innov":38},{"From":0,"To":33,"W":0.13815722208087752,"Innov":39}]}}

  the same record whole is 8885 bytes on one line, of which 144 links

  one died record, whole
{"r":"died","id":21,"tick":1024,"why":"empty"}

  what a born record is made of, over every born record in the file
    block                                  mean bytes      share
    the wiring: its links and its biases       8773.6      98.3%
    the ten body factors                         50.4       0.6%
    the three temperament genes                  25.4       0.3%
    the four look genes                          31.9       0.4%
    identity, tick, species and parents          46.4       0.5%

  the archive after 2 years
    born records                                            57
    died records                                            49
    records in all                                         106
    bytes                                               511174
    a born record, mean                                 8927.7
    a died record, mean                                   46.8
    sha256 cb4b9fc346c00d294c15c3bb9ef803151b787c440929078045f8b41ea1627054

Read the founder's line first, because it is the only readable one this file holds. Ten body factors at 1, which is what a founder is; three temperament genes and four look genes at the middle of their ranges; then the wiring, which on the page has had a hundred and forty-two of its links and thirty-six of its nodes taken out to make it fit. Whole, that line is 8,885 bytes. A died record is forty-six bytes and holds four things.

The block table is the arithmetic that matters, and it is measured rather than guessed: each block is taken out of every born record in turn and the record written again, and the difference is what that block costs. A born record is 98.3 percent controller. The ten body factors, which decide what an animal weighs and how big a mouthful it takes and what a gram of it costs to keep every tick of its life, come to fifty bytes. The three temperament genes come to twenty-five, and the four look genes to thirty-two.

Unread temper genes in the archive

Aggression, wariness and tameness are inherited. They are crossed like every other gene, mutated like every other gene, clamped between nought and one like every other gene, and they are archived. No line of code in this volume reads any of the three. There is no predation here, no second kind of animal, no creature-on-creature anything: an aggression gene has nothing to be aggressive at. Every one of the six actions an animal can take is about ground, plants and its own store.

A gene the world ignores is a gene somebody will propose deleting, so here is the case for keeping it, made entirely out of the file above. The price is twenty-five bytes in a record of nearly nine thousand, three-tenths of one percent, so whatever the argument against those three numbers is, it is not what they cost.

The other half is what it would take to add them afterwards, and that is the part that settles it. Say the run finishes without them and a year later somebody wants to know whether temperament drifts. The animals are dead. The process is gone. The genomes exist in exactly one place, which is this file, and the three numbers are not in it, so the only way to get them is to run the two centuries again with the genes installed. That run will not be this run. A mutation pass in this world spends two numbers on every gene of the genome, always, whether or not the gene moves, and a crossing spends one; three more genes is nine more numbers off the streams at every single birth. The first birth of the new run lands on the same numbers as the first birth of the old one, and every draw after it is somewhere else. Different children, different deaths, a different valley by the end of the first year. The old run cannot be extended and it cannot be repeated. Its history is whatever went into the file while it was alive, and nothing else will ever be added to it.

A column can be added to a program any day of the week. It cannot be added to a run that has finished, and an archive is made of finished runs.

Archive reader from the file

The reader gets no pool, no roster and no valley. It gets a path. Everything below is built out of records, which is the only honest way to test an archive: a reader that can reach the live objects will quietly lean on them and never notice what the file is missing.

// internal/gene/roll.go

// Genome is the genome a born record describes, built back out of the
// numbers in the line.
//
// It is checked as it is built, in the one place the file it came from
// is still known. Every block a genome has is a block a born record has
// to carry: a line missing one of them describes a creature this world
// has no prices for, and it is refused here rather than handed back
// with zeros in the gap where the numbers should have been. A wiring
// whose links name nodes it does not hold is refused by the same rule,
// and a died record is not a genome at all and says so.
func (rec Record) Genome() (*Genome, error) {
	if rec.R != Born {
		return nil, fmt.Errorf("archive: creature %d: a %q record carries no genome", rec.ID, rec.R)
	}
	if len(rec.Body) != Bodies {
		return nil, fmt.Errorf("archive: creature %d: %d body factors, not %d",
			rec.ID, len(rec.Body), Bodies)
	}
	if len(rec.Temper) != Tempers {
		return nil, fmt.Errorf("archive: creature %d, born on tick %d: %d temperament genes, not %d",
			rec.ID, rec.Tick, len(rec.Temper), Tempers)
	}
	if len(rec.Look) != Looks {
		return nil, fmt.Errorf("archive: creature %d, born on tick %d: %d look genes, not %d",
			rec.ID, rec.Tick, len(rec.Look), Looks)
	}
	g := &Genome{}
	copy(g.Body[:], rec.Body)
	copy(g.Temper[:], rec.Temper)
	copy(g.Look[:], rec.Look)
	if rec.Wire != nil {
		w, err := rec.Wire.Graph()
		if err != nil {
			return nil, fmt.Errorf("archive: creature %d: %w", rec.ID, err)
		}
		g.Wire = w
		return g, nil
	}
	g.Mind = append(g.Mind, rec.Mind...)
	return g, nil
}

// internal/gene/line.go

// Lineage is an archive read back: every creature the file names, who
// made it, and when it stopped.
//
// It is built out of records and holds no valley, no roster and no
// pool. Everything below is a question about a file, asked by somebody
// who was not there when it was written, which is the only way an
// archive is ever read.
type Lineage struct {
	born map[sim.EntityID]Record
	end  map[sim.EntityID]int
	ids  []sim.EntityID
}

// Deep is how many generations stand between this creature and the
// founder at the top of its line, and Stem is that founder.
//
// Both walk the parent of record and ignore the second parent, and that
// is a choice the caller has to know about. A pedigree with two parents
// in it is a mesh and not a chain: count generations through both and a
// creature has as many depths as it has ways up. The parent of record
// is the one that carried the price of the birth, so the chain it makes
// is the line the valley itself counted when it stamped a generation
// number on the child.
func (l *Lineage) Deep(id sim.EntityID) (int, error) {
	at, n := id, 0
	for {
		rec, ok := l.born[at]
		if !ok {
			return 0, fmt.Errorf("archive: creature %d has no born record", at)
		}
		if rec.Of == 0 {
			return n, nil
		}
		at, n = rec.Of, n+1
	}
}

// Above is every ancestor of one creature and the fewest parent-steps
// that reach it, the creature itself at nought.
//
// Both parents are walked, so this is the whole mesh and not one chain,
// and the count is the shortest way up rather than the only way: two
// creatures that are cousins on one side and half-siblings on the other
// are related by the shorter of the two.
func (l *Lineage) Above(id sim.EntityID) map[sim.EntityID]int {
	steps := map[sim.EntityID]int{id: 0}
	queue := []sim.EntityID{id}
	for len(queue) > 0 {
		at := queue[0]
		queue = queue[1:]
		rec, ok := l.born[at]
		if !ok {
			continue
		}
		for _, up := range [2]sim.EntityID{rec.Of, rec.With} {
			if up == 0 {
				continue
			}
			if _, seen := steps[up]; seen {
				continue
			}
			steps[up] = steps[at] + 1
			queue = append(queue, up)
		}
	}
	return steps
}

// Meet is the most recent common ancestor of two creatures: of every
// ancestor both of them have, the one born latest.
//
// It hands back false when the two share nobody at all, which is a real
// answer and not a failure: two lines running back to two different
// founders that have never crossed have no common ancestor anywhere in
// this world, because the valley was founded with creatures nobody
// made.
//
// Ties are broken by identity, which is the order creatures were made
// in, so two ancestors born on one tick come out in the order the
// archive names them and the answer is a fact about the file rather
// than about how a map felt that morning.
func (l *Lineage) Meet(a, b sim.EntityID) (Meeting, bool) {
	mine, theirs := l.Above(a), l.Above(b)
	var m Meeting
	found := false
	for _, id := range l.ids {
		up, ok := mine[id]
		if !ok {
			continue
		}
		down, ok := theirs[id]
		if !ok {
			continue
		}
		m.Shared++
		rec := l.born[id]
		if !found || rec.Tick > m.Tick || (rec.Tick == m.Tick && id > m.Who) {
			m.Who, m.Tick, m.First, m.Second = id, rec.Tick, up, down
			found = true
		}
	}
	return m, found
}

$ go run ./cmd/roll -mode walk -years 8
roll: 16x12 valley, tick 901, year 1 summer, 90 plants standing at 4654.1 grams

  25 creatures founded on stream 12, each written down as it stood up
  every birth and every death appended to walk.jsonl, and nothing in the valley reads it
  the temperament and look genes go in the file: yes

  25 founded, 546 born, 404 struck off, 167 still walking after 28350 ticks

  the archive after 8 years
    born records                                           571
    died records                                           404
    records in all                                         975
    bytes                                              5213895
    a born record, mean                                 9096.8
    a died record, mean                                   48.6
    sha256 0151d46a4cafb979fac5f70561e1e4413dcdf14bb95396679a2c348ecea574ac

  every born record built back into a genome
    genomes rebuilt                                        571
    mean Bulk factor over all of them                   0.6198
    mean links in a wiring                               144.7

  what the file says about who came from whom
    creatures it names                                     571
    of those, ones it never wrote a death for              167
    founders: a record with no parents in it                25
    founders any of the survivors came from                  1
    the deepest line, in generations                        33
    and the creature at the end of it                      491

  that creature's line of descent, parent of record at every step
        up   creature      tick    year    born to     with  species
         0        491     27390       8        476      413        1
         1        476     27262       8        464      317        1
         2        464     27187       8        453      333        1
         3        453     27102       8        444      236        1
       ...        ...
        33         23       901       1          0        0        1

  the oldest and the youngest creature still walking, and where their lines meet
                                             creature    born on
    the oldest still walking                      105      15365
    the youngest still walking                    571      29239
    the last ancestor they have in common         105      15365
    ancestors the two of them share in all                  15
    steps up from the oldest to reach it                     0
    and from the youngest                                    6
    the oldest is itself the ancestor: it is up the youngest's line

Five megabytes for eight years, and the block underneath it is the check that makes the rest of the page mean anything: all five hundred and seventy-one born records turned back into genomes, links, biases, body factors and all, with nothing refused. The mean wiring holds 144.7 links against the 144 a founder is wired with, so a handful of lines had grown a nerve by year eight, and the mean bulk factor sits at 0.6198: the ledger had already spent eight years making the animals smaller than the row they were founded from.

Then the pedigree, and the fourth line of it is the one to stop at. Twenty-five founders stood up in year one. Of the hundred and sixty-seven creatures still walking in year eight, every single one descends from one of them. Twenty-four founding lines went out inside eight years, and the file says so without anything in the valley having been asked to keep score: it is a walk up the of field of every survivor until it reaches a record with no parents in it. Printed out, that walk is the block below, and creature 491 stands thirty-three generations from creature 23, which stood up on tick 901 with nobody above it. The with column says most of those births had a second parent as well, and none of them was followed to get the thirty-three.

The last block shows why the mesh matters. Take the oldest animal still standing, creature 105, and the youngest, creature 571, born four years later. The last ancestor they have in common is creature 105 itself: the oldest survivor is directly up the youngest one's line, six steps above it. They share fifteen ancestors, and the walk picked the most recent of the fifteen rather than the first it met. Nothing in the valley recorded that. It came out of two fields on nine hundred and seventy-five lines of text.

$ go test -count=1 ./internal/gene/ -run 'ARecordHandsBack|AnArchiveSpendsNo|TwoRunsOfOneSeed|ACreatureBornTwice|TheMeetingIsThe|AFrugalArchive' -v
=== RUN   TestARecordHandsBackTheGenomeItWasWrittenFrom
--- PASS: TestARecordHandsBackTheGenomeItWasWrittenFrom (0.00s)
=== RUN   TestAnArchiveSpendsNoNumbers
--- PASS: TestAnArchiveSpendsNoNumbers (0.02s)
=== RUN   TestTwoRunsOfOneSeedWriteTheSameArchive
--- PASS: TestTwoRunsOfOneSeedWriteTheSameArchive (0.02s)
=== RUN   TestACreatureBornTwiceIsRefused
--- PASS: TestACreatureBornTwiceIsRefused (0.00s)
=== RUN   TestTheMeetingIsTheLatestAncestorBothOfThemHave
--- PASS: TestTheMeetingIsTheLatestAncestorBothOfThemHave (0.00s)
=== RUN   TestAFrugalArchiveWillNotBuildAGenome
--- PASS: TestAFrugalArchiveWillNotBuildAGenome (0.00s)
PASS
ok  	theworld/internal/gene	0.044s

The first is the promise the archive makes: a genome written to the file and read back out of it is the same genome, gene for gene and link for link, checked on a grown wiring and on a flat row. The second is what lets an archive be switched on inside a valley that was finished two chapters ago: a pool with a file open and a pool without one spend the same numbers and make the same creatures. The third says one seed writes the same bytes twice running. The fourth refuses a file with two born records under one identity, which is what a second run appended to yesterday's archive looks like. The fifth is the ancestry walk on a pedigree of six creatures small enough to check by eye, and the sixth is the failure below.

// internal/gene/roll.go — inside type Roll struct
	// Frugal is the archive as it is tempting to write it: only the
	// blocks something in this valley actually reads. The ten body
	// factors are priced every tick and the wiring drives every
	// action; the temperament and look genes are read by nothing, so
	// a frugal archive leaves them out of the line. It is kept behind
	// a flag so what a missing column costs can be run instead of
	// described, and it is false in every run this book ships.
	Frugal bool

$ go run ./cmd/roll -mode walk -years 8 -frugal
roll: 16x12 valley, tick 901, year 1 summer, 90 plants standing at 4654.1 grams

  25 creatures founded on stream 12, each written down as it stood up
  every birth and every death appended to frugal-walk.jsonl, and nothing in the valley reads it
  the temperament and look genes go in the file: no

  25 founded, 546 born, 404 struck off, 167 still walking after 28350 ticks

  the archive after 8 years
    born records                                           571
    died records                                           404
    records in all                                         975
    bytes                                              5138759
    a born record, mean                                 8965.2
    a died record, mean                                   48.6
    sha256 b03bc84382bd512cf2b3592b0fba62dcf94de2f435683a4645748ff44e9e37c1

  every born record built back into a genome
roll: archive: creature 1, born on tick 901: 0 temperament genes, not 3
exit status 1

Temper genes as a control group

Thirteen of the numbers in a genome are something other than the controller. Ten are the body: bulk, the size of a mouthful, what a gram costs to keep, how fast the animal can walk, how much of a bite it converts. Every one of the ten is read on every tick of every animal's life, and an animal whose ten are badly chosen puts less in its store, cannot afford a child, and ends as a carcass. The other three are temperament, and nothing reads them at all.

Both sets go through the same machinery. The same crossing awards each of them to one parent or the other, the same mutation pass draws its two numbers for each, the same clamp holds each inside its own range. The only difference anywhere in the program is that ten of them have consequences and three do not, which makes the three a control group that costs twenty-five bytes a record. Before any of it can be read, though, there has to be something to read it against. What does a gene do when nothing is pushing it at all?

A law worked out on paper is a claim, so here it is run. One founder, two thousand copies of it, and then every copy copied and mutated again and again, each line going its own way and none of them ever meeting. Nothing is selected, nothing is crossed, no wiring grows: the only thing happening in this bench is the mutation pass, over and over, on genes nobody is judging.

// cmd/roll/main.go

// Step is the spread one generation of copying opens up in a gene that
// starts in the middle of its own range, and Rest is the spread that
// gene ends at once it has been copied enough times to have forgotten
// where it began. Neither is measured off a run: both come off the two
// mutation rates, and they are what the law on this page is checked
// against.
//
// Step is the square root of the wander one pass adds. A gene is
// replaced outright once in a hundred, and a number thrown evenly over
// a range is a twelfth of that range squared away from the middle of it
// on average; it is nudged eight times in a hundred by a number spread
// over a tenth of the range, which is a twelfth of a tenth squared.
// Rest is where the wandering stops: a gene that has been replaced
// enough times is a number thrown evenly over its range, and no amount
// of further copying makes it more spread than that.
var (
	Step = math.Sqrt(gene.Usual.Replace/12 +
		(gene.Usual.Perturb-gene.Usual.Replace)*(2*gene.Usual.Step)*(2*gene.Usual.Step)/12)
	Rest = 1 / math.Sqrt(12)
)

	m := gene.NewMint(gene.Wide, beast.Acts)
	from := gene.Root(mind.Brains(seed), m)
	herd := make([]*gene.Genome, lines)
	for i := range herd {
		herd[i] = from.Copy()
	}
	mut := gene.Mutations(seed)

	fmt.Printf("  %7s %12s %12s %14s %12s\n",
		"gen", "temper sd", "body sd", "s x sqrt(gen)", "temper/law")
	for g := 1; g <= gens; g++ {
		for i, at := range herd {
			kid := at.Copy()
			gene.Usual.Mutate(kid, mut)
			herd[i] = kid
		}
		if g&(g-1) != 0 && g != gens {
			continue
		}
		t, b := spread(herd, gene.Tempers, temper), spread(herd, gene.Bodies, body)
		law := Step * math.Sqrt(float64(g))
		fmt.Printf("  %7d %12.4f %12.4f %14.4f %12.3f\n", g, t, b, law, t/law)
	}

$ go run ./cmd/roll -mode drift
roll: how far a gene wanders when nothing is pushing it

  2000 lines copied from one founder, each birth one copy and one mutation pass
  0.01 of genes replaced outright, 0.08 nudged by up to 5 percent of the range
  spreads are shares of each gene's own range, so a body factor and a
  temperament gene are the same size of number here

      gen    temper sd      body sd  s x sqrt(gen)   temper/law
        1       0.0310       0.0416         0.0300        1.033
        2       0.0425       0.0579         0.0424        1.001
        4       0.0594       0.0797         0.0600        0.990
        8       0.0854       0.1130         0.0849        1.006
       16       0.1191       0.1541         0.1200        0.992
       32       0.1599       0.2021         0.1697        0.942
       64       0.2062       0.2504         0.2400        0.859
      128       0.2510       0.2817         0.3394        0.740
      256       0.2818       0.2910         0.4800        0.587
      512       0.2907       0.2901         0.6788        0.428

  s is the step one generation of copying opens up, 0.0300, worked out from the two
  rates and not from this run, and the law is s times the square root of the
  generations. a gene copied enough times is a number thrown evenly over its
  own range, whose spread is 0.2887, and the law arrives there at generation 93

Thirteen-gene 200-year archive

Now the run this chapter was built for: two centuries of the valley with the archive open from before the first founder stands up, and then the file read back by something that has never seen the valley.

$ go run ./cmd/roll -mode herd -years 200 -every 20
roll: 16x12 valley, tick 901, year 1 summer, 90 plants standing at 4654.1 grams

  25 creatures founded on stream 12, each written down as it stood up
  every birth and every death appended to herd.jsonl, and nothing in the valley reads it
  the temperament and look genes go in the file: yes

   year   alive     born     died   records       bytes
      1      25        0        0        25      222466
      2       9       14       30        69      348905
      3       8       32       49       106      511174
      4      19       53       59       137      700928
      5      15       74       84       183      892439
     21     333     1091      783      1899    10217286
     41     333     1696     1388      3109    15783367
     61     336     1718     1407      3150    15985299
     81     335     1718     1408      3151    15985350
    101     336     1720     1409      3154    16003741
    121     337     1722     1410      3157    16021998
    141     336     1724     1413      3162    16040506
    161     336     1724     1413      3162    16040506
    181     336     1724     1413      3162    16040506
    201     336     1725     1414      3164    16049646

  25 founded, 1725 born, 1414 struck off, 336 still walking after 719550 ticks

  the archive at the end of two hundred years
    born records                                          1750
    died records                                          1414
    records in all                                        3164
    bytes                                             16049646
    a born record, mean                                 9131.2
    a died record, mean                                   49.5
    sha256 577e17a1978bd5dbc6329e46b059b135f27f20d0b3db7d8a136e1a742a8a8538
    records a year, over 200 years                           15.8
    bytes a year                                         80248

  what the file says about who came from whom
    creatures it names                                    1750
    of those, ones it never wrote a death for              336
    founders: a record with no parents in it                25
    founders any of the survivors came from                  1
    the deepest line, in generations                        50
    and the creature at the end of it                     1738

  that creature's line of descent, parent of record at every step
        up   creature      tick    year    born to     with  species
         0       1738    155205      44       1736      842        1
         1       1736    155149      44       1731     1658        1
         2       1731    154449      43       1722     1454        1
         3       1722    146601      41       1718     1421        1
       ...        ...
        50         23       901       1          0        0        1

  the oldest and the youngest creature still walking, and where their lines meet
                                             creature    born on
    the oldest still walking                      105      15365
    the youngest still walking                   1750     709514
    the last ancestor they have in common         105      15365
    ancestors the two of them share in all                  15
    steps up from the oldest to reach it                     0
    and from the youngest                                    6
    the oldest is itself the ancestor: it is up the youngest's line

  what two hundred years did to thirteen genes, read out of the archive
  spreads are shares of each gene's own range, so a body factor and a
  temperament gene are the same size of number
  generations are counted up the line of record, and creatures are pooled
  into bands of 5 so that every row is read off enough of them
        gen  creatures    temper sd      body sd  s x sqrt(gen)   temper/law
          0         25       0.0000       0.0000         0.0000        0.000
        1-5         45       0.0363       0.0206         0.0537        0.676
       6-10         56       0.0741       0.0480         0.0869        0.853
      11-15        186       0.0954       0.0649         0.1102        0.865
      16-20        362       0.1383       0.0711         0.1275        1.085
      21-25        418       0.1575       0.0699         0.1440        1.094
      26-30        327       0.1446       0.0747         0.1569        0.922
      31-35        168       0.1692       0.0763         0.1719        0.984
      36-40        112       0.1938       0.0841         0.1846        1.050
      41-45         35       0.1264       0.0848         0.1979        0.639
      46-50         16       0.1338       0.0673         0.2053        0.652

  the thirteen genes at the end, over the 336 creatures still walking
  every one of them started at the same number in every founder
  gene              block   founding       mean         sd   sd / range
  Bulk               BODY     1.0000     0.3685     0.1223       0.0326
  Full               BODY     1.0000     0.8519     0.1734       0.0462
  Basal              BODY     1.0000     1.0077     0.1127       0.0300
  Work               BODY     1.0000     1.0199     0.2787       0.0743
  Bite               BODY     1.0000     1.9092     0.6549       0.1746
  Convert            BODY     1.0000     0.9906     0.1138       0.0304
  Reach              BODY     1.0000     1.0552     0.0686       0.0183
  Sight              BODY     1.0000     1.1641     0.2240       0.1493
  Top                BODY     1.0000     1.1926     0.4220       0.1125
  Swing              BODY     1.0000     1.1436     0.1749       0.0467
  aggression       TEMPER     0.5000     0.6332     0.2669       0.2669
  wariness         TEMPER     0.5000     0.4922     0.2317       0.2317
  tameness         TEMPER     0.5000     0.4679     0.1071       0.1071

  719550 ticks in 7m13.935s, 1658 ticks a second (measured here; yours will differ)

Read the year table as the archive's own account of when this valley had a history. By the twenty-first year the file holds 1,899 records and ten megabytes; by the forty-first, 3,109; by the two hundred and first, 3,164. Fifty-five records in a hundred and sixty years, in a valley where three hundred and thirty-six animals are standing on the ground the whole time. The run prints 15.8 records a year and that mean describes no year in it: the first twenty-one years wrote sixty percent of its records and the last century wrote forty-six kilobytes. Every byte and every record on that table is the same on any machine. The seven and a quarter minutes at the foot of it are an eight-core Ryzen 7 3700X and nothing else: they carry a label saying so, and nothing on this page is worked out of them.

What stopped is births. A cell of ground carries one body, the ground filled up, and after that a birth has to wait for somebody to die; in a valley where the standing animals are well fed, almost nobody does. The archive is a faithful record of a world that ran out of things to record.

The file came to 16,049,646 bytes across 3,164 records, and its sha256 is the one printed: run the same command again and the same three thousand lines come out in the same order to the byte. A born record now averages 9,131 bytes against the 8,885 a founder took, and the difference is drift showing up as text: a founder's genes are 1 and 0.5, four characters apiece, while a gene copied fifty times is something like 0.36851472066103135 and costs nineteen.

The pedigree block says one founding line out of twenty-five is left, and that the deepest creature, number 1738, stood fifty generations from creature 23. The block below it is the clearest picture of what a saturated valley is. The youngest animal still walking was born in year 198 and the oldest in year 5, and they are not distant cousins: the older one is directly up the younger one's line, six steps above it, and has been standing beside its own descendants for a hundred and ninety-three years.

Ledger records body and temper differently

The drift table is read out of the archive alone: every born record turned back into a genome, every creature's generation counted by walking the of field to a founder, and the creatures pooled into bands of five so that no row rests on a handful of animals.

The temperament column climbs, and it climbs at the rate the law says: 0.0363 in the first five generations and 0.1938 by the thirty-sixth to fortieth, against a law wanting 0.0537 and 0.1846 at those depths. The last column crosses 1.000 in both directions and sits between 0.85 and 1.09 through the middle of the table, which is what a measured curve does when it is following a law and being read off a few hundred animals at a time. Three genes that nothing in this program looks at have wandered a fifth of the way across their range in forty generations, by copying alone, as fast as a bench with nothing alive in it says they predict.

The body column does not climb. It reaches 0.0711 by the twentieth generation and is still 0.0841 at the fortieth, by which point the temperament column is nearly two and a half times it. That gap is the ledger: ten genes that decide whether an animal can afford a child, held inside a tenth of their range for fifty generations, while three genes with no consequences drift off wherever the numbers take them.

And the gap is wider than the two columns make it look, which is what the bench had to run first to show. Left alone, a body factor drifts faster than a temperament gene, because it starts a fifth of the way up its range. The bench put an unselected body block past 0.20 by thirty-two generations; the valley's is 0.0763 at thirty-five. The ten are held to somewhere near a third of what they would do if nothing were watching, and the three are held to nothing at all.

Two wrinkles before the table at the bottom. The last two bands dip to 0.1264 and 0.1338 against a law that wants 0.20, and they are read off thirty-five creatures and sixteen: animals that deep are the tail of one fast-breeding family, closer to each other than two picked at random at that depth would be. The first band sits low too, at 0.676 of the law, because a gene four copies old has usually not been touched at all.

The thirteen rows at the end tell it one gene at a time, over the animals still standing. All of them started identical in every founder: ten body factors at 1.0000, three temperament genes at 0.5000. Bulk has moved to 0.3685 and Bite to 1.9092, both tightly held at 0.0326 and 0.1746 of their ranges; Convert has hardly moved at all, at 0.9906 with a spread of 0.0304, which says a mouthful converted badly is a dead animal and there is nothing else to add. The valley picked a small animal with a big mouth and then stopped letting anybody argue.

The three temperament genes have means of 0.6332, 0.4922 and 0.4679 and spreads of 0.2669, 0.2317 and 0.1071, and they do not agree with each other. Aggression is most of the way to the 0.2887 a gene thrown at random sits at; tameness is lower than the law wants at fifty generations. Nothing in the code separates them. What separates them is that this population came through a bottleneck of one founding line, and the spread of a gene nobody is judging is itself a wandering number: the law describes it on average over many lineages, and this run is one lineage. Reading the three as three measurements would be reading noise. Reading them together against the ten is the measurement.

Why append-only history is enough

Three things here outlive this valley, and none of them is about evolution.

The columns of a log are the questions you will be allowed to ask. While a run is alive a column costs bytes, and this one cost three-tenths of one percent. Once the run is over the same column cannot be bought at any price: the objects are gone, and a fresh run with the column added is a different run, because in a seeded system the extra field changes the arithmetic that produced the numbers. The rest of this archive falls out of that asymmetry. It is append-only because a tick cannot afford to reorganise a file, it carries two kinds of record because a death is news that arrives long after the birth, and it writes a cause word with one value in it so the second value has somewhere to go. Log what you are not using. What you are using, you can usually work out again.

A reader that can see only the file is the only honest test of the file. The pedigree here holds no pool, no roster and no valley: it is a map from an identity to a record, and every question on this page was answered out of that. A reader allowed to reach the live objects leans on them without meaning to, and the leaning shows up on the one day it cannot be fixed. The same instinct puts the checks at the boundary, where a complaint can still name a creature and a tick.

Measure something you are not touching. "The ten body factors were 0.08 spread after forty generations" means nothing on its own. It becomes a measurement when something beside it went through identical machinery with no consequences attached: three genes at 0.19, and a law from the mutation rates saying 0.18. A control turns a number into a finding, and controls cannot be added afterwards any more than columns can, which is why the two halves of this chapter are one chapter.

One smaller thing, because it turns up far outside this book. A spread growing with the square root of the steps is what independence looks like: each step knows nothing about the last, so the squares add and the roots do not. Measure a spread growing in proportion to time instead and the steps are not independent, and something is pushing. And every square-root law ends, because the thing being measured has walls: here they are 0.00 and 1.00, and the law runs into them at ninety-three generations.

Three hundred and thirty-six animals are standing on that ground at the end of the run, and the file says every one of them is a different set of numbers: a bulk factor a third of what its founders carried, a bite nearly twice theirs, a wiring with links its great-grandparents never held, and four look genes that have been copied fifty times and drifted as freely as the temperament ones. On the screen they are all the same handful of pixels in the same colour: the file can tell any two of these animals apart, and nothing looking at the valley can.