The World Vol 7 · The Living Valley
ch 80 / 105
Chapter 80

Weeks Without You

The daemon boundary

Every run this volume has printed was watched. Somebody typed a command, sat there, and read what came out. Every one of them ended, because a bench is a program with an end written into it, and the moment the last line was printed the world it had been running stopped existing.

A daemon's loop has exactly one boundary in it, the gap between the tick that has returned and the tick that has not started, and everything that is not a tick happens there: the save, the signal that asks it to stop, and the line of history it writes. A tick that is interrupted is a tick that never happened, and the world comes back from the last boundary it wrote down.

The Hollow is not short of anything it needs to be a world. Ground weathers. Plants seed and die back. Two rows of animals have a road for meat between them. Controllers grow connections their ancestors did not have. Seven tables sit underneath it, and a founding document named it before it had any history to name.

What it has never had is a minute of its own. A world that exists only while somebody is looking at it is a demonstration.

This page turns the valley into a process that is left alone. It has to tick without being told to, which is a loop with no end in it. It has to write itself down without being told to, which is the save chapter's snapshot put on a schedule.

It also has to survive being stopped at a moment no operator chose: a power cut, an out-of-memory killer, somebody pulling a cable, none of which waits for a tick to finish.

The second and third meet in one place. A snapshot taken between two ticks describes a moment the world was in; one taken half way through a creature phase describes a moment the world was never in, and it is the same size, holds the same eight parts and passes every completeness check there is. A daemon is the hardest place in this book to keep that rule, because it is the first program here whose job is to write to a database while a world runs, and the only thing between it and a torn save is where one line of code sits.

By the end of this page there is a binary that founds a world out of the founding document, ticks it unwatched and saves it every so many ticks; ten valley-years run straight through, and the same ten run by two processes with a kill in the middle of a creature phase between them; two tapes held against each other byte for byte; a chronicle that records a world picked up twice and put down once; the valley-years an hour of wall clock buys, measured with a live population and a database writing underneath it; and a governor that changes when a tick happens and nothing about what it does.

That last clause is the rate rule this volume is built on, and it belongs on the page before any code does, because it forbids the obvious optimisation. Running unwatched in this volume is the same tick, taken faster. There is no coarse resolver here, no summary pass, no second cheaper simulation that runs while the operator is away. A valley that ran at ten ticks a second while somebody watched runs at whatever the machine will carry when the operator is away, and every number it produces is the number it would have produced at ten in the same reference environment. The rate is a governor and a measurement, never a change to the simulation rules.

That defines this daemon, not every field lifecycle. A paused field advances no simulation time. Exact catch-up runs every missed tick with the same ordered inputs and pays for all of that work. An approximate offline resolver skips work under different rules and produces a different history, even if its own seed makes that history repeatable. HOT/FROZEN/ARCHIVED fields must declare which policy they use; freezing alone does not make a settlement grow. No approximate path replaces the canonical tick loop on this page.

One rule from the chapter the database arrived in survives all of this: nothing in internal/sim, internal/terra, internal/beast, internal/gene or internal/mind imports internal/store, and nothing inside a tick calls into it. Every SQL statement on this page is sent from between two ticks, and nothing in the loop below can block on a socket while a creature is half stepped, because there is no line inside a tick from which a socket is reachable.

The binary is called weeks and it is a daemon wearing a bench's clothes. Given nothing it is a process with no end in it. Given -ticks it stops on a named boundary, and given -die-at it is killed inside a creature phase on a named tick. Those two flags exist so that a page can quote a run and a machine can check one, and neither of them changes a single number the world produces.

The tick loop save point

The loop that runs a world is three statements: take a tick, fold it into the digest, and wait a moment if something is holding you to a rate. The interesting part is what is allowed to sit beside those three, and the answer is that everything else in the loop happens at the boundary and nowhere else.

▣ Build · stage 1 — a loop with no end in it
// cmd/weeks, the daemon's loop
for stop == "" {
	// The boundary. Everything in this loop that is not a tick
	// happens here and nowhere else, because here is the one place
	// where no phase is part way through, no view is open, no
	// creature is half stepped, and the mint's table cannot be
	// read again.
	if w.tick()%every == 0 && w.tick() != saved {
		bytes, dropped := keep(ctx, db, w, name, hold)
		saved = w.tick()
		...
	}
	select {
	case <-asked:
		stop = "asked to stop, and the tick it was in finished first"
	default:
	}
	if stop == "" && until > 0 && w.tick() >= until {
		stop = "stopping on the boundary it was told to stop on"
	}
	if stop != "" {
		break
	}

	w.v.Tick()
	w.fold()
	ran, folded = ran+1, folded+1
	if hz > 0 {
		govern(start, ran, hz)
	}
}

Read the select first, because it is the part that looks wrong and is not. A signal handler in Go runs on its own goroutine, so the operating system can deliver a SIGTERM at any instant it likes, including half way through a creature phase. All the handler does with it is put one value in a channel. The loop takes that value out at the top, where nothing is in flight, so a stop always happens between two ticks even though the signal that asked for it did not. default: is what makes the read cost nothing: the loop looks, finds nothing, and carries on, several thousand times a second.

The two ways this process stops meet on one line. A signal writes a sentence into stop, a tick count on the command line writes a different one, and after that there is one path out of the loop and one place the last save is taken. A daemon whose graceful shutdown is a separate code path from its ordinary exit has two shutdowns, one run every day and one run on the afternoon everything is already going wrong.

The save is guarded by w.tick() != saved, and that guard is doing real work. A process that has restored a world is standing on the tick a snapshot was taken at, and writing that snapshot again would be a second row with the same world and the same tick, which the schema refuses outright.

Which leaves the question of how the process gets a world at all, and there are two answers behind one call. A database with no world row in it has never been founded, so the process reads the founding document out of the binary and writes one. A database that has a world row was founded by an earlier process, possibly a long time ago, so this one reads the newest snapshot and puts the world back.

▣ Build · stage 2 — founded, or picked up
// cmd/weeks, the whole of how a process starts
func begin(ctx context.Context, db *store.DB, w *world, seed uint64) (store.Doc, int, bool) {
	n, err := db.Count(ctx, "world")
	if err != nil {
		die(err)
	}
	if n == 0 {
		raw, err := configs.Files.ReadFile(configs.Genesis)
		if err != nil {
			die(err)
		}
		doc, err := store.ReadDoc(raw)
		if err != nil {
			die(err)
		}
		if err := db.Genesis(ctx, doc, seed); err != nil {
			die(err)
		}
		held, err := db.Doc(ctx)
		if err != nil {
			die(err)
		}
		w.stood()
		return held, w.tick(), false
	}

	held, err := db.Doc(ctx)
	if err != nil {
		die(err)
	}
	name := held.One("world")
	tick, ok, err := db.Newest(ctx, name)
	...
	parts, err := db.Load(ctx, name, tick)
	...
	if err := w.carry(parts); err != nil {
		die(err)
	}
	return held, tick, true
}

The founding branch reads configs/genesis.json and the other does not, and that asymmetry is the founding chapter's rule arriving in the only program that could break it. A process that did not found this world has no business reading the file: the names in it are whatever is on that machine's disk this afternoon, and the names the world answers to are in the world row. Even the founding branch stops using the file the moment it has written it, calling db.Doc and holding what comes back, so from the next line onward both branches are one program getting its vocabulary from one column. It is asked once and never again: a query is a thing that can be slow and a name is a thing a running world prints constantly.

One number a restored world cannot get from its snapshot is the tick it opened on. A snapshot holds what a tick can change, and the tick a world was founded on is not one of those. It does not have to be: the terrarium is stood up half way through its first spring and ticked until the calendar reaches the first tick of summer, and a warm-up that starts and finishes on the calendar lands on one tick whatever the seed, the ground or the population. Every world founded by this recipe opens on that same valley tick, so the daemon carries it as a constant and subtracts.

That is enough to run. The database is killed and started first so that it is properly empty: its data directory is a tmpfs, so a restart is a fresh server with nothing in it, and this run founds its own world.

▣ Build · stage 3 — ten unwatched valley-years
$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go sh -c 'go build -o /tmp/weeks ./cmd/weeks && /tmp/weeks -mode run -ticks 36000 -save 3600 -tape /tmp/weeks-80/straight-80.tape'
weeks: postgres at world-db:5432, database world, as world
weeks: nothing here was founded, so this process reads the document and founds it
weeks: The World, on the ground it calls The Hollow, 29 hobbs standing at tick 0
weeks: a save every 3600 ticks, the newest 3 kept, as fast as this machine will carry it
weeks: it stops on the boundary at tick 36000

      tick  walking     born     gone  genomes      saved  dropped  the ticks since the last save
         0       29        0        0       29     298475        0  (no tick yet)
      3600       12       48       65       77     754043        0  7fc490d8b48449d5
      7200       26       89       92      118    1162894        0  64a389e5f89573e8
     10800       31      132      130      161    1588888        1  d112d46505f4f98f
     14400       20      198      207      227    2216490        1  dc3c272f63a19257
     18000       57      267      239      296    2857976        1  e21f6cadd002631c
     21600       10      339      358      368    3510713        1  7b5c4d0982dee29c
     25200       45      407      391      436    4142876        1  127af7dd42e40f69
     28800      179      707      557      736    6889280        1  2cd2bc767392b8f5
     32400      190      892      731      921    8563913        1  625d86900ed7ff28
     36000      199     1029      859     1058    9797377        1  34c1c097d0624303

weeks: stopping on the boundary it was told to stop on
weeks: already written down on the boundary at tick 36000
weeks: stopped, and every connection handed back

  36000 ticks in 12.022s, 2995 ticks a second (measured here; yours will differ)

Take the first row before anything else. Tick nought, twenty-nine animals standing, nothing born and nothing dead, and a snapshot of 298,475 bytes already in the database. A founding process saves before it runs a tick, because a world made and then killed a second later is a world nothing can carry on, and tick nought is a boundary like any other. The digest column says no tick yet because no tick has been folded into it.

The population column is the valley behaving the way this volume said it does. Founders are scattered on ground they did not grow up on, most starve, and the count falls to twelve in the first year and to ten in the sixth before the survivors' children fill the place up to a hundred and ninety-nine. The two columns beside it are the ones a daemon cares about: a thousand and twenty-nine births in ten valley-years, and a pool holding a thousand and fifty-eight genomes, every one still there because a pool never forgets a dead animal.

One line of that block belongs to a machine and not to a world, and it is the last one. Every timing on this page was taken on an idle eight-core Ryzen 7 3700X with the database in a container beside the daemon; every other figure in every block is arithmetic and comes out the same anywhere.

Which is what makes the saved column climb the way it does. The snapshot goes from 298,475 bytes to 9,797,377 over ten valley-years, and the growth is almost entirely genomes: the ground under all of it is the same handful of kilobytes in year ten as in year one. Nothing here is leaking. A world that has bred a thousand animals has a thousand animals' worth of heredity to write down.

So the last column of that table is a rule the daemon has to have, and it is the first thing on this page that is an operational decision instead of a correctness one.

▣ Build · stage 4 — the saves it will never need
// internal/store/keep.go
// Forget drops every save of this world but the newest few, and hands
// back how many rows it took away.
//
// A snapshot is not a backup. What a daemon needs in order to come
// back is the last moment it wrote down, and every save older than that
// is a world nobody is ever going to restore. They are not small: this
// world's saves are megabytes apiece and most of that is genomes, so a
// process left running with no rule about this fills a disk at a rate
// somebody can work out in advance and nobody ever does.
//
// The delete is one statement and the parts go with it: snapshot_part
// has ON DELETE CASCADE on its foreign key, so a row leaving the
// snapshot table takes its eight parts with it and there is no second
// statement anybody can forget to write.
func (d *DB) Forget(ctx context.Context, world string, keep int) (int64, error) {
	if keep < 1 {
		return 0, fmt.Errorf("store: a world that keeps %d saves cannot be restarted", keep)
	}
	const q = `DELETE FROM snapshot
	            WHERE world = $1
	              AND tick NOT IN (
	                  SELECT tick FROM snapshot
	                   WHERE world = $1
	                ORDER BY tick DESC
	                   LIMIT $2)`
	tag, err := d.pool.Exec(ctx, q, world, keep)
	if err != nil {
		return 0, fmt.Errorf("store: forgetting the old saves of %q: %w", world, err)
	}
	return tag.RowsAffected(), nil
}

The dropped column in the run above is what that returns: nought for the first three saves and one for every save afterwards, which is what keeping three of something looks like from the inside. The number is a dial and not a law. The one value that is not a dial is nought, and the function refuses it, because a world that keeps no saves is a world that cannot be restarted.

The cascade is the part to copy into other work. The two tables are written together inside one transaction and they come apart together inside one statement, because the foreign key from a part to its snapshot says ON DELETE CASCADE. There is no second delete to write, so there is no second delete to forget, and the bug where the parent rows go and eight megabytes of orphaned parts stay has been made unreachable instead of tested for.

Three claims have been made now and all three can be checked without a database anywhere near them. Taking a snapshot does not change the world it read. A restart is exact from any boundary and not only from the newest one. And a world that keeps no saves is refused.

▣ Build · stage 5 — the three claims, as tests that open no socket
$ go test ./internal/store/ -run 'ASaveOnEveryBoundaryChangesNothing|ARestartFromAnOlderSaveLandsOnTheNewer|AWorldThatKeepsNoSaveIsRefused' -v
=== RUN   TestASaveOnEveryBoundaryChangesNothing
    keep_test.go:32: 200 saves taken between ticks, not one number moved, and 200 ticks on both worlds are at 86869c94c1bb3dd7
--- PASS: TestASaveOnEveryBoundaryChangesNothing (0.52s)
=== RUN   TestARestartFromAnOlderSaveLandsOnTheNewer
    keep_test.go:78: restored from the older save, run 200 ticks, and all 8 parts came out the newer one: 235941 bytes
--- PASS: TestARestartFromAnOlderSaveLandsOnTheNewer (0.13s)
=== RUN   TestAWorldThatKeepsNoSaveIsRefused
    keep_test.go:89: store: a world that keeps 0 saves cannot be restarted
--- PASS: TestAWorldThatKeepsNoSaveIsRefused (0.00s)
PASS
ok  	theworld/internal/store	0.659s

The first is the claim everything else rests on and the easiest to assume instead of checking. Two valleys are founded off one seed; one has a snapshot taken after every single tick for two hundred ticks and the other has none; then both run on and fold. The count of numbers drawn is compared as well as the digest, because a snapshot that spent a draw and reached the same digest by luck is exactly what surfaces in year forty.

The second is what makes keeping three saves a choice about a disk. A world is run to a boundary and saved, run on and saved again, and a third world is built out of the older of those two and run forward the same distance. All eight of its parts come out identical to the later save, byte for byte, so any boundary would have done. The daemon uses the newest because it costs the least to catch up from.

Killing the creature phase

Now the forced stop. A creature phase does six things in order: it builds the view, casts every fan, steps the roster, sweeps the struck, buries the dead and mints the births. Catch it in the middle and the process is holding an index describing a valley that has already moved, a roster half of which has had this tick, a list of dead nothing has acted on, and a mint holding marks for a tick still being worked. There is no way to write that down and no reason to want to.

So the daemon does not try. It arranges instead for that moment to be survivable, and the only test of whether it is survivable is to go there and be killed. The kill hangs off the seam a valley leaves for whatever lives in it, after the view is built and before a single creature has been stepped.

▣ Build · stage 6 — a process that stops between two machine instructions
// cmd/weeks, the kill
//
// The kill hangs off the seam a phase leaves, which is after the
// view has been built and before a single creature has been
// stepped. Nothing catches SIGKILL: this process stops between one
// machine instruction and the next, with a phase half open, which
// is the worst moment there is and therefore the only one worth
// proving anything about.
if dieAt > 0 {
	w.mid = func() {
		if w.tick() == dieAt {
			fmt.Printf("weeks: killed inside the creature phase of tick %d\n", dieAt)
			syscall.Kill(os.Getpid(), syscall.SIGKILL)
		}
	}
}

SIGKILL and not SIGTERM, and the difference is the whole point. A SIGTERM is a request: it goes into the channel, the loop reads it at the boundary, and the world is written down on the way out. SIGKILL cannot be caught, blocked or handled. The kernel takes the process away: no deferred function runs, no connection is handed back, nothing is said on the way out, and any transaction that happened to be open is rolled back by a server that finds out when the socket closes.

The process sends the signal to itself so that a page can quote a run. A kill from outside lands wherever a machine happens to be when it arrives, at a different tick every time, and a chapter cannot print such a thing twice. Sent from inside on a named tick it is the same death by the same signal at the same seam, arranged so two people can compare answers. The line the process prints on its way out arrives because Go's fmt.Printf writes straight to the file descriptor with no buffer in the way.

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go sh -c '/tmp/weeks -mode run -ticks 36000 -save 3600 -tape /tmp/weeks-80/killed-80.tape -die-at 19500; echo "exit status $?"'
weeks: postgres at world-db:5432, database world, as world
weeks: nothing here was founded, so this process reads the document and founds it
weeks: The World, on the ground it calls The Hollow, 29 hobbs standing at tick 0
weeks: a save every 3600 ticks, the newest 3 kept, as fast as this machine will carry it
weeks: it stops on the boundary at tick 36000

      tick  walking     born     gone  genomes      saved  dropped  the ticks since the last save
         0       29        0        0       29     298475        0  (no tick yet)
      3600       12       48       65       77     754043        0  7fc490d8b48449d5
      7200       26       89       92      118    1162894        0  64a389e5f89573e8
     10800       31      132      130      161    1588888        1  d112d46505f4f98f
     14400       20      198      207      227    2216490        1  dc3c272f63a19257
     18000       57      267      239      296    2857976        1  e21f6cadd002631c
weeks: killed inside the creature phase of tick 19500
Killed
exit status 137

Killed is the shell saying what happened to its child, and 137 is how a shell reports a signal: 128 plus the signal's number, and SIGKILL is 9. No error message comes from the program because the program was not asked. The world it held existed in one process's memory and that memory is gone. What is left is what reached the database, and the last thing that did was the save at tick 18,000: fifteen hundred ticks were finished after it and the next one was half done, and none of them is anywhere now.

Now start another process against the same database. It is told nothing about what happened; there is nothing to tell it.

▣ Build · stage 7 — a process that finds a world already going
$ podman exec -w /bench world-go /tmp/weeks -mode run -ticks 36000 -save 3600 -tape /tmp/weeks-80/killed-80.tape
weeks: postgres at world-db:5432, database world, as world
weeks: The World was founded before this process existed, and its newest save is at tick 18000
weeks: restored 152 plants, 57 hobbs, 296 genomes; founded nothing and scattered nobody
weeks: a save every 3600 ticks, the newest 3 kept, as fast as this machine will carry it
weeks: it stops on the boundary at tick 36000

      tick  walking     born     gone  genomes      saved  dropped  the ticks since the last save
     21600       10      339      358      368    3510713        1  7b5c4d0982dee29c
     25200       45      407      391      436    4142876        1  127af7dd42e40f69
     28800      179      707      557      736    6889280        1  2cd2bc767392b8f5
     32400      190      892      731      921    8563913        1  625d86900ed7ff28
     36000      199     1029      859     1058    9797377        1  34c1c097d0624303

weeks: stopping on the boundary it was told to stop on
weeks: already written down on the boundary at tick 36000
weeks: stopped, and every connection handed back

  18000 ticks in 8.211s, 2192 ticks a second (measured here; yours will differ)

The second line is the one to sit with. This process cannot tell that anything went wrong. It asked the database what the newest save was, got tick 18,000, built a world out of it and started ticking, and a process picking up after a clean shutdown would have printed the same sentence. There is no crash flag in the schema, no unclean-shutdown marker and no recovery path, because there is nothing for one to do: the state a restart begins from is a committed transaction either way.

The fifteen hundred ticks the first process ran past its last save are not recovered. They are run again, from the same numbers, by different code in a different process, and they come out the same. That is the whole of what the boundary buys and it is why the page can be this short: there is no repair here, only a world that starts from a moment it was definitely in.

The rate on the last line is lower than the straight run's, and the reason is not the restart. This process began with fifty-seven animals on the ground and a pool of two hundred and ninety-six genomes, where the founding run began with twenty-nine animals in a nearly empty valley, and a tick in a full valley costs more than a tick in an empty one. Reading the world back took a fraction of a second out of eight.

Both runs wrote a tape: one line a save, carrying the world's own numbers and a digest folded over every tick since the previous line. The digest is opened empty at each save, so a line covers exactly the span between two boundaries, which is the one unit a restarted process can match. The straight run's tape was written by one process. The other was written by two, with a kill between them, appending to the same file.

▣ Build · stage 8 — the two tapes
$ podman exec -w /bench world-go /tmp/weeks -mode same -dir /tmp/weeks-80 -seam 18000
weeks: killed and restarted, against run straight through

                                               bytes  sha256, first 16
  run straight through                          1080  0e83f8e8cb96ec8e
  killed and restarted                          1080  0e83f8e8cb96ec8e
  the same bytes, and therefore the same run

   line      tick  run straight through killed and restarted
      1      3600  7fc490d8b48449d5     7fc490d8b48449d5     the same line
      2      7200  64a389e5f89573e8     64a389e5f89573e8     the same line
      3     10800  d112d46505f4f98f     d112d46505f4f98f     the same line
      4     14400  dc3c272f63a19257     dc3c272f63a19257     the same line
      5     18000  e21f6cadd002631c     e21f6cadd002631c     the same line  the first process stopped here
      6     21600  7b5c4d0982dee29c     7b5c4d0982dee29c     the same line
      7     25200  127af7dd42e40f69     127af7dd42e40f69     the same line
      8     28800  2cd2bc767392b8f5     2cd2bc767392b8f5     the same line
      9     32400  625d86900ed7ff28     625d86900ed7ff28     the same line
     10     36000  34c1c097d0624303     34c1c097d0624303     the same line

One thousand and eighty bytes each and the same digest over the lot. Lines one to five were written by the process that was killed and lines six to ten by the process that found its world in a database, and the file they make between them cannot be told from the file one process wrote straight through. Line six is the interesting one: it covers the span from tick 18,000 to 21,600, which includes the fifteen hundred ticks the first process had already run and thrown away, and it folds to the same sixteen characters either way.

The tape is written after the transaction that saved the world commits, and that order is deliberate: a line about a save that did not happen is worse than a missing line about one that did. The tape is a measuring instrument here and not part of the world, whose own record is the snapshot and the chronicle.

And the chronicle did notice, in the only way an append-only history can. The daemon writes one line when it takes a world up and one when it puts it down, and the second of those needs a process that is still alive to write it.

$ podman exec world-db psql -U world -d world -c "SELECT entry, tick, kind, text FROM chronicle WHERE kind IN ('started', 'stopped') ORDER BY entry;"
 entry | tick  |  kind   |                           text                            
-------+-------+---------+-----------------------------------------------------------
    12 |     0 | started | a process took The World up at tick 0
    13 | 18000 | started | a process took The World up at tick 18000
    14 | 36000 | stopped | a process put The World down at tick 36000, on a boundary
(3 rows)

One world taken up twice and put down once. The row that is missing is the crash, and it is the only trace of it anywhere, because the thing that would have logged an error was killed. A reader coming to this history cold can see that something went wrong in the first process's life by noticing that nothing said it finished, and see roughly when by reading the tick on the row after it.

Both kinds are about a process and neither is about the world, and that distinction is the whole of what makes them safe. Two started rows is not a duplicate: a world really was picked up twice. The next box is what happens when the same daemon writes a line about the world instead.

One kill, and the ticks it costs A strip of ticks running left to right with boundaries between them. A save is taken at the boundary of tick 18000 and written to the database. The process then runs on and is killed inside the creature phase of tick 19500, which is drawn opened up into its six steps with the kill landing between casting the fans and stepping the roster. The ticks between the save and the kill are marked as run twice. A second process starts underneath, reads the save at tick 18000 out of the database and runs the same ticks again, reaching tick 21600 and taking the next save. A panel at the bottom lists what happens at a boundary and what never happens inside a tick. ONE KILL, AND THE TICKS IT COSTS the first process save 18000 1500 ticks tick 19500 SIGKILL: gone inside that tick view fans roster sweep burial births nothing here can be written down: it is a moment the world was never in the second process reads that save out of the database save 18000 1500 ticks again 2100 more save 21600 the same numbers, in a different process, on the same tape line AT A BOUNDARY take the snapshot, in one transaction drop the saves too old to want read the signal, tell the chronicle INSIDE A TICK no transaction, no socket, no query no line of history, no signal read nothing at all but the world
Figure 80.1 — the kill lands inside a tick and costs that tick and every one since the last save. The second process starts from a moment the world was definitely in and runs the lost ticks again, which is why the tape it writes cannot be told from the tape of an untouched run.
⚠ Worked failure — the year that happened twice

The daemon writes two kinds of chronicle line and both of them are about a process. It is an obvious improvement to have it write a line about the world as well: a census once a valley-year, the way an annalist would, so that the history is a history of the place and not a log of restarts. The founding chapter's own bench writes exactly such a line, and the code is eight lines.

// cmd/weeks, at the boundary, which is where everything else already is
if yearly && w.tick()%terra.Year == 0 && w.tick() != from {
	text := fmt.Sprintf("year %d: %d %s standing in %s, %d born and %d gone",
		w.tick()/terra.Year, len(w.r.Live), doc.Many("living"), doc.One("ground"),
		w.p.Made, w.r.Gone)
	if err := db.Tell(ctx, w.tick(), "count", text); err != nil {
		die(err)
	}
}

It is at the boundary, it takes its numbers from the world, its names come out of the database and not out of a file, and everything this chapter has said so far it obeys. Run it with a save every three valley-years, so that a year can fall between two saves, and kill the process in the middle of a creature phase as before.

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go sh -c '/tmp/weeks -mode run -ticks 36000 -save 10800 -yearly -die-at 19500; echo "exit status $?"'
weeks: postgres at world-db:5432, database world, as world
weeks: nothing here was founded, so this process reads the document and founds it
weeks: The World, on the ground it calls The Hollow, 29 hobbs standing at tick 0
weeks: a save every 10800 ticks, the newest 3 kept, as fast as this machine will carry it
weeks: it stops on the boundary at tick 36000

      tick  walking     born     gone  genomes      saved  dropped  the ticks since the last save
         0       29        0        0       29     298475        0  (no tick yet)
     10800       31      132      130      161    1588888        0  35614ebe6e6488af
weeks: killed inside the creature phase of tick 19500
Killed
exit status 137
$ podman exec -w /bench world-go /tmp/weeks -mode run -ticks 21600 -save 10800 -yearly
weeks: postgres at world-db:5432, database world, as world
weeks: The World was founded before this process existed, and its newest save is at tick 10800
weeks: restored 145 plants, 31 hobbs, 161 genomes; founded nothing and scattered nobody
weeks: a save every 10800 ticks, the newest 3 kept, as fast as this machine will carry it
weeks: it stops on the boundary at tick 21600

      tick  walking     born     gone  genomes      saved  dropped  the ticks since the last save
     21600       10      339      358      368    3510713        0  77952a963dd43414

weeks: stopping on the boundary it was told to stop on
weeks: already written down on the boundary at tick 21600
weeks: stopped, and every connection handed back

  10800 ticks in 2.607s, 4143 ticks a second (measured here; yours will differ)

Nothing complained. The world came back, the tape would match, the save is where it should be, and every number in both processes is right. Now read the history.

$ podman exec world-db psql -U world -d world -c "SELECT entry, tick, kind, text FROM chronicle WHERE kind <> 'genesis' ORDER BY entry;"
 entry | tick  |  kind   |                              text                              
-------+-------+---------+----------------------------------------------------------------
    12 |     0 | started | a process took The World up at tick 0
    13 |  3600 | count   | year 1: 12 hobbs standing in The Hollow, 48 born and 65 gone
    14 |  7200 | count   | year 2: 26 hobbs standing in The Hollow, 89 born and 92 gone
    15 | 10800 | count   | year 3: 31 hobbs standing in The Hollow, 132 born and 130 gone
    16 | 14400 | count   | year 4: 20 hobbs standing in The Hollow, 198 born and 207 gone
    17 | 18000 | count   | year 5: 57 hobbs standing in The Hollow, 267 born and 239 gone
    18 | 10800 | started | a process took The World up at tick 10800
    19 | 14400 | count   | year 4: 20 hobbs standing in The Hollow, 198 born and 207 gone
    20 | 18000 | count   | year 5: 57 hobbs standing in The Hollow, 267 born and 239 gone
    21 | 21600 | count   | year 6: 10 hobbs standing in The Hollow, 339 born and 358 gone
    22 | 21600 | stopped | a process put The World down at tick 21600, on a boundary
(11 rows)

Year four happened twice and so did year five. Entries 16 and 19 are the same sentence word for word, at the same tick, with different entry numbers, and so are 17 and 20. Nothing here is a lie. Both times the fourth year ended there really were twenty animals standing and there really had been a hundred and ninety-eight births, because the world really did run that year twice: once in a process that was killed before it could write the year down anywhere that survives, and once in the process that ran it again.

The reasoning from symptom back to cause runs through one row, and it is entry 18. Every other row here has a tick greater than or equal to the row before it, which is what an append-only history of a world looks like. Entry 18 says tick 10,800 directly after a row that says 18,000: the clock went backwards, exactly once, and the row that did it says a process took the world up. The cause is then one step, because a process starting at tick 10,800 runs ticks 10,801 onward and does again whatever it does at a boundary on the way.

So the fault sits in what those four lines write, and not in the lines themselves or in the chronicle that took them. Anything a daemon does outside the transaction that saves the world is done again after a crash. The two lines the daemon does write are safe from that because of what they say: a started line records that a process took the world up, and a second process really did take the world up, so a second row is the truth. A count line records that a year happened in the world, and the year happened once.

There are three repairs and only one is cheap. Write the census inside the snapshot's own transaction, which is correct and means a census can never be more frequent than the saves. Put a unique constraint on the tick and the kind, which is correct here and turns the chronicle from a history somebody appends to into a table with an opinion about what may be said twice. Or give the daemon a rule about what it may say, which is what the version on this page has: a daemon writes history about itself, and the world's own history is written by whatever reads the world afterwards. The archive already works that way.

Valley-years per wall hour

A daemon's whole point is that time passes in it while you are somewhere else, so the number an operator actually wants is an exchange rate: how much of the world's time an hour of yours is worth. That has two halves. How dear a tick is, which is a fact about the machine and the valley standing on it, and how many ticks make a year, which is a fact about this world and has not moved since the terrarium was built.

Start with the first half, and measure it instead of guessing, because most of the cost of a tick turns out to be things that are alive. The same ground is ticked three times with more on it each time: bare rock and soil with nothing standing, then the terrarium with plants and seasons and no animals, then the valley this volume has been running.

▣ Build · stage 9 — what a tick costs, three times over
$ go run ./cmd/weeks -mode pace -years 3
weeks: what a tick costs, on one ground, with more and more alive on it

  a 16x12 valley, 10800 ticks apiece, which is 3 valley-years

  bare ground: no plant standing and no animal walking
  10800 ticks in 75ms, 144885 ticks a second (measured here; yours will differ)
    0 plants and 0 animals at the end of it

  the terrarium: plants, seeds and the seasons, and nothing that walks
  10800 ticks in 1.572s, 6870 ticks a second (measured here; yours will differ)
    121 plants and 0 animals at the end of it

  the valley this volume has been running: plants, 21 browsers and 8 hunters
  10800 ticks in 1.941s, 5564 ticks a second (measured here; yours will differ)
    145 plants and 31 animals at the end of it, 132 born and 130 gone

  the third of those is the one a daemon runs, and what separates it from the
  first is the whole of what a tick spends on things that are alive

Bare ground is nearly free: a hundred and forty-four thousand ticks a second for the calendar, the rain, the sun and the rotting. Put plants on it and the rate falls to 6,870, so 144,885 ÷ 6,870 = 21.1 times as dear a tick for about a hundred and twenty standing things that grow, seed, drop litter and die. Put animals on top and it falls again to 5,564, another 6,870 ÷ 5,564 = 1.23 times, and that last comparison is loose on purpose, because the third valley also finished with more plants than the second. What the three lines settle is the ranking: almost all of a tick is plants, animals are a quarter again on top of them, and the ground itself is free.

None of those three had a database underneath it. The daemon does, and its rate is the one that matters, because it includes what happens at every boundary: a world marshalled into eight parts, nine statements sent, a commit waited on, and the old saves deleted.

∑ Math Interlude — an hour of yours, in years of its

Numbers before symbols, and start with the rate the ten-year run printed: 2,995 ticks a second, which belongs to this machine and not to yours. A valley-year is 3,600 ticks, fixed in the terrarium volume and never moved since. An hour is 3,600 seconds, which no one arranged and which is a piece of luck for this page. So that machine runs 2,995 × 3,600 = 10,782,000 ticks in an hour, and 10,782,000 ÷ 3,600 = 2,995 valley-years. The number of valley-years an hour buys is the number of ticks a second the machine holds. The two 3,600s cancel and the units go with them.

Against that, the rate somebody watching sees. The tick budget has been ten a second since the first volume of this book, so a watched valley-year is 3,600 ÷ 10 = 360 seconds and an hour of watching buys ten valley-years. Unattended, the same hour buys 2,995 ÷ 10 = 299.5 times as much, and that factor is the whole of what accelerated time is here.

The second quantity is the one that might grow without anybody deciding it should, and it is a rate too. This world writes down every genome it ever handed out, the dead included, so a snapshot's size follows births and never the count of animals standing. Divide the bytes of the last snapshot by the valley-years that world had lived and the answer is bytes a valley-year; multiply by the valley-years an hour buys and it is bytes an hour; multiply by 168 and it is a week.

That division has to be done twice, over the whole run and over its later half, because an average taken across a valley filling up is an average of two different things. A valley fills once. What it costs afterwards is the number to plan with, and the run at the end of this section measures both.

Yticks in a valley-year: 3,600, and the same in every volume of this book
Hseconds in an hour: 3,600
rticks a second the daemon holds, the one number here that belongs to a machine
wticks a second when somebody is watching: 10, since the first volume
y = r × H ÷ Yvalley-years an hour buys, which is r because H and Y are the same number
Sbytes of the last snapshot a run wrote
nvalley-years that world had lived by then
S ÷ nbytes a valley-year costs written down, on average over the run
a × ba multiplied by b
a ÷ ba divided by b

Before any of that arithmetic means anything, the claim it rests on has to be run. A governor changes when a tick happens; it must change nothing about what the tick does. Here is the same valley for one year, once as fast as the machine will carry it and once held to ten ticks a second, each in its own empty database, each writing a tape.

▣ Build · stage 10 — the same year at two rates
// cmd/weeks
// govern holds the run to a rate. It is a governor and not a mode:
// every number this world produces is the number it would have
// produced at any other rate, and the only thing that changes is when.
func govern(start time.Time, ran int, hz float64) {
	due := start.Add(time.Duration(float64(ran) / hz * float64(time.Second)))
	if wait := time.Until(due); wait > 0 {
		time.Sleep(wait)
	}
}
$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go /tmp/weeks -mode run -ticks 3600 -save 1200 -tape /tmp/weeks-80/quick-80.tape | tail -4
weeks: already written down on the boundary at tick 3600
weeks: stopped, and every connection handed back

  3600 ticks in 552ms, 6523 ticks a second (measured here; yours will differ)
$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go /tmp/weeks -mode run -ticks 3600 -save 1200 -hz 10 -tape /tmp/weeks-80/slow-80.tape | tail -4
weeks: already written down on the boundary at tick 3600
weeks: stopped, and every connection handed back

  3600 ticks in 6m0.033s, 10 ticks a second (measured here; yours will differ)
$ podman exec -w /bench world-go /tmp/weeks -mode same -dir /tmp/weeks-80 -a quick-80.tape -b slow-80.tape -as 'as fast as it goes|governed to ten'
weeks: governed to ten, against as fast as it goes

                                               bytes  sha256, first 16
  as fast as it goes                             324  96889e2c1304d352
  governed to ten                                324  96889e2c1304d352
  the same bytes, and therefore the same run

   line      tick  as fast as it goes   governed to ten     
      1      1200  f11643d83d80f58e     f11643d83d80f58e     the same line
      2      2400  12f5d5b03e77bda4     12f5d5b03e77bda4     the same line
      3      3600  b7843ad130a6e33f     b7843ad130a6e33f     the same line

Six minutes against half a second, six hundred and fifty times the rate, and the two worlds are the same world: the same ticks in the same order, the same numbers off the same generators, the same plants, and a difference only in what a clock on the wall said while it happened. That is the volume's rule about accelerated time, run instead of asserted.

The code earns its place by what it forbids. Once the rate is a sleep between two ticks there is nowhere for a cheaper tick to hide: a design that ran faster by doing less arithmetic while unwatched would have to change the tick itself, and the tape would say so on its first line.

▣ Build · stage 11 — the exchange rate, worked
$ go run ./cmd/weeks -mode plan -rate 2176 -bytes 19447023 -over 1632 -was 18630844 -back 200 -keep 2
weeks: what an hour with nobody watching buys, at 2176 ticks a second

  the two numbers underneath it, and they are the same number
    ticks a valley-year                                  3600
    seconds an hour                                      3600
    so valley-years an hour is ticks a second, with the units cancelled

                                                 unattended        watched
  ticks a second                                       2176             10
  wall clock a valley-year costs                     1.654s           6m0s
  valley-years an hour                                 2176             10
  valley-years a day                                  52224            240
  valley-years a week                                365568           1680
  the first against the second                        217.6

  and what a valley-year costs written down, which is the half that grows
    the last snapshot this run wrote                 19447023 bytes
    valley-years it had lived by then                    1632
    a valley-year, averaged over the whole run          11916 bytes
    a snapshot at an earlier mark of the run         18630844 bytes
    valley-years it had lived then                        200
    a valley-year, over the span between them             570 bytes
    the average against the later span                   20.9 times

  taking the later span, because a valley fills up once and then stops
    a snapshot after an hour of it                   20687250 bytes
    a snapshot after a week of it                   227805211 bytes
    2 of those, which is what this daemon keeps     455610421 bytes
    a snapshot after a week governed to 10           20404551 bytes

  a week with nobody watching is 365568 valley-years and 227805211 bytes in one
  snapshot, and the second of those is small because most of the first is a
  valley that had already filled up. It is a straight line drawn through two
  measured points and carried 255 times past the further one

Six numbers go in and every one was measured by the run at the end of this section: a rate, a last snapshot, the valley-years that world had lived, and one earlier snapshot with the valley-years it had lived by then. Nothing here runs a valley or opens a socket. The top half is the cancellation worked out: a week unwatched is 365,568 valley-years against the 1,680 an operator sitting there would see.

The bottom half is the one that would have been got wrong. Averaged over the whole run a valley-year costs 11,916 bytes; measured over the span after the valley had filled up it costs 570, which is 20.9 times less. Both are honest arithmetic over one run and only the second is any use for planning, because the expensive part is a valley filling up and a valley fills up once. The last line says the rest: 227 megabytes after a week is a straight line through two points carried two hundred and fifty-five times past the further one, which is the best estimate available and still an estimate.

⌥ Tool — podman, and a process you mean to leave running

This page starts the daemon with podman exec inside a container that was already running, which is right for a page and wrong for an operator. A daemon of your own goes in a container of its own: podman run -d with the binary as the entry point, on the same --internal network as the database. podman logs -f follows what it prints. podman stop sends SIGTERM and waits, which is the boundary stop this chapter built, and its --time wants to be longer than one tick or the wait ends in a SIGKILL. podman kill sends SIGKILL straight away, and now you know what that costs. For a container that comes back after a reboot, --restart=always is the small answer and a Quadlet unit file is the one that puts it under systemd; both are at docs.podman.io.

Which leaves the run this chapter is named for. Everything above is bounded because everything above is quoted. The daemon has no bound in it: started with no tick count it founds a world and ticks until a signal arrives. Two columns appear below that are in no other run here, the wall clock and the rate over each span, and they are behind a flag because a run somebody will hold against another must carry nothing a second machine would print differently. This block is an exemplar. Yours will differ, in every figure in those two columns and in nothing else; the machine is the one named above.

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go sh -c '/tmp/weeks -mode run -clock -save 360000 -keep 2 -tape /tmp/weeks-80/long-80.tape & p=$!; sleep 2700; kill -TERM $p; wait $p'
weeks: postgres at world-db:5432, database world, as world
weeks: nothing here was founded, so this process reads the document and founds it
weeks: The World, on the ground it calls The Hollow, 29 hobbs standing at tick 0
weeks: a save every 360000 ticks, the newest 2 kept, as fast as this machine will carry it
weeks: it has no end in it; SIGTERM to stop on the next boundary

      tick  walking     born     gone  genomes      saved  dropped     wall a second  the ticks since the last save
         0       29        0        0       29     298475        0       0s        -  (no tick yet)
    360000      238     1970     1761     1999   18392359        0    2m50s     2124  1025291c9bc45dbe
    720000      237     1996     1788     2025   18630844        1    5m33s     2202  6ce011ed8ccd1d2c
   1080000      236     2010     1803     2039   18756838        1    8m17s     2198  b9b0279a79529fa7
   1440000      235     2010     1804     2039   18756582        1    11m0s     2202  9b4fa1886c6dbb2b
   1800000      237     2016     1808     2045   18811341        1   13m43s     2209  06c65ace69433ca6
   2160000      238     2040     1831     2069   19033133        1   16m28s     2192  513da339bb8deff3
   2520000      240     2045     1834     2074   19078268        1   19m12s     2185  5d25e0e5f5f1eb1d
   2880000      241     2050     1838     2079   19123126        1   21m58s     2174  fb889b51946ada4a
   3240000      244     2054     1839     2083   19158017        1   24m44s     2166  9dd72abfa877d2c7
   3600000      243     2054     1840     2083   19158991        1   27m31s     2158  d5315593077e73dc
   3960000      242     2054     1841     2083   19157558        1   30m17s     2170  c6fb2d25f8ccfd59
   4320000      244     2057     1842     2086   19178733        1    33m3s     2167  4e9a5b953729b5ad
   4680000      242     2059     1846     2088   19198225        1   35m48s     2176  44d05f3413b8834f
   5040000      242     2068     1855     2097   19281823        1   38m34s     2176  75ef12f13d2ef4f8
   5400000      243     2085     1871     2114   19437415        1   41m20s     2169  e70f4483cbd3f3e0
   5760000      244     2086     1871     2115   19446370        1    44m6s     2164  8eeb3223af5bd111

weeks: asked to stop, and the tick it was in finished first
weeks: written down on the boundary at tick 5876782, in 8 parts and 19447023 bytes
weeks: stopped, and every connection handed back

  5876782 ticks in 45m0.51s, 2176 ticks a second (measured here; yours will differ)

Forty-five minutes, and 5,876,782 ÷ 3,600 = 1,632 valley-years of history no operator sat and watched. The kill -TERM at the end of that command line is a person deciding they have had enough, and the two lines under the table are what the boundary did with it: the signal landed in the middle of a tick, that tick finished, and the world was written down at 5,876,782, a hundred and sixteen thousand ticks past the last scheduled save. A boundary stop saves where it stands.

The interesting column is born, and it shows something this chapter did not set out to show. Of the 2,086 animals born in those sixteen hundred valley-years, 1,970 arrived in the first hundred. After that the valley is full: the count settles between 235 and 244 and the snapshot goes from 18,392,359 bytes to 19,447,023 over fifteen hundred more valley-years. Two rows of the saved column are smaller than the row above them, which looks impossible in a world that never forgets a genome: in those spans nothing was born and something died, and the roster part is a list of what is walking. This valley does not fill a disk; it runs out of things to do first.

Which is the honest thing to say about the word in the title. Forty-five minutes is not weeks, and no page hands anybody a fortnight it did not spend. What it can hand over is why a fortnight is arithmetic here and not a hope: nothing in the loop changes with the length of the run. No counter grows a tick at a time, no list lengthens per tick, no handle is opened and left open, and the one quantity that does grow was measured above. The mechanism is the same at minute one and at minute twenty thousand.

Why stop boundaries define loss

Underneath the loop and the signal and the transaction there is one idea, smaller than the machinery around it. A program that can be stopped at any instant has, for every instant, an answer to the question what would be lost, and most programs have never been asked, so the answer differs at every instant and no process knows any of them. Picking one moment, writing the whole state down there and doing nothing anywhere else replaces all of those answers with one sentence: everything since the last boundary.

That is also the answer to how often to save. The unit of loss is the gap between two saves: save every tick and lose nothing and pay for a snapshot every tick, save once a century and lose a century. The daemon counts that gap in ticks, because a rule in wall clock would make how much work is at risk depend on how fast the machine happened to be that afternoon, and this whole chapter is about a world whose numbers do not know what a second is.

The second idea is what the transaction is for here, which is not what it was for on the save chapter's page. There a snapshot was whole or absent because nine statements went in together. Here a process can also die between two whole snapshots, and the question is what else it did in that gap. The answer this page arrived at through a failure is short: everything a process does outside the transaction that saves the world is done again after a crash. That holds well past databases, and it is the reason so much production code is full of idempotency keys.

There is a test for which side of that line a write falls on: ask what the row is about. A row saying a process took the world up is true once for each time a process took the world up, so a crash and a restart produce two and two is right. A row saying a year happened in the world is true once for each year, so a crash and a restart produce two and two is wrong. Every write a daemon makes can be sorted that way before it is written, which is cheaper than finding out in year forty.

The third idea is what makes an exact restart possible at all, and it belongs to this world and not to daemons. A restored valley runs the lost ticks again and produces the same numbers because nothing in it can tell it has run them before. No creature reads a clock, and nothing here has an outside: no client watching, no message sent, no file the world depends on. Give a world an outside and a re-run stops being invisible, and the choice is between making every effect idempotent and putting the effect inside the boundary's own transaction. There is no third answer.

And the last one is about measurement. A run that will be compared holds nothing a different machine would print differently; a run that is a measurement says so and names the machine. Mixing the two gives you a log that looks like evidence and cannot be checked, which is the state most performance numbers in most projects are in, and keeping them apart costs one flag.

✓ Checkpoint — the boundary, the kill and the exchange rate
  • The handler puts a value in a channel and the loop reads it at the boundary. Say what a handler that stopped the world where it stood would have to write down, and name two things it could not.
  • A founding process saves at tick nought before running anything and a restarted one saves nothing until it has run a tick. Give the reason for each, and say which is enforced by the schema.
  • The killed process ran fifteen hundred ticks past its last save and the next one ran them again. Say what makes those two spans the same ticks, and name the part of the snapshot without which they would not be.
  • Two started rows is a true history and two count rows for year four is not. State the rule that separates them, then apply it to a line about how much rain fell and a line about which process took a save.
  • Ticks a second and valley-years an hour are the same number here. Say why, and what would have to change about this world for them to stop being.
  • A snapshot's size follows births and not the count of animals standing. Say what that means for a valley full for a century and for one in its first year.
⚡ Exercises — try first, then reveal
Exercise 1 — move the kill. The kill on this page landed fifteen hundred ticks after a save. Put it one tick before a save instead, with -die-at 3599 on a fresh database, and predict what changes in the tape before you run the comparison.

Nothing changes in the tape, and that is the answer the run needs. What changes is how much work the second process has to do over again: killed at tick 3,599 the world falls all the way back to the save at tick nought and re-runs a whole valley-year, where killed at 19,500 it fell back fifteen hundred ticks.

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go sh -c '/tmp/weeks -mode run -ticks 36000 -save 3600 -tape /tmp/weeks-80/early-80.tape -die-at 3599; echo "exit status $?"'
weeks: postgres at world-db:5432, database world, as world
weeks: nothing here was founded, so this process reads the document and founds it
weeks: The World, on the ground it calls The Hollow, 29 hobbs standing at tick 0
weeks: a save every 3600 ticks, the newest 3 kept, as fast as this machine will carry it
weeks: it stops on the boundary at tick 36000

      tick  walking     born     gone  genomes      saved  dropped  the ticks since the last save
         0       29        0        0       29     298475        0  (no tick yet)
weeks: killed inside the creature phase of tick 3599
Killed
exit status 137
$ podman exec -w /bench world-go /tmp/weeks -mode run -ticks 36000 -save 3600 -tape /tmp/weeks-80/early-80.tape
weeks: postgres at world-db:5432, database world, as world
weeks: The World was founded before this process existed, and its newest save is at tick 0
weeks: restored 90 plants, 29 hobbs, 29 genomes; founded nothing and scattered nobody
weeks: a save every 3600 ticks, the newest 3 kept, as fast as this machine will carry it
weeks: it stops on the boundary at tick 36000

      tick  walking     born     gone  genomes      saved  dropped  the ticks since the last save
      3600       12       48       65       77     754043        0  7fc490d8b48449d5
      7200       26       89       92      118    1162894        0  64a389e5f89573e8
     10800       31      132      130      161    1588888        1  d112d46505f4f98f
     14400       20      198      207      227    2216490        1  dc3c272f63a19257
     18000       57      267      239      296    2857976        1  e21f6cadd002631c
     21600       10      339      358      368    3510713        1  7b5c4d0982dee29c
     25200       45      407      391      436    4142876        1  127af7dd42e40f69
     28800      179      707      557      736    6889280        1  2cd2bc767392b8f5
     32400      190      892      731      921    8563913        1  625d86900ed7ff28
     36000      199     1029      859     1058    9797377        1  34c1c097d0624303

weeks: stopping on the boundary it was told to stop on
weeks: already written down on the boundary at tick 36000
weeks: stopped, and every connection handed back

  36000 ticks in 12.067s, 2983 ticks a second (measured here; yours will differ)

The second process restores ninety plants and the twenty-nine founders, which is the world exactly as it was made, and then writes ten tape lines where the process on the main page wrote five. Every one of those ten is a line already printed further up this chapter, digest and all. A restart is not made more or less correct by falling further back; it is only made more or less work.

Exercise 2 — price the cadence. Work out from the ten-year run what saving three times as often would cost in bytes written and what it would buy in ticks at risk, then check the first half with -save 1200 on a fresh database.

Add up the saved column of the ten-year run and it comes to 41,782,925 bytes across eleven saves. Saving every 1,200 ticks instead of every 3,600 is three times as many saves over the same ten valley-years, so about three times that many bytes marshalled, sent and written through the server's log. What it buys is what a crash costs: up to 1,200 ticks of work to run again instead of up to 3,600. What it does not change is storage, because the daemon keeps three saves whatever the cadence.

$ podman kill world-db && podman start world-db
world-db
world-db
$ podman exec -w /bench world-go sh -c '/tmp/weeks -mode run -ticks 36000 -save 1200 | tail -7'
     36000      199     1029      859     1058    9797377        1  49fa1a4206621640

weeks: stopping on the boundary it was told to stop on
weeks: already written down on the boundary at tick 36000
weeks: stopped, and every connection handed back

  36000 ticks in 14.167s, 2541 ticks a second (measured here; yours will differ)
$ podman exec world-db psql -U world -d world -c "SELECT count(DISTINCT tick) AS saves, sum(octet_length(bytes)) AS bytes FROM snapshot_part WHERE world = 'The World' GROUP BY world ORDER BY world;"
 saves |  bytes   
-------+----------
     3 | 28089303
(1 row)

Two things in that pair are the answer. The last save is 9,797,377 bytes at either cadence, to the byte, because taking a snapshot changes nothing about the world being snapshotted. The digest beside it is a different sixteen characters from the one further up the page, and that is not a disagreement: it covers the twelve hundred ticks since the previous line instead of three thousand six hundred. Three saves are kept either way, and they come to 28,089,303 bytes here, because saves taken closer together are saves of more nearly the same size.

Exercise 3 — the exchange rate on more ground. A valley twice as wide and twice as tall has four times the cells. Predict what that does to the valley-years an hour, then measure it with go run ./cmd/weeks -mode pace -cols 32 -rows 24 -years 3.

Four times the cells, and what a tick does is walk everything standing on the ground, so a tick should cost about four times as much and the rate should fall to about a quarter. Valley-years an hour is the rate, so that falls to a quarter too: the same wall clock buys a quarter as much history on four times the world.

$ go run ./cmd/weeks -mode pace -cols 32 -rows 24 -years 3
weeks: what a tick costs, on one ground, with more and more alive on it

  a 32x24 valley, 10800 ticks apiece, which is 3 valley-years

  bare ground: no plant standing and no animal walking
  10800 ticks in 287ms, 37641 ticks a second (measured here; yours will differ)
    0 plants and 0 animals at the end of it

  the terrarium: plants, seeds and the seasons, and nothing that walks
  10800 ticks in 6.292s, 1716 ticks a second (measured here; yours will differ)
    661 plants and 0 animals at the end of it

  the valley this volume has been running: plants, 24 browsers and 8 hunters
  10800 ticks in 6.988s, 1546 ticks a second (measured here; yours will differ)
    649 plants and 8 animals at the end of it, 71 born and 95 gone

  the third of those is the one a daemon runs, and what separates it from the
  first is the whole of what a tick spends on things that are alive

A quarter, and closer to it than the prediction deserved. The terrarium line goes from 6,870 ticks a second to 1,716, which is 6,870 ÷ 1,716 = 4.00 on exactly four times the ground, and the bare ground goes from 144,885 to 37,641, which is 3.85. The plants did not scale that way at all: 121 standing on the small ground against 661 on the large one, five and a half times as many. What a tick costs tracks the cells more closely than it tracks what is standing on them, so an hour on this ground buys 1,546 valley-years where the same hour bought 5,564.

So the volume closes with a valley that does not need anybody. Ground that weathers and litter that feeds it back. Plants that seed, sleep through a winter and die of old age, and two rows of animals above them with a road for meat between the rows, the upper of which cannot make a living unless something is killed. A blow priced at both ends by one gene, armour paid for by another every tick whether or not anybody comes, and a third gene inherited, crossed, archived and read by nothing. A population whose height is set by the worst month of its year. Seven tables holding a history that answers questions, a founding document written at tick zero into rows nothing rewrites, and a process that was killed in the middle of a creature phase and came back as the same run.

What it cannot do is a shorter list than it was, and every item on it is real. Nobody can look at this world: there is no picture of it anywhere, and the only way to see what is happening in The Hollow is to read numbers out of a database. The archive chapter climbed thirty-five generations of one line without once crossing a species boundary, on ground small enough that every animal on it is a neighbour. A hunter's fan reports a creature and never says which row it belongs to, so hunting is easy to learn here and avoiding a hunter is not. And the valley settles: a few centuries in it stands at the count it reached in its second, with almost nothing born and almost nothing dying. A world that has run out of things to do is cheap to keep and thin to read.

The sharpest limit is the one hiding inside the result. A restart here is exact because this world has no outside. Nothing was watching while those fifteen hundred ticks happened the first time, nothing was told about them, and no file outside the transaction recorded that they had. That is the only reason running them again is invisible, and it is the assumption the census line broke by writing a sentence somebody could read. A world with anything at all on the far side of it, a person looking at a screen, a message sent, a payment taken, would have to answer for having done those ticks twice, and no amount of byte-identical replay would answer for it. This valley got the easy case, and it got it by being alone.