What the World Remembers
Append-only history
The loop from last chapter works, and that is the problem. The rule for memory is severe on purpose: every event is written once, as one line, at the moment it happens, to a file that is only ever appended to.
Ten times a second Step runs, walkers move and the ground changes under
them; ten times a second the world overwrites what it just was. Print the map at the end
of a run and you learn where everything stands right now.
Where a walker went, when the pond swallowed a square of soil, which of the two walkers reached the water first: gone, every one of them, overwritten by the tick that came after.
That is fine for a program you watch. It is fatal for a server meant to run for months. Ask anything about the past and the running process has no answer: when the pond last grew, which walker was standing where at tick 80, whether something changed the ground at 3 a.m. or it has always looked like this.
Debugging is the smallest of those needs. A world that keeps no record cannot be replayed, cannot be audited, cannot be told about, and cannot teach anything to anyone later, because teaching runs on what was written down.
No line is edited. No line is deleted. History grows at one end and is frozen everywhere else. The evidence is a twelve-second run that accumulates two hundred lines of testimony, then yields to shell tools older than the format they are reading.
Spring and walkers
Start with what there is to remember, because a log is only as interesting as the world underneath it. The two laws written last chapter were placeholders chosen to exercise the tick, not to be lived in. Run them on the twelve-by-eight valley for 120 ticks and watch what they leave behind.
// cmd/worldd/main.go — last chapter's laws, unpaced, 120 ticks
for w.Tick() < ticks {
must(w.Step())
}
fmt.Print(w.Render())
fmt.Println("ticks:", w.Tick(), " water cells:", w.Ground.Count(sim.Water))
$ go run ./cmd/worldd
############
#~~~~~~~~~~#
@~~~~~~~~~~#
#~~~~~~~~~~#
#~~~~~~~~~~#
#~o~~~~~~~~#
#~~~~~~~~@~#
############
ticks: 120 water cells: 60
Sixty water cells is every square of the interior. The flood that copies itself into each neighbouring soil cell every tick reaches the far wall in six ticks, six tenths of a second, and the remaining 114 ticks have nothing left to flood. The walkers fare no better: one pressed west until the rim stopped it and has stood at the left-hand wall ever since, the other took a single step and hit water. A log of that run would be a burst of lines followed by two minutes of silence. Replace both laws with ones that keep producing news.
// internal/sim/tick.go — the valley's two real laws
// SpringPeriod is how many ticks pass between one seep of the spring
// and the next: every SpringPeriod ticks, one soil cell touching
// water becomes water.
const SpringPeriod = 40
// steps are the four cells reachable from any cell in one move.
// The order is fixed, because a draw into it decides where a walker
// goes and the world's history has to be reproducible.
var steps = []Coord{{X: 1, Y: 0}, {X: -1, Y: 0}, {X: 0, Y: 1}, {X: 0, Y: -1}}
// Step advances the world by exactly one tick.
func (w *World) Step() error {
w.tick++
if err := w.spring(); err != nil {
return err
}
if err := w.wander(); err != nil {
return err
}
return nil
}
// spring seeps once every SpringPeriod ticks: one soil cell that
// touches water, chosen from the seeded stream, becomes water. The
// candidates are collected in a fixed scan order so the draw always
// picks from the same list in the same arrangement.
func (w *World) spring() error {
if w.tick%SpringPeriod != 0 {
return nil
}
g := w.Ground
candidates := make([]Coord, 0, 16)
for y := 0; y < g.H; y++ {
for x := 0; x < g.W; x++ {
c := Coord{X: x, Y: y}
if t, _ := g.At(c); t != Soil {
continue
}
for _, d := range steps {
if n, err := g.At(c.Offset(d.X, d.Y)); err == nil && n == Water {
candidates = append(candidates, c)
break
}
}
}
}
if len(candidates) == 0 {
return nil
}
return w.Flood(candidates[w.rng.IntN(len(candidates))])
}
// wander steps every walker one cell in a direction drawn from the
// world's stream, in ID order. A walker the ground refuses stays
// where it is.
func (w *World) wander() error {
for _, e := range w.Roster() {
if e.Kind != Walker {
continue
}
d := steps[w.rng.IntN(len(steps))]
err := w.Move(e.ID, e.At.Offset(d.X, d.Y))
switch {
case err == nil:
case errors.Is(err, ErrBlocked), errors.Is(err, ErrOutOfBounds):
// nowhere to go this tick; the walker stays
default:
return err
}
}
return nil
}
// internal/sim/world.go — the world takes a seed of its own
type World struct {
Ground *Grid
tick uint64 // sim time: how many ticks this world has taken
rng *rand.Rand
ents map[EntityID]*Entity
nextID EntityID
}
// NewWorld wraps a terrain grid in a world with nothing standing on
// it. The seed is the same number the grid was generated from; the
// world takes its own draw stream from it, so terrain generation and
// the laws never share a position in one sequence.
func NewWorld(g *Grid, seed uint64) *World {
return &World{
Ground: g,
rng: rand.New(rand.NewPCG(seed, 1)),
ents: make(map[EntityID]*Entity),
nextID: 1,
}
}
$ go run ./cmd/worldd
############
#..........#
#...~....@.#
#.~~~~~....#
#.~~~~~~...#
#.o~~~~~...#
#...~.~....#
#######@####
ticks: 120 water cells: 19
Sixteen cells of pond at the start, nineteen after twelve seconds: three seeps, one every forty ticks, each of them a soil square that happened to be touching water when the spring drew.
The walkers are somewhere unpredictable instead of pinned against a wall, and both laws can keep producing events for an hour. That is the property a log needs.
A world that finishes changing in six tenths of a second has no history to keep, and a valley whose creatures always go west has one you could write down from memory.
The important line is the signature. NewWorld(g) in chapter 5 took a grid
and nothing else, because a world that only held things had no decisions to make.
Motion does, so it is now NewWorld(grid, seed), and the world keeps a
generator built from that seed.
The stream number matters: the same seed the terrain came from, but fed to
rand.NewPCG(seed, 1) where Generate used
rand.NewPCG(seed, 0). PCG's second argument selects a stream, a different
sequence of numbers from the same seed, so carving the pond can never consume a draw
that a walker was going to make.
Both are decided by the seed. Neither can disturb the other.
Two details in spring exist for the same reason. The candidate list is
built by scanning rows top to bottom and columns left to right, so the same set of
eligible cells always arrives in the same arrangement, and drawing index 3 means the
same square on every machine.
wander iterates Roster(), which sorts by ID, so walker 2
always draws before walker 3. Iterate the entity map instead, whose order Go randomizes
on purpose, and the two walkers would swap draws between runs.
Determinism survives only where every list the code walks has a defined order. The seed does not grant it by itself.
A 70-byte move line
Before adopting a format, price it. A log is a file that grows every tick forever, so the cost per line multiplied by a year is a number you want to meet early instead of discovering it on a full disk. Pricing means writing one real line, so start with what an event is: a tick number, a word for what happened, and whichever details that kind of happening has. A spawn has an entity and a destination. A move has an entity, a departure, and an arrival. A terrain change has a square and a new kind of ground.
// internal/sim/log.go
package sim
// Event is one thing that happened in the world, stamped with the
// tick it happened on. Fields a given kind of event has nothing to
// say about are left out of the line entirely.
type Event struct {
Tick uint64 `json:"t"`
Kind string `json:"ev"`
ID EntityID `json:"id,omitempty"`
From *Coord `json:"from,omitempty"`
To *Coord `json:"to,omitempty"`
What string `json:"what,omitempty"`
}
// ptr gives an event a coordinate it is also allowed to not have.
func (c Coord) ptr() *Coord { return &c }
// internal/sim/coord.go — Coord gains two tags and no new behaviour
type Coord struct {
X int `json:"x"`
Y int `json:"y"`
}
// cmd/worldd/main.go — a throwaway main, back for one stage
func main() {
spawn := sim.Event{Tick: 0, Kind: "spawn", ID: 3,
To: &sim.Coord{X: 10, Y: 6}, What: "walker"}
move := sim.Event{Tick: 17, Kind: "move", ID: 3,
From: &sim.Coord{X: 10, Y: 6}, To: &sim.Coord{X: 10, Y: 5}}
flood := sim.Event{Tick: 40, Kind: "terrain",
To: &sim.Coord{X: 6, Y: 4}, What: "water"}
for _, e := range []sim.Event{spawn, move, flood} {
line, err := json.Marshal(e)
if err != nil {
panic(err)
}
fmt.Printf("%s (%d bytes)\n", line, len(line)+1)
}
}
$ go run ./cmd/worldd
{"t":0,"ev":"spawn","id":3,"to":{"x":10,"y":6},"what":"walker"} (64 bytes)
{"t":17,"ev":"move","id":3,"from":{"x":10,"y":6},"to":{"x":10,"y":5}} (70 bytes)
{"t":40,"ev":"terrain","to":{"x":6,"y":4},"what":"water"} (58 bytes)
json.Marshal takes any value and returns the bytes of its JSON encoding,
reaching struct fields by reflection, so only exported fields appear. A lowercase field
is invisible to the encoder exactly as it is invisible to other packages.
The backtick strings after each field are struct tags, metadata the compiler
stores and hands to whoever asks. encoding/json asks, and obeys two
instructions here: the name (t, ev) replaces the Go field
name in the output, and omitempty drops a field whose value is the zero
one.
That is why the terrain line carries no "id" and the spawn line carries
no "from": an event only pays for the facts it has.
From and To are *Coord, pointers, and the reason
is omitempty itself. It considers a struct value never empty, so a plain
Coord field would print {"x":0,"y":0} on every event that
has no such coordinate.
That would put the top-left corner of the map in lines that mean nothing of the kind.
A pointer has a genuine empty value, nil, so the field vanishes when there
is nothing to say.
The +1 in the byte count is the newline each line ends with. Call it 70
bytes for a move, the event kind that dominates the event log.
The tick rate is 10 per second. Suppose The Hollow eventually holds 200 walking creatures, each moving on about half of all ticks. Events per second is then 200 × 10 × 0.5 = 1,000, and at 70 bytes each that is 70,000 bytes a second. A day holds 86,400 seconds, so a day of history costs 70,000 × 86,400 = 6,048,000,000 bytes, near enough 6.05 GB, and a year of it about 2.2 TB. Large, but affordable on one disk, and that number is the reason the log records events and not snapshots. Writing the whole map every tick instead, for a 512-by-512 world at one byte per cell, costs 262,144 × 10 = 2.6 MB a second: 226 GB a day, thirty-seven times more, to store mostly cells that did not change. The event log's economy is that it says only what changed.
JSON is not the cheapest way to hold those facts. A move packed as binary fields, two coordinates and an ID, fits in about 20 bytes, so this format costs roughly three and a half times the minimum. What the extra bytes buy is that every tool on the machine can already read the file, including tools written decades before the format existed and tools nobody has written yet. For a log whose whole purpose is to be read later, by programs and people you cannot name today, that trade is the right way round.
The obvious container for many events is one JSON array holding all of them:
[{...},{...},{...}]. It is a single valid document, which sounds tidy
until a live server tries to use it. Appending an event means
rewriting the closing bracket, so the file is only valid between writes; reading it
means parsing every event ever recorded to see the last one; and a process that dies
partway through a write leaves a document that no parser will accept.
JSON Lines fixes all three by giving up the outer document. One JSON object per line,
no commas, no brackets around the whole thing, and the file is not one value but a
stream of them. A crash costs at most the line being written. Reading the last event
is tail -1. Two runs of history compare with diff, line by
line, the same way two source files do. The format has no committee and barely has a
specification, which is the point: any tool that understands lines already understands
half of it, and any JSON parser understands the other half.
// internal/sim/log.go — continued
import (
"encoding/json"
"fmt"
"os"
)
// Log is the world's memory: one JSON object per line, appended and
// never rewritten.
type Log struct {
f *os.File
enc *json.Encoder
n uint64
}
// OpenLog opens the event log at path for appending, creating it if
// this is the world's first run. An existing log is never truncated.
func OpenLog(path string) (*Log, error) {
f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o644)
if err != nil {
return nil, fmt.Errorf("open event log %s: %w", path, err)
}
return &Log{f: f, enc: json.NewEncoder(f)}, nil
}
// Append writes one event as one line.
func (l *Log) Append(e Event) error {
if err := l.enc.Encode(e); err != nil {
return fmt.Errorf("append event at tick %d: %w", e.Tick, err)
}
l.n++
return nil
}
// Count reports how many lines this Log has written.
func (l *Log) Count() uint64 { return l.n }
// Close releases the file.
func (l *Log) Close() error { return l.f.Close() }
// cmd/worldd/main.go — write three events by hand, twice
func main() {
lg, err := sim.OpenLog("demo.jsonl")
must(err)
defer lg.Close()
must(lg.Append(sim.Event{Tick: 0, Kind: "start", What: "seed 5"}))
must(lg.Append(sim.Event{Tick: 0, Kind: "spawn", ID: 1,
To: &sim.Coord{X: 2, Y: 5}, What: "shrub"}))
must(lg.Append(sim.Event{Tick: 4, Kind: "move", ID: 3,
From: &sim.Coord{X: 10, Y: 6}, To: &sim.Coord{X: 10, Y: 5}}))
fmt.Println("wrote", lg.Count(), "events")
}
$ go run ./cmd/worldd && cat demo.jsonl
wrote 3 events
{"t":0,"ev":"start","what":"seed 5"}
{"t":0,"ev":"spawn","id":1,"to":{"x":2,"y":5},"what":"shrub"}
{"t":4,"ev":"move","id":3,"from":{"x":10,"y":6},"to":{"x":10,"y":5}}
$ go run ./cmd/worldd && wc -l < demo.jsonl
wrote 3 events
6
Two mechanisms carry this stage. os.OpenFile takes a set of flags
combined with |, and the three here mean write only
(O_WRONLY), create the file if it is absent (O_CREATE),
and, the important one, O_APPEND: before every write the kernel moves
the file offset to the current end of the file. Not to where this program last
wrote. To the end, checked at write time, atomically with the write. Nothing this
process does can put bytes anywhere but after everything already there, so
"append-only" stops being a promise the code makes and becomes a property of how the
file is open. The permission 0o644 is the usual owner-writes,
everyone-reads.
json.Encoder is the streaming half of encoding/json: where
Marshal hands you bytes, Encode writes them straight to an
io.Writer, and it appends a newline after each value. That newline is
the whole reason this code has no line-joining logic. The format the world writes
falls out of what the encoder already does, and every Encode is one
write of one complete line to the file, with nothing held back in a buffer for
later. The second run proves the flags: three more events, six lines total, and the
first run's history still exactly where it was. The file has two lives in it now,
and so each run opens with its own start line naming the seed.
The world's history records the world's name.
The reflex when a program writes small pieces often is to wrap the file in a
bufio.Writer, and here that reflex costs exactly what the format was
chosen to protect. A buffer holds the last few kilobytes of history in memory and
hands them to the kernel later; a process that dies in between loses every event it
was told to remember since the last flush. The log is small (a few thousand bytes a
second at valley scale) and the kernel is good at absorbing small writes, so this
book pays for a write per event and keeps the guarantee.
Figure 8.1: the same four events in both formats: a stack of finished lines, or one document that has to be reopened and reclosed to grow.
Recording events
Now put the log where the events are. Every fact the log records already passes
through one of the hardened methods from chapter 6: an entity comes into the world
through Spawn, changes position through Move, and the ground
changes through Flood, which the spring is the only caller of. Those
methods are the only routes to state, so they are the only places an event can be
missed, and each gets one line added at the point where the change has definitely
happened.
// internal/sim/world.go — the world learns to keep records
// Record hands the world a log to write its events to.
func (w *World) Record(l *Log) { w.log = l }
// record writes one event, remembering the first failure instead of
// pretending nothing happened.
func (w *World) record(e Event) {
if w.log == nil || w.logErr != nil {
return
}
if err := w.log.Append(e); err != nil {
w.logErr = err
}
}
// Faulted returns the first error the world's log ran into.
func (w *World) Faulted() error { return w.logErr }
// internal/sim/world.go — three methods, one new line each
// in Spawn, after the entity is in the map:
w.record(Event{Tick: w.tick, Kind: "spawn", ID: id,
To: at.ptr(), What: k.String()})
// in Move, after e.At = to:
w.record(Event{Tick: w.tick, Kind: "move", ID: id,
From: from.ptr(), To: to.ptr()})
// in Flood, after the ground is water:
w.record(Event{Tick: w.tick, Kind: "terrain",
To: c.ptr(), What: Water.String()})
// cmd/worldd/main.go — last chapter's loop, now with a memory
func main() {
const seed = 5
const ticks = 120
lg, err := sim.OpenLog("world.jsonl")
must(err)
defer lg.Close()
must(lg.Append(sim.Event{Kind: "start",
What: fmt.Sprintf("seed %d", seed)}))
w := sim.NewWorld(sim.Generate(12, 8, seed), seed)
w.Record(lg)
_, err = w.Spawn(sim.Shrub, sim.Coord{X: 2, Y: 5})
must(err)
_, err = w.Spawn(sim.Walker, sim.Coord{X: 10, Y: 6})
must(err)
_, err = w.Spawn(sim.Walker, sim.Coord{X: 2, Y: 2})
must(err)
fmt.Println("worldd", version, "seed", seed, "— logging to world.jsonl")
next := time.Now()
for w.Tick() < ticks {
must(w.Step())
must(w.Faulted())
next = next.Add(tickDur)
time.Sleep(time.Until(next))
}
fmt.Print(w.Render())
fmt.Println("ticks:", w.Tick(), " events written:", lg.Count())
}
$ go run ./cmd/worldd
worldd 0.0.1 seed 5 — logging to world.jsonl
############
#..........#
#...~....@.#
#.~~~~~....#
#.~~~~~~...#
#.o~~~~~...#
#...~.~....#
#######@####
ticks: 120 events written: 210
Twelve seconds of world, 120 ticks, 210 lines. Read which events exist and which
do not. Two walkers taking one step each per tick could produce 240 moves; the log
holds 203, because a walker that draws a direction into water or off the map gets an
error back from Move and nothing happens, and an event that did not
happen gets no line. The log is a record of changes to the world, not of intentions
about it. That distinction keeps the file honest and, at valley scale, keeps it a
great deal smaller.
record is deliberately quiet in one direction and loud in another. It
takes no error return, because a caller like Move has nothing useful to
do about a failed write and its own error already means something specific to the
caller. But it does not swallow the failure either: the first write error is kept in
logErr, and the tick loop asks Faulted once per tick. A
world whose disk has filled should stop, immediately and noisily, instead of
cheerfully advancing another million ticks whose history nobody is keeping. A
simulation that cannot remember what it is doing has no business continuing to do
it.
The first version of Log was simpler and felt cleaner: collect the
events in a slice, and write them all as one array when the world shuts down. One
open, one write, one valid JSON document at the end.
// internal/sim/log.go — the version that lost a run
type Log struct {
path string
events []Event
}
func OpenLog(path string) (*Log, error) { return &Log{path: path}, nil }
func (l *Log) Append(e Event) error {
l.events = append(l.events, e)
return nil
}
func (l *Log) Close() error {
blob, err := json.Marshal(l.events)
if err != nil {
return fmt.Errorf("marshal history: %w", err)
}
return os.WriteFile(l.path, blob, 0o644)
}
// cmd/worldd/main.go — inside the tick loop, a power cut modelled
if w.Tick() == 7 {
fmt.Println("*** the machine loses power at tick 7 ***")
os.Exit(1)
}
$ go run ./cmd/worldd; ls -l history.json
worldd 0.0.1 seed 5 — logging to history.json
*** the machine loses power at tick 7 ***
exit status 1
ls: cannot access 'history.json': No such file or directory
Not a truncated file. Not a corrupt file. No file. Seven ticks of world happened,
sixteen events were dutifully handed to Append, and every one of them
died in a slice when the process did. os.Exit was chosen to model the
outage precisely because it skips deferred calls, which is what a power cut does to
defer lg.Close() and to every other tidy-up a program had planned. The bug is not the array format by itself, but that the moment of writing was moved away
from the moment of remembering, and everything in between existed only in a
process that is not guaranteed to survive.
Put the appending version back, keep the same power cut, and the difference is the whole chapter:
$ go run ./cmd/worldd; wc -l < world.jsonl; tail -1 world.jsonl
worldd 0.0.1 seed 5 — logging to world.jsonl
*** the machine loses power at tick 7 ***
exit status 1
16
{"t":7,"ev":"move","id":3,"from":{"x":1,"y":3},"to":{"x":0,"y":3}}
Sixteen lines survived a process that never got to shut down, and the last of them is the last thing that happened before the lights went out: walker 3, at tick 7, stepping west onto the rim. Nothing was flushed, because nothing was waiting. Each line was already on disk before the next event existed, and that is the property the append-only rule is really about: durability per event instead of per run.
Append-only storage
Append-only storage is a bargain in which you surrender one power to gain several. The power surrendered is editing: you cannot go back and correct line 400. Concurrency gets simpler, because writers contend for one place, the end. Reading gets simpler, because a line, once written, never changes under a reader. Crash recovery gets simpler, because the only damaged region possible is the tail. Correcting a mistake stays possible too, in the form every serious record-keeping system uses: a later entry that supersedes an earlier one, leaving both visible.
The hardware agrees with the design. A spinning disk writing sequentially never seeks, and an SSD's flash is erased in large blocks, so appending suits it far better than rewriting a middle. Database engines use the same file pattern under the name write-ahead log: record the change, durably and in order, before touching the structure it changes, and a crash becomes a question of replaying a suffix. What is being taught here is not a trick for a toy simulation. It is the shared foundation of filesystems, databases, and replicated systems.
There is one more reason, particular to this world. The file being written is not debugging output; it is the world's testimony about itself, the only account of The Hollow's past that will ever exist. Every creature that lives and dies and every square that floods passes through a line in a file like this one, timestamped by tick and never altered afterward. History that can be edited is not history. Keep it append-only from the first walker and the record stays trustworthy for as long as the world runs.
Reading the log
A record nobody can read is a record in name only, so end where the format pays off.
Nothing below is written for this file. wc, grep,
awk, sort and uniq know nothing about worlds,
ticks or JSON; they know lines. Because history is lines, they are enough to
interrogate it.
$ wc -l world.jsonl; ls -l world.jsonl | awk '{print $5, "bytes"}'
210 world.jsonl
14268 bytes
$ awk -F'"ev":"' '{split($2, a, "\""); print a[1]}' world.jsonl | sort | uniq -c
203 move
3 spawn
1 start
3 terrain
$ grep '"ev":"terrain"' world.jsonl
{"t":40,"ev":"terrain","to":{"x":4,"y":2},"what":"water"}
{"t":80,"ev":"terrain","to":{"x":6,"y":6},"what":"water"}
{"t":120,"ev":"terrain","to":{"x":3,"y":5},"what":"water"}
$ grep '"id":1,' world.jsonl
{"t":0,"ev":"spawn","id":1,"to":{"x":2,"y":5},"what":"shrub"}
The census by event kind is one awk line: split each line on the text
"ev":", take what follows up to the next quote, then let
sort and uniq -c tally. 14,268 bytes for 210 events is 68
bytes a line on average, within a rounding error of the 70 the arithmetic predicted.
The three terrain lines answer a question the final map could not: not just that the
pond is bigger, but that it grew on ticks 40, 80 and 120, one square at a time, and
in which order those three squares fell. Shrub 1 was spawned at {2 5} and never
appears again, which is correct, since shrubs do not move.
The tick-80 flood is where this run keeps its surprise. Ask the file for the lines on either side of it:
$ grep -n -B1 -A1 '"ev":"terrain","to":{"x":6,"y":6}' world.jsonl
136-{"t":79,"ev":"move","id":2,"from":{"x":7,"y":6},"to":{"x":6,"y":6}}
137:{"t":80,"ev":"terrain","to":{"x":6,"y":6},"what":"water"}
138-{"t":80,"ev":"move","id":2,"from":{"x":6,"y":6},"to":{"x":6,"y":7}}
Three consecutive lines, and they convict the simulation. On tick 79 walker 2
stepped onto {6 6}. On tick 80 the spring turned that exact square to water, with
the walker standing on it. And then, later in the same tick,
the walker moved off south under its own draw, so the run continued as if nothing
had happened. Move refuses to put a walker into water;
Flood never asks who is already there. A creature stood in the pond for
one tick and the only reason it got out is that its direction that tick happened to
point away. The final map cannot show any of this: by the time it is printed the
walker is three squares away, and the cell is ordinary water. A printed map shows a
state. A history lets you prosecute it.
- Write a law that draws from the world's own seeded stream, and say why
NewWorldnow takes a seed when chapter 5's version did not. - Name the two orderings that keep a seeded law reproducible: the spring's
row-by-row candidate scan and
Roster()'s sort by ID. - Define an
Eventstruct whose JSON keys and omitted fields are controlled by struct tags, and explain whyFromandTohave to be pointers foromitemptyto work on them. - Price a log format before adopting it: bytes per line times events per second times 86,400, with the day-sized answer for a 200-creature valley.
- Open a file with
O_WRONLY|O_CREATE|O_APPEND, say what the kernel does at each write, and explain what the array-at-shutdown logger lost when the power went out at tick 7. - Catch a bug in the simulation using
grepalone, by reading two adjacent lines of a finished run that describe the same square one tick apart.
Exercise 1: the cell that flooded underfoot. The log
caught the spring turning a square to water while a walker stood on it, and the
file says so only because two lines happened to sit next to each other. Make it
explicit: have Flood look for entities standing on the square it is
about to drown and write a line naming each one. Run 120 ticks and grep for the
new kind.
In Flood, after the terrain line is recorded, walk
Roster() for an entity whose At equals the flooded
coordinate (struct equality works, since Coord is two comparable
ints) and record
Event{Tick: w.tick, Kind: "stranded", ID: e.ID, To: c.ptr(), What: e.Kind.String()}.
Recording the terrain change first and the stranding second matters: anyone
reading the file forward then sees the cause before the consequence. With seed
5 the run grows from 210 lines to 211, and the new one is
{"t":80,"ev":"stranded","id":2,"to":{"x":6,"y":6},"what":"walker"}.
Nothing else in the history moves, because the exercise adds a line and changes
no decision. One stranding in 120 ticks is also a measurement: rare enough to be
easy to miss by watching, which is what the log is for.
Exercise 2: measure a day. Raise ticks to
1,200, run the two minutes, and use the resulting file to predict what a full day
of this world costs. How close does the prediction get to the interlude's
method?
The two-minute run writes 1,768 events and 122,516 bytes, which is 102.1 bytes per tick. At 10 ticks a second that is about 1,021 bytes a second, so a day of this three-entity world costs roughly 88 MB and a year about 32 GB, for one spring and two walkers. Rerun it and the byte count is identical, since the run is deterministic from the seed: the same 1,768 events in the same order. The interlude's estimate assumed 200 creatures moving half the time and got 6 GB a day, so the two answers differ by the population, not by the method. The lesson to carry: measure a real run and multiply, instead of trusting an estimate you have not weighed on a scale.
Exercise 3: forge a line. With the server stopped, append
a plausible but false event to world.jsonl by hand:
echo '{"t":60,"ev":"move","id":9,"to":{"x":1,"y":1}}' >> world.jsonl.
Nothing objects. What does that tell you about what "append-only" is protecting,
and what it is not?
grep '"id":9' finds your forgery sitting in the record as if it
belonged there, and no part of the program noticed. The O_APPEND
flag constrains one process's writes; it does not make the file immutable to its
owner. Append-only, at this stage, is a property of how the world writes its own
history, which is what protects it from the world's own bugs and crashes. That
covers the failures this book is currently up against. Protecting a record from
someone with write access is a different problem with different tools, from
filesystem permissions and immutable-file flags to a hash of each line folded
into the next, and none of them are needed until somebody untrusted can reach
the file. Delete the forged line before continuing, because the next thing this
world does with its history is take it seriously.