Three Kinds and No Words

The impression row

The first impression row reads 1 300 420 seen 4 0 3: the database's identity, the speaker, the tick, the kind, the body, the grams and the weight. The row has no sentence tucked beside those values and no vector waiting behind it.

A speaker keeps five numbers Go writes, one generated identity and one of three kinds; the animal has no words in storage and no meaning search. Text appears only when a later prompt renders the row. Until then the memory is the world's own accounting: who came into sight, who put food in the speaker's hands, who took food out.

That is smaller than the villager stream on purpose. A villager remembers a line, a speaker, a vector and later thoughts drawn from earlier rows. A browser-speaker cannot be told a sentence, cannot answer one here, and cannot ask a question of its own stream. A vector table would price a door the animal has no use for.

The speaker and impression tables

The volume's database starts with the ten migrations the village already had. The next two files are 0011_speaker.sql for the animal row and 0012_impression.sql for the append-only stream.

-- migrations/0011_speaker.sql
CREATE TABLE speaker (
    body     bigint NOT NULL,
    row_name text   NOT NULL,
    genome   jsonb  NOT NULL,
    founded  bigint NOT NULL,

    CONSTRAINT speaker_key PRIMARY KEY (body),
    CONSTRAINT speaker_names_its_roll_row CHECK (row_name <> ''),
    CONSTRAINT speaker_founded_on_a_real_tick CHECK (founded >= 0)
);

-- migrations/0012_impression.sql
CREATE TABLE impression (
    id      bigint GENERATED ALWAYS AS IDENTITY,
    speaker bigint NOT NULL,
    tick    bigint NOT NULL,
    kind    text   NOT NULL,
    body    bigint NOT NULL,
    grams   int    NOT NULL,
    weight  int    NOT NULL,

    CONSTRAINT impression_key PRIMARY KEY (id),
    CONSTRAINT impression_of_a_speaker FOREIGN KEY (speaker) REFERENCES speaker (body),
    CONSTRAINT impression_on_a_real_tick CHECK (tick >= 0),
    CONSTRAINT impression_is_one_of_three_kinds CHECK (kind IN ('seen', 'given', 'taken')),
    CONSTRAINT impression_names_a_body CHECK (body > 0),
    CONSTRAINT impression_grams_match_the_kind CHECK (
        (kind = 'seen' AND grams = 0) OR
        (kind IN ('given', 'taken') AND grams > 0)
    ),
    CONSTRAINT impression_weighs_one_to_ten CHECK (weight BETWEEN 1 AND 10)
);

CREATE INDEX impression_by_speaker ON impression (speaker, tick, id);

$ go run ./cmd/speakers -mode schema
speakers: the wordless memory schema, off embedded migrations

  migrations in build                                 12
  migrations applied on this run                      12
  0011_speaker.sql                           1 statement, 5bd16482f437754d
  0012_impression.sql                        2 statement, 2d65cff42bb344b7
  speaker                                    4 columns, 1 indexed column(s)
  impression                                 7 columns, 3 indexed column(s)
  impression text columns                               0
  impression vector tables                              0

The catalogue is the authority, so the contract also asks Postgres. The result is ordered by table and ordinal position; no page quotes the server's accidental order.

$ podman exec world-db psql -U world -d world -Atc "SELECT table_name || '|' || column_name || '|' || data_type FROM information_schema.columns WHERE table_schema='public' AND table_name IN ('speaker','impression') ORDER BY table_name, ordinal_position;"
impression|id|bigint
impression|speaker|bigint
impression|tick|bigint
impression|kind|text
impression|body|bigint
impression|grams|integer
impression|weight|integer
speaker|body|bigint
speaker|row_name|text
speaker|genome|jsonb
speaker|founded|bigint

Boundary writes, not tick writes

The witness has two inputs at the boundary: the eyes and the transfer tally. It does not write while a tick is running. The second boundary below sees the same two bodies as the first one and writes no row, because seen means coming into sight, not standing in sight again.

type ImpressionKind string

const (
	Seen  ImpressionKind = "seen"
	Given ImpressionKind = "given"
	Taken ImpressionKind = "taken"
)

type Impression struct {
	ID      int64
	Speaker sim.EntityID
	Tick    int
	Kind    ImpressionKind
	Body    sim.EntityID
	Grams   int
	Weight  int
}

func (w *Witness) Boundary(tick int, inSight []sim.EntityID, moved []Transfer) ([]Impression, []Work, error) {
	var out []Impression
	var works []Work
	now := map[sim.EntityID]bool{}
	for _, id := range inSight {
		now[id] = true
		if w.EveryTick || !w.seen[id] {
			row, work, err := w.Stream.Append(Event{Tick: tick, Kind: Seen, Body: id})
			if err != nil {
				return nil, nil, err
			}
			out = append(out, row)
			works = append(works, work)
		}
	}
	for _, tr := range moved {
		row, work, err := w.Stream.Append(Event{Tick: tick, Kind: tr.Kind, Body: tr.Body, Grams: tr.Grams})
		if err != nil {
			return nil, nil, err
		}
		out = append(out, row)
		works = append(works, work)
	}
	w.seen = now
	return out, works, nil
}

$ go run ./cmd/speakers -mode impressions
speakers: boundary impressions, five written numbers plus a generated identity

  weight table      base seen=2 given=5 taken=7, first +1, gram bands +1 over 5 and +2 over 30
  second boundary   the same two bodies still in view, rows written 0

  identity  speaker  tick  kind   body  grams  base  first  band  weight
         1      300   420  seen      4      0     2      1     0       3
         2      300   420  seen      7      0     2      1     0       3
         3      300   422  given     4     32     5      1     2       8
         4      300   460  seen      9      0     2      1     0       3
         5      300   460  taken     7      6     7      1     1       9

Row 3 is worth 8 because 5 plus 1 plus 2 is 8. Row 5 is worth 9 because 7 plus 1 plus 1 is 9. The database gives identities 1 through 5 after the append. The program writes the other five numbers.

Recall from recency and importance

A villager's retrieval asks a question, so relevance is a real term there. A speaker turn has no question yet. It needs the rows that matter now, not the rows closest to a sentence that nobody asked.

{
  ...
  "recall": {
    "half_life": 50,
    "recency": 1,
    "importance": 1,
    "k": 5
  }
}

func (r Recall) Rank(now int, rows []Impression) ([]Scored, error) {
	out := make([]Scored, 0, len(rows))
	for _, m := range rows {
		if err := m.Check(); err != nil {
			return nil, err
		}
		if m.Tick > now {
			return nil, fmt.Errorf("speaker: impression %d is at tick %d, after tick %d", m.ID, m.Tick, now)
		}
		s := Scored{Impression: m, Age: now - m.Tick}
		s.Recency = math.Pow(0.5, float64(s.Age)/float64(r.HalfLife))
		s.Importance = float64(m.Weight) / 10
		s.Score = r.Recency*s.Recency + r.Importance*s.Importance
		out = append(out, s)
	}
	sort.SliceStable(out, func(i, j int) bool {
		if out[i].Score != out[j].Score {
			return out[i].Score > out[j].Score
		}
		return out[i].ID < out[j].ID
	})
	if len(out) > r.K {
		out = out[:r.K]
	}
	return out, nil
}

$ go run ./cmd/speakers -mode recall
speakers: recall with no query vector

  half-life 50 ticks, recency weight 1, importance weight 1, k 5, relevance term 0

  rank  id  body  kind   age  recency  importance  score
     1   4     7  taken   10   0.8706      0.9000  1.7706
     2   3     4  given   48   0.5141      0.8000  1.3141
     3   1     4  seen    50   0.5000      0.3000  0.8000
     4   2     7  seen    50   0.5000      0.3000  0.8000

Rows 1 and 2 tie at 0.8000, so identity decides their order. The ranking prints no cosine, no embedding digest and no vector count because the package never asks for one.

Every-tick sight writes 20 rows

The tempting implementation writes a seen row every tick a body remains in view. The counterfactual flag leaves the witness otherwise unchanged, so the broken design can be run against the same recall code.

$ go run ./cmd/speakers -mode flood
speakers: the every-tick sighting counterfactual

  run                          rows written  top bodies at k=5
  coming-into-sight stretch               1  4
  every tick in view                     20  4,4,4,4,4

The console refuses words aimed at a speaker

The console route for say checks a speaker name before it looks for a villager. The refusal belongs to the verb, because there is no heard row to append and no seam to consult.

$ printf 'say browser-00 hello\n' | go run ./cmd/village -speakers -habit -roll village-24.json -ground 24,16 -plants 12 -settle 4000 | grep 'ears for words'
  tick   4901  browser-00 has no ears for words

This is the limit the design keeps. An animal can remember what a body did near it. It cannot be poisoned by a sentence here because there is no place to put the sentence, and because the console refuses the attempt before any row exists.

What the wordless stream can do

Change one base weight