A plain-language walkthrough of the coordination experiment we are about to run — on 29 books of political philosophy, using Codex agents inside Herdr.
When a team of AI agents gets bigger, what should hold the team together — conversation, or a shared set of files everyone reads and writes?
That's it. That's the whole experiment. Everything below is about how to answer it honestly instead of just believing the answer we already like.
That sentence is the claim under test. We think it's right. The experiment is designed so it can come out wrong.
Imagine two teams doing the same job in the same office, with the same boss structure — a manager, team leads, and workers under each lead.
everything lives in conversation
The boss tells a lead what to do. The lead tells the workers. Workers report back by talking. When a rule changes — "actually, keep the old chapter numbers" — the boss says it out loud, and it spreads by people telling each other.
everything important is written down
Same boss, same leads, same workers. But the objective lives in a file. The current state of the work lives in a file. Every decision gets written down and given a number. Talking still happens — but only to say "go look at the board, something changed."
Both teams have the same boss structure. That's deliberate. We're not testing whether hierarchy helps — both teams have it. We're testing one thing only: where the team's memory lives.
An experiment needs real work, not a toy puzzle — otherwise you learn about the puzzle, not about coordination. Our teams will do a genuine job that is already queued up and waiting.
29 books of political philosophy — Plato's Republic through Foucault's Discipline and Punish — have been acquired, extracted, and frozen into 1,610 "source units" (chapters, or fixed 10-page slices for PDFs). Every one of them is stuck at the same place, waiting for the same next step.
A book climbs an eight-rung ladder. A checker program decides which rung it's on — and it stops at the first rung the book can't prove, so nobody can claim progress that isn't there.
verified 2026-08-22 · node scripts/booklib/derive-milestone.mjs examples/republic → none (blocked at mapped)
To clear rung 1, each of a book's source units needs a written record with a heading, a synopsis, and a role — what this chunk of the book is doing. Rung 2 then asks for a coverage record per unit, and requires that a second agent independently reviewed it.
meaningful() that rejects any field reading unknown, pending, tbd, todo, placeholder, none, or n/a. An agent cannot type "TBD" 1,610 times and claim a rung. This is why we can trust the score.
Below is the map. It answers one question: who talks to whom, and where does the team's memory live? Use the buttons to trace one flow at a time — the rest fades but stays in place so you don't lose your bearings.
| 1 | Remy → Chief | priorities and anything the chief isn't allowed to decide alone |
| 2 | Chief → Lead | ASSIGN — one book, its objective, its constraints |
| 3 | Lead → Workers | a task contract: what to do, what you may touch, what counts as done |
| 4 | Workers → Lead | REPORT what was found, or BLOCK with the question |
| 5 | Lead → Chief | structured handoff, or escalation when it's outside the lead's authority |
| 6 | Chief ↔ files | writes priorities and cross-book decisions |
| 7 | Lead ↔ files | sole writer of state.md — synthesizes what workers found |
| 8 | Workers ↔ files | read the brief and decisions; publish their own report |
state.md; each worker owns only its own report. Nobody edits anyone else's file. This turns "who's in charge of what" into something the filesystem enforces rather than something agents negotiate — and it means two agents can never silently overwrite each other.
An agent only knows what its briefing tells it. We deliberately stripped away every background instruction this machine would otherwise inject — so the briefing is now the entire specification of how an agent behaves.
Every briefing has the same ten parts. And here is the part that matters: seven of them are word-for-word identical between the two teams. That isn't tidiness. It's the experiment.
state.md, write the final handoff, tell the chief both paths.”three parts differ · seven are byte-identical · full text in .ai/analyses/006_spike_agent_contract_architecture.md
Part 5 is the difference — so the tempting thing is to simply leave it out of Team A's briefing. Team A has no shared files, so why include the section?
Before every single run, a linter renders both briefings and refuses to let the run start if any supposedly-identical part differs by even one character, if a differing part says more things in one team than the other, or if any file path is written as a shortcut instead of in full.
Every message begins with its type as the first word, in both teams. This is also how we later count what kind of coordination was happening.
ASSIGN <book> work handed down REPORT <book> findings handed up BLOCK <book> stopped — with the question that would unblock you UPDATE <book> something you already know about has changed
Cadence. Report when a deliverable is complete, or when you found something that changes what someone else should do — not on progress. Block immediately when stuck, never silently and never by working around it. Send no acknowledgements, no status pings, no "still working." Don't ask a question you could answer from the sources part 5 gave you.
The original design says what the message types are but never when to send them. Left unstated, the two teams would end up differing in chattiness for reasons that have nothing to do with where files live — which would corrupt the main measurement directly.
Worker how to do the assigned task; what evidence to gather;
reversible changes inside its own scope
Lead how to split the book up; who does what; the order;
the approach within this one book
Chief priorities across books; anything with reach beyond
one book; retiring or reframing a book
Remy strategy, anything irreversible, and publishing
The closing line is identical in every briefing: “Escalate exceptions, not activity.” A system where everything escalates doesn't scale; a system where anything can be decided anywhere doesn't stay aligned.
The shape is the same in both teams — only where it gets delivered changes.
REPORT <book> OBSERVATION what is true, in a sentence or two EVIDENCE full file paths, commands run, and their actual output CONFIDENCE high | medium | low — and what would raise it RECOMMENDATION what you think should happen next OPEN QUESTIONS anything you could not resolve
Why evidence must be raw output, not a summary of it. One of the things we're measuring is what gets lost when a lead compresses several workers' findings into one shared view. If the worker already summarized the detail away, there's nothing left to measure the lead's compression against.
Every file has exactly one writer. The lead owns the shared state file; each worker owns only its own report; nobody edits anyone else's.
We're honest in the briefing about what this is: policy plus the lead reviewing the diff — not machinery. An earlier version of this system tested it directly by asking a read-only agent to edit a file, and it complied. So the lead's briefing carries an explicit obligation to review diffs before accepting work, and read-only agents are told to report it if they break the rule.
An agent that violates its briefing is data, not a broken run — those are precisely the coordination failures we're counting. Only two things void a run: a briefing that failed the linter, and an agent that turned out to be running the wrong model.
An experiment is only as good as its scoreboard. Ours has one rule: the agents don't get to grade themselves.
We call the score Y. It is the number of ladder rungs the books climbed during a trial, added up across every book in the batch. If a book slips backwards, that counts against us.
There is one honest catch, and we've written it into the plan: progress within a rung is invisible. A team can do six hours of genuine work and score zero if it didn't cross a boundary. That's why the seed books are chosen sitting just below a rung, and why each book's starting rung is recorded before the trial begins.
| What we count | Where it comes from | Why it's separate |
|---|---|---|
| coordination on the wire |
the text of every message sent between agents, plus the bytes of shared files read and written | This is "how much talking did it take." Team B pays a file cost here that Team A doesn't — and it still has to win on the total. |
| context consumed |
tokens the chief and leads burn per turn, taken from Codex's own accounting | This is "how much did the manager have to hold in their head." Team B may lose here — pointing agents at files means they read files. |
Here's a nice piece of luck. Codex writes a complete diary of every session to disk on its own — ~/.codex/sessions/…/rollout-*.jsonl. We don't have to instrument the agents at all. We read the diaries afterwards.
Each diary entry records the working directory, the exact prompt text, the tokens used — split into fresh, cached, output and reasoning — and, on every single turn, which model and which reasoning effort was actually running.
gpt-5.6-sol at high effort. But asking isn't getting — we found that the flag commonly used to set effort is silently ignored, so agents quietly inherit whatever the machine is set to. Because the rollout records what actually ran, we can check instead of trusting. Any trial whose turns weren't all gpt-5.6-sol/high is thrown away.
Two more deliberate choices. We never ask agents "what do you think the current state is?" mid-run — that question is itself a coordination message, and it would inflate the very cost we're measuring. We work it out from the record instead. And the raw diaries are archived somewhere the agents cannot read — because in Team A the raw worker output only exists there, and if agents could read it, Team A would have accidentally been given a whiteboard.
This is the part that makes it an experiment rather than a demo. We wrote these down before running anything, so we can't move the goalposts later.
state.md eats more time than the summary saves.state.md goes stale faster than it gets read, so agents act on old information.And the strongest possible result isn't "Team B won." It's "the gap grew as the team got bigger." A win at three agents could be luck. A widening gap across scale is a scaling property, which is the thing actually worth knowing.
Before writing a line of harness code, we ran the commands. Each of these was a belief that turned out to be wrong.
| We assumed | Actually | So |
|---|---|---|
| Setting reasoning effort on the command line works | reasoning_effort isn't a real key — it's ignored, and the machine default (xhigh) wins. The real key is model_reasoning_effort. |
Use the right key and verify per turn from the rollout |
| Codex agents would be harder to observe than Claude ones | The opposite. Rollouts record working directory, session id, model and effort per turn, and token counts split four ways. | Better instrumentation than the original design assumed |
| There's a coordinator process to disable so it doesn't compete with our org | bookops coordinator run is documented but not wired into the CLI at all. |
No conflict to resolve — our org just drives the CLI |
| A tracked git baseline could reseed the books | 121 of 123 book folders are gitignored, so they're local-only. And BookOps' own doctor reports the pipeline blocked because its private folder isn't ignored — while 387 of its files sit tracked in git. |
Hence the tarball seed, and a hygiene fix to hand back upstream |