dirigence/reliquary
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
83f6bac3f2014396e2a8fffdfee7b6d264948215
reliquary/docs/workpad/designs/2026-10-16-01-headless-forks.md
T

45 KiB
Raw Blame History

Design: Trained implementer, forked per block (spire:forks)

Date: 2026-10-16 Designer: model id claude-opus-5-5 (stage 3; does not plan or build) Goal: goals/headless-forks.md, including its "Decided 2026-10-16" section Inputs: research/2026-10-16-00-claude-headless-fork-sessions.md (claim ids such as 3a, F4c refer to it), reports/2026-10-15-23-athanor-trunk-fork-provenance.md, reports/2026-10-15-22-spire-vs-superpowers-plan-and-build-experiment.md §5 Touches: new skill plugins/spire/skills/forks/ (SKILL.md, forks.mjs, preamble.md, templates/), plugins/spire/agents/athanor.md, plugins/spire/skills/implementation/SKILL.md, references/spawning-claude-code.md, references/run-log.md, references/models.md, plugins/spire/.claude-plugin/plugin.json (version) Status: one approach, for the planner (stage 4) and the attacker (4b)

Marking: Fact means verified in the research (claim id given) or checked by the designer today (claude --help of 2.1.283, node --version, which). Assumed means not verified; the planner's preflight must check it (§10).

1. The shape in one picture

Igor ──Agent──► Athanor (Opus, plan level: plan, design, trunk notes, reports, run log; no code reads)
                  │
                  │ Bash (foreground, ≤ 9 min per call)
                  ▼
                node forks.mjs train / fork / rebase / continue / wait / usage / list / expire
                  │ spawns detached supervisor per call
                  ▼
                supervisor ──► claude -p --resume <base> --fork-session  (headless, scrubbed env,
                  │                                                        full fingerprint)
                  └─► registry.json, ledger.jsonl, results/<job>.json   (next to the run log)

Base B1 (trained once: reads code, writes trunk notes) ──fork──► F1.1  F1.2  F1.3 …  (one per complex block)
   └─rebase at phase boundary──► B2 ──fork──► F2.1 …
Mechanical / distant blocks: fresh Agent-tool agents reading the trunk notes (unchanged path).

The Athanor never builds a block itself and never reads code. The helper owns every claude -p command line, the registry, the cache bookkeeping and the usage ledger. The Athanor writes briefs, reads handoff notes and helper JSON, runs mechanical gates and test commands, and keeps the run log.

2. The skill: spire:forks

Name: forks (directory plugins/spire/skills/forks/). Short, says what it does, and lifts to Dirigent as one folder.

Files:

File Role
SKILL.md Contract: when to use, commands, the rules the Athanor follows, the JSON it gets back. Short.
forks.mjs The helper. One file, Node built-ins only, no npm install.
preamble.md Appended system prompt for every base (--append-system-prompt-file): role of a trained implementer that will be forked, handoff-note format, "do this yourself; do not dispatch agents", stamp model id. Part of the fingerprint (hashed).
templates/train.md, templates/fork.md, templates/rebase.md Fixed prompt frames the helper wraps around the Athanor's brief. Versioned with the skill.

Who loads it, when: the Athanor, by frontmatter preload (skills: gets spire:forks), so it is present from intake. The implementation skill names it in "Context" (§7) as the build path for complex blocks. Igor does not load it; Igor only sees its numbers in the run log. Any other agent may load it for a non-build use, but the Athanor procedure is the only contract this design defines.

SKILL.md head (pattern of adepts):

---
name: forks
description: Trains one headless implementer per build and forks it per block via `claude -p --resume --fork-session`. Owns the command line, the cache bookkeeping and the usage ledger. Loaded by the Athanor.
allowed-tools: Bash(node:*)
---
!`node "${CLAUDE_SKILL_DIR}/forks.mjs" where`

where prints the absolute path of forks.mjs, the resolved binary and its version, the node version, and whether the env scrub list matches the host. The Athanor uses the printed path in every later call. Assumed: a preloaded skill's ! injection runs at agent start, and allowed-tools: Bash(node:*) pre-approves the later calls (the adepts skill relies on the same pattern for bash; its own comment calls it novel).

3. The helper forks.mjs

3.1 Runtime choice: Node

Checked on this host today (Git Bash): node v24.8.0 at /c/Program Files/nodejs/node; bash 4.4.23; no jq; no python/python3 (Python 3.7 directory is on PATH but no executable resolves); Strawberry perl present.

  • Node, chosen. JSON is native (the whole job is parsing stream-json, modelUsage and the registry). child_process.spawn takes an explicit env object, so the scrub is a dict operation, not env -u quoting through Git Bash. Detached spawn, kill-tree, atomic rename, open(..., 'wx') lockfiles and path normalisation work on Windows and POSIX. The research already used node for JSON here (claim 7).
  • Assumed about other hosts: node ≥ 18 is on PATH wherever Spire runs (the native Claude Code installer does not need Node, so this is not guaranteed). where fails loudly with "node not found" and the Athanor falls back to the fresh-agent path, logged.
  • Rejected: bash + jq (no jq here; JSON in bash is fragile). Python (not on PATH). PowerShell (not on Linux remote hosts; the repo's scripts are bash). Perl (works, but nobody in this repo maintains Perl). The Agent SDK in TypeScript (needs npm install, spawns the CLI anyway; that is the Dirigent-later path).

3.2 State location: next to the run log, per run

A trained base belongs to one build plan, and the 1 h TTL (F4a) means no base usefully outlives a run. The state lives beside the run log so a successor Athanor and the reviewer find it:

<workpad>/reports/<YYYY-MM-DD-hh>-<slug>-run.md          the run log (unchanged name rule)
<workpad>/reports/<YYYY-MM-DD-hh>-<slug>-run-forks/      helper state, owned by forks.mjs
  registry.json        bases, jobs, config; never hand-edited
  ledger.jsonl         one line per finished call, append-only
  briefs/<job>.md      the exact prompt sent (frame + Athanor brief + delta header)
  notes/<block>.md     handoff notes written by forks (one file each: parallel-safe)
  results/<job>.json   the per-call summary `wait` returns
  logs/                gate and test output written by forks and the Athanor
  raw/<job>.jsonl      stream-json stdout       ┐ gitignored by the helper's own
  raw/<job>.err        stderr                   ┘ .gitignore (raw/, *.lock)

Every command takes --run <path to run log .md>; the state dir is derived from it. The run log header gets one line, Forks: <state dir>. This is a dated subfolder for a cascade of related files, as the workpad skill allows.

3.3 Fingerprint

Any difference in model, tools, permission mode or binary loses the cache (3b, 4b, F5b), and flags are not inherited by a fork (4b, 4c). So the Athanor passes launch settings once, at train; fork, rebase, continue and pings read them from the base record. The Athanor cannot get a fork's flags wrong because it never supplies them.

Launch fields (hashed into fingerprint.hash, canonical JSON, sha256):

Field Source / default Why
binary.path --binary, else $CLAUDE_CODE_EXECPATH if set, else command -v claude Two installs exist (F5a). The Desktop's bundled binary matches the host and runs the newest models: Sonnet 5.5 needs ≥ 2.1.284 (F2c), the PATH install is 2.1.283.
binary.version, binary.size, binary.mtime <binary> --version, stat Path alone is not enough: a native install can auto-update in place. Hashing a large exe on every call is too slow; size + mtime + version is the check.
model required, full id (^claude-[a-z0-9-]+(\[1m\])?$); aliases refused Aliases move with the CLI version (F2c); plan mode swaps Haiku to Sonnet (F2a). Goal decision: always a full id. Assumed: --model accepts the [1m] suffix.
effort required (--effort), skill default high The host passes CLAUDE_EFFORT=medium (F2b) and user settings.json has effortLevel: medium; a user changing effort in the Desktop mid-run would change a fork's effort. Assumed: effort changes the request (thinking settings) and so the cache.
permission_mode default dontAsk Never plan (F2a). dontAsk denies anything not pre-allowed (4a) and so needs the explicit list below.
tools default Bash,Read,Edit,Write,Glob,Grep,Skill No Task/Agent: forks are leaves, which also removes the depth question (F1c, F1d). Assumed: Skill is a valid --tools name.
allowed_tools, disallowed_tools default: allow = tools; deny = Bash(git push:*), Bash(git rebase:*), Bash(git reset --hard:*), Bash(claude:*), plus project additions Same guard rails as today's children; no nested claude from a fork.
mcp default --strict-mcp-config --mcp-config <state>/mcp-empty.json Not restored on resume (4c), so it is passed every time. Pins the tool list against claude.ai connectors toggling mid-run. Assumed: strict mode also drops claude.ai account connectors (F3b shows they load otherwise).
agent optional --agent <name>; stored as name + sha256 of the resolved file Goal: a project agent file can specialise the implementer. Assumed: --agent resolves project .claude/agents/<name>.md; plugin-namespaced names untested; how the agent body combines with the default system prompt untested.
append_system_prompt preamble.md, stored as sha256 Recorded at first request and reused (4c; --system-prompt-snapshot on is the default, designer read it in --help today). Passed identically every time anyway.
cwd --cwd, default the git toplevel; stored normalised Forks must run from the base's cwd (F4d); the transcript path derives from it (6d).
env.pinned CLAUDE_CODE_PROMPT_CACHE_TTL=1h, DISABLE_AUTOUPDATER=1 TTL constant across base and forks (F4b). Assumed: DISABLE_AUTOUPDATER stops an in-place update of the PATH install mid-run.
env.scrubbed list in §4

Runtime fields, taken from the system/init event of every call and compared with the base's: sorted tool names (hash), permissionMode, plugin names with versions, agent names, apiKeySource (assumed field name). A difference sets fingerprint: drift:[fields] on the result. Before every launch the helper re-reads binary.version/size/mtime; a change is a binary drift (rule in §5).

init.model is not recorded as the model. It lies under plan mode (F2a, F2c); the real model comes from modelUsage (§6).

3.4 Commands

All commands print one JSON object on stdout and use exit codes: 0 done/ok, 2 usage or refused, 3 wait timed out with jobs still running, 4 a job failed.

Command Inputs Does Returns
where — Preflight print for SKILL.md. script path, binary, versions, node, scrub check
init --run, launch fields (§3.3), --parallel-cap (default 2), thresholds (§5) Creates the state dir, .gitignore, mcp-empty.json, the config block in registry.json. Idempotent. config
train --run, --base <name>, --brief <file>, optional launch overrides (else the init config) New session: claude -p --session-id <uuid> with the full fingerprint and templates/train.md around the brief. On finish records base_commit (git rev-parse HEAD), training cost, context size, last hit. {job, base, session_id}
fork --run, --base, --block <id>, --phase <n>, --brief <file>, --mode sequential|parallel, --max-minutes (default 90), --max-budget-usd (optional) --resume <base session> --fork-session with the base's fingerprint, templates/fork.md + delta header + brief. Refuses a different model or cwd (that is a new base), refuses when running jobs on the run would exceed the parallel cap, refuses parallel mode unless the brief names its file set. {job, base, base_state: warm|expiring|cold, warm_until, rebase_due, retrain_due, reasons[]}
rebase --run, --base, --as <new name>, --brief <file> A fork whose job is to absorb what changed (templates/rebase.md: read the listed notes and diffs, do not edit code, update the trunk notes, reply READY). Its session becomes base <new name> with parent; the old base is superseded. as train
continue --run, --job <fork job>, --brief <file> --resume <fork session> without --fork-session: one more turn in that fork (gate failed, test fallout). Same fingerprint. Not a SendMessage resume, so no notification-routing hazard. {job}
wait --run, [job…] | --all, --any, --timeout <s> (default 540, max 570) Blocks until the condition holds or the timeout. Runs the keep-warm tick (§5) every 30 s while it waits. {done:[result…], running:[{job, block, elapsed_s, last_event_s_ago, turns}], pings:[…]}
usage --run, --phase, --block, --format json|md Aggregates ledger.jsonl. md prints rows for the run log. totals and rows (§6)
list --run Bases (status, warm state, warm_until, context tokens, training cost, forks, lineage) and running jobs.
expire --run, --base <name> | --all Marks bases retired: no forks without --force, no pings. Transcripts stay on disk (30 days, 6d).
kill --run, --job Kills the supervisor and its claude process tree (taskkill /T /F on Windows, process group on POSIX); result killed.

Internal: _supervise <job> (the detached worker) and tick (one keep-warm pass, also callable by hand).

3.5 Launch and supervision

  • Each launching command writes the job record, then spawns node forks.mjs _supervise <job> detached (detached: true, windowsHide: true, stdio to raw/) and returns at once. The supervisor spawns claude with the scrubbed env, --output-format stream-json --verbose, and the prompt from briefs/<job>.md on stdin.
  • The supervisor reads the stream as it arrives: the system/init event gives the session id and the runtime fingerprint; each assistant event bumps turns and last_event_at; the final result event gives usage, modelUsage, total_cost_usd, num_turns, is_error, permission_denials. Assumed: the stream's result event carries the same fields as --output-format json (1a verified them for json).
  • On exit it writes results/<job>.json, appends to ledger.jsonl, updates registry.json under the lock (registry.lock, open wx, stale after 30 s, write-temp-then-rename).
  • Watchdogs: --max-minutes exceeded → kill, status timeout. No stream output for 15 min → status flag stalled (reported by wait, not killed). permission_denials non-empty → needs_attention (a denied git commit would otherwise pass silently, 4a).
  • Fork session id: taken from init. Unknown whether --session-id combines with --resume --fork-session; if it does, the helper may pre-assign ids, but nothing depends on it.
  • Fallback if detached children do not survive on this host (R1): --attach runs the supervisor in the calling process; the Athanor starts it with the Bash tool's run_in_background: true and still polls with wait. The Athanor never relies on the background notification.

4. Env scrub and auth

Dropped from the child env (the twelve the research removed in E1 with auth still working, F3a, F3b):

CLAUDE_CODE_MESSAGING_SOCKET, CLAUDE_CODE_MESSAGING_TOKEN, CLAUDE_CODE_HOST_SESSION_ID, CLAUDE_CODE_ENTRYPOINT, CLAUDE_CODE_DESKTOP_APP_VERSION, CLAUDE_CODE_SDK_HAS_HOST_AUTH_REFRESH, CLAUDE_CODE_TERMINAL_MCP_TOOLS, CLAUDE_CODE_REPORT_FINDINGS, CLAUDE_CODE_SESSION_ID, CLAUDE_AGENT_SDK_VERSION, CLAUDE_CODE_ENABLE_ASK_USER_QUESTION_TOOL, CLAUDE_CODE_ENABLE_SDK_FILE_CHECKPOINTING.

Also dropped, because they would change the fingerprint or the auth path:

  • CLAUDE_EFFORT (effort comes from --effort), ANTHROPIC_MODEL, ANTHROPIC_DEFAULT_*_MODEL, ANTHROPIC_SMALL_FAST_MODEL, CLAUDE_CODE_SUBAGENT_MODEL (none present today, F2b; dropped so a future host cannot slip one in).
  • ENABLE_PROMPT_CACHING_1H, FORCE_PROMPT_CACHING_5M, CLAUDE_CODE_SUBAGENT_PROMPT_CACHE_TTL (TTL is pinned, §3.3).
  • ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, CLAUDE_CODE_USE_BEDROCK, CLAUDE_CODE_USE_VERTEX: with any of these the run would bill elsewhere and get 5 min TTL (F4a). Goal decision: OAuth.

Kept as in the tested E1 run: CLAUDECODE, CLAUDE_CODE_CHILD_SESSION, AI_AGENT, CLAUDE_CODE_EXECPATH, CLAUDE_PID, CLAUDE_CODE_GIT_BASH_PATH, PATH, HOME/USERPROFILE. Each kept var's role is unverified (F3a); keeping E1's set means the design runs the configuration that was tested.

Auth stays OAuth: no --bare (it refuses OAuth, F3d), no key in the env. The helper checks each init's apiKeySource (assumed field; expected none under OAuth) and the observed TTL (§5): a call that wrote only ephemeral_5m means the account fell out of the 1 h bucket (over plan limit, F4a), and wait reports ttl_downgrade: true. The scrub list lives in one constant in forks.mjs; where reports host vars that look Desktop-specific (CLAUDE_CODE_* not on either list) so a new host variable shows up instead of leaking silently.

Not settled by the scrub: whether the nested process's own messaging pipe can reach the host (F3c). The scrub removes the host socket and token; nothing more is claimed.

5. Keep warm, expire, rebase, re-train: the rule

All inputs come from the registry and the ledger; the Athanor only obeys the flags.

Per base:

ttl        = 3600 s if the base's last call wrote ephemeral_1h tokens, or read cache and the
             previous known ttl was 3600; 300 s if it wrote only ephemeral_5m; else 3600
             (config default for subscription).                                   [F4a, 3c]
last_hit   = time of the first assistant event of the most recent call on this base or any of
             its forks/pings (the first request reads the base prefix).           [F4c]
warm_until = last_hit + ttl
margin     = max(120 s, ttl / 10)              → 360 s at 1 h, 120 s at 5 min
state      = warm      if now <  warm_until - margin
             expiring  if now <  warm_until
             cold      otherwise

Assumed: a hit on the base prefix refreshes that prefix's entry (F4c is docs, not tested over time), and only a request that reads exactly the base prefix (a fork's first request) refreshes it; later requests in a fork read longer prefixes. Using the first assistant event, not the call end, is the conservative reading.

Keep warm (pings): a ping is --resume <base> --fork-session, prompt Reply OK, --max-turns 1, same fingerprint, logged as kind ping. Price ratios against the base input price, from Anthropic API pricing (assumed current, not re-checked in the research): cache read 0.1, 1 h write 2.0, 5 min write 1.25. One ping costs about 0.1 × prefix; one cold start about (write − read) × prefix, 1.9 × prefix at 1 h. So pings pay while fewer than break_even = floor((write − read) / read) are needed: 19 at 1 h, 11 at 5 min.

ping base when:  base.status == active
             and state == expiring
             and the run has a running job or a wait call in the last 10 min   (the Athanor is in its turn)
             and pings since the base's last real call < break_even

The tick runs inside every supervisor (every 60 s) and inside wait (every 30 s). No daemon: when the Athanor has handed back and nothing is running, nobody pings, and the base cools after its TTL. That is intended: between phases the base is rebased anyway.

Expire: a base is retired by expire, superseded by rebase or a re-train. A cold base stays forkable: the history is on disk (2b, 6d), only the cache is lost. Coldness is never a reason to re-train. A fork of a cold base is logged cold_start with its creation tokens.

Rebase due (flag on fork's return; the Athanor rebases before forking):

  • phase boundary (procedural: the first fork of a new phase always comes from a rebased base);
  • delta since base_commit above threshold: more than 6 commits, or more than 1,500 changed lines (git diff --shortstat), or a delta header plus listed notes above about 12k tokens (chars / 4);
  • binary drift (version, size or mtime changed): the rebase runs under the new binary and becomes the new fingerprint; one cold write, recorded.

Re-train due (fresh train, new lineage):

  • the requested model, cwd, tools, permission mode, agent or preamble differs from the base (these define the base; a fork cannot carry a change);
  • base context above max_base_tokens (default 250k), measured from the last assistant message's usage in the base transcript (~/.claude/projects/<cwd slug>/<id>.jsonl, 6d; assumed message-level usage is present in the transcript lines), leaving room under the ~600k comfortable figure (models.md, a guesstimate);
  • four rebases in one lineage (prefix growth).

Thresholds are starting values in init; the trial (§9) tunes them. They are numbers, not judgment.

6. Usage capture

Per call, from the result event:

  • Real model: the keys of modelUsage whose counts grew in this call. Not init.model (F2a, F2c).
  • Tokens, four columns: input, output, cache_creation, cache_read, plus cache_creation split into 1h and 5m from usage.cache_creation. Source: the per-model delta of modelUsage against the session's totals before the call (modelUsage and total_cost_usd are cumulative across the restored session, 3d, F4e). Top-level usage is stored too; the T2 numbers (creation 343 against the base's 8,661) suggest it is per call, so the two must agree; disagreement sets usage_mismatch (assumed semantics for multi-turn calls, R5).
  • Cost: cost_delta_usd = total_cost_usd(call) − total_cost_usd(session before call) (3d, F4e). For a fork, "session before" is the base's cumulative total at the fork point; for continue, the fork's own last total. Never sum total_cost_usd across calls. These are Claude Code's client-side estimates; under a subscription they are notional, which is still the right unit for comparison.
  • Cache verdict: hit if cache_read ≥ 0.9 × base prefix, partial if > 0, miss if 0 (a miss on a fork is a fingerprint bug or a cold base; wait says which).
  • Also: wall_s, num_turns, is_error, denials count, fingerprint verdict, binary version.

Ledger line (ledger.jsonl):

{"job":"j0007","kind":"fork","phase":1,"block":"1.3","base":"B1","session":"7ccea6b5-…",
 "parent_session":"433df7b7-…","started":"2026-10-16T14:02:11+02:00","ended":"…","wall_s":1310,
 "binary":"2.1.299","model_requested":"claude-sonnet-5-5","models_used":["claude-sonnet-5-5"],
 "tokens":{"input":5120,"output":38210,"cache_creation":61200,"cache_creation_1h":61200,
           "cache_creation_5m":0,"cache_read":2840100},
 "cost_delta_usd":1.92,"cost_cumulative_usd":4.37,"turns":57,"cache":"hit","base_state":"warm",
 "fingerprint":"ok","denials":0,"status":"ok","note":"notes/1.3.md"}

Into the run log: the Athanor fills each block row from usage --block <id> --format md and pastes usage --phase <n> --format md into the phase close: per-kind totals (train, rebase, fork, continue, ping), cold starts, cache hit ratio, priced total. The ledger file stays the source; the run log carries the copy the reviewer and Igor read. Goal said "writes them to the run log": the helper writes the ledger and renders the rows; the Athanor pastes them, because a script appending to a markdown file the Athanor is also editing would interleave. This is a design decision (see the end).

Agent-tool agents (mechanical blocks) report usage only in their completion result; the Athanor copies what it sees, as today. The root and the Athanor's own tokens are counted from transcripts in the trial (§9), not by the helper.

7. Blocking and polling

The headless process is invisible to Agent-tool tracking (6c), and a background Bash notification from inside a subagent is not something to build on (routing hazard in the spawning reference; unverified for Bash). So:

  • Launch returns at once (detached supervisor). Waiting is a foreground Bash call to wait, which blocks up to 540 s, under the Bash tool's 600 s ceiling (6c). Exit code 3 means "still running, call again". The Athanor loops: wait --all --timeout 540 until exit 0 or 4. A 36-minute block costs about four wait turns of a plan-level Athanor context, mostly cache reads.
  • Parallel forks: only for blocks the plan marks disjoint (the implementation skill's existing rule). Launch each with --mode parallel, then wait --all. Cap 2 by default (init --parallel-cap): forks on a subscription share plan usage, and an over-limit account drops to 5 min TTL (F4a). Two forks from one base in parallel are verified (2c); tool-using parallel forks in one worktree are not (R3).
  • Timeouts: per job --max-minutes (default 90) enforced by the supervisor; --max-budget-usd optional per call (counts the call's own spend, F4e). stalled is reported, not acted on; the Athanor kills or waits per the procedure (one more wait, then kill and flag).
  • Hand back only with nothing running: before the deliverable, list must show no running jobs, else kill and flag.
  • The Athanor never uses SendMessage on a fork and never needs TaskStop; continue and kill cover it.

8. Athanor procedure

8.1 What the Athanor holds

The build plan, the design, the run log, trunk notes, handoff notes, review reports, helper JSON, git log --oneline, git diff --stat/--numstat, gate and test summaries. Not source files. Its context stays plan-sized, so claude-opus-5-5[1m] keeps its pin but is no longer what pays for reading code (in the experiment, 53 M of 55.6 M Athanor tokens were cache reads of a 1M-context session that had read the trunk).

8.2 One phase per turn

  1. Intake. Read plan, design, run log, trunk notes, the last review. Check the branch (git rev-parse --abbrev-ref HEAD). First phase: forks init with the launch fields the build plan or project config names (model = the plan's owner model for complex blocks, full id; effort; agent file if the project has one).
  2. Train or rebase.
    • First phase: train --base B1 with a brief naming the plan, design, the phase's blocks, the areas to read, and the trunk-notes path. The base reads the code, writes the trunk notes, changes no code, replies READY with a short summary. Trunk notes are now the base's product, not the Athanor's.
    • Later phases, and whenever fork returns rebase_due: rebase --as B<n+1> with the delta (handoff notes, review findings, commit range). retrain_due: train a new lineage. The helper decides which; the Athanor follows.
  3. Route blocks. Complex block in the trained area → fork. Mechanical or distant block → fresh Agent-tool agent in the foreground with the trunk notes, as today. A complex block whose owner model differs from the base model either becomes a fork anyway (logged deviation) or gets its own base; it never gets a fork with --model (that loses the cache, 3b, and the helper refuses it).
  4. Fork per block with a delta brief (goal default): block id and plan section, the handoff notes of blocks done since the base, open review flags touching the block, the mode, the gate and test commands. The helper prepends the commit range since base_commit itself. Then wait until done, read notes/<block>.md and the result JSON.
  5. Check, mechanically. Commit present (git log); the evidence log under logs/ exists and is non-empty (experiment proposal 4); git diff --numstat on test paths and a count of removed assertion lines (git diff <range> -- <test paths> | grep -c '^-.*assert', pattern from project config). A non-zero count goes to the fork's note as a question and into the reviewer flags. This replaces the Athanor's former "read the fix's diff for loosened assertions", which was a code read; the fresh phase reviewer still looks for weakened checks.
  6. Gate between blocks: the plan's command, output to logs/. Red → continue --job <fork> with the log path, once; still red → flag and go on per "what may stay broken".
  7. Tests once at phase end: the Athanor runs the test command to a file under logs/ (never through tail, experiment incident 4; first run after a fresh worktree is a warm-up, rerun once). Fallout → one fork from the current base (or continue of the block that caused it) with the failing log as the delta.
  8. Run log: block rows from usage, notes appended from notes/, phase close with the usage totals and every flag.
  9. Hand back with no running jobs. Do not expire the base: if Igor's review and the fix pass come back within the TTL, the base is still warm; if not, it is cold but forkable.

Fix pass (Igor resumes the Athanor with findings): forks of the current base with the findings as the delta; rebase_due decides whether to rebase first.

8.3 Commits and tests: who runs them

The experiment had index-lock races between parallel children and a red commit when the Athanor committed a child's block before calibrating it (M5).

Case Who commits Who runs tests
Sequential fork (default) The fork, by explicit paths, after its block gate and the block's named tests pass. Calibrate, then commit. The fork (block-scoped tests, output to logs/).
Parallel forks (disjoint files only) Nobody inside the forks (no index-lock race). After wait --all, the Athanor runs the gate and each block's named test command, then commits each block by the explicit paths its note lists, one at a time. Mechanical; no code read. The Athanor, after all parallel forks finish. Forks run nothing that touches shared state unless project config says parallel test runs are safe.
Fresh Agent-tool agent As a sequential fork. As a sequential fork.
Phase test suite — The Athanor, once, to a file.

Only one committer exists at any moment: sequential forks commit one after another; parallel forks never commit.

8.4 Text changes

  • agents/athanor.md: frontmatter skills: adds spire:forks. Intake: drop "the code the phase touches … That reading is your trunk"; the Athanor reads plans and reports and trains the base. Step 2: replace "a fork of yourself (it inherits your model and, assumed, your context)" with "a headless fork of the trained implementer (spire:forks)"; fresh agents stay for mechanical or distant blocks. Remove inline building by the Athanor. Keepers: add "you do not read code; the base and its forks do", "nothing running at hand back", "commit rules per §8.3". Replace "read the fix's diff for loosened assertions" with the mechanical check of §8.2 step 5. Deliverable: add the usage totals and base state.
  • skills/implementation/SKILL.md "Context: fresh, fork, or keep": rewrite "trunk and forks" as "trained implementer and forks": one base per build, trained early, forked per complex block with a delta brief, rebased at phase boundaries, fresh agents for mechanical blocks; cite spire:forks for mechanics. The Agent-tool fork path is gone. "Spawning and models": add that headless forks take a full model id. Keep the general "if one agent can do the whole change in one go, do it inline" for non-Athanor use.
  • references/run-log.md: header gets Forks: <state dir>; block table becomes Block | Owner (model id from modelUsage) | Session | Commit | in | out | cache write | cache read | cost Δ; phase close gets the usage --phase table; trunk notes are written by the base at training.
  • references/models.md: "Future work: orchestrator introspection" gets a dated note that headless forks report per-call usage; add the plan-mode Haiku→Sonnet observation and the alias-by-version note (F2a–F2c).
  • Out of scope, flagged for Igor: skills/igor/SKILL.md:66 ("forking yourself … to brief a new expert") is now close to what the base does; it may want a line.

9. Spawning-reference updates (references/spawning-claude-code.md)

Fork section (Agent tool), rows changed:

Row today New text and status
"fork is actually offered … unverified" Not offered on 2026-10-05 or 2026-10-15. Spire no longer uses Agent-tool forks (goal decision 2026-10-16). observed
"A fork copies the caller … assumed" A fork inherits the current conversation only; no parameter names another agent. verified 2026-10-15 (docs sub-agents, provenance §5)
"A fork starts with the caller's full transcript … assumed" Same system prompt, tools, model, history as the forking session. verified 2026-10-15 (docs)
"Can a fork fork again? … unknown" Agent-tool: "A fork can't spawn further forks." verified 2026-10-15 (docs). Headless --fork-session: a fork of a fork works. verified 2026-10-16 (2d, T6)
new Fork mode is on by default in interactive sessions (v2.1.232+), off under -p/SDK unless CLAUDE_CODE_FORK_SUBAGENT=1; with it on, all subagents run in the background and run_in_background is removed. verified 2026-10-15 (docs)
new Agent-tool forks and subagents use the 5 min cache bucket by default; the main conversation (incl. -p) gets 1 h on a subscription within plan usage. verified 2026-10-16 (F4a, docs)

New section "Headless sessions (claude -p)", rows, each with its research id and date 2026-10-16:

  • --output-format json result fields; stream-json init fields (1a). --resume by id across projects; --session-id to choose an id (1b, 1d); don't use --continue for fan-out (1c).
  • --fork-session: new id, copy of history, base unchanged, forks isolated, parallel forks fine, resuming one id in two processes interleaves (2a–2c).
  • Cache: hits need identical tools, model, system prompt; flags are not inherited by a fork; pass them every time (3a, 3b, 4b, 4c). Model is restored on resume, --model overrides it (4d). The system prompt is recorded once and reused (4c; --system-prompt-snapshot, --help 2.1.283).
  • Cost: total_cost_usd/modelUsage are cumulative across a resumed session; per-call cost is a difference (3d, F4e).
  • TTL: 1 h main bucket on subscription, 5 min otherwise; configurable; a hit resets the timer; same cwd for shared prefix (F4a–F4d).
  • Permissions under -p: nothing can prompt; denials are listed and the run continues (4a).
  • init.model is not the model that ran; plan mode upgrades Haiku to Sonnet; aliases change with CLI version (5d, F2a–F2c).
  • Discovery: without --bare, plugins, skills, hooks, MCP and CLAUDE.md load (5a); --bare needs an API key (5b, F3d).
  • Nesting: inherits CLAUDECODE=1, no recursion guard (6a); Agent depth is per process and not passed down; at the limit the Agent tool is simply absent, no error (F1a–F1d).
  • Tracking: invisible to task notifications, ListAgents, TaskStop; --bg does not combine with -p (6c, the invisibility row stays assumed).
  • Transcripts per fork under ~/.claude/projects/<cwd slug>/ (6d).
  • Desktop env inheritance and the scrub (6e as corrected by F3a–F3c).
  • Two binaries; pin one and a full model id (F5a, F5b).

Models section: the alias-only row stays for the Agent tool; add "headless --model takes full ids, so headless forks pin by full id".

Rules of thumb: add "complex blocks go to headless forks via spire:forks; never resume or fork a headless session by hand".

10. The success-criteria trial

Preflight first (Phase 0 of the plan, scratch repo, before any Spire text changes ship). Each item maps to a risk in §11:

  1. Detached supervisor survives the end of the Bash call that started it, from inside an Agent-spawned subagent on this Windows host (R1). If not, test --attach under run_in_background.
  2. A base trained to ≥ 100k tokens with real tool use; three sequential forks that read and write files; each fork's first request hits the base prefix (R2).
  3. Two parallel tool-using forks on disjoint files in one worktree; no index lock (they don't commit); gate afterwards (R3).
  4. modelUsage delta equals top-level usage on a multi-turn fork; stream-json result has the json fields (R5).
  5. A ping after 50 min keeps a fork at 60+ min warm; a fork after 65 min without ping is cold (R7). This is the only test that needs wall time; it can run beside the others.
  6. --effort, --strict-mcp-config, --agent, Skill in --tools, [1m] in --model, apiKeySource in init: each accepted and visible in init (R8, R9, R15).

Any failure in 1, 2 or 4 stops the plan and goes back to Igor (§11, "what would make this wrong").

The trial (goal: "Proven, not assumed"):

  • Plan: one real multi-block plan the user picks, with at least two phases and at least three complex blocks in one area (otherwise forks have nothing to share). Not this skill's own build: the tool under test cannot measure itself.
  • Arms: F = Athanor with spire:forks (this design). C = the current 0.2.2 shape (Athanor reads the trunk, builds or delegates per its current text). Same build plan, same base commit, separate worktrees, same reviewer model and review structure (one fresh Opus reviewer per phase, one fix pass, re-check), Igor as root in both.
  • Isolation: sealed inputs per arm and a transcript audit with a positive control, as in the 2026-10-15 experiment (§1 there). Arm F's headless transcripts live under the worktree's project slug and are audited too.
  • Quality: blind implementation judge (Fable, as before) on the two diffs; reviewer findings by severity per phase; tests and gates.
  • Tokens, all four columns, priced, for every agent including the root: Igor, the Athanor, Agent-tool children, every headless call (the ledger, cross-checked against the fork transcripts with copied base history counted once, deduplicated by message id; assumed copied messages keep their ids), reviewers. Price with one published per-model table, written into the report.
  • Wall time: from date and git, start to last re-check, per arm; Athanor active time per phase.
  • Mechanism metrics (arm F): cache verdicts per fork, cold starts, pings and their cost, rebases, re-trains, permission denials, fingerprint drifts, ttl_downgrade.
  • Report: reports/<stamp>-headless-forks-trial.md. Thresholds in §5 are tuned from it.

11. Risks and unknowns for the planner and attacker

# Risk / unknown Check
R1 Detached children may be killed when the Bash tool call ends (Windows job object). Untested. Preflight 1; fallback --attach + run_in_background.
R2 Large tool-using bases are untested (research used tiny contexts). Cache behaviour, compaction inside a fork, and whether the recorded system prompt is reused for a 100k+ base. Preflight 2.
R3 Parallel tool-using forks in one worktree: file collisions, shared test resources. Preflight 3; parallel only for disjoint files; cap 2.
R4 Permissions. The Athanor's node calls may prompt in the host session (closed door). Forks under dontAsk deny silently. allowed-tools on the skill (assumed effective); needs_attention on denials; the user may need an allow rule for node */forks.mjs *.
R5 Top-level usage per call vs. aggregate over turns; stream-json result field parity with json. Preflight 4.
R6 --session-id with --fork-session. Not depended on.
R7 TTL refresh on hit and the base-prefix refresh semantics are docs-only; cross-cwd forks untested. Preflight 5; forks always run in the base's cwd.
R8 --effort vs. CLAUDE_EFFORT and effortLevel precedence, and whether effort affects the cache. Preflight 6; env var dropped, flag pinned.
R9 Mid-run drift outside the fingerprint: plugin updates, SessionStart hooks injecting context on resume, claude.ai connectors. Runtime fingerprint compare on every init; --strict-mcp-config.
R10 Whether the nested process's pipe can reach the host (F3c). Not claimed; scrub only removes the host socket and token.
R11 Plan usage. Parallel Opus/Sonnet forks consume the subscription; over the limit the TTL drops to 5 min and usage may bill as credits. ttl_downgrade flag; parallel cap; per-call --max-budget-usd.
R12 Binary changes mid-run (in-place auto-update; a Desktop update replacing the $CLAUDE_CODE_EXECPATH directory). Version/size/mtime check before each launch; DISABLE_AUTOUPDATER=1 (assumed honoured); binary drift → rebase.
R13 A fork's recorded system prompt carries the base's git status and CLAUDE.md from training time. The helper's delta header lists commits since base_commit; the fork template says the snapshot is stale.
R14 Node may be missing on other hosts. where fails loudly; fresh-agent fallback, logged.
R15 --agent with project and plugin agent files; how the agent body combines with the default system prompt and --append-system-prompt-file; agent tools: frontmatter against --tools. Preflight 6.
R16 Safety. Forks hold full Bash under dontAsk; deny patterns are easy to sidestep. Same exposure as today's children; project deny rules apply; the brief's scope fence.
R17 Delta briefs may leave forks with a stale view of neighbouring blocks, so they collide or contradict. The rebase threshold; the trial's quality arm.

What would make this approach wrong:

  • Forks of a large trained base do not hit the cache reliably (R2). Every fork then pays full creation at 2× input price on top of the training, and fresh agents with trunk notes are cheaper. Stop and go back to Igor.
  • Headless calls cannot run longer than one Bash call from inside a subagent on this host (R1 and its fallback both fail). The 9-minute ceiling is shorter than real blocks (11–36 min in dirigent).
  • The trial shows no quality gain over arm C at higher cost. The goal's intent then still stands, but the evidence says no; that is the user's call.
  • Plan-usage policy makes nested headless sessions impractical (R11).

Rejected alternatives

  • Agent-tool forks, or the trained implementer forking itself. No fork-source parameter; "a fork can't spawn further forks"; fork mode forces background; depth would sit at the 3-layer limit (provenance §5). The goal drops this path.
  • Carry one implementer and resume it with SendMessage. Accumulates context (the 960k drift) and has the subagent-resume routing hazard.
  • MCP server now. The goal puts it later; a script is quicker to prove and lifts into Dirigent or an MCP wrapper unchanged in contract.
  • --bare + API key. User chose OAuth; --bare would also cost the 1 h TTL.
  • Background Bash notifications or Monitor as the wait mechanism. Unverified inside a subagent; polling files with wait works for any caller.
  • A long-lived --input-format stream-json base process fed per block. It continues one session; it cannot branch it, so forks would still need --resume --fork-session.
  • Registry in the user's home or the OS temp dir. Survives across runs nobody needs, and the reviewer and a successor Athanor could not find it next to the run log.

Decisions this design made that the goal did not settle

  1. Skill name forks; helper in Node (no jq, no Python here).
  2. State per run, in a -forks/ folder beside the run log; raw streams gitignored.
  3. Usage reaches the run log by paste, not by append: the helper writes the ledger and renders markdown rows; the Athanor pastes them. Deviates from the goal's literal "writes them to the run log" to avoid interleaved writes.
  4. Launch flags supplied once at init/train; forks take only base, block and brief.
  5. Defaults: dontAsk, tools Bash,Read,Edit,Write,Glob,Grep,Skill (no Agent), --strict-mcp-config with no servers, explicit --effort (default high), CLAUDE_CODE_PROMPT_CACHE_TTL=1h, DISABLE_AUTOUPDATER=1, binary $CLAUDE_CODE_EXECPATH first.
  6. Coldness never triggers a re-train; pings only while the Athanor is in its turn, capped at the break-even count; numeric rebase and re-train thresholds as starting values.
  7. Trunk notes are written by the base at training, not the Athanor.
  8. The Athanor's diff read for loosened assertions becomes a mechanical count; the phase reviewer keeps the real check.
  9. Commit ownership: sequential forks commit after calibrating; parallel forks never commit, the Athanor commits them by explicit paths after the gate.
  10. The Athanor no longer builds blocks inline.
  11. The trial needs a Phase 0 preflight on a scratch repo; failures in R1, R2 or R5 stop the plan.

Amendment 2026-10-16 01:33 — runtime (user)

The user decided that the helper is Python, run with uv (uv run, single file with PEP 723 inline metadata, standard library preferred), not Node. uv 0.5.13 is installed at ~/.local/bin/uv; python is not on PATH, so it is always invoked through uv run. Follow the conventions in plugins/project-uv/. The rest of the design stands. The planner was told mid-run. Source: session journal.

Reference in New Issue View Git Blame Copy Permalink