Files
reliquary/docs/workpad/research/2026-10-05-21-athanor-usage-spacecraft.md
T
g4borg b0b315136e 📝 Add lean Athanor workpad (goal, journal, research, reports)
Captures the lean-build redesign work: goal, session journal, Athanor
usage research from dirigent and spacecraft, and the implementation
and review reports.
2026-10-05 22:21:55 +02:00

77 lines
13 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Athanor usage in spacecraft — survey of experience reports
**Date**: 2026-10-05
**Scope**: read-only survey of `/home/g4b/Projects/spacecraft` docs/workpad and `.claude/` for documents reporting on Athanor orchestration runs (problems, cost, model behaviour, delegation issues, proposed changes).
## 0. Critical disambiguation — spacecraft's "athanor" is not Spire's `athanor` agent
Spacecraft does **not** use Spire's `athanor.md` orchestration agent as such. Its docs use "athanor" purely as an informal nickname for runs of the **superpowers** plugin's `dispatching-orchestrator` agent (`.claude/agents/dispatching-orchestrator.md`, spawned with `model: "opus"`). This is stated explicitly:
> "The spire plugin (`athanor.md`) is installed only for `/home/g4b/Projects/dirigent`, so it is not active here. Here, 'athanor' just means our `dispatching-orchestrator`."
> — `docs/workpad/reports/2026-09-29-13-orchestration-overhead-attribution.md:34`
As of today, spacecraft's `.claude/settings.json` lists `"spire@reliquary": true` in `enabledPlugins`, and the plugin cache (`~/.claude/plugins/cache/reliquary/spire/0.1.1/agents/athanor.md`) exists and was last updated 2026-10-05. However, `~/.claude/plugins/installed_plugins.json` still shows spire's project-scope install pointed at `/home/g4b/Projects/dirigent`, and spacecraft's own `.claude/agents/` directory contains **no** `athanor.md` override — only `dispatching-orchestrator.md`. The most recent journal (2026-10-05) still refers to dispatches as "dispatching-orchestrator (opus, bg)", not as the Spire agent type. So: **everything below is about the superpowers `dispatching-orchestrator`, colloquially "athanor," not about Spire's athanor agent.** There is no evidence spacecraft has ever run Spire's actual athanor agent, and therefore **no local overrides of the Spire athanor agent or its skills exist in spacecraft.**
## 1. Ranked list of relevant files
1. **`docs/workpad/reports/2026-09-29-13-orchestration-overhead-attribution.md`** (2026-09-29) — **primary report.** Detailed attribution of orchestration overhead across an 11-element pipeline (review loops, model tiers, mutation checks, journals, layering), with ledger evidence and ranked cost-cutting recommendations.
2. **`docs/workpad/reports/2026-09-27-13-strategy-mvp-retrospective/report.md`** (2026-09-27) — **primary report.** Full retrospective on why Round 1 (~80 tasks, 7 athanor/orchestrator instances) took ~60 hours; quantifies cost per seat, per-model, and gives concrete recommendations.
3. **`docs/workpad/reports/2026-09-29-23-strategy-igor-handoff-b5.md`** (2026-09-29) — handoff doc summarizing the switch to a "lean review process" after B3 (halved time per task: ~75min → ~40min) and listing recurring problem zones to watch (performance regressions, weakened checks, untestable tests, stalls).
4. **`docs/workpad/brainstorm/the_assistants_mindset.md`** (section "Lean pipelines: don't stack quality gates", 2026-09-29) — distilled lessons/principles for running athanors, plus meta-lessons about the coordinating assistant (Igor) over-extrapolating user feedback.
5. **`docs/workpad/journals/2026-09-28-20-strategy-g1b-continue.md`** — session journal recording concrete stall incidents ("B3 athanor stalled twice … nudged with results") that motivated the foreground-dispatch rule.
6. **`docs/workpad/journals/2026-10-05-00-strategy-mvp-continue.md`** (most recent, 2026-10-05) — latest status: G1-C dispatched with per-phase review (even leaner than B5's per-task-group review), then paused mid-run by the user; notes an orchestrator resumed via SendMessage instead of fresh agents (flagged as a mistake, corrected to "fresh foreground agents").
7. Supporting/secondary: `docs/workpad/reports/2026-09-26-22-strategy-mvp-r1-delivery-report.md`, the seven `docs/workpad/plans/strategy-mvp/2026-09-24-r1-athanor-handoff*.md` handoff files (raw per-slice handoffs, less synthesized), and the chain of delivery journals under `docs/workpad/journals/` (g1b-b3/b4/b5/b6, g1c) which record ledger-level detail behind reports #1–#2.
## 2. Dense summaries of top documents
### #1 — Orchestration overhead attribution (2026-09-29)
Scope: the "assistant → dispatching-orchestrator ('athanor') → leaf agents" pipeline across G1-B slices B1–B3.
- **Superpowers supplies only the skeleton** (per-task review loop, ledger, 5-round fix cap, ruling format). "Most of the weight is ours": Opus 5.5 on every review/re-review/fix round, a fresh fixer every fix round, mutation checks in fresh `CARGO_TARGET_DIR`s, minors carried forward, a final review per slice (not per plan), three prose records per slice, oracle/golden ceremony, and a third coordination layer (assistant above orchestrator).
- **First reviews earned their cost**: 12/15 found Important or Critical issues; the only clean ones were Sonnet run-and-record tasks. Mutation checks produced 6–9 of the Important findings — "the most productive single check in the pipeline."
- **Re-reviews and deferred minors did not pay off**: 11/13 re-reviews only confirmed fixes; of 16 deferred minors none were ultimately must-fix (5 fixed incidentally, 9 still open, 1 promoted to its own task).
- **Cost of the third layer (E5)**: "Athanor went idle: its background children notify the root session [not it]" (`journals/2026-09-24-10-strategy-game-mvp.md:33`); "B3 athanor stalled twice … Nudged with results" (`journals/2026-09-28-20-strategy-g1b-continue.md:25,28,38`). Each stall cost a user-visible nudge cycle.
- **Ranked cuts** (all achievable without editing the superpowers plugin, via CLAUDE.local.md / plan execution model / orchestrator-written templates / memories): (1) skip per-task review for run-and-record/gate/metric tasks, (2) re-reviews on Sonnet not Opus 5.5, (3) minors: ledger-only, never fixed/carried, (4) cheaper/narrower mutation runs, (5) batching becomes default, (6) one final review per plan not per slice, (7) drop the delivery journal, (8) collapse layers (run orchestrator as top-level session), (9) resume drafter for fix rounds when drafter was already Opus 5.5, (10) relax re-bless policy.
- Model mentions: "Opus 5.5" used pervasively for review/fix/plan; "Opus 4.6" for drafts; Sonnet for run-and-record; no mention of Fable or Opus 5.x beyond 5.5 in this doc.
### #2 — Strategy MVP retrospective (2026-09-27)
Scope: why R1 (~80 tasks, 7 orchestrator ("athanor") instances, handoffs 1–7) took ~60 wall-clock hours.
- Three multipliers identified: **(1) scope** — Opus 5.5 plans were ~11× larger than any earlier plan, driving 77k lines of Rust instead of a plausible 10–20k for a "4.6 just build it" MVP; **(2) per-task pipeline cost doubled mid-run** — active time per task rose from ~10 min (plans A/B) to ~40 min (D2/E1) to ~70 min (E2), driven by "a mandatory fresh Opus 5.5 fixer on every task" whose fixer seat "often cost more than the draft"; **(3) serial execution with long tails** — only 2.7 of 45.8 delivery-session hours had two workers active simultaneously; one unbudgeted balance agent ran 7.1h; one blocking `AskUserQuestion` waited 5.9h overnight.
- **Housekeeping bug noted**: "the `opus46-*` agent names and the `dispatching-orchestrator` description still say 'pinned to 4.6'. In practice, spawning them with `model: 'opus'` overrides the pin (the alias resolved to 5.5 in 184 of 184 spawns since 2026-09-23)." — i.e., agents believed to run Opus 4.6 were actually all running Opus 5.5; naming is misleading in transcripts/journals.
- Model-to-seat table: orchestrators ("7 athanors") = Opus 5.5; design/plans = Opus 5.5; drafts (80) = Opus 4.6 (intended); fix rounds = Opus 5.5 (intended); reviews mostly 5.5, some Sonnet 5, **Fable 5.1 used for two design reviews** (the one Fable mention found in this survey).
- **Recommendations**: give the 5.5 planner a scope budget (cap plan size, e.g. ≤60KB/file); plan coarser (contracts/invariants per phase, not signatures per task); right-size the review loop (cheap review per task, 5.5-with-mutation only per plan file or hotspot; minors batched not upgraded; fixes go to the resumed drafter, fresh 5.5 fixer only for Criticals); budget judgment tasks by count; never block overnight (default + timeout, or park and continue); allow limited parallelism across non-overlapping crates; tie scope strictly to the stated goal bar ("working machinery, not perfect gameplay").
- Proposes a controlled lean-vs-current-policy experiment (§6) measuring defects-remaining-per-hour rather than trusting the process's own review counts.
### #3 — Igor handoff B5 (2026-09-29)
- Confirms the lean process adopted mid-B3 "halved the time per task (about 75 min → about 40 min) and kept review quality."
- Lean rules: one Opus 5.5 review per hotspot/design task (combined review for groups of small tasks); run-and-record steps get no review; re-review only for Critical/Important, by a fresh Sonnet; fix rounds resume the implementer **within one task only** — "Never carry an implementer across tasks. In B3 it drove one implementer to about 960k tokens"; minors fixed or dropped same round, no batch/final-slice review; dispatch sequential steps in foreground, no file watchers/Monitor tasks (background grandchildren notify the user/Igor, not the athanor).
- **Problem zones to watch** (recurring across slices): (1) performance trending wrong way but non-gating, must be recorded as explicit finding with profiling; (2) "the reflex to weaken a check when it fails" happened three times (p1 oracle in B1, R15 in B3, R26 in B4) — each caught and reversed, rule: no check weakened silently, any report-only downgrade needs an explicit ruling; (3) tests that cannot fail recur (Sonnet-authored, and two B4 implementers skipped RED capture) — mutation checks are the backstop; (4) design text lagging behind code via accretion; (5) notification routing — B4 ran with zero stalls once foreground dispatch was enforced.
### #4 — Assistant's mindset: "Lean pipelines: don't stack quality gates" (2026-09-29)
- Core lesson: "Every review layer re-reads the same code … One strong review sized to the risk beats three small ones." The prior additions (mutation checks everywhere, batch reviews on top of per-task reviews, a final whole-slice review, minor carry-lists, briefs restating the ruleset) "cost more time and tokens than they returned."
- Routing guidance: "Opus for mechanical and design-heavy work. Sonnet for run-and-record and checklist work: it is clean there, but it wrote tests that could not fail and could not finish an open-ended performance task. Haiku for runs and appliers, with its self-reports treated as claims to verify (it has reported 'no deviations' alongside misplaced edits)."
- Meta-lessons about Igor (the coordinating assistant) itself: don't over-extrapolate a single user remark into multiple new rules; when corrected, don't overcorrect in the opposite direction either — engage with what the remark actually implies; change rules at a clean cut (soft-retire the running athanor with a checkpoint, start fresh on new rules) rather than patching a live run.
### #5 — Latest journal, 2026-10-05 (strategy-mvp-continue)
- G1-C dispatched with an even leaner regime: "review once per phase, not per task; more code first, one fix run later" (explicit user instruction, folded into memory `feedback-lean-review-process`).
- Notable operational event: "G1-C interim hand-back (orchestrator resumed its C.1 implementer via SendMessage → bg → yielded)... Resumed the orchestrator for C.2+; told it to use fresh foreground agents instead of SendMessage resumes" — i.e., a SendMessage/background-resume pattern was tried and explicitly walked back in favor of fresh foreground agents, consistent with the stall/notification problems in reports #1 and #3.
- Run was then paused mid-slice by direct user request (not a failure): C.1 done and pushed, C.5 done locally unpushed, C.6 left as uncommitted WIP in the worktree.
## 3. Local overrides of athanor/Spire agent or skills
None found. Specifically:
- No `athanor.md` exists anywhere under spacecraft's `.claude/` (checked `.claude/agents/`, worktree copies).
- The only orchestrator agent file present is `.claude/agents/dispatching-orchestrator.md` (from the **superpowers** plugin's convention, not Spire), preloading `superpowers:subagent-driven-development`.
- Spire (`spire@reliquary`) appears enabled in spacecraft's `.claude/settings.json`, but the installed-plugin record still scopes it to `/home/g4b/Projects/dirigent`, and nothing in spacecraft's docs/workpad or `.claude/skills` references Spire's athanor agent or its skills directly.
- The steering/override mechanisms spacecraft actually uses for "athanor" behavior are, per report #1: `CLAUDE.local.md`, the plan's "Execution model" section, orchestrator-written templates (`review-common.md`, `implementer-contract.md`), project memories (e.g. `prefer-opus46-for-detail-work`, `feedback-lean-review-process`, `feedback-count-based-agent-budgets`, `one-orchestrator-at-a-time`), and `.claude/skills/assistant/SKILL.md` — none of which are Spire files.
## Bottom line
There is substantial, high-quality material (two deep quantitative reports plus several handoffs/journals), but it is all about the **superpowers `dispatching-orchestrator`** informally nicknamed "athanor" in spacecraft's own vocabulary — not about Spire's actual `athanor` agent, which spacecraft has not been shown to run or override.