Files
reliquary/docs/workpad/reports/2026-10-05-22-lean-athanor-review.md
g4borg b0b315136e 📝 Add lean Athanor workpad (goal, journal, research, reports)
Captures the lean-build redesign work: goal, session journal, Athanor
usage research from dirigent and spacecraft, and the implementation
and review reports.
2026-10-05 22:21:55 +02:00

12 KiB
Raw Permalink Blame History

Lean Athanor — review

Reviewer: Opus 5.5, model id claude-opus-5-5 (from my system prompt). Fresh general-purpose agent dispatched by Igor; did the work myself, no subagents. Date: 2026-10-05 Scope: commits e515207, 146a48c on main, measured against goals/lean-athanor.md. Implementer report: 2026-10-05-22-lean-athanor-implementation.md.

Verdict

Pass with fixes. The skill is lean, it reads as judgment rather than a checklist, and the attack step is framed correctly. The skill states the attack as adversarial-against-code first, then a reshape, in one pass, run outside the Athanor, with its risk list kept from the reviewer. Superpowers is demoted correctly and the stale guidance is gone. Three Important gaps remain, all in the model and routing rules on Igor's side: Igor's own examples break the new alias rule; nothing makes sure plan, attack and review actually run on 5.5; and one stale routing rule still hands the Athanor unplanned work.

Counts: Critical 0 · Important 3 · Minor 8.

Goal checkboxes

# Criterion Status Evidence
L1 Intent + defaults with reasons, room for judgment, no incident rules met implementation/SKILL.md:8-12, :36-41 (each default carries its why)
L2 Short; detail in references met 85 lines vs 81 before; three references
L3 Harness/model traits in references; project choices from spire:project met SKILL.md:62-66, :77-79. Note: athanor.md:55 hard-codes "Stage explicit paths" (M6)
A1 Fresh agent attacks the plan against the code, then reshapes it, in one step met SKILL.md:26; igor/SKILL.md:95 (4b)
A2 This step pays for dropping per-task review met SKILL.md:28; igor/SKILL.md:95
A3 Described by intent, not by output fields met SKILL.md:26-30 is prose; examples, not required fields
A4 Risk list is not the reviewer's scope met SKILL.md:30; igor/SKILL.md:95 (6)
A5 Outside the Athanor; user sees reshaped plan met SKILL.md:17, :22; igor/SKILL.md:95; athanor.md:51
B1 Gate between blocks; tests once; one review, one fix, C/I re-check, stop met SKILL.md:39, :53-54; igor/SKILL.md:100-101. Re-check failure path unclear (M4)
B2 Igor-dispatched fresh reviewer, stamped id, answers flags, hunts weakened checks met SKILL.md:51-52; igor/SKILL.md:95
B3 Extra reviews as a reasoned choice met SKILL.md:55; igor/SKILL.md:106
T1 One phase per turn: intake/trunk notes → blocks → gate → tests → run log → hand back met athanor.md:27-36. Trunk notes optional elsewhere (M1)
T2 Igor resumes for fix pass / next phase, or starts fresh met igor/SKILL.md:108; athanor.md:40, :68
T3 Plan sets block/owner defaults; deviation logged met athanor.md:38; SKILL.md:41
T4 Athanor does not plan, attack, or self-review met athanor.md:10-11, :51 (but see I3)
T5 Trunk and forks as an option, verified vs assumed marked, forks inherit the model partial Reference marks it correctly (spawning-claude-code.md:58-61). But SKILL.md:47 and athanor.md:32 state "inherits its context" as fact (M5)
M1 5.5 default for Athanor, plan, attack, review; Athanor pinned partial Pin at athanor.md:13; defaults at models.md:11, igor/SKILL.md:95. Nothing secures 5.5 for general-purpose planner/attacker/reviewer (I2)
M2 Alias-over-pin; omit model for pinned agents; stamp partial Rule at igor/SKILL.md:89, model-tiers.md:36. Contradicted by igor/SKILL.md:84,133,142,151 (I1); its example is confounded (M2)
M3 Model notes are dated observations, user's points kept met models.md:13-43
M4 Stale guidance removed met model-tiers.md:34-37; igor/SKILL.md:173; old "run tests between agents" gone
C1 Context figures are guesstimates; judgment between extremes met models.md:5, :19, :49; SKILL.md:45
C2 Future work: introspection / signals met models.md:51
S1 No subagent-driven-development preload; lean rules win met athanor.md:15-18, :60; SKILL.md:84
S2 Plan-writing swappable met SKILL.md:32
J1 Journals and run log are the record; journals are for later mining met SKILL.md:73

Findings

Important

  • I1. Igor's own adept examples break the new pin rule in the same file. The rule says pinned agents get no model unless Igor deliberately overrides. Yet the delegation table and all three spawn templates pass model: "opus" to the 4.6[1m]-pinned adept, with no note that this is an override. An Igor that copies the templates silently overrides the pin, and may also lose the [1m] context. Evidence: igor/SKILL.md:84, :133, :142, :151. Fix: remove model: "opus" from the row and the three templates so they match :89. Whether adepts should move off 4.6 is the user's call.
  • I2. Nothing makes sure planning, attack and review actually run on 5.5. These roles are dispatched as general-purpose agents with the opus alias. Igor is pinned to claude-opus-4-6[1m], and the user's own observation is that the alias follows the caller. Under that observation, a 4.6 Igor gets 4.6 reviewers, which is the dirigent "Pass" failure the goal is built around. The stamp rule only records the model; it does not enforce it, since a review stamped 4.6 still "counts". Evidence: igor/SKILL.md:95 ("Opus 5.5 agent" with no means), SKILL.md:51, agents/igor.md:9. Fix now: add one sentence to Igor's model rule: for the plan, the attack and the phase review, check the stamped id; if it is not 5.5, re-dispatch or tell the user before acting on the result. Follow-up: pinned 5.5 planner/reviewer role agents (the redesign report and the dirigent §5.4 both propose them).
  • I3. A stale routing rule hands the Athanor unplanned build work. "Research + implementation combined goes to athanor as a single brief" bypasses the plan and the attack. It also makes the Athanor plan, which athanor.md:51 forbids. Evidence: igor/references/model-tiers.md:19. Fix: route implementation through the cycle; the Athanor gets reshaped build plans, or non-build briefs (research, audits) only.

Minor

  • M1. Trunk notes are mandatory in one file and optional in another. The Athanor must write them, and Igor's fresh-start path depends on them, but the run-log reference calls them optional. Evidence: athanor.md:31, igor/SKILL.md:108 vs run-log.md:61. Fix: drop "(optional)".
  • M2. The alias-over-pin example does not prove what it is cited for. The old Athanor was pinned to claude-opus-4-6, so a 4.6 Athanor is equally explained by the pin, and "either way the alias, not the pin, decided" does not follow. The schema (verified) and the spacecraft 184/184 observation do support the rule. Evidence: spawning-claude-code.md:51, igor/SKILL.md:89. Fix: cite 184/184 as the evidence; mark the dirigent case as consistent with both the alias and the pin. This also leaves open whether I2 bites in practice, and the stamp check fixes it either way.
  • M3. The Athanor pin's context window is unverified. claude-opus-5-5 carries no [1m], while every other full-id pin in Spire does, and the notes assume about 600k is comfortable plus a 100k–250k trunk read. If the bare id means a smaller window, trunk and forks breaks. Evidence: athanor.md:13, models.md:19, :49; the question was already raised in reports/2026-09-14-spire-skill-audit.md:39. Fix: check the window and record the result as a row in the spawning reference's Models table.
  • M4. The stop rule does not say what happens when the re-check still fails. "Then stop and go to the next phase" reads as moving on with an open Critical. Evidence: igor/SKILL.md:101, SKILL.md:54. Fix: an unresolved Critical/Important after the re-check goes to the user, or into a new plan, not to the next phase.
  • M5. A fork "inherits its context" is stated as fact, but the reference marks it assumed. Evidence: SKILL.md:47, athanor.md:32 vs spawning-claude-code.md:59-60. Fix: write "(assumed)" inline.
  • M6. Staging is hard-coded in the Athanor although the goal lists it as a project setting. Evidence: athanor.md:55. Fix: "stage per project config (default: explicit paths)".
  • M7. One durable rule from the old skill was lost: a subagent's "tests pass" is a hypothesis, so read the diff. Only the phase reviewer holds it now, while the Athanor fixes test fallout through children at phase end. Evidence: old SKILL.md:23-33 vs athanor.md:34. Fix: add one line to step 4: check the fallout fix's diff for loosened assertions before claiming green.
  • M8. The run-log template records each block's timing twice and carries a one-off incident as advice. The per-phase table and every handoff note both hold block start and end. Separately, the Haiku line turns a single incident into advice. Evidence: run-log.md:28 and :38; models.md:43. Fix: keep times in the notes only; reword the Haiku line as plain observation. Also trivial: "six stages" now has a 4b (igor/SKILL.md:93).

Flagged items

  1. Adept model: "opus" over the 4.6 pin. Fix now; this is I1. Make the examples follow the rule; the adept pin itself is a separate user decision.
  2. Igor, adept and zathras pinned to 4.6. Split. The cheap guard (Igor checks the stamped id for plan, attack and review) is in scope now; this is I2. Re-pinning Igor, or adding pinned 5.5 role agents, is a follow-up for the user, since the goal does not set Igor's model.
  3. "Verified" means read from the tool schema, not tested at runtime. The labelling is acceptable; it is a follow-up for the preflight, plus one cheap edit now: the legend at spawning-claude-code.md:7 should say "read from the tool schema text" explicitly. Two caveats to record: (a) in this reviewer's session, the schema mentions subagent_type: "fork", but fork is not among the agent types offered, so the fork's availability is not verified, only its documented behaviour; (b) "every agent sees its exact model id" was checked only for general-purpose agents, not for plugin agents with their own bodies.
  4. spire:project template lacks build-setting fields. Follow-up. The skill reads them from spire:project (SKILL.md:79) and they fit under "Verification commands" and "Conventions" today (igor/references/project.md:20-22). Add explicit fields to the template and /spire:init when the next project adopts the lean loop.

Re-check (commit 5909221)

Re-checker: Opus 5.5, model id claude-opus-5-5 (from my system prompt). Fresh agent, scoped to I1–I3 plus a Critical/Important scan of the fix diff; did the work myself, no subagents.

  • I1 — confirmed fixed. The adept row now reads subagent_type: "adept" (model omitted; pinned) at igor/SKILL.md:84, and the three spawn templates at :133, :142, :151 no longer pass model. This matches the rule at :89 and model-tiers.md:36. No model: "opus" remains for adepts anywhere in plugins/spire; the adept pin (agents/adept.md:11, claude-opus-4-6[1m]) now governs.
  • I2 — confirmed fixed (the guard part in scope). igor/SKILL.md:89 now says to check the stamped id of the plan, the attack and the phase review before acting, and to re-dispatch or tell the user if it is not the intended model (Opus 5.5 by default). spawning-claude-code.md:52 marks alias resolution as unknown and points to the same check. Nothing contradicts it: implementation/SKILL.md:51,65 require the stamp, and the planner, attacker and reviewer are general-purpose agents, for which the model id in the prompt is observed (spawning-claude-code.md:53). Pinned 5.5 role agents stay a user follow-up, as the review flagged.
  • I3 — confirmed fixed. model-tiers.md:19 now sends implementation through plan → attack → build, gives the Athanor only reshaped build plans, and allows a single brief only for non-build fan-out. This agrees with athanor.md:8-12 and :51, with model-tiers.md:12, and with igor/SKILL.md:85,87.
  • New Critical/Important in the fix diff: none. The other touched files (athanor.md, implementation/SKILL.md, models.md, run-log.md, spawning-claude-code.md) only took the minor fixes, and none of them contradicts the touched Igor files. Observation, not new and not blocking: the pre-existing "Planned implementation → implementer agent" row (igor/SKILL.md:81, model-tiers.md:11) does not say the plan must already be attacked. Worth one clause the next time those tables are touched.