📝 Add Dirigent process reports as input for the superpowers replacement
Retrospective of the ChatGPT importer run, the lean process report and the lean-finish handoff, copied from dirigent/docs/workpad/reports.
This commit is contained in:
@@ -0,0 +1,93 @@
|
||||
# ChatGPT importer: handoff for the lean finish
|
||||
|
||||
**Date**: 2026-10-15
|
||||
**Written by**: Igor, from journal [2026-10-14-23-chatgpt-importer-continue.md](../journals/2026-10-14-23-chatgpt-importer-continue.md)
|
||||
**Supersedes**: [2026-10-14-chatgpt-importer-handoff.md](2026-10-14-chatgpt-importer-handoff.md) for process and the remaining steps. Keep that file's **rules section** (private data, archive safety, git, worktrees, models); everything else here wins.
|
||||
**Branch**: `main`, not pushed. Phase 8 sits unmerged on its own branch (below).
|
||||
|
||||
## Why this file exists: the lean rules (binding)
|
||||
|
||||
The importer took ~25 h wall clock for what the user expected to take 1–2 h ([retrospective](2026-10-14-chatgpt-importer-retrospective.md)). The user ordered testing and review time cut. The heavy gates were the coordinator's rules, never the user's. **Every brief to every agent from here on carries these rules verbatim:**
|
||||
|
||||
1. **Between blocks:** `cargo check` on the changed crate only. No test suites.
|
||||
2. **Once at the end of a step:** `cargo test` only for crates that step changed. `just check-all` only if UI/web/api changed. Never `dirigent_api` or `dirigent_rest` tests unless those crates changed.
|
||||
3. **No mutation runs**, except for a Critical fix. One test per fix is enough.
|
||||
4. **No real-export runs**, except one smoke at the very end if import/mapping behaviour changed after Phase 6. UI-only changes need none.
|
||||
5. **Run gates once, in the foreground.** No queue of background gates watched one by one.
|
||||
6. **One review total** (below), one fix pass for Critical and Important only, **no re-check**. Minors are logged, never fixed in a loop.
|
||||
7. **No new scope.** No hostile-input audit, no perf/memory work, no extra compatibility checks. Anything the review suggests beyond Critical/Important goes into a list for the user.
|
||||
8. **Docs:** crate `CLAUDE.md` files are 20–50 line navigation maps (`.claude/skills/documentation.md`). API detail goes to trunk notes or `docs/`, never into `CLAUDE.md`.
|
||||
9. **Models:** builders and the reviewer are Opus 5.5 (`athanor` pinned, no `model` override); merge, compile and test runners may be Sonnet or Haiku. Stamp the model id.
|
||||
|
||||
Known failures that don't count against a gate (verified at origin/main `e6aa0532`): `dirigent_archivist` has exactly 13 Windows path/lock failures and `dirigent_api --features server` exactly 9 (see the old handoff for the names). There are also 8 type errors in `dirigent_web` test targets, which `just check-all` doesn't build.
|
||||
|
||||
## Timetable to beat
|
||||
|
||||
From the pause: **1.5–2.5 h total**. Aim to come in under it.
|
||||
|
||||
| Step | Budget |
|
||||
|---|---|
|
||||
| Merge Phase 8 + Phase 7 remainder (gates once, one smoke, UI check, close note) | ≤ 40 min |
|
||||
| One final review | ≤ 25 min |
|
||||
| One fix pass (C/I only) | ≤ 30 min, 0 if nothing C/I |
|
||||
| CLAUDE.md shrink + close | ≤ 10 min |
|
||||
|
||||
If a step runs past its budget, the coordinator stops and asks the user. It does not extend the step on its own.
|
||||
|
||||
## Where things stand (at the pause)
|
||||
|
||||
| Phase | Status |
|
||||
|---|---|
|
||||
| 1–5 | closed |
|
||||
| 6 Assets | **closed**. Part A in the old handoff; part B `3ffa6ebb` (I2 abort), `b3820b3c`, `d560b087`, `fdcc9145`. User approved: no separate re-check; the final review covers it. |
|
||||
| 7 UI | **soft-paused, all 5 blocks built and committed**, nothing uncommitted. `cargo check` is clean on archivist, api, web, protocol and chatgpt. Commits: `4818a7d3`, `74b199bc`, 7C `f5d1a969` `a38f69a1` `addb3f4d` `70e7c57c` (**converter version 4**), 7B1 `655acf10`, 7B2 `c0a10caf`, 7B3 `60ac0c02` `69d10447`, 7A `7279c781` `573922d2`, run log `aae6243e` ("Phase 7 — paused"). Block notes: [phase7-block-notes/](2026-10-15-chatgpt-importer-phase7-block-notes/). |
|
||||
|
||||
**Phase 7 remainder (this is all of it):**
|
||||
1. Phase-end tests, once each, for the crates Phase 7 changed: chatgpt, protocol, archivist (known 13), api `--features server` (known 9), web with and without `server`, rest. Then `just check-all`. **Run these once on `main` after the Phase 8 merge (resume step 3), not before**, so they aren't run twice.
|
||||
2. One real-export smoke run, because 7C changed the mapping (converter 4): v1 alone and the union, with the digest counts. Into a temp archive, counts only. This is the single smoke run that lean rule 4 allows. Do it after the merge too.
|
||||
3. A short manual UI check: `dx serve` on a temp `DIRIGENT_DATA_DIR`, using the click steps in the 7A, 7B2 and 7B3 notes.
|
||||
4. Fold the block notes into the run log as a "Phase 7 close" section. Short.
|
||||
|
||||
Final-reviewer flags from Phase 7:
|
||||
- 7C edits `crates/dirigent_chatgpt/src/citations.rs`, which is in Phase 8's crate, so watch the merge there.
|
||||
- The hub sorts warnings by matching the reader's warning *text*, which Phase 8 may have changed. Only the account hint is structured.
|
||||
- Missing assets reach the web as a separate map. Audio placeholders match by prefix. Library files have no link, because no route serves stored files.
|
||||
- The Dioxus 0.7 `key` remount works only inside an `if`/`for` block, and the archived transcript relies on that.
|
||||
- Five crate `CLAUDE.md` files gained API detail before the stop order. All five get the close-step rewrite.
|
||||
| 8 Integration tests | **built, not merged.** Worktree `.claude/worktrees/agent-a9adadbc70966bb15`. The branch was reported as `chatgpt-importer-phase8` and also as `worktree-agent-a9adadbc70966bb15`; run `git branch --contains 6efcc533` to find which. Logs in that worktree: `docs/workpad/reports/2026-10-14-chatgpt-importer-build-run-phase8.md` and `...-trunk-notes-phase8.md`. The user chose to **keep** its perf and reader-fix commits (`8fe18a8f`, `57728bbd`, `02cc5488`, `a4de6d0b`, `6cddcaef`, `fc0cf029`, `175f5ef6`, `5a14c616`). Tests: `95817c6c`, `763bbbd1`, `6efcc533`; cancel fix `db4c8c37`. Gates on the branch: chatgpt 153/0, archivist 975 plus the known 13. |
|
||||
|
||||
Decisions since the old handoff (all from the user):
|
||||
- Limits 1–4 approved, then amended:
|
||||
- Phase 8 was cut to integration tests; the hostile-input audit and perf work were dropped (the commits that already existed are kept).
|
||||
- The separate Phase 7 review was replaced by **one final review**.
|
||||
- Phase 8 merges into `main` after Phase 7.
|
||||
|
||||
## Resume steps, in order
|
||||
|
||||
1. **Check state:**
|
||||
- `git log --oneline fdcc9145..HEAD` on `main`;
|
||||
- the run log's "Phase 7 — paused" section;
|
||||
- `git status --short` for any uncommitted Phase 7 files.
|
||||
|
||||
Re-read git; don't trust the state of agents that no longer exist.
|
||||
2. **No more Phase 7 building.** Every block is committed. The remainder above is verification, and it runs after the merge.
|
||||
3. **Merge Phase 8 into `main`, then do the Phase 7 remainder items 1–4 in a single pass.** Use a Sonnet agent and an ordinary merge commit (no rebase or history rewrite).
|
||||
- Expected conflicts: `crates/dirigent_archivist/src/import/mod.rs` (the post-loop cancel block), `backends/jsonl/mapping.rs` and `backend.rs`, the `crates/dirigent_chatgpt/src/` reader files, and both crates' `CLAUDE.md`.
|
||||
- New public items from Phase 8: `ReaderWarning::Omitted` and `Limits::max_read_cache`. Check for exhaustive matches or struct literals that Phase 7 added.
|
||||
- Gates, once: chatgpt and archivist tests, plus `just check-all`.
|
||||
- Then remove the Phase 8 worktree and its target dir.
|
||||
4. **One final review.** A fresh Opus 5.5 reviewer covers the Phase 6 fix pass (I1, I2), all of Phase 7 and the Phase 8 merge. It answers the handoff flags in the run logs, reports Critical/Important/Minor and writes `docs/workpad/reviews/2026-10-15-chatgpt-importer-final-review.md`. It doesn't scope to hostile input.
|
||||
5. **One fix pass**, for Critical and Important only, with lean gates. No re-check.
|
||||
6. **Close:**
|
||||
- Rewrite `crates/dirigent_chatgpt/CLAUDE.md` (currently a ~61-line API reference) into a 20–50 line navigation map, and move the API reference to `docs/architecture/chatgpt_importer.md`. Do the same for the other four crate `CLAUDE.md` files that Phase 7 extended: archivist, api, web and protocol (check `git log -p` from `e6aa0532`).
|
||||
- Then the user reviews and pushes. Never push for them.
|
||||
|
||||
## Carry-overs for later (not this run)
|
||||
|
||||
- Phase 8 reviewer flags:
|
||||
- the read cache now keeps all shards (~80 MB on the real export);
|
||||
- the mapping-table cache misses a same-length rewrite within one mtime tick;
|
||||
- a single-export re-import into a v0+v1 union re-converts shared conversations;
|
||||
- an archived-tier digest mismatch is counted in no notice.
|
||||
- Pre-existing failures (13 archivist, 9 api, 8 dirigent_web test type errors) are worth fixing separately.
|
||||
- Skill-level lesson for spire: the full cycle (design → plan → attack → per-phase review/fix/re-check) is too heavy for a build of this size. See the retrospective's budget table.
|
||||
Reference in New Issue
Block a user