📝 Add orbit research tree, goal, and session journal
Research tree on using the orbit code-graph CLI for review in Rust projects: the Python/Django field guide recorded as phase 0, orbit 0.133 capabilities on spacecraft and dirigent, fit with Spire's review stage, and tested review recipes (changed definitions, blast radius, blindness preflight). Includes the orbit-skill goal and the Igor session journal.
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
# Goal: an orbit skill, and knowing how to use orbit well
|
||||
|
||||
Captured by Igor on the user's behalf, 2026-09-29 (session journal: `journals/2026-09-29-17-orbit.md`).
|
||||
|
||||
## Objective
|
||||
|
||||
Turn hands-on orbit experience into a reusable skill, so agents in any project use the `orbit` code-graph CLI where it helps (structure questions) and fall back to grep where it doesn't. Along the way, learn how orbit can be used better than the first field guide managed.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- A research tree under `research/2026-09-29-17-orbit/` exists, starting from the recorded Python/Django field guide (phase 0), and its claims have been checked against the currently installed orbit version.
|
||||
- A skill exists that tells an agent when to reach for orbit versus grep, how to keep the index fresh and avoid the known pitfalls, and gives it a cookbook of queries that have actually been tested.
|
||||
- The skill does not depend on a single language: it covers Python at least and says what is known or unknown for other languages.
|
||||
- Guidance for briefing subagents (for example Spire's search, scout and researcher agents) to use orbit is included or linked.
|
||||
|
||||
## Open
|
||||
|
||||
- Where the skill lives: a standalone plugin, or part of Spire.
|
||||
- Which repositories to use as test beds for re-verification.
|
||||
|
||||
## Refinement 2026-09-29
|
||||
|
||||
- The prototype is enabled locally on spacecraft and dirigent. It records feedback about its own usage (queries, results, verdicts) so findings can be mined later. Compiling these projects is expensive, so anything that needs a build is learned from real usage.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Orbit research and skill — Igor Session Journal
|
||||
|
||||
**Date**: 2026-09-29
|
||||
**Topic**: orbit (code-graph CLI): research tree, toward a skill
|
||||
**Worktree/branch**: main (markdown only)
|
||||
|
||||
## Log
|
||||
|
||||
<!-- Append-only. This journal is the testimony of one path through the problem: record the tangents and dead ends, not only the results. The path can't be walked again; the record can be read. -->
|
||||
|
||||
- 17:00 — The user provided `local/orbit_guide.md`, a field guide on orbit from a Python/Django project, and wants a small research tree plus, eventually, a skill.
|
||||
- 17:00 — `orbit` 0.133.0 is installed at `~/.local/bin/orbit`; the guide assessed 0.124.0.
|
||||
- 17:00 — Recorded the guide as `research/2026-09-29-17-orbit/phase-0-field-guide-python-django.md` (with a provenance header; the user's intro note dropped). Wrote `goals/orbit-skill.md` and a draft tree plan `research/2026-09-29-17-orbit/plan.md`. Waiting for the user to confirm the plan, the test-bed repositories and where the skill should live.
|
||||
- 17:05 — Direction from the user: the Python guide is baseline guidance for an early prototype skill. The target is Rust Spire projects, with review as the first use case. Plan revised.
|
||||
- 17:08 — Dispatched the phase-1 rust-capabilities probe on dirigent (Opus). The user stopped it: I had misread the direction. Correction: the only off-limits repo is the Python one the guide came from. Test beds are spacecraft and dirigent, and any `~/Projects` repo is allowed. Plan updated. Waiting for the go to relaunch.
|
||||
- 17:12 — The user gave the go. Relaunched phase 1 in parallel: rust-capabilities (Opus; spacecraft and dirigent, read-only, scratch in ~/.cache/orbit-probe/) → `phase-1-rust-capabilities.md`; review-fit (Sonnet; Spire review stage, skill placement) → `phase-1-review-fit.md`.
|
||||
- 17:15 — review-fit came back → `phase-1-review-fit.md`. There is no pinned reviewer agent (stage 6 is an ad-hoc Opus reviewer). `search` is the best fit to run orbit. Recommended placement: a standalone `plugins/orbit/`, following the tome precedent. `project.md` can flag orbit availability through its existing domain-skills slot. Waiting for rust-capabilities.
|
||||
- 17:17 — The user notes that orbit had already indexed the projects beforehand. Told rust-capabilities that the existing `orbit list` entries are the user's and must not be removed, and asked it to compare each indexed commit with HEAD (a staleness check).
|
||||
- 17:48 — rust-capabilities came back → `phase-1-rust-capabilities.md`.
|
||||
- **CALLS:** 85–100% coverage for concrete calls; 0–14% for `dyn Trait` calls.
|
||||
- **Missed code:** calls inside macro arguments, bevy systems registered via `add_systems`, and Dioxus components used in `rsx!`.
|
||||
- **Blind spots:** all Cargo features treated as off (defaults too) and `#[cfg(test)]` skipped; 176 of 989 dirigent files have no definitions, including the whole server backend.
|
||||
- **Timeouts:** under default threads some large files time out and are skipped nondeterministically; use `-t 4` and check `skipped_files`.
|
||||
- **Traits:** impls are only linked when in the same file as the type.
|
||||
- **MCP:** still three tools, and `run_sql` is unscoped.
|
||||
- **Staleness:** a stale commit gives little or no warning.
|
||||
- **Branch comparison works:** a detached worktree plus one SQL query. `--db` gives an isolated DB, and orbit has no delete command.
|
||||
- **Leftover:** a spacecraft worktree entry `bfbbcd96` remains in the shared DB `~/.gitlab/orbit/graph.duckdb`.
|
||||
- **Re-indexed:** spacecraft and dirigent, which were stale. Orbit writes nothing into repos. Scripts are in `~/.cache/orbit-probe/`.
|
||||
- 17:55 — User direction: the research must not compile anything. Worktrees are fine. The skill should record feedback data and be enabled locally on spacecraft and dirigent, with findings mined later. Accreted this to the plan and the goal. Dispatching phase 2 review-workflows (Opus). The leftover DB entry is still waiting on the user.
|
||||
- 17:58 — User decision: remove leftovers at the end of the session. That covers the `bfbbcd96` entry in the shared DB, `~/.cache/orbit-probe/` (worktrees, isolated DBs, scratch), and `local/orbit_guide.md` once the user has confirmed.
|
||||
- 18:00 — User decision: keep `local/orbit_guide.md` as an artifact for now. It is not part of the cleanup.
|
||||
- 18:10 — The user flagged scope creep ("a forever task"). Narrowed phase 2 mid-run: one repo, recipes 1, 2 and 5 tested, the rest as untested drafts, feedback format included. Phase 3 synthesis dropped: next step is drafting the prototype skill directly.
|
||||
- 19:35 — Phase 2 came back after about 96 minutes → `phase-2-review-workflows.md`. The scope cut arrived too late: the report covers both repos, 7 ranges and all 6 recipes. Lesson: send a scope cut early or not at all; this brief was oversized for a prototype.
|
||||
- **Changed definitions:** exact in the code orbit can see.
|
||||
- **Preflight:** 6–71% of changed lines are invisible to orbit, so the preflight is mandatory.
|
||||
- **Blast radius:** 49% of dependents per definition, 93% per file.
|
||||
- **Feedback log:** JSONL at `~/.local/share/orbit-skill/feedback.jsonl`. Scripts are in `~/.cache/orbit-probe/recipes/`. Worktrees and isolated DBs are cleaned up; the shared DB is unchanged, so `bfbbcd96` is still there.
|
||||
- **Corrects phase 1:** orbit indexes the working tree, including uncommitted and untracked files, but labels the index with HEAD's commit.
|
||||
- 19:36 — Dispatched the prototype plugin build (Opus): `plugins/orbit/`, with the recipes carried into the plugin. No design or plan stage: this is a prototype, and the user asked for speed.
|
||||
- 19:45 — Prototype plugin built in about 8.5 minutes → `plugins/orbit/` (skill, references, scripts, `fb.sh` feedback helper, `freshness.sh`), and registered in `marketplace.json`. It is uncommitted.
|
||||
- **Tested:** the review pass ran end to end on dirigent `cd4e8f70` in a scratch clone and isolated DB (23 s, 8 KB of output, results match phase 2). `claude plugin validate` passes. The shared DB is unchanged.
|
||||
- **Enabling:** `claude plugin install orbit@reliquary --scope local` (after a push), or `--plugin-dir` before that.
|
||||
- **Rough edges:** untracked files are not reviewed, the preflight thresholds are guesses, `${CLAUDE_PLUGIN_ROOT}` expansion in SKILL.md is unverified, and the boundary rules may drift.
|
||||
- 19:50 — The user said commit, push and wrap up.
|
||||
- **Leftover DB entry:** removed `bfbbcd96` from the shared orbit DB. Orbit has no delete command, so I backed up the DB, then deleted that project's rows from the four gl_* tables, its edges (0 ID collisions with other projects), its gl_def_doc table, its FTS schema and its manifest row. I checked the copy with `orbit list`, `sql` and `grep` and swapped it in only after confirming by sha256 that the original was unchanged.
|
||||
- **Concurrent re-index:** another session re-indexed spacecraft (`82c45927`) right after the swap. The DB stays consistent: 3 entries, and spacecraft matches HEAD.
|
||||
- **Scratch:** deleted `~/.cache/orbit-probe/` and the backup. No probe worktrees remain in either repo.
|
||||
- Kept `local/orbit_guide.md` (uncommitted, as before).
|
||||
- 19:52 — Wrap-up. There are no adepts to close. Committing the research, goal, journal and `plugins/orbit/`, then pushing. The user's other uncommitted changes are left alone.
|
||||
|
||||
## Adepts
|
||||
|
||||
- Roster empty at boot. An `orbit` adept may earn its place if consultation recurs.
|
||||
- None spawned this session.
|
||||
|
||||
## Artifacts
|
||||
|
||||
- `research/2026-09-29-17-orbit/phase-0-field-guide-python-django.md`
|
||||
- `research/2026-09-29-17-orbit/plan.md`
|
||||
- `goals/orbit-skill.md`
|
||||
- `research/2026-09-29-17-orbit/phase-1-review-fit.md`
|
||||
- `research/2026-09-29-17-orbit/phase-1-rust-capabilities.md`
|
||||
- `research/2026-09-29-17-orbit/phase-2-review-workflows.md`
|
||||
- `plugins/orbit/` (prototype plugin, uncommitted)
|
||||
|
||||
## Open threads
|
||||
|
||||
- Enable the plugin locally on spacecraft and dirigent (`claude plugin install orbit@reliquary --scope local`, now that the plugin is pushed).
|
||||
- Mine `~/.local/share/orbit-skill/feedback.jsonl` after some real reviews. The phase-2 report lists the 11 questions it should answer.
|
||||
- Rough edges in the prototype: untracked files are not reviewed, the preflight thresholds are guesses, `${CLAUDE_PLUGIN_ROOT}` expansion in SKILL.md is unverified, the boundary rules are hand-derived, and worktrees linger if teardown is skipped.
|
||||
- Optional later: wire orbit into Spire (a `project.md` flag through the domain-skills slot, and briefs for the search agent). A dedicated reviewer agent for stage 6 does not exist yet.
|
||||
- `goals/orbit-skill.md` still lists "Open" items that are now answered (standalone plugin; spacecraft and dirigent). Leave them for the curator or the user to settle.
|
||||
|
||||
## Graduation candidates
|
||||
|
||||
- **Orbit's blind regions in Rust** (cfg(test), all Cargo features treated as off, macro, `dyn`, `add_systems` and `rsx!` gaps; impls linked only when in the same file as the type). Anyone using orbit on Rust would otherwise re-derive this at great cost. It already lives in the skill; the research reports hold the evidence.
|
||||
- **Orbit labels a working-tree index with HEAD's commit.** This is non-obvious, it corrected phase 1, and it affects every "is my index fresh" judgement.
|
||||
- **Removing an entry from orbit's DuckDB by hand** (manifest row, gl_* rows by `project_id`, edges by ID, `gl_def_doc_<pid>` table, FTS schema). Orbit has no delete command, and this took a schema dig to work out.
|
||||
|
||||
## Self-review
|
||||
|
||||
- Over-scoped. The phase-2 brief (6 recipes × 2 repos × 7 ranges) was sized for a finished skill, not a prototype, and ran about 96 minutes. The user had to flag it as a "forever task". The cut came too late to matter: scope has to be sized at dispatch, not trimmed mid-run.
|
||||
- Misread the user's direction twice at the start: first "no Python repo" as "no Python at all", then the test-bed question. I relaunched an agent that could have been redirected if I had asked one sharper question first.
|
||||
- Delegation worked where it mattered: the research and the plugin build went to agents that wrote their own files. The prototype build, briefed tightly with a timebox, took about 8 minutes.
|
||||
- I did the DB cleanup, the commit and the journal myself rather than delegating. That was defensible for a careful, stateful operation on the user's shared DB, but strictly off-mindset.
|
||||
- The race on the shared DB (another session indexing during the swap) was caught only by checking after the swap. Future DB surgery should take a lock or ask other sessions to pause.
|
||||
@@ -0,0 +1,545 @@
|
||||
---
|
||||
source: field guide from another project (Python/Django monolith), recorded 2026-09-29 from local/orbit_guide.md
|
||||
orbit-version-assessed: 0.124.0 (installed locally now: 0.133.0)
|
||||
status: phase-0 input — prior art, not re-verified
|
||||
---
|
||||
|
||||
# Orbit Field Guide: Using a Local Code Graph on a Large Django Monolith
|
||||
|
||||
A consolidated guide to the `orbit` CLI, compiled from two hands-on investigations on the `acme-platform` repository: an **architecture survey** (planning a shared service layer across API surfaces) and an **OpenAPI schema-warning cleanup** (triaging ~200 drf-spectacular warnings). It merges the command reference, the gotchas, the query cookbook, both case studies, and the corrections found when orbit's claims were fact-checked afterwards.
|
||||
|
||||
Environment-specific details (machine paths, local task runners, ticket and PR numbers, branch names) are left out. Project names are anonymised: the company is "Acme", the repository is `acme-platform`, and the Python package root is `acme/`.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
[TL;DR](#1-tldr)
|
||||
[What orbit is](#2-what-orbit-is)
|
||||
[Index lifecycle](#3-index-lifecycle)
|
||||
[Command reference](#4-command-reference)
|
||||
[Graph schema](#5-graph-schema)
|
||||
[MCP server](#6-mcp-server)
|
||||
[Gotchas](#7-gotchas)
|
||||
[Orbit or grep: decision table](#8-orbit-or-grep-decision-table)
|
||||
[Query cookbook](#9-query-cookbook)
|
||||
[Case study A: architecture survey for a service layer](#10-case-study-a-architecture-survey-for-a-service-layer)
|
||||
[Case study B: OpenAPI schema-warning cleanup](#11-case-study-b-openapi-schema-warning-cleanup)
|
||||
[Reusable workflows](#12-reusable-workflows)
|
||||
[Guidance for agents and a future orbit skill](#13-guidance-for-agents-and-a-future-orbit-skill)
|
||||
[Open questions](#14-open-questions)
|
||||
---
|
||||
|
||||
## 1. TL;DR
|
||||
|
||||
- **Orbit is not a better grep.** Its value is answering **structure** questions that text search cannot answer reliably: "who inherits from X", "what is the shape of class X", "which modules import from which", "what does this module depend on outside itself".
|
||||
- **Best at:** definition inventories, inheritance trees (`repo-map extends`), import maps between surfaces, compact API summaries you can paste into LLM context (`repo-map api`, `repo-map class`), and arbitrary SQL joins over definitions and edges.
|
||||
- **Weak at:** dynamic dispatch. Django manager and queryset calls such as `qs.visible_to(request)` produce almost no `CALLS` edges. It also cannot follow inheritance through third-party base classes (DRF's `APIView`). Its `grep` is ranked, not exhaustive or exact.
|
||||
- **Workflow order:** get the real targets and counts from a live tool run or plain grep first, then use orbit to find the structural relationships between them, then verify the prediction by running the tool again.
|
||||
- **Hygiene:** re-index before you query (a stale index returns wrong-commit results without warning). Leave `orbit sql` at its default scope; `--all` multiplies rows by the number of indexed worktrees.
|
||||
- **Orbit predicts structure, not runtime behaviour.** In case study B, one of two base-class fixes that orbit made look obvious had no effect, because the framework already handled the failure. Always confirm with the real tool.
|
||||
|
||||
---
|
||||
|
||||
## 2. What orbit is
|
||||
|
||||
Orbit is a CLI that indexes Git repositories into a **DuckDB graph** of files, directories, definitions (classes, functions, methods, fields…), imports and cross-references. You can query it in three ways:
|
||||
|
||||
- **SQL**: read-only queries against the graph tables.
|
||||
- **Structural subcommands**: `repo-map`, `context`, `grep`.
|
||||
- **MCP server** (`orbit mcp serve`) for AI agents. It exposes a subset (see §6).
|
||||
|
||||
The version assessed was **0.124.0**. The graph lives in a single DuckDB file in the user's home directory. One graph can hold several repositories and several commits (branches and worktrees) of the same repository at once.
|
||||
|
||||
---
|
||||
|
||||
## 3. Index lifecycle
|
||||
|
||||
| Action | Command | Notes |
|
||||
|---|---|---|
|
||||
| Index / re-index current checkout | `orbit index .` | About 2 seconds for the whole `acme-platform` monolith. |
|
||||
| See what is indexed | `orbit list` | Shows each repo with branch, commit, status and last-indexed time. |
|
||||
| Inspect tables | `orbit schema` | Run before writing SQL. |
|
||||
|
||||
Rules:
|
||||
|
||||
- **Re-index after every branch switch, pull or commit you care about.** Orbit does not warn when the index is stale. It returns results for whatever commit was indexed last.
|
||||
- **Every worktree is indexed as a separate entry.** This is useful for comparisons but causes the duplication pitfall (§7.1).
|
||||
|
||||
---
|
||||
|
||||
## 4. Command reference
|
||||
|
||||
### `orbit grep [QUERY...] [--path PATH] [--kind KINDS] [--limit N]`
|
||||
|
||||
Searches definition **names, paths and bodies**. It returns ranked `Definition:<id>` references with source for the top matches.
|
||||
|
||||
- `--path acme/webapi/public/alerts/` restricts the search to a directory.
|
||||
- `--kind Class,Method` restricts it to definition types.
|
||||
- `--limit 20` sets the number of results (default 10).
|
||||
- You can pass several queries in one call.
|
||||
|
||||
**Strength:** it finds usages inside function bodies that the structural graph misses, such as a queryset method called on a dynamic receiver.
|
||||
**Limitation:** it is ranked text matching, not type-aware and not exact. It also matches comments and strings, and it can rank near-misses (`*_extended_*`, `*_intervals*`) above the exact identifier you want. Do not use it to count.
|
||||
|
||||
### `orbit context <TARGET...> [--tests]`
|
||||
|
||||
Reads definitions by ID, or reads a file.
|
||||
|
||||
- Pass a `Definition:<id>` from `orbit grep` output to get the source plus its relationships.
|
||||
- Pass a file path (`orbit context acme/webapi/common/conventions/tenant_scoping.py`) to get the source with definition boundaries marked.
|
||||
- `--tests` also shows test connections that are otherwise hidden.
|
||||
|
||||
### `orbit sql [QUERY] [-f FILE] [-F FORMAT] [--all]`
|
||||
|
||||
Runs read-only SQL against the graph.
|
||||
|
||||
- **Default scope: the current checkout's indexed commit.** You do not need a branch or project filter.
|
||||
- `--all` queries every indexed repository and commit. Results multiply by the number of indexed worktrees.
|
||||
- `-F json|ndjson|csv|table` sets the output format (default `table`).
|
||||
- `-f query.sql` reads the SQL from a file.
|
||||
|
||||
### `orbit repo-map [SUBCOMMAND] [--ext EXT]`
|
||||
|
||||
Compact structural summaries designed for LLM context. It is always scoped to the current commit.
|
||||
|
||||
| Subcommand | Purpose | Example |
|
||||
|---|---|---|
|
||||
| `overview` | Languages, top directories, key types (default) | `orbit repo-map` |
|
||||
| `tree [PATH]` | Definitions grouped by file | `orbit repo-map tree acme/webapi/public/alerts/` |
|
||||
| `api [PATH]` | Types + callables + signatures | `orbit repo-map api acme/webapi/public/alerts/` |
|
||||
| `class NAME` | Members + signatures of one class (all classes with that name) | `orbit repo-map class AlertRepository` |
|
||||
| `extends NAME` | Descendant tree of a base class, with depth | `orbit repo-map extends TenantScopeMixin` |
|
||||
| `imports PATTERN` | Files that import a symbol or path | `orbit repo-map imports ActorScope` |
|
||||
|
||||
### `orbit schema`, `orbit index [PATH]`, `orbit list`
|
||||
|
||||
See §3.
|
||||
|
||||
---
|
||||
|
||||
## 5. Graph schema
|
||||
|
||||
### `gl_definition`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | BIGINT | Primary key |
|
||||
| `file_path` | VARCHAR | Repo-relative path |
|
||||
| `fqn` | VARCHAR | Fully qualified name |
|
||||
| `name` | VARCHAR | Short name |
|
||||
| `definition_type` | VARCHAR | See below |
|
||||
| `start_line`, `end_line` | BIGINT | Line range (`end_line - start_line + 1` = size) |
|
||||
|
||||
Python definition types: `Class`, `DataClass`, `DecoratedClass`, `Function`, `DecoratedFunction`, `AsyncFunction`, `DecoratedAsyncFunction`, `Method`, `DecoratedMethod`, `DecoratedAsyncMethod`, `StaticMethod`, `Field`, `Variable`, `TypeAlias`, `Module`, `Lambda`.
|
||||
|
||||
When you look for classes, always include `DecoratedClass` next to `Class`. The same applies to functions and methods.
|
||||
|
||||
### `gl_file`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | BIGINT | Primary key |
|
||||
| `path` | VARCHAR | Repo-relative path |
|
||||
| `extension` | VARCHAR | `.py`, `.tsx`, … |
|
||||
| `language` | VARCHAR | |
|
||||
| `size_bytes` | BIGINT | |
|
||||
|
||||
### `gl_edge`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `source_id` | BIGINT | FK to a definition or file |
|
||||
| `source_kind` | VARCHAR | `Definition` or `File` |
|
||||
| `relationship_kind` | VARCHAR | `CALLS`, `CONTAINS`, `DEFINES`, `EXTENDS`, `IMPORTS` |
|
||||
| `target_id` | BIGINT | FK to a definition or file |
|
||||
| `target_kind` | VARCHAR | `Definition` or `File` |
|
||||
|
||||
### `gl_imported_symbol`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `file_path` | VARCHAR | Importing file |
|
||||
| `import_type` | VARCHAR | `FromImport`, `Import`, `AliasedImport`, `WildcardImport`, `FutureImport` |
|
||||
| `import_path` | VARCHAR | Module path (`acme.core.models`) |
|
||||
| `identifier_name` | VARCHAR | Imported name |
|
||||
| `identifier_alias` | VARCHAR | Alias, if `as` was used |
|
||||
|
||||
### `gl_directory`
|
||||
|
||||
Directories with path and name. Use it for structural roll-ups.
|
||||
|
||||
Some tables also carry commit/branch columns, which is how the default scoping works. Check `orbit schema` before you filter on them by hand.
|
||||
|
||||
---
|
||||
|
||||
## 6. MCP server
|
||||
|
||||
Orbit can run as an MCP server (`orbit mcp serve`) and be registered in an agent's MCP configuration. It exposes **only three tools**:
|
||||
|
||||
- `run_sql`: same as `orbit sql`
|
||||
- `get_graph_schema`: same as `orbit schema`
|
||||
- `index`: same as `orbit index`
|
||||
|
||||
It does **not** expose `grep`, `context` or `repo-map`. Those are the most useful commands for structure questions, so an agent limited to MCP can only write SQL.
|
||||
|
||||
In practice the server was configured but its tools did not load into the agent session. The working setup was **the CLI through the shell**. If you brief subagents, tell them to call `orbit` from bash rather than to look for MCP tools.
|
||||
|
||||
---
|
||||
|
||||
## 7. Gotchas
|
||||
|
||||
### 7.1 Worktree duplication (the biggest pitfall)
|
||||
|
||||
Each indexed worktree is a separate entry. Queries with `orbit sql --all`, or SQL that bypasses the default scope, return **one row per indexed worktree**.
|
||||
|
||||
- A verified example: `AlertRepository` returned 2 rows with default scoping and 6 with `--all` (2 classes × 3 worktrees).
|
||||
- **This produced a real false finding.** The architecture survey first reported "three `AlertRepository` classes". A plain-grep fact check found **two**. The extra one came from unscoped results.
|
||||
|
||||
**Fix:** use default `orbit sql` (no `--all`). `grep`, `repo-map` and `context` are always scoped correctly. If you really need `--all` for a cross-worktree comparison, group or filter by the commit/branch column.
|
||||
|
||||
### 7.2 Dynamic method calls are invisible to `CALLS` edges
|
||||
|
||||
Calls on receivers whose type the static analyser cannot resolve, such as Django manager and queryset chains (`Model.objects.visible_to(request)`, `self.manager.filter(...)`), produce **no `CALLS` edge**.
|
||||
|
||||
- Impact: `visible_to` has about **120** real call sites in `acme/webapi/`, but orbit had only **4** `CALLS` edges to it.
|
||||
- Self-calls inside one class hierarchy (`self.get_alert()`) resolve correctly. Cross-class dynamic dispatch does not. Coverage is therefore inconsistent: a "who calls X" query can look complete when it is not.
|
||||
|
||||
**Workaround:** `orbit grep "visible_to" --path acme/webapi/` finds definitions whose body contains the string. For an exhaustive count, use plain `grep -rn`.
|
||||
|
||||
### 7.3 `repo-map imports` only follows import statements
|
||||
|
||||
`orbit repo-map imports visible_to` returns nothing, because a method accessed as an attribute is never imported. Only `from X import Y` and `import X` are tracked. The same applies to `gl_imported_symbol`.
|
||||
|
||||
### 7.4 Third-party base classes are not graph nodes
|
||||
|
||||
`orbit repo-map extends APIView` returned **nothing**. Inheritance is resolved only through in-repo base classes. To enumerate subclasses of framework classes (DRF `APIView`, Django `Model`, …), fall back to `grep -n "^class .*APIView"`.
|
||||
|
||||
### 7.5 `orbit grep` is ranked, not exact
|
||||
|
||||
A search for a specific property name returned unrelated partial matches and **missed the actual definition**. For "where is `def exact_name` defined", use `grep -rn "def exact_name"`.
|
||||
|
||||
### 7.6 No type, annotation or decorator semantics
|
||||
|
||||
Orbit tracks structure (definitions, containment, inheritance, imports, some calls). It does not model field types, type annotations, return types, or what a decorator does. A query like "all `SerializerMethodField` getters without a return annotation" cannot be expressed. Get that from the tool that reports the problem, then map names back to classes.
|
||||
|
||||
### 7.7 Stale index
|
||||
|
||||
After a branch change, pull or commit, the graph still shows the old commit until you run `orbit index .`. Nothing warns you. Re-index at the start of every session and after each commit you want to query.
|
||||
|
||||
### 7.8 Ticket numbers vs. the code
|
||||
|
||||
The service-layer ticket claimed "~199 `visible_to` call sites". The real number was ~120. That was not orbit's fault, but it shows why estimates from tickets should be recounted from the code or a live tool run before you plan work around them.
|
||||
|
||||
---
|
||||
|
||||
## 8. Orbit or grep: decision table
|
||||
|
||||
| Question | Tool |
|
||||
|---|---|
|
||||
| Find all classes named / matching X | `orbit sql` or `orbit grep --kind Class X` |
|
||||
| What does class X look like? | `orbit repo-map class X` |
|
||||
| Are two same-named classes the same thing? | `orbit repo-map class X` (lists every match with file) |
|
||||
| Who inherits from in-repo class X? | `orbit repo-map extends X` |
|
||||
| Who inherits from a framework class? | `grep -n "^class .*Base"` |
|
||||
| Who imports X? | `orbit repo-map imports X` |
|
||||
| API surface of a directory | `orbit repo-map api path/` |
|
||||
| How do packages / surfaces depend on each other? | `orbit sql` on `gl_imported_symbol` |
|
||||
| External dependencies of a module | `orbit sql` with `gl_edge` `CALLS` joins |
|
||||
| All call sites of a dynamic method | `orbit grep "name"` to explore, `grep -rn` to count |
|
||||
| Count occurrences of a string | `grep -rn` (orbit grep is ranked) |
|
||||
| Where is `def exact_name`? | `grep -rn "def exact_name"` |
|
||||
| Does file X exist? | `ls` / `find` |
|
||||
| What changed recently? | `git log` / `git diff` (orbit indexes snapshots, not history) |
|
||||
|
||||
---
|
||||
|
||||
## 9. Query cookbook
|
||||
|
||||
All SQL below uses the default scope (current checkout). Do not add `--all`.
|
||||
|
||||
### 9.1 Size API surfaces by definitions and files
|
||||
SELECT
|
||||
CASE
|
||||
WHEN file_path LIKE 'acme/webapi/ui/%' THEN 'ui'
|
||||
WHEN file_path LIKE 'acme/webapi/public/%' THEN 'public'
|
||||
WHEN file_path LIKE 'acme/webapi/frontend/%' THEN 'frontend'
|
||||
WHEN file_path LIKE 'acme/webapi/endpoints/%' THEN 'endpoints'
|
||||
WHEN file_path LIKE 'acme/webapi/common/%' THEN 'common'
|
||||
ELSE 'other-webapi'
|
||||
END AS surface,
|
||||
COUNT(DISTINCT file_path) AS files,
|
||||
COUNT(*) AS definitions
|
||||
FROM gl_definition
|
||||
WHERE file_path LIKE 'acme/webapi/%'
|
||||
GROUP BY surface
|
||||
ORDER BY definitions DESC;
|
||||
### 9.2 Inventory classes matching a pattern, with sizes
|
||||
SELECT name, file_path, (end_line - start_line + 1) AS lines
|
||||
FROM gl_definition
|
||||
WHERE name LIKE '%Repository%'
|
||||
AND definition_type IN ('Class', 'DecoratedClass')
|
||||
AND file_path LIKE 'acme/%'
|
||||
ORDER BY file_path;
|
||||
### 9.3 Import matrix between surfaces
|
||||
SELECT
|
||||
CASE WHEN file_path LIKE 'acme/webapi/ui/%' THEN 'ui'
|
||||
WHEN file_path LIKE 'acme/webapi/public/%' THEN 'public'
|
||||
ELSE 'other' END AS source,
|
||||
CASE WHEN import_path LIKE 'acme.webapi.ui%' THEN 'ui'
|
||||
WHEN import_path LIKE 'acme.webapi.public%' THEN 'public'
|
||||
WHEN import_path LIKE 'acme.webapi.common%' THEN 'common'
|
||||
ELSE 'other' END AS target,
|
||||
COUNT(*) AS imports
|
||||
FROM gl_imported_symbol
|
||||
WHERE file_path LIKE 'acme/webapi/%'
|
||||
AND import_path LIKE 'acme.webapi%'
|
||||
GROUP BY source, target
|
||||
ORDER BY imports DESC;
|
||||
### 9.4 Isolation guard: should stay at 0
|
||||
|
||||
Run it before and after a refactor to confirm that one surface still does not import from another.
|
||||
SELECT COUNT(*)
|
||||
FROM gl_imported_symbol
|
||||
WHERE file_path LIKE 'acme/webapi/public/%'
|
||||
AND import_path LIKE 'acme.webapi.ui%';
|
||||
### 9.5 External dependencies of a module (outgoing `CALLS`)
|
||||
SELECT DISTINCT tgt.file_path, tgt.name, tgt.definition_type
|
||||
FROM gl_edge e
|
||||
JOIN gl_definition src ON e.source_id = src.id AND e.source_kind = 'Definition'
|
||||
JOIN gl_definition tgt ON e.target_id = tgt.id AND e.target_kind = 'Definition'
|
||||
WHERE e.relationship_kind = 'CALLS'
|
||||
AND src.file_path LIKE 'acme/webapi/public/alerts/%'
|
||||
AND tgt.file_path NOT LIKE 'acme/webapi/public/alerts/%'
|
||||
ORDER BY tgt.file_path;
|
||||
Remember §7.2: this is a lower bound, because dynamic calls are missing.
|
||||
|
||||
### 9.6 Shared domain dependencies of two surfaces
|
||||
|
||||
This finds the domain types both surfaces call. They are candidates for a shared layer.
|
||||
SELECT a.dependency_name, a.dependency_file
|
||||
FROM (
|
||||
SELECT DISTINCT tgt.name AS dependency_name, tgt.file_path AS dependency_file
|
||||
FROM gl_edge e
|
||||
JOIN gl_definition src ON e.source_id = src.id AND e.source_kind = 'Definition'
|
||||
JOIN gl_definition tgt ON e.target_id = tgt.id AND e.target_kind = 'Definition'
|
||||
WHERE e.relationship_kind = 'CALLS'
|
||||
AND src.file_path LIKE 'acme/webapi/public/<resource>/%'
|
||||
AND tgt.file_path NOT LIKE 'acme/webapi/%'
|
||||
) a
|
||||
INNER JOIN (
|
||||
SELECT DISTINCT tgt.name AS dependency_name, tgt.file_path AS dependency_file
|
||||
FROM gl_edge e
|
||||
JOIN gl_definition src ON e.source_id = src.id AND e.source_kind = 'Definition'
|
||||
JOIN gl_definition tgt ON e.target_id = tgt.id AND e.target_kind = 'Definition'
|
||||
WHERE e.relationship_kind = 'CALLS'
|
||||
AND src.file_path LIKE 'acme/webapi/ui/resources/<resource>/%'
|
||||
AND tgt.file_path NOT LIKE 'acme/webapi/%'
|
||||
) b ON a.dependency_name = b.dependency_name AND a.dependency_file = b.dependency_file;
|
||||
### 9.7 Structural commands worth remembering
|
||||
orbit index . # always first
|
||||
orbit repo-map extends TenantScopeMixin # adoption of a shared mixin
|
||||
orbit repo-map extends BaseRepository # inheritance tree
|
||||
orbit repo-map imports ActorScope # who imports a symbol
|
||||
orbit repo-map api acme/webapi/ui/resources/alerts/ # surface summary for LLM context
|
||||
orbit repo-map class AlertRepository # every class with that name
|
||||
orbit grep "visible_to" --path acme/webapi/ui/resources/<resource>/ --limit 50
|
||||
orbit context acme/webapi/common/conventions/tenant_scoping.py
|
||||
---
|
||||
|
||||
## 10. Case study A: architecture survey for a service layer
|
||||
|
||||
**Goal:** plan a shared service / use-case layer that the internal UI API (`ui`) and the public management API (`public`) can both call. The survey needed to know how isolated the two surfaces are, how much shared infrastructure exists, and where business logic actually lives.
|
||||
|
||||
**Verdict:** orbit was **highly useful** for this kind of structural and architectural analysis. It should be a primary tool, alongside targeted grep for the call-site analysis it cannot do.
|
||||
|
||||
### 10.1 What worked
|
||||
**Repository inventory across surfaces** (§9.2). It found 11+ repository classes with line counts and confirmed the duplication the plan wanted to fix. Examples: `PipelineRepository` (public, 446 lines), `CredentialsRepository` (ui, up to 599 lines), `BackfillRepository` (ui, 387), `TeamMappingRepository` (ui, 321), `InvoicesRepository` (ui, 305), `AlertRepository` (public, 96).
|
||||
**Import matrix between surfaces** (§9.3). This was the most important finding:
|
||||
| From → To | Imports |
|
||||
|---|---|
|
||||
| ui → public | **0** |
|
||||
| public → ui | **0** |
|
||||
| public → common | 78 |
|
||||
| ui → common | **1** |
|
||||
| other → ui / public | 6 / 4 (spec-export management commands) |
|
||||
|
||||
The surfaces were already cleanly isolated, so a new layer could be added between them without first untangling cross-dependencies. The UI surface hardly uses the shared `common` layer.
|
||||
**Mixin adoption with `repo-map extends`.**
|
||||
```
|
||||
depth 0: TenantScopeMixin (common/conventions/tenant_scoping.py:51)
|
||||
depth 1: PipelineRepository (public/pipelines/repository.py:50)
|
||||
depth 1: AlertRepository (public/alerts/repository.py:25)
|
||||
```
|
||||
Only two repositories use the shared scoping mixin. The UI repositories each implement their own scoping.
|
||||
**Shared domain dependencies** (§9.6). Both surfaces call the domain `Alert` and `ThresholdAlert` models. The UI side has **30+** distinct domain dependencies (constants, utilities, schemas, rule types, evaluation and scheduling logic), while the public side has **6**. That puts a number on the asymmetry: the UI owns most business logic directly in its routes, and the public API delegates to a thin repository.
|
||||
**Comparing surfaces with `repo-map api`.**
|
||||
- Public alerts: 5 route functions, 1 repository (12 methods), 4 serializers.
|
||||
- UI alerts: 17 route functions, plus 15 incident routes and 3 analytics routes, ~30 serializers, **no repository**.
|
||||
**Comparing members with `repo-map class`.** `orbit repo-map class AlertRepository` returned both classes with that name at once:
|
||||
- domain `acme.alerts.repository.AlertRepository` (40 lines): `get_by_uuid`, `create`, `update`, `delete`
|
||||
- `acme.webapi.public.alerts.repository.AlertRepository` (95 lines): tenant-scoped list/get/assign/unassign
|
||||
This showed the split straight away: the domain repository does CRUD, and the API repository does scoping and assignment.
|
||||
**Module call graph** (§9.5). It mapped the outgoing dependencies of the public alerts module to 13 files across core, the alerts domain, webapi common and webapi auth. This is the input you need to draw layer boundaries.
|
||||
**Sizing surfaces** (§9.1):
|
||||
| Surface | Files | Definitions |
|
||||
|---|---|---|
|
||||
| ui | 190 | 6,407 |
|
||||
| other-webapi | 165 | 3,751 |
|
||||
| public | 57 | 1,027 |
|
||||
| frontend | 50 | 883 |
|
||||
| endpoints | 31 | 504 |
|
||||
| common | **9** | **65** |
|
||||
**`orbit context <file>`** for reading source with definition boundaries marked, without leaving the orbit workflow.
|
||||
### 10.2 What did not work
|
||||
|
||||
- **`visible_to` call sites:** 4 `CALLS` edges against ~120 real sites (§7.2). Body grep was the workaround.
|
||||
- **`gl_imported_symbol` / `repo-map imports`** cannot see attribute-accessed methods (§7.3).
|
||||
- **Duplicate rows** from multiple indexed worktrees in raw SQL (§7.1).
|
||||
- **No reliable reverse call graph** ("who calls `_apply_tenant_scope`") across dynamic dispatch boundaries.
|
||||
|
||||
### 10.3 Findings after the fact check
|
||||
|
||||
The three "surprises" orbit reported were checked with plain grep:
|
||||
|
||||
| Orbit claim | Result |
|
||||
|---|---|
|
||||
| UI alerts has no repository; logic is inline in 17+ routes | **Confirmed** |
|
||||
| Zero imports between ui and public | **Confirmed** |
|
||||
| Three `AlertRepository` classes exist | **Wrong: there are two.** The extra one came from unscoped SQL across worktrees (§7.1). |
|
||||
|
||||
Two more findings from the same survey:
|
||||
|
||||
- **The ticket's `visible_to` count (~199) was high.** The real count is ~120.
|
||||
- **Orbit showed the structure, but not the context.** A separate documentation review found that a core repository layer (actor and scope types) already existed on the main branch, was planned in an architecture decision record, and overlapped with the proposed service layer. Orbit answers "what is in the code", not "what is already planned". Read the design docs and decision records next to the graph.
|
||||
|
||||
### 10.4 Consequences for the design
|
||||
|
||||
- The pilot extraction is **not** "merge two repositories". It is "move the inline business logic out of 17+ UI route functions into a shared layer".
|
||||
- Because the surfaces are already isolated, the layer can be added without first untangling dependencies.
|
||||
- The shared `common` layer (9 files, 2 mixin adopters) has barely started. §9.4 and `repo-map extends TenantScopeMixin` can serve as progress metrics.
|
||||
|
||||
---
|
||||
|
||||
## 11. Case study B: OpenAPI schema-warning cleanup
|
||||
|
||||
**Goal:** remove the warnings drf-spectacular prints during OpenAPI spec generation. An earlier pass took the count from 225 to 196. The question was whether the remaining 196 were really ~196 separate edits, or whether shared base classes would let a few edits fix many warnings.
|
||||
|
||||
**Method:** run the spec check live and categorise the output with grep → pull out the class and method names → re-index → use `orbit repo-map extends` and `repo-map class` to find shared bases → plan base-class fixes first → verify each fix by running the spec check again.
|
||||
|
||||
### 11.1 Live category counts
|
||||
|
||||
| # | Message pattern | Count |
|
||||
|---|---|---|
|
||||
| 1 | `unable to resolve type hint for function "X"` (`SerializerMethodField` getter without a return type) | **108** |
|
||||
| 2 | `unable to guess serializer` (APIView or function view with no serializer) | **47** |
|
||||
| 3 | `could not derive type of path parameter "X"` | **24** |
|
||||
| 4 | `exception raised while getting serializer` | **14** |
|
||||
| 5 | `could not resolve field on model` | **2** |
|
||||
| 6 | `could not resolve serializer field` (custom field type) | **1** |
|
||||
| | **Total** | **196** |
|
||||
|
||||
The ticket's estimates undercounted category 2 by a few. The live run is the source of truth.
|
||||
|
||||
### 11.2 Places where one base-class fix covers many warnings
|
||||
|
||||
**1a. `TypeSerializer`: 3 lines for 10 warnings.** `orbit repo-map extends TypeSerializer` returned 4 descendants (`IntegrationTypeSerializer`, `PipelineTypeSerializer`, `DestinationTypeSerializer`, `PluginPipelineTypeSerializer`, 2 levels deep) in 4 files. All 10 warnings (`get_name`×4, `get_slug`×3, `get_canonical_name`×3) traced back to three untyped methods on the base class. Adding `-> str` there should fix all of them.
|
||||
|
||||
**1b. `PipelineRunNestedSerializer`: one class, 6 warnings.** `repo-map class` showed it is **one class reused as a nested field** in two unrelated parents. Its three untyped URL getters each fire twice, once per parent. The sibling `PipelineRunSerializer` in the same file has the same three method names plus three more, but it is **not a subclass**, only a class with a similar name. It needs its own fix. Two edits fixed all 12 warnings in that file.
|
||||
|
||||
**1c. `StageBaseSerializer`: fix the base before the subclasses.** `repo-map extends` showed a 3-level tree: base → `{Full, Minimal}` → `{Import*, Processing*}`. The base defines four untyped getters. The file had 18 of the 108 warnings, so typing the base first reduces what is left for whoever does the subclasses.
|
||||
|
||||
**Categories 3 and 4 appeared to share one cause.** Four polymorphic ViewSets each produced both an "exception raised while getting serializer" error and 1–3 "could not derive type of path parameter" warnings for the same missing URL kwarg. `orbit repo-map extends GenericPolymorphicViewSet` confirmed that three of them share one base, whose `get_ctype()` reads `self.kwargs["parent_lookup_type_id"]` unconditionally. The hypothesis was that one `swagger_fake_view` guard in the base would fix 3 exceptions and 7 path warnings. A fourth ViewSet, `DestinationLinkViewSet`, has an unrelated base and would need its own guard.
|
||||
|
||||
**Category 2, framework-view exclusions.**
|
||||
- `orbit repo-map extends WorkflowEngineAPIView` found a 2-level tree that covers all 5 warnings in an internal-tooling views file. One exclusion at the base would fix all 5.
|
||||
- `orbit repo-map extends HeartbeatEndpoint` found one subclass: one edit for 2 warnings.
|
||||
- 5 warnings are function views (`@api_view` webhooks and OAuth callbacks). There is no base to share, so each needs its own decorator.
|
||||
- `orbit repo-map extends APIView` returned nothing (§7.4). Grep showed that the rest inherit directly from `APIView` with no shared in-repo base: they need the same edit, but made separately.
|
||||
|
||||
**Category 5.** Two warnings came from one model property read by two sibling serializers, so one annotation should fix both. `orbit grep` did not find the property definition (§7.5); `grep -rn "def …"` did.
|
||||
|
||||
**Category 6.** One custom field that drf-spectacular cannot introspect. The fix copies an existing `OpenApiSerializerFieldExtension` added in the earlier pass.
|
||||
|
||||
**Plan:** make the base-class fixes first, as separate commits. Four edits were predicted to remove 29 warnings (TypeSerializer −10, polymorphic guard −10, DestinationLink guard −4, WorkflowEngine exclusion −5). After that, split the truly separate remainder by file across parallel workers.
|
||||
|
||||
### 11.3 What actually happened
|
||||
|
||||
| Step | Result |
|
||||
|---|---|
|
||||
| Fix 1: `TypeSerializer` return types | **Worked as predicted:** 196 → 186 (−10) |
|
||||
| Fix 2: polymorphic `swagger_fake_view` guard | **Blocked. It had no effect.** drf-spectacular already catches the `KeyError` and produces the same warnings either way. Fixes 3 and 4 were not attempted, because the plan said to stop when a prediction failed. |
|
||||
| Quick wins (several small grouped fixes) | 186 → 162 (−24). One fix beat its estimate (−14 vs. −12) because related files had extra warnings. |
|
||||
| Bulk file-by-file work | 162 → 61 (−101). Category 1 was fully cleared; category 2 went from 40 to 25. |
|
||||
| **Overall** | **225 → 61 (−164, 73%)** |
|
||||
|
||||
Deferred afterwards:
|
||||
- ~25 "unable to guess serializer" warnings on public endpoints. These need real response schemas, not exclusion.
|
||||
- The polymorphic ViewSet cluster (~14 errors plus ~14 related path warnings). The guard approach was proven wrong; these need per-method schema annotations.
|
||||
- A large integration view whose exclusion is dangerous (below).
|
||||
|
||||
### 11.4 Lessons
|
||||
|
||||
- **Orbit's structural predictions were right; the behaviour predictions were not.** The inheritance trees were correct every time. What failed was the assumption about how the framework reacts to a change at the base (Fix 2). Treat an orbit-derived plan as a hypothesis, and verify each step by running the real tool again.
|
||||
- **Stop when a prediction fails.** Checking the count after each base-class fix, and stopping at the first mismatch, kept a wrong theory from spreading into later fixes.
|
||||
- **Base-class changes spread in both directions.** In the bulk phase, a worker added `@extend_schema(exclude=True)` to one large integration view base. The generated spec **shrank by about 17,400 lines**, because every endpoint that inherits from that base disappeared. The change was caught from the spec diff size and reverted. Before an exclusion or other schema-wide change on a base class, run `orbit repo-map extends <Base>` to see how many descendants it affects, and check the spec diff size afterwards.
|
||||
- **Guards in runtime code need tests, not only a spec diff.** The polymorphic guard would have touched `get_queryset`/`get_ctype`, which is runtime logic. Run the affected serializer and ViewSet test suites as well as the spec check.
|
||||
|
||||
---
|
||||
|
||||
## 12. Reusable workflows
|
||||
|
||||
### 12.1 Finding one fix that covers many (warnings, lint errors, deprecations)
|
||||
Run the real tool. Count and categorise its output with grep. Do not trust estimates from tickets.
|
||||
Pull out the class and method names from each category.
|
||||
`orbit index .`
|
||||
For each class name: `orbit repo-map extends <Name>` (and walk up the tree with `repo-map class`) to look for a shared in-repo base.
|
||||
Where a base is shared, fix the base once. Order the fixes by warnings removed per edit.
|
||||
Verify the count after **each** base fix, and stop at the first mismatch.
|
||||
Split the truly separate remainder by file for parallel work.
|
||||
### 12.2 Telling apart classes with the same or similar names
|
||||
|
||||
Run `orbit repo-map class <Name>` and read every match. Decide whether you have:
|
||||
- one class **reused** in several places (one fix covers all uses),
|
||||
- a **subclass** (a fix at the base may be enough), or
|
||||
- a **similar-named class** with no inheritance link (needs its own fix).
|
||||
|
||||
### 12.3 Architecture survey of a large package
|
||||
Size with §9.1 to see where the mass is.
|
||||
Map dependencies between areas with §9.3 to see how isolated they are.
|
||||
Inventory the key abstractions with §9.2.
|
||||
Check adoption of shared bases with `repo-map extends <Mixin>`.
|
||||
Compare two implementations with `repo-map api <dirA>` and `repo-map api <dirB>`.
|
||||
Get the domain dependencies of each area with §9.5 and §9.6.
|
||||
**Fact-check every surprising claim with plain grep** before it goes into a design document.
|
||||
Read the existing design docs and decision records. The graph cannot tell you what is already planned.
|
||||
### 12.4 Orbit for structure, grep for the rest
|
||||
|
||||
Use orbit for inheritance, shape, imports and dependencies between modules. Use grep for dynamic dispatch, framework base classes, exact identifiers and counts.
|
||||
|
||||
---
|
||||
|
||||
## 13. Guidance for agents and a future orbit skill
|
||||
|
||||
Put this in any brief to an agent that is expected to use orbit:
|
||||
|
||||
- **Call orbit from the shell.** The MCP server offers only SQL, schema and index, and it may not load at all.
|
||||
- **Always run `orbit index .` first**, and again after each commit you want to query.
|
||||
- **Never use `orbit sql --all`** unless you are deliberately comparing worktrees. Any count that looks too high by a whole multiple is probably duplication.
|
||||
- **Treat `CALLS` edges as a lower bound.** For Django managers, querysets and other dynamic receivers, count call sites with `grep -rn`.
|
||||
- **`orbit grep` explores; it does not count or find exact matches.** Use `grep -rn` for both.
|
||||
- **Framework base classes are not in the graph.** `repo-map extends APIView` returning nothing does not mean there are no subclasses.
|
||||
- **Report orbit findings as hypotheses** and fact-check surprising structural claims with grep before they reach a design document.
|
||||
- **Verify predictions with the real tool** (tests, linters, spec generators). Orbit knows the structure, not the runtime behaviour.
|
||||
- **Check how far a base-class change reaches** with `repo-map extends` before editing a widely inherited base, and check the diff size afterwards.
|
||||
|
||||
A good description for an orbit skill: *"Use orbit to answer inheritance, class-shape, import-dependency and module-dependency questions in a large Python repository; not as a replacement for grep."*
|
||||
|
||||
---
|
||||
|
||||
## 14. Open questions
|
||||
|
||||
- Would a type-aware indexer (or orbit with type stubs for Django) resolve manager and queryset calls into `CALLS` edges? This is the largest gap for Django codebases.
|
||||
- Can framework base classes be indexed (by including site-packages or stubs) so that `repo-map extends APIView` works?
|
||||
- Will the MCP server expose `grep`, `context` and `repo-map`? Without them, MCP-only agents lose most of the value.
|
||||
- How complete is `CALLS` coverage for non-Django Python (plain classes, dependency injection)? It has only been assessed on a Django-heavy codebase.
|
||||
- Frontend (TypeScript/TSX) indexing was not assessed.
|
||||
@@ -0,0 +1,190 @@
|
||||
---
|
||||
phase: 1
|
||||
topic: review-fit
|
||||
depends-on: phase-0-field-guide-python-django.md
|
||||
status: findings
|
||||
---
|
||||
|
||||
# Phase 1: How orbit fits Spire's review stage
|
||||
|
||||
## TL;DR
|
||||
|
||||
- Spire's development cycle (`docs/workpad/designs/2026-09-22-development-cycle.md`) has a **Review** stage (Stage 6): "a fresh Opus reviewer measures output against plan and design," fed the Athanor's diff + plan reference. **There is no dedicated `reviewer` agent file yet** — Stage 6 is currently a generic Opus subagent dispatch, not a pinned agent like `search`/`scout`/`researcher`/`adept`/`athanor`. This is the single biggest opening for an orbit skill: it can be the thing that turns "a fresh Opus reviewer" into "a fresh Opus reviewer with a code-graph."
|
||||
- Orbit is a strong fit for **exactly the questions review asks and grep answers poorly**: what did this diff touch structurally, what else depends on it (blast radius), does it break a trait contract, does it cross a module boundary it shouldn't, how does the branch's graph differ from main's. All five map to concrete orbit subcommands per the phase-0 field guide's §8/§12/§13.
|
||||
- **`search`** (read-only, Sonnet) is the natural agent to carry orbit calls during review — it already has Bash and no write tools, matching orbit's "call from the shell, not MCP" guidance (guide §13). **`researcher`** would use orbit for one-off deep investigations (e.g., an architecture-survey-style review). **`adept`** could keep a running "blast radius" map for a domain across sessions. **`athanor`** decomposes review work but shouldn't itself be the one running orbit queries — it should dispatch a `search` (or a future `reviewer`) subagent to do so.
|
||||
- Skills in this marketplace are per-plugin folders (`plugins/<name>/skills/<skill>/SKILL.md`, optional `references/`); no central registry — a plugin just needs the file to exist under `skills/`, and agents preload skills via a `skills:` frontmatter list (as `adept.md`, `athanor.md`, `zathras.md` do).
|
||||
- **Recommendation: standalone plugin `plugins/orbit/`** with one skill (`orbit:review` or similar), not folded into Spire. Reasons in §4. Spire agents pick it up the same way they already pick up `tome` (an optional sibling plugin): check for it, use it if present, degrade gracefully (grep) if absent.
|
||||
- `.claude/spire/project.md` (scaffolded by `plugins/spire/skills/init/SKILL.md`) already has a "Domain skills to route to subagents when relevant" field — this is the natural place to flag "orbit available for this project," no new mechanism needed.
|
||||
|
||||
---
|
||||
|
||||
## 1. How review works in Spire today
|
||||
|
||||
The development cycle is documented in `docs/workpad/designs/2026-09-22-development-cycle.md` and mirrored into `plugins/spire/skills/igor/SKILL.md` (§"Development Cycle", lines 91–110). Six stages, review is the decision point:
|
||||
|
||||
> `docs/workpad/designs/2026-09-22-development-cycle.md:30`
|
||||
> | 6. Review | fresh Opus reviewer (never the Athanor that cooked it) | diff + plan | `reports/` — findings with severity |
|
||||
|
||||
> `docs/workpad/designs/2026-09-22-development-cycle.md:42`
|
||||
> **Review** measures output against the plan and design, not taste. Severity: `low` (nits), `medium` (wrong but contained), `high` (load-bearing, or plan not fulfilled).
|
||||
|
||||
The review decision tree (same file, lines 44–52; repeated in `igor/SKILL.md:99-106`) routes findings: none/low → done, medium/high → new plan (short loop back to Stage 4), wrong approach → back to design (Stage 3), unresearched unknown → back to research (Stage 2), wrong goal → stop, user's call.
|
||||
|
||||
**Inputs to review**, per the table: the diff produced by the Athanor, plus the plan it was given. The design doc is implicitly in scope too ("wrong approach... back to design"), though the table only lists "diff + plan" as formal input.
|
||||
|
||||
**Who reviews**: "fresh Opus reviewer (never the Athanor that cooked it)" — `athanor.md` reinforces this from the producer's side:
|
||||
|
||||
> `plugins/spire/agents/athanor.md:39`
|
||||
> **Output goes to a fresh reviewer.** Your deliverable (diff + plan reference) is reviewed by an independent agent in Stage 6 — never by you. Do not self-review.
|
||||
|
||||
Searching the whole `plugins/spire` tree for "reviewer" turns up exactly three hits — the two above plus one in `plugins/spire/skills/implementation/SKILL.md:10`:
|
||||
|
||||
> "Its output (diff + plan reference) goes to a fresh reviewer for Stage 6 (Review)."
|
||||
|
||||
**There is no `plugins/spire/agents/reviewer.md`.** The full agent roster is `adept.md, athanor.md, igor.md, researcher.md, scout.md, search.md, zathras.md` — no reviewer. Stage 6 today means Igor dispatching a plain Opus subagent (per the delegation table in `igor/SKILL.md:76-85`, closest match is "Complex summaries, git fixes, small code changes → Sonnet subagent" or an ad-hoc Opus dispatch; none of the pinned tiers is explicitly "review"). This is a gap, not a hidden agent — worth flagging back to the goal owner if a `reviewer` agent is ever formalized, since that would be the natural place to pin orbit as a preloaded skill (the same pattern `adept.md`/`athanor.md`/`zathras.md` use for `spire:workpad`, `spire:project`, etc.).
|
||||
|
||||
There's also no dedicated "review skill" in `plugins/spire/skills/` — review is described entirely in `igor/SKILL.md` and the design doc, not factored into its own skill file the way implementation is (`plugins/spire/skills/implementation/SKILL.md`).
|
||||
|
||||
## 2. Which Spire agents benefit from orbit, and where
|
||||
|
||||
| Agent | Benefit | Concrete review question orbit answers |
|
||||
|---|---|---|
|
||||
| **search** (`plugins/spire/agents/search.md`) | Highest-value integration point. Already Bash + read-only, no Agent tool — matches orbit's own guidance ("call orbit from the shell," guide §13). Igor already dispatches `search` for "code/file search, symbol tracing" (`igor/SKILL.md:80`); during review this becomes "what does the diff touch, structurally." | Changed-definitions inventory: for each file in the diff, `orbit repo-map class/api <file>` to get the shape instead of eyeballing a raw diff. |
|
||||
| **the (future) reviewer / current ad-hoc Opus reviewer at Stage 6** | This is where orbit's value concentrates — reviewing a diff against a plan is exactly "did this change reach further than the plan said, and correctly." | Blast radius: `orbit repo-map extends <Trait/Class>` and CALLS-edge SQL joins to answer "what depends on the code this diff changed" before signing off severity. Trait implementor coverage: `repo-map extends <Trait>` lists every impl — check the diff didn't leave one impl un-updated after a trait method signature change. Module boundary violations: `orbit repo-map imports <path>` / `orbit sql` over `gl_imported_symbol` to check a diff didn't introduce a downward or cross-boundary import it shouldn't have. Branch-vs-main structural diff: orbit indexes per-worktree snapshots (guide §7/§13 "orbit indexes snapshots, not history"); indexing both the branch worktree and main and diffing `repo-map api`/`class` output for the touched modules gives a structural diff a text diff can't — new/removed public surface, changed method signatures — directly answering "does this diff fulfill the plan" (Stage 6's stated job). |
|
||||
| **researcher** (`plugins/spire/agents/researcher.md`) | Has Edit/Write and does deep multi-source investigation; a good fit for the rarer "review found the approach is wrong, back to design" loop, where an architecture-survey-style orbit workflow (guide §12.3) is warranted rather than a quick diff check. | E.g., "is the new trait actually a good abstraction boundary" — inventory of existing implementors and their shape before recommending a design change. |
|
||||
| **adept** (`plugins/spire/agents/adept.md`) | Long-lived, domain-scoped, accumulates `key-files.md`/`architecture.md`. Could keep a standing note of a domain's key trait hierarchies or module boundaries (from a one-time orbit survey), refreshed via its existing "drift check" (`git log <verified-at>..HEAD -- <covers paths>`, `adept.md:30-34`) rather than re-running orbit every consultation. Orbit would seed the doc; the drift check already tells the adept when to re-verify. | Not a review-question answerer directly, but a place blast-radius/boundary knowledge could live between review sessions instead of being recomputed each time. |
|
||||
| **athanor** (`plugins/spire/agents/athanor.md`) | Should not itself run orbit — it dispatches. But when decomposing a large implementation brief, it could dispatch a `search` subagent with an orbit query to establish blast radius *before* implementing, feeding the plan rather than only the post-hoc review (implementation-support use case, explicitly deferred per the plan's "Direction" note: "First use case: review... Implementation support comes later"). Flag this as later-phase, not now. | — |
|
||||
| **scout** (`plugins/spire/agents/scout.md`) | Too lightweight/Haiku-tier for orbit's multi-step index→query workflow; orbit's own guide (§13) stresses re-indexing discipline and hypothesis-then-grep-verify, which doesn't fit scout's "one grep, one read, done" scope (`scout.md:27-29`). Not a good fit. | — |
|
||||
| **igor** | Never runs tools directly (`igor/SKILL.md:15-23`, Core Directive) — it routes. Igor's role re: orbit is purely: know an orbit skill exists, and brief `search`/a reviewer with it when the task is structural. | — |
|
||||
|
||||
Net: **search** for the diff-touches-what step, **the Stage-6 reviewer** (ad-hoc today, ideally its own pinned agent later) for blast-radius/trait-coverage/boundary/branch-diff, **researcher** for the rarer redesign-triggering deep dive, **adept** as a place to cache the results across a session.
|
||||
|
||||
## 3. Skill structure in this marketplace
|
||||
|
||||
Confirmed by inventory (`find plugins -iname SKILL.md`, 25 hits across `bevy`, `dioxus`, `g4b_ai`, `gitea`, `project-uv`, `spire`, `tome`). Structure:
|
||||
|
||||
```
|
||||
plugins/<plugin-name>/
|
||||
.claude-plugin/plugin.json # name, version, description, author — no skill list needed
|
||||
skills/
|
||||
<skill-name>/
|
||||
SKILL.md # required: YAML frontmatter + body
|
||||
references/ # optional: longer supporting docs, loaded on demand
|
||||
*.md
|
||||
scripts/ (or *.sh at skill root, e.g. spire/skills/adepts/roster.sh)
|
||||
agents/
|
||||
<agent-name>.md # frontmatter can preload skills via `skills:`
|
||||
```
|
||||
|
||||
**`.claude-plugin/plugin.json`** (e.g. `plugins/spire/.claude-plugin/plugin.json`, `plugins/tome/.claude-plugin/plugin.json`) is just plugin metadata — `name`, `version`, `description`, `author`. **No per-skill registration** — any `skills/<name>/SKILL.md` under the plugin directory is auto-discovered. Enabling the plugin (`"spire@reliquary": true` in `.claude/settings.json` → `enabledPlugins`) is what exposes all its skills/agents to a project.
|
||||
|
||||
**`SKILL.md` frontmatter fields observed** (from `plugins/spire/skills/*/SKILL.md`):
|
||||
- `name` (required)
|
||||
- `description` (required — long, keyword-dense, written to match model-invocation triggers; see `svg/SKILL.md:2` or `workpad/SKILL.md:2` for the house style — enumerates trigger phrases explicitly)
|
||||
- `allowed-tools` — restricts what the skill may invoke, e.g. `project/SKILL.md`: `allowed-tools: Bash(cat:*)`; `init/SKILL.md`: a longer allowlist
|
||||
- `disable-model-invocation: true` — for human-triggered-only skills (`init/SKILL.md`)
|
||||
- Skill bodies can contain a leading `!`command`` line that runs and injects output before the harness even shows the skill body — `project/SKILL.md` does this to cat `project.md` inline.
|
||||
|
||||
**How agents load skills**: agent frontmatter has an optional `skills:` list of `<plugin>:<skill>` names, preloaded at agent spawn:
|
||||
|
||||
> `plugins/spire/agents/adept.md:12-14`
|
||||
> ```
|
||||
> skills:
|
||||
> - spire:workpad
|
||||
> - spire:adepts
|
||||
> ```
|
||||
|
||||
> `plugins/spire/agents/athanor.md:14-20`
|
||||
> ```
|
||||
> skills:
|
||||
> - spire:project
|
||||
> - spire:workpad
|
||||
> - spire:implementation
|
||||
> - superpowers:using-superpowers
|
||||
> - superpowers:subagent-driven-development
|
||||
> - superpowers:dispatching-parallel-agents
|
||||
> ```
|
||||
|
||||
Cross-plugin skill references work fine (`superpowers:*` skills loaded into `athanor`, `tome:navigate` referenced conditionally in `athanor.md:25` and `igor` boot). This is the precedent for a standalone `plugins/orbit/` plugin whose skill gets referenced from Spire agents without merging code.
|
||||
|
||||
**Minimal template matching house style:**
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: <skill-name>
|
||||
description: >
|
||||
<One paragraph, keyword-dense, states WHEN to use it — enumerate trigger
|
||||
phrases and situations explicitly, the way workpad/SKILL.md and svg/SKILL.md
|
||||
do, so model-invocation matches reliably. State what it is NOT for too,
|
||||
if there's a nearby skill it could be confused with (e.g. "not a
|
||||
replacement for grep").>
|
||||
---
|
||||
|
||||
# <Skill Title>
|
||||
|
||||
<One-paragraph statement of what decision or workflow this skill governs —
|
||||
not "how to write code," but "how to decide X" (cf. workpad/SKILL.md's
|
||||
opening: "This skill answers two questions only...").>
|
||||
|
||||
## <first decision point / section>
|
||||
|
||||
...
|
||||
|
||||
## Composition with other skills
|
||||
|
||||
- **`<related-skill>`** governs <adjacent concern>; defer to it for that.
|
||||
```
|
||||
|
||||
References subfolder is used for material too long to inline (`svg/skills/references/inline-svg.md`, `layout-legibility.md`; `igor/skills/references/mindset.md`, `model-tiers.md`, `harness-notes.md`, `journal-template.md`) — the SKILL.md stays a decision-routing document and `@references/foo.md` is inlined via the `@`-include syntax seen in `igor/SKILL.md:56` (`@references/mindset.md`).
|
||||
|
||||
## 4. Placement options for an orbit skill
|
||||
|
||||
**(a) Inside Spire** (`plugins/spire/skills/orbit/SKILL.md` or similar):
|
||||
- Pro: zero extra enablement step for anyone who already has Spire; trivially preloadable into `search`/`athanor`/a future `reviewer` via their existing `skills:` frontmatter lists, same as `spire:workpad`.
|
||||
- Con: **orbit is an external tool, not every project has it, and not every Spire project is Rust** (the plan's own framing: first use case is "Rust projects that use Spire," implementation support later — implying non-Rust and non-Spire uses exist too). Bundling it into Spire ties an optional, project-conditional capability to a plugin that's supposed to be broadly applicable orchestration machinery. It also means every Spire user who doesn't have orbit installed carries dead skill-description weight in their agents' trigger-matching search space.
|
||||
- Con: mixes concerns — Spire's own `CLAUDE.md` frames it as "agent orchestration," not "a place for tool-specific integrations." `tome` was split out as a separate plugin for exactly this reason (knowledge navigation is optional per project, and it's referenced conditionally — `athanor.md:25`: "If the project has the `tome` plugin installed, use `tome:navigate`... otherwise orient via direct file reads").
|
||||
|
||||
**(b) Standalone plugin `plugins/orbit/`** (mirroring `tome`'s shape):
|
||||
- Pro: matches the `tome` precedent exactly — an optional capability plugin, enabled per-project (`enabledPlugins: {"orbit@reliquary": true}` only where orbit is installed and useful), referenced conditionally from Spire agents ("if the project has the `orbit` plugin installed, use `orbit:review`; otherwise fall back to grep" — same sentence shape as the tome line above).
|
||||
- Pro: reusable outside Spire entirely — any project/agent setup in this marketplace could enable `orbit@reliquary` without pulling in Spire's orchestration machinery. Matches the plan's stated aim that implementation support (non-review) comes later and might have a different consumer than the review skill.
|
||||
- Pro: can iterate/version independently — this is explicitly an "early prototype" (plan.md), and a standalone plugin can be broken or rewritten without touching Spire's stability.
|
||||
- Con: one more plugin to enable per project; Spire agents need an explicit conditional check (cheap — same pattern already used for tome).
|
||||
|
||||
**(c) Other** (e.g., a project-level skill under `.claude/skills/` outside the marketplace, or folding into an existing plugin like `gitea`): no evidence in the repo of a per-project (non-plugin) skill mechanism being used anywhere; `gitea` is a different concern (ticket workflow) and not a natural home. Not recommended — no structural precedent for it in this marketplace, whereas (a)/(b) both have precedent (Spire itself, and the tome/Spire split).
|
||||
|
||||
**Recommendation: (b), standalone `plugins/orbit/`.** It mirrors the one existing precedent in this exact repo for "optional, tool-dependent, project-conditional capability that Spire agents use if present" — `tome`. A Spire agent picks it up the same way `athanor.md:25` already picks up tome: a conditional line ("if `.claude/spire/project.md` (or plugin.json availability) flags orbit, use `orbit:review`; otherwise use grep/read"), with the actual availability fact recorded once per project rather than re-detected every session (see §5).
|
||||
|
||||
## 5. Per-project facts and orbit availability
|
||||
|
||||
`.claude/spire/project.md` is scaffolded by `plugins/spire/skills/init/SKILL.md` (Phase 3, "Scaffold project.md"). It is read at agent startup by the tiny `spire:project` skill:
|
||||
|
||||
> `plugins/spire/skills/project/SKILL.md`
|
||||
> ```
|
||||
> ---
|
||||
> name: project
|
||||
> description: Loads per-project Spire configuration (override or bundled template).
|
||||
> allowed-tools: Bash(cat:*)
|
||||
> ---
|
||||
> !`cat "${CLAUDE_PROJECT_DIR}/.claude/spire/project.md" 2>/dev/null || echo "No project config found. Run /spire:init to set up this project."`
|
||||
> ```
|
||||
|
||||
The scaffolded template (`init/SKILL.md`, Phase 3.2) already includes exactly the right slot:
|
||||
|
||||
> ```markdown
|
||||
> - **Tome skill**: <detected tome skill or "none detected...">
|
||||
> ...
|
||||
> - **Domain skills to route to subagents when relevant**: <!-- detect and list, or leave as placeholder -->
|
||||
> ```
|
||||
|
||||
This is the natural, zero-new-mechanism place to flag orbit: extend `init`'s Phase 2 detection (2.5, "Tome presence" — `init/SKILL.md`, section 2.5) with a parallel "2.6 — Orbit presence" check (e.g. `which orbit`, or a `.orbit/` index directory in the repo root, or an `orbit@reliquary` entry in `enabledPlugins`), and populate the "Domain skills to route to subagents when relevant" field (or add a dedicated "Orbit" line next to "Tome skill") with either the detected skill name or "none detected — fall back to grep." Agents (`search` during review, a future `reviewer`) then read this fact for free via the existing `spire:project` preload — no new per-project config file or convention needed, just one more detection phase in `init` and one more line in the template it writes.
|
||||
|
||||
## Sources
|
||||
|
||||
- `docs/workpad/designs/2026-09-22-development-cycle.md`
|
||||
- `plugins/spire/skills/igor/SKILL.md`
|
||||
- `plugins/spire/agents/athanor.md`, `adept.md`, `researcher.md`, `search.md`, `scout.md`, `zathras.md`
|
||||
- `plugins/spire/skills/implementation/SKILL.md`, `workpad/SKILL.md`, `research-tree/SKILL.md`, `project/SKILL.md`, `init/SKILL.md`, `svg/SKILL.md`
|
||||
- `plugins/spire/.claude-plugin/plugin.json`, `plugins/tome/.claude-plugin/plugin.json`, `plugins/tome/skills/navigate/SKILL.md`
|
||||
- `.claude/settings.json` (enabledPlugins)
|
||||
- `docs/workpad/goals/orbit-skill.md`, `docs/workpad/research/2026-09-29-17-orbit/plan.md`
|
||||
- `docs/workpad/research/2026-09-29-17-orbit/phase-0-field-guide-python-django.md` (§1, §7, §8, §12, §13)
|
||||
@@ -0,0 +1,415 @@
|
||||
---
|
||||
phase: 1
|
||||
stream: rust-capabilities
|
||||
date: 2026-09-29
|
||||
orbit-version: 0.133.0 (~/.local/bin/orbit, embedded skill "orbit-cli" v0.18.2)
|
||||
test-beds: spacecraft (Bevy game, 25-crate workspace, 903 .rs files), dirigent (Tokio/Dioxus server+web, 35-crate workspace, 989 .rs files)
|
||||
status: done
|
||||
---
|
||||
|
||||
# Orbit 0.133 on Rust: capabilities probe
|
||||
|
||||
Legend: **[V]** = verified here with command output. **[I]** = inferred (reasoned from evidence, not tested directly).
|
||||
|
||||
Scratch scripts used for the measurements are in `~/.cache/orbit-probe/` (`calls.py` = CALLS recall against grep, `cfgcheck.py` = cfg-gated item presence, `ext.py` = impl-for against EXTENDS, `defdiff.sql`, `impl-heuristic.sql`). Phase 2 can reuse them.
|
||||
|
||||
## TL;DR
|
||||
|
||||
- **Orbit understands Rust well where code is visible.** Name resolution is type-aware, and it follows `use` paths across workspace crates. Recall for "who calls X" was **85–100 %** for free functions, `Type::new`-style associated functions and methods on concretely typed receivers. It even resolved calls through `match` bindings on enum payloads. [V]
|
||||
- **It does not see everything.** cfg is evaluated with **every Cargo feature off (default features included)**, `test` off and `debug_assertions` off. Feature-gated modules drop out entirely. In dirigent the whole `server` feature is invisible: `dirigent_core/src/runtime/mod.rs` (180 KB) and `config.rs` (105 KB) have **0 definitions**, and 176 of 989 Rust files have none. `#[cfg(test)] mod tests` never appears, so unit tests are invisible. Only `tests/` integration tests are indexed. [V]
|
||||
- **Trait graph is partial.** `impl Trait for Type` gives `EXTENDS Type→Trait` only when the impl block **is in the same file as the type** (60/62 hits same-file, 0/10 cross-file). Supertraits work. External traits (`Plugin`, `Display`, `Serialize`, …) give nothing, and the result is silently empty (216 `impl Plugin for` in spacecraft). [V]
|
||||
- **Weak spots for CALLS:** calls through `dyn Trait` (0/4 and 1/12 callers found), calls inside macro arguments (`assert!(f(..))`), Bevy systems passed to `add_systems` (they are not calls), and Dioxus components used in `rsx!` (4 of 260 have any incoming edge). [V]
|
||||
- **Review use is feasible.** Two commits (main plus a detached `git worktree`) can be indexed at once and diffed at definition level with one `--all` SQL query. Orbit has **no command to delete an index entry**, so the worktree's rows stay in the shared DB. [V]
|
||||
- **Hygiene:**
|
||||
- Index with `-t 4`. With the default thread count the per-file watchdog skipped 4–7 large files per run, a different set each time.
|
||||
- Check `processing.skipped_files` in the index JSON.
|
||||
- Leave `orbit sql` at its default scope. MCP `run_sql` is **always unscoped**. [V]
|
||||
|
||||
## Probe context and state notes
|
||||
|
||||
- **Pre-existing entries.** These belong to the user, who had indexed the repos before this probe. They are not leftovers from the interrupted attempt, and I removed none of them. All had status `indexed`; there were no `indexing`/`error` rows. At probe start (17:12 local), each indexed commit compared with the repo's HEAD as follows [V]:
|
||||
|
||||
| repo_path | indexed commit (last_indexed_at, UTC) | HEAD at probe start | stale? |
|
||||
|---|---|---|---|
|
||||
| ~/Projects/spacecraft | `3ff5f27` (14:37:01) | `8d9ec30a` (main) | **yes** |
|
||||
| ~/Projects/dirigent | `dd8dfbb2` (14:36:49), branch `feature/w1c-live-acp` | `5de6c726` (same branch) | **yes** |
|
||||
| ~/Projects/sandcage | `2e70b44` (14:37:22) | `2e70b44` (main, dirty worktree: 6 modified docs) | no; uncommitted edits are not visible to orbit [I] |
|
||||
|
||||
**Staleness detection on the stale spacecraft entry (before any re-index)** [V]:
|
||||
- `orbit repo-map` → `Error: current commit 8d9ec30a… is not indexed … run: orbit index .` (hard fail).
|
||||
- `orbit sql "select count(*) from gl_definition"` → note on stderr: "commit 8d9ec30a … is not indexed; querying every indexed commit (as with --all)". It returned **34881**, the rows of all three repos.
|
||||
- `orbit grep new` → header `grep "new" @ 8d9ec30a` and results, with **no warning**.
|
||||
- `orbit context <file>` → returned a file map with no warning.
|
||||
|
||||
I then re-indexed spacecraft and dirigent at their HEADs. Re-indexing a path replaces that path's entry. The entries are now `spacecraft@8d9ec30a` and `dirigent@8c64bfd2` (dirigent got a new commit from another session during the probe). The "writes into the repo" check below is therefore a **re-index** check on existing entries, plus a first-index check on a fresh worktree (§10).
|
||||
- **Concurrency.** Other sessions were active during the probe. spacecraft was re-indexed by someone else at 17:12 local, between two of my queries. dirigent received a commit (`8c64bfd2`) and a file edit. The `strategy-mvp` worktree in spacecraft moved. The DuckDB file `~/.gitlab/orbit/graph.duckdb` is **shared by every agent on the host**, and one row per `repo_path` is overwritten on re-index. [V]
|
||||
- **`orbit index` writes nothing into the repo.** Evidence:
|
||||
- spacecraft (re-index): `git status --porcelain --ignored` was identical before and after.
|
||||
- Scratch worktree (first index of a fresh checkout): identical, and no new files appeared.
|
||||
- dirigent: the only diffs came from the concurrent session (a test file edit, later committed).
|
||||
- All output goes to `~/.gitlab/orbit/graph.duckdb`, which grew from 46.9 MB to 70 MB. [V]
|
||||
|
||||
## 1. CLI surface: what is new since 0.124
|
||||
|
||||
`orbit --help` (0.133.0) lists these subcommands: `version index grep context sql schema list mcp repo-map skills setup uninstall query status ontology dsl tools graph-status config`. [V]
|
||||
|
||||
| Area | 0.124 guide | 0.133 (verified from `--help` and use) |
|
||||
|---|---|---|
|
||||
| `index` | `orbit index .` | `-t/--threads` (0 = auto), `-s/--stats` (per-file and per-language timings, skipped files), `-v`, `--db PATH`. PATH may be a directory that holds several repos. Prints JSON stats, including `processing.skipped_files`. |
|
||||
| `grep` | several queries, ranked | **One** query; `'a\|b\|c'` for OR. `--repo`; repeatable `--path` with globs; `--kind`; omit the query with `--path` to get a file **outline**. Each result is labelled `exact-name` / `name/path` / `body-only ×N` and shows the first body line that matched. |
|
||||
| `context` | `--tests` flag | **`--tests` removed** (error: unexpected argument). Test/fixture/generated connections always print in their own section. Accepts `Definition:<id>`, `File:<id>`, exact FQNs, paths, `path:start-end`, directories. `--repo`. |
|
||||
| `sql` | default scope, `--all` | Adds `--repo PATH` (scope to another checkout) and `-` to read stdin. **New:** if the current commit is not indexed, it falls back to `--all` and prints a note **on stderr**. |
|
||||
| `schema` | whole schema | `orbit schema gl_definition gl_edge` (filter by table), `--raw` (JSON). |
|
||||
| `list` | table | `-F table\|json\|ndjson\|csv`. |
|
||||
| `repo-map` | overview/tree/api/class/extends/imports | Same subcommands. `--repo`, `--ext`. `class` also covers traits and modules. `overview` adds "Key abstractions" (most EXTENDS descendants), "Most-imported defined symbols" and "Most-called callables". Refuses to run on an unindexed commit. |
|
||||
| new | – | `skills [get orbit [path]]`: embedded vendor skill `orbit-cli` v0.18.2, with references `local/cli.md`, `local/sql.md`, `local/repo_map.md`. `setup`/`uninstall`: write instructions, hooks, skill and MCP config into claude/codex/duo/opencode/pi, with `--dry-run` and `--project`. `config` (`~/.gitlab/orbit/settings.json`, only `telemetry.enabled`). |
|
||||
| new, remote-only | – | `query`, `status`, `ontology`, `dsl`, `tools`, `graph-status` talk to the remote GitLab Orbit API. Not probed. |
|
||||
| Schema | – | `_orbit_manifest` now has `project_id` and `parent_repo_path` (links a worktree to its main repo). `gl_file.reason` (`skip_excluded_extension`, `skip_oversize`, `skip_not_utf8`, `skip_binary`). Byte and char offsets on definitions and imports. `gl_edge` has **no** `commit_sha`. |
|
||||
| Env | – | `ORBIT_DATA_DIR` appears in the binary's strings [I: untested]. `--db` exists on every local command. |
|
||||
|
||||
**Changelog:** not found locally. There is no changelog subcommand or file, and nothing under `~/.local/share` or `~/.cache`. The embedded skill says its source is `gitlab-org/orbit/knowledge-graph`. [V: absence locally]
|
||||
|
||||
## 2. MCP
|
||||
|
||||
`orbit mcp serve` is a stateless stdio server. I checked it with an `initialize` + `tools/list` + `tools/call` pipe (`timeout 20`). The process exited, and `pgrep` showed no orbit process left. [V]
|
||||
|
||||
- Server `orbit-cli 0.133.0`, protocol `2024-11-05`. **Still only three tools:**
|
||||
- `get_graph_schema(db?)`
|
||||
- `index(path, threads?, stats?, db?)`
|
||||
- `run_sql(sql: string[], db?)`: now takes an **array** of statements and returns one result set per statement. "Orbit rejects large result sets."
|
||||
- **`run_sql` has no repo/commit scoping.** `select count(*) from gl_definition` gave **52114 rows across 4 commits** over MCP, against **17110** with CLI default scope in spacecraft. Every MCP query therefore behaves like `--all`: filter by `commit_sha`, or join `_orbit_manifest`. [V]
|
||||
- `grep`, `context` and `repo-map` are still CLI-only. The guide's advice holds: use the CLI through bash.
|
||||
|
||||
## 3. Rust schema
|
||||
|
||||
**`definition_type` for `.rs` files** [V]:
|
||||
|
||||
| type | spacecraft | dirigent | notes |
|
||||
|---|---|---|---|
|
||||
| Field | 6356 | 5359 | struct fields and enum-variant fields (`Variant::0`, `Variant::reason`) |
|
||||
| Function | 2961 | 3430 | free fns, including `#[test]` fns in `tests/` |
|
||||
| Struct | 2288 | 954 | |
|
||||
| Method | 1272 | 1960 | fns with `self`, and trait method declarations (fqn `Trait::m`) |
|
||||
| Variant | 1270 | 1931 | enum variants |
|
||||
| Constant | 794 | 367 | |
|
||||
| Module | 642 | 682 | inline `mod x {}` and `mod x;` |
|
||||
| AssociatedFunction | 517 | 529 | fns without `self` inside `impl` (`new`, `default`, …) |
|
||||
| Enum | 285 | 456 | |
|
||||
| TypeAlias | 29 | 62 | |
|
||||
| Trait | 18 | 30 | |
|
||||
| Macro | 10 | 7 | `macro_rules!` only |
|
||||
| Static | 6 | 35 | |
|
||||
|
||||
- There is **no `Impl` node.** Methods from every impl block (in any file) hang off the type through fqn `crate::path::Type::method`. [V]
|
||||
- A trait impl's methods cannot be told apart from inherent methods: the fqn carries no trait. [V]
|
||||
- **fqn root is the cargo target name.** This covers lib crates, `[[bin]] name` and integration tests (file stem, e.g. `archetypes::…`). In spacecraft the root package lib `warpspace` and the bin `warpspace` (from `warpspace_app`) **share the root `warpspace`**. Group by `file_path`, not by fqn prefix. [V]
|
||||
- **`start_line` includes doc comments and attributes.** Example: `Gamemode` shows at `mod.rs:177`, but `pub trait` is at line 182. [V]
|
||||
|
||||
**Edges** (`relationship_kind`, `source_kind` → `target_kind`) [V]:
|
||||
|
||||
| edge | spacecraft | dirigent |
|
||||
|---|---|---|
|
||||
| DEFINES File→Definition | 17110 | 17152 |
|
||||
| DEFINES Definition→Definition (nesting) | 9253 | 8718 |
|
||||
| IMPORTS File→ImportedSymbol | 12912 | 11074 |
|
||||
| CALLS Definition→Definition | 5570 | 10028 |
|
||||
| CALLS Definition→ImportedSymbol (unresolved/external) | 877 | 2377 |
|
||||
| CALLS File→… (top-level) | 69 | 74 |
|
||||
| CONTAINS Directory→File/Directory | 4285 | 3220 |
|
||||
| EXTENDS Definition→Definition | 43 (31 Rust) | 34 |
|
||||
|
||||
- CALLS targets include `Struct` (1157 in spacecraft) and `Variant` (406). These are **construction sites**: struct literals, tuple-struct and variant constructors. Type mentions in signatures, `Query<&T>` or generics produce **no** edge. [V]
|
||||
- **Proc-macro invocations resolve.** `hex_color!(…)` has 202 grep sites and 200 raw CALLS edges to the `#[proc_macro] fn hex_color`. `macro_rules!` invocations resolve at statement level (4/4, 7/7, 3/3). `seam!` invoked inside `vec![…]` has 0/63. [V]
|
||||
|
||||
## 4. Traits and impls
|
||||
|
||||
- **In-repo trait, implementor in the same file:** `EXTENDS Struct→Trait`. 30/30 in spacecraft; 30/32 in dirigent (two misses in `dirigent_server/src/adapters/process.rs`, cause unknown). [V]
|
||||
- **Impl block in a different file from the type:** **no EXTENDS** (spacecraft 0/5, dirigent 0/5). Example: `JsonlBackend` implements 6 archivist traits in 6 files. Only `ArchiveBackend` is linked, because that impl is in `backend.rs` with the struct. The methods are still attached to `JsonlBackend` (`repo-map class JsonlBackend` lists them with their files). [V]
|
||||
- **Extension traits on external types** (`impl RegisterGameUpdate for App`, `impl IntoPlayerId for u32`): no edge, because the implementing type is not a node. [V]
|
||||
- **Blanket impls** (`impl<T> AbilityClone for T`): no edge. [V]
|
||||
- **Supertraits work.** `trait Ability: Debug + Send + Sync + AbilityClone` gives `EXTENDS Ability→AbilityClone`, so `repo-map extends AbilityClone` shows `Ability` at depth 1. External supertraits (`Resource`, `Debug`) are dropped. [V]
|
||||
- **External traits:** `orbit repo-map extends Plugin` prints only the header, with no rows and no warning. spacecraft has 216 `impl Plugin for`. dirigent has 348 impls of external traits. [V]
|
||||
- **Enums implementing in-repo traits:** neither repo had a case, so this is untested. [I: likely the same same-file rule]
|
||||
- **Trait-object users are missing from EXTENDS by construction.** `repo-map extends <Trait>` is a **lower bound**. Cross-check with `git grep -n "impl .*<Trait> for"`, or use the method-name overlap heuristic (cookbook C7). [V]
|
||||
|
||||
## 5. CALLS coverage
|
||||
|
||||
Method used: for each identifier, compare the set of **enclosing definitions** of `git grep -w` hits (comments, `use` lines and the definition line excluded) with the set of CALLS sources pointing at the target. Script: `~/.cache/orbit-probe/calls.py`.
|
||||
|
||||
| repo | identifier | kind | grep callers | orbit callers | recall | notes |
|
||||
|---|---|---|---|---|---|---|
|
||||
| spacecraft | `generate_ship` | free fn, cross-crate | 31 | 30 | 97 % | |
|
||||
| spacecraft | `themed_button` | free fn | 25 | 25 | 100 % | |
|
||||
| spacecraft | `micro_exec_tick` | free fn | 17 | 17 | 100 % | |
|
||||
| spacecraft | `border_radius_small` | method (`theme.x()`) | 40 | 34 | 85 % | misses inside nested `with_children(\|p\| …)` closures and on `Res<UiTheme>` receivers |
|
||||
| spacecraft | `register_session_resource` | extension-trait method on `App` | 19 | 19 | 100 % | |
|
||||
| spacecraft | `validate` | fn plus same-named method | 46 | 35 | 76 % | misses: `assert!(validate(..))`, `matches!(validate(..))`, `field.field_type.validate(v)` |
|
||||
| spacecraft | `MonitorShared::new` | assoc fn | 1 | 1 | 100 % | |
|
||||
| spacecraft | `Gamemode::{check_victory, spawn_player_units, room_for_joiner, initial_camera_transform}` | via `Box<dyn Gamemode>` | 4 | 0 | **0 %** | |
|
||||
| spacecraft | `update_flex_centering` | Bevy system | 2 | 0 | 0 % | passed to `add_systems`, not called |
|
||||
| dirigent | `throttled` | free fn | 79 | 73 | 92 % | |
|
||||
| dirigent | `use_workspace` | Dioxus hook | 41 | 40 | 98 % | |
|
||||
| dirigent | `show_toast` | free fn | 29 | 29 | 100 % | |
|
||||
| dirigent | `SchemaValidator::new` | assoc fn | 47 | 46 | 98 % | |
|
||||
| dirigent | `handle_request` | method | 29 | 28 | 97 % | |
|
||||
| dirigent | `put_session` | via `Arc<dyn ArchiveBackend>` field | 12 | 1 | **8 %** | `reg.backend.put_session(..).await` |
|
||||
| dirigent | `count_messages` | dyn / field chain | 7 | 1 | **14 %** | |
|
||||
| dirigent | `#[component]` fns in `rsx!` | Dioxus | 260 comps | 4 with any edge | ~0 % | |
|
||||
|
||||
All rows above are [V].
|
||||
|
||||
- **Trait dispatch resolves when the concrete type is known.** `ReplayMover::update_movement` does `match self { ReplayMover::Pid(m) => m.update_movement(..) }`, and edges go to **each concrete impl method** (`PIDMovement::update_movement`, …). They do not go to the trait declaration. [V]
|
||||
- **Calls to the trait method declaration are rare**, so "who calls `Trait::m`" undercounts generic and `dyn` callers. [V]
|
||||
- **Cross-crate CALLS resolve.** Cross-crate edges: spacecraft 807 of 4038 crate→crate edges, dirigent 613 of 8810. [V]
|
||||
- **Default `orbit sql` scope deduplicates edges to one per (source, target).** `--all` shows **one edge per call site**: `border_radius_small` has 34 edges scoped against 45 with `--all`, and `hex_color` 4 against 200. `repo-map overview` "callers" counts raw call sites. [V]
|
||||
- **Share of callables with no incoming CALLS:**
|
||||
- spacecraft: Function 1840/2961, Method 1019/1272, AssociatedFunction 405/517.
|
||||
- dirigent: Function 1848/3430, Method 1145/1960, AssociatedFunction 222/529.
|
||||
- Most of these are Bevy systems, trait impl methods, handlers and `main`s. **"No callers" is not "dead code".** [V]
|
||||
|
||||
## 6. Imports (`gl_imported_symbol`)
|
||||
|
||||
- **`import_type` values for Rust** [V]:
|
||||
- `Use`: `use a::b::C;`
|
||||
- `UseGroup`: `use a::{B, c::D}`; nested groups are flattened to full paths.
|
||||
- `GlobUse`: `use bevy::prelude::*`; `import_path` = module, `identifier_name` empty.
|
||||
- `AliasedUse`: `as`.
|
||||
- `PubUseGroup`, `ReExport`, `ReExportGlob`, `ReExportAliased`: the `pub use` forms.
|
||||
- `ModDeclaration`: `mod x;` also lands here.
|
||||
- `ExternCrate`.
|
||||
- **Paths are raw and not normalised.** `crate::theme::widgets::themed_button`, `super::theme::widgets::themed_button`, `super::widgets::themed_button` and edition-2018 relative `game_states::is_client_world` all appear as written. Cross-crate imports carry the crate name (`warpspace_engine::game_states::InSession`). To match a symbol, use `identifier_name`, or `import_path LIKE '%::name'`. [V]
|
||||
- **Imports are not resolved to definitions.** `context` labels them `external/unresolved`, even in-repo ones. There is no ImportedSymbol→Definition edge. Re-export chains (`pub use`) are not followed. [V]
|
||||
- **`repo-map imports PATTERN` returns aggregated counts per `(symbol, import_path)`, not the file list.** Use SQL for the files (cookbook C4). [V]
|
||||
- **Imports are kept in files whose definitions were dropped** (cfg-excluded modules, `#[cfg(test)]` files, `vendor/`). Example: `runtime/mod.rs` has 37 import rows and 0 definitions. An import matrix therefore **includes** feature-gated and test code, while definitions and CALLS **exclude** it. [V]
|
||||
- **Crate-to-crate import matrix works** (dirigent: `dirigent_web→dirigent_api` 217, `dirigent_core→dirigent_protocol` 201). [V]
|
||||
|
||||
## 7. Workspaces, crates, tests
|
||||
|
||||
- **All workspace members are indexed.** spacecraft: 25 crates plus the root package. dirigent: 35 crates. The file set is **git-tracked files only** (903 of 2424 `.rs` files on disk in spacecraft; `target/` and untracked files are ignored). [V]
|
||||
- **Code outside the workspace crate trees gets no definitions,** only imports and `macro_rules`. spacecraft `vendor/` (88 tracked `.rs` files of vendored bevy_replicon/lightyear) has 5 definitions, all macros. `archive/*.rs` has 0. [V: counts; I: exact rule, probably module-tree reachability]
|
||||
- **Tests** [V]:
|
||||
- `#[cfg(test)] mod tests { … }` (144 inline in spacecraft, 389 in dirigent) and files pulled in by `#[cfg(test)] mod tests;` (e.g. all of `warpspace_testing/src/tests/*.rs`) have **0 definitions**. `fqn LIKE '%::tests::%'` returns 0.
|
||||
- `crates/*/tests/*.rs` integration tests **are** indexed. Their fns show up in `context` under "Test, fixture, or generated connections". Example: `generate_ship` has 30 test callers from `archetypes::…`, `determinism::…`.
|
||||
- Integration test files with `#![cfg(feature = "…")]` have 0 definitions.
|
||||
- `context --tests` no longer exists.
|
||||
- **cfg evaluation.** Features are **all off**, including `default = [...]`. `debug_assertions` off, `test` off, `target_os = linux`/`unix` on.
|
||||
- Evidence:
|
||||
- `cfg(not(feature="server"))` items are kept, `cfg(feature="server")` items dropped.
|
||||
- Of the four `platform_id` variants, only the `cfg(all(unix, not(macos)))` one is kept.
|
||||
- `cfg(debug_assertions)` consts and fns are dropped.
|
||||
- spacecraft `update_steam_status` (`feature="steam"`, a **default** feature) is dropped.
|
||||
- `#[cfg]` on **methods inside an impl block is not evaluated.** Example: `set_client_latency` under a non-default feature is kept. [V]
|
||||
- Scale: files with zero definitions are spacecraft **212/903** (2.9 of 11.6 MB) and dirigent **176/989** (3.0 of 13.6 MB). Test files are included in those counts. The largest non-test blind spots in dirigent are `dirigent_core/src/runtime/mod.rs`, `config.rs`, `connectors/gateway/mod.rs` and `connectors/opencode.rs`, `dirigent_api/src/acp_controller.rs`, and `dirigent_web/src/server.rs`. Most of dirigent's backend sits behind `server`. [V]
|
||||
- No flag, config key or env var to enable features was found in `--help`, `orbit config list` or `strings`. [I]
|
||||
|
||||
## 8. repo-map quality and timings
|
||||
|
||||
| step | spacecraft | dirigent |
|
||||
|---|---|---|
|
||||
| `orbit index . -t 4` | **4.4–4.7 s**, 0 skipped | **5.3 s**, 0 skipped |
|
||||
| `orbit index .` (auto, 20 cores) | 7.7–8.4 s, **4–7 files skipped** (`timeout_sentinel`, "per-file watchdog killed analysis") | 5.7 s, 0 skipped |
|
||||
| `orbit index . -t 1` | 7.2–8.0 s, 0 skipped | – |
|
||||
| `repo-map` overview | 0.67 s, 132 lines | 0.62 s, 141 lines |
|
||||
| `repo-map tree <dir>` | – | 0.54 s |
|
||||
|
||||
All timings are [V]. The skipped files were 700–2000-line Bevy UI/lab files. A watchdog skip under default threads silently drops those files' definitions. The first index I saw (by another agent) had 16456 definitions, against 17110 with no skips.
|
||||
|
||||
- **`overview`: useful.**
|
||||
- Per-crate files/types/callables table.
|
||||
- "Key abstractions": LabScene 15, MovementBehaviour 6, Gamemode 3, EditTarget 3. It **misses** `ArchiveBackend` because of the cross-file impl gap.
|
||||
- "Most-imported defined symbols" (`PlayerId` 83, `InSession` 74, `UiTheme` 57) is a good anchor list for review context.
|
||||
- "Most-called callables" counts call sites and mixes in Python scripts.
|
||||
- The Languages row "unknown 2853" is noise from md/toml/assets.
|
||||
- **`tree <path>`: good.** Example: `Enum ArchiveCapability @L9 / TypeAlias CapabilitySet @L20 / Trait ArchiveBackend @L44 …`.
|
||||
- **`class <Name>`: good for structs and traits.**
|
||||
- It lists fields and methods across impl files, with locations.
|
||||
- **Signatures are one-line extracts.** Multi-line signatures truncate to `fn apply_command(` or `async fn append_messages(`, and some show only the name (`delete_session_if_unchanged`, `trash_session`).
|
||||
- **`extends <Trait>`:** correct depth tree, subject to the same-file and in-repo limits (§4).
|
||||
- **`imports <pattern>`:** counts per import path (§6).
|
||||
- **`api <path>`:** same data as `tree` plus signatures, with the same truncation. [I: not sampled separately; the signature extractor is shared with `class`]
|
||||
- **`grep`: much better than 0.124.** Match classes make exact-name hits obvious (`put_session` → trait declaration plus `JsonlBackend::put_session`, then body-only hits with the matching line). `grep --path file.rs` with no query prints an outline. It still ranks, caps with "… N more", and does not search excluded code (cfg/test). [V]
|
||||
|
||||
## 9. The guide's §7 gotchas for Rust
|
||||
|
||||
| Guide gotcha | Rust / 0.133 status |
|
||||
|---|---|
|
||||
| 7.1 worktree duplication | **Carries over** for `--all`. Worse for **MCP `run_sql`, which is always unscoped** [V]. Also new: `--all` edges are per call site, while default scope deduplicates them [V]. |
|
||||
| 7.2 dynamic calls invisible | **Much smaller in Rust.** Concrete receivers resolve at 85–100 %. It **still applies** to `dyn Trait` / `Arc<dyn …>` field receivers (0–14 %), macro arguments, Bevy systems registered by fn reference, and Dioxus `rsx!` components [V]. |
|
||||
| 7.3 imports only via statements | **Carries over.** Paths are also raw (`crate::`/`super::`/relative), not resolved, and re-exports are not followed [V]. |
|
||||
| 7.4 third-party bases not nodes | **Carries over** for external traits (silently empty) [V]. New: cross-file impls give no EXTENDS even for in-repo traits [V]. |
|
||||
| 7.5 grep ranked, not exact | **Still true for counting.** Match classes make exact-name lookups reliable. Excluded code (cfg-off, `#[cfg(test)]`) is never searched [V]. |
|
||||
| 7.6 no type/decorator semantics | **Mostly carries over.** `repo-map class` shows (truncated) signatures. Type mentions (fn params, `Query<&T>`, generics) are not edges. `#[derive]` and attributes are not modelled [V]. |
|
||||
| 7.7 stale index, silent | **Changed.** `repo-map` refuses on an unindexed commit. `sql` falls back to **all commits** with a stderr note, which is easy to miss in scripts and mixes repos [V]. `grep` printed `@ <new sha>` against an unindexed commit without a warning [V; what it searched is I]. Re-indexing the same path **replaces** the old commit's entry [V]. |
|
||||
| 7.8 ticket numbers | Not orbit-specific. Skipped. |
|
||||
|
||||
### New Rust-specific gotchas
|
||||
|
||||
1. **cfg blindness.** All features off, `test` off, `debug_assertions` off. Feature-gated modules, whole crates' backends, and all `#[cfg(test)]` unit tests have no definitions and no CALLS. A review of feature-gated code must use `git grep`/read the files. Check `gl_file` rows with zero definitions before trusting a "no callers" or "not found" result (cookbook C6). [V]
|
||||
2. **Watchdog skips under default threads.** Index with `-t 4`. Assert `processing.skipped_files == 0` in the JSON. [V]
|
||||
3. **EXTENDS only for same-file impls, in-repo traits and in-repo types.** "Implementors of X" is a lower bound. [V]
|
||||
4. **`dyn Trait` dispatch has no callers.** For trait methods, search call sites with `orbit grep '<method>'` (body-only hits) or `git grep '\.method('`. [V]
|
||||
5. **Macros.** Calls inside macro arguments are missed (`assert!`, `matches!`, `vec![seam!(..)]`, `rsx!`). Statement-level `macro_rules!` and proc-macro invocations do produce CALLS. [V]
|
||||
6. **ECS / framework wiring is not calls.** Bevy `add_systems(Update, f)`, observers and Dioxus components give no incoming edges. [V]
|
||||
7. **CALLS→Struct means construction, not usage.** "Who uses component `Health`": 4 of 52 enclosing definitions found. Use `repo-map imports` or SQL on imports, or grep. [V]
|
||||
8. **fqn root is the cargo target name, and collides** (lib vs bin `warpspace`). Integration tests are rooted at the file stem. Group by `file_path`. [V]
|
||||
9. **`start_line` includes doc comments and attributes.** This helps when mapping diff hunks to definitions, but a hunk touching only a doc comment counts as touching the item. [V]
|
||||
10. **Default scope deduplicates edges; `--all` does not.** Do not compare counts across the two. [V]
|
||||
11. **SQL papercuts:**
|
||||
- `target` is a reserved word in DuckDB (`AS target` fails); use `tgt`.
|
||||
- An empty result prints **nothing** (no header), exit 0.
|
||||
- zsh loops: don't name a variable `path`. That is a shell issue, not orbit. [V]
|
||||
12. **Shared DB and concurrent agents.** One `graph.duckdb` per user. Other sessions re-index the same repos, and `repo_path` rows are replaced. **`--db <file>` isolates cleanly.** `orbit index . -t 4 --db ~/.cache/orbit-probe/scratch.duckdb` built a 22.8 MB DB with 17110 definitions. `list`, `sql` and `repo-map extends` with the same `--db` worked, `rm` of the file was the whole cleanup, and the shared DB was untouched (still 4 entries). [V] `ORBIT_DATA_DIR` is untested. [I]
|
||||
|
||||
## 10. Multi-commit / worktree comparison
|
||||
|
||||
Steps [V]:
|
||||
|
||||
1. `git -C ~/Projects/spacecraft worktree add --detach ~/.cache/orbit-probe/wt-spacecraft bfbbcd96`. This commit is 4 commits behind `main`, with 6 `.rs` files changed (+363).
|
||||
2. `orbit index . -t 4` inside the worktree: 3.8 s, 17074 definitions. `git status --porcelain --ignored` was unchanged.
|
||||
3. `_orbit_manifest` gained a row with `repo_path=/home/g4b/.cache/orbit-probe/wt-spacecraft`, `parent_repo_path=/home/g4b/Projects/spacecraft`, its own `project_id`, and `branch='HEAD detached at bfbbcd96'`.
|
||||
4. Definition IDs are unique per commit: 0 shared IDs between the two commits. Join across commits on `(fqn, definition_type, file_path)`.
|
||||
5. `orbit sql --repo <worktree> …` and `orbit repo-map --repo <worktree> …` scope to that commit.
|
||||
6. `orbit sql --all -f defdiff.sql` (cookbook C8) listed exactly the expected changes:
|
||||
- added: `warpspace_ui::replay_browser::*` (plugin, structs, fns), `MenuAction::OpenReplay`, `MenuState::ReplayBrowser`, `warpspace::start_replay_session`, `auto_start_replay`, `Args::replay`
|
||||
- resized: `handle_menu_actions` 70→82, `main` 248→264, `spawn_main_menu_root` 181→188
|
||||
- Python script adds and removes in the same range
|
||||
7. Re-creating the worktree at another commit and re-indexing the same path **replaced** that path's entry.
|
||||
8. Cleanup: `git worktree remove` + `git worktree prune`. `git worktree list` is back to `main` + `.worktrees/strategy-mvp`.
|
||||
|
||||
What is left over:
|
||||
|
||||
- **Orbit keeps the index entry.** `orbit list` still shows `/home/g4b/.cache/orbit-probe/wt-spacecraft @ bfbbcd96`, with 17074 definitions in the DB.
|
||||
- There is **no CLI command to remove an entry**: no prune/forget/remove in `--help`. `orbit index <deleted path>` fails with "failed to find git top-level".
|
||||
- Removing the entry would need direct DuckDB surgery or a separate `--db`. I did neither, because the DB is shared.
|
||||
- Scratch files in `~/.cache/orbit-probe/` (about 250 KB of scripts and outputs) remain on purpose for phase 2. No scratch DB.
|
||||
|
||||
Implication for the skill: branch-vs-main review works well at definition level. Only length changes are detected, so pair it with `git diff -U0` hunks for body edits. Worktree entries accumulate in the shared DB, so a dedicated `--db` per review is the clean pattern. `--db` isolation and deleting the file afterwards were verified (gotcha 12). Indexing two commits into one scratch DB was not tried, but it should behave like the shared DB [I].
|
||||
|
||||
## Tested cookbook
|
||||
|
||||
All of these ran on the repos above. Default scope unless marked.
|
||||
|
||||
```bash
|
||||
# Index (always check skipped_files)
|
||||
orbit index . -t 4 | grep -E '"definitions"|"skipped_files"'
|
||||
orbit list
|
||||
|
||||
# Orientation
|
||||
orbit repo-map # overview
|
||||
orbit repo-map tree crates/dirigent_archivist/src/backend
|
||||
orbit repo-map class JsonlBackend # fields + methods across impl files
|
||||
orbit repo-map extends MovementBehaviour # same-file in-repo implementors (lower bound)
|
||||
orbit repo-map imports themed_button # counts per import path
|
||||
orbit grep put_session --kind Method --path crates/dirigent_archivist
|
||||
orbit grep --path crates/dirigent_archivist/src/backend/health.rs # outline
|
||||
orbit context Definition:<id> # source + callers/callees + test callers
|
||||
```
|
||||
|
||||
**C1. Definition types (Rust only)**
|
||||
|
||||
```sql
|
||||
SELECT d.definition_type, count(*) n
|
||||
FROM gl_definition d JOIN gl_file f ON f.path = d.file_path
|
||||
WHERE f.language = 'rust' GROUP BY 1 ORDER BY 2 DESC;
|
||||
```
|
||||
|
||||
**C2. Callers of one definition** (distinct callers under default scope; add `--all` plus a commit filter to count call sites)
|
||||
|
||||
```sql
|
||||
SELECT s.fqn AS caller, s.file_path, s.start_line
|
||||
FROM gl_edge e
|
||||
JOIN gl_definition s ON s.id = e.source_id
|
||||
JOIN gl_definition t ON t.id = e.target_id
|
||||
WHERE e.relationship_kind = 'CALLS'
|
||||
AND t.fqn = 'dirigent_web::components::toast::show_toast'
|
||||
ORDER BY s.file_path;
|
||||
```
|
||||
|
||||
**C3. Cross-crate call matrix**
|
||||
|
||||
```sql
|
||||
SELECT split_part(s.file_path,'/',2) src_crate, split_part(t.file_path,'/',2) dst_crate, count(*) n
|
||||
FROM gl_edge e JOIN gl_definition s ON s.id=e.source_id JOIN gl_definition t ON t.id=e.target_id
|
||||
WHERE e.relationship_kind='CALLS' AND s.file_path LIKE 'crates/%' AND t.file_path LIKE 'crates/%'
|
||||
AND split_part(s.file_path,'/',2) <> split_part(t.file_path,'/',2)
|
||||
GROUP BY ALL ORDER BY n DESC;
|
||||
```
|
||||
|
||||
**C4. Files importing a symbol; crate import matrix** (includes cfg-off and test files)
|
||||
|
||||
```sql
|
||||
SELECT file_path, import_type, import_path FROM gl_imported_symbol
|
||||
WHERE identifier_name = 'themed_button' ORDER BY 1;
|
||||
|
||||
SELECT split_part(i.file_path,'/',2) from_crate, split_part(i.import_path,'::',1) to_crate, count(*) n
|
||||
FROM gl_imported_symbol i
|
||||
WHERE i.file_path LIKE 'crates/%' AND i.import_path LIKE 'dirigent_%'
|
||||
AND split_part(i.file_path,'/',2) <> split_part(i.import_path,'::',1)
|
||||
GROUP BY ALL ORDER BY n DESC;
|
||||
```
|
||||
|
||||
**C5. Trait implementors via EXTENDS** (lower bound)
|
||||
|
||||
```sql
|
||||
SELECT s.fqn implementor, s.definition_type, t.name AS trait_name, s.file_path
|
||||
FROM gl_edge e JOIN gl_definition s ON s.id=e.source_id JOIN gl_definition t ON t.id=e.target_id
|
||||
WHERE e.relationship_kind='EXTENDS' AND t.definition_type='Trait' ORDER BY trait_name;
|
||||
```
|
||||
|
||||
**C6. Blind-spot check: Rust files with no definitions** (cfg-off, test-only, vendor)
|
||||
|
||||
```sql
|
||||
SELECT f.path, f.size_bytes FROM gl_file f
|
||||
WHERE f.language='rust'
|
||||
AND NOT EXISTS (SELECT 1 FROM gl_definition d WHERE d.file_path=f.path)
|
||||
ORDER BY f.size_bytes DESC LIMIT 20;
|
||||
```
|
||||
|
||||
**C7. Candidate implementors by method-name overlap** (covers cross-file impls, but noisy: it also matched the `Archivist` coordinator with 7/21)
|
||||
|
||||
```sql
|
||||
WITH tm AS (
|
||||
SELECT m.name FROM gl_definition t
|
||||
JOIN gl_definition m ON m.fqn LIKE t.fqn || '::%' AND m.definition_type='Method'
|
||||
WHERE t.definition_type='Trait' AND t.name='ArchiveBackend')
|
||||
SELECT regexp_replace(d.fqn,'::[^:]+$','') implementor, count(DISTINCT d.name) matched,
|
||||
(SELECT count(*) FROM tm) trait_methods
|
||||
FROM gl_definition d WHERE d.definition_type='Method' AND d.name IN (SELECT name FROM tm)
|
||||
GROUP BY 1 HAVING count(DISTINCT d.name) >= 3 ORDER BY matched DESC;
|
||||
```
|
||||
|
||||
**C8. Definition diff between two indexed commits** (`orbit sql --all -f defdiff.sql`; SHAs from `orbit list`)
|
||||
|
||||
```sql
|
||||
WITH a AS (SELECT fqn, definition_type, file_path, end_line-start_line+1 len
|
||||
FROM gl_definition WHERE commit_sha='<old_sha>'),
|
||||
b AS (SELECT fqn, definition_type, file_path, end_line-start_line+1 len
|
||||
FROM gl_definition WHERE commit_sha='<new_sha>')
|
||||
SELECT CASE WHEN a.fqn IS NULL THEN 'added' WHEN b.fqn IS NULL THEN 'removed' ELSE 'resized' END change,
|
||||
coalesce(b.fqn,a.fqn) fqn, coalesce(b.definition_type,a.definition_type) kind,
|
||||
coalesce(b.file_path,a.file_path) file, a.len old_len, b.len new_len
|
||||
FROM a FULL OUTER JOIN b USING (fqn, definition_type, file_path)
|
||||
WHERE a.fqn IS NULL OR b.fqn IS NULL OR a.len <> b.len ORDER BY file, fqn;
|
||||
```
|
||||
|
||||
**C9. Callables with no incoming CALLS** (triage only; Bevy systems, trait impls and handlers dominate)
|
||||
|
||||
```sql
|
||||
SELECT d.definition_type, count(*) total,
|
||||
count(*) FILTER (WHERE NOT EXISTS (SELECT 1 FROM gl_edge e
|
||||
WHERE e.relationship_kind='CALLS' AND e.target_id=d.id)) no_incoming
|
||||
FROM gl_definition d
|
||||
WHERE d.file_path LIKE '%.rs' AND d.definition_type IN ('Function','Method','AssociatedFunction')
|
||||
GROUP BY 1;
|
||||
```
|
||||
|
||||
## Open points for phase 2
|
||||
|
||||
- Blast radius across two commits: join CALLS on fqn between the worktree and main entries. Measure how much the cfg and `dyn` gaps hurt on a real diff.
|
||||
- Run a full per-review lifecycle in one scratch `--db`: index main and the branch worktree, diff, delete the file. Single-commit `--db` isolation is already verified.
|
||||
- Decide how the skill should state cfg blindness. Options: always run C6 first, or grep feature-gated files touched by the diff.
|
||||
|
||||
|
||||
## Refinement 2026-09-29 (from phase 2)
|
||||
|
||||
Phase 2 (`phase-2-review-workflows.md`) found that orbit indexes the **working tree**, including uncommitted and untracked files, but labels the index with HEAD's commit. Any claim above that an index reflects a commit exactly should be read with this in mind.
|
||||
@@ -0,0 +1,497 @@
|
||||
---
|
||||
phase: 2
|
||||
stream: review-workflows
|
||||
date: 2026-09-29
|
||||
orbit-version: 0.133.0
|
||||
depends-on: phase-1-rust-capabilities.md, phase-1-review-fit.md, phase-0-field-guide-python-django.md
|
||||
test-beds: spacecraft (2 ranges), dirigent (5 ranges), all historical, indexed from detached worktrees into per-review isolated DBs
|
||||
status: findings
|
||||
---
|
||||
|
||||
# Phase 2: Review workflows with orbit + grep (Rust)
|
||||
|
||||
Legend: **[V]** = verified here with command output and a ground-truth comparison. **[I]** = inferred, not tested directly.
|
||||
|
||||
Scripts, SQL and rules files are in `~/.cache/orbit-probe/recipes/` (full list in §11). Raw outputs for every range are in `~/.cache/orbit-probe/out/`. Nothing was compiled. The shared DB `~/.gitlab/orbit/graph.duckdb` was not written: its sha256 was the same before and after the run.
|
||||
|
||||
## TL;DR
|
||||
|
||||
- **Orbit gives a reviewer a structural map of a Rust diff that the diff text does not.** It lists which definitions were added, removed, changed in signature or touched in body; where their callers and importers are; which trait implementors must follow a trait change; and whether new crate edges appeared. A full pass indexes base and tip into a throwaway DB and runs six scripts. It takes **about 30 s** and prints **about 16 KB**, compared with a 56–411 KB raw diff. [V]
|
||||
- **Within the code orbit can see, the changed-definitions map is exact:** precision 100 % and recall 100 % across 7 ranges. **But orbit cannot see much of Rust:**
|
||||
- Of all item declarations the diffs added or removed, 27–100 % were visible depending on the range. **Every miss** was in `#[cfg(test)]` code, a `cfg(feature)` region, or a file gated at a parent `mod`.
|
||||
- The preflight measured **6 % to 71 % of changed lines** as invisible to orbit. dirigent's `server` feature hides whole connector modules.
|
||||
- **Run the blindness preflight first. It decides how much to trust everything after it.** [V]
|
||||
- **Blast radius: one level down, orbit misses half; one level up, it misses little.** Recall at **definition level** was **49 %** (129 of 265 grep-confirmed dependents). At **file level**, callers plus importers reached **93 %** (138/148). The gaps:
|
||||
- `cfg(test)` code: 36
|
||||
- enum variant literals and multi-line patterns: 32
|
||||
- Bevy systems passed as fn refs: 21
|
||||
- gated files: 17
|
||||
- `Type::default()`-style calls: 6
|
||||
- `rsx!` bodies: 4
|
||||
- `dyn` receivers: 2
|
||||
|
||||
Report orbit's callers as a **lower bound**. Close each gap with the grep named in its class. [V]
|
||||
- **The review findings that no plain `git diff` shows:**
|
||||
- **Doc drift:** a new crate edge `warpspace_replay → warpspace_sync_core` contradicts the dependency graph in AGENTS.md.
|
||||
- **Re-export shims:** 8 of 8 removed duplicate types were correctly traced to `pub use` shims that keep the old paths alive for 1–13 importers.
|
||||
- **Hidden trait implementor:** a cfg-gated `MockBackend` of a changed trait. Orbit cannot see it; the grep fallback in the trait recipe does.
|
||||
- **Stop signal:** a branch where 71 % of changed lines are feature-gated. There the preflight says to stop relying on orbit. [V]
|
||||
- **New facts about orbit:**
|
||||
- **Orbit indexes the working tree, including uncommitted and untracked files, but labels the rows with HEAD's sha.** [V]
|
||||
- `repo-map api` accepts **directory** prefixes only. A file path silently returns nothing. [V]
|
||||
- No import rows are recorded for `use` inside `#[cfg(test)] mod` blocks (0 of 621) or inside fn bodies (mostly none). Inline paths such as `other_crate::x::Y` never become import rows. [V]
|
||||
- **Feedback log:** one JSONL line per orbit query that informed a decision. Written by a tested helper, `fb.sh`, and stored by default outside git at `~/.local/share/orbit-skill/feedback.jsonl` (§9).
|
||||
|
||||
---
|
||||
|
||||
## 1. Test beds and ranges
|
||||
|
||||
| id | repo | range (base..tip) | what it is | .rs files | +/− lines |
|
||||
|---|---|---|---|---|---|
|
||||
| sc-replay | spacecraft | `21cdd1fd..bfbbcd96` (`45c50f07^..bfbbcd96`) | replay feature FL-1..3 + refactor, 13 commits | 72 | +2245/−361 |
|
||||
| sc-unify | spacecraft | `00325961..8df804e1` (`a8711df5^..8df804e1`) | "unify duplicate types / delete dead code" series | 39 | +364/−1361 |
|
||||
| dg-tokens | dirigent | `9e9374be..d3f99fb7` (merge PR #28) | session token stats | 17 | +1043/−32 |
|
||||
| dg-acp | dirigent | `0c40e871..5de6c726` (merge-base(main)..branch) | W1c live ACP branch (typed updates, generic `SessionUpdate<Id>`) | 58 | +4331/−2020 |
|
||||
| dg-archivist | dirigent | `313c0f11..22034c02` (merge) | reading pane + archivist redesign | 32 | +5033/−170 |
|
||||
| dg-trash | dirigent | `300f4319^..300f4319` | adds `ArchiveBackend::trash_session` | 17 | +1153/−120 |
|
||||
| dg-move | dirigent | `cd4e8f70^..cd4e8f70` | adds `delete_session_if_unchanged`, removes `log_stamp` from the trait | 8 | +324/−142 |
|
||||
|
||||
Every index used `-t 4`: `skipped_files=0` and `errored_files=0` on all 16 indexes. [V]
|
||||
|
||||
## 2. Per-review setup (shared by all recipes)
|
||||
|
||||
```bash
|
||||
R=~/.cache/orbit-probe/recipes
|
||||
$R/review-setup.sh ~/Projects/dirigent 300f4319^ 300f4319 dg-trash > out/dg-trash.env
|
||||
# -> DB=~/.cache/orbit-probe/dg-trash.duckdb BASE_WT=…/worktrees/dg-trash-base TIP_WT=…/worktrees/dg-trash-tip BASE_SHA TIP_SHA REPO
|
||||
...recipes...
|
||||
$R/review-teardown.sh ~/Projects/dirigent dg-trash # git worktree remove + prune + rm DB
|
||||
```
|
||||
|
||||
- **Both commits go into one isolated DB.** `orbit sql --repo <worktree> --db <db>` scopes each query to one commit, so recipes query base and tip separately and join them in Python on `(fqn, kind)`. Definition IDs differ between commits. [V]
|
||||
- **Time:** about 6 s per index with `-t 4`, plus worktree checkout. `review-pass.sh` (all six recipes) ran in **30 s** wall time on dg-tokens. [V]
|
||||
- **Orbit indexes the working tree, not the commit.** I appended a fn to a tracked file and created an untracked `.rs` file in a scratch worktree, then indexed it into a throwaway DB. Both appeared: `orbit_probe_dirty_marker` and `zz_probe_untracked.rs`. The manifest still said `commit_sha=cd4e8f70` (HEAD). [V] This has two consequences:
|
||||
- (a) An index labelled with sha X proves nothing if the tree was dirty. The preflight checks `git status --porcelain`.
|
||||
- (b) An **uncommitted** diff, for example Athanor output before commit, can be reviewed by indexing the live checkout into an isolated `--db` as "tip". [I: tested only in a scratch worktree, not on a live checkout] Phase 1's statement that "uncommitted edits are not visible" is corrected.
|
||||
- **Do not reuse the shared DB for review.** Its per-repo row is replaced by whichever agent indexed last, and is often stale (phase 1).
|
||||
|
||||
---
|
||||
|
||||
## 3. Recipe 1: Changed definitions
|
||||
|
||||
**Answers:** which definitions the diff added, removed, moved, changed in signature, or touched only in body, with a one-line signature delta and field/variant membership changes. Also which changed lines no definition covers; that tally feeds recipe 5.
|
||||
|
||||
**Command:** `python3 $R/defdiff.py out/<id>.env --json out/<id>.defdiff.json`
|
||||
|
||||
**How it works:**
|
||||
1. `git diff -U0 -M base tip -- '*.rs'` gives changed line numbers on both sides.
|
||||
2. Each line maps to its innermost orbit definition. Fields and variants roll up to their parent type; `Module` is used only if nothing smaller covers the line.
|
||||
3. Definitions are matched across commits on `(fqn, kind)`, same file first, following renames.
|
||||
4. The signature is re-read from the source: the declaration up to `{` or `;`, skipping docs and attributes. It is compared as normalised text.
|
||||
5. Children of an added or removed type collapse into `(+N members)`.
|
||||
6. Uncovered lines are classified as cfg-test, cfg-feature, cfg-other, file-has-no-defs, import, impl-header/brace or blank/comment.
|
||||
|
||||
A single-hunk manual variant is in `sql/hunk-to-defs.sql`.
|
||||
|
||||
**Sample output** (sc-replay, excerpt):
|
||||
```
|
||||
crates/warpspace_debug/src/bin/ui_main.rs
|
||||
S Function spawn_embedded_server_on_menu L234-273
|
||||
sig: …on<Res<EmbeddedServer>>, [--]{+replay_path: Option<Res<ReplayPath>>, +})
|
||||
+ Struct ReplayPath L47-49
|
||||
~ Struct Args L13-45 29->33 lines members +replay
|
||||
crates/warpspace_domains/src/orders/core.rs
|
||||
~ Enum orders::core::OrderOrigin L155-173 15->19 lines members +External.peer_debug +External.player -External.0
|
||||
src/sync/game_record.rs
|
||||
S Constant sync::game_record::FORMAT_VERSION L413-418
|
||||
sig: …st FORMAT_VERSION: u32 = [-5-]{+6+};
|
||||
```
|
||||
The `OrderOrigin::External` change from a tuple variant to a struct variant is visible at a glance. It forced edits in 35 test files, none of which orbit can see (recipe 5).
|
||||
|
||||
**Measured against ground truth** (`gt_items.py`). Ground truth is a regex extraction of item declarations (`fn|struct|enum|trait|const|static|type|mod|macro_rules!`) from both file versions; the table compares added and removed items per file:
|
||||
|
||||
| range | added: text / orbit / recall | removed: text / orbit / recall | precision | every miss is in… |
|
||||
|---|---|---|---|---|
|
||||
| sc-replay | 87 / 78 / 90 % | 3 / 1 / 33 % | 100 % | cfg-test (5), file with no defs (6: `warpspace_testing/src/tests/*`) |
|
||||
| sc-unify | 5 / 5 / 100 % | 79 / 71 / 90 % | 100 % | cfg-test (8) |
|
||||
| dg-tokens | 40 / 15 / 38 % | 0 | 100 % | cfg-test (23), cfg-feature (2) |
|
||||
| dg-acp | 152 / 41 / 27 % | 32 / 2 / 6 % | 100 % | cfg-test (109), file gated by `server` (32) |
|
||||
| dg-archivist | 189 / 104 / 55 % | 0 | 100 % | gated file (44), cfg-test (40), cfg-feature (1) |
|
||||
| dg-trash | 52 / 37 / 71 % | 1 / 0 | 100 % | cfg-test (13), cfg-feature (3) |
|
||||
| dg-move | 14 / 5 / 36 % | 5 / 4 / 80 % | 100 % | cfg-test (6), gated file (3), cfg-feature (1) |
|
||||
|
||||
- **Precision is 100 % everywhere, and recall on orbit-visible code is 100 %.** Every missed item fell into a cfg or gated category. [V]
|
||||
- **Signature detection** was checked against fn declaration lines that change on both sides of the diff:
|
||||
- Every one was either reported as `S`, or sat in gated/test code (dg-acp: 9 in `server`-gated or test code).
|
||||
- Two single-line fns (`new`, `world`) whose declaration line changed only in the body were correctly reported as body-only (`~`). [V]
|
||||
- **Moves** (`>`, same fqn in another file) did not occur in any range. The logic is untested. [I] Moves across crates change the fqn and show as `-`/`+`. The script annotates a `+` with "same name removed in …" when the name and kind match.
|
||||
|
||||
**Blind spots:**
|
||||
- Everything under cfg-off: all features off, `test` off, `debug_assertions` off.
|
||||
- Files gated at a parent `mod`.
|
||||
- Changes to `impl` headers or where-clauses. There is no Impl node, so the line is unattributed.
|
||||
- Changes to `#[derive]` or attributes count as touching the item, because `start_line` includes attributes.
|
||||
- Signature text over 12 lines is cut.
|
||||
|
||||
**Output size:** 2.5–12.7 KB, which is **3–8 % of the raw diff** (sc-replay 12.7 KB against 214 KB; dg-acp 12.3 KB against 411 KB). About 1–3.5k tokens. Runtime about 1.5 s. [V]
|
||||
|
||||
## 4. Recipe 2: Blast radius
|
||||
|
||||
**Answers:** for each changed definition (`S`, `-`, `>`, and `~` with member changes), who depends on it:
|
||||
- CALLS callers. For a type, this includes callers of its methods, associated fns and variant constructors.
|
||||
- Importing files.
|
||||
- EXTENDS implementors, for traits.
|
||||
|
||||
Dependents are split into **in-diff** and **OUTSIDE diff**. The outside ones are what a reviewer has to check. For removed definitions it also reports:
|
||||
- whether a same-name definition survives elsewhere, and
|
||||
- whether the tip still imports the old path. If so, is the path **kept alive by a re-export**, or does it dangle?
|
||||
|
||||
**Command:** `python3 $R/blast.py out/<id>.env out/<id>.defdiff.json [--measure]`. Standalone SQL: `sql/dependents.sql`, `sql/importers.sql`.
|
||||
|
||||
**Sample output** (dg-acp, then sc-unify):
|
||||
```
|
||||
~ Enum conversation::MessagePart [crates/dirigent_protocol/src/conversation.rs] callers=16 outside=12 importers=37 outside=17
|
||||
<- crates/dirigent_protocol/src/merge.rs:447 apply_to_message
|
||||
<- crates/dirigent_archivist/src/record_upgrade.rs:511 unreadable_placeholder
|
||||
… 6 more outside callers
|
||||
- Struct info::UnitInfo [crates/warpspace_combat/src/info.rs] callers=0 outside=0 importers=9 outside=9
|
||||
= same-name Struct survives at tip in: crates/warpspace_core/src/info_vocab.rs
|
||||
= old path '…info::UnitInfo' kept alive by re-export at crates/warpspace_combat/src/lib.rs:17 (8 importers use it)
|
||||
```
|
||||
|
||||
**Measured** (`--measure`). Ground truth was `git grep` at the same commit, with these rules:
|
||||
- Method names are matched as `.name`/`::name`, associated fns as `Type::name`.
|
||||
- A site counts only if its file can see the name: it imports the name, glob-imports the owner module, is the defining file, or uses a qualified `owner::name`.
|
||||
- Import lines, type mentions, impl headers, pattern matches and const reads are excluded. These produce no edge by design, and are not "callers".
|
||||
- Names defined more than once at that commit were skipped as ambiguous (66 targets, mostly sc-unify's duplicates).
|
||||
|
||||
| range | dependents (grep ∪ orbit) | orbit found | def-level recall | file-level (callers ∪ importers) |
|
||||
|---|---|---|---|---|
|
||||
| sc-replay | 41 | 15 | 37 % | 27/31 = 87 % |
|
||||
| sc-unify | 13 | 5 | 38 % | 10/11 = 91 % |
|
||||
| dg-tokens | 5 | 2 | 40 % | 2/2 |
|
||||
| dg-acp | 179 | 91 | 51 % | 80/84 = 95 % |
|
||||
| dg-archivist | 19 | 8 | 42 % | 13/14 |
|
||||
| dg-trash | 3 | 3 | 100 % | 3/3 |
|
||||
| dg-move | 5 | 5 | 100 % | 3/3 |
|
||||
| **total** | **265** | **129** | **49 %** | **138/148 = 93 %** |
|
||||
|
||||
**Why orbit missed a dependent, with the grep that closes each gap** (136 misses, all [V]):
|
||||
|
||||
| gap class | n | example | fallback |
|
||||
|---|---|---|---|
|
||||
| inside `#[cfg(test)]` | 36 | `game_record.rs:2437 RecordCounters::default()` | `git grep -nw Name` limited to test modules; read the test module |
|
||||
| enum variant literal / multi-line match pattern | 32 | `SessionUpdateVariant::AgentMessageChunk(chunk)` | `git grep -nE 'Enum::\w+'`. For an **added variant**, find every `match` on the enum: exhaustiveness is invisible to orbit |
|
||||
| fn passed by reference (Bevy system, callback) | 21 | `app.add_systems(FixedFirst, conductor_gate_system);` | `git grep -nw fn_name`. For a Bevy system a changed signature is harmless, so ask "is it still registered?" |
|
||||
| file gated at parent `mod` / tests dir | 17 | `connectors/opencode.rs` (`server` feature) | read the file; `git grep` |
|
||||
| `Type::default()`, derive/trait associated calls | 6 | `RecordCounters::default()` | `git grep -n 'Type::'` |
|
||||
| other (struct literal `Type { … }`, return type in signature) | 12 | `let runner = HeadlessRunner {` | `git grep -nw Type` |
|
||||
| inside a multi-line `rsx!` body | 4 | `let title = format_token_title(` in a `rsx!{}` | `git grep -nw` |
|
||||
| cfg-other / cfg-feature | 6 | `matches!(&err, ArchivistError::LogUnreadable …)` | read the region |
|
||||
| method on an unresolved receiver (dyn/field/closure) | 2 | `.translate_notification(…)` | `git grep -n '\.method('` |
|
||||
|
||||
- **Orbit-only dependents** (19 in dg-acp, 7 in sc-replay) are real. For a changed type, they are callers of its methods, which a grep for the type name cannot find. Examples: `HeadlessRunner <- fast_forward_to`, `ArchivistError <- backfill_connector_sessions`. For types, orbit and grep complement each other. [V]
|
||||
- **The re-export check was 8/8 correct** on sc-unify. All 8 removed types whose old path the tip still imports are kept alive by `pub use`. I checked 5 by hand, for example `input/runtime.rs:14 pub use warpspace_core::input_vocab::{ActionState, Actions, InputRuntime};`. A naive "name still imported" check would have flagged all 8 as dangling. [V]
|
||||
|
||||
**Output size:** brief mode (no `--measure`) is 0.9–5 KB for feature ranges and 14 KB for sc-unify's 48 removal targets. Runtime about 5 s. [V]
|
||||
|
||||
## 5. Recipe 3: Trait-impl coverage
|
||||
|
||||
**Answers:** for each trait the diff touches (the trait itself, or a method whose parent is a trait):
|
||||
- its methods at tip, and which were added, removed or changed in signature;
|
||||
- every implementor, from three sources:
|
||||
- `E`: EXTENDS; only same-file impls.
|
||||
- `G`: `git grep 'impl … Trait … for Type'`. Text, so it also sees gated and test impls.
|
||||
- `H`: method-name overlap of at least 60 %.
|
||||
- per implementor, how many trait methods it defines and which the diff touched;
|
||||
- for implementors orbit cannot see, a **text check that the new method exists in the impl file**.
|
||||
|
||||
**Command:** `python3 $R/traitcov.py out/<id>.env out/<id>.defdiff.json [--trait Name] [--all-traits]`. Standalone SQL: `sql/implementors.sql` (E ∪ H).
|
||||
|
||||
**Sample output** (dg-trash):
|
||||
```
|
||||
Trait dirigent_archivist::backend::traits::ArchiveBackend [crates/dirigent_archivist/src/backend/traits.rs:33] methods=21 +['trash_session']
|
||||
[EGH] JsonlBackend defines 20/21 touched:trash_session
|
||||
[G ] MockBackend type not indexed (external/gated) impl-in: crates/dirigent_archivist/src/backend/mock.rs cfg-test new method(s) ['trash_session'] found by text in impl file (orbit can't see it)
|
||||
```
|
||||
`MockBackend` lives in a file gated by `#![cfg(any(test, feature = "test-utils"))]`. `git show 300f4319:…/mock.rs` confirms `async fn trash_session` at line 369. `20/21` means `as_search`, a default method, is not overridden. That is expected, not a finding. [V]
|
||||
|
||||
**Measured.** Over every in-repo trait at tip, grep impls were taken as ground truth (`--all-traits`):
|
||||
|
||||
| repo @ tip | grep impls | found by E or H | only grep: gated | only grep: type not indexed | only grep: cross-file/other | H false positives | E found, grep regex missed |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| dirigent @ 5de6c726 (30 traits) | 67 | 41 (61 %) | 20 | 5 | 1 | 8 (facades such as `Archivist`, `ServerRuntime`) | 0 |
|
||||
| spacecraft @ bfbbcd96 (18 traits) | 42 | 33 (79 %) | 1 | 4 | 4 | 1 | 1 (multi-line impl header) |
|
||||
|
||||
- Diff-scoped runs caught the one new trait in sc-replay (`ReplayHarnessAssertions`). Its only impl sits in another file (`[G]`), so EXTENDS has no edge; the text check found it.
|
||||
- sc-unify removed the trait `AbilityLike`. The recipe prints the grep that checks for leftover impls. [V]
|
||||
|
||||
**Blind spots:**
|
||||
- External traits (`Plugin`, `Serialize`, …) are not nodes, so there is no trait to anchor on.
|
||||
- Blanket impls.
|
||||
- `impl` on external types (`impl Ext for App`).
|
||||
- The H heuristic cannot tell a delegating facade from an implementor. Confirm H-only rows by reading.
|
||||
- The G regex misses multi-line `impl` headers.
|
||||
|
||||
**Output size:** under 1 KB per touched trait. [V]
|
||||
|
||||
## 6. Recipe 4: Module and crate boundary guards
|
||||
|
||||
**Answers:**
|
||||
- new crate→crate import edges (tip minus base, whole repo);
|
||||
- new import rows in changed files, checked against a per-repo **rules file** (allow-lists and deny patterns);
|
||||
- new intra-crate module edges (`crate::a → crate::b`);
|
||||
- two grep fallbacks, because imports alone miss edges:
|
||||
- (a) inline qualified paths (`other_crate::x::Y` with no `use`);
|
||||
- (b) **Cargo.toml dependency changes**, checked against the allow-list.
|
||||
|
||||
`--audit` checks every tip import against the rules. Use it to validate the rules themselves.
|
||||
|
||||
**Command:** `python3 $R/boundary.py out/<id>.env out/<id>.defdiff.json $R/rules/<repo>.rules [--audit] [--measure]`. Standalone SQL: `sql/crate-edges-delta.sql` (run with `--all`).
|
||||
|
||||
**Boundaries inferred from the repos:**
|
||||
- **spacecraft** (`rules/spacecraft.rules`):
|
||||
- allow-lists per crate, from AGENTS.md "Dependency Graph";
|
||||
- `warpspace_core` must not import domains/combat/hero/client (`crates/warpspace_core/CLAUDE.md`: "Dependency floor");
|
||||
- `warpspace_combat` must not import hero/client (`crates/warpspace_combat/CLAUDE.md`).
|
||||
- **dirigent** (`rules/dirigent.rules`):
|
||||
- `dirigent_acp_api` must not depend on `dirigent_core` or `dirigent_archivist` (acp_api and core CLAUDE.md);
|
||||
- `dirigent_logs` depends on no dirigent crate;
|
||||
- `dirigent_diff` has no Dioxus;
|
||||
- `dirigent_web` must not import `OpenCodeClient` (docs/architecture/rules.md);
|
||||
- `dirigent_appservice` has no workspace path deps.
|
||||
|
||||
**Sample output** (sc-replay):
|
||||
```
|
||||
new crate edges: 1
|
||||
+ warpspace_replay -> warpspace_sync_core (4 imports, e.g. crates/warpspace_replay/src/speed.rs)
|
||||
crate edges seen only as inline paths in changed files (no import row):
|
||||
~ warpspace_replay -> warpspace_protocol e.g. crates/warpspace_replay/src/harness.rs:583 [NEW: not in base imports or base inline paths]
|
||||
Cargo.toml workspace-dep changes: ['+warpspace_replay in crates/warpspace_debug/Cargo.toml', '+warpspace_sync_core in crates/warpspace_replay/Cargo.toml', '+warpspace_protocol in …']
|
||||
!! warpspace_replay gains dependency warpspace_sync_core, not in documented allow-list
|
||||
new intra-crate module edges: 3
|
||||
+ warpspace_client: interaction -> spectator
|
||||
```
|
||||
|
||||
**Findings:**
|
||||
- **Real finding, verified:** the replay feature added `warpspace_sync_core` and `warpspace_protocol` to `warpspace_replay`'s Cargo.toml. AGENTS.md still lists replay's deps as `(warpspace, domains, combat, client, engine)`. That is documentation drift a reviewer should flag. [V]
|
||||
- `warpspace_protocol` is used only through inline paths such as `warpspace_protocol::ids::UnitNetId`. The import-based edge check **missed** it; the inline-path scan caught it. [V]
|
||||
|
||||
**Measured:**
|
||||
- **Rule audits at tip:**
|
||||
- spacecraft: 0 violations with the rules as documented, and 4 once replay's documented allow-list is added. Those 4 are exactly the drift above.
|
||||
- dirigent: 7 violations, all false: `dirigent_appservice` importing its own lib by crate name. I fixed this by skipping `dst == src`, but did not re-run after the fix. [I: post-fix count expected 0]
|
||||
- **New-import detection, file level:** compared with files whose diff adds a `use` line, recall per range was 100 % (sc-replay), 85 % (sc-unify), 75 % (dg-tokens), 91 % (dg-acp), 90 % (dg-trash) and 100 % (dg-archivist). [V] The misses were:
|
||||
- `use` inside fn bodies: dg-tokens `archivist.rs` (a `use` inside a `cfg(feature)` fn), dg-move `coordinator/tests.rs`;
|
||||
- `use` lines that were only reformatted (sc-unify).
|
||||
- **Which `use` statements become import rows** (dirigent @ cd4e8f70, all 5,547 `use` lines):
|
||||
|
||||
| where the `use` line sits | recorded |
|
||||
|---|---|
|
||||
| visible code, module level | 2870/3080 (93 %; most misses are probably multi-line groups whose row line differs [I]) |
|
||||
| visible code, nested (fn body or inline mod) | 337/357 |
|
||||
| inside `#[cfg(test)]` regions | **0/621** |
|
||||
| files gated at a parent `mod`, top level | 843/897 |
|
||||
| files gated at a parent `mod`, nested | **0/350** |
|
||||
| `cfg(feature)` / other regions | 192/289 |
|
||||
|
||||
This refines phase 1's "imports are kept in gated files": that holds at top level of file-gated modules, but not inside `#[cfg(test)] mod` blocks. [V]
|
||||
|
||||
**Blind spots:**
|
||||
- Inline qualified paths. The text scan covers them only in changed files.
|
||||
- `super::`/relative paths are attributed to the file's own crate.
|
||||
- Re-exports are not followed, so a dependency through a re-exporting crate shows as an edge to that crate.
|
||||
- The compiler already enforces the Cargo graph. The guard is valuable for **documented** rules beyond Cargo (doc drift, intra-crate layering, "web must not use OpenCodeClient") and for review awareness ("first import from X to Y").
|
||||
|
||||
**Output size:** 1–2 KB. [V]
|
||||
|
||||
## 7. Recipe 5: Blindness preflight
|
||||
|
||||
**Answers:** before any orbit answer is trusted, what can orbit not see **for this diff**?
|
||||
1. **Index freshness:** manifest sha equals worktree HEAD equals the requested sha, `status=indexed`, the tree is clean, and `skipped_files`/`errored_files` are 0. A dirty tree matters because orbit indexes the working tree (§2).
|
||||
2. **Changed files with no non-macro definitions,** with size and a reason. The script walks up the `mod` chain to the gating `#[cfg]`. `macro_rules!` are indexed even in unreachable files, so they are excluded from the count.
|
||||
3. **Share of changed lines** in cfg-test, cfg-feature, other cfg regions, or gated files.
|
||||
4. **Cargo features of the touched crates.** Orbit evaluates all features off, default features included.
|
||||
5. **Added lines with constructs CALLS misses:** `add_systems`/observers, `rsx!`, `dyn`, and macro arguments.
|
||||
|
||||
**Command:** `python3 $R/preflight.py out/<id>.env out/<id>.defdiff.json`. Standalone SQL: `sql/blind-files.sql`.
|
||||
|
||||
**Sample output** (dg-acp, excerpt):
|
||||
```
|
||||
[OK] index tip: manifest=5de6c726 head=5de6c726 dirty=0 skipped=0 errored=0
|
||||
[PARTIAL] 15/58 changed .rs files have NO definitions in orbit (read/grep them):
|
||||
crates/dirigent_core/src/connectors/acp/connector.rs (238 KB) — gated at crates/dirigent_core/src/lib.rs:75 by cfg(feature = "server")
|
||||
crates/dirigent_core/src/runtime/mod.rs (176 KB) — gated at crates/dirigent_core/src/lib.rs:58 by cfg(feature = "server")
|
||||
crates/dirigent_protocol/src/adapters/opencode.rs (57 KB) — gated at crates/dirigent_protocol/src/adapters/mod.rs:5 by cfg(feature = "adapters")
|
||||
[PARTIAL] 4539/6351 changed lines (71%) are in code orbit cannot see; unattributed: cfg-test=3176, file-has-no-defs=1363, import=25, …
|
||||
[INFO] added lines with CALLS-blind constructs: assert!/matches!/vec! args (missed calls): 302
|
||||
```
|
||||
|
||||
**Measured** [V]:
|
||||
|
||||
| range | changed .rs files with no defs | changed lines orbit cannot see |
|
||||
|---|---|---|
|
||||
| sc-unify | 2/39 | 111/1725 = 6 % |
|
||||
| dg-archivist | 3/32 | 1192/5203 = 23 % |
|
||||
| dg-trash | 4/17 | 290/1273 = 23 % |
|
||||
| sc-replay | 35/72 (all `warpspace_testing/src/tests/*`) | 733/2606 = 28 % |
|
||||
| dg-tokens | 3/17 | 373/1075 = 35 % |
|
||||
| dg-move | 3/8 | 191/466 = 41 % |
|
||||
| dg-acp | 15/58 | 4539/6351 = **71 %** |
|
||||
|
||||
- **Reason lookup:** the reason given for each blind file matched the actual gate in every case I checked. Examples:
|
||||
- `connector.rs` → `lib.rs:75 #[cfg(feature = "server")] pub mod connectors;`
|
||||
- `dirigent_web/src/server.rs` → `main.rs:27`
|
||||
- The cfg-region scanner is text-based (brace matching). It handled struct-field-level `#[cfg]`, multi-line fn signatures and `#![cfg]` file attributes after two fixes, and gives the same categories as the definition-based check. [V]
|
||||
|
||||
**Output size:** 1.3–2.8 KB. [V]
|
||||
|
||||
## 8. Recipe 6: Compact context for the reviewer
|
||||
|
||||
**Answers:** the shape of the touched area as one line per type or callable, marked with change ops, within a byte budget. Changed items always show. Unchanged items show only for small files; otherwise they collapse to a count and a pointer (`orbit grep --path <file>`). Non-test files come first.
|
||||
|
||||
**Command:** `python3 $R/context.py out/<id>.env out/<id>.defdiff.json [--budget 8000]`
|
||||
|
||||
```
|
||||
## crates/dirigent_archivist/src/coordinator/meta.rs (4 changed / 18 items)
|
||||
~ pub async fn cleanup_empty_sessions(&self, archive: Option<String>) -> Result<(usize, usize)> L456
|
||||
+ pub async fn storage_usage(&self) -> Vec<ArchiveStorageUsage> L583
|
||||
… 14 unchanged items (orbit grep --path crates/dirigent_archivist/src/coordinator/meta.rs)
|
||||
## crates/dirigent_archivist/src/backend/traits.rs (2 changed / 54 items)
|
||||
+ async fn trash_session(&self, scroll_id: Uuid, trash_dir: &TrashDir) -> Result<TrashOutcome>; L117
|
||||
```
|
||||
|
||||
**Measured and compared** [V]:
|
||||
- `context.py` produced 5.3 KB (dg-trash), 8.3 KB (dg-tokens), 9.9 KB (sc-replay) and 10.5 KB (dg-acp) at budget 8000.
|
||||
- `orbit repo-map api` over the **directories** touched by dg-trash is **206 KB even after squeezing table padding**. The raw output is about 3× larger.
|
||||
- `repo-map api <file.rs>` returns `(no matching definitions under <file>/)`. It appends `/` and accepts directory prefixes only; use `orbit grep --path <file>` for a single-file outline. [V]
|
||||
- **Signature quality in `repo-map api`** (754 entries in `dirigent_archivist/src`, 367 in `dirigent_protocol/src`):
|
||||
- full signature: 71 % and 92 %;
|
||||
- truncated at `(`: 18 % and 3 %;
|
||||
- name only: 9 % and 4 %;
|
||||
- **garbage**: 2 % and 0 %. These are `let …` lines shown as signatures, e.g. `Function let base = format!("{stamp}-{suffix}"); [L438]`.
|
||||
|
||||
`context.py` re-reads declarations from source and is not affected.
|
||||
- `orbit repo-map class <Trait>` is a good one-off for a touched trait: 21 methods with signatures. Two multi-line signatures showed only the name (`trash_session`, `replace_session_messages`).
|
||||
|
||||
**Output size:** 5–10.5 KB at the default budget (about 1.3–2.6k tokens). [V]
|
||||
|
||||
---
|
||||
|
||||
## 9. End-to-end "orbit review pass"
|
||||
|
||||
This is written for Spire's `search` agent or an ad-hoc Opus reviewer. One command runs steps 1–7: `review-pass.sh <repo> <base> <tip> <name> [rules]`. It was tested at 30 s and 16 KB for dg-tokens.
|
||||
|
||||
1. **Pick the range.** For a branch, use `base = git merge-base main <branch>` and `tip = <branch>`. For an uncommitted Athanor diff, use tip = the working tree: index the checkout into an isolated DB, noting the dirty flag. For a merged PR, use `<merge>^1..<merge>`.
|
||||
2. **Set up:** `review-setup.sh`. **Stop condition:** `skipped_files > 0` or `errored_files > 0` → re-index. If it persists, list the skipped files as blind.
|
||||
3. **Preflight (recipe 5).** Decide the trust level:
|
||||
- blind share under about 25 % → orbit-led review with grep on the listed gaps;
|
||||
- 25–60 % → orbit for the visible part only; read the blind files directly;
|
||||
- over 60 % (dg-acp) → **stop using orbit for this review** except for recipes 1 and 4. Say so in the report.
|
||||
4. **Changed definitions (recipe 1).** Compare against the plan: is every planned item present, and is anything present that was not planned? Read every `S` row, every removed row, and every enum or struct with member changes.
|
||||
5. **Blast radius (recipe 2)** for `S`/`-`/member-changed rows. For each OUTSIDE-diff dependent, ask whether it needed a change it didn't get. Then run the grep for each gap class the preflight flagged:
|
||||
- **added enum variant** → grep every `match` on it;
|
||||
- **changed Bevy system signature** → check it is still registered;
|
||||
- **removed item** → confirm the old path is re-exported or gone.
|
||||
6. **Trait coverage (recipe 3)** for touched traits. Every implementor marked `!!` or `[G]` gets a read.
|
||||
7. **Boundary guard (recipe 4)** with the repo's rules file. Also compare any new crate edge or Cargo.toml dependency against the documented graph.
|
||||
8. **Context (recipe 6)** goes into the reviewer brief together with the plan.
|
||||
9. **Report:**
|
||||
- **Fact** = something orbit states and grep or a file read confirmed. Examples: "`X` signature changed", "implementor `Y` does not define new method `m` (checked in `file:line`)".
|
||||
- **Hypothesis** = any orbit count or absence ("no other callers", "no implementors", "no boundary crossing") that has not been grep-closed. Label it "lower bound (orbit)" and name the gap classes that apply.
|
||||
- **Blind** = the preflight's blind files and share, stated once at the top of the review.
|
||||
- Log one feedback record per orbit result that changed a decision, and one per result that proved wrong (§10).
|
||||
10. **Teardown:** `review-teardown.sh <repo> <name>`. Check with `git worktree list` and `pgrep -a orbit`.
|
||||
|
||||
**Stop conditions** (any one stops the pass):
|
||||
- the preflight blind share is above 60 %;
|
||||
- the index is not fresh or not clean;
|
||||
- the plan's key items are all in blind files;
|
||||
- the question is behavioural (runtime ordering, Bevy schedule, async). Orbit shows structure, not runtime.
|
||||
|
||||
## 10. Feedback record format (for the prototype skill)
|
||||
|
||||
**Format:** append-only **JSONL**, one line per orbit query whose result was used, or should have been used, in a decision. It must be cheap to write: one shell call, and the helper fills in the context.
|
||||
|
||||
```json
|
||||
{"ts":"2026-09-29T18:42:48+02:00","repo":"dirigent","commit":"d3f99fb7","dirty":false,"orbit":"0.133.0",
|
||||
"agent":"search","recipe":"blast","purpose":"callers of format_token_title","cmd":"blast.py dg-tokens",
|
||||
"rows":0,"verdict":"grep-needed","gap":"rsx","note":"both call sites inside rsx! block; git grep found them"}
|
||||
```
|
||||
|
||||
| field | required | meaning |
|
||||
|---|---|---|
|
||||
| ts, repo, commit, dirty, orbit | auto | filled in by the helper. `repo` is resolved through `git-common-dir`, so worktrees log under the main repo name |
|
||||
| agent | env `ORBIT_AGENT` | `search`, `reviewer`, `researcher`, … |
|
||||
| recipe | yes | `defdiff`, `blast`, `traitcov`, `boundary`, `preflight`, `context`, `adhoc-sql`, `grep`, `repo-map`, `context-cmd` |
|
||||
| purpose | yes | the question in a few words |
|
||||
| cmd | yes | the orbit command or script call |
|
||||
| rows | yes | result size (rows, or items reported) |
|
||||
| verdict | yes | `useful`, `wrong`, `incomplete`, `grep-needed`, `noise`, `unused` |
|
||||
| gap | optional | `cfg-test`, `cfg-feature`, `gated-file`, `dyn`, `fn-ref`, `macro`, `rsx`, `cross-file-impl`, `external-trait`, `inline-path`, `pattern-match`, `stale-index`, `sig-truncated`, `other` |
|
||||
| note | optional | one sentence. Say what grep or reading found that orbit didn't, or vice versa |
|
||||
|
||||
**Helper:** tested; it appended exactly the record above.
|
||||
```
|
||||
ORBIT_AGENT=search ~/.cache/orbit-probe/recipes/fb.sh blast grep-needed "callers of format_token_title" "blast.py dg-tokens" 0 "both call sites inside rsx! block; git grep found them" rsx
|
||||
```
|
||||
|
||||
**Where it lives** (options):
|
||||
1. **Default: `~/.local/share/orbit-skill/feedback.jsonl`.** Central, outside every repo, and survives cache cleaning. One file keeps mining across spacecraft and dirigent to a single `jq` call. `ORBIT_FEEDBACK` overrides it.
|
||||
2. **Per repo: `<repo>/local/orbit-feedback.jsonl`.** `local/` is git-ignored in both spacecraft (`.gitignore:22 /local/**`) and dirigent (`local/.gitignore: *`). It is visible to the user in the project. Worktrees have no `local/`, so the helper would have to resolve to the main checkout.
|
||||
3. **Tracked:** raw logs are not recommended. The mined **summaries** belong in `reliquary/docs/workpad/research/…/` (for example a later "phase-4 field feedback" report).
|
||||
|
||||
**Mining** [I: not built]. Example: `jq -s 'group_by(.recipe, .verdict) | map({recipe: .[0].recipe, verdict: .[0].verdict, n: length})' feedback.jsonl`, and `group_by(.gap)` for the gap histogram.
|
||||
|
||||
## 11. Questions only real usage can answer (the feedback log should capture these)
|
||||
|
||||
1. **Does the pass change review outcomes?** How often does an OUTSIDE-diff dependent (recipe 2) or an `[G]`/`!!` implementor (recipe 3) lead to a real finding, compared with noise? (`verdict=useful` vs `noise` per recipe)
|
||||
2. **Blast radius on compiled code:** since the tip compiles, how often does "caller outside the diff" matter at all? When it does, is it cfg-gated code the compiler never built? Real usage can record builds the probe was not allowed to run.
|
||||
3. **Which gap classes dominate in day-to-day diffs,** as opposed to these 7 historical ranges? In particular: Bevy fn refs in spacecraft, `rsx!` in dirigent_web, `server` gating in dirigent_core. (`gap` histogram)
|
||||
4. **Where to set the preflight thresholds:** 25 % and 60 % are guesses. Record the blind share and whether the review still found orbit useful.
|
||||
5. **Uncommitted review:** does indexing the live checkout (dirty working tree) into an isolated DB work reliably while other sessions edit the same checkout? Are there races with half-written files? (`dirty=true` records)
|
||||
6. **Is the boundary rules file worth maintaining?** How often does it catch drift, compared with how often it needs editing? (`recipe=boundary`)
|
||||
7. **Token budget:** is about 16 KB of pass output the right size for an Opus reviewer? Which sections do reviewers actually quote? (`verdict=unused` on sections)
|
||||
8. **Agent fit:** can Spire's Sonnet `search` agent run the pass and relay only the facts and hypotheses, or does the reviewer need the raw output? (`agent` field)
|
||||
9. **Moves and renames:** `>` is untested. Do real refactors produce same-fqn moves, or mostly cross-crate `-`/`+` pairs?
|
||||
10. **Feature-on indexing:** if orbit ever gains a features flag, how much of the blind share goes away? Until then, is a text-only fallback good enough for `server`-gated dirigent code?
|
||||
11. **Staleness in the wild:** how often do reviewers hit a stale or dirty index in practice? (`gap=stale-index`)
|
||||
|
||||
## 12. Corrections and additions to phase 1
|
||||
|
||||
- **Orbit indexes the working tree** (uncommitted edits and untracked non-ignored files), and the manifest records HEAD's sha. Phase 1's "uncommitted edits are not visible [I]" is wrong. [V]
|
||||
- **Imports inside `#[cfg(test)] mod` blocks are not recorded** (0/621). Neither are nested `use` statements in gated files (0/350). Phase 1's "imports kept in gated files" holds only for top-level `use` in files gated at the parent `mod`. [V]
|
||||
- **Inline qualified paths are not import rows.** Crate-edge matrices from `gl_imported_symbol` undercount. [V]
|
||||
- **`repo-map api` needs a directory prefix.** A file path returns empty silently. About 2 % of its "signatures" are `let` statements. [V]
|
||||
- **No CALLS edges target `Constant`/`Static`.** Constant reads are not edges. CALLS targets seen at tip: Function, Struct, Variant, Method, Field, AssociatedFunction, Macro, plus Python kinds. [V]
|
||||
|
||||
## 13. Scripts and files (`~/.cache/orbit-probe/recipes/`)
|
||||
|
||||
| file | recipe | purpose |
|
||||
|---|---|---|
|
||||
| `review-setup.sh` | setup | detached worktrees for base and tip, index both with `-t 4 --db <name>.duckdb`, assert `skipped_files==0`, print the env file |
|
||||
| `review-teardown.sh` | cleanup | `git worktree remove` for both, `prune`, delete the DB |
|
||||
| `review-pass.sh` | end-to-end | setup plus recipes 1–6, prints in reading order |
|
||||
| `ob.py`, `regions.py` | lib | `orbit sql` wrapper (fails on "not indexed" fallback); cfg-region scanner |
|
||||
| `defdiff.py` | 1 | changed definitions, signature deltas, unattributed-line tally, JSON for the other recipes |
|
||||
| `gt_items.py` | 1 (eval) | regex ground truth for added and removed items |
|
||||
| `blast.py` | 2 | dependents in/outside the diff, re-export check for removed items, `--measure` grep ground truth |
|
||||
| `traitcov.py` | 3 | E/G/H implementors, new-method coverage, text fallback |
|
||||
| `boundary.py`, `rules/{spacecraft,dirigent}.rules` | 4 | crate and module edge delta, rules, inline-path and Cargo.toml fallbacks |
|
||||
| `preflight.py` | 5 | blindness preflight |
|
||||
| `context.py` | 6 | budgeted API brief |
|
||||
| `fb.sh` | feedback | JSONL append helper |
|
||||
| `sql/hunk-to-defs.sql`, `dependents.sql`, `importers.sql`, `implementors.sql`, `crate-edges-delta.sql`, `blind-files.sql` | 1, 2, 2, 3, 4, 5 | standalone SQL, each tested against a real DB |
|
||||
|
||||
Phase-1 scripts (`calls.py`, `ext.py`, `defdiff.sql`, …) remain in `~/.cache/orbit-probe/`.
|
||||
|
||||
## 14. Hygiene report
|
||||
|
||||
- **Worktrees:** 16 detached worktrees were created under `~/.cache/orbit-probe/worktrees/`. All were removed with `git worktree remove` plus `git worktree prune`. At the end, `git worktree list` shows:
|
||||
- spacecraft: `main` and the user's `.worktrees/strategy-mvp`;
|
||||
- dirigent: the main checkout only.
|
||||
- **Isolated DBs:** eight per-review DBs (`*.duckdb`) and one throwaway dirty-tree DB were deleted.
|
||||
- **Shared DB:** `~/.gitlab/orbit/graph.duckdb` is unchanged: same size (72,626,176 bytes), same mtime (17:23:09) and same sha256 `c9784dbe…` before and after. The `wt-spacecraft` entry from phase 1 is still there. I did not touch it.
|
||||
- **Scratch worktree edits:** the dirty-tree test edited only my own scratch worktree. It was reverted (`git checkout -- <file>` inside that worktree, and removal of the untracked file) before the worktree was removed. No repository refs or files were modified.
|
||||
- **Processes:** no compilation was run. `pgrep -a orbit` found no orbit process at the end.
|
||||
- **Scratch size:** `~/.cache/orbit-probe/` is 1.2 MB (scripts plus text outputs in `out/`). Nothing was written to `/tmp`.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Orbit research tree: plan
|
||||
|
||||
Goal: `goals/orbit-skill.md`. Phase 0 input: `phase-0-field-guide-python-django.md` (assessed orbit 0.124.0; 0.133.0 is installed locally).
|
||||
|
||||
## Direction (user, 2026-09-29)
|
||||
|
||||
- The Python guide is **baseline guidance only**. The repository it came from is off limits.
|
||||
- Test beds: **spacecraft** and **dirigent** (both Rust) are the main ones. Any other project in `~/Projects` may be used.
|
||||
- First use case: **review**, for example the Spire review stage. Implementation support comes later, after experimenting.
|
||||
- Aim: an **early prototype skill**, not a finished one.
|
||||
|
||||
## Phase 1 (parallel)
|
||||
|
||||
- **rust-capabilities**: what orbit 0.133 does with spacecraft and dirigent. Covers the subcommand surface, the definition types and edges for Rust (struct, enum, trait, impl, fn, method, mod, macro), how traits and impls map to `EXTENDS`, how much of `CALLS` is covered (trait dispatch, generics, method calls on receivers), workspace and crate handling, the MCP tool list, and index speed. It also checks which of the guide's gotchas carry over to Rust. Output: `phase-1-rust-capabilities.md`.
|
||||
- **review-fit**: how review works in Spire today (the review stage, reviewer briefs, the search and scout agents) and where an orbit skill would sit (in Spire or as a standalone plugin). Output: `phase-1-review-fit.md`.
|
||||
|
||||
## Phase 2 (needs phase 1)
|
||||
|
||||
- **review-workflows**: concrete review recipes, tested on a real branch or diff:
|
||||
- which definitions a change touches
|
||||
- the blast radius, meaning which code depends on what changed
|
||||
- trait-impl coverage
|
||||
- module and crate boundary guards
|
||||
- comparing a branch worktree against main (the multi-worktree index is a feature here, not only a pitfall)
|
||||
|
||||
Output: `phase-2-review-workflows.md`.
|
||||
|
||||
## Phase 3
|
||||
|
||||
- **prototype-skill**: a draft skill built from phases 0–2 (the design stage takes it from there). `index.md` is written last.
|
||||
|
||||
## Refinement 2026-09-29 (after phase 1; see `journals/2026-09-29-17-orbit.md`)
|
||||
|
||||
- **No compilation during research.** Other sessions are working in spacecraft and dirigent. Worktrees are fine as long as nothing is compiled. Anything that needs a build (for example, checking what `cfg` or feature-gated code hides) comes from **real usage later**, not from probes.
|
||||
- **The prototype skill collects feedback.** It records what was queried, what orbit returned, and whether that was useful or wrong, in a place we can read. It is enabled locally on spacecraft and dirigent only, and findings are mined later.
|
||||
- Phase 2 **review-workflows** still runs non-compiling, using isolated `--db` DBs and detached worktrees outside any checkout. The feedback format is an input for phase 3 (prototype skill).
|
||||
Reference in New Issue
Block a user