reliquary/docs/workpad/plans/2026-10-17-00-gremlins-mvp-spec.md

Gremlins MVP: implementation spec

• Date: 2026-10-17 · Author model: claude-opus-5-5 · Timebox: 20 min, one agent
• Inputs: code verdict (§3 keep, §5 minimal), post-mortem RC1–RC4, brainstorm "Steering sentences cut both ways"
• Prototype: .claude/worktrees/headless-forks/plugins/spire/skills/gremlins/gremlins.py @ ab1d945 (cited as P:<line>)

How to read this. Every module, function and CLI command names the acceptance box (B1–B8) it serves. Anything that serves no box carries a one-line reason, or it is out. The builder does the same in the hand-back: code that traces to no box gets a one-line reason, and a missing failure path that a box names counts as a gap. Size figures are estimates to justify deviations against, not caps. Going over is fine with a reason, and so is coming in under, as long as no box's failure path was dropped to get there.

Acceptance boxes

	Box
B1	train creates a base with one blocking claude -p call and returns its session id. The launch fingerprint (binary, full model id, flags, cwd, appended prompt) is stored once.
B2	spawn forks the base (--resume <base> --fork-session) with a brief, as one blocking call that reuses the fingerprint verbatim, so the prompt cache hits.
B3	Several gremlins run at once. The orchestrator runs each spawn through Bash run_in_background and gets the completion notice. The helper has no detach or supervisor code.
B4	TaskStop on the Bash call stops a gremlin. Verified on Windows 2.1.299, so there is no kill command (see §7 for the orphaned in-flight command).
B5	Usage per call comes from the result JSON: the real model from modelUsage, four token columns, and the cost delta against the base. One appended JSON line per call in a ledger.
B6	Warmth is reported only (last cache hit, TTL left). No pings.
B7	rebase resumes the base without forking and passes a short summary of the changes. Optional, tiny.
B8	No platform-specific process code: Windows Git Bash, WSL, Linux, macOS. The env scrub stays.

1. Ecosystem inventory: what already exists, so the helper must not rebuild it

Provider	Gives us	So the helper does NOT build
Harness	Bash run_in_background with a completion notice that carries stdout. A call that passes its timeout moves to the background and keeps running (post-mortem RC1). TaskStop. The orchestrator's own task list.	detach, supervisor, wait, heartbeats, the orphan watchdog, reaping of lost jobs, a "running jobs" poller (B3, B4)
claude -p	--session-id <uuid> (id known before launch), --resume … --fork-session, --append-system-prompt-file, --strict-mcp-config, --output-format stream-json --verbose. Its events: system/init; assistant with per-request usage; result with session_id, usage, modelUsage, total_cost_usd, is_error, api_error_status. Session storage. Price maths.	price tables, transcript reading, cost computation, session bookkeeping beyond the ids (B1, B2, B5)
Python stdlib	subprocess.Popen plus terminate() (portable), signal, json, uuid, argparse, string.Template, os.replace, shutil.which	ctypes, CIM, Toolhelp32, pid-reuse guards, process-tree walks, a lock manager (B8)
uv	an interpreter for a PEP 723 script with no dependencies (uv run --script)	interpreter discovery (supervisor_interpreter)
git	log / diff --shortstat for the change summary	its own change tracking (B7)

Concurrency without locks (B3): each job writes only its own jobs/<J>.json. Only train and rebase write bases/<B>.json. The ledger is append-only, one write() per line. Warmth is derived from the ledger (B6), so spawns never write base files.

2. Module structure (plugins/spire/skills/gremlins/)

Entry point: gremlins.py, a PEP 723 script with dependencies = [] that puts its own directory on sys.path and imports the package grm/. Run it as uv run --script gremlins.py <cmd> …. The templates and preamble.md stay where they are.

Module	Purpose	Public functions	Boxes	Est. lines
grm/store.py	state-dir layout, atomic JSON, ledger append, run/base/job records	state_dir(run) -> Path; read_json(p) -> dict|None; write_json(p, obj); ledger_append(run, line: dict); ledger_lines(run) -> list[dict]	B1, B3, B5	70
grm/launch.py	fingerprint, env scrub, binary lookup, argv	resolve_binary(explicit=None) -> str|None; scrubbed_env(binary, environ=None) -> dict; build_fingerprint(*, binary, model, effort, cwd, agent, preamble_path) -> dict; train_argv(fp, sid) -> list; fork_argv(fp, base_sid, sid) -> list; resume_argv(fp, base_sid) -> list	B1, B2, B7, B8	110 (40 are the kept scrub constants)
grm/stream.py	stream reader and accounting (pure functions, no IO)	StreamState.feed(ev, t); classify(ss, exit_code, stopped) -> (status, reason); usage_from_result(result, before_mu, cost_before) -> dict; api_error_reason(res); cache_verdict(first_usage, base_ctx); infer_ttl(tokens, prev); warmth(last_hit_ts, ttl_s, now) -> dict	B2, B5, B6	170
grm/call.py	the only subprocess code: run one claude -p in the foreground, tee the stream to raw/<J>.jsonl, forward a stop	run_claude(argv, prompt: str, env, cwd, raw_path, on_start) -> (StreamState, exit_code, stopped: bool)	B1–B4, B8	60
grm/cli.py	argparse plus the commands; each command is one record → call → record sequence	main(argv) -> int; cmd_init/train/spawn/rebase/status(args) -> dict	all	200
tests/	see §6			about 300

Total: about 610 lines of code (the verdict estimated 600–800).

call.py details:

• The prompt goes in on stdin, which avoids the Windows argv length limit.
• on_start(child_pid) records the pid in the job file.
• A handler for SIGTERM and SIGINT, plus getattr(signal, "SIGBREAK", None), calls child.terminate() and sets stopped. Reason for the getattr: it is the one portable way to see Ctrl-Break on Windows, and it is not process code.
• It uses stream-json rather than --output-format json. Reason: the first assistant usage is the only way to prove the B2 cache hit, and a stopped call (B4) leaves a partial trace behind.

Data model

• run.json (B1; written once by init, never rewritten):

  {run, created, fingerprint: {binary, binary_version, model: "claude-opus-5-5[1m]", effort,
   permission_mode, tools, agent, cwd, append_prompt: {path, sha256}, mcp_config, env_pinned},
   argv: [...full fingerprinted argv...], fp_hash}
  

  Spawn reuses argv as stored (B2) and only appends --resume/--fork-session/--session-id.

• bases/<B>.json (B1, B7):

  {base, session_id, train_job, commit, trained_at, context_tokens, model_usage, cost_cumulative_usd, ttl_s, rebases}
  

• jobs/<J>.json (B3, B4):

  {job, kind: train|spawn|rebase, base, session_id, status: running|ok|error|stopped, reason,
   started, ended, child_pid, raw, note, cache, result_excerpt}
  

• ledger.jsonl, one line per finished or stopped call (B5):

  {ts, run, job, kind, base, session_id, model, models_used,
   tokens: {input, output, cache_creation, cache_read, cache_creation_1h, cache_creation_5m},
   cost_delta_usd, cost_cumulative_usd, cache, status, wall_s}
  

  model is models_used[0] from the modelUsage delta. The 1h/5m split stays because infer_ttl needs it (B6).

• Warmth (B6): the last ledger line for base B whose tokens.cache_read > 0 gives last_hit. The ttl is infer_ttl over that line's tokens. Output: {base_state: warm|expiring|cold, warm_until, ttl_left_s}.

3. Reuse from the prototype (copy, then strip review-ID comments)

Into	Prototype source	Change
stream.py	StreamState P:1459–1522, context_tokens P:1525, usage_from_result P:1573–1601 (+MU_FIELDS P:158), _API_ERROR_TEXT/api_error_reason P:1604–1637, cache_verdict P:1640 (+CACHE_HIT_RATIO P:153), infer_ttl P:1650, wrote_only_5m P:1663, base_warmth P:1675–1690 (+TTL_1H/5M P:137)	base_warmth becomes warmth(last_hit, ttl, now), margin max(120, ttl/10), with no warm_margin_s. Round costs once, at 6 decimals (verdict §6 rounding finding).
stream.py classify	the if-chain inside _conclude P:1771–1790	rewrite it as one match over (stopped, exit_code, ss.result, api_error_reason(...))
launch.py	ENV_PINNED_LAUNCH, SCRUB_* P:86–109, scrub_class / _venv_prefixes / _under / scrubbed_env P:1235–1282, resolve_binary(_via) P:1309–1335, claude_argv P:2652–2665, canonical_json / _sha256_* P:240–250	drop scrub_report/scrub_unknown (only where uses them); norm_path becomes os.path.abspath
store.py	_retry, atomic_write_bytes, write_json_atomic, read_bytes_retry P:829–866; ledger_append P:1046–1059	the ledger append drops its RegistryLock precondition and per-job dedupe: one call writes once
cli.py	render_template P:1738; delta_header P:2677–2697 + git P:1214	used as is
files	templates/{train,spawn,rebase}.md, preamble.md	remove ${commit_rule} and ${mode} from spawn.md. Reason: the default rule contradicted no-commit briefs (verdict §3), so the brief now carries commit instructions.

Do not port: Platform/WindowsPlatform/PosixPlatform, RegistryLock, Supervisor, SupClock, reap_lost, kill_tree/process_tree, ping and tick, due_flags and the drift checks, wait, where, model_admission, --method.

4. CLI surface (G = uv run --script gremlins.py; every command prints one JSON object on stdout)

• G init --run R --model M [--effort E] [--cwd D] [--agent A] [--binary P] writes run.json with the fingerprint and the rendered preamble (B1). It refuses to overwrite an existing run.
• G train --run R --base B --brief F makes one blocking call with --session-id, then writes bases/B.json and prints {status, session_id, cost, …} (B1).
• G spawn --run R --base B --job J --brief F makes one blocking fork with the spawn template plus delta_header and prints {status, cache, note, cost, …} (B2, B3, B5). Reason for the delta: a fork knows only the base's commit, and it reuses B7's function.
• G rebase --run R --base B --brief F resumes the base without a fork; the prompt is the rebase template + delta_header + the brief. Afterwards it updates bases/B.json from the result. It warns when jobs of B are still marked running (B7).
• G status --run R [--job J] shows bases with their warmth, plus the job records (B6; B3 for reading a result again).
• There is no kill command. B4 was verified on Windows with 2.1.299: TaskStop on the backgrounded Bash call kills claude.exe and its bash wrappers.
• Out of the MVP, each with its reason: continue (no box), ping (B6 says no), usage aggregation (no box; the ledger lines are greppable), list/wait/expire (the harness covers them).

5. Build order (sequencing, not a ranking of value)

1. store + launch + stream (ported) + call + init/train/spawn. Gate: a live run of init → train → spawn shows cache: hit and two ledger lines with real model, tokens and cost delta. Nothing else gets built before this gate passes.
2. B3: two spawns via run_in_background at once; both notices arrive, two job files, two ledger lines.
3. B6: status with warmth.
4. B4: the stop-signal path in call.py, then one live TaskStop on a backgrounded spawn to confirm the job ends without a ledger line and nothing beyond an in-flight command survives.
5. B7: rebase.
6. B8: run step 1's gate on one non-Windows host (WSL is enough).

6. Tests (stdlib unittest; a fake claude = [sys.executable, tests/fake_claude.py] written into run.json["argv"])

The tests cover the happy paths and the failure paths each box names. Add no other tests without a one-line reason.

• Happy (B1, B2, B5): train stores the base with the session id from --session-id. spawn's argv starts with the stored argv byte for byte, then --resume <base> --fork-session. The ledger line has model, four token columns and cost_delta = total − base cumulative.
• B1 failures: the binary is missing (error before launch, no job file); a non-zero exit or no result event gives status: error and no base file; an API error maps through api_error_reason.
• B2 failures: unknown base gives an error with no launch; first-usage cache_read of 0 gives cache: miss, which is reported, not an error.
• B3: two spawns in parallel threads give two job files and two whole ledger lines.
• B4: SIGTERM to a helper running a fake that sleeps leaves the child gone, the job stopped, and the partial raw file kept.
• B5: modelUsage with two models gives both in models_used; a missing total_cost_usd gives cost_delta: null.
• B6: no hit gives cold; 5m-only writes give ttl_s 300; a hit at now − ttl + 60 gives expiring.
• B7: the rebase argv has --resume and no --fork-session; the base record takes the result's session_id.
• B8: the scrub drops SCRUB_DROP and UV_*, sets the pinned vars, and removes venv PATH entries.

7. Open risks per box

• B4 (verified, with a known leftover). TaskStop kills claude.exe and its bash wrappers. A command the gremlin had in flight (in the check, a sleep 120 run through the gremlin's Bash) survives as an orphan and runs to its own end. That is accepted for the MVP. A later sweep is an open item, outside MVP scope; a full tree kill would need a Job Object, and B8 forbids that. On a hard kill, the helper's signal handler may never run, so the job file can stay running with no ledger line. The orchestrator knows which calls it stopped, so status reports the record as written. Reason the handler stays anyway: on POSIX and with Ctrl-C it does run, and it costs about 10 lines.
• B7. Resuming without a fork may give the session a new id in -p mode. Take session_id from the result; never assume it stays the same. A spawn started during a rebase forks a session that is still being written. The helper only warns; keeping them apart is the orchestrator's job.
• B2. A cache hit needs more than the same argv: the same env pins, the same cwd, and a system prompt that does not change (for example, the date rolls over at midnight). The helper reports cache; it does not prevent misses.