dirigent/crates/dirigent_core/CLAUDE.md

# Package: dirigent_core

Core orchestration engine for multi-connector agent system management.

## Quick Facts
- **Type**: Library
- **Main Entry**: src/lib.rs
- **Dependencies**: dirigent_protocol, tokio, axum, serde, uuid

## Architecture Overview

The dirigent_core package provides a **runtime-based architecture** for managing long-lived connections to external agent systems (OpenCode.ai, ACP agents, etc.). The core abstraction is the **Connector**, which represents a bidirectional communication channel to an agent system.

### Core Components

#### CoreRuntime
The central orchestrator for managing connectors. It maintains:
- Registry of active connectors (keyed by ConnectorId)
- Global event broadcast channel for system-wide events
- User registry for ownership and authorization
- Configuration state (with persistence support)

#### CoreHandle
Lightweight, cloneable wrapper around CoreRuntime. Uses Arc internally for cheap cloning across async tasks and server functions.

#### Connector Trait
Defines the interface for connector implementations:
- Command channel (mpsc) for control operations
- Event broadcast channel for publishing events
- State tracking (Initializing, Connecting, Ready, Error, Stopped)
- User ownership for authorization

#### ConnectorHandle
Concrete implementation of the Connector trait that wraps:
- Metadata (id, kind, owner, title)
- Shared state (protected by RwLock)
- Command and event channels
- Optional task handle for lifecycle management

### Connector Implementations

#### OpenCodeConnector
Connector for OpenCode.ai REST + SSE API:
- HTTP client for session/message operations
- SSE event stream for real-time updates
- Background task loop for command processing
- State machine for connection lifecycle
- **TurnComplete emission**: Uses `TurnCompleteTrigger::ExplicitSignal` after `MessageCompleted` events (based on upstream session.idle signals)

#### AcpConnector (Future)
Connector for Agent-Client Protocol:
- WebSocket or HTTP/2 transport
- ACP message protocol handling
- Tool execution and streaming support

## Key Files

### Core Runtime
- `src/runtime.rs` - CoreRuntime and CoreHandle implementation
- `src/types.rs` - Core types (ConnectorId, ConnectorState, User, etc.)
- `src/error.rs` - Error types for the runtime
- `src/config.rs` - Configuration types and template system

### Connectors
- `src/connectors/mod.rs` - Connector trait and ConnectorHandle
- `src/connectors/opencode/mod.rs` - OpenCode connector implementation
- `src/connectors/opencode/config.rs` - OpenCode-specific configuration
- `src/connectors/acp/mod.rs` - ACP connector implementation (in progress)

### ACP Protocol Implementation
- `src/acp/protocol/initialize.rs` - Protocol initialization and capability negotiation
- `src/acp/protocol/authenticate.rs` - Optional authentication flow
- `src/acp/protocol/session.rs` - Session lifecycle (new, load, set_mode, cancel)
- `src/acp/protocol/prompt.rs` - Prompt requests with content blocks (Phases 3-7 complete)
- `src/acp/protocol/streaming.rs` - Session update notifications and handlers
- `src/acp/protocol/stop_reason.rs` - Stop reason interpretation and actions
- `src/acp/protocol/cancellation.rs` - Cancellation and disconnect handling
- `src/acp/protocol/error.rs` - Error classification and retry logic
- `src/acp/connector_state.rs` - Connection and session state management
- `src/acp/transport/mod.rs` - Transport abstraction layer
- `src/acp/transport/stdio.rs` - Stdio transport (process spawning)
- `src/acp/transport/http.rs` - HTTP+SSE transport

### Bidirectional Request Handling Pattern

The ACP connector implements **true bidirectional communication** where both client and agent can send JSON-RPC requests at any time. This creates an architectural challenge: how to maintain synchronous request/response semantics (async/await) while handling incoming agent requests during outgoing client requests.

**The Challenge:**

When the connector sends `session/prompt` to the agent:
1. It calls `send_request()` which awaits the JSON-RPC response
2. The agent may send permission requests (e.g., `tools/write`) before responding
3. These agent requests arrive as `ConnectorCommand::AgentResponse` via the command channel
4. If `send_request()` only polls transport and response channels, the command channel isn't polled
5. **Result**: Deadlock - agent waits for permission → permission stuck in channel → client waits for prompt response

**The Solution (src/connectors/acp/connector.rs:1253-1523):**

The `send_request()` method uses `tokio::select!` to poll **three** sources simultaneously:
- **Response channel** (`response_rx`) - Waiting for the correlated JSON-RPC response
- **Transport channel** - Receiving messages/notifications from agent (may trigger response)
- **Command channel** (`cmd_rx`) - Receiving commands from event bridge (e.g., `AgentResponse`)

When an `AgentResponse` command arrives during `send_request()`:
1. Extract the response payload from the command
2. Send it to the agent via transport immediately
3. Remove from pending requests map
4. Continue waiting for the original prompt response

This pattern is **idiomatic async Rust** for implementing synchronous abstractions over bidirectional transports. It's similar to:
- gRPC bidirectional streaming with request/response correlation
- WebSocket clients with RPC-style method calls
- HTTP/2 multiplexing with concurrent streams

**Why Not Separate Tasks?**

Alternative architectures (separate command processing task, full actor model) introduce:
- Complex synchronization between tasks
- Race conditions on shared state
- Message ordering guarantees across channels
- Significantly more code and cognitive overhead

The single-task, multi-channel select pattern keeps all state local and eliminates these issues.

**Key Invariant:**

Any method that blocks waiting for a response MUST also poll the command channel to process `AgentResponse` commands, otherwise bidirectional flows deadlock.

## Main Exports

### Runtime
- `CoreRuntime` - Main orchestrator
- `CoreHandle` - Cloneable runtime handle
- `CoreConfig` - Runtime configuration

### Types
- `ConnectorId`, `UserId` - Type aliases for IDs
- `ConnectorKind` - Enum of connector types (OpenCode, Acp, Mock)
- `ConnectorState` - Lifecycle state enum
- `ConnectorSummary` - Lightweight connector view
- `User` - User information

### Connectors
- `Connector` - Trait for connector implementations
- `ConnectorHandle` - Handle to a running connector
- `ConnectorCommand` - Commands sent to connectors
- `OpenCodeConnector` - OpenCode.ai integration
- `OpenCodeConfig` - OpenCode connector configuration

### Configuration
- `ConnectorConfig` - Configuration for creating connectors
- `apply_template()` - Apply connector templates with patches

### ACP Protocol Types (Phases 3-7)
- **Prompt Turn**:
  - `SessionPromptRequest`, `SessionPromptResponse` - Prompt requests/responses
  - `ContentBlock` - Text, Image, Audio, Resource, ResourceLink content
  - `EmbeddedResource` - Text or Blob embedded resources
  - `StopReason` - EndTurn, MaxTokens, MaxTurnRequests, Refusal, Cancelled
  - `PromptError` - Timeout, JsonRpcError, TransportError, Cancelled, ValidationError

- **Streaming Updates**:
  - `SessionUpdate` - All update types (agent_message_chunk, tool_call, plan, etc.)
  - `SessionUpdateNotification` - Notification wrapper with session_id
  - `ToolCallInfo` - Tool call tracking with status and content
  - `ToolKind` - Read, Edit, Search, Execute, Think, Other
  - `ToolCallStatus` - Pending, InProgress, Completed, Failed, Cancelled
  - `ToolCallContent` - Content, Diff, Terminal output
  - `MessageAccumulator` - Message chunk accumulation helper
  - `PlanEntry`, `Command` - Plan and command structures

- **Stop Reason Handling**:
  - `StopReasonAction` - Complete, ShowWarning, ShowError, ShowInfo
  - `handle_stop_reason()` - Interpret stop reasons
  - `is_continuable()`, `is_error()` - Stop reason classification

- **Cancellation**:
  - `handle_cancellation()` - Cancel pending operations
  - `cancel_pending_tool_calls()` - Mark tool calls as cancelled
  - `handle_disconnect()` - Handle transport disconnect

- **Error Classification**:
  - `ClassifiedError` - Error with class, message, details, retry_after
  - `ErrorClass` - Transient, Terminal, User
  - `ErrorSeverity` - Info, Warning, Error, Fatal
  - `ErrorAction` - Retry, Reconnect, CheckConfig, ContactSupport, Dismiss
  - `classify_jsonrpc_error()`, `classify_transport_error()` - Classify errors
  - `exponential_backoff()` - Calculate retry delays

## Usage Examples

### Creating a Runtime

```rust
use dirigent_core::{CoreRuntime, CoreConfig, CoreHandle};

// Load config from file or use default
let config = CoreConfig::load_config(None)?;

// Create runtime
let runtime = CoreRuntime::new(config);

// Wrap in handle for cheap cloning
let handle = CoreHandle::new(runtime);
```

### Creating a Connector

```rust
use dirigent_core::{ConnectorConfig, ConnectorKind, OpenCodeConfig};
use serde_json::json;

// Build OpenCode connector config
let config = OpenCodeConfig {
    base_url: "http://localhost:12225".to_string(),
    title: "My OpenCode".to_string(),
    initial_session: None,
};

// Serialize to JSON for ConnectorConfig
let params = serde_json::to_value(&config)?;

let connector_config = ConnectorConfig {
    id: None, // Runtime generates ID
    kind: ConnectorKind::OpenCode,
    owner: Some("user-123".to_string()),
    title: Some("My OpenCode".to_string()),
    params,
};

// Create connector via runtime
let connector_id = handle.create_connector(
    "user-123".to_string(),
    connector_config
).await?;
```

### Using Templates

```rust
use dirigent_core::{apply_template, ConnectorKind};
use serde_json::json;

// Use default template with custom URL
let connector_config = apply_template(
    ConnectorKind::OpenCode,
    "default",
    json!({
        "base_url": "http://localhost:8080",
        "title": "Custom OpenCode"
    })
)?;

let connector_id = handle.create_connector(
    "user-123".to_string(),
    connector_config
).await?;
```

### Managing Connectors

```rust
// List all connectors
let all_connectors = handle.list_connectors(None).await;

// List connectors for a specific user
let user_connectors = handle.list_connectors(Some("user-123".to_string())).await;

// Get a specific connector
let connector = handle.get_connector(&connector_id).await;

// Stop a connector
handle.stop_connector(&connector_id).await?;

// Restart a stopped connector
handle.restart_connector(&connector_id).await?;

// Remove a connector
handle.remove_connector(&connector_id).await?;
```

### Connector Lifecycle with Restart

Connectors can be restarted after being stopped or entering an error state:

```rust
// Create and start a connector
let connector_id = handle.create_connector(
    "user-123".to_string(),
    connector_config
).await?;
// Connector is now in Ready state

// Stop the connector
handle.stop_connector(&connector_id).await?;
// Connector is now in Stopped state

// Restart the connector (recreates background task with fresh channels)
handle.restart_connector(&connector_id).await?;
// Connector transitions: Stopped → Initializing → Connecting → Ready

// Restart preserves:
// - Connector ID (same instance)
// - Configuration (base_url, title, etc.)
// - Event broadcast channel (subscribers continue receiving)
// - State Arc (observers see real-time updates)

// Restart recreates:
// - Command channel (new sender/receiver pair)
// - Connector instance (fresh OpenCodeConnector)
// - Background task (new spawn)
// - Task handle (new JoinHandle)
```

### Sending Commands

```rust
use dirigent_core::connectors::ConnectorCommand;

// Get connector handle
let connector = handle.get_connector(&connector_id).await.unwrap();

// Subscribe to events
let mut events = connector.subscribe();

// Send a command
let cmd_tx = connector.command_tx();
cmd_tx.send(ConnectorCommand::ListSessions).await?;

// Receive events
while let Ok(event) = events.recv().await {
    match event {
        Event::SessionsListed { sessions } => {
            println!("Got {} sessions", sessions.len());
            break;
        }
        Event::Error { message } => {
            eprintln!("Error: {}", message);
            break;
        }
        _ => {}
    }
}
```

### Global Event Stream

```rust
// Subscribe to every event on the SharingBus. Callers can also pick an
// `EventFilter` via `subscribe_filtered()` to receive only the events
// they care about.
let mut bus_rx = handle.sharing_bus().subscribe_all().await;

tokio::spawn(async move {
    while let Some(bus_event) = bus_rx.rx.recv().await {
        println!("Bus event: {:?}", bus_event.event);
    }
});
```

## Configuration

### Runtime Configuration (dirigent.toml or dirigent.json)

```toml
port = 3000
project_dir = "."
project_name = "my_project"
templates_enabled = true

[[connectors]]
id = "opencode-1"
kind = "OpenCode"
owner = "user-123"
title = "OpenCode Local"

[connectors.params]
base_url = "http://localhost:12225"
title = "OpenCode Local"
initial_session = null
```

### Templates

Available templates:
- `opencode/default` - Standard localhost OpenCode connector
- `acp/claude-default` - Claude API connector (stub for future)

## Connector Event Emission Patterns

### TurnComplete Event Semantics

All connectors emit `Event::TurnComplete` to signal that a turn/message is finalized. This is the **primary signal** for:
- Archivist to finalize and write message to disk
- UI cache to lock message state as immutable
- Conductor bridge to flush response to upstream

**Event ordering guarantee**:
```text
MessageCompleted → TurnComplete → SessionIdle
```

### Connector-Specific TurnComplete Strategies

Different connectors use different strategies to determine when a turn is complete:

#### OpenCode Connector
- **Trigger**: `TurnCompleteTrigger::ExplicitSignal`
- **Strategy**: Relies on upstream `session.idle` events from OpenCode.ai
- **Implementation**: After translating `MessageCompleted`, emits `TurnComplete` then `SessionIdle`
- **Code location**: `crates/dirigent_core/src/connectors/opencode.rs:550-575`

#### ACP Connector (stdio transport)
- **Trigger**: `TurnCompleteTrigger::ResponseReceived`
- **Strategy**: JSON-RPC response message is the final message in a turn
- **Implementation**: Emits `TurnComplete` after receiving the JSON-RPC response to `session/prompt`
- **Code location**: `crates/dirigent_core/src/connectors/acp/connector.rs:328`

#### Gateway Connector
- **Trigger**: `TurnCompleteTrigger::OperationsComplete`
- **Strategy**: Tracks pending tool calls and emits when all operations resolve
- **Implementation**: Monitors tool call status changes and emits when last pending call completes
- **Code location**: `crates/dirigent_core/src/connectors/gateway/mod.rs:464,556,626,678`

**Important**: Connectors MUST emit `TurnComplete` exactly once per turn. Duplicate emissions can cause archiving issues and UI state corruption.

### Gateway Session Transfer Mechanics

The Gateway connector serves as an **entry point** for incoming ACP connections. Sessions can be transferred to real agent connectors (Claude, etc.) via `/select-connector` commands.

#### Key Principle: New Connector Is Authority

**After transfer, the target connector becomes the sole authority for session configuration.**

```text
Before transfer:
  Gateway has placeholder modes: "ask", "write", "yolo"
  Gateway has placeholder models: "simple", "default", "high"

After transfer to Claude:
  Claude's actual modes/models become authoritative
  Gateway's placeholders are irrelevant
  Editor receives config_option_update with Claude's real options
```

#### Transfer Flow

1. User sends `/select-connector claude` in Gateway session
2. Gateway emits `SessionTransferRequest` to CoreRuntime
3. CoreRuntime creates/loads session in target connector
4. Target connector emits `SessionCreated` with its modes/models
5. CoreRuntime extracts modes/models from `SessionCreated` event
6. CoreRuntime emits `SessionTransferred` event with modes/models
7. Event bridge sends `config_option_update` to editor with target's modes/models

#### What Does NOT Happen

- Gateway does NOT adjust or map its values to the target connector
- Gateway does NOT remain involved after transfer completes
- Target connector does NOT inherit Gateway's mode/model selections
- No "mapping" between Gateway placeholders and real connector values

#### Code Locations

- Transfer request handling: `src/runtime.rs:execute_transfer()`
- Gateway commands: `src/connectors/gateway/commands.rs`
- SessionTransferred event: `dirigent_protocol/src/events/mod.rs`
- config_option_update emission: `dirigent_acp_api/src/event_bridge.rs:handle_session_transferred_internal()`

## Architecture Patterns

### Request-Response Pattern
For operations that need results (list_sessions, list_messages):
1. Subscribe to connector events
2. Send command via command channel
3. Wait for corresponding response event (with timeout)
4. Return result or error

### Fire-and-Forget Pattern
For operations that stream results (send_message):
1. Send command via command channel
2. Return immediately
3. Clients subscribe to event stream for updates

### Lifecycle Management
Connectors progress through states:
1. **Initializing** - Created but not connecting
2. **Connecting** - Attempting to establish connection
3. **Ready** - Connected and operational
4. **Error** - Encountered failure (with error message)
5. **Stopped** - Shutdown or unrecoverable error

#### Restart Support
Connectors in `Stopped` or `Error` state can be restarted:
- **restart_connector()** recreates the connector's background task with fresh channels
- Preserves connector identity (ID, owner, configuration)
- Preserves event broadcast channel (existing subscribers continue receiving)
- Recreates command channel and background task
- State transitions: `Stopped`/`Error` → `Initializing` → `Connecting` → `Ready`

### Configuration Persistence
The runtime automatically saves configuration to disk when:
- A connector is created
- A connector is removed
- Configuration is explicitly saved

This ensures connectors are restored on server restart.

## Session Tracking Responsibilities

**IMPORTANT**: CoreRuntime is a **stateless orchestrator** for session operations. It does NOT maintain session state or cache message history.

### What CoreRuntime Does

- **Route Commands**: Forward session/message commands to appropriate connectors
- **Broadcast Events**: Relay connector events to global event stream
- **Manage Connectors**: Track which connectors are active and available
- **Persist Configuration**: Save/load connector configuration (not session data)

### What CoreRuntime Does NOT Do

- **Cache Sessions**: Does not maintain lists of sessions or session metadata
- **Store Messages**: Does not retain message content or history
- **Buffer Events**: Does not cache events for replay or historical access
- **Track Session State**: Does not know which sessions exist or their current state

### Stateless Design Rationale

1. **Multi-Connector Support**: With multiple connectors (OpenCode, ACP, etc.), caching sessions would require complex invalidation
2. **Memory Efficiency**: Long-running server should not accumulate unbounded session history
3. **Single Source of Truth**: External APIs (OpenCode.ai, ACP agents) are authoritative for session state
4. **Scalability**: Stateless design supports future horizontal scaling

### Data Flow for Session Operations

**List Sessions**:
```
Server Function → CoreRuntime.get_connector() → Send Command → Connector queries API
                       ↓                                              ↓
                   Returns handle                              Broadcasts SessionsListed
                                                                      ↓
                                                           Server function returns to UI
                                                           (CoreRuntime does NOT cache list)
```

**Send Message**:
```
Server Function → CoreRuntime.get_connector() → Send Command → Connector sends to API
                       ↓                                              ↓
                   Returns handle                              Broadcasts MessageSent
                                                                      ↓
                                                           SSE pushes to UI in real-time
                                                           (CoreRuntime does NOT store message)
```

All session data passes through CoreRuntime but is **never cached**. See `docs/architecture/session_tracking.md` for the complete three-layer architecture (CoreRuntime, UI Cache, Archivist).

## Phase 4: `SharingBus` + `StreamRegistry` (2026-04-21)

Every `Event` emitted by a connector or the runtime is published onto a
single `SharingBus` that owns:

- A `broadcast::Sender<BusEvent>` as the internal multicast.
- A worker task that receives from that broadcast and dispatches per-
  subscriber `mpsc::Sender<BusEvent>` pipes with filter matching.
- A `HashMap<(connector_id, native_session_id), scroll_id>` cache that
  late-binds `routing.scroll_id` for events emitted before their
  `SessionRegistered` event arrived.

### Subscriber model

`SharingBus::subscribe_all()` and `subscribe_filtered(EventFilter, cap)`
return a `BusReceiver { id, rx, lagged }`. Filters are applied by the
worker — subscribers never allocate a closure for skipped events.

`EventFilter` variants: `All | ScrollId | ConnectorUid | Kinds | AnyOf
| AllOf`.

### Stream registry

The runtime owns a `StreamRegistry` and a `StreamFactoryRegistry`.
Streams are attached via `CoreRuntime::attach_stream(StreamConfig)`:

- Factory registry resolves the `kind` string → concrete
  `Arc<dyn SessionStream>`.
- The new stream gets its own `BusReceiver` scoped via `scope_to_filter`
  (Session → ScrollId, Connector → ConnectorUid, ArchiveWide → All).
- A worker task pumps events into `stream.on_event(&bus_event).await`.

Health drift: `record_failure` / `record_success` in `sharing/health.rs`
transitions `HealthStatus` on 5 consecutive failures
(`Healthy → Degraded → Unavailable`).

### Replay

`CoreRuntime::replay_session_to_stream(scroll_id, stream_id, opts)`
loads archived messages via the archivist's read API and calls
`stream.on_event` directly, bypassing the bus. Each replayed event has
`origin = EventOrigin::Replay { replay_id }` and
`routing.scroll_id = Some(scroll_id)`.

### Config

`[[streams]]` blocks in `dirigent.toml` are parsed into `StreamsConfig`
and applied at boot (best-effort: failures log + continue).

## Related Packages
- **api** - Server functions that wrap CoreRuntime operations
- **web** - Dioxus UI that calls server functions
- **dirigent_protocol** - Shared event and message types
- **opencode_client** - Low-level OpenCode.ai HTTP client (used by OpenCodeConnector)

## Documentation
- README: ./README.md
- Architecture: ../../docs/architecture/overview.md
- Migration Guide: ../../docs/migration/singleton_to_runtime.md