Codewalkers/docs/test-inventory.md

# Test Inventory

Comprehensive catalog of every test in the suite, what it verifies, and where coverage is weak or redundant.

**41 test files** | **33 unit** | **7 E2E** | **3 integration (mocked)** | **5 integration (real providers)**

Last audited: 2026-03-02

---

## Coverage Matrix

Subjective 0–10 scores. 0 = no coverage, 5 = happy paths only, 10 = comprehensive with edge cases.

| Module | Happy Path | Error/Edge | Integration | Race/Concurrency | Notes |
|--------|:---------:|:----------:|:-----------:|:----------------:|-------|
| Agent Manager | 8 | 6 | 6 | 9 | Missing: `getResult` with actual result, multi-provider spawn |
| Agent OutputHandler | 6 | 4 | 5 | 8 | Missing: `processAgentOutput` with error status, no-result-line case |
| Agent FileIO | 8 | 7 | 8 | — | Missing: round-trip write→read, `writeInputFiles` content verification |
| Agent ProcessManager | 6 | 4 | 3 | 0 | Heavy mocking; missing: `killProcess`, `cleanupWorktree`, error paths |
| Agent Lifecycle | 8 | 7 | 3 | 0 | Missing: controller integration, `analyzeError→shouldRetry` chain |
| DB Repositories | 9 | 7 | 9 | — | task.test.ts thin: missing `findByParentTaskId`, approval, category queries |
| DB Cascades | 10 | — | 10 | — | Excellent: full graph verified |
| Dispatch Manager | 7 | 5 | 8 | 0 | "dependency scenario" test is misleading (deps not implemented) |
| Phase Dispatch | 9 | 7 | 8 | 0 | Best dependency tests (diamond pattern, blocking propagation) |
| Coordination/Merge | 7 | 5 | 7 | 0 | Overlap with ConflictResolutionService |
| Git Manager | 9 | 7 | 9 | 0 | Real git repos, no mocks. Excellent. |
| Git BranchManager | 6 | 5 | 6 | — | Only `remoteBranchExists` tested. Other methods untested. |
| Process Manager/Registry | 8 | 6 | — | 0 | Registry is pure and clean. Manager has mock fragility. |
| Logging | 8 | 6 | 8 | 0 | Real filesystem. Good. |
| Events Bus | 8 | 3 | — | 0 | Missing: handler-throws, handler ordering |
| Server/HTTP | 8 | 6 | 9 | 0 | Real HTTP + real PID files |
| Shutdown | 5 | 2 | — | 0 | Missing: signal→handler→shutdown chain, error resilience |
| tRPC Router | 3 | 1 | 3 | 0 | **Only `health` and `status` tested. All domain procedures untested.** |
| Preview | 7 | 5 | 4 | 0 | Missing: `discoverConfig`, `docker-client`, `health-checker` unit tests |
| E2E Workflows | 8 | 6 | 8 | 0 | Good lifecycle coverage. Output parsing never tested E2E. |

---

## Test Files — Detailed Inventory

### Agent Module (`src/agent/`)

#### `manager.test.ts` — MultiProviderAgentManager

| Test | Verifies |
|------|----------|
| spawn / creates agent record with provided name | `repository.create` called with name |
| spawn / rejects duplicate agent names | Unique name constraint enforced |
| spawn / emits AgentSpawned event with name | Event has agent name in payload |
| spawn / writes diagnostic files for workdir verification | `spawn-diagnostic.json` written with metadata |
| spawn / uses custom cwd if provided | `child_process.spawn` gets custom cwd |
| stop / stops running agent and updates status | DB updated to `stopped` |
| stop / kills detached process if running | Status update after spawn+stop (does NOT verify kill) |
| stop / throws if agent not found | Error on missing ID |
| stop / emits AgentStopped event | Event has `user_requested` reason |
| list / returns all agents with names | `findAll` returns correct data |
| get / finds agent by id | `findById` returns agent |
| get / returns null if not found | `findById` returns null |
| getByName / finds agent by name | `findByName` returns agent |
| getByName / returns null if not found | `findByName` returns null |
| resume / resumes agent waiting for input | Spawn called with `--resume` + session args |
| resume / rejects if not waiting for input | Error on wrong status |
| resume / rejects if no session | Error on null sessionId |
| resume / emits AgentResumed event | Event has sessionId |
| getResult / returns null when no result | Agent result is null |
| delete / deletes agent and clears active state | `repository.delete` called |
| delete / emits agent:deleted event | Event has agent name |
| delete / throws if not found | Error on missing ID |
| delete / handles missing workdir gracefully | No error when workdir absent |

**Mocking**: 5 `vi.mock()` calls (child_process, git/manager, node:fs, node:fs/promises, file-tailer). Brittle.
**Gaps**: No test for `getResult` with actual result. No multi-provider tests. `findByTaskId`/`findBySessionId` mocked but never exercised.

---

#### `output-handler.test.ts` — OutputHandler

| Test | Verifies |
|------|----------|
| processAgentOutput / parse questions from Claude output | `waiting_for_input` status + questions stored |
| processAgentOutput / handle malformed questions | Falls to `crashed` status |
| processAgentOutput / handle "done" status | No `waiting_for_input` set |
| getPendingQuestions / retrieve stored questions | JSON round-trip from DB |
| getPendingQuestions / return null when none | Null for no questions |
| formatAnswersAsPrompt / format normal answers | String formatting correct |
| formatAnswersAsPrompt / instruction-enhanced answers | `__instruction__` prepended |
| formatAnswersAsPrompt / instruction with whitespace | Trimmed correctly |
| formatAnswersAsPrompt / only instruction, no answers | Instruction + header only |
| formatAnswersAsPrompt / empty answers | Header with no entries |

**Gaps**: No test for `processAgentOutput` with `"error"` status. No test for output with no result line. Uses real provider registry (fragile).

---

#### `mutex-completion.test.ts` — handleCompletion mutex

| Test | Verifies |
|------|----------|
| prevent concurrent completion | Same agentId blocked |
| allow sequential completion | Same agentId allowed after first finishes |
| clean up on exception | Lock released on error |
| per-agent mutex isolation | Different agentIds run concurrently |

**Excellent focused design.** 4 complementary scenarios. Uses real timers (50ms delays).

---

#### `file-io.test.ts` — File I/O utilities

| Test | Verifies |
|------|----------|
| generateId / non-empty string | Returns truthy |
| generateId / unique values | 100 unique IDs |
| writeInputFiles / initiative.md | File created |
| writeInputFiles / phase.md | File created |
| writeInputFiles / task.md | File created |
| writeInputFiles / pages | Files in pages/ subdir |
| writeInputFiles / empty options | No crash |
| readSummary / with frontmatter | Parsed correctly |
| readSummary / missing file | Returns null |
| readSummary / no frontmatter | Body only |
| readSummary / empty files_modified | Empty array |
| readPhaseFiles / from phases/ dir | Parsed with frontmatter |
| readPhaseFiles / missing dir | Returns [] |
| readPhaseFiles / no dependencies | Defaults to [] |
| readTaskFiles / from tasks/ dir | Parsed with frontmatter |
| readTaskFiles / defaults category/type | `execute`/`auto` defaults |
| readTaskFiles / missing dir | Returns [] |
| readDecisionFiles / from decisions/ dir | Parsed |
| readDecisionFiles / missing dir | Returns [] |
| readPageFiles / from pages/ dir | Parsed with pageId |
| readPageFiles / missing dir | Returns [] |
| readPageFiles / ignores non-.md | Filter works |

**Real filesystem, no mocks. Good.** Gaps: `writeInputFiles` only checks `existsSync`, not file content. No write→read round-trip test.

---

#### `process-manager.test.ts` — ProcessManager

| Test | Verifies |
|------|----------|
| getAgentWorkdir | Path computation |
| createProjectWorktrees | Worktree creation for initiative projects |
| createProjectWorktrees / throws on failure | Error propagation |
| createStandaloneWorktree | Path returned |
| createStandaloneWorktree / throws on failure | Error propagation |
| spawnDetached / validates cwd exists | existsSync called |
| spawnDetached / throws if cwd missing | Error thrown |
| spawnDetached / passes correct cwd | spawn args verified |
| spawnDetached / writes prompt file | PROMPT.md written |
| buildSpawnCommand / native prompt mode | Command+args assembled |
| buildSpawnCommand / flag prompt mode | Command+args assembled |
| buildResumeCommand / flag style | Resume args assembled |
| buildResumeCommand / no resume support | Error thrown |

**6 `vi.mock()` calls.** Most brittle test file. Missing: `killProcess`, `cleanupWorktree`.

---

#### `mock-manager.test.ts` — MockAgentManager (testing the test double)

43 tests covering: default scenario, configured delay, error/questions scenarios, resume, stop, list, get/getByName, scenario overrides, event emission order, name uniqueness, constructor options, agent modes (execute/discuss/plan/detail), structured questions.

**Thorough.**

---

#### Agent Lifecycle (`src/agent/lifecycle/`)

**`error-analyzer.test.ts`** — 20 tests. Auth failure (4), usage limit (4), timeout (2), missing signal (3), process crash (4), unknown (2), context preservation (2). Pure behavioral classification tests.

**`instructions.test.ts`** — 6 tests. Constant content verification + `addInstructionToPrompt` pure function tests. Appropriate for 28-line module.

**`retry-policy.test.ts`** — 12 tests. Config (2), shouldRetry per error type (8), backoff delays (2). Clean decision-function tests. Missing: `AgentExhaustedError`/`AgentFailureError` class tests.

**`signal-manager.test.ts`** — 18 tests. clearSignal (2), checkSignalExists (2), readSignal (7), waitForSignal (3), validateSignalFile (4). Real filesystem, no mocks. `waitForSignal` has real-timer flakiness risk.

---

### DB Repositories (`src/db/repositories/drizzle/`)

All use real in-memory SQLite via `createTestDatabase()`. No mocks. Integration tests against real SQL.

**`agent.test.ts`** — 20 tests. Full CRUD + findByStatus, findByTaskId, findBySessionId. Gap: never tests `initiativeId` field.

**`cascade.test.ts`** — 4 tests. **High value.** Validates entire FK cascade graph: initiative→phases→tasks→pages→junction rows. Verifies SET NULL on agents/conversations. Before/after assertions prove setup correctness.

**`initiative.test.ts`** — 12 tests. Standard CRUD + findByStatus. Simplest repo, clean.

**`message.test.ts`** — 22 tests. **Most comprehensive repo test.** Covers sender/recipient polymorphism, threading (parentMessageId), status lifecycle (pending→read→responded), filtering (findPendingForUser, findRequiringResponse, findReplies).

**`phase.test.ts`** — 21 tests. CRUD + 12 dependency tests (createDependency, getDependencies, getDependents, findDependenciesByInitiativeId). Direct-only (not transitive) verified. Cross-initiative scoping verified.

**`task.test.ts`** — 12 tests. **Thinnest relative to feature surface.** Missing: `findByParentTaskId`, `findByInitiativeId`, `findPendingApproval`, category filtering, `requiresApproval` behavior.

**Shared pattern**: 8 tests use `setTimeout(resolve, 10)` for timestamp differentiation (~80ms total delay).

---

### Dispatch (`src/dispatch/`)

**`manager.test.ts`** — 13 tests. Queue, getNextDispatchable (priority, queuedAt ordering), completeTask, blockTask, dispatchNext (no tasks, no agents, success), getQueueState, dependency scenario. Real DB + mock event bus + mock agent manager. Gap: "dependency scenario" test admits dependencies aren't implemented — just retests priority ordering.

**`phase-manager.test.ts`** — 17 tests. queuePhase (status validation, deps), getNextDispatchablePhase (dependency blocking), dispatchNextPhase, completePhase, blockPhase, diamond dependency scenario. **Best dependency tests**: full diamond pattern (A→B, A→C, B→D, C→D) with step-by-step verification. Gap: mock DispatchManager never asserted (task-level dispatch not verified).

---

### Coordination (`src/coordination/`)

**`manager.test.ts`** — 14 tests. queueMerge, getNextMergeable (priority ordering), processMerges (success path, conflict detection, resolution task creation, message creation), getQueueState, handleConflict, error handling (undefined repos).

**`conflict-resolution-service.test.ts`** — 10 tests. handleConflict (resolution task properties, original task blocking, agent message, event emission, optional deps, error cases, multiple conflicts, parentTaskId preservation).

**Overlap**: 5+ tests in `manager.test.ts` duplicate conflict-handling logic from `conflict-resolution-service.test.ts`.

---

### Git (`src/git/`)

**`manager.test.ts`** — 22 tests. create, remove, list, get, diff (clean/added/modified/deleted), merge (success/conflict/abort/cleanup), edge cases. **Real git repos on filesystem. No mocks. Excellent.** Most confident test file in the suite.

**`simple-git-branch-manager.test.ts`** — 4 tests. `remoteBranchExists` only. Real git with bare repo + clones. Gap: only one method tested; other methods untested.

---

### Process (`src/process/`)

**`manager.test.ts`** — 18 tests. spawn, stop, stopAll, restart, isRunning (live/dead/stopped), event emission. Uses mocked `execa`. Good separation of event bus concern.

**`registry.test.ts`** — 14 tests. **Pure unit tests, no mocks.** register, get, getAll, updateStatus, unregister, getByPid, clear, size. Exemplary.

---

### Logging (`src/logging/`)

**`manager.test.ts`** — 13 tests. getBaseDir, ensureLogDir, ensureProcessDir, getProcessDir, getLogPath, listLogs (filtering), cleanOldLogs (retainDays). Real filesystem.

**`writer.test.ts`** — 17 tests. open, writeStdout/writeStderr (string/Buffer, timestamps, multi-line), close (flush, idempotent), write-after-close, append mode, event emission, backwards compatibility. Real filesystem + real EventBus.

---

### Events (`src/events/`)

**`bus.test.ts`** — 10 tests. emit/on, event isolation, once, off, multiple handlers, typed events, timestamp preservation, factory. Pure behavioral tests. Gap: no handler-throws test, no handler ordering test.

---

### Server (`src/server/`)

**`index.test.ts`** — 16 tests. Lifecycle (start/stop/double-start/PID conflict/idempotent stop/isRunning), HTTP endpoints (health/status/404/405), PID file management (create/remove/stale cleanup), event emission. Real HTTP server + real fetch. Gap: no port-already-in-use test.

**`shutdown.test.ts`** — 6 tests. shutdown() call order (server.stop before processManager.stopAll), signal handler installation (SIGTERM/SIGINT/SIGHUP). Gap: **doesn't verify signal→handler→shutdown chain**. Doesn't test error resilience (what if server.stop() throws).

---

### tRPC Router (`src/trpc/`)

**`router.test.ts`** — 14 tests. health procedure (shape, Zod, uptime calc, null startedAt, processCount), status procedure (shape, Zod, startedAt, pid, uptime), Zod schema validation.

**Major gap**: **Only `health` and `status` are tested.** All domain procedures untested: initiatives, tasks, phases, agents, pages, proposals, accounts, conversations, previews.

---

### Preview (`src/preview/`)

**`compose-generator.test.ts`** — 10 tests. Pure function tests. YAML generation, build config handling, Caddy deps, Caddyfile routing (single/multi/internal/specificity), labels. No mocks. Strong.

**`config-reader.test.ts`** — 6 tests. `parseCwPreviewConfig` only. Minimal/multi-service, validation errors, internal services, build normalization. Gap: `discoverConfig` (filesystem discovery) is untested — always mocked in manager.test.ts.

**`manager.test.ts`** — 20 tests. start (full lifecycle, phaseId, Docker unavailable, project not found, compose-up failure, health-check failure, no healthcheck), stop (cleanup, missing labels), list (all/filtered/skip non-preview/skip incomplete), getStatus (running/failed/stopped/building/null), stopAll (all/error resilience/empty). 6 `vi.mock()` calls.

**`port-allocator.test.ts`** — 4 tests. Base port, skip used ports, gap filling, real TCP port contention. One real TCP server test. Gap: no upper-bound/exhaustion test.

---

### E2E Tests (`src/test/e2e/`)

All use `createTestHarness()` with in-memory DB, MockAgentManager, MockWorktreeManager, real dispatch/coordination managers.

**`happy-path.test.ts`** — 6 tests. Single task flow, sequential deps, parallel dispatch, full merge flow, complex dependency flow (diamond), fixture dependency DB verification. Gap: merge flow requires manual agent+worktree bridging (harness not fully integrated).

**`architect-workflow.test.ts`** — 8 tests. Discuss mode (complete, pause/resume), plan mode (complete, persist phases), plan conflict detection (reject dup, auto-dismiss stale, different initiatives), full discuss→plan→phases workflow. Gap: architect output is never parsed — mock discards structured data.

**`decompose-workflow.test.ts`** — 10 tests. Detail mode (complete, pause/resume, multiple questions), detail conflict detection (3 tests), task persistence (create, all types, dependencies), full workflow. Gap: filename says "decompose" but describe says "Detail". Task dependency rows never verified in DB.

**`phase-dispatch.test.ts`** — 7 tests. **Cleanest E2E file.** Independent parallel dispatch, sequential deps, diamond dependency (A→B,C→D), approval gate (pending/in_progress rejection), blocked phase (no dispatch, downstream blocking). No fake timers, no agent involvement.

**`recovery-scenarios.test.ts`** — 9 tests. Queue state in DB, in-progress task recovery after crash, blocked state persistence, merge queue recovery, agent Q&A (question/resume, structured questions, status transitions, multiple questions).

**`edge-cases.test.ts`** — 12 tests. Agent crash (events, task status, error message), agent waiting (1 test), task blocking (4 tests), merge conflict handling (4 tests).

**`extended-scenarios.test.ts`** — 6 tests. Conflict hand-back round-trip (full lifecycle, context preservation, sequential conflicts), multi-agent parallel work (parallel completion, merge ordering, mixed outcomes).

---

### Integration Tests (`src/test/integration/`)

**`agent-workdir-verification.test.ts`** — 2 tests. Skipped by default (`REAL_WORKDIR_TESTS`). Spawns real Claude agent, verifies working directory path via diagnostic files and agent-created files.

**`crash-race-condition.test.ts`** — 3 tests. Always runs. Tests signal.json-based completion detection vs crash marking. Covers questions, no-signal crash, and slim-wildebeest scenario. Accesses private methods via `as any`.

**`real-claude.test.ts`** — 3 tests. Skipped by default. Direct Claude CLI contract tests: done/questions/error status outputs validated against Zod schema. ~$0.10.

---

### Real Provider Tests (`src/test/integration/real-providers/`)

All skipped by default. ~$0.50 total per full run.

**`claude-manager.test.ts`** — 7 tests. Output parsing (text_delta, init/session, result), questions flow, session resume (2 tests), error handling. Resume test accepts `crashed` as valid (too lenient).

**`codex-manager.test.ts`** — 3 tests. Spawn, output parsing, provider config. **All accept `crashed` as valid** — pass vacuously when Codex isn't installed.

**`conversation.test.ts`** — 1 test. **Most ambitious**: two real Claude agents doing actual coding while communicating via `cw ask`/`listen`/`answer`. Verifies 2 conversations, file creation, content, and interleaved coding+conversation. ~$0.30, 5min timeout.

**`crash-recovery.test.ts`** — 4 tests. Server restart simulation (resume streaming, mark dead agent, process output for dead agent), event idempotency (no duplicate events on restart).

**`schema-retry.test.ts`** — 9 tests. Valid output (done, questions, multiple questions), retry logic (bad→good, code blocks, surrounding text), mode-specific schemas (discuss, plan, detail). Tests 5-6 have conditional assertions (silently pass on crash).

---

## Remaining Redundancy

Tests covering the same scenario multiple times.

| Scenario | Tests |
|----------|-------|
| Task blocking (DB status) | `edge-cases.test.ts`, `recovery-scenarios.test.ts` |
| Merge conflict → resolution task | `edge-cases.test.ts`, `extended-scenarios.test.ts` (full round-trip) |
| Conflict resolution task creation | `coordination/manager.test.ts` (5 tests), `conflict-resolution-service.test.ts` (10 tests) |

---

## Coverage Gaps (Priority Order)

### Critical

1. **tRPC domain procedures entirely untested** — initiatives, tasks, phases, agents, pages, proposals, accounts, conversations, previews. Only `health`/`status` have tests. This is the single largest gap.

2. **Agent output parsing never tested E2E** — Architect helpers (`setArchitectPlanComplete`, `setArchitectDetailComplete`) discard structured output. The pipeline from agent output → DB persistence is exercised nowhere.

3. **`task.test.ts` thin relative to feature surface** — Missing: `findByParentTaskId`, `findByInitiativeId`, `findPendingApproval`, `approveTask`, category filtering, `requiresApproval` field behavior. These features exist in production but have zero direct unit test coverage.

### Important

4. **`discoverConfig` (preview config discovery) untested** — The filesystem walk logic (.cw-preview.yml → docker-compose.yml → Dockerfile) is always mocked away.

5. **`docker-client.js` and `health-checker.js` have no unit tests** — Always mocked. No verification of the Docker shell-out layer.

6. **Signal handler → shutdown chain untested** — `shutdown.test.ts` verifies listeners are installed and `shutdown()` works when called directly, but never tests signal → handler → shutdown.

7. **No concurrent dispatch/merge tests** — For a multi-agent system, racing `dispatchNext()` or concurrent merges are never exercised.

8. **`processAgentOutput` missing error status test** — "done" and "questions" tested, "error" not tested.

### Minor

9. **EventBus handler-throws behavior untested** — Does emit swallow or propagate handler errors?

10. **Port allocator upper-bound/exhaustion untested** — What happens when all ports 9100-9200 are taken?

11. **`error-analyzer` → `retry-policy` integration untested** — Each tested independently but the chain (`analyzeError` output fed to `shouldRetry`) is never verified end-to-end.

12. **Git `BranchManager` incomplete** — Only `remoteBranchExists` tested. Other methods (if any) untested.

13. **`ProcessManager.killProcess` and `cleanupWorktree` untested** — No unit test for these methods.

---

## Fragility Assessment

### High Fragility (refactoring will break tests)

- **`agent/process-manager.test.ts`** — 6 `vi.mock()` calls. Any import restructuring breaks tests.
- **`agent/manager.test.ts`** — 5 `vi.mock()` calls. Same concern.
- **`preview/manager.test.ts`** — 6 `vi.mock()` calls. Same concern.
- **`crash-race-condition.test.ts`** — Accesses private `handleCompletion` via `as any`.

### Low Fragility (behavioral tests)

- **DB repository tests** — Real in-memory SQLite, test public API only.
- **`git/manager.test.ts`** — Real git repos, no mocks.
- **`file-io.test.ts`** — Real filesystem, pure function tests.
- **`process/registry.test.ts`** — Pure unit tests, no mocks.
- **`preview/compose-generator.test.ts`** — Pure function tests.
- **`events/bus.test.ts`** — Pure behavioral tests.
- **E2E tests** — Test through public harness API.

---

## Mock Style Inconsistencies

The test suite uses 4 different mocking patterns with no shared convention:

| Pattern | Files | Concern |
|---------|-------|---------|
| `vi.mock()` module-level | manager.test.ts, process-manager.test.ts, preview/manager.test.ts | Import-structure coupled |
| Factory function (`createMockX()`) | output-handler.test.ts, coordination/*.test.ts, server/*.test.ts | Cleaner but copy-pasted |
| Inline const object | mutex-completion.test.ts | Simple but mutated per-test |
| Hand-rolled class | harness.ts (MockWorktreeManager, CapturingEventBus) | Best for shared infrastructure |

`createMockEventBus()` is independently defined in 5+ files. Should be extracted to shared test utility.