Codewalkers/docs/preview.md

# Preview Deployments

`apps/server/preview/` — Docker-based preview deployments for reviewing changes in a running application.

## Overview

Preview deployments let reviewers spin up a branch in local Docker containers. A single shared **Caddy gateway** handles subdomain routing for all active previews, accessed at `http://<previewId>.localhost:<gatewayPort>`.

Two modes:
- **Preview mode**: Checks out target branch into a git worktree, builds Docker images, serves production-like output.
- **Dev mode**: Mounts the agent's worktree into a container with a dev server image (e.g. `node:20-alpine`), enabling hot reload.

**Auto-start**: When a phase enters `pending_review`, a preview is automatically started for the phase's branch (if the initiative has exactly one project).

**Key design decision: No database table.** Docker IS the source of truth. Instead of persisting rows, we query Docker directly via compose project names, container labels, and `docker compose` CLI commands.

## Setting Up Preview Deployments for a Project

### Prerequisites

- **Docker** installed and running (`docker --version` / `docker compose version`)
- **Project registered** in Codewalkers (`cw project add` or via the UI)
- **Browser**: Chrome or Firefox recommended. Safari requires manual `/etc/hosts` entries for `*.localhost` subdomains.

### Quick Start

1. **Add a `.cw-preview.yml`** to your project root. This tells Codewalkers how to build and run your app:

```yaml
version: 1
services:
  app:
    build: .
    port: 3000
```

That's it for a single-service app with a Dockerfile. Run:

```sh
cw preview start --initiative <id> --project <id> --branch <branch>
```

Open the URL printed in the output (e.g. `http://abc123.localhost:9100`).

### Don't Have a `.cw-preview.yml`?

The config reader auto-discovers in this order:

1. **`.cw-preview.yml`** — full control (recommended)
2. **`docker-compose.yml` / `compose.yml`** — uses your existing compose file
3. **`Dockerfile`** — builds a single `app` service on port 3000

If your project already has a Dockerfile or compose file, previews work out of the box with zero config.

### Multi-Service Example

A typical full-stack app with frontend, backend, and database:

```yaml
version: 1
services:
  frontend:
    build:
      context: .
      dockerfile: apps/web/Dockerfile
    port: 3000
    route: /                      # serves the root path
    healthcheck:
      path: /
      interval: 5s
      retries: 10
    env:
      VITE_API_URL: /api
    dev:                          # optional: used in dev mode
      image: node:20-alpine
      command: npm run dev -- --host 0.0.0.0
      workdir: /app

  backend:
    build:
      context: .
      dockerfile: packages/api/Dockerfile
    port: 8080
    route: /api                   # proxied under /api/*
    healthcheck:
      path: /health
    env:
      DATABASE_URL: postgres://db:5432/app

  db:
    image: postgres:16-alpine
    port: 5432
    internal: true                # not exposed through the proxy
    env:
      POSTGRES_PASSWORD: preview
```

Requests to `http://<id>.localhost:9100/` hit `frontend:3000`, requests to `/api/*` hit `backend:8080`, and `db` is only reachable by other services.

### Config Reference

Each service in `.cw-preview.yml` supports:

| Field | Required | Description |
|-------|----------|-------------|
| `build` | yes* | Build context — string (`"."`) or object (`{context, dockerfile}`) |
| `image` | yes* | Docker image to pull (e.g. `postgres:16-alpine`) |
| `port` | yes** | Container port the service listens on |
| `route` | no | URL path prefix for gateway routing (default: `/`) |
| `internal` | no | If `true`, not exposed through the proxy (e.g. databases) |
| `healthcheck` | no | `{path, interval?, retries?}` — polled before marking ready |
| `env` | no | Environment variables passed to the container |
| `volumes` | no | Additional volume mounts |
| `seed` | no | Array of shell commands to run inside the container after health checks pass |
| `dev` | no | Dev mode overrides: `{image, command?, workdir?}` |

\* Provide either `build` or `image`, not both.
\** Required unless `internal: true`.

### Seeding

If a service needs initialization (database migrations, fixture loading, etc.), add a `seed` array. Commands run inside the container via `docker compose exec` after all health checks pass, before the preview is marked ready.

```yaml
services:
  app:
    build: .
    port: 3000
    seed:
      - npm run db:migrate
      - npm run db:seed
```

Seeds execute in service definition order. Each command has a 5-minute timeout. If any seed command fails (non-zero exit), the preview fails and all containers are cleaned up.

### Dev Mode

Dev mode skips the Docker build and instead mounts your source code into a container running a dev server. Useful for hot reload during active development.

To use dev mode, add a `dev` section to the service:

```yaml
services:
  app:
    build: .
    port: 3000
    dev:
      image: node:20-alpine          # base image with your runtime
      command: npm run dev -- --host 0.0.0.0   # dev server command
      workdir: /app                   # where source is mounted
```

Start with:

```sh
cw preview start --initiative <id> --project <id> --branch <branch> --mode dev
```

The project directory is mounted at `workdir` (default `/app`). An anonymous volume is created for `node_modules` to prevent host/container conflicts.

### Healthchecks

If a service has a `healthcheck`, the preview waits for it to respond with HTTP 200 before reporting ready. Without a healthcheck, the service is considered ready as soon as the container starts.

```yaml
healthcheck:
  path: /health     # required: endpoint to poll
  interval: 5s      # optional: time between checks (default: 5s)
  retries: 10       # optional: max attempts before failing (default: 10)
```

### Auto-Start on Review

When a phase transitions to `pending_review`, a preview is automatically started if:
- The initiative has exactly one registered project
- The project has a discoverable config (`.cw-preview.yml`, compose file, or Dockerfile)

No manual `cw preview start` needed — just push your branch and move the phase to review.

## Architecture

```
.cw-previews/
  gateway/
    docker-compose.yml       ← single Caddy container, one port
    Caddyfile                ← regenerated on each preview add/remove
  <previewId>/
    docker-compose.yml       ← per-preview stack (no published ports)
    source/                  ← git worktree (preview mode only)

Docker:
  network: cw-preview-net    ← external, shared by gateway + all previews
  cw-preview-gateway         ← Caddy on one port, subdomain routing
  cw-preview-<id>            ← per-preview compose project (services only)

Routing:
  <previewId>.localhost:<gatewayPort> → cw-preview-<id>-<service>:<port>
```

```
PreviewManager
  ├── GatewayManager (shared Caddy gateway lifecycle + Caddyfile generation)
  ├── ConfigReader (discover .cw-preview.yml / compose / Dockerfile)
  ├── ComposeGenerator (generate per-preview docker-compose.yml)
  ├── DockerClient (thin wrapper around docker compose CLI + network ops)
  ├── HealthChecker (poll service healthcheck endpoints via subdomain URL)
  ├── PortAllocator (find next available port 9100-9200 for gateway)
  └── Worktree helper (git worktree add/remove for preview mode)
```

### Lifecycle

1. **Start**: ensure gateway → discover config → create worktree (preview) or use provided path (dev) → generate compose → `docker compose up --build -d` → update gateway routes → health check → run seed commands → emit `preview:ready`
2. **Stop**: `docker compose down --volumes --remove-orphans` → remove worktree → clean up `.cw-previews/<id>/` → update gateway routes → stop gateway if no more previews → emit `preview:stopped`
3. **List**: `docker compose ls --filter name=cw-preview` → skip gateway project → parse container labels → reconstruct status
4. **Shutdown**: `stopAll()` called on server shutdown — stops all previews, then stops gateway

### Gateway

The `GatewayManager` class manages a single shared Caddy container:

- **`ensureGateway()`** — idempotent. Creates the `cw-preview-net` Docker network, checks if gateway is already running, allocates a port (9100-9200) if needed, writes compose + Caddyfile, starts Caddy with `--watch` flag.
- **`updateRoutes()`** — regenerates the full Caddyfile from all active previews. Caddy's `--watch` flag auto-reloads on file change (no `docker exec` needed).
- **`stopGateway()`** — composes down the gateway, removes the Docker network, cleans up the gateway directory.

Gateway Caddyfile format:
```
{
  auto_https off
}

abc123.localhost:9100 {
  handle_path /api/* {
    reverse_proxy cw-preview-abc123-backend:8080
  }
  handle {
    reverse_proxy cw-preview-abc123-frontend:3000
  }
}

xyz789.localhost:9100 {
  handle {
    reverse_proxy cw-preview-xyz789-app:3000
  }
}
```

Routes are sorted by specificity (longer paths first) to ensure correct matching.

### Subdomain Routing

Previews are accessed at `http://<previewId>.localhost:<gatewayPort>`.

- **Chrome / Firefox**: Resolve `*.localhost` to `127.0.0.1` natively. No DNS config needed.
- **Safari**: Requires a `/etc/hosts` entry: `127.0.0.1 <previewId>.localhost` for each preview.

### Docker Labels

All preview containers get `cw.*` labels for metadata retrieval:

| Label | Purpose |
|-------|---------|
| `cw.preview` | `"true"` — marker for filtering |
| `cw.initiative-id` | Initiative ID |
| `cw.phase-id` | Phase ID (optional) |
| `cw.project-id` | Project ID |
| `cw.branch` | Branch name |
| `cw.port` | Gateway port |
| `cw.preview-id` | Nanoid for this deployment |
| `cw.mode` | `"preview"` or `"dev"` |
| `cw.agent-id` | Agent ID (optional, set when started via `--agent`) |

### Compose Project Naming

- **Gateway**: `cw-preview-gateway` (single instance)
- **Previews**: `cw-preview-<nanoid>` — filtered via `docker compose ls --filter name=cw-preview`, gateway excluded from listings
- **Container names**: `cw-preview-<id>-<service>` — unique DNS names on the shared network

### Networking

- **`cw-preview-net`** — external Docker bridge network shared by gateway + all preview stacks
- **`internal`** — per-preview bridge network for inter-service communication
- Public services join both networks; internal services (e.g. databases) only join `internal`

## Config Discovery

See [Setting Up Preview Deployments](#setting-up-preview-deployments-for-a-project) above for the full config reference. Discovery order:

1. **`.cw-preview.yml`** — explicit CW preview config (recommended)
2. **`docker-compose.yml` / `compose.yml`** — existing compose file with gateway network injection
3. **`Dockerfile`** — single-service fallback (builds from `.`, assumes port 3000)

## Module Files

| File | Purpose |
|------|---------|
| `types.ts` | PreviewConfig, PreviewStatus, labels, constants, dev config types |
| `config-reader.ts` | Discovery + YAML parsing (including `dev` section) |
| `compose-generator.ts` | Per-preview Docker Compose YAML + label generation |
| `gateway.ts` | GatewayManager class + Caddyfile generation |
| `worktree.ts` | Git worktree create/remove helpers |
| `docker-client.ts` | Docker CLI wrapper (execa) + network operations |
| `health-checker.ts` | Service readiness polling via subdomain URL |
| `port-allocator.ts` | Port 9100-9200 allocation with TCP bind test |
| `manager.ts` | PreviewManager class (start/stop/list/status/stopAll + auto-start) |
| `index.ts` | Barrel exports |

## Events

| Event | Payload |
|-------|---------|
| `preview:building` | `{previewId, initiativeId, branch, gatewayPort, mode, phaseId?}` |
| `preview:ready` | `{previewId, initiativeId, branch, gatewayPort, url, mode, phaseId?}` |
| `preview:stopped` | `{previewId, initiativeId}` |
| `preview:failed` | `{previewId, initiativeId, error}` |

## Auto-Start

`PreviewManager.setupEventListeners()` listens for `phase:pending_review` events:
1. Loads the initiative and its projects
2. If exactly one project: auto-starts a preview in `preview` mode
3. Branch is derived from `phaseBranchName(initiative.branch, phase.name)`
4. Errors are caught and logged (best-effort, never blocks the phase transition)

## Agent Integration

Agents can spin up and tear down preview deployments using simplified commands that only require their agent ID — the server resolves initiative, project, branch, and mode automatically.

### Agent-Simplified Commands

When an agent has a `<preview_deployments>` section in its prompt (injected automatically if the initiative's project has `.cw-preview.yml`), it can use:

```
cw preview start --agent <agentId>    # Server resolves everything, starts dev mode
cw preview stop --agent <agentId>     # Stops all previews for this agent
```

### Prompt Injection

Preview instructions are automatically appended to agent prompts when all conditions are met:
1. Agent mode is `execute`, `refine`, or `discuss`
2. Agent has an `initiativeId`
3. Initiative has exactly one linked project
4. Project clone directory contains `.cw-preview.yml`

The injected `<preview_deployments>` block includes prefilled `cw preview start/stop --agent <agentId>` commands.

### Agent ID Label

Previews started with `--agent` receive a `cw.agent-id` Docker container label. This enables:
- **Auto-teardown**: When an agent stops (`agent:stopped` event), all previews labeled with its ID are automatically torn down (best-effort).
- **Agent-scoped stop**: `cw preview stop --agent <id>` finds and stops previews by label.

### Setup Command

`cw preview setup` prints inline setup instructions for `.cw-preview.yml`. With `--auto --project <id>`, it creates an initiative and spawns a refine agent to analyze the project and generate the config file.

## tRPC Procedures

| Procedure | Type | Input |
|-----------|------|-------|
| `startPreview` | mutation | `{initiativeId, phaseId?, projectId, branch, mode?, worktreePath?, agentId?}` |
| `startPreviewForAgent` | mutation | `{agentId}` |
| `stopPreview` | mutation | `{previewId}` |
| `stopPreviewByAgent` | mutation | `{agentId}` |
| `listPreviews` | query | `{initiativeId?}` |
| `getPreviewStatus` | query | `{previewId}` |

`mode` defaults to `'preview'`. Set to `'dev'` with a `worktreePath` for dev mode. `startPreviewForAgent` always uses dev mode.

## CLI Commands

```
cw preview start --initiative <id> --project <id> --branch <branch> [--phase <id>]
cw preview start --agent <id>
cw preview stop <previewId>
cw preview stop --agent <id>
cw preview list [--initiative <id>]
cw preview status <previewId>
cw preview setup [--auto --project <id> [--provider <name>]]
```

## Frontend

The Review tab shows preview status inline:
- **No preview**: "Start Preview" button
- **Building**: Spinner + "Building preview..."
- **Running**: Green dot + `http://<id>.localhost:<port>` link + Stop button
- **Failed**: Error message + Retry button

Polls `getPreviewStatus` with `refetchInterval: 3000` while active.

## Container Wiring

- `PreviewManager` instantiated in `apps/server/container.ts` with `(projectRepository, eventBus, workspaceRoot, phaseRepository, initiativeRepository)`
- Added to `Container` interface and `toContextDeps()`
- `GracefulShutdown` calls `previewManager.stopAll()` during shutdown
- `requirePreviewManager(ctx)` helper in `apps/server/trpc/routers/_helpers.ts`

## Codewalkers Self-Preview

Codewalkers itself has a Dockerfile, `.cw-preview.yml`, and seed script for preview deployments. This lets you demo the full app — initiatives, phases, tasks, agents with output, pages, and review tab with real git diffs.

### Files

| File | Purpose |
|------|---------|
| `Dockerfile` | Multi-stage: deps → build (server+web) → production (Node + Caddy) |
| `.cw-preview.yml` | Preview config: build, healthcheck, seed |
| `scripts/Caddyfile` | Caddy config: SPA file server + tRPC reverse proxy |
| `scripts/entrypoint.sh` | Init workspace, start backend, run Caddy |
| `scripts/seed-preview.sh` | Create demo git repo + run Node.js seed |
| `apps/server/preview-seed.ts` | Populate DB with demo initiative, phases, tasks, agents, log chunks, pages, review comments |

### Container Architecture

Single container running two processes:
- **Node.js backend** on port 3847 (tRPC API + health endpoint)
- **Caddy** on port 3000 (SPA file server, reverse proxy `/trpc` and `/health` to backend)

### Seed Data

The seed creates a "Task Manager Redesign" initiative with:
- 3 phases (completed, pending_review, in_progress)
- 9 tasks across phases
- 3 agents with JSONL log output
- Root page with Tiptap content
- 3 review comments on the pending_review phase
- Git repo with 3 branches and real commit diffs for the review tab

### Manual Testing

```sh
docker build -t cw-preview .
docker run -p 3000:3000 cw-preview
# Wait for startup, then seed:
docker exec <container> sh /app/scripts/seed-preview.sh
# Browse http://localhost:3000
```

## Dependencies

- `js-yaml` + `@types/js-yaml` — for parsing `.cw-preview.yml`
- `simple-git` — for git worktree operations
- Docker must be installed and running on the host