docs: Fix migration workflow — document hand-written SQL + journal registration
drizzle-kit generate has been broken since migration 0008 (stale snapshots). The actual workflow is hand-written SQL + manual _journal.json registration. Updated CLAUDE.md and database-migrations.md to reflect reality and prevent future migrations from silently failing to apply.
This commit is contained in:
@@ -24,7 +24,7 @@ Pre-implementation design docs are archived in `docs/archive/`.
|
|||||||
|
|
||||||
## Key Rules
|
## Key Rules
|
||||||
|
|
||||||
- **Database**: Never use raw SQL for schema initialization. Use `drizzle-kit generate` and the migration system. See [docs/database-migrations.md](docs/database-migrations.md).
|
- **Database migrations**: Hand-write SQL in `apps/server/drizzle/NNNN_name.sql` AND register it in `apps/server/drizzle/meta/_journal.json` (increment `idx`, set `tag` to filename without `.sql`). Both files must be committed together. Do NOT use `drizzle-kit generate` — snapshots are stale since migration 0008. See [docs/database-migrations.md](docs/database-migrations.md).
|
||||||
- **Logging**: Use `createModuleLogger()` from `apps/server/logger/index.ts`. Keep `console.log` for CLI user-facing output only.
|
- **Logging**: Use `createModuleLogger()` from `apps/server/logger/index.ts`. Keep `console.log` for CLI user-facing output only.
|
||||||
- **Hexagonal architecture**: Repository ports in `apps/server/db/repositories/*.ts`, Drizzle adapters in `apps/server/db/repositories/drizzle/*.ts`. All re-exported from `apps/server/db/index.ts`.
|
- **Hexagonal architecture**: Repository ports in `apps/server/db/repositories/*.ts`, Drizzle adapters in `apps/server/db/repositories/drizzle/*.ts`. All re-exported from `apps/server/db/index.ts`.
|
||||||
- **tRPC context**: Optional repos accessed via `require*Repository()` helpers in `apps/server/trpc/routers/_helpers.ts`.
|
- **tRPC context**: Optional repos accessed via `require*Repository()` helpers in `apps/server/trpc/routers/_helpers.ts`.
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
# Database Migrations
|
# Database Migrations
|
||||||
|
|
||||||
This project uses [drizzle-kit](https://orm.drizzle.team/kit-docs/overview) for database schema management and migrations.
|
This project uses [drizzle-orm](https://orm.drizzle.team/) for database schema management with **hand-written SQL migrations**.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
- **Schema definition:** `apps/server/db/schema.ts` (drizzle-orm table definitions)
|
- **Schema definition:** `apps/server/db/schema.ts` (drizzle-orm table definitions)
|
||||||
- **Migration output:** `apps/server/drizzle/` directory (SQL files + meta journal)
|
- **Migration output:** `apps/server/drizzle/` directory (SQL files + `meta/_journal.json`)
|
||||||
- **Config:** `drizzle.config.ts`
|
- **Config:** `drizzle.config.ts`
|
||||||
- **Runtime migrator:** `apps/server/db/ensure-schema.ts` (calls `drizzle-orm/better-sqlite3/migrator`)
|
- **Runtime migrator:** `apps/server/db/ensure-schema.ts` (calls `drizzle-orm/better-sqlite3/migrator`)
|
||||||
|
|
||||||
@@ -13,17 +13,55 @@ This project uses [drizzle-kit](https://orm.drizzle.team/kit-docs/overview) for
|
|||||||
|
|
||||||
On every server startup, `ensureSchema(db)` runs all pending migrations from the `apps/server/drizzle/` folder. Drizzle tracks applied migrations in a `__drizzle_migrations` table so only new migrations are applied. This is safe to call repeatedly.
|
On every server startup, `ensureSchema(db)` runs all pending migrations from the `apps/server/drizzle/` folder. Drizzle tracks applied migrations in a `__drizzle_migrations` table so only new migrations are applied. This is safe to call repeatedly.
|
||||||
|
|
||||||
|
The migrator discovers migrations via `apps/server/drizzle/meta/_journal.json` — **not** by scanning the filesystem. A migration SQL file that isn't registered in the journal will never be applied.
|
||||||
|
|
||||||
## Workflow
|
## Workflow
|
||||||
|
|
||||||
### Making schema changes
|
### Making schema changes
|
||||||
|
|
||||||
|
**Do NOT use `drizzle-kit generate`** — the snapshots in `meta/` have been stale since migration 0008 and `drizzle-kit generate` will produce incorrect interactive prompts. All migrations since 0008 are hand-written.
|
||||||
|
|
||||||
1. Edit `apps/server/db/schema.ts` with your table/column changes
|
1. Edit `apps/server/db/schema.ts` with your table/column changes
|
||||||
2. Generate a migration:
|
2. Create a new SQL migration file: `apps/server/drizzle/NNNN_descriptive_name.sql`
|
||||||
```bash
|
- Number it sequentially (check the last migration number)
|
||||||
npx drizzle-kit generate
|
- Write the SQL (ALTER TABLE, CREATE INDEX, etc.)
|
||||||
```
|
3. **Register it in the journal**: edit `apps/server/drizzle/meta/_journal.json`
|
||||||
3. Review the generated SQL in `apps/server/drizzle/NNNN_*.sql`
|
- Add a new entry at the end of the `entries` array:
|
||||||
4. Commit the migration file along with your schema change
|
```json
|
||||||
|
{
|
||||||
|
"idx": <next_number>,
|
||||||
|
"version": "6",
|
||||||
|
"when": <unix_timestamp_ms>,
|
||||||
|
"tag": "NNNN_descriptive_name",
|
||||||
|
"breakpoints": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
- `idx`: sequential (previous entry's idx + 1)
|
||||||
|
- `tag`: migration filename **without `.sql` extension**
|
||||||
|
- `when`: any timestamp in milliseconds (e.g., previous + 86400000)
|
||||||
|
4. Commit **both** the SQL file and the updated `_journal.json` together
|
||||||
|
5. Run `npm run build && npm link` to pick up the changes
|
||||||
|
|
||||||
|
### Example
|
||||||
|
|
||||||
|
Adding a column to an existing table:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- apps/server/drizzle/0032_add_comment_threading.sql
|
||||||
|
ALTER TABLE review_comments ADD COLUMN parent_comment_id TEXT REFERENCES review_comments(id) ON DELETE CASCADE;
|
||||||
|
CREATE INDEX review_comments_parent_id_idx ON review_comments(parent_comment_id);
|
||||||
|
```
|
||||||
|
|
||||||
|
```json
|
||||||
|
// In meta/_journal.json entries array:
|
||||||
|
{
|
||||||
|
"idx": 32,
|
||||||
|
"version": "6",
|
||||||
|
"when": 1772323200000,
|
||||||
|
"tag": "0032_add_comment_threading",
|
||||||
|
"breakpoints": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
### Applying migrations
|
### Applying migrations
|
||||||
|
|
||||||
@@ -31,20 +69,12 @@ Migrations are applied automatically on server startup. No manual step needed.
|
|||||||
|
|
||||||
For tests, the same `ensureSchema()` function is called on in-memory SQLite databases in `apps/server/db/repositories/drizzle/test-helpers.ts`.
|
For tests, the same `ensureSchema()` function is called on in-memory SQLite databases in `apps/server/db/repositories/drizzle/test-helpers.ts`.
|
||||||
|
|
||||||
### Checking migration status
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# See what drizzle-kit would generate (dry run)
|
|
||||||
npx drizzle-kit generate --dry-run
|
|
||||||
|
|
||||||
# Open drizzle studio to inspect the database
|
|
||||||
npx drizzle-kit studio
|
|
||||||
```
|
|
||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
- **Never hand-write migration SQL.** Always use `drizzle-kit generate` from the schema.
|
- **Always hand-write migration SQL.** Do not use `drizzle-kit generate` (stale snapshots).
|
||||||
|
- **Always register in `_journal.json`.** The migrator reads the journal, not the filesystem.
|
||||||
|
- **Commit SQL + journal together.** A migration file without a journal entry is invisible to the migrator.
|
||||||
- **Never use raw CREATE TABLE statements** for schema initialization. The migration system handles this.
|
- **Never use raw CREATE TABLE statements** for schema initialization. The migration system handles this.
|
||||||
- **Always commit migration files.** They are the source of truth for database evolution.
|
|
||||||
- **Migration files are immutable.** Once committed, never edit them. Make a new migration instead.
|
- **Migration files are immutable.** Once committed, never edit them. Make a new migration instead.
|
||||||
- **Test with `npx vitest run`** after generating migrations to verify they work with in-memory databases.
|
- **Test with `npm test`** after creating migrations to verify they work with in-memory databases.
|
||||||
|
- **Keep schema.ts in sync.** The schema file is the source of truth for TypeScript types; migrations are the source of truth for database DDL. Both must reflect the same structure.
|
||||||
|
|||||||
Reference in New Issue
Block a user