julian
a8e808e71c
Initial commit. Establishes the .planning/ tree mirroring processor's shape (ROADMAP.md as nav hub + per-phase folders with READMEs and granular task files). Six phases: 1. Slice 1 schema + deploy pipeline — what Rally Albania 2026 needs. Org catalog (orgs, users, vehicles, devices) + event participation (events, classes, entries, entry_crew, entry_devices). db-init/ for the positions hypertable + faulty column. snapshot/apply tooling. Gitea CI dry-run. Dogfood seed of Rally Albania 2026. Nine task files with full Goal / Deliverables / Specification / Acceptance criteria / Risks / Done sections. 2. Course definition — stages, segments, geofences, waypoints, SLZs. PostGIS extension introduced here. 3. Timing & penalty tables — co-developed with processor Phase 2. entry_segment_starts, entry_crossings, entry_penalties, stage_results, penalty_formulas. 4. Permissions & policies — Directus 11 dynamic-filter Policies per logical role. Deployment-time work, deferred to keep early phases focused on the data model. 5. Custom extensions — TypeScript hooks/endpoints implementing the cross-plane workflows the schema implies (faulty-flag → Redis stream emit, stage-open materializer, etc.). 6. Future / optional — retroactivity preview UI, command-routing Flows, audit trails, federation rule import. Not committed. Non-negotiable design rules captured in ROADMAP.md: schema authority in Directus + snapshot-as-code + db-init for non-Directus DDL + sequential idempotent migrations + entrypoint apply order + no application logic in Flows + permissions deferred to Phase 4. Architectural anchors point at the wiki at ../docs/wiki/ — the schema draft, the Rally Albania 2025 source page, plus the existing processor/postgres-timescaledb/live-channel pages. Each task file calls out the wiki refs an implementing agent should read first. README.md mirrors the processor service README structure: quick start, local Docker test, prod/stage deployment notes, env vars, CI behavior.
61 lines
4.7 KiB
Markdown
61 lines
4.7 KiB
Markdown
# Task 1.2 — db-init runner script
|
|
|
|
**Phase:** 1 — Slice 1 schema + deploy pipeline
|
|
**Status:** ⬜ Not started
|
|
**Depends on:** 1.1
|
|
**Wiki refs:** `docs/wiki/entities/postgres-timescaledb.md`, `docs/wiki/entities/directus.md` (Schema management section)
|
|
|
|
## Goal
|
|
|
|
Implement `scripts/apply-db-init.sh` — the boot-time runner that walks `db-init/*.sql` in numeric order, applies each via `psql` against the configured Postgres, and records successful applications in a `migrations_applied` guard table so re-runs are no-ops. This is the foundation Phase 1 (and every later phase) depends on for non-Directus DDL.
|
|
|
|
## Deliverables
|
|
|
|
- `scripts/apply-db-init.sh` — POSIX-compatible bash. Does the following, in order:
|
|
1. **Wait for Postgres readiness.** Loop calling `pg_isready -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_DATABASE` until success or timeout (configurable, default 60 s). Exit non-zero on timeout with a clear log message.
|
|
2. **Bootstrap the guard table.**
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS migrations_applied (
|
|
filename TEXT PRIMARY KEY,
|
|
applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
|
checksum TEXT NOT NULL
|
|
);
|
|
```
|
|
3. **Walk `db-init/*.sql` in numeric-prefix order** (sorted lexically; the `NNN_` prefix enforces order). For each file:
|
|
- Compute `sha256sum` of the file contents → `checksum`.
|
|
- Query `migrations_applied WHERE filename = <basename>`.
|
|
- If a row exists and the checksums match → log `skip filename` and continue.
|
|
- If a row exists and checksums DON'T match → log error and exit non-zero. (Migrations are append-only; never edit a file once applied.)
|
|
- If no row exists → apply the file via `psql -v ON_ERROR_STOP=1 -f <path>`. On success, insert the row. On failure, exit non-zero with the SQL error.
|
|
4. **Log a one-line summary** at the end: `db-init complete: <N> applied, <M> skipped`.
|
|
|
|
## Specification
|
|
|
|
- **Environment variables expected:** `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASSWORD`, `DB_DATABASE`. Plus `DB_INIT_DIR` (default `/directus/db-init`) and `DB_INIT_TIMEOUT_SECONDS` (default `60`).
|
|
- **Use `PGPASSWORD` for psql auth** — exported in the script before `psql` calls, never printed in logs.
|
|
- **Each migration runs in a single transaction** by virtue of `psql -v ON_ERROR_STOP=1 -1 -f`. The `-1` flag wraps the whole file in `BEGIN/COMMIT`. (Some statements like `CREATE EXTENSION` or `CREATE INDEX CONCURRENTLY` can't run in a transaction — those go in their own files without `-1` if needed. Document the exception inline.)
|
|
- **Numeric-prefix convention.** `001_`, `002_`, …, `999_`. Pad to 3 digits; gives 999 slots which is well beyond what we'll need.
|
|
- **Filename uniqueness.** Two files can't share a prefix. Lint check at script start: detect collisions, error out before applying anything.
|
|
- **Logging.** One line per file at INFO level. Failure logs include the psql exit code and the offending file. No SQL output to stdout (verbose `psql` output goes to stderr and is suppressed unless `DEBUG=1` is set).
|
|
- **Idempotency.** Running the script twice in a row → second run does zero psql work beyond the readiness check + guard-table query.
|
|
- **Exit codes.** `0` = success, `1` = readiness timeout, `2` = checksum mismatch, `3` = psql error, `4` = filename collision.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] Script is executable (`chmod +x`), shebang is `#!/usr/bin/env bash`.
|
|
- [ ] `set -euo pipefail` at the top.
|
|
- [ ] Against a fresh Postgres, no `db-init/*.sql` files yet → script creates `migrations_applied` table, prints "0 applied, 0 skipped", exits 0.
|
|
- [ ] After 1.3 lands, script applies all three migrations on first run (3 applied, 0 skipped), no-ops on second run (0 applied, 3 skipped).
|
|
- [ ] Manually editing an applied file → next run exits 2 with a clear "checksum mismatch" error.
|
|
- [ ] Adding two files with the same numeric prefix → script exits 4 before applying anything.
|
|
- [ ] Killing Postgres mid-run during file 002 → script exits 3 with the psql error; on next run, file 002 retries cleanly.
|
|
|
|
## Risks / open questions
|
|
|
|
- **`CREATE EXTENSION` inside a transaction.** Some Postgres extensions can be created inside a transaction (timescaledb, postgis), some cannot (pg_partman with parallel apply). For Phase 1 the only extension is timescaledb, which is fine. Re-evaluate per phase.
|
|
- **Concurrent boots.** If two Directus containers boot against the same DB at the same time (rolling deploy), both will try to apply migrations. The guard table's `PRIMARY KEY` on `filename` makes the insert race-safe, but two containers running the *same* `psql -f` at once is risky. Mitigation for Phase 1: assume single-replica boot during deploy; Phase 3+ revisit if rolling deploy is a goal.
|
|
|
|
## Done
|
|
|
|
(Fill in commit SHA + one-line note when this lands.)
|