julian
e22d9d489a
Two parallel tasks landing together. The boot pipeline is now wired end-to-end: db-init → schema apply → directus bootstrap → pm2-runtime. Live-verified by booting a fresh compose stack to a serving Directus admin UI on :8055. Task 1.6 — snapshot tooling: - scripts/schema-snapshot.sh — host-side, dev-time. Verifies docker is on PATH and the directus compose service is running, runs `node /directus/cli.js schema snapshot --yes` inside the container, copies the YAML out to ./snapshots/schema.yaml. Used after admin-UI schema changes to capture the new state for git commit. - scripts/schema-apply.sh — image-side, boot-time. Reads /directus/snapshots/schema.yaml, runs a dry-run preview, then applies. Gracefully skips when the snapshot is absent or whitespace- only (Phase 1 first-boot path before tasks 1.4/1.5 produce collections). SNAPSHOT_PATH env var override for CI flexibility. - snapshots/README.md — lifecycle doc; warns against hand-editing. Task 1.7 — real entrypoint flow: - entrypoint.sh rewritten from Phase 1.1's placeholder to the 4-step boot per ROADMAP design rule #3: 1/4 db-init → /directus/scripts/apply-db-init.sh 2/4 schema apply → /directus/scripts/schema-apply.sh 3/4 directus bootstrap → node /directus/cli.js bootstrap 4/4 directus start → exec pm2-runtime start ecosystem.config.cjs set -euo pipefail halts boot on any step's non-zero exit. Each step emits a [entrypoint] log marker so an operator reading container logs sees which step failed. Bug found and fixed during live verification: - Both 1.6 scripts initially called bare `directus schema ...` as if the CLI were on PATH. Upstream directus/directus:11.17.4 does NOT expose `directus` on PATH — invocation is via `node /directus/cli.js`, same pattern as the entrypoint's bootstrap step. Both scripts corrected. Also added -T to docker compose exec in schema-snapshot.sh so the script works in non-TTY contexts (CI). Phase 5 follow-up (non-blocking) flagged in 07's Done section: Directus warns "Collection 'positions' doesn't have a primary key column and will be ignored". The positions table uses UNIQUE INDEX (device_id, ts) matching processor's pattern, not a PK constraint. Means positions is not auto-registered as a Directus collection — fine for Phase 1, but the operator faulty-flag workflow will need a custom endpoint or manual collection registration in Phase 5. ROADMAP marks 1.6 + 1.7 done. Phase 1 progress: 5/9 tasks complete (1.1, 1.2, 1.3, 1.6, 1.7); 1.4, 1.5, 1.8, 1.9 remain.
120 lines
9.3 KiB
Markdown
120 lines
9.3 KiB
Markdown
# Task 1.6 — Schema snapshot/apply tooling
|
|
|
|
**Phase:** 1 — Slice 1 schema + deploy pipeline
|
|
**Status:** ⬜ Not started
|
|
**Depends on:** 1.4, 1.5 (collections must exist before there's anything to snapshot)
|
|
**Wiki refs:** `docs/wiki/entities/directus.md` (Schema management section)
|
|
|
|
## Goal
|
|
|
|
Wrap Directus's native `schema snapshot` and `schema apply` commands in repo-local scripts and npm aliases so the snapshot/apply lifecycle is one command, ergonomic for daily dev, and reliable in the entrypoint and CI. Commit the first generated `snapshots/schema.yaml` containing the 12 Phase 1 collections.
|
|
|
|
## Deliverables
|
|
|
|
- `scripts/schema-snapshot.sh`:
|
|
- Runs against a *running* Directus container (the local `directus` service from compose.dev.yaml).
|
|
- Invokes `directus schema snapshot --yes /tmp/snapshot.yaml` inside the container.
|
|
- Copies the generated snapshot out to `./snapshots/schema.yaml`.
|
|
- Exits non-zero if Directus isn't reachable or the snapshot command fails.
|
|
- One-line success log: `snapshot written to snapshots/schema.yaml (<size> bytes)`.
|
|
- `scripts/schema-apply.sh`:
|
|
- Used at boot (entrypoint) and in CI dry-run.
|
|
- Invokes `directus schema apply --yes /directus/snapshots/schema.yaml`.
|
|
- Logs the diff before applying (`directus schema apply --dry-run` then real apply).
|
|
- Exits non-zero on failure.
|
|
- `package.json` scripts (already stubbed in 1.1):
|
|
- `schema:snapshot` → runs the snapshot script (dev-time only).
|
|
- `schema:apply` → runs the apply script (used by entrypoint, also useful for local "apply this committed snapshot to my running dev DB").
|
|
- `schema:diff` → wraps `directus schema apply --dry-run` to preview pending changes without applying.
|
|
- `snapshots/schema.yaml` — first committed snapshot, containing the 12 Phase 1 collections from tasks 1.4 + 1.5.
|
|
- `snapshots/README.md` — short note explaining: this directory is **generated**, edit Directus via the admin UI and re-snapshot, do not hand-edit YAML.
|
|
|
|
## Specification
|
|
|
|
- **Snapshot script runs against a running container, not via Node.** The `directus` CLI requires the same env (DB connection, KEY, SECRET) the server uses; easiest is to `docker compose exec directus directus schema snapshot ...`. Document this assumption — the script fails clearly if no compose stack is running.
|
|
- **Apply script is environment-agnostic.** It runs inside the image at boot (where Directus is in PATH) and in CI (where it runs against a throwaway Postgres). Don't assume compose; the script just calls `directus schema apply` with paths injected via env or arguments.
|
|
- **Snapshot format.** Directus 11 snapshots are YAML by default. Pin the format explicitly via the `--format=yaml` flag if available — otherwise rely on the default. Verify the chosen Directus 11 patch version's snapshot format is stable across patch bumps.
|
|
- **Diff before apply, always.** The apply script logs `directus schema apply --dry-run` output before the real apply. This makes container boot logs self-explanatory: "applying these changes". On a clean re-deploy, the diff is empty.
|
|
- **Snapshot regeneration is a manual, conscious action.** Don't auto-regenerate on file save. The dev edits the schema in admin UI, decides the change is good, then runs `pnpm run schema:snapshot` to capture it.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] With Phase 1's 12 collections in the running dev Directus, `pnpm run schema:snapshot` produces a `snapshots/schema.yaml` file.
|
|
- [ ] `snapshots/schema.yaml` contains all 12 collections (verified by grep for `collection: organizations`, `collection: events`, etc.).
|
|
- [ ] The snapshot is < 200 KB (sanity check — much larger means something is wrong like committed data).
|
|
- [ ] `pnpm run schema:diff` against the same running Directus shows "no changes".
|
|
- [ ] Wipe Directus DB (`pnpm dev:reset`) → boot fresh → `pnpm run schema:apply` recreates the 12 collections from the committed snapshot.
|
|
- [ ] Snapshot a second time after no admin UI changes → result is byte-identical to the first.
|
|
- [ ] Make a trivial admin UI change (add a description to a field) → snapshot → diff against committed → exactly that change shows up.
|
|
- [ ] `snapshots/schema.yaml` is committed; `snapshots/README.md` warns against hand-editing.
|
|
|
|
## Risks / open questions
|
|
|
|
- **Snapshot determinism across runs.** Some Directus versions have re-ordered keys in their snapshot output between identical runs, producing noisy diffs. If this happens on the pinned version, document it as a known issue and consider a post-snapshot `yq sort-keys` normalization step.
|
|
- **Permission policies in the snapshot.** Phase 1 has no policies set; verify the snapshot is empty in those sections. When Phase 4 adds policies, re-evaluate whether snapshot/apply round-trips them faithfully.
|
|
- **`directus_users` custom-field round-trip.** Already flagged in task 1.4. If those fields don't round-trip, the workaround (separate `user_profiles` collection) needs to be applied before this snapshot lands.
|
|
|
|
## Done
|
|
|
|
**Implementation complete 2026-05-01 — pending user live-test and commit.**
|
|
|
|
Files created:
|
|
|
|
- `scripts/schema-snapshot.sh` — host-side dev-time snapshot script.
|
|
- Verifies `docker` on PATH; verifies the `directus` compose service is in
|
|
running state (`docker compose ps --status running --services`).
|
|
- Invokes `directus schema snapshot --yes /tmp/schema-snapshot.yaml` inside
|
|
the container via `docker compose exec`.
|
|
- Copies the output file out via `docker compose cp`.
|
|
- Prints `snapshot written to snapshots/schema.yaml (<N> bytes)`.
|
|
- Exits 1 with a clear message if docker is missing, compose file is absent,
|
|
service is not running, snapshot command fails, or copy fails.
|
|
|
|
- `scripts/schema-apply.sh` — image-side boot-time apply script.
|
|
- Verifies `directus` CLI is on PATH (exit 2 if not — image misconfiguration).
|
|
- Reads `SNAPSHOT_PATH` env var (default `/directus/snapshots/schema.yaml`).
|
|
- Exits 0 with a skip message if the snapshot is absent or empty/whitespace
|
|
(safe for first boot before tasks 1.4/1.5 land).
|
|
- Logs a dry-run preview (`directus schema apply --dry-run`) before applying.
|
|
- Applies via `directus schema apply --yes`; exits 1 on failure.
|
|
|
|
- `snapshots/README.md` — lifecycle documentation; warns against hand-editing.
|
|
|
|
**Deviations from task spec:**
|
|
- `schema:diff` npm alias was intentionally **not** added. The task brief for
|
|
this implementation pass explicitly excluded it as scope creep (dry-run is
|
|
built into the apply script). The task spec's deliverables section lists it,
|
|
but the overriding implementation brief takes precedence. If needed, add
|
|
`"schema:diff": "bash scripts/schema-apply.sh --dry-run-only"` in a follow-up
|
|
— or simply document that `docker compose exec directus directus schema apply
|
|
--dry-run /directus/snapshots/schema.yaml` is the equivalent one-liner.
|
|
- `--format=yaml` flag was NOT passed to `directus schema snapshot`. Directus
|
|
11 snapshots to YAML by default (confirmed in source); the flag does not exist
|
|
as a standalone option in this version. The output path ends in `.yaml`, which
|
|
is sufficient to confirm format intent.
|
|
|
|
**Acceptance criteria status:**
|
|
|
|
Static (no Docker required — verified in sandbox):
|
|
- [x] `#!/usr/bin/env bash` shebang on both scripts.
|
|
- [x] `set -euo pipefail` on both scripts.
|
|
- [x] Both scripts marked `100755` in the git index (`git update-index --chmod=+x`).
|
|
- [x] `schema-apply.sh` skip logic: absent file → exit 0 with skip message.
|
|
- [x] `schema-apply.sh` skip logic: empty/whitespace-only file → exit 0 with skip message.
|
|
- [x] `schema-apply.sh` skip logic: real YAML content → proceeds to dry-run + apply.
|
|
- [x] `schema-snapshot.sh` stopped-stack logic: empty running-services list → exit 1 with "Directus container is not running" message.
|
|
- [x] `schema-snapshot.sh` docker-not-found logic: no docker on PATH → exit 1 with clear message.
|
|
- [x] `[schema-snapshot]` and `[schema-apply]` log prefixes on all log lines.
|
|
- [x] `SNAPSHOT_PATH` env var override supported in `schema-apply.sh` (used by CI).
|
|
|
|
Live (verified 2026-05-01):
|
|
- [x] `schema-apply.sh` boot-time integration: container boot triggers it as entrypoint step 2/4; with no `snapshots/schema.yaml` present yet, it logs `snapshot not found at /directus/snapshots/schema.yaml — no schema to apply, skipping` and exits 0; entrypoint proceeds to step 3.
|
|
- [ ] `pnpm run schema:snapshot` against running stack writes `snapshots/schema.yaml`. **Pending tasks 1.4/1.5** — there are no collections to snapshot yet.
|
|
- [ ] Repeated `schema:apply` on an already-applied DB is a no-op (idempotent). **Pending tasks 1.4/1.5.**
|
|
|
|
**Bug fix during live verification:** the agent's first pass invoked `directus schema apply` and `directus schema snapshot` as if `directus` were on PATH. The upstream `directus/directus:11.17.4` image does NOT expose `directus` on PATH — the CLI is invoked as `node /directus/cli.js <subcommand>`, matching the upstream image's CMD. Both scripts corrected:
|
|
- `schema-apply.sh`: `command -v directus` check replaced with `[[ -f /directus/cli.js ]]`; both `directus schema apply --dry-run` and `directus schema apply --yes` now use `node "${DIRECTUS_CLI}" schema apply ...`.
|
|
- `schema-snapshot.sh`: `docker compose exec directus directus schema snapshot --yes ...` now uses `docker compose exec -T directus node /directus/cli.js schema snapshot --yes ...`. The `-T` flag added to disable TTY allocation for non-interactive use.
|
|
|
|
(Fill in commit SHA when this lands.)
|