directus/.planning/phase-1-slice-1-schema/06-snapshot-tooling.md

Task 1.6 — Schema snapshot/apply tooling

Phase: 1 — Slice 1 schema + deploy pipeline Status: ⬜ Not started Depends on: 1.4, 1.5 (collections must exist before there's anything to snapshot) Wiki refs: docs/wiki/entities/directus.md (Schema management section)

Goal

Wrap Directus's native schema snapshot and schema apply commands in repo-local scripts and npm aliases so the snapshot/apply lifecycle is one command, ergonomic for daily dev, and reliable in the entrypoint and CI. Commit the first generated snapshots/schema.yaml containing the 12 Phase 1 collections.

Deliverables

• scripts/schema-snapshot.sh:
  • Runs against a running Directus container (the local directus service from compose.dev.yaml).
  • Invokes directus schema snapshot --yes /tmp/snapshot.yaml inside the container.
  • Copies the generated snapshot out to ./snapshots/schema.yaml.
  • Exits non-zero if Directus isn't reachable or the snapshot command fails.
  • One-line success log: snapshot written to snapshots/schema.yaml (<size> bytes).
• scripts/schema-apply.sh:
  • Used at boot (entrypoint) and in CI dry-run.
  • Invokes directus schema apply --yes /directus/snapshots/schema.yaml.
  • Logs the diff before applying (directus schema apply --dry-run then real apply).
  • Exits non-zero on failure.
• package.json scripts (already stubbed in 1.1):
  • schema:snapshot → runs the snapshot script (dev-time only).
  • schema:apply → runs the apply script (used by entrypoint, also useful for local "apply this committed snapshot to my running dev DB").
  • schema:diff → wraps directus schema apply --dry-run to preview pending changes without applying.
• snapshots/schema.yaml — first committed snapshot, containing the 12 Phase 1 collections from tasks 1.4 + 1.5.
• snapshots/README.md — short note explaining: this directory is generated, edit Directus via the admin UI and re-snapshot, do not hand-edit YAML.

Specification

• Snapshot script runs against a running container, not via Node. The directus CLI requires the same env (DB connection, KEY, SECRET) the server uses; easiest is to docker compose exec directus directus schema snapshot .... Document this assumption — the script fails clearly if no compose stack is running.
• Apply script is environment-agnostic. It runs inside the image at boot (where Directus is in PATH) and in CI (where it runs against a throwaway Postgres). Don't assume compose; the script just calls directus schema apply with paths injected via env or arguments.
• Snapshot format. Directus 11 snapshots are YAML by default. Pin the format explicitly via the --format=yaml flag if available — otherwise rely on the default. Verify the chosen Directus 11 patch version's snapshot format is stable across patch bumps.
• Diff before apply, always. The apply script logs directus schema apply --dry-run output before the real apply. This makes container boot logs self-explanatory: "applying these changes". On a clean re-deploy, the diff is empty.
• Snapshot regeneration is a manual, conscious action. Don't auto-regenerate on file save. The dev edits the schema in admin UI, decides the change is good, then runs pnpm run schema:snapshot to capture it.

Acceptance criteria

• With Phase 1's 12 collections in the running dev Directus, pnpm run schema:snapshot produces a snapshots/schema.yaml file.
• snapshots/schema.yaml contains all 12 collections (verified by grep for collection: organizations, collection: events, etc.).
• The snapshot is < 200 KB (sanity check — much larger means something is wrong like committed data).
• pnpm run schema:diff against the same running Directus shows "no changes".
• Wipe Directus DB (pnpm dev:reset) → boot fresh → pnpm run schema:apply recreates the 12 collections from the committed snapshot.
• Snapshot a second time after no admin UI changes → result is byte-identical to the first.
• Make a trivial admin UI change (add a description to a field) → snapshot → diff against committed → exactly that change shows up.
• snapshots/schema.yaml is committed; snapshots/README.md warns against hand-editing.

Risks / open questions

• Snapshot determinism across runs. Some Directus versions have re-ordered keys in their snapshot output between identical runs, producing noisy diffs. If this happens on the pinned version, document it as a known issue and consider a post-snapshot yq sort-keys normalization step.
• Permission policies in the snapshot. Phase 1 has no policies set; verify the snapshot is empty in those sections. When Phase 4 adds policies, re-evaluate whether snapshot/apply round-trips them faithfully.
• directus_users custom-field round-trip. Already flagged in task 1.4. If those fields don't round-trip, the workaround (separate user_profiles collection) needs to be applied before this snapshot lands.

Done

Implementation complete 2026-05-01 — pending user live-test and commit.

Files created:

• scripts/schema-snapshot.sh — host-side dev-time snapshot script.

  • Verifies docker on PATH; verifies the directus compose service is in running state (docker compose ps --status running --services).
  • Invokes directus schema snapshot --yes /tmp/schema-snapshot.yaml inside the container via docker compose exec.
  • Copies the output file out via docker compose cp.
  • Prints snapshot written to snapshots/schema.yaml (<N> bytes).
  • Exits 1 with a clear message if docker is missing, compose file is absent, service is not running, snapshot command fails, or copy fails.

• scripts/schema-apply.sh — image-side boot-time apply script.

  • Verifies directus CLI is on PATH (exit 2 if not — image misconfiguration).
  • Reads SNAPSHOT_PATH env var (default /directus/snapshots/schema.yaml).
  • Exits 0 with a skip message if the snapshot is absent or empty/whitespace (safe for first boot before tasks 1.4/1.5 land).
  • Logs a dry-run preview (directus schema apply --dry-run) before applying.
  • Applies via directus schema apply --yes; exits 1 on failure.

• snapshots/README.md — lifecycle documentation; warns against hand-editing.

Deviations from task spec:

• schema:diff npm alias was intentionally not added. The task brief for this implementation pass explicitly excluded it as scope creep (dry-run is built into the apply script). The task spec's deliverables section lists it, but the overriding implementation brief takes precedence. If needed, add "schema:diff": "bash scripts/schema-apply.sh --dry-run-only" in a follow-up — or simply document that docker compose exec directus directus schema apply --dry-run /directus/snapshots/schema.yaml is the equivalent one-liner.
• --format=yaml flag was NOT passed to directus schema snapshot. Directus 11 snapshots to YAML by default (confirmed in source); the flag does not exist as a standalone option in this version. The output path ends in .yaml, which is sufficient to confirm format intent.

Acceptance criteria status:

Static (no Docker required — verified in sandbox):

• #!/usr/bin/env bash shebang on both scripts.
• set -euo pipefail on both scripts.
• Both scripts marked 100755 in the git index (git update-index --chmod=+x).
• schema-apply.sh skip logic: absent file → exit 0 with skip message.
• schema-apply.sh skip logic: empty/whitespace-only file → exit 0 with skip message.
• schema-apply.sh skip logic: real YAML content → proceeds to dry-run + apply.
• schema-snapshot.sh stopped-stack logic: empty running-services list → exit 1 with "Directus container is not running" message.
• schema-snapshot.sh docker-not-found logic: no docker on PATH → exit 1 with clear message.
• [schema-snapshot] and [schema-apply] log prefixes on all log lines.
• SNAPSHOT_PATH env var override supported in schema-apply.sh (used by CI).

Live (verified 2026-05-01):

• schema-apply.sh boot-time integration: container boot triggers it as entrypoint step 2/4; with no snapshots/schema.yaml present yet, it logs snapshot not found at /directus/snapshots/schema.yaml — no schema to apply, skipping and exits 0; entrypoint proceeds to step 3.
• pnpm run schema:snapshot against running stack writes snapshots/schema.yaml. Pending tasks 1.4/1.5 — there are no collections to snapshot yet.
• Repeated schema:apply on an already-applied DB is a no-op (idempotent). Pending tasks 1.4/1.5.

Bug fix during live verification: the agent's first pass invoked directus schema apply and directus schema snapshot as if directus were on PATH. The upstream directus/directus:11.17.4 image does NOT expose directus on PATH — the CLI is invoked as node /directus/cli.js <subcommand>, matching the upstream image's CMD. Both scripts corrected:

• schema-apply.sh: command -v directus check replaced with [[ -f /directus/cli.js ]]; both directus schema apply --dry-run and directus schema apply --yes now use node "${DIRECTUS_CLI}" schema apply ....
• schema-snapshot.sh: docker compose exec directus directus schema snapshot --yes ... now uses docker compose exec -T directus node /directus/cli.js schema snapshot --yes .... The -T flag added to disable TTY allocation for non-interactive use.

(Fill in commit SHA when this lands.)