Third CI dry-run failure: schema-apply tried to "Create migrations_applied"
and "Create positions" as Directus collections — both already exist as
raw tables created by db-init pre-schema. The conflict halts schema-apply
on a fresh CI DB.
Why these end up in the snapshot at all: `directus schema snapshot`
auto-discovers every table in the public schema, including ones owned by
db-init (positions hypertable, migrations_applied guard). It registers
them as ghost entries with no fields and no relations — just enough
metadata to make Directus aware of the table.
In local dev this never tripped because the tables existed BEFORE the
snapshot ran, and any subsequent apply was a no-op against directus_collections
which already had matching ghost rows. On a fresh CI DB the order is:
1. db-init pre-schema → creates the tables
2. bootstrap → installs Directus system tables (NOT the ghosts)
3. schema-apply → tries to "Create" the ghosts → conflict → fail
Fixes:
- snapshots/schema.yaml: stripped the migrations_applied and positions
entries (24 lines each) from the collections: section. The user
collections remain untouched.
- scripts/schema-snapshot.sh: post-process step that filters the same
ghost names from every future snapshot capture. Awk-based, applied
after `docker compose cp` writes the file out. The ghost list is a
bash array near the top of the new step — add to it when introducing
more db-init-only tables.
Snapshot is now 105 KB → ~103 KB. The user collections, fields, and
relations are unchanged. positions and migrations_applied stay as
raw Postgres tables managed by db-init/, never registered in
directus_collections, never shown in the admin UI. That matches the
schema-as-code split: Directus owns user collections; db-init owns the
positions hypertable and the runner's guard table.
Three CI iterations to get the boot pipeline right (port collision →
ordering → ghost entries). The dry-run gate has now caught three
distinct failure modes that would have damaged stage if pushed
unguarded.
Two parallel tasks landing together. The boot pipeline is now wired
end-to-end: db-init → schema apply → directus bootstrap → pm2-runtime.
Live-verified by booting a fresh compose stack to a serving Directus
admin UI on :8055.
Task 1.6 — snapshot tooling:
- scripts/schema-snapshot.sh — host-side, dev-time. Verifies docker
is on PATH and the directus compose service is running, runs
`node /directus/cli.js schema snapshot --yes` inside the container,
copies the YAML out to ./snapshots/schema.yaml. Used after admin-UI
schema changes to capture the new state for git commit.
- scripts/schema-apply.sh — image-side, boot-time. Reads
/directus/snapshots/schema.yaml, runs a dry-run preview, then
applies. Gracefully skips when the snapshot is absent or whitespace-
only (Phase 1 first-boot path before tasks 1.4/1.5 produce
collections). SNAPSHOT_PATH env var override for CI flexibility.
- snapshots/README.md — lifecycle doc; warns against hand-editing.
Task 1.7 — real entrypoint flow:
- entrypoint.sh rewritten from Phase 1.1's placeholder to the
4-step boot per ROADMAP design rule #3:
1/4 db-init → /directus/scripts/apply-db-init.sh
2/4 schema apply → /directus/scripts/schema-apply.sh
3/4 directus bootstrap → node /directus/cli.js bootstrap
4/4 directus start → exec pm2-runtime start ecosystem.config.cjs
set -euo pipefail halts boot on any step's non-zero exit. Each step
emits a [entrypoint] log marker so an operator reading container
logs sees which step failed.
Bug found and fixed during live verification:
- Both 1.6 scripts initially called bare `directus schema ...` as if
the CLI were on PATH. Upstream directus/directus:11.17.4 does NOT
expose `directus` on PATH — invocation is via `node /directus/cli.js`,
same pattern as the entrypoint's bootstrap step. Both scripts
corrected. Also added -T to docker compose exec in schema-snapshot.sh
so the script works in non-TTY contexts (CI).
Phase 5 follow-up (non-blocking) flagged in 07's Done section: Directus
warns "Collection 'positions' doesn't have a primary key column and
will be ignored". The positions table uses UNIQUE INDEX (device_id, ts)
matching processor's pattern, not a PK constraint. Means positions is
not auto-registered as a Directus collection — fine for Phase 1, but
the operator faulty-flag workflow will need a custom endpoint or
manual collection registration in Phase 5.
ROADMAP marks 1.6 + 1.7 done. Phase 1 progress: 5/9 tasks complete
(1.1, 1.2, 1.3, 1.6, 1.7); 1.4, 1.5, 1.8, 1.9 remain.
julian