Files

T

julian 25a9731070 Task 1.3 — Initial migrations

Three SQL files under db-init/ create the schema processor writes
against. All three apply cleanly via apply-db-init.sh, are idempotent
on re-run, and end with assertion blocks that catch silent
schema drift.

001_extensions.sql — registers timescaledb on the directus database.
  PostGIS deferred to Phase 2 (per Plan A). The timescaledb-ha image
  pre-creates the extension at DB init, so the IF NOT EXISTS guard
  fires as a NOTICE — expected and harmless.

002_positions_hypertable.sql — positions hypertable, exact
  column-by-column match against processor/src/db/migrations/0001_positions.sql.

  Cross-checking against processor surfaced 8 divergences from the
  original task spec; processor wins in every case (it is the writer
  and is in production). The corrections:

    - added ingested_at timestamptz NOT NULL DEFAULT now()
    - added codec text NOT NULL
    - altitude/angle/speed: real NOT NULL (not DOUBLE PRECISION nullable)
    - satellites/priority: NOT NULL
    - removed attributes DEFAULT '{}'::jsonb (processor always writes)
    - replaced PRIMARY KEY with UNIQUE INDEX positions_device_ts
      (idiomatic for TimescaleDB hypertables)
    - chunk interval 1 day, not 7 days
    - two indexes (positions_device_ts + positions_ts), not one composite

  Without these corrections every processor INSERT would have failed
  with NOT NULL violations. Spec deliverables section updated to
  reflect the correct shape so future readers see the right schema.

003_faulty_column.sql — adds the operator-controlled faulty boolean
  flag plus the partial index positions_faulty_idx ON (device_id,
  ts DESC) WHERE faulty = FALSE. The column is set only via Directus
  admin (Phase 4 permissions); processor's writer never touches it.
  The partial index optimises the hot-path read pattern (every
  processor evaluator filters faulty = FALSE); operator queries that
  look at faulty rows specifically use the broader positions_device_ts
  index from 002.

Live-verified 2026-05-01:
  - First apply: 3 applied, 0 skipped, exit 0.
  - Re-run: 0 applied, 3 skipped, exit 0.
  - All 13 columns present with correct types/nullability/defaults.
  - Hypertable registered with 1-day chunk interval.
  - Three expected indexes present.

Non-blocking observation: TimescaleDB's create_hypertable()
auto-created a fourth index (positions_ts_idx) duplicating our
explicit positions_ts. Processor's migration has the same redundancy
so stage already lives with this. Cleanup path documented in the
task spec for Phase 3 hardening (create_default_indexes => FALSE
in the create_hypertable call).

ROADMAP marks 1.3 done; 1.4 next.

2026-05-01 22:52:06 +02:00

8.5 KiB

Raw Blame History

directus — Roadmap

The TRM business plane. Directus 11 instance owning the relational schema and exposing it via REST/GraphQL/WebSockets/Admin UI. Schema-as-code via snapshots/ + db-init/, applied at container startup.

This file is the single navigation hub for all implementation planning. Each phase has its own folder with a README and granular task files. Update statuses here as work lands.

Status legend

Symbol	Meaning
⬜	Not started
🟦	Planned (designed, not coded)
🟨	In progress
🟩	Done
⏸	Paused / blocked
❄️	Frozen / future / optional

Architectural anchors

The service is specified by the wiki at ../docs/wiki/. Implementing agents should read these pages before starting any task:

Architecture — docs/wiki/sources/gps-tracking-architecture.md, docs/wiki/concepts/plane-separation.md, docs/wiki/concepts/failure-domains.md
This service — docs/wiki/entities/directus.md
Schema design — docs/wiki/synthesis/directus-schema-draft.md
Reference rulebook — docs/wiki/sources/rally-albania-regulations-2025.md (canonical real-world fixture for federation rule shapes)
Downstream / sibling — docs/wiki/entities/postgres-timescaledb.md, docs/wiki/entities/processor.md, docs/wiki/concepts/live-channel-architecture.md

Non-negotiable design rules

These rules govern every task. Any deviation must be discussed and documented as a decision before code lands.

Schema authority lives in Directus. Collections, fields, relations are defined through Directus and round-tripped via directus schema snapshot. The exception is the positions hypertable (owned by processor) and any other DDL Directus cannot represent (PostGIS-specific syntax, custom indexes, hypertable creation) — those live in db-init/*.sql.
db-init/*.sql is sequential, idempotent, and guarded. Files numbered NNN_name.sql. Each is internally idempotent (IF NOT EXISTS, ADD COLUMN IF NOT EXISTS). The runner skips files already recorded in migrations_applied. Manual application of out-of-order files is forbidden.
Apply order at boot: db-init runner → directus schema apply --yes → directus start. Any failure halts boot. Implemented in entrypoint.sh.
Snapshot lives in git, edited only via the admin UI. Hand-editing snapshots/schema.yaml is forbidden — round-trip through the UI keeps the format consistent with what directus schema snapshot produces.
One PR = one snapshot regeneration. PRs that change schema include the regenerated snapshot. CI verifies the snapshot matches what directus schema snapshot would produce against an applied database.
No application logic in Flows. Flows are reserved for declarative orchestration (notifications, simple field updates, webhook routing). Domain logic lives in extensions/ (TypeScript hooks/endpoints) where it is reviewed, tested, and version-controlled like any other code.
Permissions are a separate phase. Adding a collection in Phase 1–3 does NOT come with its access policies — those land deliberately in Phase 4. Until then collections are admin-only by default. This avoids premature commitment to role definitions before the data model is settled.
Image starts from directus/directus:11.x. No forking the upstream image. Customizations are: bundled extensions under /directus/extensions/, snapshot/db-init artifacts under /directus/snapshots/ and /directus/db-init/, and an entrypoint wrapper.

Phases

Phase 1 — Slice 1 schema + deploy pipeline

Status: 🟨 In progress (1.1, 1.2, 1.3 done; 1.4 next) Outcome: A Directus instance with the org-level catalog (orgs, users, organization_users, vehicles, devices and their org junctions) and event-participation collections (events, classes, entries, entry_crew, entry_devices) live and snapshot-tracked. db-init/ covers the TimescaleDB extension, the positions hypertable, and the faulty column. Image builds via Gitea Actions with a CI dry-run that catches snapshot drift before deploy. Rally Albania 2026 is registered as the first event in admin UI to dogfood the registration workflow. This is what Rally Albania 2026 needs.

See phase-1-slice-1-schema/README.md

#	Task	Status	Landed in
1.1	Project scaffold	🟩	pending user commit
1.2	db-init runner script	🟩	pending user commit
1.3	Initial migrations (extensions, positions hypertable, faulty column)	🟩	pending user commit
1.4	Org-level catalog collections	⬜	—
1.5	Event-participation collections	⬜	—
1.6	Schema snapshot/apply tooling	⬜	—
1.7	Image build & entrypoint	⬜	—
1.8	Gitea CI dry-run workflow	⬜	—
1.9	Rally Albania 2026 dogfood seed	⬜	—

Phase 2 — Course definition

Status: ⬜ Not started — depends on Phase 1 Outcome: Stages, segments, geofences (PostGIS polygons), waypoints, and speed_limit_zones as data-layer collections. Operators can define an event's full course before each stage. No processor logic yet — Phase 2 of processor consumes this data and writes crossings/penalties.

See phase-2-course-definition/README.md

Phase 3 — Timing & penalty tables

Status: ⬜ Not started — co-developed with processor Phase 2 Outcome: entry_segment_starts, entry_crossings, entry_penalties, stage_results, and penalty_formulas collections. The schema half of the paired schema/code work that produces real timing results. Penalty evaluator registry shipped on the processor side; rule numeric values shipped here.

See phase-3-timing-and-penalty-tables/README.md

Phase 4 — Permissions & policies

Status: ⬜ Not started — depends on Phases 1–3 Outcome: Dynamic-filter Policies per logical role (org-admin, race-director, marshal, timekeeper, participant, …) covering each collection × action. Multi-tenant isolation enforced by Directus, not by application code. Deployment-time work, not architectural.

See phase-4-permissions-and-policies/README.md

Phase 5 — Custom extensions

Status: ⬜ Not started — depends on Phase 3 Outcome: TypeScript extensions implementing the cross-plane workflows the schema implies: faulty-flag → recompute:requests stream emit; events.discipline validation hook; stage-open trigger materializing entry_segment_starts; CP closing-time computation; entry registration "copy crew from previous entry" custom endpoint.

See phase-5-custom-extensions/README.md

Phase 6 — Future / optional

Status: ❄️ Not committed See phase-6-future/README.md

Ideas on radar: retroactivity preview UI for geometry edits (Phase 2.5 of processor — needs a UI counterpart here), command-routing Flows (phase-2-commands), audit trail extensions, federation rule import tooling.

Operating model

Implementation agent contract. Each task file is self-sufficient: goal, deliverables, specification, acceptance criteria. An agent should be able to complete one task without reading the whole wiki — but should skim the wiki references at the top of the task before starting.
Sequence within a phase. Task numbering reflects intended order. Soft dependencies are explicit in each task's "Depends on" field. Tasks with no dependencies on each other can be done in parallel.
Status updates. When a task is started, change its row in this ROADMAP to 🟨 and the task file's status badge accordingly. When done, 🟩 + a one-line note in the task file's "Done" section pointing at the merging commit/PR.
Drift control. If implementation diverges from a task's spec, update the task file before the diverging code lands, with a note explaining why. Do not let plans rot — either fix the plan or fix the code.

8.5 KiB Raw Blame History Unescape Escape