Add business-plane schema draft and ingest Rally Albania 2025 regs

Substantial design artifact + canonical-source ingest for the TRM
business plane.

Schema draft (synthesis):
- wiki/synthesis/directus-schema-draft.md — working agreement for the
  multi-tenant schema. Pseudo multi-tenant under organizations; entries
  as the unit of timing; course definition (stages/segments/geofences/
  waypoints/SLZs); penalty system "numbers in DB, math in code" with an
  evaluator registry and progressive bracket math; per-entry timing
  tables; per-stage start-order strategies (manual /
  previous_stage_clean_result / inverse_top_n_then_natural /
  inverse_of_overall) covering both Tirana 24h and Rally Albania
  patterns. Two role surfaces (org role vs racing role) called out
  explicitly. Decisions captured; Open questions reduced to one
  (geometry retroactivity engine, deferred to Phase 2.5).

Source ingest:
- raw/Regulations_2025.pdf + wiki/sources/rally-albania-regulations-
  2025.md — formal ingest of the canonical Rally Albania 2025
  rulebook. Section numbers preserved as §X.Y so the schema draft and
  future SPA work can cite precisely. Flagged follow-ups: the SLZ
  formula lives in the Supplementary Regulations (don't hardcode);
  M-7 numbering bug; unmodeled neutralization zones.

Faulty-position flag (cross-plane operator workflow):
- entities/postgres-timescaledb.md, entities/processor.md,
  concepts/position-record.md — operator-controlled boolean on the
  positions hypertable; processor filters WHERE faulty = false on
  every read; flagging triggers windowed recompute via the
  recompute:requests stream.

Implementation strategy on entity pages:
- entities/directus.md — Schema management section documenting the
  snapshots/ + db-init/ convention, container-startup apply pipeline.
- entities/processor.md — Phase 2 long-lived branch model with
  PROCESSOR_PHASE_2_ENABLED flag-gating for incremental main merges;
  Phase 2.5 deferral note.

Index and log updated.
This commit is contained in:
2026-05-01 20:28:33 +02:00
parent 90d036dbf0
commit 411b08d02f
9 changed files with 774 additions and 4 deletions
+11
View File
@@ -56,6 +56,17 @@ This means **direct database writes from [[processor]] are not visible** to Dire
See [[live-channel-architecture]] for the full design, including why this split is preferable to routing telemetry writes through [[directus]]'s API or running a bridging extension inside [[directus]].
## Schema management — snapshot/apply pipeline
Schema changes flow through Directus's native snapshot mechanism, kept under git. Two artifact directories:
- **`snapshots/schema.yaml`** — Directus collections, fields, relations. Generated locally with `directus schema snapshot`. Applied at container startup with `directus schema apply --yes`. Idempotent — applies only the diff against the running DB.
- **`db-init/*.sql`** — schema Directus does not manage: the [[postgres-timescaledb]] positions hypertable, the `faulty` column, indexes that need PostGIS-specific syntax, or any DDL that predates Directus knowing about a collection. Numbered (`001_`, `002_`, …) and applied by a sidecar container or one-shot job ahead of `directus schema apply`. Tracked via a `migrations_applied` guard table to skip already-run files.
Local dev edits the schema in the admin UI, then snapshots before commit. CI builds the image with both directories baked in, spins a throwaway Postgres, and dry-runs `apply` to catch breakage before deploy. Production (Portainer) runs the same apply at container start; multi-env separation is a connection string, not different artifacts.
This treats `schema.yaml` as the source of truth and the admin UI as its editor. Don't hand-edit `schema.yaml`; round-trip through the UI to keep the format consistent.
## Phase 2 role
Directus owns the `commands` collection and is the **single auth surface** for outbound device commands. The SPA inserts command rows; a Directus Flow routes them via Redis to the Ingestion instance holding the device's socket. See [[phase-2-commands]].
+7 -1
View File
@@ -2,7 +2,7 @@
title: PostgreSQL + TimescaleDB
type: entity
created: 2026-04-30
updated: 2026-04-30
updated: 2026-05-01
sources: [gps-tracking-architecture]
tags: [infrastructure, business-plane, database]
---
@@ -22,6 +22,12 @@ The durable storage layer. PostgreSQL with the TimescaleDB extension. Holds the
Schema is **defined and migrated through [[directus]]** — see that page for why. The Processor inserts rows respecting that schema; it does not create tables.
## Positions hypertable
Stores normalized [[position-record]] rows from [[processor]]. Beyond the wire-shape fields (device_id, timestamp, lat/lon/alt, angle, speed, satellites, priority, attributes), the hypertable carries one storage-only field:
- **`faulty boolean DEFAULT false`** — set by track operators via [[directus]] when a position is unrealistic (jumpy GPS, impossible speed/coordinate). The [[processor]]'s evaluators (peak-speed, crossing detection, recompute) filter `WHERE faulty = false` on every read of position data. Untouched at write time; mutated only through the operator workflow described in the schema draft.
## Operational note
The database is the **only single point of failure** in the architecture. Everything else is restartable, replaceable, or naturally redundant. Operational attention concentrates here:
+17
View File
@@ -47,6 +47,12 @@ In multi-instance deployments, each Processor reads the [[redis-streams]] stream
Per-model IO mappings live here, not in the Ingestion layer. Example: `{ "FMB920": { "16": "odometer_km", "240": "movement" } }`. This is the boundary set by the [[teltonika]] adapter — Ingestion produces raw IO maps; the Processor names and interprets them.
## Faulty position handling
The positions hypertable in [[postgres-timescaledb]] carries a `faulty boolean DEFAULT false` column that operators can flip through [[directus]] when a position is unrealistic. **All Processor read paths against position data filter `WHERE faulty = false`** — peak-speed evaluation inside SLZs, geofence crossing detection, waypoint pass detection, replay-based recompute. The flag is never set at write time; it's a post-hoc operator action.
When an operator flips the flag (set or unset), Directus emits a webhook → Redis Stream `recompute:requests`. The Processor consumes the request and re-evaluates `entry_penalties` whose evaluation window overlaps the flagged position's timestamp. Cost sits between formula recompute (cheap) and full geometry replay (expensive) — the affected window is bounded, but the inputs (peak speed, missed-waypoint count) must be re-derived from the now-filtered position stream rather than from snapshotted values.
## Scaling
Multiple Processor instances join a Redis Streams consumer group and split the load across device IDs. Consumer-group offsets ensure a crashed instance's work is picked up by the next one.
@@ -54,3 +60,14 @@ Multiple Processor instances join a Redis Streams consumer group and split the l
## Failure mode
Crash → consumer-group offsets ensure the next instance picks up where the last left off. In-memory state is rehydrated from the database. See [[failure-domains]].
## Development workflow — Phase 2 branch model
Phase 2 (geofence engine, evaluator registry, crossings/penalties/results writers) is a substantial body of work and lives on a long-lived **`phase-2`** branch rather than landing piecemeal on `main`. Conventions:
- **Rebase weekly** against `main`, not merge. Keeps history readable and avoids merge-commit clutter when the branch eventually lands.
- **CI parity** — same workflow on `phase-2` PRs as on `main` PRs. Test coverage doesn't diverge across the branch boundary.
- **Flag-gated incremental merges** — chunks that are self-contained (a single evaluator, the geofence detector) can land on `main` behind `PROCESSOR_PHASE_2_ENABLED=false`. Off in prod, on in stage. Lets the work merge before it's user-visible without keeping the entire feature on a side branch indefinitely.
- **Single squash merge to retire the branch** — when Phase 2 is feature-complete enough to dogfood end-to-end, one squash merge retires the branch. Avoid death-by-a-thousand-merges.
Phase 2.5 (the geometry retroactivity engine in [[directus-schema-draft]]) follows the same pattern on its own branch when it starts; it is explicitly deferred until Phase 2 has shipped and the manual operator workflow for geometry edits has surfaced real pain points.