docs/log.md

# Log

Chronological activity log. Append-only. Entry headers use the format `## [YYYY-MM-DD] <op> | <title>` so they can be grepped:

```
grep "^## \[" log.md | tail -10
```

---

## [2026-04-30] note | Wiki bootstrapped

Created CLAUDE.md (schema + workflows), index.md (empty catalog), and this log. Wiki directory structure (wiki/sources, wiki/entities, wiki/concepts, wiki/synthesis) will be created on first ingest.

## [2026-04-30] ingest | gps-tracking-architecture.md + teltonika-ingestion-architecture.md

Ingested both initial architecture docs in one pass. Created:
- Source pages: [[gps-tracking-architecture]], [[teltonika-ingestion-architecture]].
- Entity pages: [[tcp-ingestion]], [[processor]], [[directus]], [[react-spa]], [[redis-streams]], [[postgres-timescaledb]], [[teltonika]].
- Concept pages: [[plane-separation]], [[protocol-adapter]], [[codec-dispatch]], [[position-record]], [[failure-domains]], [[phase-2-commands]].
- Updated index.md with all 15 new pages.

No contradictions to flag — the two docs are coherent (the Teltonika doc explicitly cites and respects the system architecture). Open follow-ups: TRM business domain not yet captured; per-model IO dictionary location TBD; Phase 2 timing unspecified.

## [2026-04-30] ingest | Teltonika Data Sending Protocols (official wiki)

Ingested the canonical Teltonika spec covering all codec families. New additions:
- Source page: [[teltonika-data-sending-protocols]].
- New concept: [[avl-data-format]] — byte-level reference for codecs 8/8E/16, including UDP envelope.

Updates to existing pages (no contradictions; refinements + additions):
- [[teltonika]] — added full codec table with hex IDs, Codec 15 (out of scope), Codec 14 ACK/nACK, packet size limits, UDP support note.
- [[codec-dispatch]] — corrected hex IDs, added directionality table covering codecs 8–15.
- [[position-record]] — concrete priority enum (0/1/2), two's-complement lat/lon note, Speed=0 means GPS invalid, Generation Type and NX section flagged.
- [[phase-2-commands]] — clarified Codec 12 vs 14 selection, added `nack` status for Codec 14 IMEI-mismatch (Type `0x11`); noted 13/15 are not part of the outbound design.

Cleanup: removed stale duplicate concept files from earlier passes (system-planes.md, protocol-adapter-pattern.md, codec-dispatch-registry.md) — superseded by plane-separation.md, protocol-adapter.md, codec-dispatch.md respectively. Fixed dangling [[protocol-adapter-pattern]] link in [[io-element-bag]].

Open questions surfaced by the canonical doc: Codec 16 Generation Type — promote to typed [[position-record]] field? Codec 8E NX values land as `Buffer` in `attributes`; needs explicit fixture coverage. SMS-based protocols (Codec 4 + binary SMS) probably out of scope but worth a deliberate decision.

## [2026-05-01] note | Stream-name canonicalization

Documented the canonical stream/key names in [[redis-streams]] — the wiki was previously silent on the actual `telemetry:teltonika` name, so anyone reading it had no way to find out what stream the services use. Added a "Stream and key naming" table covering the inbound telemetry stream, Phase 2 command streams, and registry/heartbeat keys. Also added the naming convention (`telemetry:{vendor}`) so future adapters fit predictably. Cross-referenced the actual stream name in [[processor]] and [[tcp-ingestion]] entities so each entity is self-contained but the convention has one canonical home.

Triggered by a stage-side bug where tcp-ingestion's compiled default (`telemetry:teltonika`) and processor's compiled default (`telemetry:t`) had drifted; pipeline ran with both services talking past each other for ~7 hours before symptoms surfaced. Fix landed in deploy stack (shared env var) and processor (default realigned). Wiki update closes the documentation loop.

## [2026-05-01] synthesis | Live channel architecture (corrects a wiki claim)

Researched Directus's WebSocket subscription mechanism via context7. Confirmed that subscriptions only fire for writes that go through Directus's `ItemsService` (REST/GraphQL/Admin UI mutations, not direct database INSERTs). The previous claim in [[directus]] — "When Processor writes a row, Directus broadcasts the change to subscribed clients" — was wrong.

Wrote [[live-channel-architecture]] documenting the corrected design: two WebSocket channels, each in its own plane. Processor exposes its own WebSocket endpoint for high-volume telemetry fan-out (auth via Directus-issued JWT, authorization delegated to Directus once at subscribe time). Directus's built-in WebSocket subscriptions cover business-plane events. Reasoning: preserves [[plane-separation]] and gives the gentlest failure mode (Directus down blocks only new authorizations, not the live firehose).

Updated [[processor]] (added Live broadcast section, multi-instance consumer-group plumbing note), [[directus]] (corrected the real-time-delivery section), and index.md.

## [2026-05-01] synthesis | Directus schema — working draft

Captured the business-plane schema agreement reached during today's discussion as [[directus-schema-draft]]. Marked as a working draft, open for revision.

Shape: pseudo multi-tenant under `organizations`; users / teams / vehicles / devices are all m2m with orgs (durable catalog); events scoped to a single org; `entries` is the per-event timing unit with nullable `vehicle_id` (foot races) and nullable `team_id` (lone racers); `entry_crew` and `entry_devices` are junctions off entries (no separate `crews` collection — teams already provide durable group identity). Vehicle ownership intentionally soft (`owner_user_id?`, `owner_team_id?`), not enforced. Per-event `classes`. `events.discipline` drives validation. Per-org-per-user role lives on `organization_users.role`.

Open: `entries.status` enum, permission policy definitions per role, stages/timing records (Phase 2 processor), geofences (Phase 2 processor).

## [2026-05-01] synthesis | Schema draft — course definition + penalty system

Major expansion of [[directus-schema-draft]]. Added course definition (stages → segments → geofences/waypoints/SLZs) and the full penalty system. Vehicle ownership idea dropped (org-level only, no owner FKs). `entries.status` enum pinned with semantics. Permission policies confirmed as Directus 11 dynamic-filter Policies, one per logical role.

**Penalty system landed as: numbers in DB, math in code.** A `penalty_formulas` collection holds all values (bracket multipliers, per-miss penalties); the [[processor]] holds one evaluator per `type` in a registry. Speed limit penalties are progressive slice-by-slice (income-tax math, confirmed against the Tirana 24h rulebook): each bracket contributes only the portion of the peak overspeed within its range — `slice × rate` summed across all brackets the peak crossed. Worked example with peak=58 included in the doc.

**Retroactive flag** lives on `penalty_formulas` (default `true`) and on `geofences` / `speed_limit_zones` (default `false`). Per-edit override at save time. Formula recomputes are cheap (snapshotted inputs on `entry_penalties` rows). Geometry recomputes are expensive (replay from positions hypertable) and deferred to Phase 2.5 of [[processor]].

**Other decisions:** checkpoints are typed geofences with `manual_verification=true`, not a separate collection. Stages are containers; segments (`liaison` / `special-stage` / `parc-ferme`) are the atomic rule unit. SLZs carry an `evaluation_window_meters` so the 2km rule from real federations is data, not code.

Per-entry timing layer (`entry_segment_starts`, `entry_crossings`, `entry_penalties`) and results layer (`stage_results`) are the [[processor]] Phase 2 write target. Schema is laid out so Phase 1 (positions only) can ship without it.

## [2026-05-01] note | Faulty position flagging

Added a `faulty boolean DEFAULT false` column to the positions hypertable, controlled by track operators through [[directus]] (the hypertable is exposed as a Directus collection for read+update). [[processor]] filters `WHERE faulty = false` on every read of position data — peak-speed, crossing detection, replay-based recompute. Flagging triggers a windowed recompute of affected `entry_penalties`. Updated [[postgres-timescaledb]], [[position-record]] (storage shape vs. wire shape), [[processor]] (faulty position handling), and [[directus-schema-draft]] (cross-plane operator workflow + third recompute kind).

## [2026-05-01] synthesis | Schema draft — start-order strategies + secondary observations

Read two real-world rulebooks to pin the start-order question: Tirana 24h 2017 (static every leg) and Rally Albania 2025 (dynamic, several variants). Rally Albania's §5.5–5.10 settle it — start order is per-stage, declarative, and rule-driven. Stage 1 bikes invert the top 20 of the prologue; stages 2 onward seed from previous-stage **clean** SS time (penalties explicitly excluded); epilogue inverts overall standings; intervals are decided per stage.

Updates to [[directus-schema-draft]]:
- `stages` gains `role` (prologue/regular/epilogue), `start_interval_seconds`, `start_order_strategy`, `start_order_strategy_params`, `start_order_input_stage_id`.
- New "Start order strategies" subsection enumerating `manual` / `previous_stage_result` / `previous_stage_clean_result` / `inverse_top_n_then_natural` / `inverse_of_overall` with real-world mappings. Tirana 24h covered by `manual`; Rally Albania covered by the other four.
- `entry_segment_starts` adds `start_position` and `manual_override` (latter for late-arrival reseeding by Race Marshals — both rulebooks leave that operator-driven).
- Materialization is per-category (categories share grids independently per Rally Albania §2.8 + §5.10).
- Decisions list grows: stage roles, CP-missing vs CP-late-past-closing as distinct event types sharing a formula row, reverse-stage tiebreaker.
- Open questions shrink: dropped the start-interval question (now pinned) and the permission-policy-filters question (admin/deployment task, not architectural).

## [2026-05-01] ingest | Rally Albania 2025 — Race Rules and Regulations

Formal ingest of `raw/Regulations_2025.pdf` (Motorsport Club Albania, October 2024). Created [[rally-albania-regulations-2025]] as the canonical real-world reference for federation rule shapes — classes, start-order rules, penalty taxonomy, tracking requirements, timekeeping, protests. Section numbers preserved as `§X.Y` so the schema draft and future SPA work can cite precisely.

Wired the source into [[directus-schema-draft]] (added to `sources:` frontmatter; framing note near the top; inline citation at start-order strategies section). Most of the schema-relevant content was already absorbed into the draft during the prior synthesis step — this ingest formalizes the citation chain.

Open follow-ups flagged on the source page: §12.11 SLZ formula lives in the Supplementary Regulations (not the general regs), so we shouldn't hardcode a default; M-7 numbering bug (Veteran and Female driver share the code — likely a typo); neutralization zones (§8.12) not yet modeled in the schema.

Index updated: new source row. No new entity/concept pages created — the doc supports existing pages rather than introducing new domain objects.

## [2026-05-02] note | Directus deployment wired; entity page updated

`trm/directus` Phase 1 shipped its image to the registry and the `trm/deploy` `compose.yaml` was extended with a `directus` service block (sharing the existing `postgres` service with [[processor]]). Updated [[directus]] entity page to reflect operational reality:

- New "Deployment" section: links to the deploy compose, explains the shared-Postgres model with [[processor]], spells out the 5-step boot pipeline (db-init pre-schema → bootstrap → schema apply → db-init post-schema → start), notes first-boot vs warm-boot timing.
- Schema management section: db-init split into pre-schema (`db-init/`) and post-schema (`db-init-post/`) phases. Post-schema landed because the composite UNIQUE constraints target Directus-managed tables that don't exist until schema apply runs.
- Destructive-apply hazard callout: corrected entrypoint step reference (now step 3/5, not 2/4) after the bootstrap-before-apply reorder.
- New "Network exposure" subsection inside Deployment: directus is internal-only on stage / prod (`expose: 8055` not `ports:`). A reverse proxy (Traefik / Caddy / nginx) on the host or attached to `trm_default` terminates TLS and forwards the public domain to `http://directus:8055`. The asymmetry with [[tcp-ingestion]] (which must host-publish for GPS devices) is named, and the dev compose's deliberate divergence is noted.

Three CI iterations on the directus repo's first push exposed three distinct production-breaking bugs (port collision; bootstrap-before-apply ordering + silent ERROR exit; ghost-collection apply conflict). The dry-run gate caught all of them before the image touched stage. The "ghost-collection" stripping is now automated in `scripts/schema-snapshot.sh` so future captures don't regress.

## [2026-05-02] note | Stage deploy verified + Rally Albania 2026 seed landed

Stage Directus is live at `api.stage.new.trmtracking.org` and matches the local snapshot. Verification done via the `directus-stage` MCP server:

- All 12 user collections present (`organizations`, `organization_users`, `organization_vehicles`, `organization_devices`, `vehicles`, `devices`, `events`, `classes`, `entries`, `entry_crew`, `entry_devices` + custom fields on `directus_users`).
- Field shapes, types, notes, and relations identical to local. `migrations_applied` + `positions` (db-init) and `schema_migrations` (processor migration runner) tables also present, as expected.
- Composite UNIQUE constraints landed — probed `(event_id, code)` on `classes` with a duplicate `M-1` insert, got `RECORD_NOT_UNIQUE`. Confirms `db-init-post/001` + `002` ran on stage (the post-schema phase introduced during task 1.8 CI iterations).

Rally Albania 2026 dogfood seed (task 1.9) replayed against stage: 1 org (`msc-albania`), 1 event (`rally-albania-2026`, 2026-06-06 → 2026-06-13), 18 classes (M-1..M-8, Q-1..Q-3, C-1/C-2/C-A/C-3, S-1..S-3), 1 vehicle (Toyota Land Cruiser 70), 3 devices (FMB920 chassis + FMB920 dash backup + FMB003 panic). Junction rows (`organization_vehicles` ×1, `organization_devices` ×3) wired. UUIDs differ from the local seed; record of stage UUIDs lives in `trm/directus/.planning/phase-1-slice-1-schema/09-rally-albania-2026-seed.md` Done section if needed.

End-to-end registration walkthrough (`organization_users` + `entries` + `entry_crew` + `entry_devices`) deferred to manual operator pass through the admin UI — the MCP `items` tool blocks writes to core collections like `directus_users`, so the user-attaches-to-entry flow can't be MCP-driven. That manual walkthrough is the actual dogfood acceptance gate for slice-1 schema.

Drift flagged: field notes on `events.slug`, `classes.code`, and `entries.race_number` still reference "db-init/005" — those constraints moved to `db-init-post/` during the CI fix. Cosmetic only, no behavior impact; worth a snapshot-side cleanup pass next time someone touches the schema.

## [2026-05-02] ingest | TRACCAR_MAPS_ARCHITECTURE.md

Ingested the deep architectural reference for traccar-web's maps subsystem after recognising during SPA-planning discussion that Traccar already fields the exact stack we're converging on (MapLibre GL JS + GeoJSON sources + WebSocket fan-out). Created [[traccar-maps-architecture]] (source page, with TRM divergences enumerated) and [[maps-architecture]] (concept page distilling the inherited patterns: singleton map, side-effect-only `Map*` components, two-effect setup/setData split, two-source clustered+selected design, style-swap `mapReady` gate, sprite preload, rAF coalescer at the WS boundary, geofence editing via `@mapbox/mapbox-gl-draw`, three-way camera control split).

Updated [[react-spa]] heavily: appended the new source; corrected the "talks exclusively to Directus" claim that conflicted with [[live-channel-architecture]] (the SPA connects to two endpoints — Directus for business plane, Processor for telemetry firehose); locked in the stack (raw MapLibre over `react-map-gl`, Zustand over Redux, `maplibre-google-maps` adapter as optional Google-tiles path); added an Auth section documenting the same-domain cookie + reverse-proxy pattern; rewrote Real-time rendering to point at [[maps-architecture]] and headline the rAF coalescer + per-device bounded ring buffers. One sentence + cross-reference added to [[live-channel-architecture]] flagging consumer-side throughput discipline.

Headline takeaway: Traccar's frontend architecture is mostly correct — the lag the user experienced isn't the rendering layer (which is WebGL `setData` and fast) but throughput discipline (per-message Redux dispatch cascading through selectors and rebuilding feature collections at every position arrival). TRM inherits the architecture and adds an rAF coalescer at the WS boundary plus Zustand to neutralise the failure mode. Tile-source decision unblocked: Google Maps via the official Map Tiles API is legitimate through the `maplibre-google-maps` protocol adapter (bring-your-own-key, runtime-config-gated). Dogfood-day starter set: Esri World Imagery (satellite, free) + OpenTopoMap (free) + OSM raster, with Google Satellite as an optional add when an operator provides a key.

## [2026-05-02] synthesis | Processor WebSocket contract + wiki/planning drift surfaced

Wrote [[processor-ws-contract]] as the wire-level spec for the live-position WebSocket: endpoint shape, cookie-based auth handshake, subscribe/snapshot/streaming/unsubscribe protocol, reconnect semantics, multi-instance fan-out behaviour, connection limits, versioning rules. Both the SPA and the producing service will build against this page; changes require coordinated updates on both sides.

Surfaced a real wiki/planning drift while researching: [[processor]] entity page lists "Broadcast live positions" as a top-level responsibility and [[live-channel-architecture]] specifies the design, but the processor's actual planning roadmap (`trm/processor/.planning/`) has no task for it. Phase 1 (done) is throughput-only; Phase 2 is geofence/IO/timing; Phase 3 is hardening; Phase 4 only mentions a "WebSocket gateway" as an uncommitted fallback service. The drift happened because [[live-channel-architecture]] was synthesised on 2026-05-01, after Phase 1's plan had locked — the wiki absorbed the corrected design, the processor's planning didn't reconcile.

Recommendation pending user decision: add a new processor phase ("Phase 1.5 — Live broadcast") that implements [[processor-ws-contract]] inside the processor service. Alternatives are Option B (separate `trm/live-gateway` service, aligning with the old Phase 4 framing — adds a deploy unit and contradicts the wiki) and Option C (defer the live map for the dogfood — thins the SPA's value-add over Directus admin). The synthesis page is implementation-agnostic so the contract is locked regardless of which option lands.

## [2026-05-02] note | Phase 1.5 planning landed (Option A chosen)

Promoted the Processor's WebSocket broadcast endpoint to a real planning artefact. Created `trm/processor/.planning/phase-1-5-live-broadcast/` with a phase README and six task files: 1.5.1 WS server scaffold + heartbeat, 1.5.2 cookie auth handshake, 1.5.3 subscription registry & per-event authorization, 1.5.4 broadcast consumer group & fan-out, 1.5.5 snapshot-on-subscribe, 1.5.6 integration test. Each follows the existing Phase 1 task-file shape (Goal / Deliverables / Specification / Acceptance / Risks / Done) so an implementer can pick one up self-contained.

Updated `trm/processor/.planning/ROADMAP.md` with a Phase 1.5 section between Phase 1 and Phase 2, including the per-task table. Pruned the stale "WebSocket gateway for live updates" candidate from Phase 4's README and reframed it as the documented [[live-channel-architecture]] escape hatch — to be promoted to a numbered phase only when measurements justify lifting the WS endpoint out of the Processor process. Updated [[processor-ws-contract]]'s Implementation status section to reflect "planned as Phase 1.5" instead of "designed but not scheduled."

Wiki / planning drift surfaced earlier today is now closed: the wiki's [[processor]] / [[live-channel-architecture]] / [[processor-ws-contract]] design and the processor's planning roadmap agree on what gets built, where, and how it's sequenced. Implementation can start on 1.5.1 whenever; SPA work can proceed against [[processor-ws-contract]] in parallel as long as it doesn't ship to stage before Phase 1.5 lands.

## [2026-05-02] note | Auth-mode wiki realignment (cookie → session)

SPA implementation surfaced that Directus SDK's `'cookie'` auth mode doesn't survive a hard reload cleanly — the in-memory access token is gone, and `/users/me` 401s before autoRefresh can establish a new one. Switched the SPA to `'session'` mode (`authentication('session', { credentials: 'include' })`), where the session itself lives in the httpOnly cookie and the browser sends it on every request including the WebSocket upgrade. Reload survives without any client-side state.

Updated [[react-spa]] §"Auth pattern" to describe session mode (single httpOnly session cookie, no separate access token, no `/auth/refresh` dance). Added a "Mode choice context" note explaining why session mode is the right default for an SPA that needs reload-survives behaviour.

Updated [[processor-ws-contract]] §"Auth handshake" to drop the explicit "(mode: cookie)" annotation and emphasise that the producer is **cookie-name-agnostic** — it forwards the entire `Cookie` header to `/users/me` and lets Directus identify the session. The producer's implementation was already cookie-name-agnostic in practice (the 1.5.2 implementation forwards the whole header), so no processor-side code change is needed; the wiki just now matches the implementation. Reframed "Cookie refresh while connected" open question as "Session expiry while connected" with the cleaner session-mode semantics.

Processor Phase 1.5 is fully shipped (`c07ea0e` 1.5.4, `f4b50ca` 1.5.5, `2f2cf5c` 1.5.6) — six tasks, 178/178 unit tests, 6 integration scenarios. The cookie-mode language in the processor's planning task files (1.5.2 in particular) is left as-is — it's the historical spec the implementation landed against; the implementation itself is mode-agnostic.

## [2026-05-02] note | TRM design handoff imported (deferred to SPA Phase 3.8)

User generated a design system via claude.ai/design and dropped the handoff bundle into `trm/spa/TRM_Design_System-handoff/`. Bloomberg/F1-pit-wall aesthetic — ink-on-paper base, race-flag red `#E8412B` accent, square-edged everything, sharp printed offset shadows (no blur), mono numerics for changing values, Goldplay (real licensed font, three weights) + JetBrains Mono + Inter. Four surfaces designed: dashboard / leaderboard / mobile / marketing — SPA scope covers the first two.

Adoption deferred to SPA Phase 3.8 ("Visual brand pass") because applying it now would either delay dogfood-blocking Phase 1/2 work or land partial styling that gets reworked. The bundle is committed in-tree (`trm/spa/9e6b361`) and Phase 3's README spells out the recommended approach: retheme shadcn via CSS-variable overrides + Tailwind 4 `@theme` block, don't replace primitives. Source-of-truth files for the future implementer: `colors_and_type.css` (tokens), `chats/chat1.md` (intent), the bundle's READMEs (specs), `ui_kits/` (HTML prototypes per surface).

No wiki updates yet — design system isn't part of the architectural model and the surface-level styling isn't worth a wiki entity. If/when 3.8 lands and the brand becomes a stable fixture, a brief mention in `[[react-spa]]` is the right home.

## [2026-05-02] note | trm/spa planning landed

User created `trm/spa` repo on Gitea and seeded a minimal Vite 8 + React 19 + TypeScript 6 scaffold (App.tsx returns "SPA"). Wrote the full planning structure mirroring the conventions established by `trm/processor` and `trm/directus`.

Created in `trm/spa/.planning/`:
- `ROADMAP.md` — navigation hub with status legend, architectural anchors, eight non-negotiable design rules (singleton MapLibre, side-effect-only `Map*` components, rAF coalescer, same-origin-everything, in-memory access token, role-aware UI, runtime config, native PostGIS GeoJSON), four phases.
- `phase-1-foundation/` — README + 9 task files: 1.2 stack rounding-out (Tailwind + shadcn/ui + TanStack Router/Query + Zustand + @directus/sdk + zod + react-hook-form + Prettier), 1.3 Vite dev proxy + path aliases + tsconfig hardening, 1.4 runtime config endpoint, 1.5 Directus auth client (cookie mode + refresh + Zustand auth store), 1.6 login page, 1.7 routing skeleton (TanStack Router file-based + role-aware guards), 1.8 logout flow (with cross-tab sync), 1.9 Gitea CI + Dockerfile + nginx static serve, 1.10 compose service block in `trm/deploy`.
- `phase-2-live-map/README.md` — sketched task table for the live-monitoring map; depends on processor Phase 1.5 landing. Nine tasks: MapLibre singleton, tile-source switcher, sprite preload, WS client + rAF coalescer + Zustand store, MapPositions, MapTrails, event picker, camera control trio, connection-status indicators.
- `phase-3-dogfood-readiness/README.md` — error boundaries, connection-state UI, mobile-responsive baseline, per-device detail panel, empty/loading-state polish, Vitest setup, production logging, visual brand pass.
- `phase-4-future/README.md` — geometry editor (depends on directus Phase 2), replay mode, heatmaps / deck.gl, i18n (Albanian), dark mode, Playwright E2E, leaderboard, spectator-facing public map, notifications, operator chat. None committed.

Each task file follows the existing Goal / Deliverables / Specification / Acceptance / Risks / Done shape so an implementer agent can pick one up self-contained. Phase 1 sequencing: 1.2 → 1.3 → 1.4 → 1.5 → (1.6 ‖ 1.7) → 1.8, with 1.9+1.10 (deploy plumbing) developable in parallel after 1.3 lands.

End state of Phase 1: a deployable empty shell — auth + protected routes + login/logout + CI + compose deploy block. End state of Phase 2: the dogfood-day deliverable. End state of Phase 3: actually fielded for race operators on race day, not just a tech demo.