docs/log.md

Log

Chronological activity log. Append-only. Entry headers use the format ## [YYYY-MM-DD] <op> | <title> so they can be grepped:

grep "^## \[" log.md | tail -10

---

[2026-04-30] note | Wiki bootstrapped

Created CLAUDE.md (schema + workflows), index.md (empty catalog), and this log. Wiki directory structure (wiki/sources, wiki/entities, wiki/concepts, wiki/synthesis) will be created on first ingest.

[2026-04-30] ingest | gps-tracking-architecture.md + teltonika-ingestion-architecture.md

Ingested both initial architecture docs in one pass. Created:

• Source pages: gps-tracking-architecture, teltonika-ingestion-architecture.
• Entity pages: tcp-ingestion, processor, directus, react-spa, redis-streams, postgres-timescaledb, teltonika.
• Concept pages: plane-separation, protocol-adapter, codec-dispatch, position-record, failure-domains, phase-2-commands.
• Updated index.md with all 15 new pages.

No contradictions to flag — the two docs are coherent (the Teltonika doc explicitly cites and respects the system architecture). Open follow-ups: TRM business domain not yet captured; per-model IO dictionary location TBD; Phase 2 timing unspecified.

[2026-04-30] ingest | Teltonika Data Sending Protocols (official wiki)

Ingested the canonical Teltonika spec covering all codec families. New additions:

• Source page: teltonika-data-sending-protocols.
• New concept: avl-data-format — byte-level reference for codecs 8/8E/16, including UDP envelope.

Updates to existing pages (no contradictions; refinements + additions):

• teltonika — added full codec table with hex IDs, Codec 15 (out of scope), Codec 14 ACK/nACK, packet size limits, UDP support note.
• codec-dispatch — corrected hex IDs, added directionality table covering codecs 8–15.
• position-record — concrete priority enum (0/1/2), two's-complement lat/lon note, Speed=0 means GPS invalid, Generation Type and NX section flagged.
• phase-2-commands — clarified Codec 12 vs 14 selection, added nack status for Codec 14 IMEI-mismatch (Type 0x11); noted 13/15 are not part of the outbound design.

Cleanup: removed stale duplicate concept files from earlier passes (system-planes.md, protocol-adapter-pattern.md, codec-dispatch-registry.md) — superseded by plane-separation.md, protocol-adapter.md, codec-dispatch.md respectively. Fixed dangling protocol-adapter-pattern link in io-element-bag.

Open questions surfaced by the canonical doc: Codec 16 Generation Type — promote to typed position-record field? Codec 8E NX values land as Buffer in attributes; needs explicit fixture coverage. SMS-based protocols (Codec 4 + binary SMS) probably out of scope but worth a deliberate decision.

[2026-05-01] note | Stream-name canonicalization

Documented the canonical stream/key names in redis-streams — the wiki was previously silent on the actual telemetry:teltonika name, so anyone reading it had no way to find out what stream the services use. Added a "Stream and key naming" table covering the inbound telemetry stream, Phase 2 command streams, and registry/heartbeat keys. Also added the naming convention (telemetry:{vendor}) so future adapters fit predictably. Cross-referenced the actual stream name in processor and tcp-ingestion entities so each entity is self-contained but the convention has one canonical home.

Triggered by a stage-side bug where tcp-ingestion's compiled default (telemetry:teltonika) and processor's compiled default (telemetry:t) had drifted; pipeline ran with both services talking past each other for ~7 hours before symptoms surfaced. Fix landed in deploy stack (shared env var) and processor (default realigned). Wiki update closes the documentation loop.

[2026-05-01] synthesis | Live channel architecture (corrects a wiki claim)

Researched Directus's WebSocket subscription mechanism via context7. Confirmed that subscriptions only fire for writes that go through Directus's ItemsService (REST/GraphQL/Admin UI mutations, not direct database INSERTs). The previous claim in directus — "When Processor writes a row, Directus broadcasts the change to subscribed clients" — was wrong.

Wrote live-channel-architecture documenting the corrected design: two WebSocket channels, each in its own plane. Processor exposes its own WebSocket endpoint for high-volume telemetry fan-out (auth via Directus-issued JWT, authorization delegated to Directus once at subscribe time). Directus's built-in WebSocket subscriptions cover business-plane events. Reasoning: preserves plane-separation and gives the gentlest failure mode (Directus down blocks only new authorizations, not the live firehose).

Updated processor (added Live broadcast section, multi-instance consumer-group plumbing note), directus (corrected the real-time-delivery section), and index.md.

[2026-05-01] synthesis | Directus schema — working draft

Captured the business-plane schema agreement reached during today's discussion as directus-schema-draft. Marked as a working draft, open for revision.

Shape: pseudo multi-tenant under organizations; users / teams / vehicles / devices are all m2m with orgs (durable catalog); events scoped to a single org; entries is the per-event timing unit with nullable vehicle_id (foot races) and nullable team_id (lone racers); entry_crew and entry_devices are junctions off entries (no separate crews collection — teams already provide durable group identity). Vehicle ownership intentionally soft (owner_user_id?, owner_team_id?), not enforced. Per-event classes. events.discipline drives validation. Per-org-per-user role lives on organization_users.role.

Open: entries.status enum, permission policy definitions per role, stages/timing records (Phase 2 processor), geofences (Phase 2 processor).

[2026-05-01] synthesis | Schema draft — course definition + penalty system

Major expansion of directus-schema-draft. Added course definition (stages → segments → geofences/waypoints/SLZs) and the full penalty system. Vehicle ownership idea dropped (org-level only, no owner FKs). entries.status enum pinned with semantics. Permission policies confirmed as Directus 11 dynamic-filter Policies, one per logical role.

Penalty system landed as: numbers in DB, math in code. A penalty_formulas collection holds all values (bracket multipliers, per-miss penalties); the processor holds one evaluator per type in a registry. Speed limit penalties are progressive slice-by-slice (income-tax math, confirmed against the Tirana 24h rulebook): each bracket contributes only the portion of the peak overspeed within its range — slice × rate summed across all brackets the peak crossed. Worked example with peak=58 included in the doc.

Retroactive flag lives on penalty_formulas (default true) and on geofences / speed_limit_zones (default false). Per-edit override at save time. Formula recomputes are cheap (snapshotted inputs on entry_penalties rows). Geometry recomputes are expensive (replay from positions hypertable) and deferred to Phase 2.5 of processor.

Other decisions: checkpoints are typed geofences with manual_verification=true, not a separate collection. Stages are containers; segments (liaison / special-stage / parc-ferme) are the atomic rule unit. SLZs carry an evaluation_window_meters so the 2km rule from real federations is data, not code.

Per-entry timing layer (entry_segment_starts, entry_crossings, entry_penalties) and results layer (stage_results) are the processor Phase 2 write target. Schema is laid out so Phase 1 (positions only) can ship without it.

[2026-05-01] note | Faulty position flagging

Added a faulty boolean DEFAULT false column to the positions hypertable, controlled by track operators through directus (the hypertable is exposed as a Directus collection for read+update). processor filters WHERE faulty = false on every read of position data — peak-speed, crossing detection, replay-based recompute. Flagging triggers a windowed recompute of affected entry_penalties. Updated postgres-timescaledb, position-record (storage shape vs. wire shape), processor (faulty position handling), and directus-schema-draft (cross-plane operator workflow + third recompute kind).

[2026-05-01] synthesis | Schema draft — start-order strategies + secondary observations

Read two real-world rulebooks to pin the start-order question: Tirana 24h 2017 (static every leg) and Rally Albania 2025 (dynamic, several variants). Rally Albania's §5.5–5.10 settle it — start order is per-stage, declarative, and rule-driven. Stage 1 bikes invert the top 20 of the prologue; stages 2 onward seed from previous-stage clean SS time (penalties explicitly excluded); epilogue inverts overall standings; intervals are decided per stage.

Updates to directus-schema-draft:

• stages gains role (prologue/regular/epilogue), start_interval_seconds, start_order_strategy, start_order_strategy_params, start_order_input_stage_id.
• New "Start order strategies" subsection enumerating manual / previous_stage_result / previous_stage_clean_result / inverse_top_n_then_natural / inverse_of_overall with real-world mappings. Tirana 24h covered by manual; Rally Albania covered by the other four.
• entry_segment_starts adds start_position and manual_override (latter for late-arrival reseeding by Race Marshals — both rulebooks leave that operator-driven).
• Materialization is per-category (categories share grids independently per Rally Albania §2.8 + §5.10).
• Decisions list grows: stage roles, CP-missing vs CP-late-past-closing as distinct event types sharing a formula row, reverse-stage tiebreaker.
• Open questions shrink: dropped the start-interval question (now pinned) and the permission-policy-filters question (admin/deployment task, not architectural).

[2026-05-01] ingest | Rally Albania 2025 — Race Rules and Regulations

Formal ingest of raw/Regulations_2025.pdf (Motorsport Club Albania, October 2024). Created rally-albania-regulations-2025 as the canonical real-world reference for federation rule shapes — classes, start-order rules, penalty taxonomy, tracking requirements, timekeeping, protests. Section numbers preserved as §X.Y so the schema draft and future SPA work can cite precisely.

Wired the source into directus-schema-draft (added to sources: frontmatter; framing note near the top; inline citation at start-order strategies section). Most of the schema-relevant content was already absorbed into the draft during the prior synthesis step — this ingest formalizes the citation chain.

Open follow-ups flagged on the source page: §12.11 SLZ formula lives in the Supplementary Regulations (not the general regs), so we shouldn't hardcode a default; M-7 numbering bug (Veteran and Female driver share the code — likely a typo); neutralization zones (§8.12) not yet modeled in the schema.

Index updated: new source row. No new entity/concept pages created — the doc supports existing pages rather than introducing new domain objects.

[2026-05-02] note | Directus deployment wired; entity page updated

trm/directus Phase 1 shipped its image to the registry and the trm/deploy compose.yaml was extended with a directus service block (sharing the existing postgres service with processor). Updated directus entity page to reflect operational reality:

• New "Deployment" section: links to the deploy compose, explains the shared-Postgres model with processor, spells out the 5-step boot pipeline (db-init pre-schema → bootstrap → schema apply → db-init post-schema → start), notes first-boot vs warm-boot timing.
• Schema management section: db-init split into pre-schema (db-init/) and post-schema (db-init-post/) phases. Post-schema landed because the composite UNIQUE constraints target Directus-managed tables that don't exist until schema apply runs.
• Destructive-apply hazard callout: corrected entrypoint step reference (now step 3/5, not 2/4) after the bootstrap-before-apply reorder.
• New "Network exposure" subsection inside Deployment: directus is internal-only on stage / prod (expose: 8055 not ports:). A reverse proxy (Traefik / Caddy / nginx) on the host or attached to trm_default terminates TLS and forwards the public domain to http://directus:8055. The asymmetry with tcp-ingestion (which must host-publish for GPS devices) is named, and the dev compose's deliberate divergence is noted.

Three CI iterations on the directus repo's first push exposed three distinct production-breaking bugs (port collision; bootstrap-before-apply ordering + silent ERROR exit; ghost-collection apply conflict). The dry-run gate caught all of them before the image touched stage. The "ghost-collection" stripping is now automated in scripts/schema-snapshot.sh so future captures don't regress.

[2026-05-02] note | Stage deploy verified + Rally Albania 2026 seed landed

Stage Directus is live at api.stage.new.trmtracking.org and matches the local snapshot. Verification done via the directus-stage MCP server:

• All 12 user collections present (organizations, organization_users, organization_vehicles, organization_devices, vehicles, devices, events, classes, entries, entry_crew, entry_devices + custom fields on directus_users).
• Field shapes, types, notes, and relations identical to local. migrations_applied + positions (db-init) and schema_migrations (processor migration runner) tables also present, as expected.
• Composite UNIQUE constraints landed — probed (event_id, code) on classes with a duplicate M-1 insert, got RECORD_NOT_UNIQUE. Confirms db-init-post/001 + 002 ran on stage (the post-schema phase introduced during task 1.8 CI iterations).

Rally Albania 2026 dogfood seed (task 1.9) replayed against stage: 1 org (msc-albania), 1 event (rally-albania-2026, 2026-06-06 → 2026-06-13), 18 classes (M-1..M-8, Q-1..Q-3, C-1/C-2/C-A/C-3, S-1..S-3), 1 vehicle (Toyota Land Cruiser 70), 3 devices (FMB920 chassis + FMB920 dash backup + FMB003 panic). Junction rows (organization_vehicles ×1, organization_devices ×3) wired. UUIDs differ from the local seed; record of stage UUIDs lives in trm/directus/.planning/phase-1-slice-1-schema/09-rally-albania-2026-seed.md Done section if needed.

End-to-end registration walkthrough (organization_users + entries + entry_crew + entry_devices) deferred to manual operator pass through the admin UI — the MCP items tool blocks writes to core collections like directus_users, so the user-attaches-to-entry flow can't be MCP-driven. That manual walkthrough is the actual dogfood acceptance gate for slice-1 schema.

Drift flagged: field notes on events.slug, classes.code, and entries.race_number still reference "db-init/005" — those constraints moved to db-init-post/ during the CI fix. Cosmetic only, no behavior impact; worth a snapshot-side cleanup pass next time someone touches the schema.

[2026-05-02] ingest | TRACCAR_MAPS_ARCHITECTURE.md

Ingested the deep architectural reference for traccar-web's maps subsystem after recognising during SPA-planning discussion that Traccar already fields the exact stack we're converging on (MapLibre GL JS + GeoJSON sources + WebSocket fan-out). Created traccar-maps-architecture (source page, with TRM divergences enumerated) and maps-architecture (concept page distilling the inherited patterns: singleton map, side-effect-only Map* components, two-effect setup/setData split, two-source clustered+selected design, style-swap mapReady gate, sprite preload, rAF coalescer at the WS boundary, geofence editing via @mapbox/mapbox-gl-draw, three-way camera control split).

Updated react-spa heavily: appended the new source; corrected the "talks exclusively to Directus" claim that conflicted with live-channel-architecture (the SPA connects to two endpoints — Directus for business plane, Processor for telemetry firehose); locked in the stack (raw MapLibre over react-map-gl, Zustand over Redux, maplibre-google-maps adapter as optional Google-tiles path); added an Auth section documenting the same-domain cookie + reverse-proxy pattern; rewrote Real-time rendering to point at maps-architecture and headline the rAF coalescer + per-device bounded ring buffers. One sentence + cross-reference added to live-channel-architecture flagging consumer-side throughput discipline.

Headline takeaway: Traccar's frontend architecture is mostly correct — the lag the user experienced isn't the rendering layer (which is WebGL setData and fast) but throughput discipline (per-message Redux dispatch cascading through selectors and rebuilding feature collections at every position arrival). TRM inherits the architecture and adds an rAF coalescer at the WS boundary plus Zustand to neutralise the failure mode. Tile-source decision unblocked: Google Maps via the official Map Tiles API is legitimate through the maplibre-google-maps protocol adapter (bring-your-own-key, runtime-config-gated). Dogfood-day starter set: Esri World Imagery (satellite, free) + OpenTopoMap (free) + OSM raster, with Google Satellite as an optional add when an operator provides a key.

[2026-05-02] synthesis | Processor WebSocket contract + wiki/planning drift surfaced

Wrote processor-ws-contract as the wire-level spec for the live-position WebSocket: endpoint shape, cookie-based auth handshake, subscribe/snapshot/streaming/unsubscribe protocol, reconnect semantics, multi-instance fan-out behaviour, connection limits, versioning rules. Both the SPA and the producing service will build against this page; changes require coordinated updates on both sides.

Surfaced a real wiki/planning drift while researching: processor entity page lists "Broadcast live positions" as a top-level responsibility and live-channel-architecture specifies the design, but the processor's actual planning roadmap (trm/processor/.planning/) has no task for it. Phase 1 (done) is throughput-only; Phase 2 is geofence/IO/timing; Phase 3 is hardening; Phase 4 only mentions a "WebSocket gateway" as an uncommitted fallback service. The drift happened because live-channel-architecture was synthesised on 2026-05-01, after Phase 1's plan had locked — the wiki absorbed the corrected design, the processor's planning didn't reconcile.

Recommendation pending user decision: add a new processor phase ("Phase 1.5 — Live broadcast") that implements processor-ws-contract inside the processor service. Alternatives are Option B (separate trm/live-gateway service, aligning with the old Phase 4 framing — adds a deploy unit and contradicts the wiki) and Option C (defer the live map for the dogfood — thins the SPA's value-add over Directus admin). The synthesis page is implementation-agnostic so the contract is locked regardless of which option lands.

[2026-05-02] note | Phase 1.5 planning landed (Option A chosen)

Promoted the Processor's WebSocket broadcast endpoint to a real planning artefact. Created trm/processor/.planning/phase-1-5-live-broadcast/ with a phase README and six task files: 1.5.1 WS server scaffold + heartbeat, 1.5.2 cookie auth handshake, 1.5.3 subscription registry & per-event authorization, 1.5.4 broadcast consumer group & fan-out, 1.5.5 snapshot-on-subscribe, 1.5.6 integration test. Each follows the existing Phase 1 task-file shape (Goal / Deliverables / Specification / Acceptance / Risks / Done) so an implementer can pick one up self-contained.

Updated trm/processor/.planning/ROADMAP.md with a Phase 1.5 section between Phase 1 and Phase 2, including the per-task table. Pruned the stale "WebSocket gateway for live updates" candidate from Phase 4's README and reframed it as the documented live-channel-architecture escape hatch — to be promoted to a numbered phase only when measurements justify lifting the WS endpoint out of the Processor process. Updated processor-ws-contract's Implementation status section to reflect "planned as Phase 1.5" instead of "designed but not scheduled."

Wiki / planning drift surfaced earlier today is now closed: the wiki's processor / live-channel-architecture / processor-ws-contract design and the processor's planning roadmap agree on what gets built, where, and how it's sequenced. Implementation can start on 1.5.1 whenever; SPA work can proceed against processor-ws-contract in parallel as long as it doesn't ship to stage before Phase 1.5 lands.

[2026-05-02] note | Auth-mode wiki realignment (cookie → session)

SPA implementation surfaced that Directus SDK's 'cookie' auth mode doesn't survive a hard reload cleanly — the in-memory access token is gone, and /users/me 401s before autoRefresh can establish a new one. Switched the SPA to 'session' mode (authentication('session', { credentials: 'include' })), where the session itself lives in the httpOnly cookie and the browser sends it on every request including the WebSocket upgrade. Reload survives without any client-side state.

Updated react-spa §"Auth pattern" to describe session mode (single httpOnly session cookie, no separate access token, no /auth/refresh dance). Added a "Mode choice context" note explaining why session mode is the right default for an SPA that needs reload-survives behaviour.

Updated processor-ws-contract §"Auth handshake" to drop the explicit "(mode: cookie)" annotation and emphasise that the producer is cookie-name-agnostic — it forwards the entire Cookie header to /users/me and lets Directus identify the session. The producer's implementation was already cookie-name-agnostic in practice (the 1.5.2 implementation forwards the whole header), so no processor-side code change is needed; the wiki just now matches the implementation. Reframed "Cookie refresh while connected" open question as "Session expiry while connected" with the cleaner session-mode semantics.

Processor Phase 1.5 is fully shipped (c07ea0e 1.5.4, f4b50ca 1.5.5, 2f2cf5c 1.5.6) — six tasks, 178/178 unit tests, 6 integration scenarios. The cookie-mode language in the processor's planning task files (1.5.2 in particular) is left as-is — it's the historical spec the implementation landed against; the implementation itself is mode-agnostic.

[2026-05-02] note | TRM design handoff imported (deferred to SPA Phase 3.8)

User generated a design system via claude.ai/design and dropped the handoff bundle into trm/spa/TRM_Design_System-handoff/. Bloomberg/F1-pit-wall aesthetic — ink-on-paper base, race-flag red #E8412B accent, square-edged everything, sharp printed offset shadows (no blur), mono numerics for changing values, Goldplay (real licensed font, three weights) + JetBrains Mono + Inter. Four surfaces designed: dashboard / leaderboard / mobile / marketing — SPA scope covers the first two.

Adoption deferred to SPA Phase 3.8 ("Visual brand pass") because applying it now would either delay dogfood-blocking Phase 1/2 work or land partial styling that gets reworked. The bundle is committed in-tree (trm/spa/9e6b361) and Phase 3's README spells out the recommended approach: retheme shadcn via CSS-variable overrides + Tailwind 4 @theme block, don't replace primitives. Source-of-truth files for the future implementer: colors_and_type.css (tokens), chats/chat1.md (intent), the bundle's READMEs (specs), ui_kits/ (HTML prototypes per surface).

No wiki updates yet — design system isn't part of the architectural model and the surface-level styling isn't worth a wiki entity. If/when 3.8 lands and the brand becomes a stable fixture, a brief mention in [[react-spa]] is the right home.

[2026-05-02] note | trm/spa planning landed

User created trm/spa repo on Gitea and seeded a minimal Vite 8 + React 19 + TypeScript 6 scaffold (App.tsx returns "SPA"). Wrote the full planning structure mirroring the conventions established by trm/processor and trm/directus.

Created in trm/spa/.planning/:

• ROADMAP.md — navigation hub with status legend, architectural anchors, eight non-negotiable design rules (singleton MapLibre, side-effect-only Map* components, rAF coalescer, same-origin-everything, in-memory access token, role-aware UI, runtime config, native PostGIS GeoJSON), four phases.
• phase-1-foundation/ — README + 9 task files: 1.2 stack rounding-out (Tailwind + shadcn/ui + TanStack Router/Query + Zustand + @directus/sdk + zod + react-hook-form + Prettier), 1.3 Vite dev proxy + path aliases + tsconfig hardening, 1.4 runtime config endpoint, 1.5 Directus auth client (cookie mode + refresh + Zustand auth store), 1.6 login page, 1.7 routing skeleton (TanStack Router file-based + role-aware guards), 1.8 logout flow (with cross-tab sync), 1.9 Gitea CI + Dockerfile + nginx static serve, 1.10 compose service block in trm/deploy.
• phase-2-live-map/README.md — sketched task table for the live-monitoring map; depends on processor Phase 1.5 landing. Nine tasks: MapLibre singleton, tile-source switcher, sprite preload, WS client + rAF coalescer + Zustand store, MapPositions, MapTrails, event picker, camera control trio, connection-status indicators.
• phase-3-dogfood-readiness/README.md — error boundaries, connection-state UI, mobile-responsive baseline, per-device detail panel, empty/loading-state polish, Vitest setup, production logging, visual brand pass.
• phase-4-future/README.md — geometry editor (depends on directus Phase 2), replay mode, heatmaps / deck.gl, i18n (Albanian), dark mode, Playwright E2E, leaderboard, spectator-facing public map, notifications, operator chat. None committed.

Each task file follows the existing Goal / Deliverables / Specification / Acceptance / Risks / Done shape so an implementer agent can pick one up self-contained. Phase 1 sequencing: 1.2 → 1.3 → 1.4 → 1.5 → (1.6 ‖ 1.7) → 1.8, with 1.9+1.10 (deploy plumbing) developable in parallel after 1.3 lands.

End state of Phase 1: a deployable empty shell — auth + protected routes + login/logout + CI + compose deploy block. End state of Phase 2: the dogfood-day deliverable. End state of Phase 3: actually fielded for race operators on race day, not just a tech demo.