diff --git a/log.md b/log.md index 5658ef6..254851f 100644 --- a/log.md +++ b/log.md @@ -38,6 +38,12 @@ Cleanup: removed stale duplicate concept files from earlier passes (system-plane Open questions surfaced by the canonical doc: Codec 16 Generation Type — promote to typed [[position-record]] field? Codec 8E NX values land as `Buffer` in `attributes`; needs explicit fixture coverage. SMS-based protocols (Codec 4 + binary SMS) probably out of scope but worth a deliberate decision. +## [2026-05-01] note | Stream-name canonicalization + +Documented the canonical stream/key names in [[redis-streams]] — the wiki was previously silent on the actual `telemetry:teltonika` name, so anyone reading it had no way to find out what stream the services use. Added a "Stream and key naming" table covering the inbound telemetry stream, Phase 2 command streams, and registry/heartbeat keys. Also added the naming convention (`telemetry:{vendor}`) so future adapters fit predictably. Cross-referenced the actual stream name in [[processor]] and [[tcp-ingestion]] entities so each entity is self-contained but the convention has one canonical home. + +Triggered by a stage-side bug where tcp-ingestion's compiled default (`telemetry:teltonika`) and processor's compiled default (`telemetry:t`) had drifted; pipeline ran with both services talking past each other for ~7 hours before symptoms surfaced. Fix landed in deploy stack (shared env var) and processor (default realigned). Wiki update closes the documentation loop. + ## [2026-05-01] synthesis | Live channel architecture (corrects a wiki claim) Researched Directus's WebSocket subscription mechanism via context7. Confirmed that subscriptions only fire for writes that go through Directus's `ItemsService` (REST/GraphQL/Admin UI mutations, not direct database INSERTs). The previous claim in [[directus]] — "When Processor writes a row, Directus broadcasts the change to subscribed clients" — was wrong. diff --git a/wiki/entities/processor.md b/wiki/entities/processor.md index 74f187b..f52c4d5 100644 --- a/wiki/entities/processor.md +++ b/wiki/entities/processor.md @@ -9,7 +9,7 @@ tags: [service, telemetry-plane, domain-logic] # Processor -The service where domain logic lives. Consumes normalized telemetry from [[redis-streams]] and is responsible for per-device runtime state, applying domain rules, writing durable state to [[postgres-timescaledb]], and broadcasting live position updates over WebSockets to the [[react-spa]]. +The service where domain logic lives. Consumes normalized telemetry from [[redis-streams]] (default stream `telemetry:teltonika`, consumer group `processor`) and is responsible for per-device runtime state, applying domain rules, writing durable state to [[postgres-timescaledb]], and broadcasting live position updates over WebSockets to the [[react-spa]]. ## Responsibilities diff --git a/wiki/entities/redis-streams.md b/wiki/entities/redis-streams.md index c55e387..694d8af 100644 --- a/wiki/entities/redis-streams.md +++ b/wiki/entities/redis-streams.md @@ -2,7 +2,7 @@ title: Redis Streams type: entity created: 2026-04-30 -updated: 2026-04-30 +updated: 2026-05-01 sources: [gps-tracking-architecture, teltonika-ingestion-architecture] tags: [infrastructure, telemetry-plane, queue] --- @@ -21,9 +21,23 @@ The durable in-flight queue between [[tcp-ingestion]] and [[processor]]. Also th Sufficient at current scale and adds minimal operational burden. NATS or Kafka are reasonable upgrades when **multi-region durability** or **very high throughput** become real concerns. Until then, Redis is the right choice. +## Stream and key naming + +Canonical names used across the platform. Both [[tcp-ingestion]] and [[processor]] reference these via the `REDIS_TELEMETRY_STREAM` environment variable, pinned in the deploy stack so the two services cannot drift from each other. + +| Name | Purpose | Producer | Consumer | +|---|---|---|---| +| `telemetry:teltonika` | Inbound Position records from Teltonika devices | [[tcp-ingestion]] (XADD) | [[processor]] (XREADGROUP, group=`processor`) | +| `commands:outbound:{instance_id}` | Outbound device commands routed to a specific [[tcp-ingestion]] instance | [[directus]] Flow | [[tcp-ingestion]] | +| `commands:responses` | Command ACK/nACK and replies | [[tcp-ingestion]] | [[directus]] Flow | +| `connections:registry` (hash) | IMEI → instance routing table | [[tcp-ingestion]] | [[directus]] Flow | +| `instance:heartbeat:{instance_id}` (key, `EX 90`) | Liveness signal per [[tcp-ingestion]] instance | [[tcp-ingestion]] | janitor / [[directus]] Flow | + +**Naming convention.** Telemetry streams are namespaced by vendor (`telemetry:{vendor}`) so adding a second adapter (Queclink, Concox, etc.) creates `telemetry:queclink` rather than competing for shape on the same stream. [[processor]] consumes the union by joining a consumer group on each. + ## Phase 2 usage -Outbound commands ride on per-instance streams: `commands:outbound:{instance_id}`. Responses ride on `commands:responses`. Redis is the transport; the source of truth for commands is the Directus `commands` collection. See [[phase-2-commands]]. +Outbound commands ride on per-instance streams: `commands:outbound:{instance_id}`. Responses ride on `commands:responses`. Redis is the transport; the source of truth for commands is the [[directus]] `commands` collection. See [[phase-2-commands]]. The connection registry (`connections:registry` hash) and per-instance heartbeats (`instance:heartbeat:{instance_id}` keys with `EX 90`) also live in Redis. diff --git a/wiki/entities/tcp-ingestion.md b/wiki/entities/tcp-ingestion.md index 40f5ef7..6d4325f 100644 --- a/wiki/entities/tcp-ingestion.md +++ b/wiki/entities/tcp-ingestion.md @@ -2,14 +2,14 @@ title: TCP Ingestion type: entity created: 2026-04-30 -updated: 2026-04-30 +updated: 2026-05-01 sources: [gps-tracking-architecture, teltonika-ingestion-architecture] tags: [service, telemetry-plane] --- # TCP Ingestion -The service that maintains persistent TCP connections with GPS devices, parses vendor binary protocols, ACKs frames per protocol, and hands off normalized records to the [[redis-streams]] queue. +The service that maintains persistent TCP connections with GPS devices, parses vendor binary protocols, ACKs frames per protocol, and hands off normalized records to the [[redis-streams]] queue (default stream `telemetry:teltonika` for the Teltonika adapter; see [[redis-streams]] for the full naming convention). ## Responsibility