Document canonical Redis stream names in wiki
The wiki was silent on the actual stream name used by tcp-ingestion and
processor — anyone reading it to understand the architecture had no way
to find out what stream the services use. This gap contributed to a
stage-side bug where the two services' compiled defaults drifted
(tcp-ingestion: telemetry:teltonika, processor: telemetry:t), causing
~7 hours of silent zero-throughput before symptoms surfaced.
Changes:
- entities/redis-streams.md — added "Stream and key naming" table
covering the inbound telemetry stream, Phase 2 command streams, and
registry/heartbeat keys. Documented the telemetry:{vendor} convention
so a future Queclink/Concox adapter fits predictably.
- entities/processor.md — opening paragraph names the stream and
consumer group consumed.
- entities/tcp-ingestion.md — opening paragraph names the stream
produced; defers full naming convention to redis-streams.
- log.md — note entry recording the canonicalization and the stage
incident that triggered it.
This commit is contained in:
@@ -9,7 +9,7 @@ tags: [service, telemetry-plane, domain-logic]
|
||||
|
||||
# Processor
|
||||
|
||||
The service where domain logic lives. Consumes normalized telemetry from [[redis-streams]] and is responsible for per-device runtime state, applying domain rules, writing durable state to [[postgres-timescaledb]], and broadcasting live position updates over WebSockets to the [[react-spa]].
|
||||
The service where domain logic lives. Consumes normalized telemetry from [[redis-streams]] (default stream `telemetry:teltonika`, consumer group `processor`) and is responsible for per-device runtime state, applying domain rules, writing durable state to [[postgres-timescaledb]], and broadcasting live position updates over WebSockets to the [[react-spa]].
|
||||
|
||||
## Responsibilities
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: Redis Streams
|
||||
type: entity
|
||||
created: 2026-04-30
|
||||
updated: 2026-04-30
|
||||
updated: 2026-05-01
|
||||
sources: [gps-tracking-architecture, teltonika-ingestion-architecture]
|
||||
tags: [infrastructure, telemetry-plane, queue]
|
||||
---
|
||||
@@ -21,9 +21,23 @@ The durable in-flight queue between [[tcp-ingestion]] and [[processor]]. Also th
|
||||
|
||||
Sufficient at current scale and adds minimal operational burden. NATS or Kafka are reasonable upgrades when **multi-region durability** or **very high throughput** become real concerns. Until then, Redis is the right choice.
|
||||
|
||||
## Stream and key naming
|
||||
|
||||
Canonical names used across the platform. Both [[tcp-ingestion]] and [[processor]] reference these via the `REDIS_TELEMETRY_STREAM` environment variable, pinned in the deploy stack so the two services cannot drift from each other.
|
||||
|
||||
| Name | Purpose | Producer | Consumer |
|
||||
|---|---|---|---|
|
||||
| `telemetry:teltonika` | Inbound Position records from Teltonika devices | [[tcp-ingestion]] (XADD) | [[processor]] (XREADGROUP, group=`processor`) |
|
||||
| `commands:outbound:{instance_id}` | Outbound device commands routed to a specific [[tcp-ingestion]] instance | [[directus]] Flow | [[tcp-ingestion]] |
|
||||
| `commands:responses` | Command ACK/nACK and replies | [[tcp-ingestion]] | [[directus]] Flow |
|
||||
| `connections:registry` (hash) | IMEI → instance routing table | [[tcp-ingestion]] | [[directus]] Flow |
|
||||
| `instance:heartbeat:{instance_id}` (key, `EX 90`) | Liveness signal per [[tcp-ingestion]] instance | [[tcp-ingestion]] | janitor / [[directus]] Flow |
|
||||
|
||||
**Naming convention.** Telemetry streams are namespaced by vendor (`telemetry:{vendor}`) so adding a second adapter (Queclink, Concox, etc.) creates `telemetry:queclink` rather than competing for shape on the same stream. [[processor]] consumes the union by joining a consumer group on each.
|
||||
|
||||
## Phase 2 usage
|
||||
|
||||
Outbound commands ride on per-instance streams: `commands:outbound:{instance_id}`. Responses ride on `commands:responses`. Redis is the transport; the source of truth for commands is the Directus `commands` collection. See [[phase-2-commands]].
|
||||
Outbound commands ride on per-instance streams: `commands:outbound:{instance_id}`. Responses ride on `commands:responses`. Redis is the transport; the source of truth for commands is the [[directus]] `commands` collection. See [[phase-2-commands]].
|
||||
|
||||
The connection registry (`connections:registry` hash) and per-instance heartbeats (`instance:heartbeat:{instance_id}` keys with `EX 90`) also live in Redis.
|
||||
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
title: TCP Ingestion
|
||||
type: entity
|
||||
created: 2026-04-30
|
||||
updated: 2026-04-30
|
||||
updated: 2026-05-01
|
||||
sources: [gps-tracking-architecture, teltonika-ingestion-architecture]
|
||||
tags: [service, telemetry-plane]
|
||||
---
|
||||
|
||||
# TCP Ingestion
|
||||
|
||||
The service that maintains persistent TCP connections with GPS devices, parses vendor binary protocols, ACKs frames per protocol, and hands off normalized records to the [[redis-streams]] queue.
|
||||
The service that maintains persistent TCP connections with GPS devices, parses vendor binary protocols, ACKs frames per protocol, and hands off normalized records to the [[redis-streams]] queue (default stream `telemetry:teltonika` for the Teltonika adapter; see [[redis-streams]] for the full naming convention).
|
||||
|
||||
## Responsibility
|
||||
|
||||
|
||||
Reference in New Issue
Block a user