tcp-ingestion/.planning/ROADMAP.md

tcp-ingestion — Roadmap

A Node.js TCP server that accepts persistent connections from GPS hardware, parses vendor binary protocols, and emits normalized Position records onto Redis Streams for the Processor to consume.

This file is the single navigation hub for all implementation planning. Each phase has its own folder with a README and granular task files. Update statuses here as work lands.

Status legend

Symbol	Meaning
⬜	Not started
🟦	Planned (designed, not coded)
🟨	In progress
🟩	Done
⏸	Paused / blocked
❄️	Frozen / future / optional

Architectural anchors

The service is specified by the wiki at ../docs/wiki/. Implementing agents should read these pages before starting any task:

• Architecture — docs/wiki/sources/gps-tracking-architecture.md, docs/wiki/concepts/plane-separation.md, docs/wiki/concepts/failure-domains.md
• Adapter design (this service) — docs/wiki/sources/teltonika-ingestion-architecture.md, docs/wiki/concepts/protocol-adapter.md
• Teltonika protocol — docs/wiki/entities/teltonika.md, docs/wiki/concepts/avl-data-format.md, docs/wiki/concepts/codec-dispatch.md, docs/wiki/concepts/position-record.md, docs/wiki/concepts/io-element-bag.md
• Phase 2 design — docs/wiki/concepts/phase-2-commands.md
• Canonical Teltonika spec — docs/wiki/sources/teltonika-data-sending-protocols.md

Non-negotiable design rules

These rules govern every task. Any deviation must be discussed and documented as a decision before code lands.

1. Vendor-agnostic shell. src/core/ never imports from src/adapters/. Adapters never import from each other. Each adapter folder is self-contained.
2. TCP handler never blocks on downstream work. Redis pushes are queued/awaited but never inside a path that would back-pressure a device socket beyond the TCP buffer.
3. IO element bag passes through unchanged. No naming, no unit conversion, no model lookup in the parser. attributes is keyed by numeric IO ID as string.
4. Loud failure on unknown codec. Drop the connection, do not skip-ahead, do not partial-parse.
5. NACK on CRC mismatch = simply do not ACK; device retransmits.
6. Codec dispatch is a flat registry, not a switch or inheritance hierarchy. Phase 2 must be a pure addition.
7. Fixture-based testing is mandatory for every codec parser. Hex captures + expected Position[] pairs run on every CI build.

Phases

Phase 1 — Inbound telemetry (Codec 8, 8E, 16)

Status: 🟨 In progress (core implementation done; observability + hardening + device authority paused for pilot test) Outcome: A production-ready Node.js TCP server ingesting Teltonika telemetry from any FMB/FMC/FMM/FMU device, publishing normalized Position records to Redis Streams, with full observability and CI/CD via Gitea.

See phase-1-telemetry/README.md

#	Task	Status	Landed in
1.1	Project scaffold	🟩	1e9219d
1.2	Core shell & framing types	🟩	1e9219d
1.3	Configuration & logging	🟩	1e9219d
1.4	Teltonika framing layer (envelope, CRC, handshake)	🟩	1e9219d
1.5	Codec 8 parser	🟩	381287b
1.6	Codec 8 Extended parser (incl. NX)	🟩	381287b
1.7	Codec 16 parser (incl. Generation Type)	🟩	381287b
1.8	Redis Streams publisher & main wiring	🟩	af06973
1.9	Fixture suite & testing strategy	🟩	381287b
1.10	Observability (Prometheus metrics)	⏸	deferred — see below
1.11	Dockerfile & Gitea workflow	🟩	88b742d (slim pilot variant)
1.12	Production hardening	⏸	deferred — see below
1.13	Device authority (Redis allow-list refresher)	⏸	deferred — see below

Deferred (resume after the real-device pilot test)

These three tasks are paused so we can get the service onto real hardware as fast as possible. They are paused, not cancelled — each must be completed before the service is considered production-ready.

• 1.10 Observability (Prometheus metrics). Tracking via the placeholder Metrics interface for now. Resume trigger: as soon as the pilot is generating real traffic and we want to measure it, or before any second instance is deployed (without metrics, "consumer lag" and "unknown codec" alerts cannot fire).
• 1.12 Production hardening. Graceful shutdown is a stub today; uncaught-exception handlers are minimal. Resume trigger: before the pilot graduates to "always-on" or before any deployment that does rolling restarts. Acceptable for a manual pilot where we can stop/start the process by hand.
• 1.13 Device authority (Redis allow-list refresher). Default AllowAllAuthority accepts every IMEI; observability of known | unknown is moot until 1.10 lands. Resume trigger: when Directus has a devices collection publishing the allow-list to Redis, or when the operational picture demands rejecting unknown IMEIs (STRICT_DEVICE_AUTH=true).

When resuming any of these, change the status from ⏸ back to ⬜ or 🟨 here and in the task file's status badge, and clear the deferral note in the task file.

Phase 2 — Outbound commands (Codec 12, 14)

Status: ⬜ Not started (depends on Phase 1) Outcome: Each Ingestion instance runs a parallel command consumer that reads from commands:outbound:{instance_id}, encodes Codec 12 or 14 frames, writes them to the appropriate device socket via a per-socket write queue, and publishes responses back to commands:responses. Includes the IMEI→instance connection registry and heartbeat.

See phase-2-commands/README.md

#	Task	Status
2.1	Connection registry & heartbeat	⬜
2.2	Registry janitor (stale entry cleanup)	⬜
2.3	Per-socket write queue & outstanding-command tracker	⬜
2.4	Command consumer (stream reader)	⬜
2.5	Codec 12 encoder + handler	⬜
2.6	Codec 14 encoder + ACK/nACK handler	⬜

Phase 3 — Future / optional

Status: ❄️ Not committed See phase-3-future/README.md for ideas on radar:

• Additional vendor adapters (Queclink, Concox).
• UDP transport for codecs 8/8E/16.
• Codec 15 (FMX6 RS232 modes) if such a fleet onboards.
• SMS-based protocols (Codec 4 24-position, binary SMS) if SMS fallback connectivity is needed.

Operating model

• Implementation agent contract. Each task file is self-sufficient: goal, deliverables, specification, acceptance criteria. An agent should be able to complete one task without reading the whole wiki — but should skim the wiki references at the top of the task before starting.
• Sequence within a phase. Task numbering reflects intended order. Soft dependencies are explicit in each task's "Depends on" field. Tasks with no dependencies on each other can be done in parallel.
• Status updates. When a task is started, change its row in this ROADMAP to 🟨 and the task file's status badge accordingly. When done, 🟩 + a one-line note in the task file's "Done" section pointing at the merging commit/PR.
• Drift control. If implementation diverges from a task's spec, update the task file before the diverging code lands, with a note explaining why. Do not let plans rot — either fix the plan or fix the code.