docs/wiki/sources/teltonika-ingestion-architecture.md

title, type, created, updated, sources, tags, source_path, source_kind

title	type	created	updated	sources	tags	source_path	source_kind
Teltonika Ingestion — Architecture Reference	source	2026-04-30	2026-04-30	teltonika-ingestion-architecture	teltonika ingestion protocol codec	raw/teltonika-ingestion-architecture.md	note

Teltonika Ingestion — Architecture Reference

TL;DR

Design for the Teltonika protocol adapter inside the tcp-ingestion service. Goal: ingest telemetry from any Teltonika device — including unseen models — without parser changes, by leaning on the protocol's self-description (codec ID announces framing; IO bag carries opaque model-specific telemetry). Phase 1 implements data-sending codecs 8, 8E, 16; Phase 2 will add command codecs 12, 13, 14. Six design principles govern the adapter; the codec dispatch is a registry built so Phase 2 is additive, not a rewrite.

Key claims

• Project lives at tcp-ingestion/ — a single Node.js/TypeScript project containing the vendor-agnostic shell (src/core/) and per-vendor adapters (src/adapters/). Today the only adapter is Teltonika.
• Layout rules: core/ never imports from adapters/; adapters never import from each other; the Teltonika folder is self-contained so it can be lifted into its own service via git mv later.
• Phase 1 scope = data-sending codecs 8, 8E, 16 (covers the deployed Teltonika telemetry fleet).
• Phase 2 scope = GPRS command codecs 12, 13, 14 (server → device). Deferred because they are a distinct feature with different security implications, not an incremental codec. See phase-2-commands.
• The parser is model-agnostic — what matters is the codec ID byte, not the device model. Fixed AVL fields (timestamp, lat/lon/alt, angle, satellites, speed) are codec-defined; only the IO element bag varies, and it's passed through verbatim. New Teltonika models work on day one.
• Position shape is the boundary contract — device_id, timestamp, lat, lon, alt, angle, speed, satellites, priority, attributes (raw IO map keyed by numeric ID as string). See position-record.
• Six design principles (priority order):
  1. Implement Codec 8, 8E, 16 — the closed set.
  2. Defer Codec 12, 13, 14.
  3. Pass the IO map through unchanged — naming/interpretation is a Processor concern.
  4. Log unknown codec IDs and drop the connection (loud failure > silent corruption).
  5. Validate CRC-16/IBM; NACK (no ACK) on mismatch — devices retransmit.
  6. Maintain a fixture suite of real packet captures + Teltonika-doc captures + synthetic edge cases.
• Codec dispatch is a flat registry keyed on codec ID — not an inheritance hierarchy. Codec parsers are independent because their record shapes diverge. See codec-dispatch.
• Phase 1 forward-compatibility seams for Phase 2:
  • Codec dispatch is a registry, not a switch (§8.1).
  • Session owns the socket; handlers borrow write access via a respond(bytes) callback (§8.2).
  • Per-device state is local to the socket; no shared registry today. Phase 2 adds the connection registry alongside, not woven through (§8.3).
• Phase 2 outbound command flow: SPA → Directus → Redis Streams → Ingestion → device. Directus enforces single auth surface; commands are persisted as rows in a commands collection before being routed; Redis is transport, not source of truth.
• Phase 2 connection registry: Redis hash connections:registry mapping imei → instance_id. Per-instance heartbeat keys (SET instance:heartbeat:{instance_id} EX 90) plus a registry janitor handle crash recovery — Redis hashes don't support per-field TTL.
• Phase 2 command correlation: Teltonika's command codecs carry no correlation ID, so the protocol assumes one outstanding command per connection. The Ingestion service enforces this via a per-socket write queue.

TCP session lifecycle (Phase 1 happy path)

1. Device connects to the Teltonika port.
2. IMEI handshake: device sends 2-byte length + ASCII IMEI; server responds 0x01 to accept.
3. AVL data loop: read preamble (4×0x00) + length + payload + 4-byte CRC; validate CRC; dispatch on codec ID byte; emit Position records to Redis; ACK with 4-byte big-endian record count.
4. Session ends on disconnect; no state preserved across sessions.

Notable quotes

"Ingest telemetry from any Teltonika device, including models we have never seen, without code changes to the parser."

"Naming and interpreting IO elements is explicitly a Processor concern, driven by per-model configuration."

"A loud failure (the device reconnects, fails again, shows up in logs) is strictly better than a quiet corruption."

"A fixture suite is not optional infrastructure. It is the only place the parser's correctness is actually verified."

Open questions / follow-ups

• Per-model IO dictionary: where does it live downstream — Directus collection, static config in the Processor, or both?
• Phase 2 timing: no commitment given. Driver will be the first real need to issue commands (configuration, remote actions).
• Pending-command sweeper cadence (30s) and command default TTL (5 min) — operational defaults, may want tuning once command volume is real.