julian
22b1b069df
Initialize CLAUDE.md schema, index, and log; ingest three architecture sources (system overview, Teltonika ingestion design, official Teltonika data-sending protocols) into 7 entity pages, 8 concept pages, and 3 source pages with wikilink cross-references.
64 lines
5.1 KiB
Markdown
64 lines
5.1 KiB
Markdown
---
|
||
title: Teltonika Ingestion — Architecture Reference
|
||
type: source
|
||
created: 2026-04-30
|
||
updated: 2026-04-30
|
||
sources: [teltonika-ingestion-architecture]
|
||
tags: [teltonika, ingestion, protocol, codec]
|
||
source_path: raw/teltonika-ingestion-architecture.md
|
||
source_kind: note
|
||
---
|
||
|
||
# Teltonika Ingestion — Architecture Reference
|
||
|
||
## TL;DR
|
||
|
||
Design for the Teltonika protocol adapter inside the [[tcp-ingestion]] service. Goal: ingest telemetry from any Teltonika device — including unseen models — without parser changes, by leaning on the protocol's self-description (codec ID announces framing; IO bag carries opaque model-specific telemetry). Phase 1 implements data-sending codecs **8, 8E, 16**; Phase 2 will add command codecs **12, 13, 14**. Six design principles govern the adapter; the codec dispatch is a registry built so Phase 2 is additive, not a rewrite.
|
||
|
||
## Key claims
|
||
|
||
- **Project lives at `tcp-ingestion/`** — a single Node.js/TypeScript project containing the vendor-agnostic shell (`src/core/`) and per-vendor adapters (`src/adapters/`). Today the only adapter is Teltonika.
|
||
- **Layout rules**: `core/` never imports from `adapters/`; adapters never import from each other; the Teltonika folder is self-contained so it can be lifted into its own service via `git mv` later.
|
||
- **Phase 1 scope** = data-sending codecs 8, 8E, 16 (covers the deployed Teltonika telemetry fleet).
|
||
- **Phase 2 scope** = GPRS command codecs 12, 13, 14 (server → device). Deferred because they are a **distinct feature** with different security implications, not an incremental codec. See [[phase-2-commands]].
|
||
- **The parser is model-agnostic** — what matters is the codec ID byte, not the device model. Fixed AVL fields (timestamp, lat/lon/alt, angle, satellites, speed) are codec-defined; only the IO element bag varies, and it's passed through verbatim. New Teltonika models work on day one.
|
||
- **`Position` shape** is the boundary contract — `device_id`, `timestamp`, `lat`, `lon`, `alt`, `angle`, `speed`, `satellites`, `priority`, `attributes` (raw IO map keyed by numeric ID as string). See [[position-record]].
|
||
- **Six design principles** (priority order):
|
||
1. Implement Codec 8, 8E, 16 — the closed set.
|
||
2. Defer Codec 12, 13, 14.
|
||
3. Pass the IO map through unchanged — naming/interpretation is a Processor concern.
|
||
4. Log unknown codec IDs and drop the connection (loud failure > silent corruption).
|
||
5. Validate CRC-16/IBM; NACK (no ACK) on mismatch — devices retransmit.
|
||
6. Maintain a fixture suite of real packet captures + Teltonika-doc captures + synthetic edge cases.
|
||
- **Codec dispatch is a flat registry** keyed on codec ID — not an inheritance hierarchy. Codec parsers are independent because their record shapes diverge. See [[codec-dispatch]].
|
||
- **Phase 1 forward-compatibility seams** for Phase 2:
|
||
- Codec dispatch is a registry, not a switch (§8.1).
|
||
- Session owns the socket; handlers borrow write access via a `respond(bytes)` callback (§8.2).
|
||
- Per-device state is local to the socket; no shared registry today. Phase 2 adds the connection registry alongside, not woven through (§8.3).
|
||
- **Phase 2 outbound command flow**: `SPA → Directus → Redis Streams → Ingestion → device`. Directus enforces single auth surface; commands are persisted as rows in a `commands` collection before being routed; Redis is transport, not source of truth.
|
||
- **Phase 2 connection registry**: Redis hash `connections:registry` mapping `imei → instance_id`. Per-instance heartbeat keys (`SET instance:heartbeat:{instance_id} EX 90`) plus a registry janitor handle crash recovery — Redis hashes don't support per-field TTL.
|
||
- **Phase 2 command correlation**: Teltonika's command codecs carry no correlation ID, so the protocol assumes one outstanding command per connection. The Ingestion service enforces this via a per-socket write queue.
|
||
|
||
## TCP session lifecycle (Phase 1 happy path)
|
||
|
||
1. Device connects to the Teltonika port.
|
||
2. IMEI handshake: device sends 2-byte length + ASCII IMEI; server responds `0x01` to accept.
|
||
3. AVL data loop: read preamble (4×0x00) + length + payload + 4-byte CRC; validate CRC; dispatch on codec ID byte; emit `Position` records to Redis; ACK with 4-byte big-endian record count.
|
||
4. Session ends on disconnect; no state preserved across sessions.
|
||
|
||
## Notable quotes
|
||
|
||
> "Ingest telemetry from any Teltonika device, including models we have never seen, without code changes to the parser."
|
||
|
||
> "Naming and interpreting IO elements is explicitly a Processor concern, driven by per-model configuration."
|
||
|
||
> "A loud failure (the device reconnects, fails again, shows up in logs) is strictly better than a quiet corruption."
|
||
|
||
> "A fixture suite is not optional infrastructure. It is the only place the parser's correctness is actually verified."
|
||
|
||
## Open questions / follow-ups
|
||
|
||
- Per-model IO dictionary: where does it live downstream — Directus collection, static config in the Processor, or both?
|
||
- Phase 2 timing: no commitment given. Driver will be the first real need to issue commands (configuration, remote actions).
|
||
- Pending-command sweeper cadence (30s) and command default TTL (5 min) — operational defaults, may want tuning once command volume is real.
|