Bootstrap LLM-maintained wiki with TRM architecture knowledge
Initialize CLAUDE.md schema, index, and log; ingest three architecture sources (system overview, Teltonika ingestion design, official Teltonika data-sending protocols) into 7 entity pages, 8 concept pages, and 3 source pages with wikilink cross-references.
This commit is contained in:
@@ -0,0 +1,44 @@
|
||||
---
|
||||
title: GPS Tracking Platform — Architecture Overview
|
||||
type: source
|
||||
created: 2026-04-30
|
||||
updated: 2026-04-30
|
||||
sources: [gps-tracking-architecture]
|
||||
tags: [architecture, system-design, overview]
|
||||
source_path: raw/gps-tracking-architecture.md
|
||||
source_kind: note
|
||||
---
|
||||
|
||||
# GPS Tracking Platform — Architecture Overview
|
||||
|
||||
## TL;DR
|
||||
|
||||
System-level reference for a real-time GPS telemetry platform built around four cooperating components: a TCP Ingestion service, a Processor, a Directus instance, and a React SPA. The architecture splits the system along **data velocity** and **failure domain**: a hot telemetry plane (Ingestion → Redis Streams → Processor), a business plane (Directus over PostgreSQL/TimescaleDB), and a presentation plane (React SPA). Each component scales, fails, and deploys independently.
|
||||
|
||||
## Key claims
|
||||
|
||||
- The system is structured as **three concentric planes** — telemetry, business, presentation. Boundaries are enforced by Redis Streams on one side and the database/API on the other; no component reaches across two boundaries. See [[plane-separation]].
|
||||
- **TCP Ingestion** maintains persistent device connections, parses vendor binary protocols, ACKs frames, and pushes normalized records to a Redis Stream. It does not apply business rules, write to Postgres, or serve user APIs. See [[tcp-ingestion]].
|
||||
- **Vendor abstraction** lives at the Ingestion layer via a protocol adapter interface (bytes in → normalized `Position` out). Adding a new device family means writing a new adapter; nothing downstream changes. See [[protocol-adapter]].
|
||||
- **The TCP handler never blocks on downstream work.** If the Processor or DB is slow, the Stream absorbs pressure; Ingestion keeps accepting and acknowledging. This single discipline is "the difference between a system that survives a bad day and one that doesn't."
|
||||
- **Processor** owns domain logic: hot per-device state in memory, durable writes to PostgreSQL/TimescaleDB, derived events. It is the **only writer** for high-volume telemetry tables (positions hypertable). See [[processor]].
|
||||
- **Directus** owns the schema, REST/GraphQL APIs, permissions, WebSockets, admin UI, and Flows — but is **not** in the telemetry hot path. The Processor writes directly to Directus-owned tables; schema authority stays with Directus. See [[directus]].
|
||||
- **React SPA** is the only user-facing surface. It talks exclusively to Directus (REST/GraphQL/WSS via the official SDK); it never touches Ingestion, Processor, Redis, or the database. See [[react-spa]].
|
||||
- **Redis Streams** between Ingestion and Processor provides buffering, replayability (consumer-group offsets), and horizontal scaling. NATS or Kafka are reasonable upgrades for multi-region or very high throughput; Redis is the right choice at current scale. See [[redis-streams]].
|
||||
- **Failure domains are isolated.** Ingestion crash → devices reconnect. Processor crash → resume from offset. Directus crash → telemetry continues; UI offline. The database is the only single point of failure. See [[failure-domains]].
|
||||
- **Recommended SPA stack**: Vite + React + TS, TanStack Router, TanStack Query, @directus/sdk, MapLibre GL + react-map-gl, shadcn/ui + Tailwind, Zustand, react-hook-form + Zod.
|
||||
- **Observability**: Redis Stream consumer lag is the single most important metric — it reflects the health of the entire telemetry pipeline in one number.
|
||||
|
||||
## Notable quotes
|
||||
|
||||
> "The architecture is deliberately split along the lines of **data velocity** and **failure domain**."
|
||||
|
||||
> "The TCP handler **never blocks on downstream work**. … This single discipline is the difference between a system that survives a bad day and one that doesn't."
|
||||
|
||||
> "The architecture deliberately makes the database the **only** part of the system that requires careful operational attention. Everything else is restartable, replaceable, or naturally redundant."
|
||||
|
||||
## Open questions / follow-ups
|
||||
|
||||
- No specific business domain modeled here ("intentionally generic"). Future sources should clarify the actual TRM domain (events, alerts, geofencing rules).
|
||||
- WebSocket fan-out limits cited as "tens to low hundreds" of subscribers — what's the expected scale for TRM?
|
||||
- Multi-region is mentioned as a possible future evolution. Is it on the roadmap?
|
||||
@@ -0,0 +1,121 @@
|
||||
---
|
||||
title: Teltonika Data Sending Protocols (official wiki)
|
||||
type: source
|
||||
created: 2026-04-30
|
||||
updated: 2026-04-30
|
||||
sources: [teltonika-data-sending-protocols]
|
||||
tags: [teltonika, protocol, canonical, reference]
|
||||
source_path: raw/Teltonika Data Sending Protocols - Teltonika Telematics Wiki.md
|
||||
source_url: https://wiki.teltonika-gps.com/view/Teltonika_Data_Sending_Protocols
|
||||
source_kind: article
|
||||
---
|
||||
|
||||
# Teltonika Data Sending Protocols
|
||||
|
||||
Official Teltonika Telematics wiki page documenting all codec families: data-sending (8, 8E, 16), GPRS command (12, 13, 14, **15**), and SMS-based (24-position, binary SMS). Canonical reference; supersedes our internal summaries where they conflict.
|
||||
|
||||
## TL;DR
|
||||
|
||||
The complete codec inventory: **Codec 8 (0x08), 8 Extended (0x8E), 16 (0x10)** for device-to-server telemetry; **Codec 12 (0x0C), 13 (0x0D), 14 (0x0E), 15 (0x0F)** for GPRS-message-based device/server communication; plus **Codec 4 (0x04)** for 24-position SMS and a separate binary-SMS data-sending protocol. All telemetry codecs work over both TCP and UDP. See [[avl-data-format]] for the canonical frame structure and [[teltonika]] for the per-codec summary.
|
||||
|
||||
## What's new vs. our prior wiki
|
||||
|
||||
This source corrects/extends earlier pages with several concrete details we hadn't captured:
|
||||
|
||||
- **Codec 16 hex ID = 0x10** (not just decimal 16).
|
||||
- **UDP is officially supported** for codecs 8/8E/16 — separate envelope with UDP Channel Header (Length 2B + Packet ID 2B + Not Usable Byte 1B), AVL Packet Header (AVL Packet ID 1B + IMEI Length 2B + IMEI 15B), then the AVL Data Array. Server ACK over UDP carries (AVL Packet ID 1B + Number of Accepted 1B), much shorter than the TCP 4-byte big-endian count.
|
||||
- **Codec 8 Extended has variable-size IO elements** (NX section), separate from N1/N2/N4/N8 — IO ID 2B + Length 2B + Value (variable).
|
||||
- **Codec 16 Generation Type** is a 1-byte field with 8 defined values: 0=On Exit, 1=On Entrance, 2=On Both, 3=Reserved, 4=Hysteresis, 5=On Change, 6=Eventual, 7=Periodical.
|
||||
- **Priority is an enum** with values 0=Low, 1=High, 2=Panic.
|
||||
- **Codec 14 has a nACK response type** `0x11` (not just the `0x06` ACK we previously documented). nACK is sent when the command's IMEI does not match the device's actual IMEI. Available from firmware FMB.Ver.03.25.04.Rev.00.
|
||||
- **Codec 13** is **Device→Server only** (one-way), Type always `0x06`, includes a 4-byte timestamp prepended to the command. Used only when the "Message Timestamp" parameter in RS232 settings is enabled.
|
||||
- **Codec 15 (0x0F)** is **new to our wiki** — one-way Device→Server, FMX6 professional devices, RS232 modes (TCP/UDP Ascii/Binary, ±Buffered). Includes both timestamp (4B, **seconds**, not ms) and IMEI (8B HEX). Out of scope for our deployed FMB/FMC/FMM/FMU fleet.
|
||||
- **Packet size limits**: min AVL record 45B (all IO elements disabled); max AVL record 255B; **max AVL packet 512B for FMB640/FMB641/FMC640/FMM640, 1280B for other devices**.
|
||||
- **AVL ID range**: AVL IDs > 255 are only available in Codec 16 (and, by inference from the spec, Codec 8E with its 2-byte IO ID width).
|
||||
- **Coordinate encoding**: Lat/Lon are 4-byte integers built from `((d + m/60 + s/3600 + ms/3600000) × 10⁷)` per axis. Negative coords use **two's complement** — first bit = sign. Speed = `0x0000` when GPS data is invalid.
|
||||
- **CRC scope** is precisely "from Codec ID to second Number of Data" (we had said roughly "the data section"). CRC is CRC-16/IBM, lower 2 bytes of the 4-byte field carry the value.
|
||||
- **Codec 12 GPRS command session prerequisite**: device must have sent AVL data and received a correct ACK before commands can be sent over the same socket. Recommended `Active datalink timeout` is 259200 (max) so the session stays open.
|
||||
|
||||
## Key claims (canonical)
|
||||
|
||||
### Codec table
|
||||
|
||||
| Codec | Hex ID | Direction | Purpose |
|
||||
|-------|--------|-----------|---------|
|
||||
| 8 | 0x08 | device → server | AVL telemetry; 1-byte IO IDs |
|
||||
| 8 Extended | 0x8E | device → server | AVL telemetry; 2-byte IO IDs + variable-length IO |
|
||||
| 16 | 0x10 | device → server | AVL telemetry; 2-byte IO IDs + Generation Type; supports IO IDs > 255 |
|
||||
| 12 | 0x0C | bidirectional | Server commands + device responses (ASCII text) |
|
||||
| 13 | 0x0D | device → server | One-way command upload with timestamp |
|
||||
| 14 | 0x0E | bidirectional | Like 12, addressed by IMEI; ACK 0x06 / nACK 0x11 |
|
||||
| 15 | 0x0F | device → server | One-way; timestamp + IMEI; FMX6 RS232 modes only |
|
||||
| 4 | 0x04 | SMS (one-way) | 24-position daily SMS, bit-field compressed |
|
||||
|
||||
### AVL outer envelope (TCP, codecs 8/8E/16)
|
||||
|
||||
```
|
||||
[Preamble 4B = 0x00000000][DataFieldLength 4B][CodecID 1B][N1 1B][AVL Data X B][N2 1B][CRC 4B]
|
||||
```
|
||||
|
||||
- `DataFieldLength` = bytes from `CodecID` through `N2`.
|
||||
- `CRC-16/IBM` calculated over the same range.
|
||||
- `N1` MUST equal `N2` (record count, repeated).
|
||||
- TCP server ACK = **4-byte big-endian integer** = number of records accepted.
|
||||
|
||||
### AVL record (all three telemetry codecs)
|
||||
|
||||
```
|
||||
[Timestamp 8B (UNIX ms)][Priority 1B][GPS Element 15B][IO Element X B]
|
||||
```
|
||||
|
||||
- **GPS Element**: `Longitude 4B + Latitude 4B + Altitude 2B + Angle 2B + Satellites 1B + Speed 2B` — total 15 bytes.
|
||||
- **Priority**: 0=Low, 1=High, 2=Panic.
|
||||
- IO Element layout differs per codec — see [[avl-data-format]].
|
||||
|
||||
### IMEI handshake
|
||||
|
||||
Device → server: `2-byte length` + ASCII IMEI (e.g. `000F` + 15 ASCII bytes for a 15-digit IMEI).
|
||||
Server → device: `0x01` (accept) or `0x00` (reject).
|
||||
|
||||
### UDP envelope (data codecs)
|
||||
|
||||
```
|
||||
[Length 2B][Packet ID 2B][Not Usable Byte 1B] | [AVL Packet ID 1B][IMEI Length 2B = 0x000F][IMEI 15B] | [AVL Data Array (same as TCP)]
|
||||
```
|
||||
|
||||
UDP server ACK: `[Length 2B = 0x0005][Packet ID][Not Usable 1B] | [AVL Packet ID 1B][Number of Accepted 1B]`.
|
||||
|
||||
### Codec 12 message structure
|
||||
|
||||
```
|
||||
[Preamble 4B][DataSize 4B][CodecID 0x0C][CmdQty1 1B][Type 1B][CmdSize 4B][Command X B][CmdQty2 1B][CRC 4B]
|
||||
```
|
||||
|
||||
- `Type`: `0x05` for command (server → device), `0x06` for response (device → server).
|
||||
- `CRC-16/IBM` from `CodecID` through `CmdQty2`.
|
||||
- Command/Response field is ASCII text encoded as HEX (e.g. `getinfo` → `67 65 74 69 6E 66 6F`).
|
||||
|
||||
### Codec 14 specifics
|
||||
|
||||
```
|
||||
[...][CodecID 0x0E][CmdQty 1B][Type 1B][Size 4B = command + 8 IMEI][IMEI HEX 8B][Command X B][CmdQty 1B][CRC 4B]
|
||||
```
|
||||
|
||||
- IMEI is encoded as 8-byte HEX (e.g. IMEI `123456789123456` → `01 23 45 67 89 12 34 56`).
|
||||
- **Type 0x06 = ACK** (IMEI matched, command executed); **Type 0x11 = nACK** (IMEI mismatch, command rejected).
|
||||
- Available from FMB.Ver.03.25.04.Rev.00.
|
||||
|
||||
## Notable quotes
|
||||
|
||||
> "Note that the GPRS session should remain active between the device and server, while GPRS commands are sent. For this reason, active datalink timeout (global parameters in device configuration) is recommended to be set to 259200 (maximum value)."
|
||||
|
||||
> "AVL IDs that are higher than 255 will can be used only in the Codec16 protocol."
|
||||
|
||||
> "If command message IMEI is equal to actual device IMEI, received command will be executed and response will be sent with ACK (0x06) … If the command message IMEI doesn't match … response to the server will be sent with nACK (0x11)."
|
||||
|
||||
## Open questions / follow-ups
|
||||
|
||||
- Codec 8 Extended NX (variable-size IO) parser handling — what shape do those values take in our [[position-record]] `attributes` bag? Likely raw `Buffer`, length-aware. Needs explicit handling in the parser fixture suite ([[teltonika-ingestion-architecture]] §5.6).
|
||||
- Codec 16 Generation Type — should this be promoted into a typed field in [[position-record]] (since it's codec-defined, not model-defined)? Currently we only specify `priority`; Generation Type is similar in nature.
|
||||
- Should we plan to support the SMS-based protocols (Codec 4 / binary SMS) for fallback connectivity? Probably not for now — SMS is rare in the deployed fleet, and adding it means an SMS gateway integration well outside the TCP service.
|
||||
- Codec 15 is FMX6-only and out of scope. But if the platform ever onboards an FMX6 fleet, this becomes a Phase-3 line item.
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
title: Teltonika Ingestion — Architecture Reference
|
||||
type: source
|
||||
created: 2026-04-30
|
||||
updated: 2026-04-30
|
||||
sources: [teltonika-ingestion-architecture]
|
||||
tags: [teltonika, ingestion, protocol, codec]
|
||||
source_path: raw/teltonika-ingestion-architecture.md
|
||||
source_kind: note
|
||||
---
|
||||
|
||||
# Teltonika Ingestion — Architecture Reference
|
||||
|
||||
## TL;DR
|
||||
|
||||
Design for the Teltonika protocol adapter inside the [[tcp-ingestion]] service. Goal: ingest telemetry from any Teltonika device — including unseen models — without parser changes, by leaning on the protocol's self-description (codec ID announces framing; IO bag carries opaque model-specific telemetry). Phase 1 implements data-sending codecs **8, 8E, 16**; Phase 2 will add command codecs **12, 13, 14**. Six design principles govern the adapter; the codec dispatch is a registry built so Phase 2 is additive, not a rewrite.
|
||||
|
||||
## Key claims
|
||||
|
||||
- **Project lives at `tcp-ingestion/`** — a single Node.js/TypeScript project containing the vendor-agnostic shell (`src/core/`) and per-vendor adapters (`src/adapters/`). Today the only adapter is Teltonika.
|
||||
- **Layout rules**: `core/` never imports from `adapters/`; adapters never import from each other; the Teltonika folder is self-contained so it can be lifted into its own service via `git mv` later.
|
||||
- **Phase 1 scope** = data-sending codecs 8, 8E, 16 (covers the deployed Teltonika telemetry fleet).
|
||||
- **Phase 2 scope** = GPRS command codecs 12, 13, 14 (server → device). Deferred because they are a **distinct feature** with different security implications, not an incremental codec. See [[phase-2-commands]].
|
||||
- **The parser is model-agnostic** — what matters is the codec ID byte, not the device model. Fixed AVL fields (timestamp, lat/lon/alt, angle, satellites, speed) are codec-defined; only the IO element bag varies, and it's passed through verbatim. New Teltonika models work on day one.
|
||||
- **`Position` shape** is the boundary contract — `device_id`, `timestamp`, `lat`, `lon`, `alt`, `angle`, `speed`, `satellites`, `priority`, `attributes` (raw IO map keyed by numeric ID as string). See [[position-record]].
|
||||
- **Six design principles** (priority order):
|
||||
1. Implement Codec 8, 8E, 16 — the closed set.
|
||||
2. Defer Codec 12, 13, 14.
|
||||
3. Pass the IO map through unchanged — naming/interpretation is a Processor concern.
|
||||
4. Log unknown codec IDs and drop the connection (loud failure > silent corruption).
|
||||
5. Validate CRC-16/IBM; NACK (no ACK) on mismatch — devices retransmit.
|
||||
6. Maintain a fixture suite of real packet captures + Teltonika-doc captures + synthetic edge cases.
|
||||
- **Codec dispatch is a flat registry** keyed on codec ID — not an inheritance hierarchy. Codec parsers are independent because their record shapes diverge. See [[codec-dispatch]].
|
||||
- **Phase 1 forward-compatibility seams** for Phase 2:
|
||||
- Codec dispatch is a registry, not a switch (§8.1).
|
||||
- Session owns the socket; handlers borrow write access via a `respond(bytes)` callback (§8.2).
|
||||
- Per-device state is local to the socket; no shared registry today. Phase 2 adds the connection registry alongside, not woven through (§8.3).
|
||||
- **Phase 2 outbound command flow**: `SPA → Directus → Redis Streams → Ingestion → device`. Directus enforces single auth surface; commands are persisted as rows in a `commands` collection before being routed; Redis is transport, not source of truth.
|
||||
- **Phase 2 connection registry**: Redis hash `connections:registry` mapping `imei → instance_id`. Per-instance heartbeat keys (`SET instance:heartbeat:{instance_id} EX 90`) plus a registry janitor handle crash recovery — Redis hashes don't support per-field TTL.
|
||||
- **Phase 2 command correlation**: Teltonika's command codecs carry no correlation ID, so the protocol assumes one outstanding command per connection. The Ingestion service enforces this via a per-socket write queue.
|
||||
|
||||
## TCP session lifecycle (Phase 1 happy path)
|
||||
|
||||
1. Device connects to the Teltonika port.
|
||||
2. IMEI handshake: device sends 2-byte length + ASCII IMEI; server responds `0x01` to accept.
|
||||
3. AVL data loop: read preamble (4×0x00) + length + payload + 4-byte CRC; validate CRC; dispatch on codec ID byte; emit `Position` records to Redis; ACK with 4-byte big-endian record count.
|
||||
4. Session ends on disconnect; no state preserved across sessions.
|
||||
|
||||
## Notable quotes
|
||||
|
||||
> "Ingest telemetry from any Teltonika device, including models we have never seen, without code changes to the parser."
|
||||
|
||||
> "Naming and interpreting IO elements is explicitly a Processor concern, driven by per-model configuration."
|
||||
|
||||
> "A loud failure (the device reconnects, fails again, shows up in logs) is strictly better than a quiet corruption."
|
||||
|
||||
> "A fixture suite is not optional infrastructure. It is the only place the parser's correctness is actually verified."
|
||||
|
||||
## Open questions / follow-ups
|
||||
|
||||
- Per-model IO dictionary: where does it live downstream — Directus collection, static config in the Processor, or both?
|
||||
- Phase 2 timing: no commitment given. Driver will be the first real need to issue commands (configuration, remote actions).
|
||||
- Pending-command sweeper cadence (30s) and command default TTL (5 min) — operational defaults, may want tuning once command volume is real.
|
||||
Reference in New Issue
Block a user