tcp-ingestion/.planning/phase-1-telemetry/04-teltonika-framing.md

Task 1.4 — Teltonika framing layer

Phase: 1 — Inbound telemetry Status: ⬜ Not started Depends on: 1.2 Wiki refs: docs/wiki/concepts/avl-data-format.md (envelope, IMEI handshake), docs/wiki/concepts/codec-dispatch.md, docs/wiki/sources/teltonika-data-sending-protocols.md

Goal

Implement the Teltonika adapter shell: IMEI handshake, AVL frame envelope read loop, CRC validation, and codec dispatch registry. Codec parsers themselves are tasks 1.5–1.7; this task lays the framing rails they slot into.

Deliverables

• src/adapters/teltonika/index.ts — the Adapter export consumed by src/core/. Wires the codec registry, exports { name: 'teltonika', ports: [config.TELTONIKA_PORT], handleSession }.
• src/adapters/teltonika/handshake.ts — readImeiHandshake(socket): Promise<string> performs the 2-byte length + ASCII IMEI read and returns the IMEI string. Does not write the accept/reject byte itself — that decision is made by the session loop after consulting DeviceAuthority (see "DeviceAuthority seam" below). On malformed input, throws a typed HandshakeError.
• src/adapters/teltonika/device-authority.ts — defines the DeviceAuthority interface and ships an AllowAllAuthority default implementation. The opt-in Redis-backed authority lives in task 1.13.
• src/adapters/teltonika/frame.ts — readNextFrame(socket, buffer): Promise<{ codecId: number; payload: Buffer; crcValid: boolean }> plus a small BufferedReader class that handles partial-read accumulation across socket.on('data') events.
• src/adapters/teltonika/crc.ts — pure function crc16Ibm(buf: Buffer): number. Implements CRC-16/IBM (polynomial 0xA001, init 0x0000, no reflection beyond the polynomial standard).
• src/adapters/teltonika/codec/registry.ts — internal-to-adapter codec registry: Map<codecId, CodecDataHandler>. Phase 1 registers handlers from codec/data/. Phase 2 will register from codec/command/.

Specification

IMEI handshake

Device → Server: [length 2B big-endian][IMEI bytes (ASCII, length B)]
Server → Device: 0x01 (accept) | 0x00 (reject)

Phase 1 default: accept all syntactically valid IMEIs. Authorization (the question of whether a given IMEI is expected to be in the fleet) is a soft observability concern, not a hard gate, until task 1.13 adds the opt-in allow-list refresher. The handshake consults a DeviceAuthority interface for a known | unknown label that flows into metrics and logs but does not block the handshake by default.

Parse rules:

• Length must be ≤ 32 (Teltonika IMEIs are 15 ASCII digits; we allow some headroom).
• IMEI body must match /^\d{14,16}$/ after ASCII decode.
• Anything malformed: HandshakeError, log WARN with the offending bytes (truncated), destroy socket, no 0x01.

DeviceAuthority seam

export interface DeviceAuthority {
  check(imei: string): Promise<'known' | 'unknown'>;
}

export class AllowAllAuthority implements DeviceAuthority {
  async check(): Promise<'known'> { return 'known'; }
}

Wire DeviceAuthority into the Teltonika adapter context. Default binding in main.ts is new AllowAllAuthority() — every IMEI is reported as known until a real authority is configured.

Behavior of the handshake:

const imei = await readImeiHandshake(socket);
const authority = ctx.deviceAuthority;
const knownLabel = await authority.check(imei).catch((err) => {
  ctx.logger.warn({ err, imei }, 'device authority check failed; defaulting to unknown');
  return 'unknown' as const;
});
ctx.metrics.handshake.inc({ result: 'accepted', known: knownLabel });
if (knownLabel === 'unknown' && config.STRICT_DEVICE_AUTH) {
  // Reject (rare; off by default)
  socket.write(Buffer.from([0x00]));
  ctx.logger.warn({ imei }, 'rejected unknown device under STRICT_DEVICE_AUTH');
  socket.destroy();
  return;
}
socket.write(Buffer.from([0x01]));

Three properties:

1. Default behavior is unchanged from accept-all. No business-plane dependency.
2. Unknown devices are visible via the known label on teltonika_handshake_total (see task 1.10).
3. STRICT_DEVICE_AUTH=true flips the policy to reject-unknowns. Off by default. When operators want this, they enable it; the code path is already there.

The real implementation of DeviceAuthority (Redis-backed, refreshed from a Directus-published allow-list) is task 1.13. Task 1.4 only ships the interface and the AllowAllAuthority default.

AVL frame envelope

Per avl-data-format:

[Preamble 4B = 0x00000000]
[DataFieldLength 4B big-endian]
[CodecID 1B]
[N1 1B]
[AVL records — DataFieldLength minus 2 bytes for CodecID and N1, minus 1 byte for N2]
[N2 1B]
[CRC 4B]

Important framing rules:

• DataFieldLength is NOT the size of the AVL records section — it is the size from CodecID through N2 inclusive. So the bytes to read after the length field = DataFieldLength + 4 (CRC).
• CRC is computed from CodecID through N2 (the same span as DataFieldLength).
• N1 must equal N2. Mismatch is a malformation; treat like CRC failure (no ACK, log, drop the connection — N1≠N2 is structural, not transient).
• The CRC field is 4 bytes but only the lower 2 contain the value; upper 2 are zero. Read all 4; validate the lower 16 bits.

Pseudocode for the read loop:

const reader = new BufferedReader(socket);
while (!socket.destroyed) {
  const preamble = await reader.readExact(4);
  if (preamble.readUInt32BE() !== 0) {
    logger.warn({ imei }, 'invalid preamble; dropping connection');
    socket.destroy();
    return;
  }
  const length = (await reader.readExact(4)).readUInt32BE();
  if (length < 8 || length > MAX_AVL_PACKET_SIZE) { /* log + destroy */ }
  const body = await reader.readExact(length);   // CodecID + N1 + records + N2
  const crcField = await reader.readExact(4);
  const expectedCrc = crcField.readUInt16BE(2);  // lower 2 of 4
  const computedCrc = crc16Ibm(body);
  if (expectedCrc !== computedCrc) {
    metrics.frames.inc({ codec: codecLabel(body[0]), result: 'crc_fail' });
    logger.warn({ imei, expected_crc: expectedCrc, computed_crc: computedCrc }, 'CRC mismatch');
    continue; // do NOT ack; device retransmits
  }
  const codecId = body[0];
  const handler = codecRegistry.get(codecId);
  if (!handler) {
    metrics.unknownCodec.inc({ codec_id: String(codecId) });
    logger.warn({ imei, codec_id: codecId, header: body.subarray(0, 16).toString('hex') }, 'unknown codec; dropping connection');
    socket.destroy();
    return;
  }
  const result = await handler.handle(body, ctx);
  // ACK: 4-byte big-endian count of records accepted
  const ack = Buffer.alloc(4);
  ack.writeUInt32BE(result.recordCount, 0);
  socket.write(ack);
}

CRC-16/IBM

Polynomial 0xA001 (reflected 0x8005), initial value 0x0000, no final XOR. Implementation should be a tight loop with a precomputed lookup table for performance — protocol parsing is on the hot path. The Teltonika doc references CRC-16/IBM; it's the same as ARC.

Test against the canonical doc's worked example:

• Frame body 08010000016B40D8EA30010000000000000000000000000000000105021503010101425E0F01F10000601A014E000000000000000001 (codec 8, see docs/raw/...) expected CRC = 0x0000C7CF (lower 16 bits = 0xC7CF).

MAX_AVL_PACKET_SIZE

Constant for sanity check on DataFieldLength. Use 1300 to cover both fleet caps (512B for FMB640 family, 1280B for others) with small headroom. Larger frames are malformed and we drop the connection.

Codec registry structure

export interface CodecDataHandler {
  codec_id: number;
  handle(
    body: Buffer,                                // CodecID + N1 + records + N2
    ctx: { imei: string; publish: (p: Position) => Promise<void>; logger: Logger; metrics: Metrics }
  ): Promise<{ recordCount: number }>;
}

The handle body skips the framing-level concerns (envelope, CRC, codec dispatch) — those happen above. Each codec parser receives the validated body and is responsible for parsing N1/N2, the records themselves, and producing Position records via ctx.publish.

Acceptance criteria

• CRC-16/IBM matches the canonical Teltonika example byte-for-byte.
• readImeiHandshake returns a parsed IMEI for well-formed input without writing to the socket.
• readImeiHandshake rejects malformed input by throwing without writing anything.
• The session loop, after a successful handshake, consults DeviceAuthority.check, increments teltonika_handshake_total{result, known}, and writes 0x01 (or 0x00 under STRICT_DEVICE_AUTH).
• AllowAllAuthority always returns 'known'; verified by a unit test.
• STRICT_DEVICE_AUTH=true causes an unknown device to receive 0x00 and have its socket destroyed; verified by an integration test with a stub authority.
• BufferedReader.readExact(n) correctly handles cases where bytes arrive across multiple data events.
• readNextFrame correctly identifies CRC mismatch without dropping the connection.
• readNextFrame drops the connection on unknown codec ID and logs the structured warn line.
• All paths that write to the socket use a single point of ACK emission so Phase 2 can later interpose a write queue without rewriting framing code.

Risks / open questions

• BufferedReader correctness is critical. Use a battle-tested approach — a queue of pending reads with a backing Buffer.concat accumulator. Alternatively use Node's node:stream/web async iterator if the ergonomics fit.
• The await ctx.publish(p) inside the handler is the boundary where Phase 1's "TCP handler never blocks on downstream" property is enforced. The publish must use a non-blocking strategy (fire-and-forget into a bounded queue, or guarantee Redis publish is fast enough). Task 1.8 specifies the publish strategy; this task only needs to make the await semantically correct.

Done

(Fill in once complete.)