tcp-ingestion/.planning/phase-1-telemetry/04-teltonika-framing.md

# Task 1.4 — Teltonika framing layer

**Phase:** 1 — Inbound telemetry
**Status:** 🟩 Done — landed in commit `1e9219d`
**Depends on:** 1.2
**Wiki refs:** `docs/wiki/concepts/avl-data-format.md` (envelope, IMEI handshake), `docs/wiki/concepts/codec-dispatch.md`, `docs/wiki/sources/teltonika-data-sending-protocols.md`

## Goal

Implement the Teltonika **adapter shell**: IMEI handshake, AVL frame envelope read loop, CRC validation, and codec dispatch registry. Codec parsers themselves are tasks 1.5–1.7; this task lays the framing rails they slot into.

## Deliverables

- `src/adapters/teltonika/index.ts` — the `Adapter` export consumed by `src/core/`. Wires the codec registry, exports `{ name: 'teltonika', ports: [config.TELTONIKA_PORT], handleSession }`.
- `src/adapters/teltonika/handshake.ts` — `readImeiHandshake(socket): Promise<string>` performs the 2-byte length + ASCII IMEI read and returns the IMEI string. **Does not write the accept/reject byte itself** — that decision is made by the session loop after consulting `DeviceAuthority` (see "DeviceAuthority seam" below). On malformed input, throws a typed `HandshakeError`.
- `src/adapters/teltonika/device-authority.ts` — defines the `DeviceAuthority` interface and ships an `AllowAllAuthority` default implementation. The opt-in Redis-backed authority lives in task 1.13.
- `src/adapters/teltonika/frame.ts` — `readNextFrame(socket, buffer): Promise<{ codecId: number; payload: Buffer; crcValid: boolean }>` plus a small `BufferedReader` class that handles partial-read accumulation across `socket.on('data')` events.
- `src/adapters/teltonika/crc.ts` — pure function `crc16Ibm(buf: Buffer): number`. Implements CRC-16/IBM (polynomial `0xA001`, init `0x0000`, no reflection beyond the polynomial standard).
- `src/adapters/teltonika/codec/registry.ts` — internal-to-adapter codec registry: `Map<codecId, CodecDataHandler>`. Phase 1 registers handlers from `codec/data/`. Phase 2 will register from `codec/command/`.

## Specification

### IMEI handshake

```
Device → Server: [length 2B big-endian][IMEI bytes (ASCII, length B)]
Server → Device: 0x01 (accept) | 0x00 (reject)
```

Phase 1 default: **accept all syntactically valid IMEIs.** Authorization (the question of whether a given IMEI is *expected* to be in the fleet) is a soft observability concern, not a hard gate, until task 1.13 adds the opt-in allow-list refresher. The handshake consults a `DeviceAuthority` interface for a `known | unknown` label that flows into metrics and logs but does **not** block the handshake by default.

Parse rules:
- Length must be ≤ 32 (Teltonika IMEIs are 15 ASCII digits; we allow some headroom).
- IMEI body must match `/^\d{14,16}$/` after ASCII decode.
- Anything malformed: `HandshakeError`, log `WARN` with the offending bytes (truncated), destroy socket, no `0x01`.

### `DeviceAuthority` seam

```ts
export interface DeviceAuthority {
  check(imei: string): Promise<'known' | 'unknown'>;
}

export class AllowAllAuthority implements DeviceAuthority {
  async check(): Promise<'known'> { return 'known'; }
}
```

Wire `DeviceAuthority` into the Teltonika adapter context. Default binding in `main.ts` is `new AllowAllAuthority()` — every IMEI is reported as `known` until a real authority is configured.

Behavior of the handshake:

```ts
const imei = await readImeiHandshake(socket);
const authority = ctx.deviceAuthority;
const knownLabel = await authority.check(imei).catch((err) => {
  ctx.logger.warn({ err, imei }, 'device authority check failed; defaulting to unknown');
  return 'unknown' as const;
});
ctx.metrics.handshake.inc({ result: 'accepted', known: knownLabel });
if (knownLabel === 'unknown' && config.STRICT_DEVICE_AUTH) {
  // Reject (rare; off by default)
  socket.write(Buffer.from([0x00]));
  ctx.logger.warn({ imei }, 'rejected unknown device under STRICT_DEVICE_AUTH');
  socket.destroy();
  return;
}
socket.write(Buffer.from([0x01]));
```

Three properties:

1. **Default behavior is unchanged from accept-all.** No business-plane dependency.
2. **Unknown devices are *visible*** via the `known` label on `teltonika_handshake_total` (see task 1.10).
3. **`STRICT_DEVICE_AUTH=true`** flips the policy to reject-unknowns. Off by default. When operators want this, they enable it; the code path is already there.

The real implementation of `DeviceAuthority` (Redis-backed, refreshed from a Directus-published allow-list) is task 1.13. Task 1.4 only ships the interface and the `AllowAllAuthority` default.

### AVL frame envelope

Per [[avl-data-format]]:

```
[Preamble 4B = 0x00000000]
[DataFieldLength 4B big-endian]
[CodecID 1B]
[N1 1B]
[AVL records — DataFieldLength minus 2 bytes for CodecID and N1, minus 1 byte for N2]
[N2 1B]
[CRC 4B]
```

Important framing rules:

- **DataFieldLength is NOT the size of the AVL records section** — it is the size from `CodecID` through `N2` inclusive. So the bytes to read after the length field = `DataFieldLength + 4 (CRC)`.
- **CRC is computed from `CodecID` through `N2`** (the same span as `DataFieldLength`).
- **N1 must equal N2.** Mismatch is a malformation; treat like CRC failure (no ACK, log, **drop the connection** — N1≠N2 is structural, not transient).
- **The CRC field is 4 bytes** but only the lower 2 contain the value; upper 2 are zero. Read all 4; validate the lower 16 bits.

Pseudocode for the read loop:

```ts
const reader = new BufferedReader(socket);
while (!socket.destroyed) {
  const preamble = await reader.readExact(4);
  if (preamble.readUInt32BE() !== 0) {
    logger.warn({ imei }, 'invalid preamble; dropping connection');
    socket.destroy();
    return;
  }
  const length = (await reader.readExact(4)).readUInt32BE();
  if (length < 8 || length > MAX_AVL_PACKET_SIZE) { /* log + destroy */ }
  const body = await reader.readExact(length);   // CodecID + N1 + records + N2
  const crcField = await reader.readExact(4);
  const expectedCrc = crcField.readUInt16BE(2);  // lower 2 of 4
  const computedCrc = crc16Ibm(body);
  if (expectedCrc !== computedCrc) {
    metrics.frames.inc({ codec: codecLabel(body[0]), result: 'crc_fail' });
    logger.warn({ imei, expected_crc: expectedCrc, computed_crc: computedCrc }, 'CRC mismatch');
    continue; // do NOT ack; device retransmits
  }
  const codecId = body[0];
  const handler = codecRegistry.get(codecId);
  if (!handler) {
    metrics.unknownCodec.inc({ codec_id: String(codecId) });
    logger.warn({ imei, codec_id: codecId, header: body.subarray(0, 16).toString('hex') }, 'unknown codec; dropping connection');
    socket.destroy();
    return;
  }
  const result = await handler.handle(body, ctx);
  // ACK: 4-byte big-endian count of records accepted
  const ack = Buffer.alloc(4);
  ack.writeUInt32BE(result.recordCount, 0);
  socket.write(ack);
}
```

### CRC-16/IBM

Polynomial `0xA001` (reflected `0x8005`), initial value `0x0000`, no final XOR. Implementation should be a tight loop with a precomputed lookup table for performance — protocol parsing is on the hot path. The Teltonika doc references CRC-16/IBM; it's the same as ARC.

Test against the canonical doc's worked example:
- Frame body `08010000016B40D8EA30010000000000000000000000000000000105021503010101425E0F01F10000601A014E000000000000000001` (codec 8, see `docs/raw/...`)  expected CRC = `0x0000C7CF` (lower 16 bits = `0xC7CF`).

### MAX_AVL_PACKET_SIZE

Constant for sanity check on `DataFieldLength`. Use `1300` to cover both fleet caps (512B for FMB640 family, 1280B for others) with small headroom. Larger frames are malformed and we drop the connection.

### Codec registry structure

```ts
export interface CodecDataHandler {
  codec_id: number;
  handle(
    body: Buffer,                                // CodecID + N1 + records + N2
    ctx: { imei: string; publish: (p: Position) => Promise<void>; logger: Logger; metrics: Metrics }
  ): Promise<{ recordCount: number }>;
}
```

The `handle` body skips the framing-level concerns (envelope, CRC, codec dispatch) — those happen above. Each codec parser receives the validated body and is responsible for parsing N1/N2, the records themselves, and producing `Position` records via `ctx.publish`.

## Acceptance criteria

- [ ] CRC-16/IBM matches the canonical Teltonika example byte-for-byte.
- [ ] `readImeiHandshake` returns a parsed IMEI for well-formed input without writing to the socket.
- [ ] `readImeiHandshake` rejects malformed input by throwing without writing anything.
- [ ] The session loop, after a successful handshake, consults `DeviceAuthority.check`, increments `teltonika_handshake_total{result, known}`, and writes `0x01` (or `0x00` under `STRICT_DEVICE_AUTH`).
- [ ] `AllowAllAuthority` always returns `'known'`; verified by a unit test.
- [ ] `STRICT_DEVICE_AUTH=true` causes an `unknown` device to receive `0x00` and have its socket destroyed; verified by an integration test with a stub authority.
- [ ] `BufferedReader.readExact(n)` correctly handles cases where bytes arrive across multiple `data` events.
- [ ] `readNextFrame` correctly identifies CRC mismatch without dropping the connection.
- [ ] `readNextFrame` drops the connection on unknown codec ID and logs the structured warn line.
- [ ] All paths that write to the socket use a single point of ACK emission so Phase 2 can later interpose a write queue without rewriting framing code.

## Risks / open questions

- `BufferedReader` correctness is critical. Use a battle-tested approach — a queue of pending reads with a backing `Buffer.concat` accumulator. Alternatively use Node's `node:stream/web` async iterator if the ergonomics fit.
- The `await ctx.publish(p)` inside the handler is the boundary where Phase 1's "TCP handler never blocks on downstream" property is enforced. The publish must use a non-blocking strategy (fire-and-forget into a bounded queue, or guarantee Redis publish is fast enough). Task 1.8 specifies the publish strategy; this task only needs to make the `await` semantically correct.

## Done

(Fill in once complete.)