92 lines
5.8 KiB
Markdown
92 lines
5.8 KiB
Markdown
---
|
||
name: TCP Ingestion Project Conventions
|
||
description: Tech stack, architecture patterns, ESLint quirks, and key decisions for the tcp-ingestion service at C:\Users\Administrator\projects\trm\tcp-ingestion
|
||
type: project
|
||
---
|
||
|
||
# TCP Ingestion Project
|
||
|
||
**Why:** TRM platform Teltonika GPS ingestion service — Node 22+ ESM, TypeScript strict, pnpm, vitest, pino, prom-client, ioredis, zod.
|
||
|
||
## Key conventions
|
||
|
||
- **ESM-only**: all imports use `.js` suffix on relative imports (Node ESM resolution, even for `.ts` source files)
|
||
- **No bundler**: `tsc` only, output to `dist/`, runtime is plain `node dist/main.js`
|
||
- **Strict TS**: `strict: true`, `noUncheckedIndexedAccess: true` in tsconfig.json (build); tsconfig.test.json (standalone, no extends, for ESLint + typecheck of tests)
|
||
- **pnpm** workspace — `pnpm-lock.yaml` must be committed
|
||
|
||
## ESLint flat config quirks (critical)
|
||
|
||
- ESLint uses `tsconfig.test.json` (standalone, not extending tsconfig.json which has `rootDir: src`)
|
||
- `tsconfig.test.json` must NOT extend `tsconfig.json` — stand-alone config that includes both `src/**/*` and `test/**/*`
|
||
- `import/no-restricted-paths` requires `eslint-import-resolver-typescript` + `settings["import/resolver"]` in the ESLint config to resolve `.js`-suffixed imports to `.ts` files
|
||
- `basePath: __dirname` + relative zone paths (`"src/core"`, `"src/adapters"`) is the correct format for `no-restricted-paths` on Windows
|
||
- The `join` import from `node:path` is needed in eslint.config.js for path construction
|
||
|
||
## Architecture
|
||
|
||
- `src/core/` — vendor-agnostic shell (types, registry, session, server, publish stub)
|
||
- `src/adapters/teltonika/` — Teltonika adapter (handshake, frame, CRC, device-authority, codec registry)
|
||
- `src/config/load.ts` — zod-validated env config, fail-fast on missing required vars
|
||
- `src/observability/logger.ts` — pino logger, pretty in dev, JSON in prod/test
|
||
- `src/main.ts` — wires everything, graceful SIGTERM/SIGINT shutdown
|
||
|
||
## CRC-16/IBM
|
||
|
||
- Polynomial `0xA001` (reflected 0x8005), init `0x0000`, no final XOR
|
||
- CRC scope: from CodecID byte through N2 inclusive (the body field, DataFieldLength bytes)
|
||
- CRC field in frame: 4 bytes, value in **lower 2 bytes** (bytes 2-3 of the 4-byte field)
|
||
- Canonical test vector: Codec 8 example body (54 bytes) → `0xC7CF`
|
||
|
||
## IMEI handshake
|
||
|
||
- Wire format: `[2B length BE][IMEI ASCII]`; server responds `0x01` (accept) or `0x00` (reject)
|
||
- `readImeiHandshake` ONLY reads — does NOT write; handshake accept/reject is in the adapter session loop after `DeviceAuthority.check()`
|
||
- Valid IMEI: `/^\d{14,16}$/`, max length 32 bytes before rejection
|
||
|
||
## Frame loop
|
||
|
||
- `BufferedReader` — single pending read at a time, accumulates chunks, no concurrent reads needed
|
||
- Frame: preamble (4B=0) + DataFieldLength (4B) + body (DataFieldLength B) + CRC field (4B)
|
||
- CRC mismatch: log warn, do NOT ACK, keep connection open (device retransmits)
|
||
- Unknown codec: log warn, destroy socket, no ACK
|
||
- N1≠N2: `FrameDropError(n1_n2_mismatch)`, destroy socket
|
||
|
||
## Codec parsers (tasks 1.5–1.7)
|
||
|
||
- Codec 8: `src/adapters/teltonika/codec/data/codec8.ts` — 1B IO IDs, 1B counts, no NX, no Generation Type
|
||
- Codec 8E: `src/adapters/teltonika/codec/data/codec8e.ts` — 2B IO IDs, 2B counts, NX variable-length (Buffer, zero-copy subarray)
|
||
- Codec 16: `src/adapters/teltonika/codec/data/codec16.ts` — MIXED: 2B IO IDs, 1B counts, Generation Type 1B, no NX
|
||
- Shared GPS helpers: `src/adapters/teltonika/codec/data/gps-element.ts`
|
||
- All three handlers registered by default in `createTeltonikaAdapter` (no custom registry needed)
|
||
- Special attributes: `__event` (all codecs), `__generation_type` (codec 16 only)
|
||
- Codec 16 mixed-widths trap: counts=1B (like C8), IO IDs=2B (like C8E) — easily confused
|
||
|
||
## Fixture suite (task 1.9)
|
||
|
||
- Fixtures: `test/fixtures/teltonika/codec8/`, `codec8e/`, `codec16/`
|
||
- Format: `.hex` = body only (CodecID+N1+records+N2, NO preamble/length/CRC), `.expected.json` = Position[] output
|
||
- Sentinels in JSON: `"__bigint:<decimal>"` → bigint, `"__buffer_b64:<base64>"` → Buffer (empty: `"__buffer_b64:"`)
|
||
- Loader: `test/fixtures/_loader.ts` — `loadFixturesFromDir()` for auto-discovery, `compareToExpected()` for bigint/Buffer-aware equality
|
||
- Auto-discovery: adding a `.hex` + `.expected.json` pair = new test with NO test file edits
|
||
|
||
## Redis publisher (task 1.8)
|
||
|
||
- `src/core/publish.ts` — `createPublisher(redis, config, logger, metrics): Publisher` + `connectRedis(url, logger, maxAttempts)`
|
||
- Publisher: bounded in-memory queue (PUBLISH_QUEUE_CAPACITY, default 10,000), single worker, XADD with approximate MAXLEN trim
|
||
- `Publisher.publish(position, codec)` is non-blocking (enqueue only); throws `PublishOverflowError` on overflow
|
||
- `Publisher.drain(timeoutMs)` for graceful shutdown
|
||
- **Buffer.toJSON() trap**: `JSON.stringify` with replacer calls `Buffer.prototype.toJSON()` first, converting Buffer to `{type:'Buffer',data:[...]}` before the replacer sees it. Solution: detect `instanceof Uint8Array` (direct calls) AND the `{type:'Buffer',data:[]}` shape (JSON.stringify path) in `jsonReplacer`.
|
||
- `CodecLabel` type (`'8'|'8E'|'16'`) is in `publish.ts`; `AdapterContext.publish` was updated to `(position, codec) => Promise<void>`
|
||
- Codec label injection: framing layer (`index.ts`) wraps `ctx.publish(pos)` into `(pos) => ctx.publish(pos, codecLabel)` at handler dispatch; codec parsers remain unchanged
|
||
- `testcontainers` added as devDependency for Redis integration tests
|
||
|
||
## Test patterns
|
||
|
||
- Mock sockets are `EventEmitter` + `vi.fn()` methods — NOT real `net.Socket` instances
|
||
- Use `vi.waitFor()` with timeout for async socket writes
|
||
- Session tests need `{ timeout: 10_000 }` due to async socket lifecycle
|
||
- Codec tests: use `makeTestCtx` with a collected `positions[]` array + mock logger
|
||
|
||
**How to apply:** Always check these patterns first before introducing new conventions. The ESLint/TypeScript resolver setup is particularly non-obvious.
|