Add initial documentation for Agent Memory Index and TCP Ingestion Project
This commit is contained in:
@@ -0,0 +1,4 @@
|
||||
# Agent Memory Index
|
||||
|
||||
- [TCP Ingestion Project](project_tcp_ingestion.md) — tcp-ingestion service: tech stack, conventions, ESLint quirks, CRC details
|
||||
- [User Profile](user_profile.md) — User role and working style
|
||||
@@ -0,0 +1,91 @@
|
||||
---
|
||||
name: TCP Ingestion Project Conventions
|
||||
description: Tech stack, architecture patterns, ESLint quirks, and key decisions for the tcp-ingestion service at C:\Users\Administrator\projects\trm\tcp-ingestion
|
||||
type: project
|
||||
---
|
||||
|
||||
# TCP Ingestion Project
|
||||
|
||||
**Why:** TRM platform Teltonika GPS ingestion service — Node 22+ ESM, TypeScript strict, pnpm, vitest, pino, prom-client, ioredis, zod.
|
||||
|
||||
## Key conventions
|
||||
|
||||
- **ESM-only**: all imports use `.js` suffix on relative imports (Node ESM resolution, even for `.ts` source files)
|
||||
- **No bundler**: `tsc` only, output to `dist/`, runtime is plain `node dist/main.js`
|
||||
- **Strict TS**: `strict: true`, `noUncheckedIndexedAccess: true` in tsconfig.json (build); tsconfig.test.json (standalone, no extends, for ESLint + typecheck of tests)
|
||||
- **pnpm** workspace — `pnpm-lock.yaml` must be committed
|
||||
|
||||
## ESLint flat config quirks (critical)
|
||||
|
||||
- ESLint uses `tsconfig.test.json` (standalone, not extending tsconfig.json which has `rootDir: src`)
|
||||
- `tsconfig.test.json` must NOT extend `tsconfig.json` — stand-alone config that includes both `src/**/*` and `test/**/*`
|
||||
- `import/no-restricted-paths` requires `eslint-import-resolver-typescript` + `settings["import/resolver"]` in the ESLint config to resolve `.js`-suffixed imports to `.ts` files
|
||||
- `basePath: __dirname` + relative zone paths (`"src/core"`, `"src/adapters"`) is the correct format for `no-restricted-paths` on Windows
|
||||
- The `join` import from `node:path` is needed in eslint.config.js for path construction
|
||||
|
||||
## Architecture
|
||||
|
||||
- `src/core/` — vendor-agnostic shell (types, registry, session, server, publish stub)
|
||||
- `src/adapters/teltonika/` — Teltonika adapter (handshake, frame, CRC, device-authority, codec registry)
|
||||
- `src/config/load.ts` — zod-validated env config, fail-fast on missing required vars
|
||||
- `src/observability/logger.ts` — pino logger, pretty in dev, JSON in prod/test
|
||||
- `src/main.ts` — wires everything, graceful SIGTERM/SIGINT shutdown
|
||||
|
||||
## CRC-16/IBM
|
||||
|
||||
- Polynomial `0xA001` (reflected 0x8005), init `0x0000`, no final XOR
|
||||
- CRC scope: from CodecID byte through N2 inclusive (the body field, DataFieldLength bytes)
|
||||
- CRC field in frame: 4 bytes, value in **lower 2 bytes** (bytes 2-3 of the 4-byte field)
|
||||
- Canonical test vector: Codec 8 example body (54 bytes) → `0xC7CF`
|
||||
|
||||
## IMEI handshake
|
||||
|
||||
- Wire format: `[2B length BE][IMEI ASCII]`; server responds `0x01` (accept) or `0x00` (reject)
|
||||
- `readImeiHandshake` ONLY reads — does NOT write; handshake accept/reject is in the adapter session loop after `DeviceAuthority.check()`
|
||||
- Valid IMEI: `/^\d{14,16}$/`, max length 32 bytes before rejection
|
||||
|
||||
## Frame loop
|
||||
|
||||
- `BufferedReader` — single pending read at a time, accumulates chunks, no concurrent reads needed
|
||||
- Frame: preamble (4B=0) + DataFieldLength (4B) + body (DataFieldLength B) + CRC field (4B)
|
||||
- CRC mismatch: log warn, do NOT ACK, keep connection open (device retransmits)
|
||||
- Unknown codec: log warn, destroy socket, no ACK
|
||||
- N1≠N2: `FrameDropError(n1_n2_mismatch)`, destroy socket
|
||||
|
||||
## Codec parsers (tasks 1.5–1.7)
|
||||
|
||||
- Codec 8: `src/adapters/teltonika/codec/data/codec8.ts` — 1B IO IDs, 1B counts, no NX, no Generation Type
|
||||
- Codec 8E: `src/adapters/teltonika/codec/data/codec8e.ts` — 2B IO IDs, 2B counts, NX variable-length (Buffer, zero-copy subarray)
|
||||
- Codec 16: `src/adapters/teltonika/codec/data/codec16.ts` — MIXED: 2B IO IDs, 1B counts, Generation Type 1B, no NX
|
||||
- Shared GPS helpers: `src/adapters/teltonika/codec/data/gps-element.ts`
|
||||
- All three handlers registered by default in `createTeltonikaAdapter` (no custom registry needed)
|
||||
- Special attributes: `__event` (all codecs), `__generation_type` (codec 16 only)
|
||||
- Codec 16 mixed-widths trap: counts=1B (like C8), IO IDs=2B (like C8E) — easily confused
|
||||
|
||||
## Fixture suite (task 1.9)
|
||||
|
||||
- Fixtures: `test/fixtures/teltonika/codec8/`, `codec8e/`, `codec16/`
|
||||
- Format: `.hex` = body only (CodecID+N1+records+N2, NO preamble/length/CRC), `.expected.json` = Position[] output
|
||||
- Sentinels in JSON: `"__bigint:<decimal>"` → bigint, `"__buffer_b64:<base64>"` → Buffer (empty: `"__buffer_b64:"`)
|
||||
- Loader: `test/fixtures/_loader.ts` — `loadFixturesFromDir()` for auto-discovery, `compareToExpected()` for bigint/Buffer-aware equality
|
||||
- Auto-discovery: adding a `.hex` + `.expected.json` pair = new test with NO test file edits
|
||||
|
||||
## Redis publisher (task 1.8)
|
||||
|
||||
- `src/core/publish.ts` — `createPublisher(redis, config, logger, metrics): Publisher` + `connectRedis(url, logger, maxAttempts)`
|
||||
- Publisher: bounded in-memory queue (PUBLISH_QUEUE_CAPACITY, default 10,000), single worker, XADD with approximate MAXLEN trim
|
||||
- `Publisher.publish(position, codec)` is non-blocking (enqueue only); throws `PublishOverflowError` on overflow
|
||||
- `Publisher.drain(timeoutMs)` for graceful shutdown
|
||||
- **Buffer.toJSON() trap**: `JSON.stringify` with replacer calls `Buffer.prototype.toJSON()` first, converting Buffer to `{type:'Buffer',data:[...]}` before the replacer sees it. Solution: detect `instanceof Uint8Array` (direct calls) AND the `{type:'Buffer',data:[]}` shape (JSON.stringify path) in `jsonReplacer`.
|
||||
- `CodecLabel` type (`'8'|'8E'|'16'`) is in `publish.ts`; `AdapterContext.publish` was updated to `(position, codec) => Promise<void>`
|
||||
- Codec label injection: framing layer (`index.ts`) wraps `ctx.publish(pos)` into `(pos) => ctx.publish(pos, codecLabel)` at handler dispatch; codec parsers remain unchanged
|
||||
- `testcontainers` added as devDependency for Redis integration tests
|
||||
|
||||
## Test patterns
|
||||
|
||||
- Mock sockets are `EventEmitter` + `vi.fn()` methods — NOT real `net.Socket` instances
|
||||
- Use `vi.waitFor()` with timeout for async socket writes
|
||||
- Session tests need `{ timeout: 10_000 }` due to async socket lifecycle
|
||||
- Codec tests: use `makeTestCtx` with a collected `positions[]` array + mock logger
|
||||
|
||||
**How to apply:** Always check these patterns first before introducing new conventions. The ESLint/TypeScript resolver setup is particularly non-obvious.
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
name: User Profile
|
||||
description: Julian Cuni — role, preferences, and working style
|
||||
type: user
|
||||
---
|
||||
|
||||
# User Profile
|
||||
|
||||
**Name:** Julian Cuni (juliancuni@gmail.com)
|
||||
|
||||
**Working style:** Provides thorough planning docs with goals, deliverables, specs, and acceptance criteria. Wants incremental implementation with review checkpoints — explicit "do NOT proceed to next batch" instructions. Planning docs live in `.planning/` and are self-sufficient per task.
|
||||
|
||||
**Project context:** Building TRM platform — GPS tracking system with Teltonika hardware ingestion. Codebase uses a wiki at `../docs/wiki/` for architectural specs and `../docs/raw/` for canonical vendor protocol docs.
|
||||
|
||||
**Preferences:**
|
||||
- Terse final reports — surface decisions and surprises, don't narrate every step
|
||||
- No commits — user reviews and commits themselves
|
||||
- Acceptance criteria checklists in reports, quoting the spec bullets
|
||||
- Deviations from plan documented in both report and the task file's "Done/Risks" section
|
||||
Reference in New Issue
Block a user