Implement Phase 1 tasks 1.1-1.4 (scaffold + core types + config + Postgres)
Scaffold mirrors tcp-ingestion conventions: ESM, strict TS, pnpm, vitest
with unit/integration split, ESLint flat config with no-floating-promises
+ no-misused-promises + import/no-restricted-paths (the new src/core/ →
src/domain/ boundary that protects Phase 1 from Phase 2 churn).
Core types in src/core/types.ts (Position, StreamRecord, DeviceState,
Metrics, AttributeValue) — Position is byte-equivalent to tcp-ingestion's
output. Codec in src/core/codec.ts implements sentinel reversal:
{__bigint:"..."} → bigint, {__buffer_b64:"..."} → Buffer, ISO timestamp
string → Date. CodecError surfaces malformed payload reasons with the
failing field named.
Config in src/config/load.ts (zod schema, all 13 env vars with defaults
and bounded numerics). Logger in src/observability/logger.ts matches
tcp-ingestion exactly: ISO timestamps, string level labels, pino-pretty
in development.
Postgres in src/db/: createPool with sane defaults and application_name,
connectWithRetry mirroring the ioredis retry pattern, a 30-line
migration runner using a schema_migrations table, and 0001_positions.sql
with the hypertable + (device_id, ts) unique index + ts DESC index.
Migration runner unit-tested against a mocked pg.Pool; the real
TimescaleDB round-trip is deferred to task 1.10 per spec.
Verification: typecheck, lint, build all clean; 73 unit tests passing
across 4 files. import/no-restricted-paths verified live by temporarily
adding a forbidden src/domain/ import.
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# Task 1.1 — Project scaffold
|
||||
|
||||
**Phase:** 1 — Throughput pipeline
|
||||
**Status:** ⬜ Not started
|
||||
**Status:** 🟩 Done
|
||||
**Depends on:** None
|
||||
**Wiki refs:** `docs/wiki/entities/processor.md`
|
||||
|
||||
@@ -55,4 +55,4 @@ Initialize the Node.js / TypeScript project with the directory layout from the P
|
||||
|
||||
## Done
|
||||
|
||||
(Fill in once complete: commit SHA, brief notes.)
|
||||
*(pending commit SHA)* — Scaffolded `package.json`, `tsconfig.json`, `tsconfig.test.json`, `eslint.config.js`, `.prettierrc`, `vitest.config.ts`, `vitest.integration.config.ts`, `.env.example`, `.gitignore`, `.dockerignore`, and `src/main.ts`. All tooling passes (`pnpm typecheck`, `pnpm lint`, `pnpm build`, `pnpm test`). Verified `import/no-restricted-paths` boundary rule fires on a temporary `src/core/` → `src/domain/` import. Divergence from tcp-ingestion: the restricted-paths zone targets `src/domain/` (Phase 2 boundary) instead of `src/adapters/` (tcp-ingestion boundary).
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Task 1.2 — Core types & contracts
|
||||
|
||||
**Phase:** 1 — Throughput pipeline
|
||||
**Status:** ⬜ Not started
|
||||
**Status:** 🟩 Done
|
||||
**Depends on:** 1.1
|
||||
**Wiki refs:** `docs/wiki/concepts/position-record.md`, `docs/wiki/concepts/io-element-bag.md`
|
||||
|
||||
@@ -63,4 +63,4 @@ Some Teltonika IO elements are u64 values that exceed `Number.MAX_SAFE_INTEGER`
|
||||
|
||||
## Done
|
||||
|
||||
(Fill in once complete: commit SHA, brief notes.)
|
||||
*(pending commit SHA)* — Implemented `src/core/types.ts` (Position, StreamRecord, DeviceState, Metrics, AttributeValue) and `src/core/codec.ts` (decodePosition, CodecError). All sentinel reversal rules implemented: `__bigint` → bigint, `__buffer_b64` → Buffer, timestamp ISO string → Date. 26 test cases in `test/codec.test.ts` covering round-trips, u64-max, non-UTF-8 bytes, all error paths. Judgment call: `AttributeValue` extracted as a named type alias (not inline) to aid readability in downstream tasks.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Task 1.3 — Configuration & logging
|
||||
|
||||
**Phase:** 1 — Throughput pipeline
|
||||
**Status:** ⬜ Not started
|
||||
**Status:** 🟩 Done
|
||||
**Depends on:** 1.1
|
||||
**Wiki refs:** `docs/wiki/entities/processor.md`
|
||||
|
||||
@@ -73,4 +73,4 @@ return pino({ level, base, timestamp: pino.stdTimeFunctions.isoTime, formatters
|
||||
|
||||
## Done
|
||||
|
||||
(Fill in once complete: commit SHA, brief notes.)
|
||||
*(pending commit SHA)* — Implemented `src/config/load.ts` (zod schema, loadConfig) and `src/observability/logger.ts` (createLogger, pino-pretty in dev). 37 config test cases covering all defaults, missing required vars, URL protocol validation, and bounded numeric checks. Wired into `src/main.ts`. Divergence from tcp-ingestion config: `INSTANCE_ID` defaults to a fixed `'processor-1'` string rather than a random UUID prefix; rationale: operator-visible name is more useful than randomness in a containerised environment where the instance name can be set deterministically.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Task 1.4 — Postgres connection & `positions` hypertable
|
||||
|
||||
**Phase:** 1 — Throughput pipeline
|
||||
**Status:** ⬜ Not started
|
||||
**Status:** 🟩 Done
|
||||
**Depends on:** 1.1, 1.3
|
||||
**Wiki refs:** `docs/wiki/entities/postgres-timescaledb.md`
|
||||
|
||||
@@ -86,4 +86,4 @@ Do **not** introduce a heavy framework (Knex, node-pg-migrate). The Processor ha
|
||||
|
||||
## Done
|
||||
|
||||
(Fill in once complete: commit SHA, brief notes.)
|
||||
*(pending commit SHA)* — Implemented `src/db/pool.ts` (createPool, connectWithRetry), `src/db/migrate.ts` (runMigrations — 30-line runner), and `src/db/migrations/0001_positions.sql` (hypertable + unique index + ts-desc index). Unit tests use a mocked pg.Pool throughout; the real TimescaleDB round-trip is deferred to task 1.10 per spec. The "calls process.exit(1)" pool test uses `maxAttempts=1` to avoid fake-timer unhandled-rejection noise that surfaces when a backoff setTimeout resolves after the outer promise has already thrown.
|
||||
|
||||
Reference in New Issue
Block a user