julian
95efc23139
Scaffold mirrors tcp-ingestion conventions: ESM, strict TS, pnpm, vitest
with unit/integration split, ESLint flat config with no-floating-promises
+ no-misused-promises + import/no-restricted-paths (the new src/core/ →
src/domain/ boundary that protects Phase 1 from Phase 2 churn).
Core types in src/core/types.ts (Position, StreamRecord, DeviceState,
Metrics, AttributeValue) — Position is byte-equivalent to tcp-ingestion's
output. Codec in src/core/codec.ts implements sentinel reversal:
{__bigint:"..."} → bigint, {__buffer_b64:"..."} → Buffer, ISO timestamp
string → Date. CodecError surfaces malformed payload reasons with the
failing field named.
Config in src/config/load.ts (zod schema, all 13 env vars with defaults
and bounded numerics). Logger in src/observability/logger.ts matches
tcp-ingestion exactly: ISO timestamps, string level labels, pino-pretty
in development.
Postgres in src/db/: createPool with sane defaults and application_name,
connectWithRetry mirroring the ioredis retry pattern, a 30-line
migration runner using a schema_migrations table, and 0001_positions.sql
with the hypertable + (device_id, ts) unique index + ts DESC index.
Migration runner unit-tested against a mocked pg.Pool; the real
TimescaleDB round-trip is deferred to task 1.10 per spec.
Verification: typecheck, lint, build all clean; 73 unit tests passing
across 4 files. import/no-restricted-paths verified live by temporarily
adding a forbidden src/domain/ import.
59 lines
4.6 KiB
Markdown
59 lines
4.6 KiB
Markdown
# Task 1.1 — Project scaffold
|
|
|
|
**Phase:** 1 — Throughput pipeline
|
|
**Status:** 🟩 Done
|
|
**Depends on:** None
|
|
**Wiki refs:** `docs/wiki/entities/processor.md`
|
|
|
|
## Goal
|
|
|
|
Initialize the Node.js / TypeScript project with the directory layout from the Phase 1 README, install the agreed tooling, and produce a minimal `main.ts` that the rest of Phase 1 builds on. Mirror the `tcp-ingestion` scaffold conventions exactly so the two services feel uniform.
|
|
|
|
## Deliverables
|
|
|
|
- `package.json` declaring:
|
|
- `"type": "module"` (ESM only).
|
|
- `"engines": { "node": ">=22" }`.
|
|
- Scripts: `build`, `dev`, `start`, `test`, `test:watch`, `test:integration`, `lint`, `format`, `typecheck`.
|
|
- Dependencies: `ioredis`, `pg`, `pino`, `prom-client`, `zod`.
|
|
- Dev dependencies: `typescript`, `@types/node`, `@types/pg`, `vitest`, `@vitest/coverage-v8`, `eslint`, `@typescript-eslint/parser`, `@typescript-eslint/eslint-plugin`, `eslint-plugin-import`, `eslint-import-resolver-typescript`, `prettier`, `pino-pretty`, `tsx`, `testcontainers`.
|
|
- `tsconfig.json` — same as `tcp-ingestion`: `strict: true`, `target: ES2022`, `module: NodeNext`, `moduleResolution: NodeNext`, `outDir: dist`, `rootDir: src`, `noUncheckedIndexedAccess: true`.
|
|
- `eslint.config.js` (flat config) with `@typescript-eslint/recommended-type-checked`, `@typescript-eslint/no-floating-promises`, `@typescript-eslint/no-misused-promises`. Add `import/no-restricted-paths` enforcing **`src/core/` cannot import from `src/domain/`**. (`src/domain/` doesn't exist yet — the rule is preemptive so Phase 2 can't violate the boundary by accident.)
|
|
- `.prettierrc` — match `tcp-ingestion` (2 spaces, single quotes, semis).
|
|
- `.gitignore` — `node_modules/`, `dist/`, `coverage/`, `.env`, `.env.local`, `*.log`.
|
|
- `.dockerignore` — same as `.gitignore` plus `.git/`, `.planning/`, `test/`, `*.md` except `README.md`.
|
|
- `vitest.config.ts` — unit-test config that excludes `*.integration.test.ts`.
|
|
- `vitest.integration.config.ts` — integration-test config with `hookTimeout: 120_000`, `testTimeout: 60_000`. Include only `*.integration.test.ts`.
|
|
- `.env.example` documenting every env var (with descriptions and defaults). Required vars only: `REDIS_URL`, `POSTGRES_URL`. All others have sensible defaults.
|
|
- Empty directories with `.gitkeep` files where Phase 1 will fill them in:
|
|
- `src/core/`, `src/db/migrations/`, `src/config/`, `src/observability/`
|
|
- `test/`
|
|
- `src/main.ts` — minimal stub: imports nothing yet, prints `processor starting` to stdout, exits with code 0.
|
|
- `README.md` — short description pointing at `.planning/ROADMAP.md` for the work plan, and at `../docs/wiki/entities/processor.md` for the architectural specification.
|
|
|
|
## Specification
|
|
|
|
- **Package manager:** pnpm. Commit `pnpm-lock.yaml`. The Dockerfile (task 1.11) will use `pnpm fetch` for layer-cache friendliness.
|
|
- **Module style:** ESM throughout. Relative imports use `.js` suffix per Node ESM resolution. No `paths` aliases.
|
|
- **No bundler.** Build is `tsc` only. Runtime is plain Node consuming `dist/`.
|
|
- **Linting style:** Configure ESLint to enforce no-floating-promises and no-misused-promises — both critical in a stream consumer where unhandled rejection silently loses work.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] `pnpm install` succeeds with no warnings other than peer deps.
|
|
- [ ] `pnpm typecheck` succeeds on the empty project.
|
|
- [ ] `pnpm lint` succeeds.
|
|
- [ ] `pnpm build` produces `dist/main.js`.
|
|
- [ ] `pnpm start` runs the compiled output and prints the startup message.
|
|
- [ ] `pnpm test` runs (with no tests) and exits successfully.
|
|
- [ ] `pnpm dev` runs `main.ts` via `tsx` and prints the startup message.
|
|
- [ ] Repository builds reproducibly: deleting `node_modules` and `dist`, then `pnpm install --frozen-lockfile && pnpm build` produces identical output.
|
|
|
|
## Risks / open questions
|
|
|
|
- The `import/no-restricted-paths` rule is preemptive and will be silently inert until Phase 2 introduces `src/domain/`. Verify it activates correctly with a temporary `src/domain/foo.ts` during scaffold setup, then remove.
|
|
|
|
## Done
|
|
|
|
*(pending commit SHA)* — Scaffolded `package.json`, `tsconfig.json`, `tsconfig.test.json`, `eslint.config.js`, `.prettierrc`, `vitest.config.ts`, `vitest.integration.config.ts`, `.env.example`, `.gitignore`, `.dockerignore`, and `src/main.ts`. All tooling passes (`pnpm typecheck`, `pnpm lint`, `pnpm build`, `pnpm test`). Verified `import/no-restricted-paths` boundary rule fires on a temporary `src/core/` → `src/domain/` import. Divergence from tcp-ingestion: the restricted-paths zone targets `src/domain/` (Phase 2 boundary) instead of `src/adapters/` (tcp-ingestion boundary).
|