Files

T

julian c314ba0902 Add planning documents for Phase 1 (throughput pipeline) and stub Phases 2-4

ROADMAP.md establishes status legend, architectural anchors pointing at the
wiki, and seven non-negotiable design rules — most importantly the
core/domain boundary that protects Phase 1 from Phase 2 churn, the
schema-authority split (positions hypertable owned here; everything else
owned by Directus), and idempotent-writes via (device_id, ts) ON CONFLICT.

Phase 1 (throughput pipeline) is fully detailed across 11 task files:
scaffold, core types + sentinel decoder, config + logging, Postgres
hypertable, Redis Stream consumer, per-device LRU state, batched writer,
main wiring, observability, integration test, Dockerfile + Gitea CI.
Observability is in Phase 1 (not deferred) — lesson learned from
tcp-ingestion task 1.10.

Phases 2-4 are stub READMEs. Phase 2 (domain logic) blocks on Directus
schema decisions and lists those open questions explicitly. Phase 3
(production hardening) and Phase 4 (future) sketch the task shape.

2026-04-30 21:16:59 +02:00

4.1 KiB

Raw Blame History

Task 1.1 — Project scaffold

Phase: 1 — Throughput pipeline Status: ⬜ Not started Depends on: None Wiki refs: docs/wiki/entities/processor.md

Goal

Initialize the Node.js / TypeScript project with the directory layout from the Phase 1 README, install the agreed tooling, and produce a minimal main.ts that the rest of Phase 1 builds on. Mirror the tcp-ingestion scaffold conventions exactly so the two services feel uniform.

Deliverables

package.json declaring:
- "type": "module" (ESM only).
- "engines": { "node": ">=22" }.
- Scripts: build, dev, start, test, test:watch, test:integration, lint, format, typecheck.
- Dependencies: ioredis, pg, pino, prom-client, zod.
- Dev dependencies: typescript, @types/node, @types/pg, vitest, @vitest/coverage-v8, eslint, @typescript-eslint/parser, @typescript-eslint/eslint-plugin, eslint-plugin-import, eslint-import-resolver-typescript, prettier, pino-pretty, tsx, testcontainers.
tsconfig.json — same as tcp-ingestion: strict: true, target: ES2022, module: NodeNext, moduleResolution: NodeNext, outDir: dist, rootDir: src, noUncheckedIndexedAccess: true.
eslint.config.js (flat config) with @typescript-eslint/recommended-type-checked, @typescript-eslint/no-floating-promises, @typescript-eslint/no-misused-promises. Add import/no-restricted-paths enforcing src/core/ cannot import from src/domain/. (src/domain/ doesn't exist yet — the rule is preemptive so Phase 2 can't violate the boundary by accident.)
.prettierrc — match tcp-ingestion (2 spaces, single quotes, semis).
.gitignore — node_modules/, dist/, coverage/, .env, .env.local, *.log.
.dockerignore — same as .gitignore plus .git/, .planning/, test/, *.md except README.md.
vitest.config.ts — unit-test config that excludes *.integration.test.ts.
vitest.integration.config.ts — integration-test config with hookTimeout: 120_000, testTimeout: 60_000. Include only *.integration.test.ts.
.env.example documenting every env var (with descriptions and defaults). Required vars only: REDIS_URL, POSTGRES_URL. All others have sensible defaults.
Empty directories with .gitkeep files where Phase 1 will fill them in:
- src/core/, src/db/migrations/, src/config/, src/observability/
- test/
src/main.ts — minimal stub: imports nothing yet, prints processor starting to stdout, exits with code 0.
README.md — short description pointing at .planning/ROADMAP.md for the work plan, and at ../docs/wiki/entities/processor.md for the architectural specification.

Specification

Package manager: pnpm. Commit pnpm-lock.yaml. The Dockerfile (task 1.11) will use pnpm fetch for layer-cache friendliness.
Module style: ESM throughout. Relative imports use .js suffix per Node ESM resolution. No paths aliases.
No bundler. Build is tsc only. Runtime is plain Node consuming dist/.
Linting style: Configure ESLint to enforce no-floating-promises and no-misused-promises — both critical in a stream consumer where unhandled rejection silently loses work.

Acceptance criteria

pnpm install succeeds with no warnings other than peer deps.
pnpm typecheck succeeds on the empty project.
pnpm lint succeeds.
pnpm build produces dist/main.js.
pnpm start runs the compiled output and prints the startup message.
pnpm test runs (with no tests) and exits successfully.
pnpm dev runs main.ts via tsx and prints the startup message.
Repository builds reproducibly: deleting node_modules and dist, then pnpm install --frozen-lockfile && pnpm build produces identical output.

Risks / open questions

The import/no-restricted-paths rule is preemptive and will be silently inert until Phase 2 introduces src/domain/. Verify it activates correctly with a temporary src/domain/foo.ts during scaffold setup, then remove.

Done

(Fill in once complete: commit SHA, brief notes.)

4.1 KiB Raw Blame History