julian
c314ba0902
ROADMAP.md establishes status legend, architectural anchors pointing at the wiki, and seven non-negotiable design rules — most importantly the core/domain boundary that protects Phase 1 from Phase 2 churn, the schema-authority split (positions hypertable owned here; everything else owned by Directus), and idempotent-writes via (device_id, ts) ON CONFLICT. Phase 1 (throughput pipeline) is fully detailed across 11 task files: scaffold, core types + sentinel decoder, config + logging, Postgres hypertable, Redis Stream consumer, per-device LRU state, batched writer, main wiring, observability, integration test, Dockerfile + Gitea CI. Observability is in Phase 1 (not deferred) — lesson learned from tcp-ingestion task 1.10. Phases 2-4 are stub READMEs. Phase 2 (domain logic) blocks on Directus schema decisions and lists those open questions explicitly. Phase 3 (production hardening) and Phase 4 (future) sketch the task shape.
59 lines
4.1 KiB
Markdown
59 lines
4.1 KiB
Markdown
# Task 1.1 — Project scaffold
|
|
|
|
**Phase:** 1 — Throughput pipeline
|
|
**Status:** ⬜ Not started
|
|
**Depends on:** None
|
|
**Wiki refs:** `docs/wiki/entities/processor.md`
|
|
|
|
## Goal
|
|
|
|
Initialize the Node.js / TypeScript project with the directory layout from the Phase 1 README, install the agreed tooling, and produce a minimal `main.ts` that the rest of Phase 1 builds on. Mirror the `tcp-ingestion` scaffold conventions exactly so the two services feel uniform.
|
|
|
|
## Deliverables
|
|
|
|
- `package.json` declaring:
|
|
- `"type": "module"` (ESM only).
|
|
- `"engines": { "node": ">=22" }`.
|
|
- Scripts: `build`, `dev`, `start`, `test`, `test:watch`, `test:integration`, `lint`, `format`, `typecheck`.
|
|
- Dependencies: `ioredis`, `pg`, `pino`, `prom-client`, `zod`.
|
|
- Dev dependencies: `typescript`, `@types/node`, `@types/pg`, `vitest`, `@vitest/coverage-v8`, `eslint`, `@typescript-eslint/parser`, `@typescript-eslint/eslint-plugin`, `eslint-plugin-import`, `eslint-import-resolver-typescript`, `prettier`, `pino-pretty`, `tsx`, `testcontainers`.
|
|
- `tsconfig.json` — same as `tcp-ingestion`: `strict: true`, `target: ES2022`, `module: NodeNext`, `moduleResolution: NodeNext`, `outDir: dist`, `rootDir: src`, `noUncheckedIndexedAccess: true`.
|
|
- `eslint.config.js` (flat config) with `@typescript-eslint/recommended-type-checked`, `@typescript-eslint/no-floating-promises`, `@typescript-eslint/no-misused-promises`. Add `import/no-restricted-paths` enforcing **`src/core/` cannot import from `src/domain/`**. (`src/domain/` doesn't exist yet — the rule is preemptive so Phase 2 can't violate the boundary by accident.)
|
|
- `.prettierrc` — match `tcp-ingestion` (2 spaces, single quotes, semis).
|
|
- `.gitignore` — `node_modules/`, `dist/`, `coverage/`, `.env`, `.env.local`, `*.log`.
|
|
- `.dockerignore` — same as `.gitignore` plus `.git/`, `.planning/`, `test/`, `*.md` except `README.md`.
|
|
- `vitest.config.ts` — unit-test config that excludes `*.integration.test.ts`.
|
|
- `vitest.integration.config.ts` — integration-test config with `hookTimeout: 120_000`, `testTimeout: 60_000`. Include only `*.integration.test.ts`.
|
|
- `.env.example` documenting every env var (with descriptions and defaults). Required vars only: `REDIS_URL`, `POSTGRES_URL`. All others have sensible defaults.
|
|
- Empty directories with `.gitkeep` files where Phase 1 will fill them in:
|
|
- `src/core/`, `src/db/migrations/`, `src/config/`, `src/observability/`
|
|
- `test/`
|
|
- `src/main.ts` — minimal stub: imports nothing yet, prints `processor starting` to stdout, exits with code 0.
|
|
- `README.md` — short description pointing at `.planning/ROADMAP.md` for the work plan, and at `../docs/wiki/entities/processor.md` for the architectural specification.
|
|
|
|
## Specification
|
|
|
|
- **Package manager:** pnpm. Commit `pnpm-lock.yaml`. The Dockerfile (task 1.11) will use `pnpm fetch` for layer-cache friendliness.
|
|
- **Module style:** ESM throughout. Relative imports use `.js` suffix per Node ESM resolution. No `paths` aliases.
|
|
- **No bundler.** Build is `tsc` only. Runtime is plain Node consuming `dist/`.
|
|
- **Linting style:** Configure ESLint to enforce no-floating-promises and no-misused-promises — both critical in a stream consumer where unhandled rejection silently loses work.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] `pnpm install` succeeds with no warnings other than peer deps.
|
|
- [ ] `pnpm typecheck` succeeds on the empty project.
|
|
- [ ] `pnpm lint` succeeds.
|
|
- [ ] `pnpm build` produces `dist/main.js`.
|
|
- [ ] `pnpm start` runs the compiled output and prints the startup message.
|
|
- [ ] `pnpm test` runs (with no tests) and exits successfully.
|
|
- [ ] `pnpm dev` runs `main.ts` via `tsx` and prints the startup message.
|
|
- [ ] Repository builds reproducibly: deleting `node_modules` and `dist`, then `pnpm install --frozen-lockfile && pnpm build` produces identical output.
|
|
|
|
## Risks / open questions
|
|
|
|
- The `import/no-restricted-paths` rule is preemptive and will be silently inert until Phase 2 introduces `src/domain/`. Verify it activates correctly with a temporary `src/domain/foo.ts` during scaffold setup, then remove.
|
|
|
|
## Done
|
|
|
|
(Fill in once complete: commit SHA, brief notes.)
|