julian
c314ba0902
ROADMAP.md establishes status legend, architectural anchors pointing at the wiki, and seven non-negotiable design rules — most importantly the core/domain boundary that protects Phase 1 from Phase 2 churn, the schema-authority split (positions hypertable owned here; everything else owned by Directus), and idempotent-writes via (device_id, ts) ON CONFLICT. Phase 1 (throughput pipeline) is fully detailed across 11 task files: scaffold, core types + sentinel decoder, config + logging, Postgres hypertable, Redis Stream consumer, per-device LRU state, batched writer, main wiring, observability, integration test, Dockerfile + Gitea CI. Observability is in Phase 1 (not deferred) — lesson learned from tcp-ingestion task 1.10. Phases 2-4 are stub READMEs. Phase 2 (domain logic) blocks on Directus schema decisions and lists those open questions explicitly. Phase 3 (production hardening) and Phase 4 (future) sketch the task shape.
87 lines
5.0 KiB
Markdown
87 lines
5.0 KiB
Markdown
# Task 1.11 — Dockerfile & Gitea workflow
|
|
|
|
**Phase:** 1 — Throughput pipeline
|
|
**Status:** ⬜ Not started
|
|
**Depends on:** 1.10
|
|
**Wiki refs:** —
|
|
|
|
## Goal
|
|
|
|
Containerize the service and add the Gitea Actions workflow that builds and publishes `git.dev.microservices.al/trm/processor:main` on every push to `main`. Mirror `tcp-ingestion`'s slim variant — same multi-stage Dockerfile, same single-job workflow with path filters.
|
|
|
|
## Deliverables
|
|
|
|
- `Dockerfile` — multi-stage: deps → build → runtime. Match `tcp-ingestion/Dockerfile` line for line, adjusting only:
|
|
- `EXPOSE 9090` (only — Processor has no TCP listener).
|
|
- `HEALTHCHECK` pointing at `/readyz` on `${METRICS_PORT}`.
|
|
- `CMD ["node", "dist/main.js"]`.
|
|
- `.gitea/workflows/build.yml` — single-job workflow matching `tcp-ingestion/.gitea/workflows/build.yml`:
|
|
- Trigger: `push` to `main` (path filters: `src/`, `test/`, `package.json`, `pnpm-lock.yaml`, `tsconfig.json`, `Dockerfile`, `.gitea/workflows/build.yml`) + `workflow_dispatch`.
|
|
- Steps: checkout, setup-node@v4 (Node 22, pnpm), install, typecheck, lint, test (unit only), docker buildx build-push to `git.dev.microservices.al/trm/processor:main`.
|
|
- Uses `secrets.REGISTRY_USERNAME` / `secrets.REGISTRY_PASSWORD`.
|
|
- Final step: trigger Portainer webhook on success (uncommented; same as `tcp-ingestion` after the `:main` -> webhook auto-deploy got working).
|
|
- `compose.dev.yaml` — local-build variant with `build: .`, named `processor-dev`, depends on a Redis service and a TimescaleDB service. Useful for verifying Dockerfile changes without the registry round-trip.
|
|
- `README.md` (the repo-level one, already a stub) — flesh out with:
|
|
- Quick-start (local: `pnpm install && cp .env.example .env && pnpm dev`).
|
|
- "Run the Docker build locally" section (`docker compose -f compose.dev.yaml up --build`).
|
|
- Production-deployment note: image is pulled by the `deploy/` repo's stack; do not run standalone.
|
|
- Pin to a specific commit via `PROCESSOR_TAG=<sha>` in the deploy stack.
|
|
- Tests section (unit vs. integration).
|
|
- CI behavior summary.
|
|
- "Pilot deployment notes" section if anything is paused (Phase 1 has nothing paused — note this and remove the section if so).
|
|
|
|
## Specification
|
|
|
|
### Dockerfile parity with `tcp-ingestion`
|
|
|
|
Open `tcp-ingestion/Dockerfile` and copy structure verbatim. The only diffs from a Phase 1 Processor are:
|
|
- No `EXPOSE 5027` — there's no TCP listener.
|
|
- `HEALTHCHECK` URL path is `/readyz` (already true for `tcp-ingestion`).
|
|
- Image label: `org.opencontainers.image.source` should point to the `processor` repo URL.
|
|
|
|
This parity matters: when a future engineer needs to debug a build, having two services build the same way reduces cognitive load.
|
|
|
|
### Workflow parity with `tcp-ingestion`
|
|
|
|
Same. Open `tcp-ingestion/.gitea/workflows/build.yml`, copy, change image name and (if needed) path filters. The webhook step at the end should be uncommented so `:main` builds auto-deploy through Portainer.
|
|
|
|
### Stage deploy
|
|
|
|
Phase 1 ships ready to land in the `deploy/compose.yaml` (`trm/deploy` repo) as a new service. **Do not edit `deploy/compose.yaml` from this task.** Surface it in the final report: "Add `processor` service to `deploy/compose.yaml` with image, env, depends_on Redis + Postgres." That is a deploy-side change, made by the user.
|
|
|
|
The `deploy/compose.yaml`'s service block will look roughly like:
|
|
|
|
```yaml
|
|
processor:
|
|
image: git.dev.microservices.al/trm/processor:${PROCESSOR_TAG:-main}
|
|
depends_on:
|
|
redis: { condition: service_healthy }
|
|
postgres: { condition: service_healthy }
|
|
environment:
|
|
NODE_ENV: production
|
|
INSTANCE_ID: ${PROCESSOR_INSTANCE_ID:-processor-1}
|
|
REDIS_URL: redis://redis:6379
|
|
POSTGRES_URL: postgres://...
|
|
LOG_LEVEL: ${LOG_LEVEL:-info}
|
|
restart: unless-stopped
|
|
```
|
|
|
|
Plus a Postgres service (TimescaleDB image) added to the stack — the stack currently only has Redis + tcp-ingestion. That's the user's deploy decision to make.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] `docker build .` succeeds locally; resulting image runs and exposes `/healthz` on 9090.
|
|
- [ ] `docker compose -f compose.dev.yaml up --build` boots Redis + TimescaleDB + Processor; `/readyz` reports 200 once everything is up.
|
|
- [ ] Pushing to `main` (or hitting `workflow_dispatch`) builds the image, runs typecheck/lint/test, and pushes `:main` to the registry.
|
|
- [ ] Portainer webhook fires on successful push and the stage stack picks up the new image (assuming the `deploy/` stack is set up).
|
|
- [ ] Image size is reasonable (target < 250 MB final stage; the `tcp-ingestion` slim variant lands around there).
|
|
|
|
## Risks / open questions
|
|
|
|
- **Re-pull on stack redeploy.** The same Portainer issue we hit with `tcp-ingestion` (stack redeploy doesn't pull new images by default) will apply here. Make sure the same fix is in place ("Re-pull image" toggle, or per-commit-SHA tags) before this lands. Cross-reference the `tcp-ingestion` deploy note in `deploy/README.md`.
|
|
- **HEALTHCHECK `wget` availability.** `node:22-alpine` includes `wget`. If we ever switch base image, revisit.
|
|
|
|
## Done
|
|
|
|
(Fill in once complete: commit SHA, brief notes.)
|