julian
be48da9baa
src/observability/metrics.ts — full prom-client implementation. All 10
Phase 1 metrics registered (processor_consumer_reads_total,
_records_total, _lag, _decode_errors_total, processor_position_writes_total
{status}, _write_duration_seconds, processor_acks_total,
processor_device_state_{size,evictions_total}) plus nodejs_* defaults.
node:http server with /metrics, /healthz, /readyz. /readyz checks
redis.status === 'ready' AND a 5s-cached SELECT 1 Postgres probe.
processor_consumer_lag sampled every 10s via XINFO GROUPS, falling back
to a no-op when the consumer group hasn't been created yet.
src/main.ts — replaces the trace-logging shim with createMetrics() and
startMetricsServer(); shutdown closes the metrics server before
redis.quit() and pool.end().
test/metrics.test.ts — 22 unit tests: exposition format, every metric
type behaviour, all four HTTP endpoint paths including /readyz 503 cases.
test/pipeline.integration.test.ts — testcontainers Redis 7 +
TimescaleDB latest-pg16. Four scenarios: happy path with bigint+Buffer
attribute round-trip, idempotency on (device_id, ts), malformed payload
stays in PEL (decode_errors_total increments), writer failure → retry
(weaker variant per spec: stop Postgres before publish, restart, verify
row appears). Skip-on-no-Docker pattern verified — exits 0 without
Docker.
Dockerfile — multi-stage matching tcp-ingestion. EXPOSE 9090 only,
HEALTHCHECK on /readyz, image-source label points at processor repo.
.gitea/workflows/build.yml — single-job workflow mirroring
tcp-ingestion. Path filters cover src/, test/, build config, Dockerfile.
Portainer webhook step uncommented for :main auto-deploy.
compose.dev.yaml — local-build variant with Redis + TimescaleDB +
processor-dev for verifying Dockerfile changes without the registry
round-trip.
README.md — fleshed out from stub: quick-start, Docker build, deployment
note, env vars, tests (unit vs. integration), CI behavior. Flags the
deploy-side change needed: deploy/compose.yaml needs a TimescaleDB
service and a processor service entry added.
Verification: typecheck, lint clean; 134 unit tests passing across 8
files (+22 from this batch). pnpm test:integration runs cleanly under
the no-Docker skip pattern.
Phase 1 is now complete. Service is pilot-ready.
69 lines
1.6 KiB
YAML
69 lines
1.6 KiB
YAML
name: Build and Push processor
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths:
|
|
- 'src/**'
|
|
- 'test/**'
|
|
- 'package.json'
|
|
- 'pnpm-lock.yaml'
|
|
- 'tsconfig.json'
|
|
- 'vitest.config.ts'
|
|
- 'vitest.integration.config.ts'
|
|
- 'eslint.config.js'
|
|
- 'Dockerfile'
|
|
- '.dockerignore'
|
|
- '.gitea/workflows/build.yml'
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Set up Node 22
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
|
|
- name: Enable pnpm
|
|
run: corepack enable && corepack prepare pnpm@latest-9 --activate
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Typecheck
|
|
run: pnpm typecheck
|
|
|
|
- name: Lint
|
|
run: pnpm lint
|
|
|
|
- name: Test
|
|
run: pnpm test
|
|
|
|
- name: Set up Docker Buildx
|
|
uses: docker/setup-buildx-action@v3
|
|
with:
|
|
driver: docker-container
|
|
|
|
- name: Login to Gitea Registry
|
|
uses: docker/login-action@v3
|
|
with:
|
|
registry: git.dev.microservices.al
|
|
username: ${{ secrets.REGISTRY_USERNAME }}
|
|
password: ${{ secrets.REGISTRY_PASSWORD }}
|
|
|
|
- name: Build and Push
|
|
uses: docker/build-push-action@v5
|
|
with:
|
|
context: .
|
|
push: true
|
|
tags: git.dev.microservices.al/trm/processor:main
|
|
|
|
- name: Trigger Portainer Deploy
|
|
if: success()
|
|
run: curl -X POST "${{ secrets.PORTAINER_WEBHOOK_URL }}"
|