julian
155f034f61
The Redis-publisher integration test uses testcontainers to spin up a real Redis. On the Gitea CI runner, `container.start()` hangs (likely image-pull delay or restricted Docker access), and the 60s beforeAll timeout fails the suite even though both tests ultimately would skip. The skip-on-error path only fires when start() throws, not when it times out. Fix: separate unit tests (default) from integration tests (opt-in). The default `pnpm test` now runs only `test/**/*.test.ts` excluding `*.integration.test.ts`. A new `pnpm test:integration` script runs them via `vitest.integration.config.ts` with generous hook/test timeouts for container startup. CI runs `pnpm test` and is unaffected by Docker availability. Integration tests can be run locally or in a future CI job that explicitly provisions Docker.
85 lines
3.8 KiB
Markdown
85 lines
3.8 KiB
Markdown
# tcp-ingestion
|
|
|
|
Node.js TCP server that accepts persistent connections from Teltonika GPS hardware (FMB/FMC/FMM/FMU series), parses Codec 8, 8E, and 16 AVL frames, and publishes normalized `Position` records to a Redis Stream for downstream consumers.
|
|
|
|
For the full architectural specification see `../docs/wiki/`. For the work plan and task status see `.planning/ROADMAP.md`.
|
|
|
|
---
|
|
|
|
## Quick start (local)
|
|
|
|
**Prerequisites:** Node.js 22+, pnpm, a local Redis instance (or use compose below).
|
|
|
|
```bash
|
|
git clone <repo-url>
|
|
cd tcp-ingestion
|
|
pnpm install
|
|
cp .env.example .env
|
|
# Edit .env — at minimum set REDIS_URL
|
|
pnpm dev
|
|
```
|
|
|
|
`pnpm dev` uses `tsx watch` for hot-reload during development. The server listens on `TELTONIKA_PORT` (default `5027`).
|
|
|
|
---
|
|
|
|
## Test the Docker build locally
|
|
|
|
`compose.dev.yaml` builds the image from source and runs it next to a Redis container. Useful for verifying Dockerfile changes before pushing:
|
|
|
|
```bash
|
|
docker compose -f compose.dev.yaml up --build
|
|
```
|
|
|
|
For day-to-day development, prefer `pnpm dev` directly — it has hot reload and faster iteration.
|
|
|
|
---
|
|
|
|
## Production / stage deployment
|
|
|
|
This service is **not** deployed standalone. It runs as part of the platform stack defined in the [`deploy/`](https://git.dev.microservices.al/trm/deploy) repo, which Portainer pulls and runs on the stage and production hosts.
|
|
|
|
The image itself is published to `git.dev.microservices.al/trm/tcp-ingestion:main` on every push to `main` (see CI behavior below). The `deploy/` repo's `compose.yaml` references that image; updates flow through there, not through this repo.
|
|
|
|
To pin a specific commit in production, set `TCP_INGESTION_TAG=<sha>` in the deploy stack's environment variables.
|
|
|
|
---
|
|
|
|
## Environment variables
|
|
|
|
See `.env.example` for all variables with descriptions and defaults. The only required variable is `REDIS_URL` — all others have sensible defaults.
|
|
|
|
---
|
|
|
|
## Tests
|
|
|
|
- `pnpm test` — unit tests only. Fast (~2s), no external dependencies. **This is what CI runs.**
|
|
- `pnpm test:integration` — integration tests that need Docker (testcontainers spins up a real Redis). **Opt-in.** Run locally before changes to the Redis publisher, or in a separate CI job with Docker access.
|
|
|
|
Integration tests live in `test/**/*.integration.test.ts` and are excluded from the default run by `vitest.config.ts`.
|
|
|
|
---
|
|
|
|
## CI behavior
|
|
|
|
Gitea Actions workflow is at `.gitea/workflows/build.yml`.
|
|
|
|
- **Push to `main`** (only when `src/`, `test/`, build config, Dockerfile, or the workflow file itself changes): runs `typecheck`, `lint`, `test` (unit tests only), then builds and pushes the Docker image tagged `:main`. Auto-deploys to stage if a Portainer webhook is configured.
|
|
- **Manual trigger** (`workflow_dispatch`): same flow, run on demand.
|
|
|
|
Integration tests are not run in CI — they need Docker access on the runner, which we don't currently configure. Run them locally as needed.
|
|
|
|
The workflow uses `secrets.REGISTRY_USERNAME` and `secrets.REGISTRY_PASSWORD` for the Gitea registry login — these must be configured in the repo's (or org's) Actions secrets.
|
|
|
|
---
|
|
|
|
## Pilot deployment notes
|
|
|
|
This service is running in pilot form. The following tasks are **paused** — they are not missing by accident, they are deferred by design to get onto real Teltonika hardware first:
|
|
|
|
- **Observability (task 1.10):** No `/metrics`, `/healthz`, or `/readyz` HTTP endpoints exist yet. `METRICS_PORT` is in the config schema but nothing listens on it. The Docker `HEALTHCHECK` is also absent for this reason.
|
|
- **Production hardening (task 1.12):** Graceful shutdown is a functional stub; uncaught-exception handling is minimal.
|
|
- **Device authority (task 1.13):** `AllowAllAuthority` is active — every IMEI is accepted. `STRICT_DEVICE_AUTH=true` is wired but the Redis allow-list refresher is not yet implemented.
|
|
|
|
See `.planning/ROADMAP.md` for the resume triggers for each deferred task.
|