deploy/README.md

# TRM Deploy

Deployment configuration for the TRM platform. This repo holds the multi-service `compose.yaml` and per-environment overrides; service code lives in sibling repos under `git.dev.microservices.al/trm/`.

## Layout

```
deploy/
├── compose.yaml         ← Portainer's Compose path
├── .env.example         ← documented variables; copy to .env locally
├── .gitignore
└── README.md
```

## Services in the stack

Currently:

- **redis** — telemetry queue + future Phase 2 connection registry. Internal-only, persisted via named volume.
- **tcp-ingestion** — Teltonika telemetry TCP server. Image built by [`trm/tcp-ingestion`](https://git.dev.microservices.al/trm/tcp-ingestion)'s Gitea workflow.
- **postgres** — PostgreSQL 16 + TimescaleDB + PostGIS via `timescale/timescaledb-ha`. Schema authority is split: the `positions` hypertable is owned by [`trm/processor`](https://git.dev.microservices.al/trm/processor)'s migration runner; everything else is owned by `trm/directus` via its snapshot YAML. Internal-only, persisted via named volume.
- **processor** — consumes telemetry from Redis, writes to Postgres. Image built by [`trm/processor`](https://git.dev.microservices.al/trm/processor)'s Gitea workflow.
- **directus** — business-plane API + admin UI + schema authority. Image built by [`trm/directus`](https://git.dev.microservices.al/trm/directus)'s Gitea workflow. Boot pipeline runs db-init pre-schema → bootstrap → schema-apply → db-init post-schema → start; first boot on a fresh DB takes ~60–90 s.
- **spa** — React + TypeScript front-end (static bundle served by nginx). Image built by [`trm/spa`](https://git.dev.microservices.al/trm/spa)'s Gitea workflow. Internal-only; the reverse proxy routes the public domain root to it. Runtime config is overridden via a host-path volume mount onto `/usr/share/nginx/html/config.json` — one image, multiple environments.

See `../docs/wiki/` for the full architecture.

## First-deploy checklist

Run through this once per environment before clicking deploy. It covers the security-critical secrets (which must NOT use the compose.yaml placeholder defaults) and the Portainer setup.

### 1. Generate per-environment secrets

These values are unique per environment — never reuse them across stage/prod, and never reuse them after a compromise. Run on any machine with `openssl` and `uuidgen` (Linux/macOS/WSL):

```bash
echo "POSTGRES_PASSWORD=$(openssl rand -base64 32 | tr -d '/+=')"
echo "DIRECTUS_KEY=$(uuidgen)"
echo "DIRECTUS_SECRET=$(openssl rand -hex 64)"
echo "DIRECTUS_ADMIN_PASSWORD=$(openssl rand -base64 24 | tr -d '/+=')"
```

Keep the output somewhere safe (1Password, Vaultwarden, etc.) — you'll paste it into Portainer next, and you'll need `DIRECTUS_ADMIN_PASSWORD` again to log in for the first time.

> The compose defaults for these (`trm-pilot-change-me`, `REPLACE-ME-WITH-A-UUID`, `REPLACE-ME-WITH-A-LONG-RANDOM-STRING`, `CHANGE-ON-FIRST-LOGIN`) are deliberately broken-looking. Anything still using them after deploy is a misconfiguration.

### 2. Set Portainer stack environment variables

Stack → **Environment variables** in Portainer's Add stack form. Required for first deploy:

| Variable | Value |
|---|---|
| `POSTGRES_PASSWORD` | from step 1 |
| `DIRECTUS_KEY` | from step 1 |
| `DIRECTUS_SECRET` | from step 1 |
| `DIRECTUS_ADMIN_EMAIL` | the email you'll log in with |
| `DIRECTUS_ADMIN_PASSWORD` | from step 1 |
| `DIRECTUS_PUBLIC_URL` | external URL of the Directus admin UI (e.g. `https://directus.stage.example.com`). Used in password-reset emails and OAuth redirects. |

Recommended for any non-throwaway environment:

| Variable | Value |
|---|---|
| `TCP_INGESTION_TAG` | a specific commit SHA (not `main`) for reproducibility |
| `PROCESSOR_TAG` | same |
| `DIRECTUS_TAG` | same |
| `LOG_LEVEL` | `info` or `warn` for prod; `debug` only for active troubleshooting |
| `LOG_STYLE` | `json` for log aggregators; default is already `json` |

Optional:

| Variable | When to set |
|---|---|
| `TCP_INGESTION_PORT` | Change if `5027` is already in use on the host. GPS devices need a real host port — this one is published. |
| `SPA_TAG` | A specific commit SHA for production reproducibility; `main` for stage's auto-tracking. |
| `SPA_CONFIG_FILE` | Path on the host to the SPA's runtime config file (default `./spa-config.json`). See "Runtime config override" below. |
| `DIRECTUS_CORS_ENABLED` / `DIRECTUS_CORS_ORIGIN` | Leave disabled — same-origin proxy means CORS isn't in play. Enable only if you ever need cross-origin access from a partner integration. |

> **Directus is internal-only by design.** It listens on `8055` inside the `trm_default` Compose network and is **not** published to the host. Wire a reverse proxy (Traefik / Caddy / nginx) on the host or attached to the network and forward your public domain to `http://directus:8055`. The proxy handles TLS, optional WAF / rate-limit, and any auth-header rewriting. Set `DIRECTUS_PUBLIC_URL` to the proxy-served URL (e.g. `https://directus.stage.example.com`) so password-reset emails and OAuth redirects work. The dev compose in `trm/directus` does host-publish `8055` for local iteration; this stage/prod stack deliberately does not.

### 3. Authenticate the host to the Gitea registry

First deploy only (Portainer's **Registries** UI is preferred over manual login):

```bash
docker login git.dev.microservices.al
```

### 4. Deploy the stack

Stack → **Add stack** → **Repository** → fill in repo URL, branch, compose path → **Deploy the stack**.

### 5. Watch the first boot

Directus's first boot runs ~30–45 s of internal migrations on top of the project's own boot pipeline. Total time-to-healthy on a fresh DB is ~60–90 s. Tail the logs in Portainer → Stack → directus → Logs:

Expected progression:

```
[entrypoint] step 1/5: db-init (pre-schema)
[db-init] db-init complete: 3 applied, 0 skipped
[entrypoint] step 2/5: directus bootstrap
INFO: Initializing bootstrap...
INFO: Installing Directus system tables...
INFO: Running migrations...
... ~80 internal migrations ...
INFO: Setting up first admin role...
INFO: Adding first admin user...
INFO: Done
[entrypoint] step 3/5: directus schema apply
INFO: Snapshot applied successfully
[entrypoint] step 4/5: db-init (post-schema)
[db-init] db-init complete: 2 applied, 0 skipped
[entrypoint] step 5/5: directus start (pm2-runtime)
PM2 log: App [directus:0] online
INFO: Server started at http://0.0.0.0:8055
```

The healthcheck flips to `healthy` once the server is serving (~5–10 s after the "Server started" log line).

### 6. First login

Browse to `${DIRECTUS_PUBLIC_URL}` (or `http://<host>:8055` if you didn't put it behind a proxy). Log in with `DIRECTUS_ADMIN_EMAIL` + `DIRECTUS_ADMIN_PASSWORD`.

**Change the admin password immediately** via the user's own profile in the admin UI. The `DIRECTUS_ADMIN_PASSWORD` env var is only the first-boot seed — changing it post-deploy has no effect on the running user. Same goes for `POSTGRES_PASSWORD`: it's baked into the persistent volume on first boot and must be rotated via `ALTER USER` inside psql, not by changing the env var.

### 7. Verify the schema landed

Admin UI → **Settings → Data Model**. You should see 12 user collections: `organizations`, `users` (built-in `directus_users` with custom fields), `organization_users`, `vehicles`, `organization_vehicles`, `devices`, `organization_devices`, `events`, `classes`, `entries`, `entry_crew`, `entry_devices`. If a collection is missing, the schema-apply step failed and the boot logs will say so.

> **Schema-as-code reminder:** do NOT add or remove collections via the admin UI on this stage instance. Schema changes flow through `trm/directus` (admin-UI edit → `pnpm run schema:snapshot` → commit → CI dry-run → image rebuild → redeploy). Edits made directly here will be DROPPED on the next image rebuild — schema-apply enforces the committed snapshot. See `docs/wiki/entities/directus.md` "destructive-apply hazard."

### 8. Verify the SPA loads

Browse to the public root (e.g. `https://stage.trmtracking.org/`). Expected behaviour:

- Unauthenticated → redirected to `/login`.
- Submit valid Directus credentials → redirected to `/`. Operator sees the placeholder home page (live-monitoring map lands in Phase 2).
- Hard refresh on `/` while authenticated stays on `/` (session cookie carries the session — no client-side state needed across reloads).
- Sign out → redirect back to `/login`. Refresh stays on `/login`.

If `/config.json` returns a 404 or a malformed JSON, the SPA renders an explicit "Runtime config error" panel listing what's missing — that's a sign the volume mount didn't pick up the override file. Re-check the `SPA_CONFIG_FILE` path and that the file is reachable from inside the container.

---

## Deploy via Portainer (Repository Stack)

1. **Stack → Add stack → Repository** in Portainer.
2. Repository URL: `https://git.dev.microservices.al/trm/deploy`
3. Branch: `main`
4. Compose path: `compose.yaml`
5. Environment variables: leave empty for defaults, or set per `.env.example`.
6. (Optional) Enable **Automatic updates** with a webhook. Portainer will poll or accept a webhook to redeploy when this repo's `main` changes.

Before the first deploy, the Portainer host must be authenticated to the Gitea registry:

```bash
docker login git.dev.microservices.al
```

(Or configure registry credentials in Portainer's **Registries** UI — preferred.)

## Deploy without Portainer (manual)

```bash
git clone https://git.dev.microservices.al/trm/deploy
cd deploy
cp .env.example .env
# edit .env if you want to override defaults
docker compose pull
docker compose up -d
```

## Updating

When a service publishes a new image (Gitea workflow on push to `main`):

```bash
docker compose pull
docker compose up -d
```

Portainer with automatic updates does this automatically.

To pin a specific build for production, set the relevant `*_TAG` variable in `.env` (or in Portainer's stack environment) to a commit SHA — e.g. `TCP_INGESTION_TAG=af06973`.

## Network model

- One internal Compose network (`trm_default`).
- **Redis**, **postgres**, **directus**, **processor**, and **spa** are not bound to host ports — only reachable from other services in the stack via service-name DNS (`redis://redis:6379`, `postgres:5432`, `http://directus:8055`, `http://spa:80`).
- **tcp-ingestion**'s TCP port (`5027` by default) is bound to the host so GPS devices can reach it. This is the only host-published port in the stack.
- **HTTP/WS access** goes through a reverse proxy (Traefik / Caddy / nginx) on the host or attached to the `trm_default` network. **Same-origin is non-negotiable** — the SPA's session cookie and the Processor's WebSocket upgrade both rely on browser-attached cookies that only flow within one origin. Route everything under one public domain:
  - `/api/*` → `http://directus:8055/...`
  - `/ws-business` → `ws://directus:8055/websocket`
  - `/ws-live` → `ws://processor:<live-ws-port>` (Phase 1.5; default `8081`)
  - everything else → `http://spa:80` (the SPA's nginx falls through to `index.html` for any unmatched path so client-side routes resolve)
  
  The proxy itself is not part of this stack — add it as a sibling stack in Portainer or run it on the host.
- Other Redis / Postgres instances on the same host can keep using their default ports freely; this stack does not collide with them. The default postgres-on-host you may already have running on `5432` is untouched — this stack's postgres is internal-only.

## Runtime config override (SPA)

The SPA reads `/config.json` at first paint to discover its backend URLs and feature toggles. The image bakes a default suitable for dev (relative paths matching the Vite proxy); stage and prod override via a host-path volume mount.

1. Copy the example file:
   ```bash
   cp spa-config.example.json spa-config.json
   ```
2. Edit `spa-config.json`:
   - Set `env` to `"stage"` or `"prod"`.
   - Add `googleMapsKey` if operators have a Google Maps Tiles API key (optional; if absent the Google tile sources are hidden in the UI).
   - URLs stay relative (`/api`, `/ws-live`, `/ws-business`) — the reverse proxy routes them to the right backends. Only switch to absolute URLs if the SPA ever needs to call cross-origin (which breaks cookie auth — avoid).
3. Keep `spa-config.json` outside source control. The repo's `.gitignore` already excludes it.
4. In Portainer, ensure `SPA_CONFIG_FILE` points at the on-host path of the file. Stack redeploy mounts it read-only into the SPA container.

Changing the file does not require a SPA image rebuild — just `docker compose up -d spa` (or a stack redeploy in Portainer) to pick up the new file. The SPA refetches `/config.json` on every page load.

## Environment variables

See `.env.example` for the documented set with defaults and explanations.

## Why a separate repo

Compose covers multiple services; placing it inside any one service repo creates ownership ambiguity (which service "owns" the Postgres definition? the Redis volume?). Keeping deploy config in its own repo means:

- Compose changes are versioned independently of any service's code.
- Portainer's Repository stack tracks one source of truth.
- Per-environment overrides (e.g. `compose.stage.yaml`, `compose.prod.yaml`) can be added cleanly later.
- Adding a new service is a one-file change here, not a coordinated edit across repos.