docs/wiki/entities/processor.md

title, type, created, updated, sources, tags

title	type	created	updated	sources	tags
Processor	entity	2026-04-30	2026-05-01	gps-tracking-architecture	service telemetry-plane domain-logic

Processor

The service where domain logic lives. Consumes normalized telemetry from redis-streams (default stream telemetry:teltonika, consumer group processor) and is responsible for per-device runtime state, applying domain rules, writing durable state to postgres-timescaledb, and broadcasting live position updates over WebSockets to the react-spa.

Responsibilities

• Maintain per-device runtime state — last position, derived metrics, current zone, accumulators.
• Apply domain rules that turn raw telemetry into meaningful events.
• Write durable state — both raw position history and any derived events.
• Broadcast live positions to subscribed react-spa clients over a WebSocket endpoint. See live-channel-architecture for the full design and rationale.
• Emit events for downstream consumers (Directus Flows, notification services, dashboards).

Where tcp-ingestion is about throughput and protocol correctness, the Processor is about correctness of meaning. It is the component most likely to evolve as requirements grow, which is why it is isolated from the sockets on one side and the API surface on the other.

State management

• Static reference data (spatial assets, configurations) loaded at startup; refreshed on a known cadence or via explicit invalidation.
• Per-device state held in memory keyed by device identifier (last seen, current segment, accumulators).
• Durable state written asynchronously to the database.

The database is the source of truth for replay/analysis; in-memory state is the source of truth for the current decision. On restart, hot state is rehydrated from the DB — this is a recovery path, not a hot path.

Database writes

• The Processor is the only writer for high-volume telemetry tables (e.g. the positions hypertable). directus does not insert positions; it reads them.
• For derived business entities (events, violations, alerts), the Processor writes directly to tables directus also knows about. Schema is owned by Directus; the Processor inserts rows respecting that schema.
• This keeps the hot write path off the Directus HTTP stack while still letting Directus expose the data through API and admin UI.

Live broadcast

The Processor exposes a WebSocket endpoint that the react-spa connects to for live position updates. The endpoint authenticates connections by validating Directus-issued JWTs and authorizes subscriptions by delegating to Directus's permission system once at subscribe time — never per record.

This decouples the live channel from directus's failure domain (Directus down blocks only new authorizations, not the live firehose) and preserves plane-separation (telemetry stays in the telemetry plane end-to-end). directus's built-in WebSocket subscriptions remain the right channel for changes to the business-plane tables it writes to (timing edits, configuration, manual overrides). See live-channel-architecture for the full design.

In multi-instance deployments, each Processor reads the redis-streams stream on two consumer groups: a shared processor group for durable writes (work-split across instances) and a per-instance live-broadcast-{instance_id} group for fan-out (every instance reads every record for its own connected clients).

IO element interpretation

Per-model IO mappings live here, not in the Ingestion layer. Example: { "FMB920": { "16": "odometer_km", "240": "movement" } }. This is the boundary set by the teltonika adapter — Ingestion produces raw IO maps; the Processor names and interprets them.

Faulty position handling

The positions hypertable in postgres-timescaledb carries a faulty boolean DEFAULT false column that operators can flip through directus when a position is unrealistic. All Processor read paths against position data filter WHERE faulty = false — peak-speed evaluation inside SLZs, geofence crossing detection, waypoint pass detection, replay-based recompute. The flag is never set at write time; it's a post-hoc operator action.

When an operator flips the flag (set or unset), Directus emits a webhook → Redis Stream recompute:requests. The Processor consumes the request and re-evaluates entry_penalties whose evaluation window overlaps the flagged position's timestamp. Cost sits between formula recompute (cheap) and full geometry replay (expensive) — the affected window is bounded, but the inputs (peak speed, missed-waypoint count) must be re-derived from the now-filtered position stream rather than from snapshotted values.

Scaling

Multiple Processor instances join a Redis Streams consumer group and split the load across device IDs. Consumer-group offsets ensure a crashed instance's work is picked up by the next one.

Failure mode

Crash → consumer-group offsets ensure the next instance picks up where the last left off. In-memory state is rehydrated from the database. See failure-domains.

Development workflow — Phase 2 branch model

Phase 2 (geofence engine, evaluator registry, crossings/penalties/results writers) is a substantial body of work and lives on a long-lived phase-2 branch rather than landing piecemeal on main. Conventions:

• Rebase weekly against main, not merge. Keeps history readable and avoids merge-commit clutter when the branch eventually lands.
• CI parity — same workflow on phase-2 PRs as on main PRs. Test coverage doesn't diverge across the branch boundary.
• Flag-gated incremental merges — chunks that are self-contained (a single evaluator, the geofence detector) can land on main behind PROCESSOR_PHASE_2_ENABLED=false. Off in prod, on in stage. Lets the work merge before it's user-visible without keeping the entire feature on a side branch indefinitely.
• Single squash merge to retire the branch — when Phase 2 is feature-complete enough to dogfood end-to-end, one squash merge retires the branch. Avoid death-by-a-thousand-merges.

Phase 2.5 (the geometry retroactivity engine in directus-schema-draft) follows the same pattern on its own branch when it starts; it is explicitly deferred until Phase 2 has shipped and the manual operator workflow for geometry edits has surfaced real pain points.