feat(vision): wire ANPR into the read bus via VisionReader

A VisionReader polls each opt-in camera (config.anpr===true, off by default) every
VISION_POLL_MS, captures a snapshot, recognizes via VisionClient, and on a confident
plate emits deviceEvents.emitRead({kind:"plate", value}) — the same event a physical
plate reader sends, so the existing ReadDispatcher routes it to the subscription/exit
flow unchanged (no flow rewrite).

The plate stays advisory by construction: the exit flow still demands a covering
payment, the subscription flow only matches a bound plate. Guards: low-confidence reads
dropped; debounce (VISION_DEDUPE_MS) so a parked car doesn't re-fire; per-camera
in-flight guard; idle when vision is off or no camera opts in. #recognizeOn is public
for a future on-demand (loop-edge/API) trigger.

Verified end-to-end: an in-memory anpr camera (AL plate image) + live fast_alpr service
→ VisionReader emitted exactly one {kind:"plate",value:"AA558EE"} onto the bus; debounce
held it to 1 emit over 7 polls. Build + lint green. Updates opencv-anpr-service
(trigger-wiring + per-camera opt-in marked done).

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
2026-06-19 16:21:34 +02:00
parent 236cbfecab
commit 7e086ff0d7
4 changed files with 184 additions and 8 deletions
+10
View File
@@ -20,6 +20,7 @@ import { buildSigner, buildVerifier } from "./signer.js";
import { LogService, pinoDbStream } from "./log-service.js"; import { LogService, pinoDbStream } from "./log-service.js";
import { logRoutes } from "./routes/logs.js"; import { logRoutes } from "./routes/logs.js";
import { VisionClient } from "./vision-client.js"; import { VisionClient } from "./vision-client.js";
import { VisionReader } from "./vision-reader.js";
import { authRoutes } from "./routes/auth.js"; import { authRoutes } from "./routes/auth.js";
import { userRoutes } from "./routes/users.js"; import { userRoutes } from "./routes/users.js";
import { roleRoutes } from "./routes/roles.js"; import { roleRoutes } from "./routes/roles.js";
@@ -171,6 +172,15 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
const visionClient = new VisionClient(app.log); const visionClient = new VisionClient(app.log);
if (visionClient.enabled) app.log.info("vision client enabled"); if (visionClient.enabled) app.log.info("vision client enabled");
// Vision READER: polls opt-in (config.anpr) cameras, recognizes a plate via the
// vision client, and emits a kind:"plate" read onto the SAME read bus a physical
// reader uses → the dispatcher routes it to the subscription/exit flow unchanged. A
// plate stays advisory: the exit flow still demands a payment, the subscription flow
// only matches a BOUND plate. Idle when vision is disabled or no camera opts in.
const visionReader = new VisionReader(db, visionClient, app.log);
app.addHook("onReady", async () => visionReader.start());
app.addHook("onClose", async () => visionReader.stop());
// Credential capture ("enroll a card"): lets the operator present an RFID card to a // Credential capture ("enroll a card"): lets the operator present an RFID card to a
// CHOSEN reader to populate a subscription credential, without blocking the other // CHOSEN reader to populate a subscription credential, without blocking the other
// reader's live flow. Single-shot + TTL. See credential-capture.ts. // reader's live flow. Single-shot + TTL. See credential-capture.ts.
+153
View File
@@ -0,0 +1,153 @@
import { devices, eq, type Db, type DeviceRow } from "@parking/db";
import { registry, type CameraDevice } from "@parking/devices";
import type { FastifyBaseLogger } from "fastify";
import { deviceEvents } from "./device-events.js";
import { directionOf } from "./device-resolve.js";
import type { VisionClient } from "./vision-client.js";
// VisionReader — turns an ANPR camera into a virtual plate READER. It polls each
// opt-in camera, sends a snapshot to the VisionClient (apps/vision), and on a CONFIDENT
// plate emits a `DeviceReadEvent{kind:"plate"}` onto the read bus — the SAME path a
// physical reader's scan takes, so the subscription/exit flows consume it unchanged.
//
// Per the fitness assessment (wiki/entities/opencv-anpr-service.md): a plate read is an
// ADVISORY identity + evidence, never the sole authority to open a paid barrier. The
// guards that keep it advisory live below the recognition, in the flows it feeds:
// - the EXIT flow still requires a covering `payment` (a plate can't bypass it);
// - the SUBSCRIPTION flow only matches a plate BOUND to a subscription (subscriptionPlates).
// So a recognized plate that owes money is refused exactly like a scanned ticket would be.
//
// Opt-in + safety:
// - PER-CAMERA opt-in: only cameras whose config has `anpr: true` are polled (off by
// default). The VisionClient itself is also opt-in (VISION_ENABLED) and fail-soft.
// - DEBOUNCE: a parked car sits in frame across many polls; the same plate from the
// same camera is NOT re-emitted within `dedupeMs` (avoids a storm of identical reads).
// - LOW-CONFIDENCE reads are dropped (not emitted) — a shaky read must not act as an
// identity; the camera keeps polling until a confident frame (or the car leaves).
const POLL_MS = Number(process.env.VISION_POLL_MS ?? 2000);
const DEDUPE_MS = Number(process.env.VISION_DEDUPE_MS ?? 15_000);
interface CameraConfig {
readonly anpr?: boolean;
readonly [k: string]: unknown;
}
export class VisionReader {
readonly #db: Db;
readonly #vision: VisionClient;
readonly #logger: FastifyBaseLogger;
readonly #pollMs: number;
readonly #dedupeMs: number;
#timer: ReturnType<typeof setInterval> | null = null;
/** In-flight guard per camera so a slow recognize doesn't overlap its own next tick. */
readonly #busy = new Set<string>();
/** Last emitted plate + time per camera, for debounce. */
readonly #lastEmit = new Map<string, { value: string; at: number }>();
constructor(db: Db, vision: VisionClient, logger: FastifyBaseLogger, pollMs = POLL_MS, dedupeMs = DEDUPE_MS) {
this.#db = db;
this.#vision = vision;
this.#logger = logger;
this.#pollMs = pollMs;
this.#dedupeMs = dedupeMs;
}
start(): void {
if (this.#timer) return;
// Only run when vision is enabled AND at least one camera opts in — otherwise the
// timer is pure overhead. We still re-check enabled per tick (config can change).
if (!this.#vision.enabled) {
this.#logger.info("vision reader idle (VISION_ENABLED off)");
return;
}
this.#timer = setInterval(() => void this.#tick(), this.#pollMs);
this.#timer.unref?.();
this.#logger.info(`vision reader polling anpr cameras every ${this.#pollMs}ms`);
}
stop(): void {
if (this.#timer) {
clearInterval(this.#timer);
this.#timer = null;
}
}
/** One poll cycle: recognize on every opt-in camera, concurrently. Never throws. */
async #tick(): Promise<void> {
const cams = this.#anprCameras();
if (cams.length === 0) return;
await Promise.all(cams.map((c) => this.#recognizeOn(c)));
}
/** Enabled cameras with `config.anpr === true`. */
#anprCameras(): DeviceRow[] {
return this.#db
.select()
.from(devices)
.where(eq(devices.category, "camera"))
.all()
.filter((r) => r.enabled && (r.config as CameraConfig)?.anpr === true);
}
/**
* Capture from one camera → recognize → maybe emit a plate read. Public so a future
* on-demand trigger (a loop edge, an API call) can call it directly, not just the poll.
*/
async #recognizeOn(row: DeviceRow): Promise<void> {
if (this.#busy.has(row.id)) return; // skip if its previous recognize is still running
this.#busy.add(row.id);
try {
const camera = this.#buildCamera(row);
if (!camera) return;
const dir = directionOf(this.#db, row);
const direction = dir === "exit" ? "exit" : "entry"; // "both" → entry context
const shot = await camera.captureSnapshot({ direction });
const result = await this.#vision.analyze(shot.bytes, shot.contentType);
// Fail-soft: null (disabled/unreachable/timeout) or no plate ⇒ nothing to emit.
if (!result || !result.plate) return;
// Advisory gate: a low-confidence read is NOT an identity — drop it.
if (result.lowConfidence) {
this.#logger.debug(`vision low-confidence plate '${result.plate.text}' on ${row.id} — dropped`);
return;
}
const plate = result.plate.text.trim().toUpperCase();
if (!plate) return;
if (this.#isDuplicate(row.id, plate)) return; // same car still in frame
this.#lastEmit.set(row.id, { value: plate, at: Date.now() });
// Emit onto the read bus — the SAME event a physical plate reader would send, so
// the ReadDispatcher routes it to the subscription/exit flow unchanged.
deviceEvents.emitRead({
driverId: row.driverId,
deviceId: row.id,
value: plate,
kind: "plate",
at: new Date().toISOString(),
});
this.#logger.info(`vision plate '${plate}' (${result.plate.confidence.toFixed(3)}) from camera ${row.id}`);
} catch (err) {
// Never let a camera/recognition error break the poll loop.
this.#logger.warn(`vision reader ${row.id} failed: ${(err as Error).message}`);
} finally {
this.#busy.delete(row.id);
}
}
/** Debounce: true if this same plate was emitted from this camera within dedupeMs. */
#isDuplicate(cameraId: string, plate: string): boolean {
const last = this.#lastEmit.get(cameraId);
return last != null && last.value === plate && Date.now() - last.at < this.#dedupeMs;
}
/** Build a live camera adapter from a devices row, or null. */
#buildCamera(row: DeviceRow): CameraDevice | null {
const driver = registry.get(row.driverId);
if (!driver) return null;
try {
return driver.create(row.config as never) as CameraDevice;
} catch {
return null;
}
}
}
+17 -8
View File
@@ -170,14 +170,22 @@ the read is given** — and the answer splits by role:
now exists: **opt-in** (`VISION_ENABLED`, default off), **fail-soft** (any error/timeout/unreachable → now exists: **opt-in** (`VISION_ENABLED`, default off), **fail-soft** (any error/timeout/unreachable →
`null`, never throws into the lane → ticket-path fallback), and **re-applies the confidence floor** `null`, never throws into the lane → ticket-path fallback), and **re-applies the confidence floor**
(`VISION_MIN_CONFIDENCE`) so a low read is flagged advisory. Constructed in `server.ts`; verified (`VISION_MIN_CONFIDENCE`) so a low read is flagged advisory. Constructed in `server.ts`; verified
end-to-end against the live service (Node → `AA558EE` 0.999, `region=Albania`). Still NOT wired into the end-to-end against the live service (Node → `AA558EE` 0.999, `region=Albania`). (2) ✅ **DONE —
read bus. (2) **trigger wiring** — snapshots today fire *after* a barrier opens (evidence); trigger wiring (`apps/server/src/vision-reader.ts`).** A **`VisionReader`** polls each **opt-in**
plate-as-identity needs a snapshot *before* the decision, on a **per-camera opt-in** lane → emit camera (`config.anpr === true`, off by default) every `VISION_POLL_MS`, captures a snapshot →
`DeviceReadEvent{kind:"plate"}` (open item below). (3) **field-accuracy** unknown — re-benchmark/tune `VisionClient.analyze` → on a **confident** plate emits a `DeviceReadEvent{kind:"plate"}` onto the
**same read bus a physical reader uses** (`deviceEvents.emitRead`), so the `ReadDispatcher` routes it
to the subscription/exit flow **unchanged**. Guards: low-confidence reads are dropped (not an
identity); a **debounce** (`VISION_DEDUPE_MS`) stops the same plate re-firing while a car sits in
frame; an in-flight guard prevents overlapping recognizes; idle when vision is off or no camera opts
in. Verified end-to-end (live service → reader → one `AA558EE` read on the bus; debounce held it to 1
emit over 7 polls). Plate stays **advisory** — the exit flow still demands a `payment`, the
subscription flow only matches a **bound** plate. (3) **field-accuracy** unknown — re-benchmark/tune
the threshold on real on-site captures (angle/night/dirt). (4) the **weight-provenance** check (open). the threshold on real on-site captures (angle/night/dirt). (4) the **weight-provenance** check (open).
**Bottom line: consume it as a gated advisory identity source feeding the existing `kind:"plate"` path **Bottom line: consume it as a gated advisory identity source feeding the existing `kind:"plate"` path
— not as sole authority — and Job 2 is still required for the anti-spoofing value.** With the adapter — not as sole authority — and Job 2 is still required for the anti-spoofing value.** The
done, the next concrete step is the **opt-in snapshot→read trigger**, not more model work. adapter + the opt-in poll→read trigger are now **both built and verified end-to-end**; remaining is
field tuning (3), the provenance check (4), the SetupWizard `anpr` toggle, and Job 2.
## Open ## Open
@@ -190,5 +198,6 @@ done, the next concrete step is the **opt-in snapshot→read trigger**, not more
mismatch an anomaly without false-positiving on lighting/angle. mismatch an anomaly without false-positiving on lighting/angle.
- **Compute footprint** on the appliance (CPU-only vs. a small GPU/NPU) — procurement input - **Compute footprint** on the appliance (CPU-only vs. a small GPU/NPU) — procurement input
([[bom]], [[open-questions]]). ([[bom]], [[open-questions]]).
- Per-camera **opt-in** ("optionally bound", user's word): which lanes/cameras route snapshots to - Per-camera **opt-in** — ✅ **mechanism built**: `config.anpr === true` on a camera enables ANPR
the service. polling (the `VisionReader`). Remaining: expose the toggle in the **SetupWizard** (it's currently
set in raw config) and decide sensible `VISION_POLL_MS`/`VISION_DEDUPE_MS` defaults per site.
+4
View File
@@ -928,3 +928,7 @@ Q: is the scaffolded ANPR service worthy to consume in entry/exit flows? Assessm
## [2026-06-19] feat | VisionClient Node adapter (apps/server/src/vision-client.ts) ## [2026-06-19] feat | VisionClient Node adapter (apps/server/src/vision-client.ts)
Scaffolded the Node-side adapter to the host vision microservice per the fitness assessment. VisionClient calls apps/vision over localhost HTTP (POST /analyze with snapshot Buffer bytes, GET /health), returning a normalised/camelCased VisionResult (best plate + all plates + lowConfidence + modelVersion + tookMs) or null. THREE guardrails enforce "advisory, never sole authority" at the boundary: (1) OPT-IN — VISION_ENABLED (default OFF), so the appliance runs with no vision service; (2) FAIL-SOFT — disabled/unreachable/timeout/non-2xx/bad-body all resolve to null and NEVER throw into the entry/exit path (→ ticket/manual fallback, never strand a car); (3) CONFIDENCE FLOOR re-applied (VISION_MIN_CONFIDENCE) on top of the service's own low_confidence flag. Per-request AbortController timeout (VISION_TIMEOUT_MS, default 1500ms) so a slow call can't hang the lane. Constructed in server.ts (logs when enabled). VERIFIED: fail-soft (disabled→null, unreachable→null no-throw) and LIVE end-to-end (Node client → running fast_alpr service → AA558EE 0.999 region=Albania, camelCased). NOT yet wired into the read bus — the opt-in snapshot-before-decision trigger that emits DeviceReadEvent{kind:"plate"} is the next deliberate step. Build+lint green. Updated [[opencv-anpr-service]] (gap 1 marked done). Scaffolded the Node-side adapter to the host vision microservice per the fitness assessment. VisionClient calls apps/vision over localhost HTTP (POST /analyze with snapshot Buffer bytes, GET /health), returning a normalised/camelCased VisionResult (best plate + all plates + lowConfidence + modelVersion + tookMs) or null. THREE guardrails enforce "advisory, never sole authority" at the boundary: (1) OPT-IN — VISION_ENABLED (default OFF), so the appliance runs with no vision service; (2) FAIL-SOFT — disabled/unreachable/timeout/non-2xx/bad-body all resolve to null and NEVER throw into the entry/exit path (→ ticket/manual fallback, never strand a car); (3) CONFIDENCE FLOOR re-applied (VISION_MIN_CONFIDENCE) on top of the service's own low_confidence flag. Per-request AbortController timeout (VISION_TIMEOUT_MS, default 1500ms) so a slow call can't hang the lane. Constructed in server.ts (logs when enabled). VERIFIED: fail-soft (disabled→null, unreachable→null no-throw) and LIVE end-to-end (Node client → running fast_alpr service → AA558EE 0.999 region=Albania, camelCased). NOT yet wired into the read bus — the opt-in snapshot-before-decision trigger that emits DeviceReadEvent{kind:"plate"} is the next deliberate step. Build+lint green. Updated [[opencv-anpr-service]] (gap 1 marked done).
## [2026-06-19] feat | VisionReader — wire ANPR into the read bus (apps/server/src/vision-reader.ts)
Wired the vision service into the entry/exit flows via the READ BUS. VisionReader polls each OPT-IN camera (config.anpr===true, off by default) every VISION_POLL_MS, captures a snapshot → VisionClient.analyze → on a CONFIDENT plate calls deviceEvents.emitRead({kind:"plate", value:PLATE, deviceId, driverId}) — the SAME event a physical plate reader emits, so the existing ReadDispatcher routes it to the subscription/exit flow UNCHANGED (no flow rewrite). The plate stays ADVISORY by construction: the exit flow still demands a covering payment (a plate can't bypass it), the subscription flow only matches a BOUND plate (subscriptionPlates). Guards: low-confidence reads DROPPED (a shaky read isn't an identity); DEBOUNCE (VISION_DEDUPE_MS, default 15s) so a parked car in frame doesn't re-fire the same plate; per-camera in-flight guard; idle when VISION_ENABLED off or no camera opts in; #recognizeOn is public for a future on-demand trigger (loop edge / API). Direction from directionOf (both→entry context). Constructed in server.ts, start on onReady / stop on onClose. VERIFIED END-TO-END: in-memory anpr camera returning the AL plate image + live fast_alpr service → VisionReader emitted exactly one {kind:"plate",value:"AA558EE"} read onto the bus; debounce held it to 1 emit over 7 polls. Build+lint green. Updated [[opencv-anpr-service]] (trigger-wiring gap + per-camera opt-in marked done). Remaining: SetupWizard anpr toggle, field tuning, weight-provenance, Job 2.