feat(vision): wire ANPR into the read bus via VisionReader
A VisionReader polls each opt-in camera (config.anpr===true, off by default) every
VISION_POLL_MS, captures a snapshot, recognizes via VisionClient, and on a confident
plate emits deviceEvents.emitRead({kind:"plate", value}) — the same event a physical
plate reader sends, so the existing ReadDispatcher routes it to the subscription/exit
flow unchanged (no flow rewrite).
The plate stays advisory by construction: the exit flow still demands a covering
payment, the subscription flow only matches a bound plate. Guards: low-confidence reads
dropped; debounce (VISION_DEDUPE_MS) so a parked car doesn't re-fire; per-camera
in-flight guard; idle when vision is off or no camera opts in. #recognizeOn is public
for a future on-demand (loop-edge/API) trigger.
Verified end-to-end: an in-memory anpr camera (AL plate image) + live fast_alpr service
→ VisionReader emitted exactly one {kind:"plate",value:"AA558EE"} onto the bus; debounce
held it to 1 emit over 7 polls. Build + lint green. Updates opencv-anpr-service
(trigger-wiring + per-camera opt-in marked done).
Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
@@ -20,6 +20,7 @@ import { buildSigner, buildVerifier } from "./signer.js";
|
||||
import { LogService, pinoDbStream } from "./log-service.js";
|
||||
import { logRoutes } from "./routes/logs.js";
|
||||
import { VisionClient } from "./vision-client.js";
|
||||
import { VisionReader } from "./vision-reader.js";
|
||||
import { authRoutes } from "./routes/auth.js";
|
||||
import { userRoutes } from "./routes/users.js";
|
||||
import { roleRoutes } from "./routes/roles.js";
|
||||
@@ -171,6 +172,15 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
|
||||
const visionClient = new VisionClient(app.log);
|
||||
if (visionClient.enabled) app.log.info("vision client enabled");
|
||||
|
||||
// Vision READER: polls opt-in (config.anpr) cameras, recognizes a plate via the
|
||||
// vision client, and emits a kind:"plate" read onto the SAME read bus a physical
|
||||
// reader uses → the dispatcher routes it to the subscription/exit flow unchanged. A
|
||||
// plate stays advisory: the exit flow still demands a payment, the subscription flow
|
||||
// only matches a BOUND plate. Idle when vision is disabled or no camera opts in.
|
||||
const visionReader = new VisionReader(db, visionClient, app.log);
|
||||
app.addHook("onReady", async () => visionReader.start());
|
||||
app.addHook("onClose", async () => visionReader.stop());
|
||||
|
||||
// Credential capture ("enroll a card"): lets the operator present an RFID card to a
|
||||
// CHOSEN reader to populate a subscription credential, without blocking the other
|
||||
// reader's live flow. Single-shot + TTL. See credential-capture.ts.
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
import { devices, eq, type Db, type DeviceRow } from "@parking/db";
|
||||
import { registry, type CameraDevice } from "@parking/devices";
|
||||
import type { FastifyBaseLogger } from "fastify";
|
||||
import { deviceEvents } from "./device-events.js";
|
||||
import { directionOf } from "./device-resolve.js";
|
||||
import type { VisionClient } from "./vision-client.js";
|
||||
|
||||
// VisionReader — turns an ANPR camera into a virtual plate READER. It polls each
|
||||
// opt-in camera, sends a snapshot to the VisionClient (apps/vision), and on a CONFIDENT
|
||||
// plate emits a `DeviceReadEvent{kind:"plate"}` onto the read bus — the SAME path a
|
||||
// physical reader's scan takes, so the subscription/exit flows consume it unchanged.
|
||||
//
|
||||
// Per the fitness assessment (wiki/entities/opencv-anpr-service.md): a plate read is an
|
||||
// ADVISORY identity + evidence, never the sole authority to open a paid barrier. The
|
||||
// guards that keep it advisory live below the recognition, in the flows it feeds:
|
||||
// - the EXIT flow still requires a covering `payment` (a plate can't bypass it);
|
||||
// - the SUBSCRIPTION flow only matches a plate BOUND to a subscription (subscriptionPlates).
|
||||
// So a recognized plate that owes money is refused exactly like a scanned ticket would be.
|
||||
//
|
||||
// Opt-in + safety:
|
||||
// - PER-CAMERA opt-in: only cameras whose config has `anpr: true` are polled (off by
|
||||
// default). The VisionClient itself is also opt-in (VISION_ENABLED) and fail-soft.
|
||||
// - DEBOUNCE: a parked car sits in frame across many polls; the same plate from the
|
||||
// same camera is NOT re-emitted within `dedupeMs` (avoids a storm of identical reads).
|
||||
// - LOW-CONFIDENCE reads are dropped (not emitted) — a shaky read must not act as an
|
||||
// identity; the camera keeps polling until a confident frame (or the car leaves).
|
||||
|
||||
const POLL_MS = Number(process.env.VISION_POLL_MS ?? 2000);
|
||||
const DEDUPE_MS = Number(process.env.VISION_DEDUPE_MS ?? 15_000);
|
||||
|
||||
interface CameraConfig {
|
||||
readonly anpr?: boolean;
|
||||
readonly [k: string]: unknown;
|
||||
}
|
||||
|
||||
export class VisionReader {
|
||||
readonly #db: Db;
|
||||
readonly #vision: VisionClient;
|
||||
readonly #logger: FastifyBaseLogger;
|
||||
readonly #pollMs: number;
|
||||
readonly #dedupeMs: number;
|
||||
#timer: ReturnType<typeof setInterval> | null = null;
|
||||
/** In-flight guard per camera so a slow recognize doesn't overlap its own next tick. */
|
||||
readonly #busy = new Set<string>();
|
||||
/** Last emitted plate + time per camera, for debounce. */
|
||||
readonly #lastEmit = new Map<string, { value: string; at: number }>();
|
||||
|
||||
constructor(db: Db, vision: VisionClient, logger: FastifyBaseLogger, pollMs = POLL_MS, dedupeMs = DEDUPE_MS) {
|
||||
this.#db = db;
|
||||
this.#vision = vision;
|
||||
this.#logger = logger;
|
||||
this.#pollMs = pollMs;
|
||||
this.#dedupeMs = dedupeMs;
|
||||
}
|
||||
|
||||
start(): void {
|
||||
if (this.#timer) return;
|
||||
// Only run when vision is enabled AND at least one camera opts in — otherwise the
|
||||
// timer is pure overhead. We still re-check enabled per tick (config can change).
|
||||
if (!this.#vision.enabled) {
|
||||
this.#logger.info("vision reader idle (VISION_ENABLED off)");
|
||||
return;
|
||||
}
|
||||
this.#timer = setInterval(() => void this.#tick(), this.#pollMs);
|
||||
this.#timer.unref?.();
|
||||
this.#logger.info(`vision reader polling anpr cameras every ${this.#pollMs}ms`);
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
if (this.#timer) {
|
||||
clearInterval(this.#timer);
|
||||
this.#timer = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** One poll cycle: recognize on every opt-in camera, concurrently. Never throws. */
|
||||
async #tick(): Promise<void> {
|
||||
const cams = this.#anprCameras();
|
||||
if (cams.length === 0) return;
|
||||
await Promise.all(cams.map((c) => this.#recognizeOn(c)));
|
||||
}
|
||||
|
||||
/** Enabled cameras with `config.anpr === true`. */
|
||||
#anprCameras(): DeviceRow[] {
|
||||
return this.#db
|
||||
.select()
|
||||
.from(devices)
|
||||
.where(eq(devices.category, "camera"))
|
||||
.all()
|
||||
.filter((r) => r.enabled && (r.config as CameraConfig)?.anpr === true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Capture from one camera → recognize → maybe emit a plate read. Public so a future
|
||||
* on-demand trigger (a loop edge, an API call) can call it directly, not just the poll.
|
||||
*/
|
||||
async #recognizeOn(row: DeviceRow): Promise<void> {
|
||||
if (this.#busy.has(row.id)) return; // skip if its previous recognize is still running
|
||||
this.#busy.add(row.id);
|
||||
try {
|
||||
const camera = this.#buildCamera(row);
|
||||
if (!camera) return;
|
||||
const dir = directionOf(this.#db, row);
|
||||
const direction = dir === "exit" ? "exit" : "entry"; // "both" → entry context
|
||||
const shot = await camera.captureSnapshot({ direction });
|
||||
const result = await this.#vision.analyze(shot.bytes, shot.contentType);
|
||||
// Fail-soft: null (disabled/unreachable/timeout) or no plate ⇒ nothing to emit.
|
||||
if (!result || !result.plate) return;
|
||||
// Advisory gate: a low-confidence read is NOT an identity — drop it.
|
||||
if (result.lowConfidence) {
|
||||
this.#logger.debug(`vision low-confidence plate '${result.plate.text}' on ${row.id} — dropped`);
|
||||
return;
|
||||
}
|
||||
const plate = result.plate.text.trim().toUpperCase();
|
||||
if (!plate) return;
|
||||
if (this.#isDuplicate(row.id, plate)) return; // same car still in frame
|
||||
this.#lastEmit.set(row.id, { value: plate, at: Date.now() });
|
||||
|
||||
// Emit onto the read bus — the SAME event a physical plate reader would send, so
|
||||
// the ReadDispatcher routes it to the subscription/exit flow unchanged.
|
||||
deviceEvents.emitRead({
|
||||
driverId: row.driverId,
|
||||
deviceId: row.id,
|
||||
value: plate,
|
||||
kind: "plate",
|
||||
at: new Date().toISOString(),
|
||||
});
|
||||
this.#logger.info(`vision plate '${plate}' (${result.plate.confidence.toFixed(3)}) from camera ${row.id}`);
|
||||
} catch (err) {
|
||||
// Never let a camera/recognition error break the poll loop.
|
||||
this.#logger.warn(`vision reader ${row.id} failed: ${(err as Error).message}`);
|
||||
} finally {
|
||||
this.#busy.delete(row.id);
|
||||
}
|
||||
}
|
||||
|
||||
/** Debounce: true if this same plate was emitted from this camera within dedupeMs. */
|
||||
#isDuplicate(cameraId: string, plate: string): boolean {
|
||||
const last = this.#lastEmit.get(cameraId);
|
||||
return last != null && last.value === plate && Date.now() - last.at < this.#dedupeMs;
|
||||
}
|
||||
|
||||
/** Build a live camera adapter from a devices row, or null. */
|
||||
#buildCamera(row: DeviceRow): CameraDevice | null {
|
||||
const driver = registry.get(row.driverId);
|
||||
if (!driver) return null;
|
||||
try {
|
||||
return driver.create(row.config as never) as CameraDevice;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -170,14 +170,22 @@ the read is given** — and the answer splits by role:
|
||||
now exists: **opt-in** (`VISION_ENABLED`, default off), **fail-soft** (any error/timeout/unreachable →
|
||||
`null`, never throws into the lane → ticket-path fallback), and **re-applies the confidence floor**
|
||||
(`VISION_MIN_CONFIDENCE`) so a low read is flagged advisory. Constructed in `server.ts`; verified
|
||||
end-to-end against the live service (Node → `AA558EE` 0.999, `region=Albania`). Still NOT wired into the
|
||||
read bus. (2) **trigger wiring** — snapshots today fire *after* a barrier opens (evidence);
|
||||
plate-as-identity needs a snapshot *before* the decision, on a **per-camera opt-in** lane → emit
|
||||
`DeviceReadEvent{kind:"plate"}` (open item below). (3) **field-accuracy** unknown — re-benchmark/tune
|
||||
end-to-end against the live service (Node → `AA558EE` 0.999, `region=Albania`). (2) ✅ **DONE —
|
||||
trigger wiring (`apps/server/src/vision-reader.ts`).** A **`VisionReader`** polls each **opt-in**
|
||||
camera (`config.anpr === true`, off by default) every `VISION_POLL_MS`, captures a snapshot →
|
||||
`VisionClient.analyze` → on a **confident** plate emits a `DeviceReadEvent{kind:"plate"}` onto the
|
||||
**same read bus a physical reader uses** (`deviceEvents.emitRead`), so the `ReadDispatcher` routes it
|
||||
to the subscription/exit flow **unchanged**. Guards: low-confidence reads are dropped (not an
|
||||
identity); a **debounce** (`VISION_DEDUPE_MS`) stops the same plate re-firing while a car sits in
|
||||
frame; an in-flight guard prevents overlapping recognizes; idle when vision is off or no camera opts
|
||||
in. Verified end-to-end (live service → reader → one `AA558EE` read on the bus; debounce held it to 1
|
||||
emit over 7 polls). Plate stays **advisory** — the exit flow still demands a `payment`, the
|
||||
subscription flow only matches a **bound** plate. (3) **field-accuracy** unknown — re-benchmark/tune
|
||||
the threshold on real on-site captures (angle/night/dirt). (4) the **weight-provenance** check (open).
|
||||
**Bottom line: consume it as a gated advisory identity source feeding the existing `kind:"plate"` path
|
||||
— not as sole authority — and Job 2 is still required for the anti-spoofing value.** With the adapter
|
||||
done, the next concrete step is the **opt-in snapshot→read trigger**, not more model work.
|
||||
— not as sole authority — and Job 2 is still required for the anti-spoofing value.** The
|
||||
adapter + the opt-in poll→read trigger are now **both built and verified end-to-end**; remaining is
|
||||
field tuning (3), the provenance check (4), the SetupWizard `anpr` toggle, and Job 2.
|
||||
|
||||
## Open
|
||||
|
||||
@@ -190,5 +198,6 @@ done, the next concrete step is the **opt-in snapshot→read trigger**, not more
|
||||
mismatch an anomaly without false-positiving on lighting/angle.
|
||||
- **Compute footprint** on the appliance (CPU-only vs. a small GPU/NPU) — procurement input
|
||||
([[bom]], [[open-questions]]).
|
||||
- Per-camera **opt-in** ("optionally bound", user's word): which lanes/cameras route snapshots to
|
||||
the service.
|
||||
- Per-camera **opt-in** — ✅ **mechanism built**: `config.anpr === true` on a camera enables ANPR
|
||||
polling (the `VisionReader`). Remaining: expose the toggle in the **SetupWizard** (it's currently
|
||||
set in raw config) and decide sensible `VISION_POLL_MS`/`VISION_DEDUPE_MS` defaults per site.
|
||||
|
||||
@@ -928,3 +928,7 @@ Q: is the scaffolded ANPR service worthy to consume in entry/exit flows? Assessm
|
||||
## [2026-06-19] feat | VisionClient Node adapter (apps/server/src/vision-client.ts)
|
||||
|
||||
Scaffolded the Node-side adapter to the host vision microservice per the fitness assessment. VisionClient calls apps/vision over localhost HTTP (POST /analyze with snapshot Buffer bytes, GET /health), returning a normalised/camelCased VisionResult (best plate + all plates + lowConfidence + modelVersion + tookMs) or null. THREE guardrails enforce "advisory, never sole authority" at the boundary: (1) OPT-IN — VISION_ENABLED (default OFF), so the appliance runs with no vision service; (2) FAIL-SOFT — disabled/unreachable/timeout/non-2xx/bad-body all resolve to null and NEVER throw into the entry/exit path (→ ticket/manual fallback, never strand a car); (3) CONFIDENCE FLOOR re-applied (VISION_MIN_CONFIDENCE) on top of the service's own low_confidence flag. Per-request AbortController timeout (VISION_TIMEOUT_MS, default 1500ms) so a slow call can't hang the lane. Constructed in server.ts (logs when enabled). VERIFIED: fail-soft (disabled→null, unreachable→null no-throw) and LIVE end-to-end (Node client → running fast_alpr service → AA558EE 0.999 region=Albania, camelCased). NOT yet wired into the read bus — the opt-in snapshot-before-decision trigger that emits DeviceReadEvent{kind:"plate"} is the next deliberate step. Build+lint green. Updated [[opencv-anpr-service]] (gap 1 marked done).
|
||||
|
||||
## [2026-06-19] feat | VisionReader — wire ANPR into the read bus (apps/server/src/vision-reader.ts)
|
||||
|
||||
Wired the vision service into the entry/exit flows via the READ BUS. VisionReader polls each OPT-IN camera (config.anpr===true, off by default) every VISION_POLL_MS, captures a snapshot → VisionClient.analyze → on a CONFIDENT plate calls deviceEvents.emitRead({kind:"plate", value:PLATE, deviceId, driverId}) — the SAME event a physical plate reader emits, so the existing ReadDispatcher routes it to the subscription/exit flow UNCHANGED (no flow rewrite). The plate stays ADVISORY by construction: the exit flow still demands a covering payment (a plate can't bypass it), the subscription flow only matches a BOUND plate (subscriptionPlates). Guards: low-confidence reads DROPPED (a shaky read isn't an identity); DEBOUNCE (VISION_DEDUPE_MS, default 15s) so a parked car in frame doesn't re-fire the same plate; per-camera in-flight guard; idle when VISION_ENABLED off or no camera opts in; #recognizeOn is public for a future on-demand trigger (loop edge / API). Direction from directionOf (both→entry context). Constructed in server.ts, start on onReady / stop on onClose. VERIFIED END-TO-END: in-memory anpr camera returning the AL plate image + live fast_alpr service → VisionReader emitted exactly one {kind:"plate",value:"AA558EE"} read onto the bus; debounce held it to 1 emit over 7 polls. Build+lint green. Updated [[opencv-anpr-service]] (trigger-wiring gap + per-camera opt-in marked done). Remaining: SetupWizard anpr toggle, field tuning, weight-provenance, Job 2.
|
||||
|
||||
Reference in New Issue
Block a user