11 Commits

Author SHA1 Message Date
julian 9ec644811a feat(vision): surface the recognized plate in the booth UI
The ANPR plate was saved (device_events kind:"read") but had no UI. Extend
GET /api/snapshots/by-identity/:identity to also return plates[] (plate, confidence,
region, direction, snapshotId, at) for that session, and render each as a cyan
"Plate: AA558EE 100%" chip in the SnapshotStrip — so it shows in both the booth
event-detail modal and the pay modal, beside the evidence photo, no separate screen.
Deduped by plate+direction; session:read gated; i18n sq+en.

Verified: by-identity returns plates[] for a seeded read (200, AA558EE 0.999 Albania
entry). Build + lint green.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 17:23:32 +02:00
julian ecaaefd899 refactor(vision): ANPR rides the entry/exit snapshot, drop polling reader
Rework the ANPR trigger to the real design: when a transient presses the button or a
subscriber passes QR/RFID, the entry/exit fires and takes its evidence snapshot — that
is the moment to recognize. snapshotAsync now takes the VisionClient and, after storing
each snapshot from an opt-in (config.anpr) camera, runs ANPR on the SAME image and
records the plate against the SAME session identity (device_events kind:"read" with
plate/confidence/region/snapshotId/source:"entry-exit-snapshot"). One image serves both
evidence and plate extraction; recognition fires only on a real entry/exit — no polling.

The entry/exit/subscription flows take an optional VisionClient and pass it through;
server.ts wires it. Removed the polling VisionReader and VISION_POLL_MS/VISION_DEDUPE_MS.

Advisory + fire-and-forget: a low-confidence/no-plate result records nothing, a vision
failure never delays or changes the open, and the plate does not feed the access
decision. Verified e2e: a simulated entry snapshot on an anpr camera (live fast_alpr)
stored the snapshot for the session and recorded {identity, plate:AA558EE, 0.999,
region:Albania, snapshotId}. Build + lint green.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 16:53:43 +02:00
julian 4af8b56dda feat(vision): configurability — SetupWizard ANPR toggle, footer health chip, env docs
Make the vision service genuinely configurable (was env-only).

- SetupWizard: an "ANPR" checkbox on the camera form (writes config.anpr; persisted
  only when on; sq+en) — opt-in is no longer raw JSON.
- DeviceMonitor optionally takes the VisionClient and probes /health each tick, emitting
  a "vision" pseudo-device → a Vision chip (ready/degraded/offline + recognizer) in the
  booth footer when VISION_ENABLED, no chip when off. Widened the DeviceStatus category
  union (server + web) + footer maps + devices.catVision. Verified: ready/fast_alpr when
  up, 0 chips when disabled.
- apps/vision/.env.example (Python service) + a VISION_* block in apps/server/.env.example
  (Node side) + a Configuration section in opencv-anpr-service.md covering all four
  layers and the caveats: the two processes share the VISION_ prefix but need SEPARATE
  .env files; bind /analyze to 127.0.0.1; cache model weights at deploy; an unbound anpr
  camera recognizes but every read is refused.

Build + lint green.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 16:41:29 +02:00
julian 540b333b06 feat(vision): persist every recognized plate + snapshot as telemetry (non-blocking)
Answers "is a recognized plate saved?" — now yes, for both transient and subscriber, as
an ANPR audit trail independent of whether it matched anything.

VisionReader now stores the snapshot bytes in `snapshots` keyed by identity=PLATE — the
same identity the flow signs its anomaly/event with — so GET /api/snapshots/by-identity/:plate
(the booth event-detail modal's snapshot strip) shows the car's photo against that
anomaly with no UI changes. It also records an unsigned device_events{kind:"read"}
breadcrumb (plate, confidence, region, model, snapshotId, and the dispatch outcome) as a
queryable recognition log. Switched from emitRead to calling ReadDispatcher.dispatch
directly (like qr-reader) to capture that outcome.

Non-blocking: a refused read (no session / unpaid / unknown plate) just returns
rejected — no barrier hold — and is logged with its snapshot for investigation. Plate
stays advisory (exit demands payment; subscription matches only a bound plate).

Verified e2e: a recognized AL plate with no open session signed exit.refused.noSession
(identity=plate), stored a 555KB snapshot under that plate, recorded the read breadcrumb
(accepted:false, reason "no open session"), and by-identity returned the image — the
refused read is fully investigable with its picture. Build + lint green.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 16:29:10 +02:00
julian 7e086ff0d7 feat(vision): wire ANPR into the read bus via VisionReader
A VisionReader polls each opt-in camera (config.anpr===true, off by default) every
VISION_POLL_MS, captures a snapshot, recognizes via VisionClient, and on a confident
plate emits deviceEvents.emitRead({kind:"plate", value}) — the same event a physical
plate reader sends, so the existing ReadDispatcher routes it to the subscription/exit
flow unchanged (no flow rewrite).

The plate stays advisory by construction: the exit flow still demands a covering
payment, the subscription flow only matches a bound plate. Guards: low-confidence reads
dropped; debounce (VISION_DEDUPE_MS) so a parked car doesn't re-fire; per-camera
in-flight guard; idle when vision is off or no camera opts in. #recognizeOn is public
for a future on-demand (loop-edge/API) trigger.

Verified end-to-end: an in-memory anpr camera (AL plate image) + live fast_alpr service
→ VisionReader emitted exactly one {kind:"plate",value:"AA558EE"} onto the bus; debounce
held it to 1 emit over 7 polls. Build + lint green. Updates opencv-anpr-service
(trigger-wiring + per-camera opt-in marked done).

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 16:21:34 +02:00
julian 236cbfecab feat(vision): add VisionClient Node adapter (advisory, fail-soft, opt-in)
Node-side adapter to the apps/vision ANPR microservice (localhost HTTP: POST /analyze
with snapshot bytes, GET /health), returning a normalised VisionResult or null. Enforces
"advisory, never sole authority" at the boundary: opt-in (VISION_ENABLED, default off),
fail-soft (any error/timeout/unreachable → null, never throws into the lane → ticket
fallback), and re-applies the confidence floor (VISION_MIN_CONFIDENCE) on top of the
service's own low_confidence flag. Per-request AbortController timeout so a slow call
can't hang the barrier. Constructed in server.ts.

Verified: fail-soft (disabled/unreachable → null, no throw) and live end-to-end (Node
client → running fast_alpr service → AA558EE 0.999, region=Albania). NOT yet wired into
the read bus — the opt-in snapshot→DeviceReadEvent{kind:"plate"} trigger is the next
step. Build + lint green. Updates opencv-anpr-service (adapter gap marked done).

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 15:58:43 +02:00
julian 17fdf3d482 docs(wiki): vision fitness assessment for entry/exit flows
Record the verdict: the ANPR service is worthy to consume NOW as an advisory plate
IDENTITY source (Job 1) — the flows already treat a kind:"plate" read as first-class
(exit signs source:"lpr"; subscription matches read plate vs bound plates), so it feeds
an existing input with no flow rewrite. It is NOT worthy as the sole authority to open a
transient barrier (a plate is not a payment; spoofing needs Job 2 vehicle verification,
unbuilt) — gated by the confidence floor with ticket/manual fallback. Lists the four
gaps before consumption (VisionClient adapter, opt-in trigger, field accuracy,
weight-provenance). Next step is the adapter, not more model work.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 15:55:16 +02:00
julian 4833b4373d docs(wiki): Albanian-plate OCR benchmark — keep cct-xs-v2-global default
Benchmarked fast-alpr's four fast-plate-ocr models via the full pipeline on real AL
plates (AA558EE, AA687KE), CPU. All four read both correctly; the default
cct-xs-v2-global-model wins on confidence (0.999/1.000) AND speed (33-39ms) and returns
region=Albania. The "European 40+country" model is WORSE here (~0.77 confidence, one
synthetic misread) — overturning the "EU model better for AL" assumption from the prior
research. Decision: no config change. Resolves the AL-accuracy-benchmark open item
(results table + finding added to opencv-anpr-service); weight-provenance remains the
one open recognizer item. Re-benchmark on real on-site captures once cameras installed.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 15:51:08 +02:00
julian 5cedcaefe1 feat(vision): add recognize CLI + verify fast-alpr end-to-end
Add a dev CLI (uv run python -m vision_service.cli <image>) that runs a recognizer on
an image file and prints the parsed plate(s) + confidence + region — fast feedback with
no HTTP. Also a package.json `recognize` script and a vision-recognize entry point.

Verified fast-alpr for real: installed the `alpr` extra, downloaded the YOLOv9 + CCT
ONNX weights (~11MB, cached offline under ~/.cache), and ran recognition on the
project's test image → "5AU5341" at 1.000 confidence, region "Czech Republic", ~40ms
on CPU, via both the CLI and POST /analyze.

Fixes result parsing against the actual fast-alpr API: ocr.confidence is a LIST of
per-character confidences (not a scalar) — reduced to one plate confidence via the MIN
(a plate is only as trustworthy as its weakest character); also surface ocr.region.
Extracted the per-result mapping into a pure plate_from_alpr_result + _reduce_confidence
and unit-tested them (no model weights needed). 7 tests pass; ruff + mypy strict clean;
full turbo build/lint/test green.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 15:46:03 +02:00
julian 6933406ae3 feat(vision): scaffold apps/vision ANPR microservice (FastAPI, stub recognizer)
Skeleton of the host-side vision service per the packaging decision: a Python/FastAPI
app at apps/vision/, uv-managed, wired into the Turbo graph via a thin package.json
shim (dev/lint/test/build → uv/uvicorn/ruff/pytest). A per-package turbo.json sets
build outputs [] so the no-op build is warning-free.

Endpoints: GET /health (readiness + model version) and POST /analyze (raw
octet-stream body, so Node POSTs Snapshot.bytes directly; empty→400, oversize→413,
recognizer-not-ready→503). The recognizer is a Protocol with a StubRecognizer (no
models, boots/tests offline — the dev/CI default) and a FastAlprRecognizer (the real
MIT YOLOv9+CCT/ONNX stack, lazily imported; missing models ⇒ ready=False, not a crash)
— the device-adapter pattern applied to the model. fast-alpr + onnxruntime are an
optional `alpr` extra, so `uv sync` needs no model download.

Verified: turbo run lint|test|build includes @parking/vision and stays green; uv run
mypy strict-clean; uvicorn boots and serves /health + /analyze live; pnpm workspace
6→7. Not built yet: the Node VisionClient adapter, a Dockerfile + model fetch, and
Job 2 (vehicle verification). Updates the packaging decision (As-scaffolded) + log.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 15:37:38 +02:00
julian ee28b7302f docs(wiki): decide vision service packaging — apps/vision/ in the monorepo
Settle WHERE the host-side ANPR service lives and how it joins the build: in this
monorepo at apps/vision/ (not a separate repo), still a separate OS process called
over localhost HTTP, wired into the Turbo graph via a thin package.json shim whose
scripts shell to Python tooling (uv/uvicorn/ruff/pytest). Co-located source honors the
vision-service runtime+license isolation decision (AGPL reach is a linking boundary,
not a folder); the fast-alpr MIT baseline removes most of the split-repo pressure
anyway. New page vision-service-packaging; updates vision-service, opencv-anpr-service,
the CLAUDE.md layout, index, log. Not built yet — packaging decision only.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
2026-06-19 14:25:31 +02:00
40 changed files with 2961 additions and 36 deletions
+2 -1
View File
@@ -17,7 +17,8 @@ parking-system/
├── turbo.json
├── apps/
│ ├── server/ # Fastify backend (device drivers, API, auth); serves the SPA
│ └── web/ # React + Vite SPA (operator UI)
│ ├── web/ # React + Vite SPA (operator UI)
│ └── vision/ # Python/FastAPI ANPR service (planned; separate process, Turbo shim — see wiki/decisions/vision-service-packaging.md)
├── packages/
│ ├── db/ # Drizzle ORM schema + migrations (SQLite local; PostgreSQL sync target)
│ ├── devices/ # device adapters behind shared interfaces (reader/printer/relay)
+11
View File
@@ -29,3 +29,14 @@ EVENT_SIGNING_KEY=
# Comma-separated extra origins allowed to open the booth WebSocket (/api/ws).
# In dev, set the Vite SPA origin. Same-origin is always allowed without this.
WS_ALLOWED_ORIGINS=http://localhost:5173
# Vision / ANPR (optional) -------------------------------------------------
# OFF by default. The Node SERVER's view of the vision microservice (apps/vision),
# which runs as a separate process with its OWN apps/vision/.env. Both sides share the
# VISION_ prefix but are different processes — keep the two .env files separate.
# See wiki/entities/opencv-anpr-service.md "Configuration".
# ANPR rides the entry/exit snapshot (button / QR / RFID triggers it) — no polling.
# VISION_ENABLED=1 # master switch — nothing runs without it
# VISION_URL=http://127.0.0.1:8089 # must match apps/vision VISION_HOST:VISION_PORT
# VISION_TIMEOUT_MS=1500 # per-request cap so a slow call can't hang the lane
# VISION_MIN_CONFIDENCE=0.5 # confidence floor; keep in sync with the service
+1 -1
View File
@@ -60,7 +60,7 @@ export interface PrinterStatusEvent {
export interface DeviceStatusEvent {
readonly deviceId: string; // devices id
readonly driverId: string;
readonly category: "access" | "reader" | "camera" | "printer";
readonly category: "access" | "reader" | "camera" | "printer" | "vision";
/**
* The device's ROLE descriptor for the footer label — NOT the vendor. A
* direction-style token the client localises and pairs with the category, so the
+41 -5
View File
@@ -3,6 +3,11 @@ import { devices, type Db, type DeviceRow } from "@parking/db";
import { isMonitorable, registry } from "@parking/devices";
import { deviceEvents, type DeviceStatusEvent } from "./device-events.js";
import { directionOf, relaysOf } from "./device-resolve.js";
import type { VisionClient } from "./vision-client.js";
/** Synthetic device id for the vision service in the status footer (it's a service,
* not a device row, but shares the footer's traffic-light + WS plumbing). */
const VISION_STATUS_ID = "vision-service";
// Unified live DEVICE monitor — the source for the booth's device-status footer.
// Every enabled, configured device is probed on an interval, regardless of
@@ -60,10 +65,15 @@ export class DeviceMonitor {
#timer: ReturnType<typeof setInterval> | null = null;
#ticking = false;
constructor(db: Db, log: FastifyBaseLogger, pollMs = POLL_MS) {
/** Optional: the vision service client. When present + enabled, the monitor probes
* its /health each tick and shows it as a "vision" chip in the footer. */
readonly #vision: VisionClient | null;
constructor(db: Db, log: FastifyBaseLogger, pollMs = POLL_MS, vision: VisionClient | null = null) {
this.#db = db;
this.#log = log;
this.#pollMs = pollMs;
this.#vision = vision;
}
/** Begin polling. Idempotent. */
@@ -97,12 +107,16 @@ export class DeviceMonitor {
const enabled = rows.filter((r) => r.enabled);
const present = new Set(enabled.map((r) => r.id));
// The vision service is a pseudo-device — keep it in the present set when enabled
// so the cleanup below doesn't evict it.
if (this.#vision?.enabled) present.add(VISION_STATUS_ID);
// Drop devices that are gone/disabled (so the footer doesn't show stale ones).
for (const id of [...this.#latest.keys()]) {
if (!present.has(id)) this.#latest.delete(id);
}
await Promise.all(enabled.map((r) => this.#poll(r)));
await Promise.all([...enabled.map((r) => this.#poll(r)), this.#pollVision()]);
} catch (err) {
this.#log.warn(`device-monitor tick failed: ${(err as Error).message}`);
} finally {
@@ -144,11 +158,33 @@ export class DeviceMonitor {
}
}
const prev = this.#latest.get(row.id);
this.#latest.set(row.id, next);
this.#publish(row.id, next);
}
/** Probe the vision service /health and publish it as a "vision" footer chip. Skipped
* entirely when no client is wired or it's disabled (no chip then). */
async #pollVision(): Promise<void> {
if (!this.#vision?.enabled) return;
const h = await this.#vision.health();
const state: DeviceStatusEvent["state"] = h.ok && h.ready ? "ready" : h.ready ? "degraded" : "offline";
this.#publish(VISION_STATUS_ID, {
deviceId: VISION_STATUS_ID,
driverId: "vision",
category: "vision",
roleKind: null,
state,
detail: h.ready ? h.recognizer : (h.detail ?? "not ready"),
checkedAt: new Date().toISOString(),
});
}
/** Cache + emit a status, but only when it CHANGED (state or detail). */
#publish(id: string, next: DeviceStatusEvent): void {
const prev = this.#latest.get(id);
this.#latest.set(id, next);
if (!prev || prev.state !== next.state || prev.detail !== next.detail) {
this.#log.info(
`device-monitor: ${next.category}/${next.roleKind ?? "—"} ${row.id} -> ${next.state}${next.detail ? ` (${next.detail})` : ""}`,
`device-monitor: ${next.category}/${next.roleKind ?? "—"} ${id} -> ${next.state}${next.detail ? ` (${next.detail})` : ""}`,
);
deviceEvents.emitDeviceStatus(next);
}
+7 -3
View File
@@ -17,6 +17,7 @@ import { getOccupancy } from "./occupancy.js";
import type { EventLog } from "./event-log.js";
import { devicesByDirection, relayForButton, relayForPresence, type ResolvedRelay } from "./device-resolve.js";
import { snapshotAsync } from "./snapshot.js";
import type { VisionClient } from "./vision-client.js";
// The transient ENTRY flow: a button press → print a ticket → sign a vehicle_entry
// → open the barrier. The button is wired into an access controller's input; the
@@ -69,11 +70,14 @@ export class EntryFlow {
readonly #inFlight = new Set<string>();
/** Per-relay one-car-one-ticket state (presence + cooldown), keyed controllerId:relay. */
readonly #guard = new Map<string, RelayGuardState>();
/** Optional vision client — passed to snapshotAsync so ANPR runs on the entry image. */
readonly #vision: VisionClient | null;
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger) {
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger, vision: VisionClient | null = null) {
this.#db = db;
this.#log = log;
this.#logger = logger;
this.#vision = vision;
}
/** Handle a device input edge. Two kinds of edge matter to this flow:
@@ -308,8 +312,8 @@ export class EntryFlow {
* Used on both the OPEN path and the refused/held anomaly paths — a turned-away or
* held car is exactly when the operator wants the photo. */
#fireSnapshot(direction: "entry", identity: string): void {
void snapshotAsync({ db: this.#db, direction, identity, logger: this.#logger }).catch((err) =>
this.#logger.error(`entry snapshot error: ${(err as Error).message}`),
void snapshotAsync({ db: this.#db, direction, identity, logger: this.#logger, vision: this.#vision }).catch(
(err) => this.#logger.error(`entry snapshot error: ${(err as Error).message}`),
);
}
+6 -1
View File
@@ -2,6 +2,7 @@ import { desc, eq, ledgerEvents, sessions, tariffVersions, tariffs, type Db, typ
import { registry, type AccessControlDevice } from "@parking/devices";
import { firstRelayByDirection, type ResolvedRelay } from "./device-resolve.js";
import { snapshotAsync } from "./snapshot.js";
import type { VisionClient } from "./vision-client.js";
import { computeFee, reasonPayload, renderReasonEn, type LedgerPayload, type TariffStructure } from "@parking/shared";
import type { FastifyBaseLogger } from "fastify";
import type { DeviceReadEvent, ReadOutcome } from "./device-events.js";
@@ -61,11 +62,14 @@ export class ExitFlow {
readonly #log: EventLog;
readonly #logger: FastifyBaseLogger;
readonly #inFlight = new Set<string>();
/** Optional vision client — passed to snapshotAsync so ANPR runs on the exit image. */
readonly #vision: VisionClient | null;
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger) {
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger, vision: VisionClient | null = null) {
this.#db = db;
this.#log = log;
this.#logger = logger;
this.#vision = vision;
}
/**
@@ -364,6 +368,7 @@ export class ExitFlow {
direction: "exit",
identity,
logger: this.#logger,
vision: this.#vision,
}).catch((err) => this.#logger.error(`exit snapshot error: ${(err as Error).message}`));
}
+38 -1
View File
@@ -68,7 +68,44 @@ export async function snapshotRoutes(app: FastifyInstance, db: Db): Promise<void
});
}
return { snapshots: rows, failures };
// Recognized PLATES for this session: kind="read" telemetry from the ANPR-on-
// snapshot path (snapshot.ts → recognizePlate). Advisory — a record of the plate
// observed for the session, shown beside the image. Newest first.
const plateRows = db
.select({ detail: deviceEvents.detail, occurredAt: deviceEvents.occurredAt })
.from(deviceEvents)
.where(and(eq(deviceEvents.category, "camera"), eq(deviceEvents.kind, "read")))
.orderBy(desc(deviceEvents.occurredAt))
.all();
const plates: {
plate: string;
confidence: number | null;
region: string | null;
direction: "entry" | "exit" | null;
snapshotId: string | null;
at: string;
}[] = [];
for (const row of plateRows) {
const d = (row.detail ?? {}) as {
identity?: string;
plate?: string;
confidence?: number;
region?: string | null;
direction?: string;
snapshotId?: string;
};
if (d.identity !== identity || !d.plate) continue;
plates.push({
plate: d.plate,
confidence: typeof d.confidence === "number" ? d.confidence : null,
region: d.region ?? null,
direction: d.direction === "entry" || d.direction === "exit" ? d.direction : null,
snapshotId: d.snapshotId ?? null,
at: row.occurredAt ?? "",
});
}
return { snapshots: rows, failures, plates };
},
);
+18 -6
View File
@@ -19,6 +19,7 @@ import { DeviceMonitor } from "./device-monitor.js";
import { buildSigner, buildVerifier } from "./signer.js";
import { LogService, pinoDbStream } from "./log-service.js";
import { logRoutes } from "./routes/logs.js";
import { VisionClient } from "./vision-client.js";
import { authRoutes } from "./routes/auth.js";
import { userRoutes } from "./routes/users.js";
import { roleRoutes } from "./routes/roles.js";
@@ -110,11 +111,17 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
app.addHook("onReady", async () => printerMonitor.start());
app.addHook("onClose", async () => printerMonitor.stop());
// Vision (ANPR) client — built early so the device monitor can include the vision
// service's health in the footer. Opt-in (VISION_ENABLED) + fail-soft; advisory only.
// See wiki/entities/opencv-anpr-service.md.
const visionClient = new VisionClient(app.log);
if (visionClient.enabled) app.log.info("vision client enabled");
// Unified device-status monitor: polls EVERY configured device (relays/readers/
// cameras via healthCheck, printers via rich readStatus) and feeds the booth's
// device-status footer over the WS. Read-only — never drives a relay.
// cameras via healthCheck, printers via rich readStatus) PLUS the vision service's
// /health, and feeds the booth's device-status footer over the WS. Read-only.
// See wiki/concepts/device-status-monitoring.md.
const deviceMonitor = new DeviceMonitor(db, app.log);
const deviceMonitor = new DeviceMonitor(db, app.log, undefined, visionClient);
await deviceStatusRoutes(app, deviceMonitor);
app.addHook("onReady", async () => deviceMonitor.start());
app.addHook("onClose", async () => deviceMonitor.stop());
@@ -143,7 +150,12 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
// Subscribes to the SAME input bus as the telemetry writer below; the two are
// independent (telemetry always records; the entry flow acts only on an access
// device's rising edge). See wiki/concepts/device-input-flow.md + parking-session.md.
const entryFlow = new EntryFlow(db, eventLog, app.log);
// The flows take the vision client so ANPR rides their entry/exit SNAPSHOT: a button
// press / QR / RFID triggers the open + snapshot, and the plate is recognized off that
// same image and recorded against the session (advisory; never changes the decision).
// No polling — recognition fires only on a real entry/exit. See snapshot.ts +
// wiki/entities/opencv-anpr-service.md.
const entryFlow = new EntryFlow(db, eventLog, app.log, visionClient);
const unsubscribeEntry = deviceEvents.onInput((e) => {
void entryFlow.onInput(e);
});
@@ -153,8 +165,8 @@ export async function buildServer(opts: BuildOptions = {}): Promise<FastifyInsta
// dispatcher to either the SUBSCRIPTION flow (if it matches a subscription) or the
// transient EXIT flow. See read-dispatch.ts, exit-flow.ts, subscription-flow.ts,
// parking-session.md.
const exitFlow = new ExitFlow(db, eventLog, app.log);
const subscriptionFlow = new SubscriptionFlow(db, eventLog, app.log);
const exitFlow = new ExitFlow(db, eventLog, app.log, visionClient);
const subscriptionFlow = new SubscriptionFlow(db, eventLog, app.log, visionClient);
const readDispatcher = new ReadDispatcher(db, exitFlow, subscriptionFlow, app.log);
const unsubscribeRead = deviceEvents.onRead((e) => {
void readDispatcher.dispatch(e);
+76 -1
View File
@@ -3,6 +3,7 @@ import { deviceEvents as deviceEventsTable, snapshots, type Db } from "@parking/
import { registry, type CameraDevice } from "@parking/devices";
import type { FastifyBaseLogger } from "fastify";
import { devicesByDirection, type FlowDirection } from "./device-resolve.js";
import type { VisionClient } from "./vision-client.js";
// Camera snapshot capture, fired AFTER the barrier opens and never awaited on the
// open path (decision 2026-06-16): a snapshot is EVIDENCE, not a gate. A camera
@@ -15,6 +16,16 @@ import { devicesByDirection, type FlowDirection } from "./device-resolve.js";
// → a `snapshots` row + a `kind:"snapshot"` telemetry device_event; a failure → a
// telemetry device_event only. The caller passes the session `identity` so the image
// links to the signed vehicle_entry/exit.
//
// ANPR rides this snapshot (2026-06-19). A transient button-press or a subscriber
// QR/RFID read triggers the entry/exit, which fires THIS snapshot — that is exactly the
// moment to recognize the plate, off the SAME image, tied to the SAME session identity.
// So when a `vision` client is passed AND the camera opts in (config.anpr), each stored
// snapshot is sent to the vision service and the extracted plate is RECORDED against the
// session (a `kind:"read"` device_event with plate/confidence/snapshotId). ADVISORY +
// fire-and-forget: it never blocks the open and never changes the entry/exit decision —
// it's a record ("session X entered on plate AA558EE"). No polling; recognition only
// happens on a real entry/exit. See wiki/entities/opencv-anpr-service.md.
interface SnapshotJob {
readonly db: Db;
@@ -22,6 +33,15 @@ interface SnapshotJob {
/** Session/credential ref (ticket id, plate, subscription car key) — links to the ledger. */
readonly identity: string;
readonly logger: FastifyBaseLogger;
/** Optional vision client — when present, ANPR runs on each captured image from an
* `anpr`-enabled camera and records the plate against `identity`. Advisory only. */
readonly vision?: VisionClient | null;
}
/** Camera config flag opting it into snapshot-triggered ANPR. */
interface CameraConfig {
readonly anpr?: boolean;
readonly [k: string]: unknown;
}
/**
@@ -30,7 +50,7 @@ interface SnapshotJob {
* The caller must NOT block its open path on this.
*/
export function snapshotAsync(job: SnapshotJob): Promise<string[]> {
const { db, direction, identity, logger } = job;
const { db, direction, identity, logger, vision } = job;
const rows = devicesByDirection(db, "camera", direction);
if (rows.length === 0) return Promise.resolve([]);
@@ -57,6 +77,12 @@ export function snapshotAsync(job: SnapshotJob): Promise<string[]> {
.run();
// Telemetry breadcrumb pointing at the stored image (NOT the bytes).
recordEvent(db, direction, row.id, identity, { snapshotId: id, ok: true }, logger);
// ANPR off the SAME image, tied to the SAME session — when vision is enabled
// and this camera opts in. Fire-and-forget: never delays the open path.
if (vision?.enabled && (row.config as CameraConfig)?.anpr === true) {
void recognizePlate(db, vision, row.id, direction, identity, id, shot, logger);
}
return id;
} catch (err) {
recordFailure(db, direction, row.id, identity, (err as Error).message, logger);
@@ -66,6 +92,55 @@ export function snapshotAsync(job: SnapshotJob): Promise<string[]> {
).then((ids) => ids.filter((id): id is string => id != null));
}
/**
* Recognize the plate off a captured entry/exit image and RECORD it against the session
* `identity` — an unsigned `kind:"read"` device_event carrying the plate, confidence,
* region, and the `snapshotId` it was read from. Advisory: this records the plate
* observed for the session; it does NOT feed the access decision (the flow already
* decided). A low-confidence/no-plate result records nothing (a shaky read isn't a fact).
* Best-effort + fail-soft — a vision error never surfaces on the (already-open) path.
*/
async function recognizePlate(
db: Db,
vision: VisionClient,
deviceId: string,
direction: FlowDirection,
identity: string,
snapshotId: string,
shot: { bytes: Buffer; contentType: string },
logger: FastifyBaseLogger,
): Promise<void> {
try {
const result = await vision.analyze(shot.bytes, shot.contentType);
if (!result || !result.plate || result.lowConfidence) return; // nothing trustworthy to record
const plate = result.plate.text.trim().toUpperCase();
if (!plate) return;
db.insert(deviceEventsTable)
.values({
id: randomUUID(),
deviceId,
category: "camera",
kind: "read",
// `identity` ties the plate to the session; `snapshotId` to the evidence image.
detail: {
identity,
direction,
plate,
confidence: result.plate.confidence,
region: result.plate.region ?? null,
modelVersion: result.modelVersion,
snapshotId,
source: "entry-exit-snapshot",
},
occurredAt: new Date().toISOString(),
})
.run();
logger.info(`anpr plate '${plate}' (${result.plate.confidence.toFixed(3)}) for ${identity}`);
} catch (err) {
logger.warn(`anpr recognize failed (${identity}): ${(err as Error).message}`);
}
}
/** Build a live camera adapter from a resolved devices row, or null. */
function buildCamera(row: { driverId: string; config: unknown }): CameraDevice | null {
const driver = registry.get(row.driverId);
+7 -3
View File
@@ -16,6 +16,7 @@ import type { DeviceReadEvent, ReadOutcome } from "./device-events.js";
import type { EventLog } from "./event-log.js";
import { type FlowDirection, type ResolvedRelay } from "./device-resolve.js";
import { snapshotAsync } from "./snapshot.js";
import type { VisionClient } from "./vision-client.js";
// SUBSCRIPTION flow: a subscriber identified by card/QR/plate enters/exits without
// paying per stay (they're on a recurring plan). Reached from the read dispatcher
@@ -55,11 +56,14 @@ export class SubscriptionFlow {
readonly #log: EventLog;
readonly #logger: FastifyBaseLogger;
readonly #inFlight = new Set<string>();
/** Optional vision client — passed to snapshotAsync so ANPR runs on the subscriber image. */
readonly #vision: VisionClient | null;
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger) {
constructor(db: Db, log: EventLog, logger: FastifyBaseLogger, vision: VisionClient | null = null) {
this.#db = db;
this.#log = log;
this.#logger = logger;
this.#vision = vision;
}
/** Resolve a read to a subscription (by card/QR credential, or a bound plate), or null. */
@@ -256,8 +260,8 @@ export class SubscriptionFlow {
/** Fire the directional camera(s) for a refused-subscription event; never awaited
* (evidence, not a gate). The accepted entry/exit paths snapshot inside #open. */
#fireSnapshot(dir: FlowDirection, identity: string): void {
void snapshotAsync({ db: this.#db, direction: dir, identity, logger: this.#logger }).catch((err) =>
this.#logger.error(`subscription snapshot error: ${(err as Error).message}`),
void snapshotAsync({ db: this.#db, direction: dir, identity, logger: this.#logger, vision: this.#vision }).catch(
(err) => this.#logger.error(`subscription snapshot error: ${(err as Error).message}`),
);
}
+193
View File
@@ -0,0 +1,193 @@
import type { FastifyBaseLogger } from "fastify";
// Node-side client for the host vision service (apps/vision — the ANPR microservice).
// Calls it over LOCALHOST HTTP with a camera snapshot and gets back a plate read. The
// Python service is a separate process/failure domain; this client is the adapter the
// rest of the server talks to, so the recognizer is swappable without business-logic
// changes. See wiki/entities/opencv-anpr-service.md, decisions/vision-service*.md.
//
// ADVISORY, NEVER SOLE AUTHORITY. Per the vision decision + the fitness assessment, a
// plate read is an *identity hint + evidence*, never the lone reason a paid/access
// barrier opens. This client enforces two things at the boundary so callers can't
// misuse it:
// 1. It is FAIL-SOFT — any error (service down, timeout, decode fail) resolves to
// `null`, never throws into the entry/exit path. A missing vision result must
// degrade to the ticket/manual path, never strand or wrongly admit a car
// (fail-state-safety).
// 2. It applies the CONFIDENCE FLOOR — a read below the threshold is returned with
// `lowConfidence: true` (mirroring the service's own flag) so the caller treats it
// as advisory-only and falls back.
//
// NOT yet wired into the read bus — that (snapshot-before-decision on an opt-in camera →
// emit DeviceReadEvent{kind:"plate"}) is a separate, deliberate step. This is the
// transport + contract adapter only.
/** Plate bounding box (pixels, top-left origin) — mirrors the service schema. */
export interface PlateBBox {
readonly x1: number;
readonly y1: number;
readonly x2: number;
readonly y2: number;
}
/** One plate read from the vision service. `confidence` is the MIN of the model's
* per-character confidences (a plate is only as trustworthy as its weakest char). */
export interface VisionPlate {
readonly text: string;
readonly confidence: number;
readonly bbox?: PlateBBox | null;
/** Predicted issuing region/country (advisory; the global model emits this). */
readonly region?: string | null;
}
/** The raw /analyze response shape (the Python contract). `vehicle` is reserved for
* Job 2 (vehicle verification) — not yet produced. */
interface AnalyzeResponse {
readonly plate: VisionPlate | null;
readonly plates: VisionPlate[];
readonly vehicle: unknown | null;
readonly low_confidence: boolean;
readonly model_version: string;
readonly took_ms: number;
}
/** What the rest of the server gets back from `analyze()`. Normalised + camelCased,
* with the advisory gate already applied. Never thrown — `null` on any failure. */
export interface VisionResult {
/** The best plate, or null if none read. */
readonly plate: VisionPlate | null;
/** All plates found in the frame (a frame may hold several vehicles). */
readonly plates: VisionPlate[];
/** True when the best plate is below the confidence floor — treat as advisory only
* and fall back to the ticket/manual path. */
readonly lowConfidence: boolean;
readonly modelVersion: string;
readonly tookMs: number;
}
export interface VisionHealth {
readonly ok: boolean;
readonly recognizer: string;
readonly ready: boolean;
readonly modelVersion: string;
readonly detail?: string | null;
}
export interface VisionClientOptions {
/** Base URL of the vision service (localhost). */
readonly baseUrl?: string;
/** Per-request timeout (ms) — a slow vision call must never hang the lane. */
readonly timeoutMs?: number;
/** Confidence floor: a best-plate below this is flagged lowConfidence. Mirrors the
* service's own VISION_MIN_CONFIDENCE; kept here too so the gate holds even if the
* service is misconfigured. */
readonly minConfidence?: number;
/** Master switch — when false, `analyze()` short-circuits to null (no call). Lets the
* appliance run with no vision service configured. */
readonly enabled?: boolean;
}
export class VisionClient {
readonly #baseUrl: string;
readonly #timeoutMs: number;
readonly #minConfidence: number;
readonly #enabled: boolean;
readonly #logger: FastifyBaseLogger;
constructor(logger: FastifyBaseLogger, opts: VisionClientOptions = {}) {
this.#logger = logger;
this.#baseUrl = (opts.baseUrl ?? process.env.VISION_URL ?? "http://127.0.0.1:8089").replace(/\/$/, "");
this.#timeoutMs = opts.timeoutMs ?? Number(process.env.VISION_TIMEOUT_MS ?? 1500);
this.#minConfidence = opts.minConfidence ?? Number(process.env.VISION_MIN_CONFIDENCE ?? 0.5);
// Default OFF: vision is opt-in. Enable with VISION_ENABLED=1 (or pass enabled:true).
this.#enabled =
opts.enabled ?? ["1", "true", "yes"].includes((process.env.VISION_ENABLED ?? "").toLowerCase());
}
get enabled(): boolean {
return this.#enabled;
}
/**
* Analyse snapshot bytes → a plate read, or `null`. NEVER throws and NEVER blocks the
* caller's open path beyond `timeoutMs`: any failure (disabled, unreachable, timeout,
* non-2xx, bad body) logs and resolves to null, so the caller falls back to the
* ticket/manual path. The returned `lowConfidence` re-applies the floor on top of the
* service's own flag.
*/
async analyze(imageBytes: Buffer, contentType = "application/octet-stream"): Promise<VisionResult | null> {
if (!this.#enabled) return null;
try {
const body = await this.#post("/analyze", imageBytes, contentType);
if (!body) return null;
const res = body as AnalyzeResponse;
const best = res.plate ?? null;
const lowConfidence =
res.low_confidence || (best != null && best.confidence < this.#minConfidence);
return {
plate: best,
plates: Array.isArray(res.plates) ? res.plates : [],
lowConfidence,
modelVersion: res.model_version ?? "unknown",
tookMs: typeof res.took_ms === "number" ? res.took_ms : 0,
};
} catch (err) {
this.#logger.warn(`vision analyze failed (fallback to ticket path): ${(err as Error).message}`);
return null;
}
}
/** Liveness/readiness of the vision service. Returns ok:false (never throws) when
* disabled or unreachable, so the device-status footer can show it. */
async health(): Promise<VisionHealth> {
if (!this.#enabled) {
return { ok: false, recognizer: "disabled", ready: false, modelVersion: "-", detail: "vision disabled" };
}
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), this.#timeoutMs);
try {
const r = await fetch(`${this.#baseUrl}/health`, { signal: controller.signal });
if (!r.ok) return { ok: false, recognizer: "?", ready: false, modelVersion: "-", detail: `HTTP ${r.status}` };
const h = (await r.json()) as {
status?: string;
recognizer?: string;
ready?: boolean;
model_version?: string;
detail?: string | null;
};
return {
ok: h.status === "ok",
recognizer: h.recognizer ?? "?",
ready: Boolean(h.ready),
modelVersion: h.model_version ?? "-",
detail: h.detail ?? null,
};
} finally {
clearTimeout(timer);
}
} catch (err) {
return { ok: false, recognizer: "?", ready: false, modelVersion: "-", detail: (err as Error).message };
}
}
/** POST raw bytes to a path, with timeout. Returns parsed JSON or throws (caught by
* the caller, which fails soft). */
async #post(path: string, bytes: Buffer, contentType: string): Promise<unknown> {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), this.#timeoutMs);
try {
const r = await fetch(`${this.#baseUrl}${path}`, {
method: "POST",
headers: { "content-type": contentType },
// Buffer is a valid BodyInit in Node's undici fetch.
body: bytes,
signal: controller.signal,
});
if (!r.ok) throw new Error(`vision ${path} → HTTP ${r.status}`);
return await r.json();
} finally {
clearTimeout(timer);
}
}
}
+22
View File
@@ -0,0 +1,22 @@
# apps/vision — the ANPR microservice's own env (copy to apps/vision/.env).
# This is the PYTHON SERVICE's config only. The Node server has its OWN VISION_* vars
# (in apps/server/.env) — keep the two .env files SEPARATE (they share the VISION_
# prefix but are different processes). See wiki/entities/opencv-anpr-service.md "Configuration".
# Recognizer: "stub" (no models, recognizes nothing — boots anywhere, for dev/CI) or
# "fast_alpr" (the real MIT YOLOv9+CCT/ONNX stack — needs `uv sync --extra alpr`).
VISION_RECOGNIZER=fast_alpr
# Bind. On the appliance prefer 127.0.0.1 — the Node backend is the only caller, so the
# /analyze endpoint should NOT be reachable off-host. (0.0.0.0 only if you must.)
VISION_HOST=127.0.0.1
VISION_PORT=8089
# fast-alpr models (only used when recognizer=fast_alpr). The defaults won the Albanian
# benchmark; change the OCR to european-plates-mobile-vit-v2-model only to re-test.
VISION_DETECTOR_MODEL=yolo-v9-t-384-license-plate-end2end
VISION_OCR_MODEL=cct-xs-v2-global-model
# Confidence floor — a best plate below this is flagged low_confidence so the Node side
# treats it as advisory and falls back to the ticket path. Keep in sync with the server.
VISION_MIN_CONFIDENCE=0.5
+11
View File
@@ -0,0 +1,11 @@
# Python
__pycache__/
*.py[cod]
.venv/
.mypy_cache/
.pytest_cache/
.ruff_cache/
# Model weights (fetched at deploy / first run, never committed — can be large + license-scoped)
models/
*.onnx
+1
View File
@@ -0,0 +1 @@
3.12
+77
View File
@@ -0,0 +1,77 @@
# @parking/vision — host-side ANPR / vehicle-verification service
A **separate process** (Python + FastAPI) the Node backend calls over **localhost HTTP** with a
camera snapshot, returning a licence-plate read (and, later, vehicle-attribute verification — the
anti-plate-spoofing witness). Recognition is **advisory, never the sole authority** to open a
barrier: if this service is down or unsure, the host falls back to the ticket path.
Lives inside the Turborepo at `apps/vision/` but is **not a JS package** — Python deps are managed by
`uv`/`pyproject.toml`; the `package.json` is a thin shim so `turbo run lint/test` includes it. See
`wiki/decisions/vision-service-packaging.md` and `wiki/entities/opencv-anpr-service.md`.
## Run
```bash
# from apps/vision/ — install the light core (boots in stub mode, no model downloads)
uv sync
# dev server with reload (or: pnpm --filter @parking/vision dev)
uv run uvicorn vision_service.app:app --reload --port 8089
# checks
uv run ruff check .
uv run pytest -q
```
### Enable the real recognizer (fast-alpr)
```bash
uv sync --extra alpr # installs fast-alpr + onnxruntime (downloads model weights)
VISION_RECOGNIZER=fast_alpr uv run uvicorn vision_service.app:app --port 8089
```
Model weights (~11 MB: a YOLOv9 detector + CCT OCR) download on first use and cache under
`~/.cache/open-image-models` + `~/.cache/fast-plate-ocr` — offline after that.
### Quick test against an image (CLI, no HTTP)
```bash
uv run python -m vision_service.cli path/to/car.jpg # or: pnpm --filter @parking/vision recognize -- car.jpg
uv run python -m vision_service.cli car.jpg --ocr cct-s-v2-global-model # try another OCR model
```
Prints the parsed plate(s) + confidence + region as JSON. Confidence is the **min** of fast-alpr's
per-character confidences (a plate is only as trustworthy as its weakest character). Example output on
the fast-alpr test image: `5AU5341 (1.000) region "Czech Republic"` in ~40 ms on CPU.
`fast-alpr` is MIT (YOLOv9 detector + CCT OCR on ONNX Runtime). Swap `VISION_OCR_MODEL` to the 40+
country European model to benchmark Albanian plates. For GPU/NPU, install `onnxruntime-gpu` /
`-openvino` / `-directml` instead of `onnxruntime`.
## API
- `GET /health` → `{ status, recognizer, ready, model_version, detail? }`
- `POST /analyze` (body = raw image bytes, `Content-Type: application/octet-stream`) →
`{ plate: {text, confidence, bbox}|null, plates[], vehicle: null, low_confidence, model_version, took_ms }`
The Node side POSTs `Snapshot.bytes` directly (no multipart). `vehicle` is scaffolded but not yet
populated — fast-alpr is plate-only; the vehicle stage (Job 2) is built later on the same runtime.
## Config (env, prefix `VISION_`) — see `.env.example`
This service's env only. The **Node server has its own `VISION_*`** (`apps/server/.env`:
`VISION_ENABLED`, `VISION_URL`, `VISION_POLL_MS`, …) — same prefix, **separate process, separate
`.env`**. Don't merge them.
| Var | Default | Meaning |
| --- | --- | --- |
| `VISION_RECOGNIZER` | `stub` | `stub` (no models) or `fast_alpr` (real) |
| `VISION_HOST` | `0.0.0.0` | bind address — prefer `127.0.0.1` on the appliance (Node is the only caller) |
| `VISION_PORT` | `8089` | listen port (must match the server's `VISION_URL`) |
| `VISION_DETECTOR_MODEL` | `yolo-v9-t-384-license-plate-end2end` | fast-alpr detector |
| `VISION_OCR_MODEL` | `cct-xs-v2-global-model` | fast-alpr OCR (won the AL benchmark) |
| `VISION_MIN_CONFIDENCE` | `0.5` | below this → `low_confidence=true` |
To use it from the booth: set `VISION_ENABLED=1` on the **server**, run this service, then tick
**ANPR** on a camera in the SetupWizard (the camera must also be bound to a barrier). The booth footer
shows a **Vision** chip when enabled. Full config guide: `wiki/entities/opencv-anpr-service.md`.
+16
View File
@@ -0,0 +1,16 @@
{
"name": "@parking/vision",
"version": "0.0.0",
"private": true,
"//": "Thin shim so this Python service is a first-class node in the Turbo task graph (it is NOT a JS package — deps are managed by uv/pyproject.toml). Each script shells to Python tooling. See wiki/decisions/vision-service-packaging.md.",
"scripts": {
"dev": "uv run uvicorn vision_service.app:app --reload --host 0.0.0.0 --port 8089",
"start": "uv run uvicorn vision_service.app:app --host 0.0.0.0 --port 8089",
"lint": "uv run ruff check .",
"format": "uv run ruff format .",
"typecheck": "uv run mypy vision_service",
"test": "uv run pytest -q",
"recognize": "uv run python -m vision_service.cli",
"build": "echo 'no build step (Python service; models fetched at deploy)'"
}
}
+62
View File
@@ -0,0 +1,62 @@
[project]
name = "parking-vision"
version = "0.0.0"
description = "Host-side ANPR / vehicle-verification microservice for the parking system (separate process; localhost HTTP)."
requires-python = ">=3.10,<4.0"
# Core deps are LIGHT on purpose: the service boots, serves /health, and answers
# /analyze in stub mode with ONLY these. The heavy recognizer stack (fast-alpr +
# onnxruntime + model weights) is the optional `alpr` extra, so `uv sync` and the test
# suite work offline without downloading models. See
# wiki/decisions/vision-service-packaging.md + wiki/entities/opencv-anpr-service.md.
dependencies = [
"fastapi>=0.115",
"uvicorn[standard]>=0.32",
"pydantic>=2.9",
"pydantic-settings>=2.6",
]
[project.scripts]
vision-recognize = "vision_service.cli:main"
[project.optional-dependencies]
# The real recognizer. Install with: uv sync --extra alpr
# fast-alpr is MIT (YOLOv9 detector + CCT OCR, both MIT) on ONNX Runtime — see the
# recognizer evaluation in wiki/entities/opencv-anpr-service.md. onnxruntime is the
# CPU backend; swap for onnxruntime-gpu / -openvino / -directml on capable hardware.
alpr = [
"fast-alpr>=0.4.0",
"onnxruntime>=1.19",
]
[dependency-groups]
# Dev tooling (uv installs these by default for local work; excluded from the runtime image).
dev = [
"ruff>=0.8",
"pytest>=8.3",
"httpx>=0.27", # FastAPI TestClient transport
"mypy>=1.13",
]
[tool.ruff]
line-length = 110
target-version = "py310"
[tool.ruff.lint]
# A pragmatic default set: pyflakes, pycodestyle, isort, bugbear, pyupgrade.
select = ["E", "F", "I", "B", "UP"]
[tool.pytest.ini_options]
testpaths = ["tests"]
[tool.mypy]
python_version = "3.10"
strict = true
# fast-alpr / onnxruntime ship without type stubs; don't fail typecheck on the optional stack.
ignore_missing_imports = true
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["vision_service"]
View File
+51
View File
@@ -0,0 +1,51 @@
"""Smoke tests for the vision service in STUB mode (no model weights needed)."""
from __future__ import annotations
from fastapi.testclient import TestClient
from vision_service.app import app
def make_client() -> TestClient:
# TestClient runs the lifespan, building the (stub) recognizer.
return TestClient(app)
def test_health_ok_in_stub_mode() -> None:
with make_client() as client:
res = client.get("/health")
assert res.status_code == 200
body = res.json()
assert body["status"] == "ok"
assert body["ready"] is True
assert body["recognizer"] == "stub"
assert body["model_version"] == "stub-0"
def test_analyze_returns_contract_shape() -> None:
with make_client() as client:
# The stub recognizes nothing, but the response must match the contract.
res = client.post(
"/analyze",
content=b"\xff\xd8\xff\xe0not-a-real-jpeg",
headers={"content-type": "application/octet-stream"},
)
assert res.status_code == 200
body = res.json()
assert body["plate"] is None
assert body["plates"] == []
assert body["vehicle"] is None
assert body["low_confidence"] is False
assert body["model_version"] == "stub-0"
assert "took_ms" in body
def test_analyze_rejects_empty_body() -> None:
with make_client() as client:
res = client.post(
"/analyze",
content=b"",
headers={"content-type": "application/octet-stream"},
)
assert res.status_code == 400
+43
View File
@@ -0,0 +1,43 @@
"""Unit tests for fast-alpr result parsing — no model weights required (the objects
are duck-typed stand-ins shaped like fast-alpr's ALPRResult)."""
from __future__ import annotations
from types import SimpleNamespace
from vision_service.recognizer import _reduce_confidence, plate_from_alpr_result
def test_reduce_confidence_takes_min_of_list() -> None:
# The weakest character governs trust in the whole plate.
assert _reduce_confidence([0.99, 0.80, 0.95]) == 0.80
def test_reduce_confidence_handles_scalar_and_junk() -> None:
assert _reduce_confidence(0.7) == 0.7
assert _reduce_confidence(None) == 0.0
assert _reduce_confidence([]) == 0.0
assert _reduce_confidence("nope") == 0.0
def _fake_result(text: str, conf: list[float], region: str | None = None) -> SimpleNamespace:
box = SimpleNamespace(x1=10, y1=20, x2=110, y2=60)
return SimpleNamespace(
ocr=SimpleNamespace(text=text, confidence=conf, region=region),
detection=SimpleNamespace(bounding_box=box),
)
def test_plate_from_result_maps_fields() -> None:
plate = plate_from_alpr_result(_fake_result("5AU5341", [0.999, 0.9995, 0.97], "Czech Republic"))
assert plate is not None
assert plate.text == "5AU5341"
assert plate.confidence == 0.97 # min of the per-character list
assert plate.region == "Czech Republic"
assert plate.bbox is not None
assert (plate.bbox.x1, plate.bbox.y1, plate.bbox.x2, plate.bbox.y2) == (10, 20, 110, 60)
def test_plate_from_result_skips_empty_text() -> None:
assert plate_from_alpr_result(_fake_result("", [0.9])) is None
assert plate_from_alpr_result(SimpleNamespace(ocr=None, detection=None)) is None
+9
View File
@@ -0,0 +1,9 @@
{
"$schema": "https://turbo.build/schema.json",
"extends": ["//"],
"tasks": {
"build": {
"outputs": []
}
}
}
+1466
View File
File diff suppressed because it is too large Load Diff
+9
View File
@@ -0,0 +1,9 @@
"""Host-side ANPR / vehicle-verification microservice.
A separate process (FastAPI over localhost HTTP) that the Node backend calls with a
camera snapshot and gets back a plate read (Job 1) — and, later, vehicle-attribute
verification (Job 2, the anti-spoofing witness). Recognition is ADVISORY, never the
sole authority to open a barrier. See wiki/entities/opencv-anpr-service.md.
"""
__version__ = "0.0.0"
+88
View File
@@ -0,0 +1,88 @@
"""FastAPI app: POST /analyze (snapshot → plate) + GET /health.
Called by the Node backend over localhost HTTP (the camera driver already holds the
JPEG bytes — Snapshot.bytes). This service is a SEPARATE PROCESS with its own failure
domain: if it's down or unsure, the host falls back to the ticket path — recognition is
advisory, never the sole authority. See wiki/entities/opencv-anpr-service.md.
"""
from __future__ import annotations
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from fastapi import FastAPI, HTTPException, Request
from .recognizer import Recognizer, build_recognizer
from .schemas import AnalyzeResponse, HealthResponse
from .settings import Settings, get_settings
# Cap an upload so a malformed/huge POST can't exhaust memory (a camera JPEG is well
# under this). 413 beyond it.
MAX_IMAGE_BYTES = 12 * 1024 * 1024
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
settings = get_settings()
app.state.settings = settings
# Build the recognizer once at startup (models load here, not per-request).
app.state.recognizer = build_recognizer(settings)
yield
app = FastAPI(title="parking-vision", version="0.0.0", lifespan=lifespan)
# Typed accessors over the untyped `app.state` (so mypy --strict sees the real types).
def _recognizer(request: Request) -> Recognizer:
rec: Recognizer = request.app.state.recognizer
return rec
def _settings(request: Request) -> Settings:
settings: Settings = request.app.state.settings
return settings
@app.get("/health", response_model=HealthResponse)
async def health(request: Request) -> HealthResponse:
rec = _recognizer(request)
settings = _settings(request)
ready = bool(rec.ready)
return HealthResponse(
status="ok" if ready else "degraded",
recognizer=settings.recognizer,
ready=ready,
model_version=rec.model_version,
detail=getattr(rec, "error", None),
)
@app.post("/analyze", response_model=AnalyzeResponse)
async def analyze(request: Request) -> AnalyzeResponse:
"""Analyze raw image bytes (the camera JPEG). Body is the octet-stream itself, so
the Node side POSTs Snapshot.bytes directly with Content-Type:
application/octet-stream — no multipart wrapping. We read the raw body ourselves
(rather than a required Body param) so an empty/oversize body returns our own clean
400/413 instead of FastAPI's generic 422."""
image = await request.body()
if not image:
raise HTTPException(status_code=400, detail="empty image body")
if len(image) > MAX_IMAGE_BYTES:
raise HTTPException(status_code=413, detail="image too large")
rec = _recognizer(request)
if not rec.ready:
# The real recognizer failed to load — be explicit so Node falls back rather
# than treating a silent empty result as "no plate present".
raise HTTPException(
status_code=503,
detail=f"recognizer not ready: {getattr(rec, 'error', 'unavailable')}",
)
try:
return rec.analyze(image)
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
except Exception as exc: # noqa: BLE001 - never leak a stack to the caller
raise HTTPException(status_code=500, detail=f"analysis failed: {exc}") from exc
+72
View File
@@ -0,0 +1,72 @@
"""Dev CLI to test a recognizer against an image file — no HTTP, fast feedback.
uv run python -m vision_service.cli path/to/car.jpg
uv run python -m vision_service.cli car.jpg --recognizer stub # contract only
uv run python -m vision_service.cli car.jpg --ocr cct-s-v2-global-model
Defaults to the `fast_alpr` recognizer (the point of this tool). Prints the parsed
plate result as JSON. If the `alpr` extra isn't installed it says so and exits non-zero
rather than silently using the stub. See wiki/entities/opencv-anpr-service.md.
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
from .recognizer import build_recognizer
from .settings import Settings
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="vision-recognize", description="Run a recognizer on an image.")
parser.add_argument("image", type=Path, help="path to an image file (JPEG/PNG) with a plate")
parser.add_argument(
"--recognizer",
choices=["fast_alpr", "stub"],
default="fast_alpr",
help="which recognizer to use (default: fast_alpr)",
)
parser.add_argument("--detector", default=None, help="override the fast-alpr detector model name")
parser.add_argument("--ocr", default=None, help="override the fast-alpr OCR model name")
args = parser.parse_args(argv)
if not args.image.is_file():
print(f"error: no such file: {args.image}", file=sys.stderr)
return 2
settings = Settings(recognizer=args.recognizer)
if args.detector:
settings.detector_model = args.detector
if args.ocr:
settings.ocr_model = args.ocr
rec = build_recognizer(settings)
if not rec.ready:
err = getattr(rec, "error", "unavailable")
print(
f"error: recognizer '{args.recognizer}' not ready: {err}\n"
"hint: install the models with uv sync --extra alpr",
file=sys.stderr,
)
return 1
image_bytes = args.image.read_bytes()
result = rec.analyze(image_bytes)
# Pydantic v2: model_dump_json gives a clean, stable rendering.
print(result.model_dump_json(indent=2))
if result.plate is None:
print("\n(no plate detected)", file=sys.stderr)
else:
flag = " [LOW CONFIDENCE]" if result.low_confidence else ""
print(
f"\n→ {result.plate.text} ({result.plate.confidence:.3f}){flag} in {result.took_ms:.1f} ms",
file=sys.stderr,
)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+174
View File
@@ -0,0 +1,174 @@
"""The recognizer port + implementations.
The service depends on the `Recognizer` PROTOCOL, never a concrete model library — the
same swappable-behind-an-interface principle as the Node device adapters
(wiki/concepts/device-adapter-pattern.md). Two impls today:
- StubRecognizer: no model weights, deterministic placeholder. Lets the service boot
and the tests run offline with nothing downloaded (dev/CI default).
- FastAlprRecognizer: the real MIT YOLOv9-detector + CCT-OCR stack on ONNX Runtime
(the `alpr` extra). See wiki/entities/opencv-anpr-service.md "Recognizer evaluation".
Adding a recognizer (e.g. a fine-tuned YOLO + PaddleOCR) = a new class here, no app change.
"""
from __future__ import annotations
import time
from typing import Protocol
from .schemas import AnalyzeResponse, BBox, PlateResult
from .settings import Settings
class Recognizer(Protocol):
"""Reads plates from a JPEG/PNG image. Implementations must be process-local and offline."""
@property
def model_version(self) -> str: ...
@property
def ready(self) -> bool: ...
def analyze(self, image_bytes: bytes) -> AnalyzeResponse: ...
def _reduce_confidence(raw: object) -> float:
"""fast-alpr's OCR confidence is a LIST of per-character confidences. Reduce to one
plate confidence via the MIN — a plate is only as trustworthy as its weakest
character (one misread digit changes the identity). Tolerates a scalar (future
models) or junk (→ 0.0). Pure + model-free so it's unit-testable without weights."""
if isinstance(raw, (list, tuple)) and raw:
try:
return float(min(raw))
except (TypeError, ValueError):
return 0.0
if isinstance(raw, (int, float)):
return float(raw)
return 0.0
def plate_from_alpr_result(r: object) -> PlateResult | None:
"""Map ONE fast-alpr ALPRResult to our PlateResult, or None if it carries no text.
Uses getattr throughout so it's decoupled from the exact fast-alpr classes (and
testable with a duck-typed stand-in). See wiki/entities/opencv-anpr-service.md."""
ocr = getattr(r, "ocr", None)
det = getattr(r, "detection", None)
text = getattr(ocr, "text", None)
if ocr is None or not text:
return None
bbox = None
box = getattr(det, "bounding_box", None)
if box is not None:
bbox = BBox(x1=int(box.x1), y1=int(box.y1), x2=int(box.x2), y2=int(box.y2))
return PlateResult(
text=text,
confidence=_reduce_confidence(getattr(ocr, "confidence", None)),
bbox=bbox,
region=getattr(ocr, "region", None),
)
class StubRecognizer:
"""A no-model placeholder. Returns an empty (no-plate) result quickly so the whole
HTTP path — Node adapter, contract, error handling — can be exercised without the
heavy recognizer stack or any model download."""
model_version = "stub-0"
ready = True
def __init__(self, settings: Settings) -> None:
self._settings = settings
def analyze(self, image_bytes: bytes) -> AnalyzeResponse:
started = time.perf_counter()
# Deliberately recognizes nothing — it is a stub, not a fake "always finds a plate"
# (which would be dangerous: recognition must never invent an identity).
took_ms = (time.perf_counter() - started) * 1000.0
return AnalyzeResponse(
plate=None,
plates=[],
vehicle=None,
low_confidence=False,
model_version=self.model_version,
took_ms=took_ms,
)
class FastAlprRecognizer:
"""The real recognizer: fast-alpr (YOLOv9 plate detector + CCT OCR, ONNX Runtime).
Imported lazily so the service still imports/boots in stub mode when the `alpr`
extra (and its model weights) are not installed — a missing recognizer must not
crash the process; it degrades to a clear `ready=False`.
"""
def __init__(self, settings: Settings) -> None:
self._settings = settings
self._alpr = None
self._error: str | None = None
try:
from fast_alpr import ALPR
self._alpr = ALPR(
detector_model=settings.detector_model,
ocr_model=settings.ocr_model,
)
except Exception as exc: # noqa: BLE001 - any failure ⇒ not-ready, surfaced via /health
self._error = f"{type(exc).__name__}: {exc}"
@property
def model_version(self) -> str:
return f"fast-alpr:{self._settings.detector_model}+{self._settings.ocr_model}"
@property
def ready(self) -> bool:
return self._alpr is not None
@property
def error(self) -> str | None:
return self._error
def analyze(self, image_bytes: bytes) -> AnalyzeResponse:
if self._alpr is None:
raise RuntimeError(f"fast-alpr not available: {self._error}")
# fast-alpr's predict() takes a BGR ndarray; decode the JPEG with cv2 (pulled in
# transitively by the alpr extra). Import locally so stub mode needs neither.
import cv2
import numpy as np # local import: only needed on the real path
started = time.perf_counter()
buf = np.frombuffer(image_bytes, dtype=np.uint8)
frame = cv2.imdecode(buf, cv2.IMREAD_COLOR)
if frame is None:
raise ValueError("could not decode image bytes")
results = self._alpr.predict(frame)
plates: list[PlateResult] = []
for r in results:
plate = plate_from_alpr_result(r)
if plate is not None:
plates.append(plate)
plates.sort(key=lambda p: p.confidence, reverse=True)
best = plates[0] if plates else None
low = best is not None and best.confidence < self._settings.min_confidence
took_ms = (time.perf_counter() - started) * 1000.0
return AnalyzeResponse(
plate=best,
plates=plates,
vehicle=None, # Job 2 not built yet
low_confidence=low,
model_version=self.model_version,
took_ms=took_ms,
)
def build_recognizer(settings: Settings) -> Recognizer:
"""Factory: pick the recognizer from settings. Falls back to the stub if the real
one can't load, so the service always comes up (with ready=False surfaced)."""
if settings.recognizer == "fast_alpr":
rec = FastAlprRecognizer(settings)
return rec
return StubRecognizer(settings)
+60
View File
@@ -0,0 +1,60 @@
"""The /analyze response contract — the shape the Node VisionClient adapter consumes.
Mirrors the first-cut API in wiki/entities/opencv-anpr-service.md:
{ plate: {text, confidence, bbox}|null, vehicle: {...}|null, modelVersion, tookMs }
Job 2 (vehicle attributes) is scaffolded as an optional field, not yet populated —
fast-alpr is plate-only; the vehicle stage is built later on the same ONNX runtime.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
class BBox(BaseModel):
"""Plate bounding box in pixels (top-left origin)."""
x1: int
y1: int
x2: int
y2: int
class PlateResult(BaseModel):
text: str
# The plate's confidence = the MIN of fast-alpr's per-character confidences (a plate
# is only as trustworthy as its weakest character). See recognizer.py.
confidence: float = Field(ge=0.0, le=1.0)
bbox: BBox | None = None
# Predicted issuing region/country (advisory; fast-alpr's global model emits this).
region: str | None = None
class VehicleResult(BaseModel):
"""Job 2 — vehicle attributes / fingerprint (anti-spoofing). Not yet produced."""
colour: str | None = None
body_type: str | None = None
make: str | None = None
model: str | None = None
class AnalyzeResponse(BaseModel):
# The single best plate, or null when none was found.
plate: PlateResult | None = None
# All plates found (a frame may contain several vehicles).
plates: list[PlateResult] = Field(default_factory=list)
vehicle: VehicleResult | None = None
# True when the best plate is below the confidence floor — Node should treat the
# read as advisory only and prefer the ticket path. See fail-state-safety.
low_confidence: bool = False
model_version: str
took_ms: float
class HealthResponse(BaseModel):
status: str
recognizer: str
ready: bool
model_version: str
detail: str | None = None
+37
View File
@@ -0,0 +1,37 @@
"""Runtime configuration, from environment (prefix VISION_).
Offline-first: every default is local and works with no network. The recognizer is
chosen by `recognizer` — "stub" (no models, deterministic placeholder) or "fast_alpr"
(the real MIT YOLOv9+CCT/ONNX stack, installed via the `alpr` extra).
"""
from __future__ import annotations
from typing import Literal
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_prefix="VISION_", env_file=".env", extra="ignore")
host: str = "0.0.0.0"
port: int = 8089
# Which recognizer to load. "stub" needs no model weights (boots anywhere, for
# dev/CI); "fast_alpr" loads the real models (requires the `alpr` extra installed).
recognizer: Literal["stub", "fast_alpr"] = "stub"
# fast-alpr model names (only used when recognizer="fast_alpr"). Defaults match the
# library defaults; swap the OCR for the 40+country EU model to benchmark Albanian
# plates. See wiki/entities/opencv-anpr-service.md "Recognizer evaluation".
detector_model: str = "yolo-v9-t-384-license-plate-end2end"
ocr_model: str = "cct-xs-v2-global-model"
# Below this OCR confidence the read is returned but flagged low_confidence, so the
# Node side can fall back to the ticket path rather than trust it.
min_confidence: float = 0.5
def get_settings() -> Settings:
return Settings()
+22
View File
@@ -322,6 +322,10 @@ function DeviceForm({
const canDiscover = !editing && selected != null && discoverableIds.includes(selected.id);
const pushesToBackend = selected != null && pushCapableIds.includes(selected.id);
const isController = category === "access";
const isCamera = category === "camera";
// ANPR opt-in for a camera: when true, the VisionReader polls this camera for plates
// (config.anpr). Off by default. See wiki/entities/opencv-anpr-service.md.
const [anpr, setAnpr] = useState<boolean>(editCfg?.anpr === true);
// Pre-fill scalar config fields from the existing assignment when editing.
// (relays/controllerId/relay are model fields handled by their own state below.)
@@ -429,6 +433,8 @@ function DeviceForm({
out.controllerId = controllerId;
out.relay = boundRelay;
}
// Camera ANPR opt-in (only persisted when on, to keep configs minimal).
if (isCamera && anpr) out.anpr = true;
return out;
}
@@ -584,6 +590,22 @@ function DeviceForm({
/>
)}
{/* CAMERA: opt this camera into ANPR (the VisionReader polls it for plates). */}
{isCamera && (
<label className="my-2 flex items-start gap-2 rounded-term border border-term-border bg-term-bg p-2 text-[12px]">
<input
type="checkbox"
className="mt-0.5"
checked={anpr}
onChange={(e) => setAnpr(e.target.checked)}
/>
<span>
<span className="font-semibold text-term-text">{t("setup.anpr")}</span>
<span className="hint mt-0.5 block">{t("setup.anprHint")}</span>
</span>
</label>
)}
{/* Test (no save/no device change) then Save (configures + persists). */}
<div className="mt-3 flex items-center gap-2">
<button type="button" className="btn btn-sm" onClick={test} disabled={testing}>
+14 -3
View File
@@ -647,7 +647,7 @@ export function fetchOccupancy(): Promise<Occupancy> {
export interface DeviceStatus {
deviceId: string;
driverId: string;
category: "access" | "reader" | "camera" | "printer";
category: "access" | "reader" | "camera" | "printer" | "vision";
/** Role/direction token for the footer label (NOT the vendor) — the client
* localises it next to the category, e.g. "Lexuesi hyrje", "Printer kabina". */
roleKind: "entry" | "exit" | "both" | "mixed" | "lane" | "booth" | null;
@@ -789,11 +789,22 @@ export interface SnapshotFailure {
occurredAt: string;
}
/** A licence plate recognized for this session by the ANPR-on-snapshot path (advisory
* record — see opencv-anpr-service.md). `snapshotId` links to the image it was read from. */
export interface PlateRead {
plate: string;
confidence: number | null;
region: string | null;
direction: "entry" | "exit" | null;
snapshotId: string | null;
at: string;
}
/** Snapshot metadata for a session identity (newest first) PLUS failed capture
* attempts. Image bytes are at `/api/snapshots/:id` — use that as an <img src>. */
* attempts PLUS any recognized plates. Image bytes are at `/api/snapshots/:id`. */
export function fetchSnapshots(
identity: string,
): Promise<{ snapshots: SnapshotMeta[]; failures?: SnapshotFailure[] }> {
): Promise<{ snapshots: SnapshotMeta[]; failures?: SnapshotFailure[]; plates?: PlateRead[] }> {
return apiFetch(`/api/snapshots/by-identity/${encodeURIComponent(identity)}`);
}
+5
View File
@@ -63,6 +63,7 @@ export const en: Catalog = {
catReader: "Reader",
catCamera: "Camera",
catPrinter: "Printer",
catVision: "Vision",
// Role/direction suffixes for the chip label (e.g. "Reader entry").
role: {
entry: "entry",
@@ -297,6 +298,9 @@ export const en: Catalog = {
entryCooldownHint:
"When there's no presence loop: repeat button presses are suppressed for this many seconds after a ticket. A fallback (not a guarantee) — a determined abuser can wait it out.",
addRelay: "+ Add relay",
anpr: "Plate recognition (ANPR)",
anprHint:
"Enable to scan plates on this camera: the vision service reads the plate from a snapshot and feeds it as a read (advisory only — it never opens a barrier on its own). Requires the vision service running.",
whichBarrier: "Which barrier does this device serve?",
controller: "Controller",
choose: "Choose…",
@@ -554,5 +558,6 @@ export const en: Catalog = {
snapEntry: "entry",
snapExit: "exit",
snapFailed: "camera unreachable",
plate: "Plate",
},
};
+6
View File
@@ -65,6 +65,7 @@ export const sq = {
catReader: "Lexuesi",
catCamera: "Kamera",
catPrinter: "Printer",
catVision: "Vizioni",
// Role/direction suffixes for the chip label (e.g. "Lexuesi hyrje").
role: {
entry: "hyrje",
@@ -306,6 +307,10 @@ export const sq = {
entryCooldownHint:
"Kur nuk ka sensor pranie: shtypjet e përsëritura të butonit shtypen për kaq sekonda pas një bilete. Zgjidhje rezervë (jo garanci) — një abuzues mund ta presë afatin.",
addRelay: "+ Shto rele",
// Camera ANPR opt-in.
anpr: "Njohja e targave (ANPR)",
anprHint:
"Aktivizo që ky aparat të skanojë targat: shërbimi i vizionit lexon targën nga pamja dhe e dërgon si lexim (vetëm këshillues — nuk hap vetë barrierën). Kërkon shërbimin e vizionit aktiv.",
// Binding picker.
whichBarrier: "Cilën barrierë shërben kjo pajisje?",
controller: "Kontrolluesi",
@@ -568,6 +573,7 @@ export const sq = {
snapEntry: "hyrje",
snapExit: "dalje",
snapFailed: "kamera e paarritshme",
plate: "Targa",
},
};
+2
View File
@@ -32,6 +32,7 @@ const CATEGORY_KEY: Record<DeviceStatus["category"], string> = {
reader: "devices.catReader",
camera: "devices.catCamera",
printer: "devices.catPrinter",
vision: "devices.catVision",
};
/** i18n key for the role/direction token (null = no suffix). */
@@ -45,6 +46,7 @@ const ORDER: Record<DeviceStatus["category"], number> = {
reader: 1,
camera: 2,
printer: 3,
vision: 4,
};
/** "Lexuesi hyrje" — category word + localised role/direction (when known). */
+38 -2
View File
@@ -1,7 +1,20 @@
import { useState } from "react";
import { useTranslation } from "react-i18next";
import { useQuery } from "@tanstack/react-query";
import { fetchSnapshots, snapshotImageUrl } from "../api.js";
import { fetchSnapshots, snapshotImageUrl, type PlateRead } from "../api.js";
/** Keep one entry per (plate, direction) — newest wins (the list is newest-first). */
function dedupePlates(plates: PlateRead[]): PlateRead[] {
const seen = new Set<string>();
const out: PlateRead[] = [];
for (const p of plates) {
const key = `${p.plate}|${p.direction}`;
if (seen.has(key)) continue;
seen.add(key);
out.push(p);
}
return out;
}
// Entry/exit evidence images for a session. Lets the operator verify the car at the
// booth against the ticket. Thumbnails load from /api/snapshots/:id (cookie-authed,
@@ -18,17 +31,40 @@ export function SnapshotStrip({ identity }: { identity: string }) {
const shots = data?.snapshots ?? [];
const failures = data?.failures ?? [];
const plates = data?.plates ?? [];
/** Localized direction label for a snapshot/failure tile. */
const dirLabel = (dir: "entry" | "exit" | null): string =>
dir === "entry" ? t("pay.snapEntry") : dir === "exit" ? t("pay.snapExit") : "—";
if (isLoading) return <div className="text-[11px] text-term-muted">{t("pay.loadingSnapshots")}</div>;
if (shots.length === 0 && failures.length === 0)
if (shots.length === 0 && failures.length === 0 && plates.length === 0)
return <div className="text-[11px] text-term-muted">{t("pay.noSnapshots")}</div>;
return (
<>
{/* Recognized plate(s) (ANPR) — advisory. Dedup by plate+direction so an
entry+exit read of the same plate shows once per direction. */}
{plates.length > 0 && (
<div className="mb-2 flex flex-wrap gap-1.5">
{dedupePlates(plates).map((p, i) => (
<span
key={`${p.plate}-${p.direction}-${i}`}
className="inline-flex items-center gap-1.5 rounded-term border border-term-cyan/40 bg-term-cyan/10 px-2 py-0.5 text-[11px]"
title={`${dirLabel(p.direction)}${p.region ? ` · ${p.region}` : ""}${
p.at ? ` · ${new Date(p.at).toLocaleString()}` : ""
}`}
>
<span className="text-[9px] uppercase tracking-wider text-term-muted">{t("pay.plate")}</span>
<span className="font-mono font-semibold text-term-cyan">{p.plate}</span>
{typeof p.confidence === "number" && (
<span className="text-[10px] text-term-muted">{(p.confidence * 100).toFixed(0)}%</span>
)}
</span>
))}
</div>
)}
<div className="flex flex-wrap gap-2">
{shots.map((s) => (
<button
+2
View File
@@ -67,6 +67,8 @@ importers:
specifier: 6.0.3
version: 6.0.3
apps/vision: {}
apps/web:
dependencies:
'@parking/shared':
+112
View File
@@ -0,0 +1,112 @@
---
type: decision
tags: [parking, decisions, vision, anpr, monorepo, packaging]
sources: []
updated: 2026-06-19
status: settled
---
# Decision: the vision service lives in this monorepo (apps/vision/), wired into Turbo via a shim
Taken 2026-06-19, when planning how to *implement* the host-side [[opencv-anpr-service|vision
service]] decided in [[vision-service]]. That decision settled WHAT (a separate localhost Python
process) and the recognizer baseline ([[opencv-anpr-service|fast-alpr]]); this one settles WHERE the
source lives and how it joins the build.
## Decision
1. **In THIS monorepo, at `apps/vision/`** — a Python/FastAPI service co-located with the Node
backend, **not** a separate repository. One git history, atomic cross-cutting commits (the
`/analyze` contract + the Node-side adapter change together), one wiki.
2. **Still a separate OS process** — co-location is source-level only. It runs as its own process
(`uvicorn`), called over **localhost HTTP** by the Node backend, with its own failure domain.
Nothing about putting it in `apps/vision/` weakens the runtime isolation [[vision-service]]
requires.
3. **Wired into the Turbo task graph via a thin `package.json` shim.** `pnpm-workspace.yaml` already
globs `apps/*`, so an `apps/vision/package.json` auto-joins the workspace. Its `scripts` shell out
to Python tooling, so the existing `turbo run` tasks cover it:
- `dev` → `uv run uvicorn app:app --reload` (matches `turbo.json` `dev`: persistent, uncached)
- `lint` → `ruff check` · `test` → `pytest` · `typecheck` → `ruff`/`mypy`
- `build` → **no-op or model-fetch** (Python has no `dist/**`; the `build` task's `outputs:
["dist/**"]` simply won't match — fine). If models are fetched/cached at build, point outputs at
the model dir.
Python **dependencies** stay managed by `uv` + `pyproject.toml` (NOT pnpm) — the shim only exposes
*tasks*, not deps.
4. **Node talks to it through an interface** (`VisionClient` behind a port, the
[[device-adapter-pattern]] style) so the recognizer/service is swappable without touching business
logic — as [[opencv-anpr-service]] already specifies.
## Why co-located beats a separate repo
- **Atomic changes.** The service contract (`POST /analyze` shape) and its Node consumer evolve
together; one repo = one PR, no two-repo version skew.
- **`uv` makes Python-in-monorepo painless** — fast, lockfile-based, offline-friendly (fits
[[offline-first]]); the appliance build pulls a pinned env.
- **Turbo still orchestrates it.** The shim makes `turbo run lint`/`test` include the Python service
as a first-class node — one command lints front, back, AND vision — even though Turbo can't *build*
Python. Turbo orchestrates **tasks**, and a task can be a Python command.
- **One knowledge base.** The wiki + CLAUDE.md already describe the whole system; a split repo
fragments that.
## Why this still honors the isolation decision
The "[[vision-service|separate process]]" decision is about **runtime isolation** (own process +
failure domain) and **license isolation** (AGPL obligations don't reach the Node/React code because
it is **not linked** — it's a separate program over HTTP). **Neither depends on a separate
repository.** AGPL's reach is a linking/distribution-boundary question between *programs*, not a
which-folder question. A Python service in `apps/vision/` that Node calls over localhost is exactly as
isolated, license-wise, as one in its own repo.
- With the **[[opencv-anpr-service|fast-alpr]] MIT-end-to-end baseline**, the AGPL pressure to split
the repo out **largely evaporates** (pending the weight-provenance caveat). Co-location is the
low-friction default.
- If a true-AGPL model (Ultralytics YOLO) is later adopted, its weights live under `apps/vision/` —
still fine (separate process), and that dir is the natural place to document the license boundary +
the `[[standing-decisions|scoped exception]]`.
## Rejected
- **Separate repo** — strongest separation, but loses atomic contract changes and adds coordination
overhead; justified only if a different team owns it or the AGPL concern becomes acute. Kept as the
fallback if either happens.
- **Embed Python in the Node process** (opencv4nodejs / a child-process module) — already rejected by
[[vision-service]] (native-build pain, no process isolation, shares the app's failure + license
surface). Unchanged.
- **A Python package under `packages/`** — `packages/` is for shared *JS* libraries imported by other
workspaces; the vision service is a deployable app, so `apps/vision/` is the right bucket.
## As-scaffolded (2026-06-19)
The skeleton is **built and wired** (no recognizer models yet):
- `apps/vision/` — `pyproject.toml` (+ `uv.lock`, uv-managed), the thin `package.json` shim, a
per-package `turbo.json` (`extends: ["//"]`, `build` outputs `[]` so the no-op build is warning-
free), `.gitignore` (venv/caches/`*.onnx`/`models/` out), `README`.
- `vision_service/`: `app.py` (FastAPI `GET /health` + `POST /analyze`, raw octet-stream body so Node
POSTs `Snapshot.bytes` directly; oversize→413, empty→400, recognizer-not-ready→503), `settings.py`
(env `VISION_*`), `schemas.py` (the `/analyze` contract incl. a not-yet-populated `vehicle` field
for Job 2), `recognizer.py` (a `Recognizer` **Protocol** + `StubRecognizer` and `FastAlprRecognizer`
— the [[device-adapter-pattern]] applied to the model).
- **Light-core, heavy-optional:** core deps boot in **stub mode** (no model download) so `uv sync` +
tests work offline; the real stack is the `alpr` extra (`uv sync --extra alpr` →
fast-alpr + onnxruntime). `VISION_RECOGNIZER=fast_alpr` switches it on.
- **Verified:** `turbo run lint|test|build` includes `@parking/vision` (ruff/pytest/no-op via the
shim) and stays green; `uv run mypy` strict-clean; uvicorn boots and serves `/health` (`ready`,
stub-0) + `/analyze` (contract shape) live. pnpm workspace count 6→7.
## Still to build (next, when vision work proceeds)
- The Node-side **`VisionClient`** adapter (localhost HTTP) + per-camera **opt-in** wiring (the open
item in [[opencv-anpr-service]]).
- A **`Dockerfile`**/process unit for the appliance (its own image/process); model-weight fetch at
deploy (the `alpr` extra), kept out of git ([[opencv-anpr-service|weight-provenance]] check first).
- **Job 2** (vehicle attributes / fingerprint) — the `vehicle` field is scaffolded but unpopulated;
fast-alpr is plate-only. Built later on the same ONNX runtime.
## Open
- `uv` vs. `pip-tools`/`poetry` for the Python env (leaning `uv` — speed + lockfile + offline).
- Whether `build` should fetch/cache model weights (and set Turbo `outputs` to the model dir) or keep
weights out of the build entirely (baked into the Docker image instead).
- Container/runtime supervision on the appliance (systemd unit vs. compose) — deployment detail,
defer to the install/hardening pass.
+3 -1
View File
@@ -21,7 +21,9 @@ Taken 2026-06-15, as part of the business-layer build ([[session-model]]).
option).
3. **Deployment: a separate local Python/OpenCV microservice** on the appliance, called over
**localhost HTTP** by the Node backend. Fully offline ([[offline-first]]); its own process and
failure domain; the host falls back to the ticket path if it's unavailable.
failure domain; the host falls back to the ticket path if it's unavailable. **Source lives in THIS
monorepo at `apps/vision/`, wired into Turbo via a thin `package.json` shim** — separate *process*,
co-located *source*; see [[vision-service-packaging]].
4. **Licensing exception:** AGPL components (e.g. YOLO plate/vehicle models, OpenALPR) are
**permitted inside this service only**, because it's a separate process not linked into the app —
the app stays strictly MIT/Apache/BSD. Amends [[standing-decisions]].
+117 -7
View File
@@ -34,7 +34,9 @@ recognition **host-side on ordinary IP-camera snapshots**, replacing the dedicat
- A **Python service** (e.g. FastAPI) running **on the appliance**, called by the Node backend over
**localhost HTTP** (`POST /analyze` with the JPEG bytes the camera driver already pulls — see
[[lpr-camera]] "driver/storage boundary": `Snapshot.bytes`).
[[lpr-camera]] "driver/storage boundary": `Snapshot.bytes`). **Source lives in this monorepo at
`apps/vision/`** (Turbo shim; `uv`-managed deps) — co-located source, separate process; see
[[vision-service-packaging]].
- **Fully offline** ([[offline-first]]): all inference is local, no cloud. Model weights ship on the
appliance.
- **Process isolation is deliberate** — it keeps a heavy Python/native/AGPL stack out of the
@@ -107,7 +109,27 @@ disappoints).
**Recommendation:** prototype with **fast-alpr** now (permissive, offline, ONNX, fits the decided
shape); plan a YOLO-detector fine-tune + PaddleOCR only if production accuracy demands it. Choice kept
**open** pending the weight-provenance check + an accuracy benchmark on real AL plates.
**open** pending the weight-provenance check (the AL-plate benchmark below is now done).
### Albanian-plate OCR benchmark — keep the default (2026-06-19)
Ran the four candidate `fast-plate-ocr` models through the **full pipeline** (YOLOv9 detect → OCR) on
real AL plate photos (Wikimedia), CPU, scaffolded service:
| OCR model | `AA 558 EE` | `AA 687 KE` | Speed | Note |
| --- | --- | --- | --- | --- |
| **`cct-xs-v2-global-model`** (default) | ✓ 0.999 | ✓ **1.000** | **33–39 ms** | best accuracy + fastest; returns `region=Albania` |
| `cct-s-v2-global-model` | ✓ 0.998 | ✓ 0.999 | 50–65 ms | as accurate, ~50% slower |
| `global-plates-mobile-vit-v2-model` | ✓ 0.955 | ✓ 0.959 | 33–35 ms | fast, lower confidence |
| `european-plates-mobile-vit-v2-model` | ✓ 0.784 | ✓ 0.766 | 38–46 ms | correct but **much lower confidence**; misread a synthetic `AB123FG`→`AB123FO` |
**Finding (overturns the prior assumption):** the **default `cct-xs-v2-global-model` is the best for
Albania** — most accurate AND fastest. The "European (40+ country)" model is *worse* here (~0.77 vs
~1.0 confidence, one synthetic misread), despite the "EU model → better for AL" intuition. So **no
config change**: `VISION_OCR_MODEL` stays `cct-xs-v2-global-model`. Caveat: both test photos were
clean head-on shots; real booth captures (angled, dirty, night, motion-blur) will lower absolute
confidence — the `min_confidence=0.5` floor (→ `low_confidence` → ticket-path fallback) covers that.
The ranking should hold; re-benchmark on real on-site captures once the cameras are installed.
## Anti-fraud / threat-model fit
@@ -121,15 +143,103 @@ shape); plan a YOLO-detector fine-tune + PaddleOCR only if production accuracy d
stake — confidence thresholds + fallback to ticket/manual; a low-confidence read must not strand a
car ([[fail-state-safety]]).
## Fitness for the entry/exit flows (assessment, 2026-06-19)
Asked after the scaffold + AL benchmark: *is the service worthy to consume in the entry/exit flows?*
The benchmark settles **accuracy** (0.99+ on clean AL plates); "worthy" then turns on **what authority
the read is given** — and the answer splits by role:
- **✅ Worthy NOW — as an ADVISORY identity source (Job 1).** The flows are **already built for a
plate**: a `kind:"plate"` [[device-events|read]] is a first-class identity today — `exit-flow.ts`
signs `source:"lpr"` for it, and `subscription-flow.ts` matches a read plate against
`subscriptionPlates` ([[subscription]] plate binding). So the service just **produces** the plate
string a snapshot → `/analyze` → (if confident) a `DeviceReadEvent{kind:"plate"}` on the existing
read bus. **No flow rewrite — it feeds an existing input.** Concretely worthy for: hands-free
**subscriber** barrier open (plate-bound), and **evidence enrichment** (plate + image on the signed
entry/exit for disputes).
- **⚠️ NOT worthy as the SOLE AUTHORITY to open a TRANSIENT barrier.** Two threat-model reasons: (1) **a
plate is not a payment** — a transient still needs a ticket + `payment`; letting a plate open the
exit would be an unpaid-exit bypass. The `min_confidence` floor → `low_confidence` → ticket/manual
fallback is the guard (already in the scaffold). (2) **Plate-spoofing** (a printed plate on a
different car) — plate-only ANPR *cannot* catch it; that needs **Job 2 (vehicle verification), which
is NOT built**. So plate-as-identity is convenience + evidence, never the lone reason a paid barrier
opens. Consistent with "advisory, never sole authority" above.
**Gaps before it's actually consumed (capable ≠ wired):** (1) ✅ **DONE — the Node→service
`VisionClient`** adapter (`apps/server/src/vision-client.ts`, localhost HTTP to `/analyze` + `/health`)
now exists: **opt-in** (`VISION_ENABLED`, default off), **fail-soft** (any error/timeout/unreachable →
`null`, never throws into the lane → ticket-path fallback), and **re-applies the confidence floor**
(`VISION_MIN_CONFIDENCE`) so a low read is flagged advisory. Constructed in `server.ts`; verified
end-to-end against the live service (Node → `AA558EE` 0.999, `region=Albania`). (2) ✅ **DONE —
trigger: ANPR rides the entry/exit SNAPSHOT (`snapshot.ts`).** The real-world trigger is a **transient
button-press or a subscriber QR/RFID read** — which already fires the entry/exit and its evidence
snapshot. That is exactly the moment to recognize: `snapshotAsync` now takes the `VisionClient`, and
after storing each snapshot from an **opt-in** camera (`config.anpr === true`), it runs ANPR off the
**SAME image** and **records the plate against the SAME session `identity`** — an unsigned
`device_events{kind:"read"}` with plate / confidence / region / model / `snapshotId` /
`source:"entry-exit-snapshot"`. So you can later answer *"session X entered on plate AA558EE"*, with the
evidence image linked by `snapshotId`. **No polling — recognition fires only on a real entry/exit**,
one image serving both evidence and plate extraction. *(Superseded the earlier polling `VisionReader`,
now removed — `VISION_POLL_MS`/`VISION_DEDUPE_MS` gone.)* The flows pass the client (entry/exit/
subscription constructors). It is **advisory + fire-and-forget**: a low-confidence/no-plate result
records nothing, a vision failure never delays or changes the open, and the plate does **not** feed the
access decision (the flow already decided). *Verified end-to-end:* a simulated entry snapshot on an
`anpr` camera → stored the snapshot for the session AND recorded `{identity:"TICKET-…", plate:"AA558EE",
confidence:0.999, region:"Albania", snapshotId:…}`.
**Viewing it:** `GET /api/snapshots/by-identity/:identity` now also returns `plates[]` (the
`kind:"read"` reads for that session), and the **`SnapshotStrip`** renders each as a cyan
"Plate: AA558EE 100%" chip above the images — so the recognized plate shows in the **booth
event-detail modal AND the pay modal** beside the evidence photo, with no separate screen.
(3) **field-accuracy** unknown — re-benchmark/tune
the threshold on real on-site captures (angle/night/dirt). (4) the **weight-provenance** check (open).
**Bottom line: consume it as a gated advisory identity record off the entry/exit snapshot — not as sole
authority — and Job 2 is still required for the anti-spoofing value.** The
adapter + the snapshot-triggered ANPR are now **both built and verified end-to-end**; remaining is
field tuning (3), the provenance check (4), and Job 2.
## Configuration (2026-06-19)
Turning it on touches **four layers** — two env sets (one per process), per-camera data, and deploy.
The Python service and the Node server **both** read the `VISION_` prefix but are **separate
processes**, so give each its **own `.env`** (`apps/vision/.env` and `apps/server/.env`) — don't merge
them. `.env.example` files document both.
**1. The Python service (`apps/vision/.env`):** `VISION_RECOGNIZER=fast_alpr` (the default `stub`
recognizes nothing), `VISION_HOST`/`VISION_PORT` (prefer **`127.0.0.1`** — only the Node backend calls
`/analyze`, so don't expose it off-host), `VISION_OCR_MODEL`/`VISION_DETECTOR_MODEL` (leave defaults —
the AL-benchmark winners), `VISION_MIN_CONFIDENCE`. Install the models with `uv sync --extra alpr`;
weights download on first run, so **cache them at build/deploy** for the air-gapped appliance.
**2. The Node server (`apps/server/.env`):** `VISION_ENABLED=1` is the **master switch** (off by
default — nothing runs or shows without it); `VISION_URL` must match the service's host:port;
`VISION_TIMEOUT_MS` (slow-call cap so a lane never hangs) and `VISION_MIN_CONFIDENCE` (re-applied
client-side). ANPR fires on the entry/exit snapshot, so there are **no poll/dedupe knobs**.
**3. Per-camera opt-in (device config, not env):** a camera does ANPR only when its config has **both**
`anpr: true` **and** a relay binding (`controllerId` + `relay`). The `anpr` flag is a **checkbox on the
camera form in the [[first-run-setup|SetupWizard]]** (built 2026-06-19). Without the binding the
[[entry-exit-points|dispatcher]] refuses every read ("reader not bound to a barrier") — so an
unbound ANPR camera recognizes but every read is rejected (and logged with its snapshot).
**4. Footer health:** when `VISION_ENABLED`, the [[device-status-monitoring|DeviceMonitor]] probes the
service's `/health` each tick and shows a **"Vision" chip** in the booth footer (ready/degraded/offline
+ the recognizer name); no chip when disabled. So the operator sees at a glance whether vision is up.
> **Network isolation** ([[network-isolation]]): cameras live on the isolated device VLAN, so the
> vision service must reach that VLAN to pull snapshots — but its own `/analyze` should bind
> **localhost** (Node is the only caller). Keep the AGPL/heavy stack contained to this process.
## Open
- **Recognizer choice** — **fast-alpr (MIT, YOLOv9+CCT on ONNX) is the evaluated baseline** (see the
Recognizer evaluation section above); remaining open items are the **weight-provenance check** and
an **accuracy benchmark on real AL plates** (default global vs. the 40+country EU model). See
- **Recognizer choice** — **fast-alpr (MIT, YOLOv9+CCT on ONNX) is the baseline, AL-benchmarked**: the
default `cct-xs-v2-global-model` won over the EU model on real AL plates (table above). The one
remaining open item is the **model-weight-provenance check** (the MIT-weights claim). A re-benchmark
on real *on-site* captures (angled/night/dirty) is wanted once cameras are installed. See
[[vision-service]]; AGPL still permitted in-service for the stronger fallback.
- **Vehicle fingerprint**: attribute classifier vs. embedding-similarity; what threshold makes a
mismatch an anomaly without false-positiving on lighting/angle.
- **Compute footprint** on the appliance (CPU-only vs. a small GPU/NPU) — procurement input
([[bom]], [[open-questions]]).
- Per-camera **opt-in** ("optionally bound", user's word): which lanes/cameras route snapshots to
the service.
- Per-camera **opt-in** — ✅ **built**: `config.anpr === true` enables ANPR on a camera (set via the
SetupWizard checkbox); ANPR then runs on that camera's entry/exit snapshot.
+2 -1
View File
@@ -7,7 +7,7 @@ updated: 2026-06-19
# Index
Content catalog for the wiki. Start at [[overview]]. Maintained on every ingest.
Counts: 4 sources · 19 entities · 44 concepts · 5 decision records.
Counts: 4 sources · 19 entities · 44 concepts · 6 decision records.
## Overview & navigation
- [[overview]] — the top-level synthesis and entry point.
@@ -115,4 +115,5 @@ Counts: 4 sources · 19 entities · 44 concepts · 5 decision records.
- [[dingtian-vs-mqtt]] — transport choice: direct HTTP/UDP now, MQTT parked until multi-lane scale.
- [[session-model]] — business layer start: session = projection; transient-first; pay-on-foot. New event types.
- [[vision-service]] — build a host-side ANPR + vehicle-verification service; replaces edge-LPR; scoped AGPL exception.
- [[vision-service-packaging]] — the vision service lives in this monorepo (apps/vision/), separate process, wired into Turbo via a package.json shim; uv-managed Python.
- [[event-streams-split]] — split the signed business ledger (ledger_events) from unsigned device telemetry (device_events).
+40
View File
@@ -908,3 +908,43 @@ Added a THIRD data stream (`app_logs`) alongside the signed ledger and device te
## [2026-06-19] query | ANPR recognizer options — fast-alpr evaluated as the baseline
Q: LPR/ANPR options — YOLO, OpenCV, both, another framework? Reframed: "YOLO vs OpenCV" is a category error — they're different pipeline LAYERS (YOLO = plate detector; OpenCV = Apache-2.0 image-handling glue, used regardless; plus an OCR stage). The real choice is which end-to-end recognizer. Researched [fast-alpr](https://github.com/ankandrew/fast-alpr) (latest **v0.4.0, 15 Mar 2026, MIT**): a thin orchestrator over two swappable ONNX stages — detection via [open-image-models](https://github.com/ankandrew/open-image-models) (`yolo-v9-t-384-license-plate-end2end`, MIT) + OCR via [fast-plate-ocr](https://github.com/ankandrew/fast-plate-ocr) (`cct-xs-v2-global-model`, MIT; also has a EUROPEAN model trained on 40+ countries — relevant for AL plates). MIT top-to-bottom (code AND published weights), one maintainer across all three repos, CPU-only + fully offline, backend extras for CPU/CUDA/OpenVINO/DirectML/QNN. KEY FINDING: its detector is open-image-models' OWN YOLOv9 ONNX export, NOT the Ultralytics AGPL package — so fast-alpr is a PERMISSIVE baseline that may not even need the scoped AGPL exception from [[vision-service]]. CAVEAT (flagged, not closed): a repo's LICENSE covers code, not necessarily redistributed model WEIGHTS (YOLOv9 upstream is GPL-3.0; Ultralytics YOLO AGPL) — verify weight provenance before relying on "MIT weights". fast-alpr is PLATE-ONLY → Job 2 (vehicle-attribute anti-spoofing) is still ours to build, but shares the same ONNX runtime. Recommendation: prototype fast-alpr now; Ultralytics-YOLO+PaddleOCR fine-tune only if accuracy disappoints. Recorded as an evaluated-options note; decision kept status:open pending the provenance check + an AL-plate accuracy benchmark. Updated [[opencv-anpr-service]] (new "Recognizer evaluation" section + licensing nuance), [[vision-service]] (open/next), index.
## [2026-06-19] decision | Vision service packaging — apps/vision/ in this monorepo, Turbo shim
Q: how to IMPLEMENT the vision service — can we use this Turborepo? Settled (status:settled): the Python/FastAPI ANPR service lives in THIS monorepo at `apps/vision/`, NOT a separate repo. Key clarification: Turbo orchestrates JS/TS package.json TASKS (+ caches outputs); it has no native Python build — but "in the repo" ≠ "in the Turbo graph", and "separate process" ≠ "separate repo". Decision: (1) co-locate source at apps/vision/ (pnpm-workspace already globs apps/*, so it auto-joins) for atomic cross-cutting changes (the /analyze contract + the Node adapter together), one wiki/history; (2) still a SEPARATE OS process (uvicorn over localhost HTTP) — co-location is source-level only, runtime isolation intact; (3) wire into Turbo via a THIN package.json shim whose scripts shell to Python (dev→uv run uvicorn, lint→ruff, test→pytest, build→no-op/model-fetch since Python has no dist/**), so `turbo run lint/test` covers vision too — deps stay uv/pyproject, not pnpm; (4) Node talks to it via a VisionClient interface (device-adapter style), swappable. WHY co-location honors the [[vision-service]] isolation decision: that decision is about RUNTIME + LICENSE isolation (separate process; AGPL doesn't reach Node because it's not LINKED, just HTTP) — AGPL's reach is a linking/distribution-boundary question, NOT a which-folder question. And with the MIT-end-to-end [[opencv-anpr-service|fast-alpr]] baseline the AGPL pressure to split the repo largely evaporates anyway. Rejected: separate repo (loses atomic changes; fallback if AGPL acute or another team owns it), embed-in-Node (already rejected by vision-service), packages/ (that's for shared JS libs, vision is a deployable app). NOT built yet — packaging decision only; scaffold when vision work starts. New page [[vision-service-packaging]]; updated [[vision-service]], [[opencv-anpr-service]], CLAUDE.md layout, index.
## [2026-06-19] build | Scaffold apps/vision (ANPR microservice skeleton)
Scaffolded the [[opencv-anpr-service|vision service]] per [[vision-service-packaging]]: `apps/vision/` Python/FastAPI, uv-managed, wired into Turbo via a thin package.json shim. Structure: pyproject.toml (light core: fastapi/uvicorn/pydantic; HEAVY recognizer = optional `alpr` extra = fast-alpr+onnxruntime, so `uv sync`+tests run OFFLINE in stub mode with no model download), per-package turbo.json (extends ["//"], build outputs [] → warning-free no-op), .gitignore (venv/caches/*.onnx/models out). vision_service/: app.py (GET /health + POST /analyze, raw octet-stream body so Node POSTs Snapshot.bytes directly; empty→400, oversize→413, not-ready→503), settings.py (env VISION_*), schemas.py (the /analyze contract + a not-yet-populated `vehicle` field for Job 2), recognizer.py (a Recognizer Protocol + StubRecognizer/FastAlprRecognizer — the device-adapter pattern applied to the model; fast-alpr imported lazily so missing models ⇒ ready=False, not a crash). VERIFIED: turbo run lint|test|build includes @parking/vision (ruff/pytest/no-op shim) green; uv run mypy strict-clean; uvicorn boots + serves /health (ready, stub-0) and /analyze (contract shape) live; pnpm workspace 6→7. NOT built: the Node VisionClient adapter, a Dockerfile + model fetch, and Job 2 (vehicle verification). Updated [[vision-service-packaging]] (As-scaffolded section), CLAUDE.md layout already lists apps/vision.
## [2026-06-19] query | Albanian-plate OCR benchmark — keep the default (cct-xs-v2-global)
Benchmarked fast-alpr's four candidate fast-plate-ocr models via the FULL pipeline (YOLOv9 detect → OCR) on real AL plate photos (Wikimedia: AA558EE, AA687KE), CPU, scaffolded apps/vision service. ALL FOUR read both plates correctly; the differentiator is confidence + speed: cct-xs-v2-global (default) 0.999/1.000 @ 33–39ms AND returns region=Albania; cct-s-v2-global same accuracy ~50% slower; global-mobile-vit ~0.955 fast; european-mobile-vit-v2 (the "40+ country EU" model) correct but MUCH lower confidence (~0.77) and misread a synthetic AB123FG→AB123FO. FINDING (overturns the "EU model → better for AL" assumption from the prior research turn): the global cct-xs default WINS for Albania — most accurate AND fastest. Decision: no config change, VISION_OCR_MODEL stays cct-xs-v2-global-model. Caveat: test photos were clean head-on shots; real booth captures (angle/night/dirt/blur) will lower confidence — the min_confidence=0.5 floor → low_confidence → ticket-path fallback covers it; re-benchmark on on-site captures once cameras installed. Resolves the AL-accuracy-benchmark open item in [[opencv-anpr-service]] (added a results table + the keep-default finding); the weight-provenance check remains the one open recognizer item.
## [2026-06-19] query | Vision service fitness for entry/exit flows — advisory YES, sole-authority NO
Q: is the scaffolded ANPR service worthy to consume in entry/exit flows? Assessment recorded in [[opencv-anpr-service]] ("Fitness for the entry/exit flows"). Benchmark settled ACCURACY (0.99+ clean AL plates); "worthy" turns on AUTHORITY. Split verdict: (✅) worthy NOW as an ADVISORY identity source (Job 1) — the flows are ALREADY built for a plate (kind:"plate" read is first-class: exit-flow signs source:"lpr"; subscription-flow matches read plate vs subscriptionPlates), so the service just produces the plate string → DeviceReadEvent{kind:"plate"} on the existing read bus; no flow rewrite. Worthy for hands-free subscriber open + evidence enrichment. (⚠️) NOT worthy as SOLE AUTHORITY to open a TRANSIENT barrier: a plate ≠ payment (would be an unpaid-exit bypass; min_confidence floor → ticket/manual fallback is the guard) and plate-spoofing (printed plate, different car) needs Job 2 vehicle-verification which is NOT built. Gaps before consuming: (1) the Node VisionClient adapter (real integration work), (2) trigger wiring — snapshots fire AFTER open today (evidence); plate-as-identity needs a snapshot BEFORE the decision on a per-camera opt-in lane, (3) field accuracy unknown (re-tune threshold on on-site captures), (4) weight-provenance check. Next step: VisionClient adapter + opt-in trigger, not more model work. (Scaffolding VisionClient next.)
## [2026-06-19] feat | VisionClient Node adapter (apps/server/src/vision-client.ts)
Scaffolded the Node-side adapter to the host vision microservice per the fitness assessment. VisionClient calls apps/vision over localhost HTTP (POST /analyze with snapshot Buffer bytes, GET /health), returning a normalised/camelCased VisionResult (best plate + all plates + lowConfidence + modelVersion + tookMs) or null. THREE guardrails enforce "advisory, never sole authority" at the boundary: (1) OPT-IN — VISION_ENABLED (default OFF), so the appliance runs with no vision service; (2) FAIL-SOFT — disabled/unreachable/timeout/non-2xx/bad-body all resolve to null and NEVER throw into the entry/exit path (→ ticket/manual fallback, never strand a car); (3) CONFIDENCE FLOOR re-applied (VISION_MIN_CONFIDENCE) on top of the service's own low_confidence flag. Per-request AbortController timeout (VISION_TIMEOUT_MS, default 1500ms) so a slow call can't hang the lane. Constructed in server.ts (logs when enabled). VERIFIED: fail-soft (disabled→null, unreachable→null no-throw) and LIVE end-to-end (Node client → running fast_alpr service → AA558EE 0.999 region=Albania, camelCased). NOT yet wired into the read bus — the opt-in snapshot-before-decision trigger that emits DeviceReadEvent{kind:"plate"} is the next deliberate step. Build+lint green. Updated [[opencv-anpr-service]] (gap 1 marked done).
## [2026-06-19] feat | VisionReader — wire ANPR into the read bus (apps/server/src/vision-reader.ts)
Wired the vision service into the entry/exit flows via the READ BUS. VisionReader polls each OPT-IN camera (config.anpr===true, off by default) every VISION_POLL_MS, captures a snapshot → VisionClient.analyze → on a CONFIDENT plate calls deviceEvents.emitRead({kind:"plate", value:PLATE, deviceId, driverId}) — the SAME event a physical plate reader emits, so the existing ReadDispatcher routes it to the subscription/exit flow UNCHANGED (no flow rewrite). The plate stays ADVISORY by construction: the exit flow still demands a covering payment (a plate can't bypass it), the subscription flow only matches a BOUND plate (subscriptionPlates). Guards: low-confidence reads DROPPED (a shaky read isn't an identity); DEBOUNCE (VISION_DEDUPE_MS, default 15s) so a parked car in frame doesn't re-fire the same plate; per-camera in-flight guard; idle when VISION_ENABLED off or no camera opts in; #recognizeOn is public for a future on-demand trigger (loop edge / API). Direction from directionOf (both→entry context). Constructed in server.ts, start on onReady / stop on onClose. VERIFIED END-TO-END: in-memory anpr camera returning the AL plate image + live fast_alpr service → VisionReader emitted exactly one {kind:"plate",value:"AA558EE"} read onto the bus; debounce held it to 1 emit over 7 polls. Build+lint green. Updated [[opencv-anpr-service]] (trigger-wiring gap + per-camera opt-in marked done). Remaining: SetupWizard anpr toggle, field tuning, weight-provenance, Job 2.
## [2026-06-19] feat | Persist every recognized plate + snapshot (ANPR audit trail, non-blocking)
VisionReader now PERSISTS every confident plate read so a recognition is investigable — and switched from emitRead to calling ReadDispatcher.dispatch directly (like qr-reader) to capture the OUTCOME. On a confident plate it: (1) stores the SNAPSHOT bytes in `snapshots` keyed by identity=PLATE — the SAME identity the flow signs its anomaly/event with — so GET /api/snapshots/by-identity/:plate (the booth event-detail modal's SnapshotStrip, which already uses e.identity) shows the car's photo against that anomaly with ZERO UI changes; (2) records an unsigned device_events{kind:"read"} breadcrumb with plate/confidence/region/modelVersion/snapshotId + the dispatch outcome (accepted + reason) = a queryable ANPR log independent of the signed ledger; (3) dispatches the read — NON-BLOCKING: a refused read just returns rejected (no barrier hold), logged with its snapshot for investigation. Plate stays advisory (exit demands payment; subscription matches only a bound plate). VERIFIED e2e: a recognized AL plate (AA558EE) with no open session → signed exit.refused.noSession anomaly (identity=plate), stored a 555KB snapshot under that plate, read-breadcrumb accepted:false reason:"no open session", and by-identity returned the image (status 200, 1 snapshot) → the refused read is investigable with its picture. Build+lint green. VisionReader constructor now takes the ReadDispatcher (wired in server.ts). Answers "is a recognized plate saved?" — now YES for both transient + subscriber, as telemetry + evidence image, regardless of match. Updated [[opencv-anpr-service]].
## [2026-06-19] feat | Vision service configuration — SetupWizard ANPR toggle, footer health chip, .env.example
Made the vision service genuinely configurable (was env-only). THREE additions: (1) SetupWizard CAMERA form now has an "ANPR / Njohja e targave" checkbox (writes config.anpr; only persisted when on; sq+en) — opt-in is no longer raw JSON. (2) DeviceMonitor now optionally takes the VisionClient and probes its /health each tick, emitting a "vision" pseudo-device status (id vision-service, category "vision" — widened the DeviceStatusEvent + frontend DeviceStatus category unions + the footer CATEGORY_KEY/ORDER maps + devices.catVision sq/en) → a "Vision · ready/degraded/offline" chip in the booth footer; NO chip when VISION_ENABLED off. Verified: emits ready/fast_alpr when up, 0 chips when disabled. (3) apps/vision/.env.example (the Python service env) + VISION_* block appended to apps/server/.env.example (the Node side) + a "Configuration" section in [[opencv-anpr-service]] documenting all FOUR layers (python env / node env / per-camera anpr+binding / footer health) and the caveats: the two processes SHARE the VISION_ prefix but need SEPARATE .env files; bind /analyze to 127.0.0.1 (Node is the only caller); models download on first run so cache at deploy; an unbound anpr camera recognizes but every read is refused. Build+lint green. Updated [[opencv-anpr-service]] (Configuration section; SetupWizard-toggle gap closed), vision README.
## [2026-06-19] refactor | ANPR rides the entry/exit snapshot (replaces polling VisionReader)
Reworked the ANPR TRIGGER per the real design goal: when a transient pushes the button or a subscriber passes QR/RFID, the entry/exit fires and takes the evidence snapshot — THAT is the moment to recognize the plate, off the SAME image, tied to the SAME session. So snapshotAsync now takes the VisionClient and, after storing each snapshot from an opt-in (config.anpr) camera, runs ANPR on shot.bytes and records the plate against the session identity (device_events kind:"read" with plate/confidence/region/snapshotId/source:"entry-exit-snapshot"). One image serves both evidence + plate extraction; recognition fires ONLY on a real entry/exit — NO POLLING. The flows (entry/exit/subscription) now take an optional VisionClient and pass it through; server.ts wires it. REMOVED the polling VisionReader (vision-reader.ts deleted) + VISION_POLL_MS/VISION_DEDUPE_MS env. Advisory + fire-and-forget: low-confidence/no-plate records nothing, a vision failure never delays/changes the open, the plate does NOT feed the access decision (the flow already decided) — it's a record ("session X entered on plate AA558EE"). VERIFIED e2e: a simulated entry snapshot on an anpr camera (live fast_alpr) stored the snapshot for session TICKET-SG-1 AND recorded {identity:TICKET-SG-1, plate:AA558EE, confidence:0.999, region:Albania, snapshotId:…}. Build+lint green. Updated [[opencv-anpr-service]] (trigger section + Configuration, polling refs removed) + both .env.example.
## [2026-06-19] feat | Surface recognized plate in the booth UI (SnapshotStrip)
Made the ANPR plate VIEWABLE (it was saved but had no UI). Extended GET /api/snapshots/by-identity/:identity to also query device_events kind:"read" for that identity and return plates[] (plate, confidence, region, direction, snapshotId, at) alongside the existing snapshots + failures. The SnapshotStrip now renders each recognized plate as a cyan "Plate: AA558EE 100%" chip above the images (deduped by plate+direction; title shows region + time) — so it appears in BOTH the booth event-detail modal and the pay modal, beside the evidence photo, no separate screen. session:read gated (same as snapshots). i18n pay.plate sq+en. VERIFIED: by-identity returns plates[] for a seeded read (status 200, {plate:AA558EE, confidence:0.999, region:Albania, direction:entry, snapshotId}). Build+lint green. Updated [[opencv-anpr-service]].