parking_solution/wiki/entities/opencv-anpr-service.md

type, tags, sources, updated, status

type	tags	sources	updated	status
entity	parking vision anpr anti-fraud service		2026-06-15	open

OpenCV ANPR / Vision Service

A local microservice that analyses camera snapshots: reads the licence plate (ANPR) and extracts vehicle attributes for verification. Built by us (decision 2026-06-15) to do recognition host-side on ordinary IP-camera snapshots, replacing the dedicated edge-AI lpr-camera. See decision vision-service.

Two jobs

1. Identity (ANPR). snapshot → { plate, confidence, bbox }. Feeds the existing IdentitySource = "lpr" (parking-session): the plate is a session/identity key and the way a plate-bound subscription is matched.
2. Verification (anti-fraud witness). snapshot → vehicle attributes — at minimum { make?, model?, colour, bodyType }, ideally a compact visual fingerprint (an embedding). This is the answer to plate-spoofing: a fraudster prints a registered/paid plate and drives in with a different car. Plate-reading alone can't catch that; comparing the vehicle seen at entry vs. exit (and vs. the subscription's known car) can. A plate that entered on a red hatchback but exits on a black SUV is a reconciliation anomaly — exactly the independent-witness role the append-only-event-chain flags as the unbuilt gap. See reconciliation.

The two jobs are why this is worth building rather than just plate-OCR: the service is both an identity source and an independent witness, the visual analogue of the whole system's "two records that must reconcile" thesis.

Architecture — separate localhost process

• A Python service (e.g. FastAPI) running on the appliance, called by the Node backend over localhost HTTP (POST /analyze with the JPEG bytes the camera driver already pulls — see lpr-camera "driver/storage boundary": Snapshot.bytes). Source lives in this monorepo at apps/vision/ (Turbo shim; uv-managed deps) — co-located source, separate process; see vision-service-packaging.
• Fully offline (offline-first): all inference is local, no cloud. Model weights ship on the appliance.
• Process isolation is deliberate — it keeps a heavy Python/native/AGPL stack out of the Node app's process and license surface (see licensing below), and gives it its own failure domain. If the service is down/slow, the host falls back (transient ticket path) rather than blocking the lane.
• Request/response (first cut):
  • POST /analyze → { plate: {text, confidence, bbox}|null, vehicle: {colour, bodyType, make?, model?, embedding?}, modelVersion, tookMs }
  • GET /health → readiness + model versions.
• The Node side wraps it behind an internal interface (like a device adapter) so the recognizer can be swapped without touching business logic.

Licensing — scoped AGPL exception (amends the standing rule)

The app is strictly MIT/Apache/BSD (technology-stack, standing-decisions). Accurate ANPR/vehicle models were assumed to be mostly AGPL (Ultralytics YOLO detectors, OpenALPR) or commercial — but the fast-alpr stack (above) is MIT end-to-end, so a permissive ANPR baseline now looks achievable (pending the weight-provenance caveat). The exception below still matters for the strongest models (Ultralytics YOLO) and for the vehicle-verification job. Decision (2026-06-15): allow AGPL inside this service only. It is a separate process, not linked into the app, so its obligations don't reach the Node/React codebase; the app's permissive guarantee is preserved. Recorded as an explicit exception in standing-decisions / vision-service.

• OpenCV core itself is Apache-2.0 (clean either way).
• AGPL note: if the appliance is ever offered as a network service to third parties, AGPL's network-use clause could require offering the service's source — relevant only if productised beyond the on-site appliance; flag at that point.

Recognizer evaluation — fast-alpr is the leading baseline (2026-06-19)

YOLO vs OpenCV is a category error — they're different pipeline layers, not competitors. ANPR is a pipeline: (1) plate detection (find the box → YOLO-family detector), (2) plate OCR (read the crop → a CRNN/CCT or OCR engine), (3) glue (capture/crop/deskew/draw → OpenCV, Apache-2.0, always present). So the real choice is which end-to-end recognizer, and OpenCV is used regardless as the image-handling toolkit.

Leading option: fast-alpr (v0.4.0, 15 Mar 2026). A thin orchestrator over two swappable stages, both on ONNX Runtime — which matches THIS service's decided architecture (separate localhost Python process, offline, swappable behind an interface) almost exactly:

Stage	Default model	Library	License
Plate detection	yolo-v9-t-384-license-plate-end2end	open-image-models	MIT
Plate OCR	cct-xs-v2-global-model	fast-plate-ocr	MIT

• MIT top-to-bottom (library and the published model weights), one maintainer (ankandrew) across all three repos. The detector is open-image-models' own YOLOv9 ONNX export — NOT the Ultralytics AGPL package — so fast-alpr is a permissive baseline that may not even need the scoped AGPL exception below. ⚠️ Caveat (verify before relying on it): a repo's LICENSE covers its code; redistributed model weights can carry separate provenance (YOLOv9 upstream is GPL-3.0; Ultralytics YOLO is AGPL). Confirm the weight training/provenance (model card) before treating "MIT weights" as settled for compliance — the AGPL-in-service exception is the safety net if it doesn't hold.
• CPU-only + fully offline. No runtime ships by default; pick a backend extra — fast-alpr[onnx] (CPU), or [onnx-gpu]/[onnx-openvino]/[onnx-directml]/[onnx-qnn] — which maps onto the "CPU now, small GPU/NPU later" compute question (bom, open-questions).
• Albanian/EU plates: fast-plate-ocr also has a European model trained on 40+ countries (newer than the default global model) — benchmark it against the default for AL accuracy.
• Modular, no lock-in: swap either stage via BaseDetector/BaseOCR (their docs plug in Tesseract). So fast-alpr is the baseline you keep while replacing one stage if needed.

Scope: fast-alpr is plate-only — it does Job 1 (ANPR) but NOT Job 2 (vehicle verification). The anti-spoofing vehicle-attribute/fingerprint stage is still ours to build — but since fast-alpr already standardizes on ONNX Runtime + a YOLO-family detector, the vehicle stage shares that runtime (the coherent outcome). Other options, weaker: OpenALPR (permissive but largely unmaintained, the old "permissive-only, weaker" path); Ultralytics YOLO + PaddleOCR (most accurate/tunable, but YOLO is AGPL → needs the in-service exception; most build effort — the "scale" path if fast-alpr's accuracy disappoints).

Recommendation: prototype with fast-alpr now (permissive, offline, ONNX, fits the decided shape); plan a YOLO-detector fine-tune + PaddleOCR only if production accuracy demands it. Choice kept open pending the weight-provenance check (the AL-plate benchmark below is now done).

Albanian-plate OCR benchmark — keep the default (2026-06-19)

Ran the four candidate fast-plate-ocr models through the full pipeline (YOLOv9 detect → OCR) on real AL plate photos (Wikimedia), CPU, scaffolded service:

OCR model	AA 558 EE	AA 687 KE	Speed	Note
cct-xs-v2-global-model (default)	✓ 0.999	✓ 1.000	33–39 ms	best accuracy + fastest; returns region=Albania
cct-s-v2-global-model	✓ 0.998	✓ 0.999	50–65 ms	as accurate, ~50% slower
global-plates-mobile-vit-v2-model	✓ 0.955	✓ 0.959	33–35 ms	fast, lower confidence
european-plates-mobile-vit-v2-model	✓ 0.784	✓ 0.766	38–46 ms	correct but much lower confidence; misread a synthetic AB123FG→AB123FO

Finding (overturns the prior assumption): the default cct-xs-v2-global-model is the best for Albania — most accurate AND fastest. The "European (40+ country)" model is worse here (~0.77 vs ~1.0 confidence, one synthetic misread), despite the "EU model → better for AL" intuition. So no config change: VISION_OCR_MODEL stays cct-xs-v2-global-model. Caveat: both test photos were clean head-on shots; real booth captures (angled, dirty, night, motion-blur) will lower absolute confidence — the min_confidence=0.5 floor (→ low_confidence → ticket-path fallback) covers that. The ranking should hold; re-benchmark on real on-site captures once the cameras are installed.

Anti-fraud / threat-model fit

• Plate spoofing (the motivating case): vehicle-attribute / fingerprint mismatch entry↔exit or vs. a subscription's registered car → anomaly. Doesn't block on its own (recognition is probabilistic) — it flags for reconciliation and is captured in the signed record.
• The recognition result and the source image both attach to the signed append-only-event-chain entry, so the evidence is tamper-evident even though recognition itself is host-side and fallible.
• Recognition is advisory, never the sole authority to open a barrier where money/access is at stake — confidence thresholds + fallback to ticket/manual; a low-confidence read must not strand a car (fail-state-safety).

Open

• Recognizer choice — fast-alpr (MIT, YOLOv9+CCT on ONNX) is the baseline, AL-benchmarked: the default cct-xs-v2-global-model won over the EU model on real AL plates (table above). The one remaining open item is the model-weight-provenance check (the MIT-weights claim). A re-benchmark on real on-site captures (angled/night/dirty) is wanted once cameras are installed. See vision-service; AGPL still permitted in-service for the stronger fallback.
• Vehicle fingerprint: attribute classifier vs. embedding-similarity; what threshold makes a mismatch an anomaly without false-positiving on lighting/angle.
• Compute footprint on the appliance (CPU-only vs. a small GPU/NPU) — procurement input (bom, open-questions).
• Per-camera opt-in ("optionally bound", user's word): which lanes/cameras route snapshots to the service.