docs(wiki): Albanian-plate OCR benchmark — keep cct-xs-v2-global default

Benchmarked fast-alpr's four fast-plate-ocr models via the full pipeline on real AL
plates (AA558EE, AA687KE), CPU. All four read both correctly; the default
cct-xs-v2-global-model wins on confidence (0.999/1.000) AND speed (33-39ms) and returns
region=Albania. The "European 40+country" model is WORSE here (~0.77 confidence, one
synthetic misread) — overturning the "EU model better for AL" assumption from the prior
research. Decision: no config change. Resolves the AL-accuracy-benchmark open item
(results table + finding added to opencv-anpr-service); weight-provenance remains the
one open recognizer item. Re-benchmark on real on-site captures once cameras installed.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
2026-06-19 15:51:08 +02:00
parent 5cedcaefe1
commit 4833b4373d
2 changed files with 29 additions and 4 deletions
+25 -4
View File
@@ -109,7 +109,27 @@ disappoints).
**Recommendation:** prototype with **fast-alpr** now (permissive, offline, ONNX, fits the decided **Recommendation:** prototype with **fast-alpr** now (permissive, offline, ONNX, fits the decided
shape); plan a YOLO-detector fine-tune + PaddleOCR only if production accuracy demands it. Choice kept shape); plan a YOLO-detector fine-tune + PaddleOCR only if production accuracy demands it. Choice kept
**open** pending the weight-provenance check + an accuracy benchmark on real AL plates. **open** pending the weight-provenance check (the AL-plate benchmark below is now done).
### Albanian-plate OCR benchmark — keep the default (2026-06-19)
Ran the four candidate `fast-plate-ocr` models through the **full pipeline** (YOLOv9 detect → OCR) on
real AL plate photos (Wikimedia), CPU, scaffolded service:
| OCR model | `AA 558 EE` | `AA 687 KE` | Speed | Note |
| --- | --- | --- | --- | --- |
| **`cct-xs-v2-global-model`** (default) | ✓ 0.999 | ✓ **1.000** | **33–39 ms** | best accuracy + fastest; returns `region=Albania` |
| `cct-s-v2-global-model` | ✓ 0.998 | ✓ 0.999 | 50–65 ms | as accurate, ~50% slower |
| `global-plates-mobile-vit-v2-model` | ✓ 0.955 | ✓ 0.959 | 33–35 ms | fast, lower confidence |
| `european-plates-mobile-vit-v2-model` | ✓ 0.784 | ✓ 0.766 | 38–46 ms | correct but **much lower confidence**; misread a synthetic `AB123FG`→`AB123FO` |
**Finding (overturns the prior assumption):** the **default `cct-xs-v2-global-model` is the best for
Albania** — most accurate AND fastest. The "European (40+ country)" model is *worse* here (~0.77 vs
~1.0 confidence, one synthetic misread), despite the "EU model → better for AL" intuition. So **no
config change**: `VISION_OCR_MODEL` stays `cct-xs-v2-global-model`. Caveat: both test photos were
clean head-on shots; real booth captures (angled, dirty, night, motion-blur) will lower absolute
confidence — the `min_confidence=0.5` floor (→ `low_confidence` → ticket-path fallback) covers that.
The ranking should hold; re-benchmark on real on-site captures once the cameras are installed.
## Anti-fraud / threat-model fit ## Anti-fraud / threat-model fit
@@ -125,9 +145,10 @@ shape); plan a YOLO-detector fine-tune + PaddleOCR only if production accuracy d
## Open ## Open
- **Recognizer choice** — **fast-alpr (MIT, YOLOv9+CCT on ONNX) is the evaluated baseline** (see the - **Recognizer choice** — **fast-alpr (MIT, YOLOv9+CCT on ONNX) is the baseline, AL-benchmarked**: the
Recognizer evaluation section above); remaining open items are the **weight-provenance check** and default `cct-xs-v2-global-model` won over the EU model on real AL plates (table above). The one
an **accuracy benchmark on real AL plates** (default global vs. the 40+country EU model). See remaining open item is the **model-weight-provenance check** (the MIT-weights claim). A re-benchmark
on real *on-site* captures (angled/night/dirty) is wanted once cameras are installed. See
[[vision-service]]; AGPL still permitted in-service for the stronger fallback. [[vision-service]]; AGPL still permitted in-service for the stronger fallback.
- **Vehicle fingerprint**: attribute classifier vs. embedding-similarity; what threshold makes a - **Vehicle fingerprint**: attribute classifier vs. embedding-similarity; what threshold makes a
mismatch an anomaly without false-positiving on lighting/angle. mismatch an anomaly without false-positiving on lighting/angle.
+4
View File
@@ -916,3 +916,7 @@ Q: how to IMPLEMENT the vision service — can we use this Turborepo? Settled (s
## [2026-06-19] build | Scaffold apps/vision (ANPR microservice skeleton) ## [2026-06-19] build | Scaffold apps/vision (ANPR microservice skeleton)
Scaffolded the [[opencv-anpr-service|vision service]] per [[vision-service-packaging]]: `apps/vision/` Python/FastAPI, uv-managed, wired into Turbo via a thin package.json shim. Structure: pyproject.toml (light core: fastapi/uvicorn/pydantic; HEAVY recognizer = optional `alpr` extra = fast-alpr+onnxruntime, so `uv sync`+tests run OFFLINE in stub mode with no model download), per-package turbo.json (extends ["//"], build outputs [] → warning-free no-op), .gitignore (venv/caches/*.onnx/models out). vision_service/: app.py (GET /health + POST /analyze, raw octet-stream body so Node POSTs Snapshot.bytes directly; empty→400, oversize→413, not-ready→503), settings.py (env VISION_*), schemas.py (the /analyze contract + a not-yet-populated `vehicle` field for Job 2), recognizer.py (a Recognizer Protocol + StubRecognizer/FastAlprRecognizer — the device-adapter pattern applied to the model; fast-alpr imported lazily so missing models ⇒ ready=False, not a crash). VERIFIED: turbo run lint|test|build includes @parking/vision (ruff/pytest/no-op shim) green; uv run mypy strict-clean; uvicorn boots + serves /health (ready, stub-0) and /analyze (contract shape) live; pnpm workspace 6→7. NOT built: the Node VisionClient adapter, a Dockerfile + model fetch, and Job 2 (vehicle verification). Updated [[vision-service-packaging]] (As-scaffolded section), CLAUDE.md layout already lists apps/vision. Scaffolded the [[opencv-anpr-service|vision service]] per [[vision-service-packaging]]: `apps/vision/` Python/FastAPI, uv-managed, wired into Turbo via a thin package.json shim. Structure: pyproject.toml (light core: fastapi/uvicorn/pydantic; HEAVY recognizer = optional `alpr` extra = fast-alpr+onnxruntime, so `uv sync`+tests run OFFLINE in stub mode with no model download), per-package turbo.json (extends ["//"], build outputs [] → warning-free no-op), .gitignore (venv/caches/*.onnx/models out). vision_service/: app.py (GET /health + POST /analyze, raw octet-stream body so Node POSTs Snapshot.bytes directly; empty→400, oversize→413, not-ready→503), settings.py (env VISION_*), schemas.py (the /analyze contract + a not-yet-populated `vehicle` field for Job 2), recognizer.py (a Recognizer Protocol + StubRecognizer/FastAlprRecognizer — the device-adapter pattern applied to the model; fast-alpr imported lazily so missing models ⇒ ready=False, not a crash). VERIFIED: turbo run lint|test|build includes @parking/vision (ruff/pytest/no-op shim) green; uv run mypy strict-clean; uvicorn boots + serves /health (ready, stub-0) and /analyze (contract shape) live; pnpm workspace 6→7. NOT built: the Node VisionClient adapter, a Dockerfile + model fetch, and Job 2 (vehicle verification). Updated [[vision-service-packaging]] (As-scaffolded section), CLAUDE.md layout already lists apps/vision.
## [2026-06-19] query | Albanian-plate OCR benchmark — keep the default (cct-xs-v2-global)
Benchmarked fast-alpr's four candidate fast-plate-ocr models via the FULL pipeline (YOLOv9 detect → OCR) on real AL plate photos (Wikimedia: AA558EE, AA687KE), CPU, scaffolded apps/vision service. ALL FOUR read both plates correctly; the differentiator is confidence + speed: cct-xs-v2-global (default) 0.999/1.000 @ 33–39ms AND returns region=Albania; cct-s-v2-global same accuracy ~50% slower; global-mobile-vit ~0.955 fast; european-mobile-vit-v2 (the "40+ country EU" model) correct but MUCH lower confidence (~0.77) and misread a synthetic AB123FG→AB123FO. FINDING (overturns the "EU model → better for AL" assumption from the prior research turn): the global cct-xs default WINS for Albania — most accurate AND fastest. Decision: no config change, VISION_OCR_MODEL stays cct-xs-v2-global-model. Caveat: test photos were clean head-on shots; real booth captures (angle/night/dirt/blur) will lower confidence — the min_confidence=0.5 floor → low_confidence → ticket-path fallback covers it; re-benchmark on on-site captures once cameras installed. Resolves the AL-accuracy-benchmark open item in [[opencv-anpr-service]] (added a results table + the keep-default finding); the weight-provenance check remains the one open recognizer item.