898cf1953a
- lpr-camera.md: "503 Device Busy" can be PERSISTENT (main-stream saturation on the G3H) — the real fix is sub-stream selection, not just retry. - device-status-monitoring.md: QR reader health was false-healthy (hardcoded "ready") until the ICMP-ping fix; document the push-device monitoring model. - log entries for both 2026-06-26 sessions. Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
200 lines
14 KiB
Markdown
200 lines
14 KiB
Markdown
---
|
||
type: entity
|
||
tags: [parking, hardware, readers, offline-first]
|
||
sources: [parking-system-architecture]
|
||
updated: 2026-06-26
|
||
---
|
||
|
||
# LPR Camera
|
||
|
||
License-plate-recognition camera. For **casual/transient** vehicles, the **plate acts as ticket +
|
||
an independent record**. (See [[parking-system-architecture]] §8, §9.)
|
||
|
||
> **Superseded direction (2026-06-15):** recognition now runs **host-side** on snapshots from
|
||
> ordinary Hikvision/Dahua cameras via the [[opencv-anpr-service]], **not** on a dedicated edge-AI
|
||
> LPR camera — see [[vision-service]]. The edge-AI camera below is kept as the original assumption /
|
||
> a fallback option, but is no longer the planned path. The host-side service also does **vehicle
|
||
> verification** (anti-plate-spoofing), which an edge-LPR camera does not.
|
||
|
||
- **Edge AI (original assumption)**: recognition runs **on-device**, so it keeps working with no
|
||
internet — fits [[offline-first]].
|
||
- It's a **host-side** identity source: only the host sees the read; the host decides and
|
||
commands the relay open (the [[uhppote-controller]] is demoted to a commanded relay for that
|
||
lane). See [[entry-exit-readers]].
|
||
- Being host-in-the-loop is **good for fraud detection** — you get two independent records (the
|
||
host's signed [[append-only-event-chain]] entry + the controller's remote-open event) that
|
||
should reconcile one-to-one; any mismatch is an anomaly.
|
||
- Mounting: within ~15° of vehicle travel at a controlled chokepoint for best reads.
|
||
|
||
## Snapshot driver (entry/exit fraud-control record)
|
||
|
||
Separate from edge-AI LPR: the camera driver (`packages/devices/src/drivers/camera.ts`) does
|
||
**snapshot-on-event** — the host pulls a still over HTTP when an entry/exit fires and stores it,
|
||
referenced from the signed [[append-only-event-chain]] entry as an independent record. The camera
|
||
**pulls, it does not push** — so it is NOT `pushesToBackend` and the setup wizard correctly hides
|
||
the "Backend push IP" field for it (gated on the driver's `pushesToBackend` flag; only
|
||
[[dingtian-relay]] sets it).
|
||
|
||
- **Hikvision** uses **ISAPI**: `GET /ISAPI/Streaming/channels/<id>/picture` (`101` = ch1 main
|
||
stream) with **HTTP Digest** auth. The "Enable Hikvision-CGI" toggle (Network → Advanced →
|
||
Integration Protocol) is a *different* legacy CGI surface — **not** needed for ISAPI.
|
||
- **Dahua** uses CGI: `GET /cgi-bin/snapshot.cgi?channel=<n>` (0-based channel; the wizard's
|
||
1-based channel is decremented).
|
||
|
||
**Driver / storage boundary:** the driver FETCHES the image bytes (client-side HTTP Digest in
|
||
`drivers/http-digest.ts`) and returns them on `Snapshot.bytes`; **storage is the caller's job**
|
||
(the future entry/exit flow stores the bytes + mints a durable `imageRef`). This keeps the device
|
||
adapter free of any filesystem/blob-store dependency. `healthCheck()` is honest — it actually pulls
|
||
a frame (exercising reachability + auth + path/channel in one shot), not a fake `ready/stub`.
|
||
|
||
### Verified on hardware (2026-06-15)
|
||
|
||
A **Hikvision** unit ("Camera 20", MAC `94:e1:ac:…`, Hikvision OUI) at `10.0.10.121`, creds
|
||
`admin` / `admin123` (Digest), TCP 80:
|
||
|
||
- Initial `curl` test confirmed the ISAPI path returns a 2688×1520 JPEG (~306 KB).
|
||
- The **real driver** (no longer a stub) was then run end to end against it:
|
||
`healthCheck()` → `ready` (pulled a frame), `captureSnapshot()` → valid `image/jpeg`, ~322 KB,
|
||
correct JPEG magic. Digest handshake works through `HttpCamera`.
|
||
- Reaching it from the WSL dev box required forcing the source address (`config.localAddress`,
|
||
threaded into the driver) — see [[wsl-dev-networking]] (multi-subnet source-selection trap).
|
||
|
||
### HTTP 503 "Device Busy" — can be PERSISTENT; the real fix is stream selection (2026-06-26)
|
||
|
||
The snapshot endpoint returns **HTTP 503** with the ISAPI body `statusCode 2` / `"Device Busy"` /
|
||
`subStatusCode deviceBusy` (occasionally **500**). It comes in two flavours, and they need different
|
||
fixes — **don't assume it's a momentary blip**:
|
||
|
||
- **Transient** — the encoder is briefly occupied (another snapshot in flight, a stream starting).
|
||
Clears on retry within a frame or two.
|
||
- **Persistent** — the **MAIN-stream encoder is saturated** and 503s on EVERY main-stream snapshot.
|
||
Confirmed on hardware (**DS-2CD1047G3H-LIU**, 2026-06-26): `channels/101/picture` → 503 on five
|
||
consecutive probes 800 ms apart, while **`channels/102/picture` (the SUB stream) → 200 every time**,
|
||
a clean ~15 KB JPEG. So the path/API was correct (the camera answered with a structured Hikvision
|
||
status); the main encoder was simply never free. A retry loop **cannot** fix this — it just delays
|
||
the failure.
|
||
|
||
**The fix that actually works: snapshot from the SUB stream.** The Hikvision ISAPI channel id is
|
||
`<channel><stream>` (e.g. ch1 main = `101`, ch1 **sub = `102`**). The driver now has a **`stream`
|
||
config field** (`1` = main, default for back-compat; `2` = sub). Set the G3H camera to **Sub (02)** in
|
||
the setup form → its status flips `degraded → ready` (verified live: pulled a 14.7 KB JPEG in ~87 ms).
|
||
The sub-stream is also the better fit for snapshot/ANPR anyway (smaller/faster; doesn't contend with
|
||
live-view/recording for the main encoder).
|
||
|
||
Two more complementary mitigations (both BUILT, for the *transient* case):
|
||
1. **Don't cause concurrent busy.** On a vehicle entry two server paths used to snapshot the same
|
||
camera at once (the ANPR bridge + the advisory `snapshotAsync`); the 2nd concurrent GET drew a 503.
|
||
They now share ONE pull via `captureSnapshotShared` (deviceId-keyed, `apps/server/src/snapshot.ts`)
|
||
— the main cause of the slow 2026-06-25 subscriber entry. See [[lane-presence-and-anpr-entry]].
|
||
2. **Retry a transient one.** `HttpCamera.captureSnapshot` retries 503/500 with a short linear backoff
|
||
(250/500/750 ms, ≤4 attempts), then fails naming it `(device busy)`; it does NOT retry 401/404
|
||
(config errors won't self-heal). This recovers a momentary blip but, by design, still fails a
|
||
PERSISTENTLY-busy main stream — the cue to switch that camera to the sub-stream.
|
||
|
||
Covered by `packages/devices/src/drivers/camera.test.ts` (retry behaviour + the main/sub path
|
||
selection). `healthCheck()` deliberately reports a live 503 as `degraded` (it surfaces a genuinely
|
||
saturated main stream rather than hiding it behind a retry).
|
||
|
||
## Camera PUSH — "Alarm Server" event notifications (2026-06-22)
|
||
|
||
Separate from the **pull** snapshot path above: newer Hikvision firmware can **push** an event to
|
||
us. Under **Event → Smart/VCA** (e.g. line crossing / intrusion / "Vehicle Detection") the unit
|
||
exposes **Detection Target: Human / Vehicle** — selecting **Vehicle** + **Notify Surveillance
|
||
Center**, then **Alarm Settings → Alarm Server**, makes the camera **HTTP-POST an
|
||
`EventNotificationAlert`** to a URL we host on each detection. Same machine-call shape as the
|
||
[[dingtian-relay]] Input Link push — no polling.
|
||
|
||
- **Ingress:** `POST /api/devices/hikvision/:deviceId/event` (`apps/server/src/routes/hikvision-alarm.ts`).
|
||
**Source-IP guarded** (must come from the device's configured `host`) + **optional HTTP Digest**
|
||
(some firmware can't authenticate the Alarm Server call → source-IP only). NOT behind the SPA
|
||
cookie/CSRF (it's a device call), exactly like the Dingtian push.
|
||
- **Config:** added to the `hikvision` driver — `alarmPushEnabled` (bool), `pushUser`/`pushPassword`
|
||
(optional Digest). The driver is now `pushesToBackend: true`, so first-run setup offers the backend
|
||
push IP. Point the camera's Alarm Server at `http://<backend-ip>:<port>/api/devices/hikvision/<deviceId>/event`.
|
||
- **Discovery-first:** the endpoint is **permissive** — accepts ANY content-type as raw bytes (event
|
||
XML, multipart-with-JPEG, or JSON; Hik's format varies by model/firmware), records the **verbatim
|
||
body** as a `kind:"alarm"` device_event, and best-effort extracts `eventType` / `target` / `plate`
|
||
/ `dateTime` / `channelID`. The point of this first cut is to **see exactly what a given camera
|
||
sends** (inspect via `GET /api/events` or the server log) before wiring it to the read bus.
|
||
- **Not yet a barrier trigger.** It records + breadcrumbs only; it does NOT emit a `DeviceReadEvent`
|
||
or open anything. A plate read is **advisory, never the sole reason** a barrier opens
|
||
([[append-only-event-chain]], [[opencv-anpr-service]]). Two consumers were since designed off this
|
||
same vehicle event — see **[[lane-presence-and-anpr-entry]]**: (a) BUILT — advisory lane busy/free
|
||
booth lights; (b) BUILT (2026-06-22) — the ANPR "bridge" (`anpr-entry.ts`) that snapshots → ANPR →
|
||
emits a `kind:"plate"` read for a SUBSCRIBER match through the existing gated flow (a small
|
||
`apps/server` handler class, not a service). If the camera ever emits its own `<plateNumber>` we'd
|
||
use it directly; this `DS-2CD1043G2`
|
||
does not, so the server pulls the frame and hands it to the [[opencv-anpr-service|vision service]].
|
||
|
||
### Gotchas learned the hard way (2026-06-22 field session)
|
||
|
||
Several traps surfaced trying to get a real camera to push. In order of how long each cost:
|
||
|
||
- **WSL rewrites the inbound source IP.** On the dev host (WSL mirrored mode), an inbound LAN packet
|
||
arrives at our server with its **source rewritten to the host's own IP** (`10.0.10.203`), not the
|
||
camera's. The source-IP guard then rejects every push as a mismatch. Fix: a per-device
|
||
**`skipSourceIpCheck`** config flag (a Setup checkbox) that bypasses the IP guard — the signed
|
||
ledger + optional Digest remain the real guards. Leave OFF on a normal LAN.
|
||
- **The setup checkbox saved booleans as the STRING `"true"`.** The generic config-field form had no
|
||
boolean renderer, so a `type:"boolean"` field fell through to a text input. Fixed (checkbox
|
||
renderer); the server also coerces `"true"`/`1`/`yes`/`on` defensively.
|
||
- **The camera's "Test" button proves almost nothing.** It does a TCP/connectivity probe and reports
|
||
"service available" on ANY HTTP reply (even our 404) — it does **not** POST a real event to your
|
||
URL. Only a real detection (or the ISAPI `httpHosts/<id>/test`) actually exercises the path.
|
||
- **`httpBroken` latches.** Once the camera marks the host broken (from earlier failed deliveries),
|
||
it stays `true` across reboots and won't retry. Clear it by **re-PUTting** the httpHost config
|
||
(`PUT /ISAPI/Event/notification/httpHosts/1` with `<httpBroken>false</httpBroken>`).
|
||
- **"Notify Surveillance Center" ≠ the HTTP Alarm Server** on some firmware (separate upload
|
||
channels). Always confirm the **Arming Schedule** covers the test time, too (a silent killer).
|
||
- **⭐ THE ROOT CAUSE (2026-06-22): no detection AREA drawn.** This is what actually defeated us for
|
||
most of a day. On the motion/smart-detection page there's a **Draw Area** step — if **no region is
|
||
drawn on the frame, the camera detects nothing, generates NO event, and therefore posts nothing**
|
||
anywhere (httpHost, FTP, alarm stream all stay silent because there's no event upstream). Enabling
|
||
the detection + ticking Notify Surveillance Center is **not enough** — you must draw the region.
|
||
Once an area was drawn, the very first vehicle produced a clean POST. **Check this FIRST.**
|
||
|
||
### Confirmed real payload (DS-2CD1043G2-LIU, V5.8.10, 2026-06-22)
|
||
|
||
What this camera actually POSTs on a motion event with a target — captured end-to-end:
|
||
|
||
- **`Content-Type: multipart/form-data; boundary=boundary`**, one XML part named `MoveDetection.xml`
|
||
(`Content-Type: application/xml`). A real frame/JPEG *may* be attached as a second part on other
|
||
event types — our endpoint stores the readable head; splitting an image part to `snapshots` is a
|
||
forward step (not needed for plain motion).
|
||
- The XML is an `EventNotificationAlert` with the fields we care about:
|
||
- `<eventType>VMD</eventType>` (Video Motion Detection) + `<eventState>active</eventState>`
|
||
- **`<targetType>vehicle</targetType>`** — the camera classifies **vehicle vs human ON-DEVICE**.
|
||
(Field is `targetType`, NOT `detectionTarget`.) This means simple presence + class comes for
|
||
free, no vision model needed for that part.
|
||
- `<targetInfo><targetRect>` with normalized `X/Y/width/height` (0–1) — the **bounding box**.
|
||
- `<channelID>`, `<macAddress>` (provenance), `<dateTime>` — **but the dateTime is GARBAGE**
|
||
(`2032-…`) because this unit's **RTC is dead** (see below); we use our own server receive time,
|
||
never the camera's. (No `<plateNumber>` — this is a motion event, not an ANPR camera.)
|
||
|
||
### If a camera still won't push — diagnostics (read its OWN state)
|
||
|
||
Only after confirming the **detection area is drawn** + arming schedule covers now + Notify
|
||
Surveillance Center is on. These read the camera directly (no cooperation from our server):
|
||
|
||
1. **`netstat` on the camera (via SSH) while you trigger** — watch for an OUTBOUND line
|
||
`cam:port → server:3000`. It appearing = the camera fired and is delivering (then check our
|
||
`/api/devices/hikvision/alarms`). None = no event was generated (almost always: **no area drawn**).
|
||
2. **`GET /ISAPI/Event/notification/alertStream`** (Digest, needs a clean handshake) — the live event
|
||
bus. NB: a `curl --digest` tap that fails the handshake returns empty and looks like "no events"
|
||
— don't over-read silence here (this misled us); the netstat watch above is more reliable.
|
||
3. **SSH `showStatus` / `dmesg`** expose internal state. ⚠ **Caveat learned the hard way:** these
|
||
surface scary-looking strings that are **red herrings** — `EventScribe: except`, a `diskfull`
|
||
error on `Event/triggers` (on a camera with **no disk**), and `fh rtc get time error` / a 1970
|
||
clock. On our unit ALL of these were present **and the camera worked fine** once an area was
|
||
drawn. The dead RTC is real (hence the bogus `dateTime`) but **harmless** to event push. **Do NOT
|
||
conclude "dead camera / RMA" from these** — they are not proof of a broken event engine.
|
||
|
||
> **Correction (2026-06-22):** an earlier version of this page concluded this DS-2CD1043G2-LIU was a
|
||
> **defective unit needing RMA**, based on the silent alertStream + `diskfull`/`EventScribe:except` +
|
||
> dead RTC surviving a full factory reset. **That was WRONG.** The camera was healthy; the real cause
|
||
> was simply **no detection area drawn**, so no event was ever generated. The `diskfull`/RTC findings
|
||
> were unrelated quirks (RTC genuinely dead, but it doesn't block event push). Lesson: don't
|
||
> escalate to "hardware fault" while a basic config precondition (the drawn region) is unmet — and
|
||
> treat vendor status-API error strings as unreliable. The pull + [[opencv-anpr-service|vision]] path
|
||
> remains a valid fallback, but it was not needed here.
|