docs(wiki): camera 503/stream, alarm URL helper, reader ICMP liveness
- lpr-camera.md: "503 Device Busy" can be PERSISTENT (main-stream saturation on the G3H) — the real fix is sub-stream selection, not just retry. - device-status-monitoring.md: QR reader health was false-healthy (hardcoded "ready") until the ICMP-ping fix; document the push-device monitoring model. - log entries for both 2026-06-26 sessions. Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
type: concept
|
||||
tags: [parking, device, monitoring, reliability, ui]
|
||||
sources: []
|
||||
updated: 2026-06-18
|
||||
updated: 2026-06-26
|
||||
status: open
|
||||
---
|
||||
|
||||
@@ -25,6 +25,15 @@ talks only to the adapter interfaces ([[device-adapter-pattern]]), never a drive
|
||||
- **Relays / readers / cameras** → the generic `Device.healthCheck()` **reachability** probe every
|
||||
adapter implements (`ready | degraded | offline`). This is presence/up-ness, not a deep fault
|
||||
model — a relay either answers or it doesn't.
|
||||
> **Reader health was a LIE until 2026-06-26.** The QR-reader adapter (a PUSH device: it GETs our
|
||||
> backend on each scan and exposes **no TCP port**) had a hardcoded `healthCheck → { ready, "stub" }`,
|
||||
> so two genuinely-offline readers still showed **green**. A push device that's silent is
|
||||
> indistinguishable from a dead one — so claiming `ready` unconditionally is the worst failure
|
||||
> (false-healthy). Fix: an **optional reader IP** (monitor-only; scans still resolve by serial) +
|
||||
> an **unprivileged ICMP ping** (`drivers/icmp.ts`, shells `/bin/ping` in SOCK_DGRAM mode — no
|
||||
> CAP_NET_RAW, no native dep; the booth compose sets `net.ipv4.ping_group_range`). Reply → `ready`,
|
||||
> no reply → `offline`; **no IP set → `degraded` ("set IP to monitor")**, never a false green.
|
||||
> Verified on hardware: pinged the real readers on the device VLAN. See [[gee-qr-er80]].
|
||||
|
||||
Both collapse to one **traffic-light**: `ready | degraded | offline`, plus a `detail` string. Fail
|
||||
**toward "there's a problem"**, never false-healthy: a probe that throws or times out reads
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
type: entity
|
||||
tags: [parking, hardware, readers, offline-first]
|
||||
sources: [parking-system-architecture]
|
||||
updated: 2026-06-15
|
||||
updated: 2026-06-26
|
||||
---
|
||||
|
||||
# LPR Camera
|
||||
@@ -59,6 +59,42 @@ A **Hikvision** unit ("Camera 20", MAC `94:e1:ac:…`, Hikvision OUI) at `10.0.1
|
||||
- Reaching it from the WSL dev box required forcing the source address (`config.localAddress`,
|
||||
threaded into the driver) — see [[wsl-dev-networking]] (multi-subnet source-selection trap).
|
||||
|
||||
### HTTP 503 "Device Busy" — can be PERSISTENT; the real fix is stream selection (2026-06-26)
|
||||
|
||||
The snapshot endpoint returns **HTTP 503** with the ISAPI body `statusCode 2` / `"Device Busy"` /
|
||||
`subStatusCode deviceBusy` (occasionally **500**). It comes in two flavours, and they need different
|
||||
fixes — **don't assume it's a momentary blip**:
|
||||
|
||||
- **Transient** — the encoder is briefly occupied (another snapshot in flight, a stream starting).
|
||||
Clears on retry within a frame or two.
|
||||
- **Persistent** — the **MAIN-stream encoder is saturated** and 503s on EVERY main-stream snapshot.
|
||||
Confirmed on hardware (**DS-2CD1047G3H-LIU**, 2026-06-26): `channels/101/picture` → 503 on five
|
||||
consecutive probes 800 ms apart, while **`channels/102/picture` (the SUB stream) → 200 every time**,
|
||||
a clean ~15 KB JPEG. So the path/API was correct (the camera answered with a structured Hikvision
|
||||
status); the main encoder was simply never free. A retry loop **cannot** fix this — it just delays
|
||||
the failure.
|
||||
|
||||
**The fix that actually works: snapshot from the SUB stream.** The Hikvision ISAPI channel id is
|
||||
`<channel><stream>` (e.g. ch1 main = `101`, ch1 **sub = `102`**). The driver now has a **`stream`
|
||||
config field** (`1` = main, default for back-compat; `2` = sub). Set the G3H camera to **Sub (02)** in
|
||||
the setup form → its status flips `degraded → ready` (verified live: pulled a 14.7 KB JPEG in ~87 ms).
|
||||
The sub-stream is also the better fit for snapshot/ANPR anyway (smaller/faster; doesn't contend with
|
||||
live-view/recording for the main encoder).
|
||||
|
||||
Two more complementary mitigations (both BUILT, for the *transient* case):
|
||||
1. **Don't cause concurrent busy.** On a vehicle entry two server paths used to snapshot the same
|
||||
camera at once (the ANPR bridge + the advisory `snapshotAsync`); the 2nd concurrent GET drew a 503.
|
||||
They now share ONE pull via `captureSnapshotShared` (deviceId-keyed, `apps/server/src/snapshot.ts`)
|
||||
— the main cause of the slow 2026-06-25 subscriber entry. See [[lane-presence-and-anpr-entry]].
|
||||
2. **Retry a transient one.** `HttpCamera.captureSnapshot` retries 503/500 with a short linear backoff
|
||||
(250/500/750 ms, ≤4 attempts), then fails naming it `(device busy)`; it does NOT retry 401/404
|
||||
(config errors won't self-heal). This recovers a momentary blip but, by design, still fails a
|
||||
PERSISTENTLY-busy main stream — the cue to switch that camera to the sub-stream.
|
||||
|
||||
Covered by `packages/devices/src/drivers/camera.test.ts` (retry behaviour + the main/sub path
|
||||
selection). `healthCheck()` deliberately reports a live 503 as `degraded` (it surfaces a genuinely
|
||||
saturated main stream rather than hiding it behind a retry).
|
||||
|
||||
## Camera PUSH — "Alarm Server" event notifications (2026-06-22)
|
||||
|
||||
Separate from the **pull** snapshot path above: newer Hikvision firmware can **push** an event to
|
||||
|
||||
+36
@@ -1636,3 +1636,39 @@ booth `ModuleNotFoundError` is a STALE image (fix: `booth.sh update`). **Fix:**
|
||||
`dev`/`start`/`recognize` now `uv sync --extra alpr &&` first (self-healing), `.env` set to `fast_alpr`,
|
||||
+ a `dev:stub` escape hatch. Restored real ANPR locally (`/health` → `fast_alpr` ready, model loaded from
|
||||
cache, no download). Documented in [[vision-service-packaging]] ("Two runtimes, one fragile").
|
||||
|
||||
## [2026-06-26] fix | Hikvision snapshot 503 "Device Busy" — stream selection + retry + Alarm URL helper
|
||||
Three camera fixes. (1) **503 Device Busy — the REAL fix is stream selection.** First framed as
|
||||
"transient, just retry" — WRONG for this camera. Hardware probe of **DS-2CD1047G3H-LIU** (10.0.10.13):
|
||||
`channels/101/picture` (MAIN) → 503 `deviceBusy` on 5 consecutive probes 800ms apart, while
|
||||
`channels/102/picture` (SUB) → 200 clean JPEG every time. The main encoder is PERSISTENTLY saturated;
|
||||
a retry loop can't fix it. Added a **`stream` config field** to the Hikvision driver (1=main default
|
||||
for back-compat, 2=sub; ISAPI id `<channel><stream>`). Verified live: setting the camera to Sub flips
|
||||
its status degraded→ready (14.7KB JPEG in ~87ms). (2) **Transient retry** (still useful for a genuine
|
||||
momentary blip + the de-dup case): `HttpCamera.captureSnapshot` retries 503/500 with linear backoff
|
||||
(250/500/750ms ×4), fails naming it `(device busy)`, does NOT retry 401/404. Plus the already-landed
|
||||
`captureSnapshotShared` removing concurrent self-collision. `healthCheck` reports a live 503 as
|
||||
`degraded` (surfaces a saturated main stream rather than hiding it). Covered by `camera.test.ts`
|
||||
(10 tests: retry + main/sub path). (3) **Alarm Server URL helper:** the camera setup form now generates the camera's Alarm
|
||||
Settings (Destination IP / URL / Protocol / Port) ready to paste, so the operator never hunts the
|
||||
deviceId or memorises the endpoint. CRUCIAL: host/port come from the **backend address on the camera's
|
||||
subnet** (`backendIpForDevice` + server port, the same probe the push-IP picker uses) — NOT
|
||||
`window.location.origin` (the SPA's dev/proxy origin, which would wrongly say `localhost:5173`).
|
||||
Verified live: matches the on-camera config field-for-field (10.0.10.203 / …/event / HTTP / 3000).
|
||||
Shows a "save first" (needs a deviceId) then "test first" (needs the resolved backend IP) hint.
|
||||
Documented in [[lpr-camera]] ("503 Device Busy"). Devices 6 new tests; server 168 green.
|
||||
|
||||
## [2026-06-26] fix | QR reader status was a LIE (hardcoded "ready") → real ICMP liveness
|
||||
Two genuinely-OFFLINE QR readers showed GREEN in the status bar. Cause: the QR-reader adapter
|
||||
(`StubReader`) had `healthCheck → { ready, "stub" }` hardcoded — it never probed anything. These are
|
||||
PUSH devices (scan → GET our backend, resolve by serial) that expose **no TCP port**, so a connect
|
||||
probe (cameras/printers) has nothing to hit; the stub "solved" that by lying. False-healthy is the
|
||||
worst failure for a status bar. Fix: an **optional reader IP** (monitor-ONLY — scans still resolve by
|
||||
serial, operation unchanged) + an **unprivileged ICMP ping** (`drivers/icmp.ts`: shells `/bin/ping`
|
||||
`-c1`, exit-0 = reply; no native dep, no CAP_NET_RAW). `healthCheck`: IP replies → `ready`, no reply →
|
||||
`offline`, **no IP → `degraded` ("set IP to monitor")** (never a false green). Booth compose
|
||||
(`docker-compose.prod.yml`) sets `net.ipv4.ping_group_range=0 2147483647` so `/bin/ping` works
|
||||
unprivileged for the non-root container user. Verified on hardware: the readers (10.0.10.7/.8) answer
|
||||
ICMP on the device VLAN (eth1) — distinct MACs — and the UI Test connection shows "● ready — ping
|
||||
10.0.10.7". (NB: an earlier "offline" reading was a WSL wrong-route artifact, not the readers.) Covered
|
||||
by `reader.test.ts` (4 tests). Documented in [[device-status-monitoring]]. Devices +4 tests, all green.
|
||||
|
||||
Reference in New Issue
Block a user