From 547061edf921666721533380132bc30c1ae76f34 Mon Sep 17 00:00:00 2001 From: Julian Cuni Date: Mon, 22 Jun 2026 12:50:44 +0200 Subject: [PATCH] docs(camera): Hik event-push gotchas + dead-camera diagnostic method Captures the hard-won findings from the field session: the WSL source-IP rewrite + skipSourceIpCheck fix, the boolean-as-string setup bug, the unreliable "Test" button, the latching httpBroken flag, and Notify- Surveillance-Center vs HTTP-Alarm-Server. Adds a "diagnose a non-pushing camera from its OWN state" runbook (alertStream heartbeat silence, SSH showStatus EventScribe:except, dmesg RTC/UBIFS, netstat outbound watch) and documents the verified-dead DS-2CD1043G2-LIU unit (defective event engine, survives factory reset -> RMA), with the pull+vision fallback. Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V --- wiki/entities/lpr-camera.md | 47 +++++++++++++++++++++++++++++++++++++ wiki/log.md | 14 +++++++++++ 2 files changed, 61 insertions(+) diff --git a/wiki/entities/lpr-camera.md b/wiki/entities/lpr-camera.md index 3df7fdc..ec1139e 100644 --- a/wiki/entities/lpr-camera.md +++ b/wiki/entities/lpr-camera.md @@ -86,3 +86,50 @@ Center**, then **Alarm Settings → Alarm Server**, makes the camera **HTTP-POST next step once the real payload is known. If the camera emits its own plate (``), we can use it as an advisory read directly; otherwise the server hands the attached/pulled frame to the [[opencv-anpr-service|vision service]] for ANPR. + +### Gotchas learned the hard way (2026-06-22 field session) + +Several traps surfaced trying to get a real camera to push. In order of how long each cost: + +- **WSL rewrites the inbound source IP.** On the dev host (WSL mirrored mode), an inbound LAN packet + arrives at our server with its **source rewritten to the host's own IP** (`10.0.10.203`), not the + camera's. The source-IP guard then rejects every push as a mismatch. Fix: a per-device + **`skipSourceIpCheck`** config flag (a Setup checkbox) that bypasses the IP guard — the signed + ledger + optional Digest remain the real guards. Leave OFF on a normal LAN. +- **The setup checkbox saved booleans as the STRING `"true"`.** The generic config-field form had no + boolean renderer, so a `type:"boolean"` field fell through to a text input. Fixed (checkbox + renderer); the server also coerces `"true"`/`1`/`yes`/`on` defensively. +- **The camera's "Test" button proves almost nothing.** It does a TCP/connectivity probe and reports + "service available" on ANY HTTP reply (even our 404) — it does **not** POST a real event to your + URL. Only a real detection (or the ISAPI `httpHosts//test`) actually exercises the path. +- **`httpBroken` latches.** Once the camera marks the host broken (from earlier failed deliveries), + it stays `true` across reboots and won't retry. Clear it by **re-PUTting** the httpHost config + (`PUT /ISAPI/Event/notification/httpHosts/1` with `false`). +- **"Notify Surveillance Center" ≠ the HTTP Alarm Server** on some firmware (separate upload + channels); and **basic Motion Detection may not POST to the httpHost** while smart/VCA events do. + Always confirm the **Arming Schedule** covers the test time, too (a silent killer). + +### Diagnosing a non-pushing camera — read its OWN state, don't guess + +Status APIs lie (a `diskfull` error on a camera with **no disk** is a misleading label). The decisive +checks, in order — they need no cooperation from our server: + +1. **`GET /ISAPI/Event/notification/alertStream`** (Digest) — the camera's live event bus. A HEALTHY + camera streams periodic multipart heartbeat frames even when idle, and an event line when one + fires. **Total silence (no heartbeat) for 20s = the event subsystem is producing nothing** — the + fault is internal, upstream of any linkage (HTTP host, FTP, alarm stream all die together). +2. **SSH into the camera** (BusyBox "psh" restricted shell; default creds work): `showStatus` → + look for **`EventScribe: except`** (event subsystem in an exception state); `dmesg` → UBIFS + "recovery needed" on every volume (unclean flash) and **`fh rtc get time error ffffffff`** (dead + RTC — a hardware fault; the clock reads 1970, which wedges time-keyed event logic). `ps` shows the + `davinci` app. `df`/`printPart` are NOT available; partition sizes via the UBIFS lines in `dmesg`. +3. **`netstat` on the camera** while triggering — watch for an OUTBOUND line `cam:port → server:3000`. + None appearing = the camera never even attempts the POST. + +> **Verified-dead unit (2026-06-22):** a `DS-2CD1043G2-LIU` (V5.8.10, 8 months old) failed all three: +> silent alertStream, `EventScribe: except`, `diskfull` on `Event/triggers`, dead RTC, and **zero +> outbound to `:3000`** across a 3-minute netstat watch — through reboot, basic reset, AND full +> factory reset. Conclusion: **defective event engine (RMA)**, not our integration. Our endpoint was +> verified correct throughout (it accepts + records a POST the instant one actually arrives). When a +> camera's event push is dead, fall back to the **pull + [[opencv-anpr-service|vision]]** path, which +> works with the same camera (it still serves snapshots fine). diff --git a/wiki/log.md b/wiki/log.md index 8abb1ad..9adc73a 100644 --- a/wiki/log.md +++ b/wiki/log.md @@ -1384,3 +1384,17 @@ setup offers the backend push IP). NOT yet a barrier trigger or DeviceReadEvent read-bus/ANPR wiring is the next step once the real payload is captured (advisory-only rule still governs). Tests: hikvision-alarm.test.ts (6: vehicle XML summary, ANPR plate, raw JSON, wrong-IP 404, disabled 404, unknown-device 404); server 109/109, build+lint 14/14. See [[lpr-camera]]. + +## [2026-06-22] debug | Hikvision event-push field session — fixes + a verified-dead camera +Long session getting a real Hik camera to POST events. Server/integration fixes (committed): listen +on ALL methods (camera probes with GET/etc, not just POST); record rejected pushes too (kind +"alarm-rejected" + reason) so "nothing arrived" is never ambiguous; a GET /api/devices/hikvision/ +alarms read endpoint; per-device skipSourceIpCheck (WSL mirrored mode REWRITES the inbound source IP +to the host's own, so the source-IP guard rejected every push); and a real checkbox renderer for +type:"boolean" config fields (they were saving the STRING "true"). Then proved — via the camera's +OWN state, not our server — that the specific DS-2CD1043G2-LIU unit has a DEAD event engine: silent +alertStream (no heartbeat), EventScribe:except, diskfull on Event/triggers, dead RTC (rtc get time +error / clock at 1970), and ZERO outbound to :3000 over a 3-min netstat watch, surviving reboot + +basic reset + FULL factory reset. Verdict: defective camera (RMA), not our code. Captured the +diagnostic method (alertStream silence / SSH showStatus / netstat) in [[lpr-camera]]. Fallback for a +dead-push camera: pull + [[opencv-anpr-service|vision]] (the same camera still serves snapshots).