diff --git a/komodo/resources.toml b/komodo/resources.toml index 4bcbdc4..1223d4a 100644 --- a/komodo/resources.toml +++ b/komodo/resources.toml @@ -64,6 +64,39 @@ EVENT_SIGNING_KEY=[[park_lab_event_signing_key]] BACKUP_KEY=[[park_lab_backup_key]] """ +############################################################################## +# art-docker-station — second LAB bench box (hardware/dev testing, no real traffic). Same tier as +# park-lab: chases `dev` (compose files + MOVING image tag), own art_docker_station_* secret refs +# (never shared with park-lab or a real booth, even lab-to-lab — per-box blast radius). +############################################################################## + +[[stack]] +name = "art-docker-station" +[stack.config] +server = "art-docker-station" +git_provider = "git.infra.msai.al" +git_account = "komodo" +repo = "mca/parking_solution" +branch = "dev" +file_paths = [ + "docker-compose.yml", + "docker-compose.prod.yml" +] +registry_provider = "git.infra.msai.al" +registry_account = "komodo" +environment = """ +REGISTRY=git.infra.msai.al/mca/parking_solution +# Lab tier: the MOVING dev tag — redeploy pulls the latest dev build. Pin to a +# dev- only when reproducing a specific state. +TAG=dev +COOKIE_SECURE=0 +VISION_ENABLED=1 +WS_ALLOWED_ORIGINS= +JWT_SECRET=[[art_docker_station_jwt_secret]] +EVENT_SIGNING_KEY=[[art_docker_station_event_signing_key]] +BACKUP_KEY=[[art_docker_station_backup_key]] +""" + ############################################################################## [[stack]] diff --git a/wiki/concepts/printer-usb-transport.md b/wiki/concepts/printer-usb-transport.md index f5d37ac..a53aca0 100644 --- a/wiki/concepts/printer-usb-transport.md +++ b/wiki/concepts/printer-usb-transport.md @@ -2,7 +2,7 @@ type: concept tags: [parking, device, printer, transport, usb, escpos, provisioning] sources: [] -updated: 2026-07-06 +updated: 2026-08-30 status: settled --- @@ -140,5 +140,70 @@ hint. The transport option label no longer hardcodes lp0. > the monitor would mark a perfectly working printer offline/degraded. Over USB the two drivers > behave identically (reachability floor), so either works post-fix. See [[rongta-printer]]. +## Field bug — cover-open re-enumeration wedges the container's `/dev/usb` view; only `docker restart`, not a host reboot, clears it (investigated 2026-08-30, unconfirmed root cause) + +**Symptom (park-buzi, unknown/"Generic" USB printer, model not yet identified — see below):** every +time the booth operator opens the printer's paper-roll cover to reload paper, the printer's status +goes `offline`/faulty in the app and **never self-recovers** — not after the cover closes, not after +a full appliance reboot. The only fix found so far is SSH in and `docker restart server`. + +**Ruled out at the application layer.** Traced `sendRawUsb`/`probeUsb` in `printer-escpos.ts`: every +print AND every poll tick (`device-monitor.ts` 8s / `printer-monitor.ts` 5s) does a fresh +`open()` → write/probe → `close()` against the configured `devicePath`. **No fd, socket, or driver +instance is held across calls** — `driver.create(config)` is a throwaway object with no persistent +handle. So a naive "stale Node file descriptor" explanation does not fit this codebase; the +app-layer retry-by-fresh-open-every-poll should self-heal within one poll cycle if the kernel's view +of the device node is current. + +**Leading hypothesis: the container's bind-mount of `/dev/usb`, not the Node process, holds the +stale state.** Docker Compose wires the printer in as a **directory bind-mount** +(`docker-compose.prod.yml`, `volumes: - /dev/usb:/dev/usb`), chosen deliberately (per its own +comment) so the app survives the printer renumbering to a different `lpN`. But many USB thermal +printers cut power to their own USB interface board when the cover-open microswitch trips (a +hardware safety/power feature, not just a status flag) — the printer drops off the bus and +re-enumerates, potentially as a new device node, when the cover closes. The **host** kernel picks +this up fine; the **container's mount namespace**, once established, is a known Docker/OverlayFS +sharp edge for `/dev` subtree bind-mounts — it can keep resolving the old node until the mount +itself is redone. + +- `docker restart server` recreates the container's mount namespace → the `/dev/usb` bind-mount is + redone against current host state → the new node is picked up → fixed. +- A full host reboot restarts the container too (`restart: always`), but as a boot-time race: if the + container starts before the USB subsystem finishes settling, or the printer re-enumerated some + time *before* the reboot and Docker doesn't necessarily redo an already-satisfied bind-mount + target on a policy-driven restart, the container can come back up still bound to the pre-incident + view. This matches the exact reported asymmetry (reboot doesn't fix it; explicit restart does). + +**Not yet confirmed on hardware** — this is the leading theory, not a verified root cause. To +confirm at the next occurrence, BEFORE restarting anything: +```bash +# host: +ls -la /dev/usb/ && stat /dev/usb/lp1 +# container: +docker exec server ls -la /dev/usb/ && docker exec server stat /dev/usb/lp1 +``` +A major:minor or inode mismatch between host and container is the smoking gun. Also worth +capturing on the lab RONGTA (different printer, but same cover-open mechanism is plausible): +`watch -n1 lsusb` + `sudo dmesg -w | grep -i -E 'usb|disconnect'` while cycling the cover, to see +whether the Bus/Device number changes. + +**Candidate fixes, not yet implemented** (ranked cheapest-to-most-invasive): +1. A host-side watchdog/udev rule that detects re-enumeration of this printer (match vendor:product + ID) and runs `docker restart server` automatically — turns the manual SSH fix into a self-healing + one without touching app code. +2. Same idea but event-driven via a udev rule or systemd path unit watching `/dev/usb`, rather than + polling. +3. Switch the compose device wiring from the directory bind-mount to a specific `--device=` cgroup + passthrough + a udev rule pinning a stable symlink name — reintroduces the renumbering fragility + the directory bind-mount was chosen to avoid, so only worth doing alongside (1)/(2), not instead. + +**Open sub-question — printer identity.** The park-buzi unit shows as "Generic (unknown)" in the +app; not yet identified by vendor/product ID. Lab reproduction uses a **RONGTA** unit instead (not +the same hardware), so the lab cannot currently reproduce the park-buzi symptom directly — only +validate the general re-enumeration mechanism. Commands to identify the real park-buzi printer next +time it's reachable via SSH: `lsusb`, `udevadm info -q property -n /dev/usb/lp1`, `udevadm info -a +-n /dev/usb/lp1`. This mirrors the same discovery gap already noted above under "Device discovery" +(sysfs `ieee1284_id` enrichment) — once identified, fold the model into that mechanism's coverage. + Related: [[rongta-printer]], [[printer-status-monitoring]], [[printer-roles-failover]], [[appliance-provisioning]], [[network-isolation]], [[technology-stack]]. diff --git a/wiki/index.md b/wiki/index.md index a6d6c1b..316db1c 100644 --- a/wiki/index.md +++ b/wiki/index.md @@ -58,7 +58,7 @@ Counts: 4 sources · 19 entities · 47 concepts · 8 decision records. - [[hardware-signer-options]] — where the ledger signing key should live (TPM interim → USB-HSM target; ATECC608 upcoming, not on-site) so a host-owner can't forge the chain. - [[reconciliation]] — the real anti-fraud control; what remote sync actually is. - [[disk-os-hardening]] — the *why* of host hardening: LUKS FDE + TPM-sealed auto-unlock (PCR 7) + Secure Boot + GRUB edit-lock + unprivileged operator + firmware/dbx lockdown; secondary control (reconciliation is the main event). Commands → [[appliance-provisioning]]. -- [[backup-recovery]] — admin-driven encrypted full-DB backup (local/SMB/SFTP) + DR; signing key escrowed & decoupled from TPM so the ledger survives total hardware loss; restore is admin-only. +- [[backup-recovery]] — admin-driven encrypted full-DB backup (local/SMB/SFTP) + DR; signing key escrowed & decoupled from TPM so the ledger survives total hardware loss; restore is admin-only; last-success/error status + schedule are now restart-durable (migration 0025, fixed a "shows Never despite valid backups" bug). ## Concepts — device architecture & safety - [[device-adapter-pattern]] — business logic talks to interfaces; swap hardware → new adapter. @@ -69,7 +69,7 @@ Counts: 4 sources · 19 entities · 47 concepts · 8 decision records. - [[barrier-not-a-door]] — never timed-close a barrier; safety lives in barrier firmware. - [[printer-roles-failover]] — ≥2 printers by role; entry ticket falls back outside→booth. - [[printer-status-monitoring]] — live poll of paper/cover/cutter/offline via the device's status page; SSE to the booth UI. -- [[printer-usb-transport]] — ESC/POS drivers drive TCP (9100) OR local USB (/dev/usb/lp0) behind one render layer; USB = usblp char device, reachability-only status; provisioning open (oq#14). +- [[printer-usb-transport]] — ESC/POS drivers drive TCP (9100) OR local USB (/dev/usb/lp0) behind one render layer; USB = usblp char device, reachability-only status; provisioning open (oq#14); park-buzi cover-open-wedges-USB-status bug (docker restart-only fix) under investigation. - [[device-status-monitoring]] — unified live status across ALL device categories (healthCheck + printer readStatus) → the booth footer over /api/ws. - [[trust-boundary]] — the core fork: network vs. device; auditable vs. unforgeable. - [[fail-state-safety]] — entry fails closed, exit fails open; manual override; watchdog. diff --git a/wiki/log.md b/wiki/log.md index 5afec6a..20d97b5 100644 --- a/wiki/log.md +++ b/wiki/log.md @@ -2673,3 +2673,40 @@ the DS-2CD1047G3H-LIU units rather than carry an RTSP/ffmpeg workaround dependen DS-2CD1043G2-LIU (no such bug, ISAPI main-stream snapshot works natively) is the reference model going forward. RTSP main-stream capture remains documented as a proven, viable fallback if a G3H camera is ever unavoidable, but is not being built. Full sweep table + reasoning on [[lpr-camera]]. + +## [2026-08-30] update | Booth USB printer cover-open bug: leading theory is a stale container bind-mount, not a stale app-layer handle + +Live troubleshooting request (park-buzi): opening the printer's paper-roll cover reliably wedges its +status to offline/faulty, surviving a full appliance reboot; only `docker restart server` clears it. +Traced `sendRawUsb`/`probeUsb` end-to-end in `printer-escpos.ts` plus both poll loops +(`device-monitor.ts`, `printer-monitor.ts`): every print AND every poll does a fresh +open→write/probe→close with no persistent fd/socket/driver instance anywhere — ruling out a naive +"stale Node handle" explanation. Leading hypothesis instead: the cover-open microswitch cuts power +to the printer's USB interface board, causing a real bus re-enumeration; the container's directory +bind-mount of `/dev/usb` (chosen specifically to survive `lpN` renumbering) can retain a stale view +of the old device node until the container's mount namespace is recreated — which `docker restart` +does and a policy-driven reboot-time restart may not (boot-order race). Not yet confirmed on +hardware (host-vs-container `stat`/inode comparison at the next occurrence is the next step); lab +repro is blocked because the lab has a RONGTA, not the park-buzi unit's actual (still unidentified, +"Generic (unknown)") model. Full writeup, confirmation commands, and candidate fixes on +[[printer-usb-transport]]. + +## [2026-08-30] update | Backup status "Never" despite valid rotating backups — restart amnesia in BackupService, fixed + +Admin noticed park-buzi's Backup screen showed "last successful backup: Never" despite 7 real, +correctly-rotating encrypted backup files on disk, plus a 2-day gap since the last file. Traced +both symptoms to the same cause: `BackupService` tracked last-success/last-error as PLAIN +IN-PROCESS FIELDS (never written to the DB), and the daily schedule was a `setInterval(...,24h)` +measured from PROCESS START, not wall-clock time since the last real backup — so any server +restart (routine under `restart: always`: deploy/crash/OOM/host reboot) simultaneously wiped the +visible status back to "Never" and reset the 24h countdown, independent of the actual +file-writing/retention engine (`backup.ts`), which was working correctly the whole time and +explains why files existed on disk despite the UI's contradictory-seeming status. Fix: four new +nullable `site_config` columns (migration `0025_backup_last_status.sql`) persist last-success/ +error there instead of in memory; `BackupService.status()` reads them fresh each call so a new +instance (= a restart) sees the prior instance's outcome; a new `isDue()` method computes +schedule-due-ness from the persisted last-success timestamp; `server.ts`'s scheduler is now a +15-minute poll gated by `isDue()` instead of a 24h `setInterval`, making the real cadence immune +to restart timing. New test file `backup-service.test.ts` (6 tests) covers restart-durability and +`isDue()` directly; full existing suite (319 tests) still green. No API/UI contract change. Not +yet committed (holding per instruction). Full writeup on [[backup-recovery]].