docs(wiki): collector live on park-2; secrets shape, DNS vs bind, token format, every-entry sampling, CI extra rule
Claude-Session: https://claude.ai/code/session_01FWncR69HgGPuei1dLrW3cU
This commit is contained in:
@@ -45,7 +45,12 @@ The wash stream is small; the **entry camera photographs every car**, in exactly
|
|||||||
classifier is trained on, with zero domain shift. So the booth can also queue **one in N entry
|
classifier is trained on, with zero domain shift. So the booth can also queue **one in N entry
|
||||||
vehicle reads** as pure training material: the crop and the camera's class, *no* order, *no*
|
vehicle reads** as pure training material: the crop and the camera's class, *no* order, *no*
|
||||||
operator, *no* category — same crop-and-blur pipeline, same one-way path, same privacy
|
operator, *no* category — same crop-and-blur pipeline, same one-way path, same privacy
|
||||||
properties. `CARWASH_REVIEW_ENTRY_SAMPLE=N` (0/unset = off; needs the three upload settings).
|
properties. `CARWASH_REVIEW_ENTRY_SAMPLE=N` = one in N entries; **`1` = every entry, and that is
|
||||||
|
the setting park-2 runs** (user, 2026-09-07: 4 TB on the collector host, bandwidth not an issue —
|
||||||
|
the only limit was ever the reviewer's time; the reviewer labels what they have time for, the
|
||||||
|
rest waits and stays useful once a first model exists, as the unlabelled pile it is measured on).
|
||||||
|
0/unset = off; needs the three upload settings. A washed car arrives twice, as an entry sample
|
||||||
|
and as the wash decision — intended, the `kind` keeps them apart.
|
||||||
Seam: the core announces every vehicle read (`deviceEvents.emitVehicleRead`, snapshot.ts, entry
|
Seam: the core announces every vehicle read (`deviceEvents.emitVehicleRead`, snapshot.ts, entry
|
||||||
and exit) and the Car Wash module decides — it samples entry reads in-process (`sampleEntry()`,
|
and exit) and the Car Wash module decides — it samples entry reads in-process (`sampleEntry()`,
|
||||||
exactly one in N) and calls `enqueueEntry()`; the core never imports the module. Packages carry
|
exactly one in N) and calls `enqueueEntry()`; the core never imports the module. Packages carry
|
||||||
@@ -78,6 +83,24 @@ Setup → Car wash show queued / delivered / abandoned + the last error.
|
|||||||
is off and **nothing is queued** (an unbounded queue nobody drains is worse than none). Set per
|
is off and **nothing is queued** (an unbounded queue nobody drains is worse than none). Set per
|
||||||
booth in the Komodo stack env; compose forwards them.
|
booth in the Komodo stack env; compose forwards them.
|
||||||
|
|
||||||
|
- **URL by Netbird DNS name** (`http://docker-station.nb.infra:8090/ingest`): the server container
|
||||||
|
runs on the host network in prod, so it uses the booth's resolver and Netbird's DNS answers
|
||||||
|
`*.nb.infra`; a collector that moves address costs no booth change. A failed lookup behaves like
|
||||||
|
a collector outage (defer, backoff). The collector's own `COLLECTOR_BIND` must be the raw overlay
|
||||||
|
**IP** — Docker port bindings take no hostname.
|
||||||
|
- **Secrets: one per booth, two consumers.** `wash_review_token_booth_2` is referenced by the
|
||||||
|
booth's stack as its `CARWASH_REVIEW_TOKEN` *and* by the collector's stack inside
|
||||||
|
`COLLECTOR_BOOTH_TOKENS=booth-2:[[wash_review_token_booth_2]],booth-3:[[…]]` — one value, nothing
|
||||||
|
to keep in sync, rotating a booth touches one secret. (A first cut had one combined secret for
|
||||||
|
the whole list; replaced the same day — rotation was all-or-nothing and the value lived twice.)
|
||||||
|
Token format: opaque, `openssl rand -hex 32`; the collector only demands ≥ 16 chars and the list
|
||||||
|
splits on commas/whitespace, which hex never contains. Never share a token between booths — it
|
||||||
|
is what names the booth. Total Komodo secrets for one booth + the collector: two (the booth's
|
||||||
|
token, the reviewer's password).
|
||||||
|
- **The operator hash needs no variable**: `sha256(boothId + ":" + username)[:16]`, computed on
|
||||||
|
the booth from values already set; the owner recomputes it from the booth's usernames to map a
|
||||||
|
hash back, the collector never can.
|
||||||
|
|
||||||
## The collector — skeleton built 2026-09-06 (`apps/collector`)
|
## The collector — skeleton built 2026-09-06 (`apps/collector`)
|
||||||
|
|
||||||
A deliberately small Fastify + SQLite service **in this monorepo** (so it imports the payload
|
A deliberately small Fastify + SQLite service **in this monorepo** (so it imports the payload
|
||||||
@@ -114,4 +137,10 @@ Three surfaces, nothing else — it must not grow into a fleet console:
|
|||||||
never `0.0.0.0` on a host with a public interface; Netbird policy: booths → this host:8090 and
|
never `0.0.0.0` on a host with a public interface; Netbird policy: booths → this host:8090 and
|
||||||
nothing else. The host must be onboarded as a Komodo server like the booths. `TAG` is pinned
|
nothing else. The host must be onboarded as a Komodo server like the booths. `TAG` is pinned
|
||||||
and promoted with the booths (one sha for all stacks) — fine while the collector stays small;
|
and promoted with the booths (one sha for all stacks) — fine while the collector stays small;
|
||||||
its own repo the day it needs its own cadence.
|
its own repo the day it needs its own cadence. Deploy the collector BEFORE a booth that sends a
|
||||||
|
package kind it does not know (a 422 is abandoned, not retried). The export neutralises cells
|
||||||
|
that start like a spreadsheet formula (category/service names are booth-supplied text).
|
||||||
|
|
||||||
|
**Status (2026-09-07).** Live: the collector runs on `art-docker-station` and park-2 is wired to
|
||||||
|
it (`stage-dbbb051` on both stacks, every entry sampled). The review screen at
|
||||||
|
`http://docker-station.nb.infra:8090/review` is filling; no labels reviewed yet.
|
||||||
|
|||||||
@@ -87,6 +87,11 @@ The skeleton is **built and wired** (no recognizer models yet):
|
|||||||
(env `VISION_*`), `schemas.py` (the `/analyze` contract incl. a not-yet-populated `vehicle` field
|
(env `VISION_*`), `schemas.py` (the `/analyze` contract incl. a not-yet-populated `vehicle` field
|
||||||
for Job 2), `recognizer.py` (a `Recognizer` **Protocol** + `StubRecognizer` and `FastAlprRecognizer`
|
for Job 2), `recognizer.py` (a `Recognizer` **Protocol** + `StubRecognizer` and `FastAlprRecognizer`
|
||||||
— the [[device-adapter-pattern]] applied to the model).
|
— the [[device-adapter-pattern]] applied to the model).
|
||||||
|
- **CI runs WITHOUT the extra** (`uv sync --frozen` in ci.yml and build-images.yml): a test that
|
||||||
|
imports numpy/cv2 at module level breaks collection there even though it passes in a local venv
|
||||||
|
that has `alpr`. Rule (2026-09-07, after three red runs): pure post-processing tests get numpy
|
||||||
|
from the **dev group**; anything needing OpenCV uses `pytest.importorskip("cv2")`; the service
|
||||||
|
itself imports both lazily inside functions.
|
||||||
- **Light-core, heavy-optional:** core deps boot in **stub mode** (no model download) so `uv sync` +
|
- **Light-core, heavy-optional:** core deps boot in **stub mode** (no model download) so `uv sync` +
|
||||||
tests work offline; the real stack is the `alpr` extra (`uv sync --extra alpr` →
|
tests work offline; the real stack is the `alpr` extra (`uv sync --extra alpr` →
|
||||||
fast-alpr + onnxruntime). `VISION_RECOGNIZER=fast_alpr` switches it on.
|
fast-alpr + onnxruntime). `VISION_RECOGNIZER=fast_alpr` switches it on.
|
||||||
|
|||||||
+10
@@ -3143,3 +3143,13 @@ run; the Quadro FX 3800 is unusable (cc 1.3), the HD P530 irrelevant, the Xeon E
|
|||||||
(feature-extraction head in minutes, full fine-tune ~1 h); trainer image = CPU-only torch, the
|
(feature-extraction head in minutes, full fine-tune ~1 h); trainer image = CPU-only torch, the
|
||||||
compose seam drops the GPU reservation; cloud GPU rejected (crops stay on premises). Linked from
|
compose seam drops the GPU reservation; cloud GPU rejected (crops stay on premises). Linked from
|
||||||
[[opencv-anpr-service]], [[vision-review-outbox]], index. User: "No build just yet."
|
[[opencv-anpr-service]], [[vision-review-outbox]], index. User: "No build just yet."
|
||||||
|
|
||||||
|
## [2026-09-07] ingest | Collector live on park-2; secrets shape, DNS vs bind, token format, CI rule
|
||||||
|
Deployed: collector on art-docker-station + park-2 at stage-dbbb051, every entry sampled; review
|
||||||
|
screen filling. Recorded on [[vision-review-outbox]]: one secret per booth referenced by both stacks
|
||||||
|
(the combined-list secret was replaced the same day), URL by Netbird DNS name vs bind by IP, token =
|
||||||
|
`openssl rand -hex 32` (opaque, ≥16, no separators, never shared), the operator hash needs no
|
||||||
|
variable, deploy the collector before a booth that sends a new package kind, the export's formula
|
||||||
|
neutralisation (security review finding), and why every entry is sent. On
|
||||||
|
[[vision-service-packaging]]: CI syncs without the alpr extra — numpy in the dev group, cv2 tests
|
||||||
|
importorskip (three red runs on 2026-09-07).
|
||||||
|
|||||||
Reference in New Issue
Block a user