feat(collector): review collector skeleton — apps/collector, its own Komodo stack on the reviewer's host
CI / check (push) Failing after 40s
Build & push images / images (push) Failing after 32s
Build desktop / desktop (push) Successful in 5m24s

The far end of the Car Wash review outbox (wiki/concepts/vision-review-outbox.md): a small
Fastify + SQLite service in the monorepo (shares the payload contract and the class
vocabulary via @parking/shared), delivered to art-docker-station by its own stack so
nothing booth-side lands there and nothing of it on a booth.

- POST /ingest: bearer token per booth (constant-time), X-Booth-Id must match, multipart
  meta + JPEG (magic checked, 2 MB cap), meta validated against the contract, idempotent on
  the item id; crop stored at crops/<booth>/<item>.jpg on the volume + one items row.
- /review + /api/*: the reviewer's screen served by the process (Basic auth, one login):
  one pending crop at a time, operator's pick and camera's pick beside it, one button/key
  per vocabulary class + unusable + skip; stats per booth and per hashed operator
  (agree / disagree / unusable — disagree = the reviewer's class is outside the operator's
  category).
- GET /export/labels.csv: reviewed usable rows for training; formula-leading cells are
  neutralised (booth-supplied names). Crops stay on the volume for the trainer on the host.
- Booth payload now carries operatorCategory.classes so the comparison needs no site setup.
- Delivery: apps/collector/Dockerfile (monorepo context), docker-compose.collector.yml
  (bind to the overlay IP; commented `trainer` profile seam for the GPU), a third build
  step in build-images.yml, a `wash-collector` stack in komodo/resources.toml with one
  secret per booth referenced from both the collector's token list and the booth's own
  stack (park-2 lines templated, commented, DNS name for the URL).
- Tests: app.test.ts (ingest ok/dup/refusals, review + stats + export, config). Image
  built and smoke-tested locally (health, ingest, duplicate, auth, verdict, export).

Claude-Session: https://claude.ai/code/session_01FWncR69HgGPuei1dLrW3cU
This commit is contained in:
2026-09-07 08:26:22 +02:00
parent ec44547122
commit b485e9870b
22 changed files with 1045 additions and 10 deletions
+16
View File
@@ -0,0 +1,16 @@
# Car Wash review collector (wiki/concepts/vision-review-outbox.md). Runs on the
# reviewer's host (art-docker-station), reachable by the booths ONLY over the Netbird
# overlay. Deployed by its own Komodo stack (komodo/resources.toml, "wash-collector").
# COLLECTOR_HOST=0.0.0.0 # in Docker the compose file binds the published port to the overlay IP
# COLLECTOR_PORT=8090
# COLLECTOR_DATA_DIR=/data # collector.sqlite + crops/<booth>/<item>.jpg
# One bearer token per booth: "<boothId>:<token>" pairs, comma- or newline-separated. The
# booth id is the pseudonymous CARWASH_REVIEW_BOOTH_ID that booth was deployed with — never
# a site name. Generate tokens with: openssl rand -hex 32
COLLECTOR_BOOTH_TOKENS=booth-7:REPLACE,booth-9:REPLACE
# The reviewer's login for the review screen and the export (HTTP Basic over the overlay).
COLLECTOR_REVIEWER_USER=reviewer
COLLECTOR_REVIEWER_PASS=REPLACE
+48
View File
@@ -0,0 +1,48 @@
# parking-collector — the Car Wash review collector (wiki/concepts/vision-review-outbox.md).
# Built from the monorepo root (context: .) like the server image, so it shares the
# lockfile and @parking/shared. Runs on the REVIEWER's host (not a booth), delivered by
# its own Komodo stack (docker-compose.collector.yml). Data on /data: collector.sqlite +
# crops/<booth>/<item>.jpg — the trainer on the same host reads the crops off that volume.
FROM node:22-alpine AS deps
WORKDIR /app
RUN apk add --no-cache python3 make g++ # node-gyp for better-sqlite3
RUN corepack enable && corepack prepare pnpm@10.24.0 --activate
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml turbo.json ./
COPY apps/server/package.json apps/server/
COPY apps/web/package.json apps/web/
COPY apps/vision/package.json apps/vision/
COPY apps/collector/package.json apps/collector/
COPY packages/db/package.json packages/db/
COPY packages/devices/package.json packages/devices/
COPY packages/shared/package.json packages/shared/
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
pnpm fetch
FROM deps AS build
ENV CI=true
COPY . .
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
pnpm install --frozen-lockfile --offline
RUN pnpm turbo run build --filter=@parking/collector
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
pnpm --filter=@parking/collector --legacy deploy --prod /deploy
FROM node:22-alpine AS runtime
WORKDIR /app
ARG BUILD_VERSION=""
ENV BUILD_VERSION=$BUILD_VERSION
ENV NODE_ENV=production
RUN apk add --no-cache libstdc++ wget # better-sqlite3 native runtime; wget for the healthcheck
RUN addgroup -S app && adduser -S -G app app
COPY --from=build --chown=app:app /deploy ./
ENV COLLECTOR_DATA_DIR=/data
ENV COLLECTOR_HOST=0.0.0.0
ENV COLLECTOR_PORT=8090
RUN mkdir -p /data && chown app:app /data
VOLUME ["/data"]
USER app
EXPOSE 8090
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD wget -qO- "http://localhost:${COLLECTOR_PORT:-8090}/health" >/dev/null 2>&1 || exit 1
CMD ["node", "dist/index.js"]
+28
View File
@@ -0,0 +1,28 @@
{
"name": "@parking/collector",
"version": "0.0.0",
"private": true,
"type": "module",
"description": "Car Wash review collector: receives plate-blurred vehicle crops + the operator's category choice from booths over the private overlay, serves the reviewer's screen, exports labels for training. See wiki/concepts/vision-review-outbox.md.",
"scripts": {
"build": "tsc -b",
"dev": "tsx watch --env-file-if-exists=.env src/index.ts",
"start": "node --env-file-if-exists=.env dist/index.js",
"typecheck": "tsc --noEmit",
"lint": "tsc --noEmit",
"test": "vitest run"
},
"dependencies": {
"@fastify/multipart": "^9.2.1",
"@parking/shared": "workspace:*",
"better-sqlite3": "12.10.1",
"fastify": "5.8.5"
},
"devDependencies": {
"@types/better-sqlite3": "7.6.13",
"@types/node": "25.9.3",
"tsx": "4.22.4",
"typescript": "6.0.3",
"vitest": "^4.1.9"
}
}
+141
View File
@@ -0,0 +1,141 @@
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { buildCollector, type CollectorApp } from "./app.js";
import { parseBoothTokens } from "./config.js";
// The collector: one ingest surface (bearer per booth, idempotent), one review surface
// (Basic), one export. Exercised over app.inject with a hand-built multipart body.
let app: CollectorApp;
let dir: string;
const TOKENS = new Map([["booth-7", "0123456789abcdef0123456789abcdef"], ["booth-9", "fedcba9876543210fedcba9876543210"]]);
const REVIEWER = { user: "julian", pass: "review-pass-123" };
const basic = "Basic " + Buffer.from(`${REVIEWER.user}:${REVIEWER.pass}`).toString("base64");
beforeEach(async () => {
dir = await mkdtemp(path.join(tmpdir(), "collector-"));
app = await buildCollector({ host: "127.0.0.1", port: 0, dataDir: dir, boothTokens: TOKENS, reviewer: REVIEWER }, { dbFile: ":memory:" });
await app.ready();
});
afterEach(async () => {
await app.close();
await rm(dir, { recursive: true, force: true });
});
/** A minimal JPEG-looking blob (SOI marker + padding) — the collector checks the magic only. */
const JPEG = Buffer.concat([Buffer.from([0xff, 0xd8, 0xff, 0xe0]), Buffer.alloc(200, 1)]);
function meta(over: Record<string, unknown> = {}) {
return {
v: 1, booth: "booth-7", item: "item-1", order: "o-1", at: "2026-09-06T10:00:00.000Z", operator: "ab12cd34ef56ab12",
operatorCategory: { id: "car", name: "Vetura", classes: ["car", "sedan", "hatchback"] }, service: "Standard",
vision: { class: "suv", confidence: 0.91, categoryId: "suv" }, downgraded: true,
image: { width: 320, height: 200, plateBlurred: true },
...over,
};
}
function multipart(fields: Record<string, string>, file: Buffer | null): { body: Buffer; type: string } {
const b = "----collector-test";
const parts: Buffer[] = [];
for (const [k, v] of Object.entries(fields)) parts.push(Buffer.from(`--${b}\r\nContent-Disposition: form-data; name="${k}"\r\n\r\n${v}\r\n`));
if (file) parts.push(Buffer.from(`--${b}\r\nContent-Disposition: form-data; name="image"; filename="x.jpg"\r\nContent-Type: image/jpeg\r\n\r\n`), file, Buffer.from("\r\n"));
parts.push(Buffer.from(`--${b}--\r\n`));
return { body: Buffer.concat(parts), type: `multipart/form-data; boundary=${b}` };
}
async function ingest(m: Record<string, unknown>, token = TOKENS.get("booth-7")!, file: Buffer | null = JPEG, extra: Record<string, string> = {}) {
const { body, type } = multipart({ meta: JSON.stringify(m) }, file);
return app.inject({ method: "POST", url: "/ingest", headers: { authorization: `Bearer ${token}`, "content-type": type, ...extra }, payload: body });
}
describe("ingest", () => {
it("stores the crop and the decision under the token's booth; retries are idempotent", async () => {
const r = await ingest(meta());
expect(r.statusCode).toBe(201);
const row = app.collectorDb.get("item-1")!;
expect(row).toMatchObject({ booth: "booth-7", operatorCategoryName: "Vetura", visionClass: "suv", downgraded: 1, plateBlurred: 1, imagePath: "crops/booth-7/item-1.jpg" });
expect(JSON.parse(row.operatorClasses)).toEqual(["car", "sedan", "hatchback"]);
const again = await ingest(meta());
expect(again.statusCode).toBe(200);
expect(again.json()).toEqual({ ok: true, duplicate: true });
expect((await app.inject({ method: "GET", url: "/health" })).json()).toMatchObject({ ok: true, booths: 1, pending: 1 });
});
it("refuses a bad token, a booth mismatch, a non-JPEG, and malformed meta", async () => {
expect((await ingest(meta(), "nope-nope-nope-nope-nope")).statusCode).toBe(401);
expect((await ingest(meta({ booth: "booth-9" }))).statusCode).toBe(422); // token is booth-7's
expect((await ingest(meta(), TOKENS.get("booth-7")!, JPEG, { "x-booth-id": "booth-9" })).statusCode).toBe(403);
expect((await ingest(meta(), TOKENS.get("booth-7")!, Buffer.alloc(300, 7))).statusCode).toBe(415);
expect((await ingest(meta(), TOKENS.get("booth-7")!, null)).statusCode).toBe(400);
expect((await ingest(meta({ vision: { class: "spaceship", confidence: 0.5, categoryId: null } }))).statusCode).toBe(422);
expect((await ingest(meta({ item: "../../etc/passwd" }))).statusCode).toBe(422);
expect((await ingest(meta({ v: 2 }))).statusCode).toBe(422);
expect(app.collectorDb.stats().booths).toEqual([]);
});
});
describe("review + export", () => {
it("the reviewer lists pending items, sees the crop, labels it; stats compare the label with the operator's category; the export lists usable labels only", async () => {
await ingest(meta());
await ingest(meta({ item: "item-2", operator: "ab12cd34ef56ab12", vision: { class: "car", confidence: 0.8, categoryId: "car" }, downgraded: false }));
await ingest(meta({ item: "item-3", booth: "booth-9", operator: "9999999999999999" }), TOKENS.get("booth-9")!);
// No login → 401 with a challenge; nothing without a configured reviewer is tested in config.
const anon = await app.inject({ method: "GET", url: "/api/items" });
expect(anon.statusCode).toBe(401);
expect(anon.headers["www-authenticate"]).toContain("Basic");
expect((await app.inject({ method: "GET", url: "/review", headers: { authorization: basic } })).headers["content-type"]).toContain("text/html");
const list = (await app.inject({ method: "GET", url: "/api/items?status=pending", headers: { authorization: basic } })).json();
expect(list.items.map((i: { id: string }) => i.id)).toEqual(["item-1", "item-2", "item-3"]);
expect(list.items[0].imagePath).toBeUndefined();
const img = await app.inject({ method: "GET", url: "/api/items/item-1/image", headers: { authorization: basic } });
expect(img.statusCode).toBe(200);
expect(img.headers["content-type"]).toBe("image/jpeg");
expect(img.rawPayload.subarray(0, 3)).toEqual(Buffer.from([0xff, 0xd8, 0xff]));
// item-1: operator said Vetura (car/sedan/hatchback), reviewer says suv → disagree.
// item-2: reviewer says sedan → inside Vetura → agree. item-3: unusable.
const post = (id: string, label: string) =>
app.inject({ method: "POST", url: `/api/items/${id}/review`, headers: { authorization: basic, "content-type": "application/json" }, payload: { label } });
expect((await post("item-1", "suv")).json()).toMatchObject({ reviewLabel: "suv", reviewer: "julian" });
expect((await post("item-2", "sedan")).statusCode).toBe(200);
expect((await post("item-3", "unusable")).statusCode).toBe(200);
expect((await post("item-3", "spaceship")).statusCode).toBe(400);
expect((await post("nope", "suv")).statusCode).toBe(404);
const stats = (await app.inject({ method: "GET", url: "/api/stats", headers: { authorization: basic } })).json();
expect(stats.booths).toEqual([
{ booth: "booth-7", received: 2, pending: 0, reviewed: 2 },
{ booth: "booth-9", received: 1, pending: 0, reviewed: 1 },
]);
expect(stats.operators).toEqual([
{ booth: "booth-7", operatorRef: "ab12cd34ef56ab12", reviewed: 2, agree: 1, disagree: 1, unusable: 0 },
{ booth: "booth-9", operatorRef: "9999999999999999", reviewed: 1, agree: 0, disagree: 0, unusable: 1 },
]);
const csv = await app.inject({ method: "GET", url: "/export/labels.csv", headers: { authorization: basic } });
expect(csv.statusCode).toBe(200);
const lines = csv.body.trim().split("\n");
expect(lines[0]).toBe("item,booth,path,label,operator_category,operator_classes,vision_class,vision_confidence,downgraded,at,reviewed_at");
expect(lines).toHaveLength(3); // header + 2 usable labels; the unusable one is left out
expect(lines[1]).toContain('"item-1","booth-7","crops/booth-7/item-1.jpg","suv","Vetura","car|sedan|hatchback","suv"');
// A booth-supplied name that looks like a spreadsheet formula is neutralised in the export.
await ingest(meta({ item: "item-4", operatorCategory: { id: "x", name: "=HYPERLINK(\"http://evil\")", classes: ["car"] } }));
await post("item-4", "car");
const csv2 = (await app.inject({ method: "GET", url: "/export/labels.csv", headers: { authorization: basic } })).body;
expect(csv2).toContain(`"'=HYPERLINK(""http://evil"")"`);
});
});
describe("config", () => {
it("parses booth:token pairs and refuses short tokens", () => {
expect([...parseBoothTokens("a:0123456789abcdef, b:fedcba9876543210\nc:0000000000000000").keys()]).toEqual(["a", "b", "c"]);
expect(() => parseBoothTokens("a:short")).toThrow(/too short/);
expect(() => parseBoothTokens("nocolon")).toThrow(/bad pair/);
});
});
+231
View File
@@ -0,0 +1,231 @@
import { timingSafeEqual } from "node:crypto";
import { createReadStream } from "node:fs";
import { mkdir, writeFile } from "node:fs/promises";
import path from "node:path";
import Fastify, { type FastifyInstance, type FastifyReply, type FastifyRequest } from "fastify";
import multipart from "@fastify/multipart";
import { isVehicleClass } from "@parking/shared";
import type { CollectorConfig } from "./config.js";
import { CollectorDb, type ItemRow, type ReviewVerdict } from "./db.js";
import { reviewPage } from "./review-page.js";
// The collector — the far end of the booth's review outbox
// (wiki/concepts/vision-review-outbox.md). Three surfaces and nothing else:
// POST /ingest one package from one booth (bearer token per booth; idempotent)
// /review + /api/* the reviewer's screen (HTTP Basic, one login)
// GET /export/labels.csv the training set: reviewed, usable rows (crops sit beside it on
// the volume, so the trainer on this host reads them directly)
// It deliberately has no fleet features and no path back into a booth.
/** The package's `meta` part, as the booth sends it (review-outbox.ts). */
interface IngestMeta {
v: number;
booth: string;
item: string;
order: string;
at: string;
operator: string;
operatorCategory: { id: string; name: string; classes?: string[] };
service: string;
vision: { class: string; confidence: number; categoryId: string | null };
downgraded: boolean;
image: { width: number; height: number; plateBlurred: boolean };
}
const ID_RE = /^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$/;
const MAX_IMAGE_BYTES = 2 * 1024 * 1024;
function str(v: unknown, max = 200): string | null {
return typeof v === "string" && v.length > 0 && v.length <= max ? v : null;
}
/** Validate the meta part; returns a message on the first problem. */
function checkMeta(m: unknown, booth: string): { ok: true; meta: IngestMeta } | { ok: false; why: string } {
if (!m || typeof m !== "object") return { ok: false, why: "meta must be an object" };
const x = m as Record<string, unknown>;
if (x.v !== 1) return { ok: false, why: "unsupported meta version" };
if (x.booth !== booth) return { ok: false, why: "meta.booth does not match the token's booth" };
if (!str(x.item, 64) || !ID_RE.test(x.item as string)) return { ok: false, why: "bad item id" };
if (!str(x.order, 64)) return { ok: false, why: "bad order ref" };
if (!str(x.at, 40) || Number.isNaN(Date.parse(x.at as string))) return { ok: false, why: "bad timestamp" };
if (!str(x.operator, 64)) return { ok: false, why: "bad operator ref" };
const oc = x.operatorCategory as Record<string, unknown> | undefined;
if (!oc || !str(oc.id, 64) || !str(oc.name, 120)) return { ok: false, why: "bad operatorCategory" };
if (oc.classes !== undefined && (!Array.isArray(oc.classes) || !oc.classes.every(isVehicleClass))) return { ok: false, why: "bad operatorCategory.classes" };
if (!str(x.service, 120)) return { ok: false, why: "bad service" };
const v = x.vision as Record<string, unknown> | undefined;
if (!v || !isVehicleClass(v.class) || typeof v.confidence !== "number" || v.confidence < 0 || v.confidence > 1) return { ok: false, why: "bad vision read" };
if (v.categoryId != null && !str(v.categoryId, 64)) return { ok: false, why: "bad vision.categoryId" };
if (typeof x.downgraded !== "boolean") return { ok: false, why: "bad downgraded" };
const im = x.image as Record<string, unknown> | undefined;
if (!im || typeof im.width !== "number" || typeof im.height !== "number" || typeof im.plateBlurred !== "boolean") return { ok: false, why: "bad image meta" };
return { ok: true, meta: x as unknown as IngestMeta };
}
function safeEqual(a: string, b: string): boolean {
const ba = Buffer.from(a);
const bb = Buffer.from(b);
return ba.length === bb.length && timingSafeEqual(ba, bb);
}
export interface CollectorApp extends FastifyInstance {
collectorDb: CollectorDb;
}
export async function buildCollector(cfg: CollectorConfig, opts: { dbFile?: string } = {}): Promise<CollectorApp> {
await mkdir(path.join(cfg.dataDir, "crops"), { recursive: true });
const db = new CollectorDb(opts.dbFile ?? path.join(cfg.dataDir, "collector.sqlite"));
const app = Fastify({ logger: { level: process.env.LOG_LEVEL ?? "info" }, bodyLimit: 64 * 1024 }) as unknown as CollectorApp;
app.collectorDb = db;
await app.register(multipart, { limits: { fileSize: MAX_IMAGE_BYTES, files: 1, fields: 4, parts: 6 } });
app.addHook("onClose", async () => db.close());
/** Which booth this bearer token belongs to, or null. Constant-time per candidate. */
function boothForToken(req: FastifyRequest): string | null {
const h = req.headers.authorization ?? "";
if (!h.startsWith("Bearer ")) return null;
const token = h.slice(7).trim();
let found: string | null = null;
for (const [booth, t] of cfg.boothTokens) if (safeEqual(token, t)) found = booth;
return found;
}
/** HTTP Basic for the reviewer. */
async function requireReviewer(req: FastifyRequest, reply: FastifyReply): Promise<void> {
if (!cfg.reviewer) return reply.code(503).send({ error: "reviewer login not configured" });
const h = req.headers.authorization ?? "";
if (h.startsWith("Basic ")) {
const [user, ...rest] = Buffer.from(h.slice(6), "base64").toString("utf8").split(":");
const pass = rest.join(":");
if (user && safeEqual(user, cfg.reviewer.user) && safeEqual(pass, cfg.reviewer.pass)) return;
}
return reply.code(401).header("www-authenticate", 'Basic realm="wash review", charset="UTF-8"').send({ error: "unauthorized" });
}
app.get("/health", async () => {
const s = db.stats();
return { ok: true, booths: s.booths.length, pending: s.booths.reduce((n, b) => n + b.pending, 0) };
});
// --- Ingest (booths) -----------------------------------------------------------------
app.post("/ingest", async (req, reply) => {
const booth = boothForToken(req);
if (!booth) return reply.code(401).send({ error: "unauthorized" });
const claimed = req.headers["x-booth-id"];
if (typeof claimed === "string" && claimed !== booth) return reply.code(403).send({ error: "booth id does not match the token" });
if (!req.isMultipart()) return reply.code(415).send({ error: "multipart/form-data expected" });
let metaRaw: string | null = null;
let image: Buffer | null = null;
try {
for await (const part of req.parts()) {
if (part.type === "file" && part.fieldname === "image") {
image = await part.toBuffer();
} else if (part.type === "field" && part.fieldname === "meta") {
metaRaw = String(part.value);
}
}
} catch (err) {
const code = (err as { code?: string }).code;
return reply.code(code === "FST_REQ_FILE_TOO_LARGE" ? 413 : 400).send({ error: (err as Error).message });
}
if (!metaRaw) return reply.code(400).send({ error: "meta part missing" });
if (!image || image.length < 100) return reply.code(400).send({ error: "image part missing" });
if (!(image[0] === 0xff && image[1] === 0xd8 && image[2] === 0xff)) return reply.code(415).send({ error: "image must be a JPEG" });
let parsed: unknown;
try {
parsed = JSON.parse(metaRaw);
} catch {
return reply.code(400).send({ error: "meta is not JSON" });
}
const checked = checkMeta(parsed, booth);
if (!checked.ok) return reply.code(422).send({ error: checked.why });
const meta = checked.meta;
// Idempotent on the item id: a booth retrying after a lost 2xx must not duplicate.
if (db.get(meta.item)) return reply.code(200).send({ ok: true, duplicate: true });
const rel = path.posix.join("crops", booth, `${meta.item}.jpg`);
await mkdir(path.join(cfg.dataDir, "crops", booth), { recursive: true });
await writeFile(path.join(cfg.dataDir, rel), image);
db.insert({
id: meta.item,
booth,
orderRef: meta.order,
at: meta.at,
operatorRef: meta.operator,
operatorCategoryId: meta.operatorCategory.id,
operatorCategoryName: meta.operatorCategory.name,
operatorClasses: JSON.stringify(meta.operatorCategory.classes ?? []),
service: meta.service,
visionClass: meta.vision.class,
visionConfidence: meta.vision.confidence,
visionCategoryId: meta.vision.categoryId ?? null,
downgraded: meta.downgraded ? 1 : 0,
imageWidth: meta.image.width,
imageHeight: meta.image.height,
plateBlurred: meta.image.plateBlurred ? 1 : 0,
imagePath: rel,
receivedAt: new Date().toISOString(),
});
req.log.info(`ingest: ${booth} item ${meta.item} (${meta.vision.class} → ${meta.operatorCategory.name})`);
return reply.code(201).send({ ok: true });
});
// --- Review (the trusted person) -----------------------------------------------------
const page = reviewPage();
app.get("/", { preHandler: requireReviewer }, async (_req, reply) => reply.redirect("/review"));
app.get("/review", { preHandler: requireReviewer }, async (_req, reply) => reply.type("text/html; charset=utf-8").send(page));
app.get<{ Querystring: { status?: string; limit?: string; booth?: string } }>(
"/api/items",
{ preHandler: requireReviewer },
async (req) => {
const status = req.query.status === "reviewed" ? "reviewed" : "pending";
const limit = Math.min(Math.max(Number(req.query.limit) || 25, 1), 200);
return { items: db.list(status, limit, req.query.booth || undefined).map(publicItem) };
},
);
app.get<{ Params: { id: string } }>("/api/items/:id/image", { preHandler: requireReviewer }, async (req, reply) => {
const row = db.get(req.params.id);
if (!row) return reply.code(404).send({ error: "not found" });
return reply.type("image/jpeg").header("cache-control", "private, max-age=3600").send(createReadStream(path.join(cfg.dataDir, row.imagePath)));
});
app.post<{ Params: { id: string }; Body: { label?: unknown } }>("/api/items/:id/review", { preHandler: requireReviewer }, async (req, reply) => {
const label = req.body?.label;
if (label !== "unusable" && !isVehicleClass(label)) return reply.code(400).send({ error: "label must be a vehicle class or 'unusable'" });
if (!db.get(req.params.id)) return reply.code(404).send({ error: "not found" });
const row = db.review(req.params.id, label as ReviewVerdict, cfg.reviewer!.user);
return publicItem(row!);
});
app.get("/api/stats", { preHandler: requireReviewer }, async () => db.stats());
// --- Export (the training set) --------------------------------------------------------
app.get("/export/labels.csv", { preHandler: requireReviewer }, async (_req, reply) => {
const rows = db.labelled();
// Quote every cell; a cell starting like a spreadsheet formula (=, +, -, @, tab, CR)
// gets a leading apostrophe — the category/service names are booth-supplied text and
// the reviewer will open this in a spreadsheet (CSV formula injection).
const q = (s: string | number | null) => {
let v = String(s ?? "");
if (/^[=+\-@\t\r]/.test(v)) v = `'${v}`;
return `"${v.replace(/"/g, '""')}"`;
};
const head = "item,booth,path,label,operator_category,operator_classes,vision_class,vision_confidence,downgraded,at,reviewed_at";
const lines = rows.map((r) =>
[r.id, r.booth, r.imagePath, r.reviewLabel, r.operatorCategoryName, JSON.parse(r.operatorClasses).join("|"), r.visionClass, r.visionConfidence, r.downgraded, r.at, r.reviewedAt].map(q).join(","),
);
return reply.type("text/csv; charset=utf-8").header("content-disposition", 'attachment; filename="labels.csv"').send([head, ...lines].join("\n") + "\n");
});
return app;
}
/** The row as the review screen sees it (no server paths). */
function publicItem(r: ItemRow): Omit<ItemRow, "imagePath"> {
const { imagePath: _p, ...rest } = r;
return rest;
}
+36
View File
@@ -0,0 +1,36 @@
export interface CollectorConfig {
readonly host: string;
readonly port: number;
readonly dataDir: string;
/** boothId → bearer token. */
readonly boothTokens: ReadonlyMap<string, string>;
/** The single reviewer login; null = review screen and export refuse (503). */
readonly reviewer: { readonly user: string; readonly pass: string } | null;
}
/** "booth-7:abc,booth-9:def" (commas, whitespace or newlines between pairs). */
export function parseBoothTokens(raw: string): Map<string, string> {
const out = new Map<string, string>();
for (const pair of raw.split(/[,\s]+/)) {
if (!pair) continue;
const i = pair.indexOf(":");
if (i <= 0) throw new Error(`COLLECTOR_BOOTH_TOKENS: bad pair "${pair}" (want boothId:token)`);
const booth = pair.slice(0, i).trim();
const token = pair.slice(i + 1).trim();
if (!booth || token.length < 16) throw new Error(`COLLECTOR_BOOTH_TOKENS: token for "${booth}" too short (>=16 chars)`);
out.set(booth, token);
}
return out;
}
export function configFromEnv(env: NodeJS.ProcessEnv = process.env): CollectorConfig {
const user = (env.COLLECTOR_REVIEWER_USER ?? "").trim();
const pass = env.COLLECTOR_REVIEWER_PASS ?? "";
return {
host: env.COLLECTOR_HOST ?? "0.0.0.0",
port: Number(env.COLLECTOR_PORT ?? 8090),
dataDir: env.COLLECTOR_DATA_DIR ?? "/data",
boothTokens: parseBoothTokens(env.COLLECTOR_BOOTH_TOKENS ?? ""),
reviewer: user && pass.length >= 8 ? { user, pass } : null,
};
}
+179
View File
@@ -0,0 +1,179 @@
import Database from "better-sqlite3";
import type { VehicleClass } from "@parking/shared";
// One table. Each row is one booth decision: what the camera saw, what the operator
// chose, and (once reviewed) what a trusted person says the vehicle is. The crop itself
// lives on disk beside the DB (crops/<booth>/<item>.jpg) so the trainer on the same host
// reads it straight off the volume.
export interface ItemRow {
id: string;
booth: string;
orderRef: string;
at: string;
operatorRef: string;
operatorCategoryId: string;
operatorCategoryName: string;
/** The vision classes the operator's category covers at that site (its mapping) — what
* lets a reviewer's CLASS be compared with an operator's CATEGORY. JSON array. */
operatorClasses: string;
service: string;
visionClass: string;
visionConfidence: number;
visionCategoryId: string | null;
downgraded: number;
imageWidth: number;
imageHeight: number;
plateBlurred: number;
imagePath: string;
receivedAt: string;
reviewLabel: string | null; // a VehicleClass, or "unusable"
reviewedAt: string | null;
reviewer: string | null;
}
export type ReviewVerdict = VehicleClass | "unusable";
export class CollectorDb {
readonly #db: Database.Database;
constructor(file: string) {
this.#db = new Database(file);
this.#db.pragma("journal_mode = WAL");
this.#db.exec(`
CREATE TABLE IF NOT EXISTS items (
id TEXT PRIMARY KEY,
booth TEXT NOT NULL,
order_ref TEXT NOT NULL,
at TEXT NOT NULL,
operator_ref TEXT NOT NULL,
operator_category_id TEXT NOT NULL,
operator_category_name TEXT NOT NULL,
operator_classes TEXT NOT NULL DEFAULT '[]',
service TEXT NOT NULL,
vision_class TEXT NOT NULL,
vision_confidence REAL NOT NULL,
vision_category_id TEXT,
downgraded INTEGER NOT NULL DEFAULT 0,
image_width INTEGER NOT NULL,
image_height INTEGER NOT NULL,
plate_blurred INTEGER NOT NULL,
image_path TEXT NOT NULL,
received_at TEXT NOT NULL,
review_label TEXT,
reviewed_at TEXT,
reviewer TEXT
);
CREATE INDEX IF NOT EXISTS items_pending ON items (reviewed_at, received_at);
CREATE INDEX IF NOT EXISTS items_booth ON items (booth, received_at);
`);
}
close(): void {
this.#db.close();
}
static #map(r: Record<string, unknown>): ItemRow {
return {
id: r.id as string,
booth: r.booth as string,
orderRef: r.order_ref as string,
at: r.at as string,
operatorRef: r.operator_ref as string,
operatorCategoryId: r.operator_category_id as string,
operatorCategoryName: r.operator_category_name as string,
operatorClasses: r.operator_classes as string,
service: r.service as string,
visionClass: r.vision_class as string,
visionConfidence: r.vision_confidence as number,
visionCategoryId: (r.vision_category_id as string | null) ?? null,
downgraded: r.downgraded as number,
imageWidth: r.image_width as number,
imageHeight: r.image_height as number,
plateBlurred: r.plate_blurred as number,
imagePath: r.image_path as string,
receivedAt: r.received_at as string,
reviewLabel: (r.review_label as string | null) ?? null,
reviewedAt: (r.reviewed_at as string | null) ?? null,
reviewer: (r.reviewer as string | null) ?? null,
};
}
get(id: string): ItemRow | null {
const r = this.#db.prepare("SELECT * FROM items WHERE id = ?").get(id) as Record<string, unknown> | undefined;
return r ? CollectorDb.#map(r) : null;
}
insert(row: Omit<ItemRow, "reviewLabel" | "reviewedAt" | "reviewer">): void {
this.#db
.prepare(
`INSERT INTO items (id, booth, order_ref, at, operator_ref, operator_category_id, operator_category_name,
operator_classes, service, vision_class, vision_confidence, vision_category_id, downgraded,
image_width, image_height, plate_blurred, image_path, received_at)
VALUES (@id, @booth, @orderRef, @at, @operatorRef, @operatorCategoryId, @operatorCategoryName,
@operatorClasses, @service, @visionClass, @visionConfidence, @visionCategoryId, @downgraded,
@imageWidth, @imageHeight, @plateBlurred, @imagePath, @receivedAt)`,
)
.run(row);
}
list(status: "pending" | "reviewed", limit: number, booth?: string): ItemRow[] {
const where = [status === "pending" ? "reviewed_at IS NULL" : "reviewed_at IS NOT NULL"];
const params: unknown[] = [];
if (booth) {
where.push("booth = ?");
params.push(booth);
}
const order = status === "pending" ? "received_at ASC" : "reviewed_at DESC";
const rows = this.#db
.prepare(`SELECT * FROM items WHERE ${where.join(" AND ")} ORDER BY ${order} LIMIT ?`)
.all(...params, limit) as Record<string, unknown>[];
return rows.map((r) => CollectorDb.#map(r));
}
review(id: string, label: ReviewVerdict, reviewer: string): ItemRow | null {
this.#db
.prepare("UPDATE items SET review_label = ?, reviewed_at = ?, reviewer = ? WHERE id = ?")
.run(label, new Date().toISOString(), reviewer, id);
return this.get(id);
}
/** Per booth: received / pending / reviewed. Per operator (booth + hash): how often the
* reviewer's class fell inside the operator's chosen category (agree) or outside
* (disagree) — the honest-mistake / fraud rate the outbox exists for. */
stats(): {
booths: { booth: string; received: number; pending: number; reviewed: number }[];
operators: { booth: string; operatorRef: string; reviewed: number; agree: number; disagree: number; unusable: number }[];
} {
const booths = this.#db
.prepare(
`SELECT booth, COUNT(*) AS received,
SUM(CASE WHEN reviewed_at IS NULL THEN 1 ELSE 0 END) AS pending,
SUM(CASE WHEN reviewed_at IS NOT NULL THEN 1 ELSE 0 END) AS reviewed
FROM items GROUP BY booth ORDER BY booth`,
)
.all() as { booth: string; received: number; pending: number; reviewed: number }[];
const reviewed = this.#db
.prepare("SELECT booth, operator_ref, operator_classes, review_label FROM items WHERE reviewed_at IS NOT NULL")
.all() as { booth: string; operator_ref: string; operator_classes: string; review_label: string }[];
const ops = new Map<string, { booth: string; operatorRef: string; reviewed: number; agree: number; disagree: number; unusable: number }>();
for (const r of reviewed) {
const key = `${r.booth} ${r.operator_ref}`;
let o = ops.get(key);
if (!o) ops.set(key, (o = { booth: r.booth, operatorRef: r.operator_ref, reviewed: 0, agree: 0, disagree: 0, unusable: 0 }));
o.reviewed += 1;
if (r.review_label === "unusable") o.unusable += 1;
else if ((JSON.parse(r.operator_classes) as string[]).includes(r.review_label)) o.agree += 1;
else o.disagree += 1;
}
return { booths, operators: [...ops.values()].sort((a, b) => b.disagree - a.disagree) };
}
/** Reviewed, usable rows — the training set. */
labelled(): ItemRow[] {
const rows = this.#db
.prepare("SELECT * FROM items WHERE reviewed_at IS NOT NULL AND review_label != 'unusable' ORDER BY reviewed_at")
.all() as Record<string, unknown>[];
return rows.map((r) => CollectorDb.#map(r));
}
}
+16
View File
@@ -0,0 +1,16 @@
import { buildCollector } from "./app.js";
import { configFromEnv } from "./config.js";
const cfg = configFromEnv();
const app = await buildCollector(cfg);
if (cfg.boothTokens.size === 0) app.log.warn("COLLECTOR_BOOTH_TOKENS is empty — no booth can ingest");
if (!cfg.reviewer) app.log.warn("COLLECTOR_REVIEWER_USER/PASS not set — the review screen and export refuse");
app.log.info(`collector: ${cfg.boothTokens.size} booth token(s), data in ${cfg.dataDir}`);
await app.listen({ host: cfg.host, port: cfg.port });
const stop = async () => {
await app.close();
process.exit(0);
};
process.on("SIGTERM", () => void stop());
process.on("SIGINT", () => void stop());
+117
View File
@@ -0,0 +1,117 @@
import { VEHICLE_CLASSES } from "@parking/shared";
// The reviewer's screen: one pending crop at a time, the operator's pick and the camera's
// pick beside it, one button per vocabulary class + "unusable". Served by the collector
// itself (no build step, no framework) — this is deliberately the whole UI.
export function reviewPage(): string {
const classes = JSON.stringify(VEHICLE_CLASSES);
return `<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Wash review</title>
<style>
:root { --bg:#111; --panel:#1b1b1b; --text:#e8e8e8; --muted:#9a9a9a; --amber:#e0a030; --green:#4caf50; --red:#e05050; }
body { margin:0; background:var(--bg); color:var(--text); font:14px/1.4 system-ui, sans-serif; }
header { display:flex; justify-content:space-between; align-items:center; padding:.6rem 1rem; border-bottom:1px solid #333; }
header b { letter-spacing:.08em; text-transform:uppercase; color:var(--amber); font-size:.75rem; }
main { max-width:960px; margin:0 auto; padding:1rem; display:grid; gap:1rem; }
.card { background:var(--panel); border:1px solid #333; border-radius:6px; padding:1rem; }
img { max-width:100%; max-height:60vh; display:block; margin:0 auto; background:#000; border-radius:4px; }
dl { display:grid; grid-template-columns:max-content 1fr; gap:.2rem .8rem; margin:0; font-variant-numeric:tabular-nums; }
dt { color:var(--muted); }
.buttons { display:flex; flex-wrap:wrap; gap:.4rem; }
button { background:#2a2a2a; color:var(--text); border:1px solid #444; border-radius:4px; padding:.5rem .8rem; font:inherit; cursor:pointer; }
button:hover { border-color:var(--amber); }
button.mono { font-family:ui-monospace, monospace; }
button.hint { border-color:var(--amber); }
button.unusable { color:var(--red); }
button.skip { color:var(--muted); }
.muted { color:var(--muted); }
.warn { color:var(--amber); }
table { border-collapse:collapse; width:100%; font-variant-numeric:tabular-nums; }
td, th { text-align:left; padding:.2rem .5rem; border-bottom:1px solid #2a2a2a; }
th { color:var(--muted); font-weight:normal; font-size:.75rem; text-transform:uppercase; letter-spacing:.06em; }
kbd { background:#2a2a2a; border:1px solid #444; border-radius:3px; padding:0 .3rem; font-size:.75rem; }
</style>
</head>
<body>
<header><b>Wash review</b><span id="counts" class="muted"></span></header>
<main>
<section class="card" id="item">
<p class="muted">Loading…</p>
</section>
<section class="card">
<table id="stats"><thead><tr><th>booth</th><th>operator</th><th>reviewed</th><th>agree</th><th>disagree</th><th>unusable</th></tr></thead><tbody></tbody></table>
</section>
<p class="muted">Keys: <kbd>1</kbd>–<kbd>9</kbd>, <kbd>0</kbd> pick a class in order · <kbd>u</kbd> unusable · <kbd>s</kbd> skip. Skipped items come back after a reload. Your verdict is the training label; the operator's pick is only compared against it.</p>
</main>
<script>
const CLASSES = ${classes};
const skipped = new Set();
let current = null;
async function api(path, init) {
const r = await fetch(path, init);
if (!r.ok) throw new Error(path + ' → HTTP ' + r.status);
return r.json();
}
function esc(s) { return String(s).replace(/[&<>"]/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;'}[c])); }
async function loadStats() {
const s = await api('/api/stats');
const pending = s.booths.reduce((n, b) => n + b.pending, 0);
const reviewed = s.booths.reduce((n, b) => n + b.reviewed, 0);
document.getElementById('counts').textContent = pending + ' waiting · ' + reviewed + ' reviewed';
const tb = document.querySelector('#stats tbody');
tb.innerHTML = s.operators.map(o => '<tr><td>' + esc(o.booth) + '</td><td class="mono">' + esc(o.operatorRef) + '</td><td>' + o.reviewed + '</td><td>' + o.agree + '</td><td' + (o.disagree ? ' class="warn"' : '') + '>' + o.disagree + '</td><td>' + o.unusable + '</td></tr>').join('') || '<tr><td colspan="6" class="muted">nothing reviewed yet</td></tr>';
}
async function next() {
const { items } = await api('/api/items?status=pending&limit=25');
current = items.find(i => !skipped.has(i.id)) || null;
const el = document.getElementById('item');
if (!current) { el.innerHTML = '<p class="muted">Nothing waiting for review.</p>'; return; }
const it = current;
const opClasses = JSON.parse(it.operatorClasses || '[]');
el.innerHTML =
'<img src="/api/items/' + encodeURIComponent(it.id) + '/image" alt="">' +
'<dl style="margin-top:.8rem">' +
'<dt>operator chose</dt><dd><b>' + esc(it.operatorCategoryName) + '</b> <span class="muted">(' + esc(opClasses.join(', ') || 'no classes mapped') + ')</span></dd>' +
'<dt>camera saw</dt><dd class="mono">' + esc(it.visionClass) + ' <span class="muted">' + Math.round(it.visionConfidence * 100) + '%</span>' + (it.downgraded ? ' <span class="warn">flagged downgrade at the booth</span>' : '') + '</dd>' +
'<dt>service</dt><dd>' + esc(it.service) + '</dd>' +
'<dt>booth · operator</dt><dd class="mono">' + esc(it.booth) + ' · ' + esc(it.operatorRef) + '</dd>' +
'<dt>at</dt><dd>' + esc(it.at) + '</dd>' +
'</dl>' +
'<div class="buttons" style="margin-top:.8rem">' +
CLASSES.map((c, i) => '<button class="mono' + (c === it.visionClass ? ' hint' : '') + '" data-label="' + c + '" title="key ' + ((i + 1) % 10) + '">' + c + '</button>').join('') +
'<button class="unusable" data-label="unusable">unusable</button>' +
'<button class="skip" data-skip="1">skip</button>' +
'</div>';
el.querySelectorAll('button[data-label]').forEach(b => b.addEventListener('click', () => verdict(b.dataset.label)));
el.querySelector('button[data-skip]').addEventListener('click', skip);
}
async function verdict(label) {
if (!current) return;
await api('/api/items/' + encodeURIComponent(current.id) + '/review', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ label }) });
await Promise.all([next(), loadStats()]);
}
function skip() { if (current) { skipped.add(current.id); next(); } }
document.addEventListener('keydown', e => {
if (e.target.tagName === 'INPUT') return;
if (e.key === 'u') verdict('unusable');
else if (e.key === 's') skip();
else if (/^[0-9]$/.test(e.key)) { const i = e.key === '0' ? 9 : Number(e.key) - 1; if (CLASSES[i]) verdict(CLASSES[i]); }
});
next().catch(e => { document.getElementById('item').innerHTML = '<p class="warn">' + esc(e.message) + '</p>'; });
loadStats().catch(() => {});
</script>
</body>
</html>`;
}
+10
View File
@@ -0,0 +1,10 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "./src",
"outDir": "./dist"
},
"references": [{ "path": "../../packages/shared" }],
"include": ["src/**/*"],
"exclude": ["src/**/*.test.ts"]
}
+5
View File
@@ -0,0 +1,5 @@
import { defineConfig } from "vitest/config";
export default defineConfig({
test: { include: ["src/**/*.test.ts"], env: { LOG_LEVEL: "silent" } },
});