feat(snapshot): re-encode captures + disk-pressure retention
Build desktop / desktop (push) Successful in 4m13s
Build & push images / images (push) Successful in 2m56s
CI / check (push) Successful in 38s

Camera snapshots were stored RAW — the camera's full-res JPEG straight into
the BLOB, no resize/recompress. Measured on the dev DB: 300 snapshots = 81.7 MB
= ~72% of the 114 MB SQLite file (the big ones 2688×1520 / ~600 KB, Hikvision
main stream). They dominated the appliance's single backed-up DB file.

Re-encode on capture (snapshot.ts):
- Downscale each frame to SNAPSHOT_MAX_EDGE (1280px long edge) + recompress at
  SNAPSHOT_JPEG_QUALITY (80) via sharp (libvips, Apache-2.0) before storage —
  ~6-10× smaller (verified 2688×1520 → 1280×724, ~8×), plate still readable,
  clean image/jpeg (drops the camera's charset cruft). STORAGE-ONLY: recognition
  keeps the ORIGINAL full-res bytes (downscaling hurts OCR). Fail-soft — a
  re-encode error stores the original, never drops the snapshot or blocks the
  (already-open) path. sharp lives in apps/server (owns the capture path), where
  bcrypt already establishes the native-dep pattern.

Disk-pressure retention (snapshot-retention.ts) — a SAFETY VALVE, not the daily
mechanism (the re-encode does that). Daily check reads the DB filesystem used%
(statfs on db.$client.name); no-op unless ≥ SNAPSHOT_DISK_HIGH_PCT (70). Over the
mark: delete the OLDEST until an estimated SNAPSHOT_DISK_FREE_TARGET_PCT (10%) of
disk is freed — never below SNAPSHOT_MIN_KEEP (500) — then VACUUM once to return
space to the OS. A DELETE only frees SQLite pages (disk doesn't drop until VACUUM),
so the loop is driven by estimated freed bytes (SUM(length(bytes))), not a live
disk re-read; the prune owns the DB-locking VACUUM, run daily off-peak. diskUsage
is injectable for tests. None of this touches the signed ledger — snapshots are
unsigned/advisory, referenced only by id.

Tests: encodeForStorage (downscale / clean-type / no-enlarge / fail-soft) +
pruneSnapshots (no-op below mark / delete-oldest-to-target + VACUUM / MIN_KEEP
floor / skip-VACUUM-when-empty). All four snapshot env knobs documented in the
komodo env reference. Full workspace build/lint/test green; the prune smoke-verified
on a scratch DB copy (file shrank after VACUUM).

Existing ~81.7 MB of raw snapshots are unchanged (a one-off re-encode backfill is
a separate optional follow-up). Updated entry-exit-points + technology-stack wiki.

Claude-Session: https://claude.ai/code/session_01Xcm6ikLgGoCxxHrxtjkk5V
This commit is contained in:
2026-06-28 17:15:15 +02:00
parent cce99aadfd
commit 96acd6b662
11 changed files with 727 additions and 5 deletions
@@ -0,0 +1,96 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { snapshots, type Db } from "@parking/db";
import { createTestDb } from "@parking/db/testing";
import { pruneSnapshots, type DiskUsage, type SnapshotRetention } from "./snapshot-retention.js";
// Snapshot retention: DISK-PRESSURE prune. No-op unless the DB's filesystem is over the
// high-water mark; then delete the OLDEST until ~freeTargetPct of disk is freed (estimated from
// the deleted BLOB sizes), honoring a MIN_KEEP floor, then VACUUM once. Disk usage is injected
// so the test controls the trigger without touching the real filesystem.
let db: Db;
beforeEach(() => {
({ db } = createTestDb());
});
/** Insert `n` snapshots, oldest first (s-0 is the oldest), each `bytes` long. */
function seed(n: number, bytes = 1000): void {
const t0 = Date.now() - n * 1000;
for (let i = 0; i < n; i++) {
db.insert(snapshots)
.values({
id: `s-${i}`,
direction: "entry",
deviceId: "cam",
identity: `s-${i}`,
contentType: "image/jpeg",
bytes: Buffer.alloc(bytes, 1),
capturedAt: new Date(t0 + i * 1000).toISOString(), // s-0 oldest … s-(n-1) newest
})
.run();
}
}
function count(): number {
return db.select().from(snapshots).all().length;
}
function ids(): string[] {
return db.select().from(snapshots).all().map((r) => r.id).sort();
}
/** A fake disk at a given used% on a 1 GB volume. */
const disk = (usedPct: number, totalBytes = 1_000_000_000): (() => Promise<DiskUsage>) =>
() => Promise.resolve({ usedPct, totalBytes });
const ret = (o: Partial<SnapshotRetention>): SnapshotRetention => ({
highPct: 70,
freeTargetPct: 10,
minKeep: 2,
batch: 5,
...o,
});
describe("pruneSnapshots (disk-pressure)", () => {
it("no-op when disk is below the high-water mark", async () => {
seed(10);
const res = await pruneSnapshots(db, { retention: ret({}), diskUsage: disk(50) });
expect(res.deletedRows).toBe(0);
expect(res.vacuumed).toBe(false);
expect(count()).toBe(10);
});
it("over the mark: deletes the OLDEST until ~freeTargetPct is freed, then VACUUMs", async () => {
// 1 GB disk, target 10% = 100 MB. Each snapshot 20 MB → ~5 deletions reach the target.
seed(20, 20 * 1048576);
const vacuumSpy = vi.spyOn(db.$client as { exec: (s: string) => void }, "exec");
const res = await pruneSnapshots(db, { retention: ret({ minKeep: 2, batch: 100 }), diskUsage: disk(80) });
expect(res.deletedRows).toBeGreaterThanOrEqual(5);
expect(res.freedBytesEst).toBeGreaterThanOrEqual(0.1 * 1_000_000_000);
expect(res.vacuumed).toBe(true);
expect(vacuumSpy).toHaveBeenCalledWith("VACUUM");
// The survivors are the NEWEST (oldest went first).
const survivors = ids();
expect(survivors).toContain(`s-19`); // newest kept
expect(survivors).not.toContain(`s-0`); // oldest pruned
vacuumSpy.mockRestore();
});
it("honors the MIN_KEEP floor even when still over target", async () => {
// Target 10% of 1 GB = 100 MB, but only 3 tiny snapshots exist and minKeep=2 → at most 1 deleted.
seed(3, 1000);
const res = await pruneSnapshots(db, { retention: ret({ minKeep: 2, freeTargetPct: 90 }), diskUsage: disk(95) });
expect(res.deletedRows).toBe(1); // 3 − minKeep(2)
expect(count()).toBe(2);
expect(res.floorHitWhileOver).toBe(true); // couldn't reach target without crossing the floor
});
it("skips VACUUM when nothing was deleted", async () => {
seed(2); // == minKeep, so nothing to delete even over the mark
const vacuumSpy = vi.spyOn(db.$client as { exec: (s: string) => void }, "exec");
const res = await pruneSnapshots(db, { retention: ret({ minKeep: 2 }), diskUsage: disk(99) });
expect(res.deletedRows).toBe(0);
expect(res.vacuumed).toBe(false);
expect(vacuumSpy).not.toHaveBeenCalled();
vacuumSpy.mockRestore();
});
});