parking_solution/wiki/concepts/backup-recovery.md

---
type: concept
tags: [parking, durability, backup, recovery, security, crypto]
sources: []
updated: 2026-06-29
---

# Backup & Disaster Recovery

The appliance's [[sqlite]] DB **is** the signed [[append-only-event-chain]] — the whole
revenue/audit history. A disk failure or a stolen/destroyed PC currently means **total
loss** (this is [[open-questions]] #5). This page is the settled design for an on-site,
admin-driven backup that survives **total hardware loss** and restores to a fresh appliance
with the signed chain still verifying. (Designed 2026-06-29.)

## The recovery scenario it must satisfy

The driving scenario (the one that forces every decision below): **the PC is gone** — stolen
or destroyed. Its SSD is LUKS-encrypted and **TPM-sealed**, so the disk is unrecoverable *by
design* (a stolen disk won't unlock off its own TPM — see [[disk-os-hardening]], [[tpm]]). We do
**not** want the dead disk; we want to stand up a **new PC**, restore the backup, and continue
signing the **same** chain. For that to work, recovery must depend on **(a)** the backup file and
**(b) two keys held out-of-band** — never on the dead machine.

## Key custody — the load-bearing decision

This is the part the whole plan rests on, and it interacts with the secure-element question
([[open-questions]] #6). Three **independent** keys, three custodians:

| Key | Lives | Recoverable after PC loss? | Job |
| --- | --- | --- | --- |
| **`EVENT_SIGNING_KEY`** | [[fleet-deployment-komodo\|Komodo]] secret (`park_buzi_event_signing_key`), escrowed offsite | **Yes — by design** | Signs + verifies the ledger chain |
| **`park_buzi_backup_key`** *(new)* | Komodo secret, escrowed offsite, **separate** from the signing key | **Yes** | Encrypts/decrypts the backup file |
| **LUKS / TPM disk key** | The appliance's TPM only | **No — deliberately** | At-rest protection of the powered-off SSD |

- **The signing key is decoupled from the TPM** — kept an *extractable software HMAC secret*
  ([[append-only-event-chain]], `signer.ts`), held in Komodo and escrowed by the operator. This is a
  **conscious trade**: a truly non-extractable TPM-sealed signing key (the #6 upgrade) would make the
  ledger unforgeable even against a host-root attacker — but it would also make the **old ledger
  permanently unverifiable after total hardware loss** (the sealed key dies with the machine;
  `buildVerifier(keyId)` would return `undefined` forever). You cannot have *both* "key can never be
  extracted" *and* "I can rescue the key after the machine dies" — they are the same property from two
  sides. Against the [[threat-model|primary adversary]] (the **booth operator**, who has a UI login, not
  host root) an escrowed software key is already tamper-evident, so the recoverable design is chosen
  **today**; revisiting #6 means re-accepting the unverifiable-after-loss cost. See [[tpm]] "TPM vs.
  ATECC608", [[fleet-deployment-komodo]] (the "EVENT_SIGNING_KEY-in-Core is a fraud-root blast radius"
  caveat is the same trade).

- **Backup key is separate from the signing key** even though Komodo holds both — so they can be
  managed independently. Rationale: (1) the **signing key must almost never rotate** (every rotation
  fractures the chain into a new `keyId` segment — old events stay pinned to the old key forever),
  whereas the **backup key may want routine rotation** (a USB went home, a target was decommissioned);
  coupling them drags the cheap op into the expensive one. (2) The backup key **travels to every backup
  destination** (USB, NAS, SFTP); the signing key should travel *nowhere* but Komodo → process memory —
  sharing one key means every backup target conceptually exposes the signing key. (3) Keeping them
  separate keeps the **#6 TPM-migration door open** without re-wiring backups. Decided 2026-06-29
  (the "one fewer secret to escrow" simplicity of a shared key is real, but weakest here because Komodo
  already holds both).

> **The keys are never inside the backup they unlock.** A key can't decrypt the file it's locked in.
> Recovery = backup file **+** both escrowed keys, supplied out-of-band. The runbook must say this
> plainly so nobody "helpfully" stores the keys next to the backups.

## What a backup contains

**Full SQLite DB, snapshots included** — one self-contained, restore-to-identical-appliance file
(ledger + sessions + config + subscriptions + the [[entry-exit-points|snapshot]] BLOBs). Chosen for
completeness over size.

> **Size caveat (interacts with [[open-questions]] #10).** Snapshot BLOBs **dominate** DB size and
> bloat *every* backup. They are unsigned, advisory, and already disk-pressure-pruned
> ([[entry-exit-points]]). A future **"exclude snapshots" toggle** (ledger/sessions/config only — much
> smaller, signed chain still fully preserved) is the obvious knob if backup size becomes a problem; the
> default is the complete picture.

The backup is produced via SQLite **online-backup / `VACUUM INTO`** (a consistent snapshot of the
live WAL-mode DB — **never a raw file copy**, which can capture a torn WAL), then encrypted with
`park_buzi_backup_key`. **Acceptance test:** a restored copy must still pass `verifyChain` — the
signed chain is the thing being protected, so an unverifiable restore is a failed backup.

## Triggers

- **Manual** — an admin-only **"Back up now"** button runs immediately to the configured target.
- **Periodic** — an **in-process daily timer** (same pattern as the snapshot-retention prune,
  [[entry-exit-points]] / `snapshot-retention.ts`): runs only if the configured target is
  reachable/mounted; surfaces last-success / last-error in the UI. No OS cron — it lives inside the
  Fastify process, works inside the [[container-deployment|Docker container]], and is configured in
  one place. ([[offline-first]]: the periodic path must tolerate a missing/unmounted target without
  failing the app.)

## Destinations (admin-configurable)

All three supported in the first cut; the manual button and the periodic timer share them:

- **Local / USB / SATA disk** — a mounted path on an attached disk. Simplest, fully offline, matches
  the air-gapped appliance. The strong first target.
- **Network drive (SMB/NFS)** — a mounted share on the isolated LAN (a site NAS). Still
  local-network, no internet ([[network-isolation]]).
- **SFTP** — push to an SFTP endpoint, useful for an offsite copy. **FTP is excluded** (plaintext
  credentials + data); SFTP is the safe equivalent.

## Retention at the destination

**Keep last N + thinned dailies** (e.g. last 7 daily / last 4 weekly) — bounded disk use, and it
survives the "a bad/partial run clobbered the only good copy" failure. (A single rolling
overwrite-latest file was rejected for exactly that reason.)

## Threat-model fit — restore is the dangerous half

Writing a backup is benign; **restore is operator-adversary surface** ([[threat-model]]). A restored
DB *replaces* the live signed chain — so a malicious restore is a way to swap in a doctored history.
Therefore:

- **Restore is NOT a booth button.** It is an **admin-only, out-of-band runbook action** (new
  appliance, deliberate provisioning step), not something reachable from the operator console.
- The backup **target configuration** and the **"Back up now"** action are admin-gated.
- Backups do **not** weaken the chain's tamper-evidence: a restored chain is re-verified with the
  escrowed `EVENT_SIGNING_KEY`; a tampered restore fails `verifyChain` just as a tampered live DB
  would. The backup is a **durability** control, not an integrity one — integrity stays with the
  signed chain + [[reconciliation]].

## As-built (2026-06-29) — engine + local/mounted target

The first slice is **built and tested**: the backup **engine + a local/mounted target + the daily
timer + the manual route**. What landed:

- **`apps/server/src/backup.ts`** — the engine. Consistent online copy via better-sqlite3's native
  `.backup()` (a transactionally-consistent snapshot of the live WAL DB — **not** a raw file copy),
  then **AES-256-GCM** encryption with a **scrypt-derived** key from `BACKUP_KEY`. Self-describing
  header (`magic | version | salt | iv | … | authTag`) so a restore tool needs only the key + the file
  — **zero new dependencies** (Node `crypto`). The plaintext intermediate is written to **scratch**
  (not the removable/network target) and **wiped in a `finally`**, success or fail. Retention =
  **keep-last-N + one-per-day-within-N-days** (`pruneOldBackups`). Tested: round-trip decrypts to a
  **byte-identical, queryable DB**; a flipped byte or wrong key **fails GCM auth**; short key rejected;
  scratch plaintext always removed.
- **`backup-service.ts`** — the **target directory AND retention are admin-chosen** in the UI
  (`site_config.backup_target_dir`, migration 0016; `backup_keep_last` + `backup_keep_daily_days`,
  migration 0017) and read **fresh each run**, so changing them takes effect with no restart. Retention
  columns are nullable → fall back to the code default (keep-last 7, keep-daily 30) per field. The
  **encryption key is the ONLY backup env/Komodo secret** (`BACKUP_KEY`) — a key must never live in the
  DB it backs up; target+retention are operational policy, not secrets. The service **serializes**
  concurrent runs (single in-flight guard) and records last-success / last-error; `status()` exposes
  `targetDir`, `keepLast`, `keepDailyDays` + `keyPresent` so the UI distinguishes "no target" from
  "no key".
- **`routes/backup.ts`** — `GET /api/backup/status` (`backup:read`); `PUT /api/backup/config` to set/
  clear the target (`backup:update`); `POST /api/backup/test` to probe a candidate path server-side —
  exists / is-a-dir / writable (`backup:update`); `POST /api/backup/run` (`backup:create`), a clean
  **409 `backup_not_configured`** when target+key aren't both set. New `backup` permission resource
  (`backup:read/update/create`) in `@parking/shared`. **No restore route** — out-of-band by design.
- **`apps/web/src/BackupSettings.tsx`** — a Setup → **Backup** tab (gated `backup:read`): an editable
  **target-path field** with a **Test target** probe (localized ok/missing/not-a-dir/not-writable),
  **retention fields** (keep-last / keep-daily-days), one **Save**, the status panel (config state,
  last-run size/pruned/error, a distinct amber **missing BACKUP_KEY** warning), a **Back up now**
  button, and the restore-is-out-of-band note. Full i18n (sq + en).
- **Komodo wiring.** `BACKUP_KEY` is a **per-booth Komodo secret** (`[[park_buzi_backup_key]]` in
  `komodo/resources.toml`; documented in `komodo/.env.komodo.example`), escrowed offsite alongside
  `EVENT_SIGNING_KEY`. It is the *only* backup env var — target + retention are in the DB.
- **`server.ts`** — an **unref'd daily timer** (`backupService.runScheduled`), a **no-op until
  configured**, and **deliberately NOT run at startup** (a just-power-cut booth shouldn't write to a
  possibly-unmounted disk; the daily cadence + the manual button cover it).
- Env documented in `apps/server/.env.example` (with the escrow + separate-key notes).

**SMB/NFS already work** — they're just a mounted path the admin enters as the target. **Deferred to
follow-up slices:** an **SFTP** target and a **restore runbook / CLI**.

## Status

Design settled 2026-06-29; **engine + admin-configured local/mounted target + admin UI BUILT
2026-06-29** (SFTP + restore tooling pending). The target directory is **admin-chosen in the UI**
(`site_config`, migration 0016), not an env var — the on-site admin picks where backups land; only
`BACKUP_KEY` stays a server secret. Resolves the *design* half of [[open-questions]] #5 and the first
build slices; records the key-custody stance that bears on #6 (signing stays decoupled from the TPM) and
#10 (snapshots bloat backups → future exclude toggle). See [[append-only-event-chain]],
[[disk-os-hardening]], [[tpm]], [[fleet-deployment-komodo]], [[reconciliation]].