feat: landing zone backup to R2 via rclone + Litestream

Landing files (append-only JSON.gz) synced to R2 every 30 min via
systemd timer + rclone. Extraction state DB (.state.sqlite) continuously
replicated via Litestream (second DB entry). Auto-restore on container
startup for both app.db and .state.sqlite. Reuses existing R2 bucket
and credentials — no new env vars needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Deeman
2026-02-23 14:06:16 +01:00
parent 5b6c4182f7
commit 76814dade7
8 changed files with 131 additions and 1 deletions

View File

@@ -6,9 +6,12 @@
# LITESTREAM_R2_SECRET_ACCESS_KEY
# LITESTREAM_R2_ENDPOINT e.g. <account-id>.r2.cloudflarestorage.com
#
# Recovery:
# Recovery (app database):
# litestream restore -config /etc/litestream.yml /app/data/app.db
# litestream restore -config /etc/litestream.yml -timestamp "2026-01-15T12:00:00Z" /app/data/app.db
#
# Recovery (extraction state):
# litestream restore -config /etc/litestream.yml /data/landing/.state.sqlite
dbs:
- path: /app/data/app.db
@@ -19,3 +22,12 @@ dbs:
endpoint: https://${LITESTREAM_R2_ENDPOINT}
retention: 8760h
snapshot-interval: 6h
- path: /data/landing/.state.sqlite
replicas:
- url: s3://${LITESTREAM_R2_BUCKET}/padelnomics/state.sqlite
access-key-id: ${LITESTREAM_R2_ACCESS_KEY_ID}
secret-access-key: ${LITESTREAM_R2_SECRET_ACCESS_KEY}
endpoint: https://${LITESTREAM_R2_ENDPOINT}
retention: 8760h
snapshot-interval: 24h