Skip to content

chore(selfhost): verify backup and restore posture #1821

Description

@JSONbored

Parent: #1819

Context

A production self-host instance should not rely on a single SQLite volume without a tested restore path. The current stack has local backup and Litestream-oriented pieces, but we need an operator-ready backup posture before presenting the self-host stack as official community infrastructure.

Backups remain self-hosted/operator-owned. The engine should not send database backups, repo config, auth files, or private rules to Gittensory. Operators can store backups locally, or stream them to their own S3-compatible target such as AWS S3, Backblaze B2, Cloudflare R2, MinIO, or another private object store.

Requirements

  • Document and verify the supported backup modes: local backup profile, Litestream/WAL streaming, and Postgres-native backup when DATABASE_URL is used.
  • Confirm the backup path covers the app database, queue state, private repo config expectations, Grafana reporting data expectations, Qdrant snapshots when enabled, and restore order.
  • Keep backup credentials runtime-only through .env or mounted secret files; never bake them into images or docs examples with real values.
  • Ensure BACKUP_ACKNOWLEDGED=true means the operator has intentionally accepted a tested backup plan, not just silenced a warning.
  • Provide a restore drill that starts from a clean volume and proves the service reaches readiness with recovered state.
  • Include failure expectations: what happens if backup storage is unavailable, snapshots fail, or the database is locked.

Deliverables

  • Backup mode matrix covering SQLite local backup, Litestream, Postgres, Qdrant, and Grafana reporting snapshots.
  • Restore runbook with exact command sequence and verification checks.
  • Smoke test or manual validation checklist for backup + restore.
  • Documentation updates in the website self-hosting docs.
  • Optional alert/log improvement if a configured backup path is failing silently.

Acceptance criteria

  • A maintainer can run a backup and restore drill without private, one-off steps.
  • The docs clearly state that backups are self-hosted/operator-owned and not sent to Gittensory.
  • The stack warns clearly when SQLite is used without an acknowledged backup plan.
  • No secret values, auth files, private repo rules, or database contents are committed or included in images.

Validation

  • Run the focused backup script checks where practical.
  • Start a clean stack from restored data and verify /ready passes.
  • Confirm docs build and relevant self-host tests still pass.

Ownership

Maintainer-only. No gittensor:* labels.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

    Projects

    Status
    Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions