Backups that actually restore: 10 checks for self-hosters
- 1. Inventory everything you would miss
- 2. Write down RPO and RTO per system
- 3. Keep three copies, on two kinds of system, with one offsite
- 4. Keep one copy offline or immutable
- 5. Production hosts must not be able to delete backups
- 6. Automate the job, and prune on purpose
- 7. Alert on failure and on silence
- 8. Dump databases with their own tools
- 9. Encrypt, and store the keys in two other places
- 10. Check integrity, then restore for real
A backup job that has never been restored is a hope. The failures that matter are quiet: a repository that has been corrupt for months, a copy of live database files that will not start, credentials that let a compromised host delete the offsite copy, or a runbook that only lives on the server that just died.
This is a short, practical list of ten checks for small IT teams and self-hosters. Tick an item only when it is done and verified. Commands target restic 0.17+ and BorgBackup 1.4; check them against the documentation for your version, and test before you rely on them.
1. Inventory everything you would miss
List servers, laptops, SaaS, repositories, the password vault, DNS and the domain registrar. You cannot protect what you have not listed. SaaS and registrar access are the most often forgotten: providers protect their platform, not your accidental deletions or account lockouts. Give every backup job a named owner. Jobs without an owner fail silently for months.
2. Write down RPO and RTO per system
RPO is how much data you can afford to lose. RTO is how long you can be down. They decide frequency and method. A photo archive and a payment database should not share the same “nightly, maybe” policy.
3. Keep three copies, on two kinds of system, with one offsite
The classic 3-2-1 rule still earns its keep. It protects against disk failure, theft, fire and a provider outage. A second disk in the same machine is not a second system.
4. Keep one copy offline or immutable
Ransomware and compromised admin accounts delete every backup they can reach. That is the extra “1” in 3-2-1-1-0: object lock, a rotated offline disk, or a copy the production host cannot overwrite.
# enable Object Lock / versioning on the bucket first, then:
rclone copy /srv/restic-repo remote:backup-bucket --immutable
5. Production hosts must not be able to delete backups
Use separate credentials for backup storage. If an attacker owns the server, the backups must not fall with it. Append-only or write-only keys for the machine, and admin keys kept elsewhere, are the usual pattern.
6. Automate the job, and prune on purpose
Manual backups stop the first busy week. Without pruning, storage fills up and the newest backups fail.
restic -r /srv/restic-repo backup /etc /home /srv --exclude-file=excludes.txt
restic -r /srv/restic-repo forget --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --prune
For Borg 1.4:
borg prune --keep-daily=7 --keep-weekly=4 --keep-monthly=12 /srv/borg-repo
borg compact /srv/borg-repo
Borg 2.x uses different commands; check the docs before you copy this.
7. Alert on failure and on silence
A job that stopped running sends no error. A heartbeat check catches it: the job pings a monitor only after a successful run, and missing pings raise an alert.
backup.sh && curl -fsS https://<your-monitor>/ping/<id> > /dev/null
8. Dump databases with their own tools
Copying the files of a running database can produce an unusable backup. Use a logical dump or a supported hot-backup tool, then include that dump in the file-level backup.
pg_dump -Fc -d appdb -f /backup/appdb.dump
mysqldump --single-transaction --routines appdb > /backup/appdb.sql
9. Encrypt, and store the keys in two other places
Offsite and cloud copies leave your control; client-side encryption keeps them private. restic encrypts every repository by default. For Borg 1.x:
borg init --encryption=repokey-blake2 /srv/borg-repo
borg key export /srv/borg-repo /safe/borg-key.txt
A lost key means a lost backup. A key stored next to the backup protects nothing. Protect backup and cloud admin accounts with MFA, and keep recovery codes offline.
10. Check integrity, then restore for real
The final “0” in 3-2-1-1-0 is zero errors. Detect corruption before you need the data:
restic -r /srv/restic-repo check --read-data-subset=10%
borg check --verify-data /srv/borg-repo
Then hold a restore drill every quarter: one file, one database, one whole machine. Only a successful restore proves a backup works. Measure the time and the data loss and compare them with the RTO and RPO you wrote down in check 2. At least once, let someone other than the usual admin run the drill using only the documentation. Disasters rarely wait until the expert is available.
date # start
restic -r /srv/restic-repo restore latest --target /tmp/restore-test
# then restore a database dump into a scratch instance and verify
date # end — that interval is your achieved RTO
Keep the DR runbook outside the systems it describes — printed, and in a second location. The runbook must be readable when servers, wiki and email are down.
The ten checks above are the short version of our Backup & Disaster Recovery checklist, licensed CC BY 4.0 (attribution: Sats Sheets). The full checklist — a printable PDF, an XLSX/Markdown tracker and a DR runbook template — is available in English, French, Italian, Polish, Hungarian, Dutch and Turkish, paid in sats: English edition. All of our IT checklists are listed in the Sats Shop catalog.
No warranty; test before you rely on it. Always have a tested backup before changing a production system.
Write a comment