A backup you have never restored is a hypothesis

The claim

Backup coverage is the wrong metric. The only number that describes your actual risk is the elapsed time of your last successful, verified restore into a clean environment. If you cannot state that number in minutes, you do not have a recovery capability — you have a folder of files that you believe are useful.

Two numbers, defined properly

Recovery point objective is how much data you are willing to lose, measured backwards from the incident. Recovery time objective is how long you are willing to be down, measured forwards. Nightly dumps at 02:00 give you an RPO of up to 24 hours. A business that takes 60 orders a day and fails at 16:00 has just lost about 35 orders, and no amount of careful restoring brings them back.

Most owners, asked directly, will say losing a day of orders is unacceptable. Most of those same businesses are running nightly dumps and nothing else. The gap between the stated tolerance and the configured behaviour is where the actual risk lives.

Closing the RPO gap costs less than people expect

For PostgreSQL, continuous archiving takes your RPO from hours to roughly the size of one write-ahead log segment. Using pgBackRest:

[global]
repo1-path=/var/lib/pgbackrest
repo1-retention-full=2
repo1-cipher-type=aes-256-cbc

[main]
pg1-path=/var/lib/postgresql/16/main
archive-async=y

With archive_mode = on and archive_command pointed at pgBackRest, a weekly full plus daily incrementals plus continuous WAL gives you point-in-time recovery to any second. The storage cost for a 20 GB database with moderate write volume is typically under 100 GB of retained archive, which is a rounding error on object storage.

The part that is actually hard

Taking backups is easy. Restoring them under pressure is not, and the reason is almost never the database. It is everything around it: the environment variables nobody documented, the S3 credentials that were rotated, the extension that has to exist before the dump will load, the DNS TTL that means your cutover takes 40 minutes regardless of how fast the restore was.

Every one of those is discoverable in advance and only in one way. Restore the thing.

An automated restore test you can build this week

The goal is not a perfect drill. It is a job that fails loudly when the backup stops being restorable.

#!/usr/bin/env bash
set -euo pipefail

DB="verify_$(date +%s)"
createdb "$DB"
trap 'dropdb --if-exists "$DB"' EXIT

pgbackrest --stanza=main --delta --db-path=/tmp/verify restore
pg_restore -d "$DB" /backups/latest.dump

ROWS=$(psql -tAX -d "$DB" -c "SELECT count(*) FROM orders WHERE created_at > now() - interval '2 days'")
[ "$ROWS" -gt 0 ] || { echo "FAIL: no recent orders in restored copy"; exit 1; }

psql -tAX -d "$DB" -c "SELECT 1 FROM users LIMIT 1" > /dev/null
echo "OK: restored and verified, $ROWS recent orders"

The assertion query is the important line. A restore that completes without error but produces an empty orders table is a failure, and a job that only checks the exit code of pg_restore will report success. Pick two or three queries whose results you can reason about, and make the job non-zero when they surprise you.

Measure the wall clock, then publish it

Time the whole sequence, including the steps a script cannot do:

That is a realistic RTO of roughly 35 to 105 minutes for a database most people describe as small. If your stated RTO is 15 minutes, you need a warm standby, not a better backup. If 90 minutes is acceptable, you are fine — but now you know, and so does whoever is going to ask during the incident.

The three rules

1. Backups live in a different account and a different region from the thing they protect. Ransomware that reaches your host reaches everything that host can write to.
2. The restore test runs on a schedule and pages a human when it fails, the same as any other production alert.
3. Someone who is not the person who built it performs a restore, from the documentation, once a year. If they cannot, the documentation is wrong.

Until you have done that once, your backup strategy is a hypothesis. Test it on a Tuesday morning by choice rather than at 03:00 by force.