Implementing the 3-2-1 Backup Rule Concretely for a Single Server

Everyone can recite 3-2-1. Far fewer can walk to a whiteboard and point at their three copies, name the two media, and show which one is genuinely offsite and cannot be deleted. This turns the slogan into a working configuration for a single Linux host.

The 3-2-1 rule is three copies of your data, on two different media, with one copy offsite. It is a good rule precisely because it is boring: it survives disk failure, host compromise, ransomware and a fat-fingered rm, all at once. But the slogan gets recited far more than it gets built. Below is a concrete build using restic 0.17, a separate local disk, and Backblaze B2 with Object Lock, held together by systemd timers.

Why a RAID mirror is not a backup

The most common self-deception is counting a RAID mirror, or a cloud volume snapshot, as one of the three. It is not. A mirror replicates every write instantly, including the write that encrypts your files or the rm -rf that deletes them. Both drives are consistent; both are consistently ruined. A backup is defined by separation in time and control: a copy that reflects the data as it was at a point in the past, that a compromise of the live system cannot reach and overwrite.

So read the three numbers strictly. Three copies means the live data plus two independent backups, not the live data plus a mirror. Two media means the copies do not share a failure domain — a second partition on the same array is not different media; a separate physical disk is, and object storage in another region certainly is. One offsite means one copy is beyond the reach of anything that can go wrong in the room, and, in 2026, beyond the reach of a credential that has compromised the host. That last clause is the one that turns “offsite” into “immutable”.

Copy two: a local restic repo on separate media

The live data on the server is copy one. Copy two is a restic repository on a physically separate disk mounted at /mnt/backup — a different spindle or SSD, ideally on a different controller. Local, fast to restore from, and pruned aggressively because space is finite. This is the copy you will use ninety-five per cent of the time.

#!/usr/bin/env bash
# /usr/local/bin/backup-local.sh  --  copy 2, local restic repo
set -euo pipefail

export RESTIC_REPOSITORY="/mnt/backup/restic"
export RESTIC_PASSWORD_FILE="/etc/restic/local.pass"

restic backup /srv /etc /home \
  --exclude-caches \
  --tag local \
  --host "$(hostname)"

# Keep a sensible window locally, then reclaim space.
restic forget --prune \
  --keep-daily 7 --keep-weekly 4 --keep-monthly 6

Drive it from a systemd timer rather than cron, so you get journald logging, Persistent=true catch-up after downtime, and a jitter window that stops every host in a fleet hammering storage on the same second.

# /etc/systemd/system/backup-local.service
[Unit]
Description=Local restic backup (copy 2)
After=local-fs.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup-local.sh
Nice=10
IOSchedulingClass=idle

# /etc/systemd/system/backup-local.timer
[Unit]
Description=Run local restic backup hourly

[Timer]
OnCalendar=hourly
Persistent=true
RandomizedDelaySec=300

[Install]
WantedBy=timers.target

Enable with systemctl enable --now backup-local.timer. You now have two copies on two media. You do not yet have a backup that survives the host being owned.

Copy three: offsite, and it has to be immutable

Copy three lives in Backblaze B2, reached over its S3-compatible API — Object Lock is exposed through the S3 API, not restic’s native B2 backend, so use the s3: repository form. The efficient way to build it is restic copy: initialise the offsite repository with the same chunker parameters as the local one, so deduplication carries across and you upload each blob once.

# One-off: init the offsite repo sharing the local repo's chunker params
export AWS_ACCESS_KEY_ID="$(cat /etc/restic/b2-keyid)"
export AWS_SECRET_ACCESS_KEY="$(cat /etc/restic/b2-appkey)"

restic -r s3:https://s3.eu-central-003.backblazeb2.com/acme-offsite \
  --password-file /etc/restic/offsite.pass \
  init --copy-chunker-params \
  --from-repo /mnt/backup/restic \
  --from-password-file /etc/restic/local.pass

#!/usr/bin/env bash
# /usr/local/bin/backup-offsite.sh  --  copy 3, push new snapshots offsite
set -euo pipefail

export AWS_ACCESS_KEY_ID="$(cat /etc/restic/b2-keyid)"
export AWS_SECRET_ACCESS_KEY="$(cat /etc/restic/b2-appkey)"

restic -r s3:https://s3.eu-central-003.backblazeb2.com/acme-offsite \
  --password-file /etc/restic/offsite.pass \
  copy \
  --from-repo /mnt/backup/restic \
  --from-password-file /etc/restic/local.pass

Give this second job its own timer on a daily cadence. That is three copies on two media with one offsite. What makes the offsite copy trustworthy against ransomware and against a compromised root is the next step.

Making “immutable” true — and the prune trap

Two controls, layered. First, the credential the host holds should be an append-only B2 application key scoped to the one bucket, with write and read capabilities but without deleteFiles. A machine that can add snapshots but cannot erase them is a machine whose compromise cannot destroy your history. Second, put a default retention on the bucket with Object Lock, so the storage layer itself refuses deletion for a fixed window — defence even against an account-level compromise.

# Bucket created with Object Lock enabled; set a default retention.
# COMPLIANCE mode: no one, not even the account owner, can delete
# a locked object before the window elapses.
aws s3api put-object-lock-configuration \
  --bucket acme-offsite \
  --endpoint-url https://s3.eu-central-003.backblazeb2.com \
  --object-lock-configuration '{
    "ObjectLockEnabled": "Enabled",
    "Rule": { "DefaultRetention": { "Mode": "COMPLIANCE", "Days": 30 } }
  }'

Here is the trap nobody puts on the slide. Object Lock retention starts when an object is uploaded, not when you decide to expire a snapshot. But restic prune reclaims space by repacking and deleting old data files — and locked objects cannot be deleted until their window passes. So an aggressively locked offsite repository will refuse to prune, and your automated append-only key could not prune it anyway. That is not a bug; it is the whole point of immutability, and it changes how you operate the offsite copy:

• Do not run forget --prune on the offsite repo from the host. The automated path only ever adds. Pruning is a deliberate, occasional, privileged operation run with a separate key.
• Size the lock window to your minimum recovery guarantee. Thirty days of COMPLIANCE-mode immutability means thirty days during which nothing — no attacker, no mistake, no rogue admin — can shorten your recovery horizon.
• Let old, unlocked object versions age out via a bucket lifecycle rule, so storage does not grow without bound once the lock window has passed.

Verify all three independently, or you have zero

A copy you have never restored is a hypothesis, not a backup. Verify each copy on its own terms. For structural integrity, restic check validates the repository metadata; to actually read data back off the medium and confirm it is not rotting, use a rolling subset so you cover the whole repository over time without paying to download all of it at once:

# Structure plus 1/7th of the pack data each night = full coverage weekly
restic -r /mnt/backup/restic --password-file /etc/restic/local.pass \
  check --read-data-subset=1/7

# The only verification that counts: restore into a scratch dir and diff
restic -r s3:https://s3.eu-central-003.backblazeb2.com/acme-offsite \
  --password-file /etc/restic/offsite.pass \
  restore latest --target /tmp/restore-test --include /etc

Run the structural check on the local copy nightly and the restore test on the offsite copy monthly at least, because the offsite copy is the one you reach for on the worst day — and the worst day is not when you want to discover the password file was never in the runbook. Restore time is a first-class metric here: knowing you can recover is not the same as knowing how long it takes, which is worth instrumenting alongside your other operational measures rather than guessing at during an incident.

The diagram you can now draw is honest: live data on the host; a local restic repo on a separate disk you prune freely; an offsite B2 repo, append-only from the host and Object-Locked at the bucket, that nothing on the compromised side can touch. Three copies, two media, one offsite and immutable — not as a slogan on a slide, but as three files you can point at and three verifications you can show. If you cannot point at all three today, you do not have 3-2-1; you have one copy and a comforting phrase.