Setting Up BorgBackup With an Append-Only Remote to Survive Ransomware

If ransomware owns a host, it owns that host’s backups too — because the same credentials that write the backups can usually delete them. An append-only Borg repository breaks that assumption: the client can add data but is physically prevented from removing any.

Most backup advice stops at “keep an offsite copy”. That is necessary and insufficient. Modern ransomware crews reconnaissance the environment first, find the backup job, read its configuration, follow the SSH key or API token to the repository, and wipe or encrypt it before they trigger the payload on the primary. The offsite copy dies with everything else because the compromised host had standing permission to destroy it. The fix is to arrange things so that the machine being backed up can never issue a destructive command against the repository — even with full root on the client and the exact key the backup job uses. BorgBackup supports this directly with append-only mode, and the enforcement lives on the server, not in the client’s good behaviour.

The threat model, stated plainly

Assume the attacker has root on the source host and everything on it: the Borg passphrase, the SSH private key, the repository URL, the systemd unit. The question is not “can they read the backup config” — they can — but “what can that config actually do to the repository”. If the answer is “delete archives and compact away the space”, you have lost. If the answer is “append new archives and nothing else”, the attacker can add junk but cannot remove yesterday’s good backup. Pruning old archives and reclaiming disk space then become privileged operations you perform from a separate, trusted path that the compromised client has no route to. This is the same least-privilege instinct behind scoping a credential to exactly what it needs: the backup key should be able to do one verb, and that verb is “append”.

Force append-only in authorized_keys

The enforcement is a forced command on the server’s SSH key. Create a dedicated unprivileged user on the backup host — call it borg — with the repository at /home/borg/repo. Each client gets its own key pair, and the client’s public key is pinned to a borg serve --append-only command in the server’s ~borg/.ssh/authorized_keys. Because the command is forced by the server, the client cannot override it whatever it passes over the wire.

# ~borg/.ssh/authorized_keys on the backup server
# The forced command runs regardless of what the client requests.
# --append-only  : reject any operation that would delete committed data
# --restrict-to-repository : confine this key to a single repo path
# restrict        : disable pty, agent/port/X11 forwarding, ~/.ssh/rc
command="borg serve --append-only --restrict-to-repository /home/borg/repo",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA...client01 backup@client01

The restrict keyword enables every SSH restriction at once — no port forwarding, no agent forwarding, no PTY — so the key is useless for anything except talking to borg serve. Initialise the repository once from a trusted position (not through an append-only session, which cannot create a repo), then hand the client only its pinned key.

The client’s scheduled backup

On the client, the job is an ordinary borg create driven by a systemd timer. Nothing here needs elevated repository rights — appending is all it is permitted to do, and all it needs.

# /etc/systemd/system/borg-backup.service
[Unit]
Description=Append-only Borg backup
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
Environment=BORG_REPO=ssh://borg@backup.example.net/home/borg/repo
# Read the passphrase from a root-only file rather than embedding it
Environment=BORG_PASSCOMMAND=cat /root/.borg-passphrase
ExecStart=/usr/bin/borg create --stats --compression zstd,6 \
    ::'{hostname}-{now}' \
    /etc /home /srv /var/www

# systemd timer: /etc/systemd/system/borg-backup.timer
# [Timer]
# OnCalendar=*-*-* 02:30:00
# Persistent=true
# [Install]
# WantedBy=timers.target

Enable it with systemctl enable --now borg-backup.timer. If this host is later compromised, the attacker can read /root/.borg-passphrase and run Borg by hand — but every destructive verb they try is refused by the server’s forced command. They can append a worthless archive; they cannot touch the good ones.

Prune and compact from a trusted admin path

Append-only repositories still need thinning, or they grow without bound. The rule is that thinning happens from somewhere the client cannot reach. Give a separate admin host its own key on the server, pinned to a non-append-only borg serve, and run retention from there.

# Admin key entry in ~borg/.ssh/authorized_keys (no --append-only):
# command="borg serve --restrict-to-repository /home/borg/repo",restrict ssh-ed25519 AAAA...admin admin@bastion

# Run from the trusted admin host, never from a backed-up client:
export BORG_REPO=ssh://borg@backup.example.net/home/borg/repo

borg prune --list --keep-daily=7 --keep-weekly=4 --keep-monthly=6
borg compact          # THIS is what actually reclaims disk space

Here is the operational catch that surprises people. Since Borg 1.2, and still in 1.4, prune does not free space — it only tags archives as deleted in a new transaction. Only borg compact reclaims the segments, and the docs are blunt about it: “only compaction will really free repository space”. More importantly, running prune or delete through an append-only session frees nothing at all; the deletion is merely appended as a revertible transaction. So compaction must run over a read-write session — which is exactly why the admin key omits --append-only. If your disk usage never drops after prune, this is almost always why.

Reverting after a client compromise

Because append-only mode writes every change as a line in the repository’s transactions file with a UTC timestamp, recovery from a hostile client is a rollback rather than a restore-from-nothing. Identify the last transaction you trust, move the segment files created after it out of data/, and the repository is back to that known-good point. Any client that touched the repo since then must run borg delete --cache-only to rebuild its local cache against the reverted state. The archives an attacker could not delete are still there, waiting.

Test the whole thing before you trust it: from the admin path, do a real borg extract into a scratch directory and confirm the files come back byte-for-byte. A backup you have never restored is a hypothesis, and recovery time is a number your board will eventually ask you to prove — the same discipline that makes mean time to recovery a measured metric rather than a hope. Keep the prune and compact commands in a version-controlled repository with the rest of your reviewed operational scripts, so the one path that can delete data is itself under change control.

The whole design rests on one line in one file on the server. Get the forced command wrong — drop the --append-only, or let the client reach the admin key — and you are back to backups that die with the host. Get it right, and a fully-owned client becomes an attacker who can add to your history but can never rewrite it.