This fixes: restic ‘repository is already locked’ after a killed backup

A restic run got killed overnight — OOM killer, a spot instance pulled away, someone hit Ctrl-C — and now every backup fails with unable to create lock in backend: repository is already locked. The repository is fine. The lock is not.

This is one of the most common ways a restic backup schedule quietly stops working. The fix takes one command. Doing it safely — without racing a process that is genuinely still running, and without corrupting a repository that several hosts write to — takes about two minutes of checking first. Here is the locking model, the safe recovery, and the wrapper that stops it recurring.

What restic’s lock actually is

restic does not use a database or a lock server. A lock is just a small JSON file written into the locks/ folder of the repository. It records the hostname, the process ID, the username, a timestamp, and whether the lock is exclusive.

There are two kinds. A non-exclusive lock is what backup, check and restore take — several of them can coexist because they only read the existing data or add new blobs. An exclusive lock is what prune, forget --prune and index rebuilds take, because they delete or rewrite data and must be the only writer. The error you are looking at means a lock file exists that restic will not write past.

Crucially, a live restic process refreshes its lock every five minutes. A lock whose timestamp is older than thirty minutes is treated as stale, as is any lock created on the current host whose PID is no longer running. That dual rule is the whole safety story: restic can be certain a same-host lock is dead by checking the PID, but for a lock written by another machine all it has is the age.

Confirm nothing is holding it — before you touch anything

The dangerous mistake is to clear a lock that a real, working backup is still holding. On a single host that is easy to rule out. List the locks without taking one yourself, read who owns it, and check whether that process still exists.

# Inspect locks without creating one of your own
restic list locks --no-lock

# Show the detail: host, PID, user, exclusive/non-exclusive, age
restic list locks --no-lock | while read id; do
  echo "== $id =="
  restic cat lock "$id" --no-lock
done

# Is the owning process actually alive on THIS host?
ps -p 19309 -o pid,etime,cmd 2>/dev/null || echo "PID 19309 is gone"

If the lock names your own hostname and the PID is gone, it is stale and safe to clear. If it names a different host, do not guess — go to that machine and confirm no restic process is running there. A backup that is genuinely mid-run on another node will be refreshing its lock; clearing it out from under a live prune is how you corrupt a repository. When you need to reconstruct what happened around the killed run, an audit trail worth having is a great deal more useful than guessing from the lock timestamp.

unlock, and when you actually need –remove-all

Once you are satisfied nothing holds the repository, the plain command does the right thing:

# Removes only STALE locks: same-host dead PID, or older than 30 minutes
restic unlock

# Removes ALL locks, stale or not. Only after you have manually
# confirmed no restic process is running on ANY host.
restic unlock --remove-all

The distinction matters. Bare restic unlock respects the staleness rules, so if a lock is fresh and from another host, it will be left in place — which is exactly what you want, because a fresh remote lock probably belongs to a working backup. --remove-all overrides that judgement and deletes everything. It is the correct tool when a same-host lock is under thirty minutes old but you know the process died (a hard kill inside the refresh window), and it is a foot-gun everywhere else. Reach for it only after the checks above, never as the reflex.

If you find yourself running unlock as a matter of routine, the lock is a symptom, not the problem. Something is killing your backups — usually the kernel’s OOM killer reaping restic under memory pressure, or two schedules overlapping. Fix that and the locks stop appearing.

Never delete the lock file by hand on a shared repository

It is tempting, when the backend is a plain filesystem or an S3 bucket you can browse, to just rm the file in locks/. Do not. On a repository that only one host ever touches you will probably get away with it, but the moment more than one machine backs up to the same repository, hand-deletion bypasses the one check that keeps you safe: restic’s own logic that refuses to remove a lock which is still being refreshed. restic unlock reads every lock, applies the staleness rules, and leaves live ones alone. A manual rm applies no rules at all — and if you delete the exclusive lock a live prune is holding on another node, both processes now believe they own the repository. Always go through restic unlock.

Stop it recurring: a timeout and a flock wrapper

Most recurring locks come from overlap: last night’s backup ran long, tonight’s started before it finished, and now two restic processes are contending for the same repository. The clean fix is a mutual-exclusion lock on the host, so a second run never starts while the first is going. flock from util-linux does exactly this, and it costs one line.

#!/usr/bin/env bash
# /usr/local/bin/nightly-backup.sh
set -euo pipefail

LOCKFILE=/run/lock/restic-backup.lock

# Open the lock file on fd 9 and take a non-blocking exclusive lock.
# If another copy of this script already holds it, exit 0 quietly
# rather than piling a second restic process onto the repository.
exec 9>"$LOCKFILE"
if ! flock -n 9; then
  echo "restic backup already running; skipping this run" >&2
  exit 0
fi

export RESTIC_REPOSITORY="s3:https://s3.eu-central-1.amazonaws.com/acme-backups"
export RESTIC_PASSWORD_FILE=/etc/restic/password

# Cap the run so a wedged backup cannot hold the flock forever.
timeout 6h restic backup /etc /srv /var/lib \
  --tag nightly \
  --exclude-caches \
  --one-file-system

# Housekeeping under the same host lock, so prune never races a backup.
restic forget --prune \
  --keep-daily 7 --keep-weekly 4 --keep-monthly 6

Two things are doing the work here. flock -n 9 guarantees a single restic process per host, so overlapping schedules can no longer produce contending writers. And timeout 6h bounds the run: if a backup wedges — a stalled network mount, a hung backend — it is killed rather than left to hold both the flock and, eventually, a stale repository lock. If you drive this from a systemd timer instead of cron, you get the same guarantee for free by setting RuntimeMaxSec= on the service and letting systemd refuse to start a second instance.

The same discipline applies to any operation that should not stampede a shared resource, backups or otherwise — it is the same instinct as making clients back off instead of hammering an API. Serialise the writers, bound the run, and the lock file goes back to being what it is meant to be: proof that a backup is genuinely in progress, not the reason your last three nights are missing.

Free interactive tool

Website compliance checklist

What your site has to do, based on what it actually does

Answer as much or as little as you like — the list builds as you go. Nothing is stored against your name and no email is required.

Most technology problems are not technology problems. They are control problems.

The systems exist. The investment has been made. The question is whether leadership can understand, direct, evidence, and sustain what those systems produce. Find out where control exists — and where it only appears to.