Migrating a Fleet From SSH Passwords to Key-Only Auth Without Locking Anyone Out

Turning off SSH password authentication across a fleet is a one-line change. Doing it without locking half your team out of half the estate is the actual job.

Every hardening guide ends at PasswordAuthentication no. Almost none of them tell you the order to do it in, and the order is the whole thing. Set that line before every engineer has a working key on every host and you have not hardened the fleet, you have bricked your own access to it. The recovery is a console session per machine, at the worst possible time, usually on a Friday. This is the staged rollout that avoids that: distribute the keys, prove every login works, keep a break-glass path open, and only then close the door.

The order that keeps you out of trouble

There is a safe sequence and an unsafe one, and they differ only in where the disable step sits. The safe one is: collect the team’s public keys, distribute them to every account on every host, verify that each person can actually authenticate by key alone on each host, confirm you have a way back in if something goes wrong, and disable passwords last. The unsafe one is any sequence where the disable step happens before the verification step. That is the entire lesson. Everything below is mechanics.

The discipline here is the same one that governs any change to a live estate you cannot afford to break: you do not flip the switch on faith, you flip it on evidence. It is the same instinct that keeps a data migration honest when you are moving off a legacy system without breaking the audit trail — the cutover only happens once the new path is proven, not once it is merely built.

Distribute the keys

For a handful of hosts, ssh-copy-id user@host is fine and it is what it exists for. For a fleet it does not scale, because you need the same set of keys on the same set of accounts everywhere, idempotently, with a record of what you did. That is Ansible’s ansible.posix.authorized_key module. Collect each person’s public key once, keep them in a variable (or Vault), and push them:

# distribute-keys.yml — install team keys on every host, idempotently
- name: Distribute team SSH public keys
  hosts: fleet
  become: true
  vars:
    # In practice, load these from ansible-vault, not inline.
    team_keys:
      deploy:
        - "ssh-ed25519 AAAAC3NzaC1lZDI1... alice@laptop"
        - "ssh-ed25519 AAAAC3NzaC1lZDI1... bob@laptop"
  tasks:
    - name: Install each public key for the deploy account
      ansible.posix.authorized_key:
        user: deploy
        key: "{{ item }}"
        state: present
        exclusive: false        # add keys, do not wipe existing ones yet
      loop: "{{ team_keys.deploy }}"

Two flags matter. exclusive: false means you are adding keys alongside whatever is already there — critical during a migration, because you are not yet ready to be the only thing in authorized_keys. You flip that to true in a later, separate pass once every key is confirmed, and that is how you prune stale keys deliberately rather than by accident. And use ed25519 keys, not RSA, for anything issued now. Treat the public keys themselves as inventory you can account for — the same hygiene that stops API keys from becoming a liability applies to the credentials that get people onto the boxes in the first place.

Prove it before you close the door

“We think everyone has keys now” is not a state you can safely disable passwords from. You need “we confirmed it”, and the only way to confirm it is to attempt a key-only login to every host and check it succeeds. The trick is forcing SSH to refuse any fallback, so a failure is unambiguous rather than quietly masked by a password prompt. That is what BatchMode=yes and PreferredAuthentications=publickey do together:

#!/usr/bin/env bash
# verify-keys.sh — confirm the current user reaches every host by key alone.
# Each engineer runs this as themselves before the disable step.
set -u
fail=0
while read -r host; do
  [ -z "$host" ] && continue
  if ssh -o BatchMode=yes 
         -o PreferredAuthentications=publickey 
         -o ConnectTimeout=5 
         -o StrictHostKeyChecking=accept-new 
         "$host" true 2>/dev/null; then
    echo "OK   $host"
  else
    echo "FAIL $host"
    fail=1
  fi
done < hosts.txt
exit "$fail"

BatchMode=yes disables every interactive prompt, so the connection can only succeed by key. A FAIL line therefore means that person genuinely cannot get onto that host without a password — which is exactly the thing you must fix before disabling passwords, and exactly the thing that stays invisible if you test by hand and reflexively type your password when prompted. The script exits non-zero if any host fails, so it drops straight into CI as a gate. Enforcing that the gate actually runs, rather than sitting in a repo as good intentions, is the same problem as keeping a pre-commit baseline that actually runs. Do not move on until this reports clean for every engineer against the full inventory.

The drop-in ordering trap that silently re-enables passwords

This is the one that catches people on modern Ubuntu, and it is worth understanding before you write a single line of config. On Ubuntu 24.04, the main /etc/ssh/sshd_config begins with Include /etc/ssh/sshd_config.d/*.conf, and cloud images ship a file called 50-cloud-init.conf that already contains PasswordAuthentication yes. The critical detail: for most keywords, sshd uses the first value it obtains, not the last. Drop-in files are read in lexical order. So a hardening file named 99-key-only.conf is read after 50-cloud-init.conf — and loses. You set PasswordAuthentication no, reload, test, and passwords still work, because a higher-numbered file cannot override a lower-numbered one for a first-match keyword.

The fix is to make your file sort before the cloud-init one, or to neutralise the cloud-init one. Name the hardening drop-in something like 10-key-only.conf and remove or empty 50-cloud-init.conf so nothing re-asserts the old value later. Always confirm the effective setting with sshd -T | grep -i passwordauthentication, which prints the resolved configuration rather than what any single file claims.

Only now: disable passwords, and keep a way back in

With keys distributed and verified, close the door. Write the hardening drop-in, validate the syntax before it can take effect, and reload rather than restart so live sessions survive:

# disable-passwords.yml — run only after verify-keys.sh is clean everywhere
- name: Enforce key-only SSH
  hosts: fleet
  become: true
  tasks:
    - name: Write key-only drop-in (sorts before 50-cloud-init.conf)
      ansible.builtin.copy:
        dest: /etc/ssh/sshd_config.d/10-key-only.conf
        content: |
          PasswordAuthentication no
          KbdInteractiveAuthentication no
          PubkeyAuthentication yes
        owner: root
        group: root
        mode: "0644"
        validate: "sshd -t -f %s"   # refuse to install a config that will not parse
      notify: Reload ssh

    - name: Neutralise cloud-init's password-auth override
      ansible.builtin.copy:
        dest: /etc/ssh/sshd_config.d/50-cloud-init.conf
        content: "# managed by Ansible; password auth disabled in 10-key-only.confn"
        owner: root
        group: root
        mode: "0644"
      notify: Reload ssh

  handlers:
    - name: Reload ssh
      ansible.builtin.service:
        name: ssh
        state: reloaded

Note KbdInteractiveAuthentication no alongside the obvious line — on its own, disabling PasswordAuthentication can leave a keyboard-interactive path that PAM will still service with a password on some builds. Disable both. And reloaded, not restarted: a reload re-reads the config for new connections without dropping the ones you are currently holding, so a mistake does not eject you before you have noticed it.

Keep a break-glass path independent of SSH keys entirely: cloud-provider serial console, hypervisor console, or out-of-band management. That is your route back if a host ends up with a broken authorized_keys despite everything above. Do the rollout in waves — a canary group first, then batches — rather than the whole fleet in one play, so a surprise is contained to a few machines you can still reach. And keep the verification script running on a schedule afterwards; key-only is a state you have to maintain, not a switch you throw once.

The failure mode here is not subtle and it is not rare. It is disabling passwords on the assumption that keys are in place, discovering per host and per person that they were not, and doing it after you have already removed the only fallback. Verify first, and the last line is boring. Verify last, and it is an incident.