Manage SSH keys on your VPS: users, root login, rotation and revocation

A Cloud VPS or dedicated server comes with full root access. Once more than one person works on it, keys need managing: who can log in, as which user, and how to remove someone quickly. This guide assumes your key login already works. If not, start with SSH key authentication.

At setup: ask for your key

UnderHost sends VPS and dedicated server credentials by encrypted email or shows them in CustomerPanel. SSH keys and custom usernames are available on request, so you can ask for your public key to be installed when the server is set up and avoid a first login with a password. Send only the public key (the .pub file).

One user and key per person

Do not share the root login or one key across a team. Give each person their own user with their own key. Then you can remove one person without changing anyone else's access, and the logs show who did what.

# create the user
adduser alex                      # Debian, Ubuntu (asks for details)
useradd -m -s /bin/bash alex      # AlmaLinux, Rocky Linux, CloudLinux

# allow admin commands with sudo
usermod -aG sudo alex             # Debian, Ubuntu
usermod -aG wheel alex            # AlmaLinux, Rocky Linux, CloudLinux

# install the person's public key
install -d -m 700 -o alex -g alex /home/alex/.ssh
echo "ssh-ed25519 AAAA...alex-public-key... alex@laptop" > /home/alex/.ssh/authorized_keys
chown alex:alex /home/alex/.ssh/authorized_keys
chmod 600 /home/alex/.ssh/authorized_keys

On a server with cPanel/WHM, each cPanel account has its own user. Manage those users' keys in WHM or cPanel rather than by hand, and keep root for server administration.

Key-only root login

After each admin can log in with their own key and use sudo, limit root:

# check for overrides, test, then reload
grep -ri "PermitRootLogin\|PasswordAuthentication" /etc/ssh/sshd_config /etc/ssh/sshd_config.d/
sshd -t && systemctl reload sshd    # AlmaLinux, Rocky Linux, CloudLinux
sshd -t && systemctl reload ssh     # Debian, Ubuntu

See who has access

Check every user's key file, not only root's. A comment at the end of each key (like alex@laptop) tells you whose key it is, so always add one.

# every authorized_keys file, with the number of keys in each
for f in /root/.ssh/authorized_keys /home/*/.ssh/authorized_keys; do
  [ -f "$f" ] && echo "$f: $(grep -c . "$f") key(s)"
done

# fingerprints and comments for one file
ssh-keygen -lf /root/.ssh/authorized_keys

# recent SSH logins
last -n 20
journalctl -u sshd -u ssh --since "7 days ago" | grep -i accepted

Rotate a key

Replace a key when a laptop is replaced, a passphrase may be known, or as a yearly habit:

1. Create the new key

   On your computer, run ssh-keygen -t ed25519 -C "you@new-laptop".

2. Add it next to the old one

   Append the new public key to authorized_keys. Both keys work for now.

3. Test the new key

   Log in from a new window with ssh -i ~/.ssh/new_key user@server.

4. Remove the old key

   Delete the old key's line from authorized_keys, and from every other server that had it.

Remove a person's access

# lock the account and remove its keys
usermod -L -e 1 alex
rm -f /home/alex/.ssh/authorized_keys

# remove their key from root and other shared files (match the comment)
sed -i '/alex@laptop/d' /root/.ssh/authorized_keys

# end their open sessions
pkill -KILL -u alex

Also change any passwords, database logins and API keys that person knew, and remove them from CustomerPanel contacts if they had access there.

If you are locked out

• Cloud VPS: open a ticket in @CustomerPanel. Say which user and key stopped working and what you changed last, so the NOC can help restore access.
• Dedicated server: out-of-band KVM and IPMI console access is available on request for servers that need hardware-level management. With the console you can log in locally and fix authorized_keys or sshd_config. See IPMI and remote console access.
• Rescue mode and reinstalls depend on the location and your support scope. Ask in the ticket before reinstalling, because a reinstall erases the server.

FAQ

Related: SSH key authentication | Generate SSH keys | SSH hardening | Connect to your VPS via SSH

Related articles