How to Set Up SSH Keys on Linux

Why keys instead of passwords

With key-based authentication, the server stores your public key and you prove you hold the matching private key, which never leaves your machine. Unlike a password, nothing reusable crosses the network and nothing can be guessed. Once keys are in place you can disable password logins entirely, which removes the target of the constant brute-force attempts every internet-facing SSH server receives.

Step 1: create a key pair

ssh-keygen -t ed25519 -C "you@laptop"

Accept the default location (~/.ssh/id_ed25519) and set a passphrase when asked. The passphrase encrypts the private key on disk; an SSH agent (below) means you won't have to type it every time. Ed25519 keys are short, fast and the default type in current OpenSSH releases. Use -t rsa -b 4096 only if you must connect to very old systems that don't support Ed25519.

This creates two files:

• ~/.ssh/id_ed25519 — the private key. Never copy it to a server or share it.
• ~/.ssh/id_ed25519.pub — the public key. Safe to share; this is what goes on servers (and into GitHub, GitLab and so on).

Step 2: copy the public key to the server

ssh-copy-id [email protected]

ssh-copy-id logs in with your password one last time and appends your public key to ~/.ssh/authorized_keys on the server, creating the directory with the right permissions. Without it, the manual equivalent is:

cat ~/.ssh/id_ed25519.pub | ssh user@server 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'

The first time you connect to a server, SSH shows its host-key fingerprint and asks you to confirm it. To check it properly, compare it with the fingerprint printed on the server itself (via a console or your hosting provider's panel): ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub.

Step 3: test the key login

ssh [email protected]

You should be asked for your key's passphrase (or nothing at all if an agent holds it) instead of the account password. If it still asks for the server password, see troubleshooting below before going further.

Step 4: use an agent so you type the passphrase once

On GNOME and KDE desktops an SSH agent already runs in your session, and the first use of the key unlocks it for the rest of the session. Elsewhere:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l     # list loaded keys

Step 5: a client config file saves typing

Put per-host settings in ~/.ssh/config:

Host web1
    HostName server.example.com
    User deploy
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Now ssh web1, scp file web1: and rsync -a dir/ web1:dir/ all pick up those settings.

Step 6: disable password logins on the server

Current Debian, Ubuntu and Fedora configurations include every file in /etc/ssh/sshd_config.d/ at the top of the main config. For each setting, sshd uses the first value it reads, so a drop-in with a low number takes precedence over the defaults (and over files such as 50-cloud-init.conf on cloud images, which can turn password login back on):

sudo tee /etc/ssh/sshd_config.d/10-key-only.conf >/dev/null <<'EOF'
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin prohibit-password
EOF

# Check the syntax and the effective values
sudo sshd -t
sudo sshd -T | grep -Ei 'passwordauthentication|kbdinteractive|permitrootlogin'

Then reload the SSH server. The service is called ssh on Debian and Ubuntu and sshd on Fedora, RHEL, openSUSE and Arch:

sudo systemctl reload ssh      # Debian / Ubuntu
sudo systemctl reload sshd     # Fedora / RHEL / openSUSE / Arch

On Ubuntu 22.10 and later, SSH is socket-activated by default. Authentication settings like the ones above take effect without extra steps, but if you change the listening Port you also need sudo systemctl daemon-reload and sudo systemctl restart ssh.socket.

Finally, from the second terminal, confirm that password login is refused:

ssh -o PubkeyAuthentication=no [email protected]
# Expected: Permission denied (publickey).

Troubleshooting

• Still asks for the account password. Most often a permissions problem: sshd ignores authorized_keys if your home directory, ~/.ssh or the file are writable by others. Fix with chmod go-w ~ && chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys on the server. On SELinux systems (Fedora, RHEL) also run restorecon -Rv ~/.ssh.
• See what the client is trying. ssh -v user@server shows which keys are offered and why they are rejected.
• See what the server thinks. sudo journalctl -u ssh (or -u sshd) shows authentication failures and their reason.
• "Too many authentication failures". The agent is offering lots of keys. Add IdentitiesOnly yes and an explicit IdentityFile to that host's block in ~/.ssh/config.
• "REMOTE HOST IDENTIFICATION HAS CHANGED". Expected after a server reinstall; verify the new fingerprint, then remove the old entry with ssh-keygen -R server.example.com. If you didn't expect a change, stop and find out why.

Related reading

• Linux file permissions — why 700 and 600 matter here
• systemd basics — services, sockets and journalctl
• Linux security overview — firewalls, updates and hardening
• rsync home backup — works over SSH once keys are set up