Introduction
SSH is the de-facto remote management channel for Linux infrastructure. It is also one of the most targeted services on the internet — automated bots probe port 22 around the clock looking for weak passwords, stale key formats, and misconfigured daemons.
Most hardening guides stop at "disable root login and add AllowUsers." That is a start, but it leaves a substantial attack surface intact: long-lived static key pairs that never expire, outdated cipher suites that can be downgraded, no centralized revocation, and verbose banners that fingerprint your server. This guide goes further.
By the end you will have:
- An SSH Certificate Authority (CA) that issues short-lived, automatically expiring user certificates
- A hardened
sshd_configusing only modern ciphers, MACs, and key exchange algorithms - Fail2ban configured to rate-limit brute-force attempts
- Structured audit logging forwarded to journald for integration with your SIEM
Certificate-based SSH is the same model that cloud providers use internally. Certificates allow you to rotate trust centrally — revoke the CA and all certificates issued by it become invalid instantly, no per-server authorized_keys cleanup required.
Prerequisites
Before starting, confirm your environment:
# Verify OpenSSH version (need 8.x+ for certificate features)
ssh -V
# Check current daemon status
sudo systemctl status sshd
# Identify your active SSH config
sudo sshd -T | head -30Ensure you have a second terminal or out-of-band access (console, IPMI, VNC) open before modifying sshd_config. Every change in this guide should be tested with sshd -t before reloading.
Step 1 — Audit the Current Configuration
Run a baseline audit to identify weak settings before making changes:
# Test existing config for syntax errors
sudo sshd -t && echo "Config OK"
# Dump the full effective config (runtime values)
sudo sshd -T > /tmp/sshd_baseline.txt
# Check for legacy key types currently accepted
sudo sshd -T | grep -E "HostKeyAlgorithms|PubkeyAcceptedKeyTypes|Ciphers|MACs|KexAlgorithms"
# List current host keys
sudo ls -la /etc/ssh/ssh_host_*Note any entries containing dss, ecdsa (NIST curves — potentially weak), arcfour, 3des, hmac-sha1, or diffie-hellman-group1. These will be removed in later steps.
Step 2 — Generate a New RSA Host Key and Ed25519 Key
Replace legacy host keys with modern equivalents. Ed25519 is preferred; RSA 4096 covers clients that do not support Ed25519.
# Backup existing host keys
sudo cp -r /etc/ssh /etc/ssh.bak.$(date +%Y%m%d)
# Remove legacy host keys (DSA, ECDSA, RSA < 4096)
sudo rm -f /etc/ssh/ssh_host_dsa_key* /etc/ssh/ssh_host_ecdsa_key*
# Generate a fresh Ed25519 host key
sudo ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N "" -C "$(hostname)-host-$(date +%Y%m)"
# Generate a fresh RSA 4096 host key (fallback for older clients)
sudo ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N "" -C "$(hostname)-host-$(date +%Y%m)"
# Fix permissions
sudo chmod 600 /etc/ssh/ssh_host_*_key
sudo chmod 644 /etc/ssh/ssh_host_*_key.pubStep 3 — Build a Simple SSH Certificate Authority
An SSH CA is just an Ed25519 key pair kept in a secure location (ideally an offline machine or a secrets manager like HashiCorp Vault). For this guide we create it on an admin workstation — not on the server you are hardening.
# On your ADMIN WORKSTATION (not the target server)
mkdir -p ~/ssh-ca && chmod 700 ~/ssh-ca
# Generate the CA key pair
ssh-keygen -t ed25519 -f ~/ssh-ca/ssh_ca -C "infra-ssh-ca-$(date +%Y%m)" -N ""
# The public key is safe to distribute; the private key is your crown jewel
ls -la ~/ssh-ca/
# ssh_ca <- PRIVATE — guard this carefully
# ssh_ca.pub <- PUBLIC — distribute to all serversCopy ssh_ca.pub to each server you want to protect:
# From your workstation
scp ~/ssh-ca/ssh_ca.pub adminuser@server-ip:/tmp/ssh_ca.pub
# On the server — install as a trusted CA
sudo mkdir -p /etc/ssh/trusted-ca
sudo mv /tmp/ssh_ca.pub /etc/ssh/trusted-ca/ssh_ca.pub
sudo chmod 644 /etc/ssh/trusted-ca/ssh_ca.pubStep 4 — Issue a User Certificate
Sign a user's public key with the CA to create a certificate. Certificates can carry an expiry, a list of allowed principals (usernames), and source IP restrictions.
# On your ADMIN WORKSTATION
# Assume the user's public key has been sent to you
ssh-keygen -s ~/ssh-ca/ssh_ca \
-I "dylan@infra-$(date +%Y%m%d)" \ # Certificate identity (for logging)
-n dylan,ubuntu,ec2-user \ # Principals: allowed login usernames
-V +8h \ # Validity: 8 hours from now
-z 1 \ # Serial number (increment per issue)
~/.ssh/id_ed25519.pub
# This produces id_ed25519-cert.pub alongside the original key
ssh-keygen -L -f ~/.ssh/id_ed25519-cert.pub # Inspect the certificateThe -V +8h flag is the key security property: even if the certificate leaks, it expires within 8 hours. Adjust the validity window to match your operational tempo (CI pipelines might use +1h; human operators +12h).
Step 5 — Harden sshd_config
Write a hardened daemon configuration. Create the file in two parts: the main config and a drop-in for your CA trust.
sudo tee /etc/ssh/sshd_config.d/00-hardened.conf > /dev/null << 'EOF'
# === Authentication ===
PasswordAuthentication no
PermitEmptyPasswords no
PermitRootLogin no
AuthenticationMethods publickey
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
# Trust certificates signed by our CA
TrustedUserCAKeys /etc/ssh/trusted-ca/ssh_ca.pub
# === Cryptography (modern only) ===
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key
HostKeyAlgorithms ssh-ed25519,rsa-sha2-512,rsa-sha2-256
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com
# === Network ===
Port 22
AddressFamily inet
ListenAddress 0.0.0.0
TCPKeepAlive no
ClientAliveInterval 300
ClientAliveCountMax 2
LoginGraceTime 30
MaxAuthTries 3
MaxSessions 5
MaxStartups 10:30:60
# === Access control ===
AllowGroups sshusers
# AllowUsers dylan # uncomment to restrict to specific users
# === Features — disable what you don't need ===
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no
PermitTunnel no
PermitUserEnvironment no
PrintMotd no
Banner /etc/ssh/banner
# === Logging ===
SyslogFacility AUTH
LogLevel VERBOSE
EOFCreate the SSH users group and add your admin account:
sudo groupadd sshusers
sudo usermod -aG sshusers dylan # replace with your actual usernameCreate a login banner (displayed before authentication):
sudo tee /etc/ssh/banner > /dev/null << 'EOF'
*******************************************************************************
Authorized access only. All sessions are monitored and logged.
Disconnect immediately if you are not an authorized user.
*******************************************************************************
EOFValidate and apply:
sudo sshd -t && echo "Syntax OK"
sudo systemctl reload sshdStep 6 — Regenerate Diffie-Hellman Moduli
Many servers still ship the default /etc/ssh/moduli which includes 1024-bit DH groups vulnerable to Logjam. Remove them:
# Filter out weak moduli (keep only >= 3071 bits)
sudo awk '$5 >= 3071' /etc/ssh/moduli | sudo tee /etc/ssh/moduli.safe > /dev/null
sudo mv /etc/ssh/moduli.safe /etc/ssh/moduli
# Verify
awk '{print $5}' /etc/ssh/moduli | sort -n | uniq -c | headIf moduli ends up empty (some minimal images don't ship it), generate new ones — this takes a few minutes:
sudo ssh-keygen -G /tmp/moduli.candidates -b 4096
sudo ssh-keygen -T /etc/ssh/moduli -f /tmp/moduli.candidatesStep 7 — Install and Configure Fail2ban
Fail2ban monitors auth logs and bans IPs that exceed authentication failure thresholds using iptables/nftables.
# Ubuntu/Debian
sudo apt install fail2ban -y
# RHEL/Rocky
sudo dnf install epel-release -y && sudo dnf install fail2ban -yCreate a local override (never edit /etc/fail2ban/jail.conf directly):
sudo tee /etc/fail2ban/jail.d/sshd.local > /dev/null << 'EOF'
[sshd]
enabled = true
port = ssh
filter = sshd
backend = systemd
maxretry = 3
findtime = 300
bantime = 3600
ignoreip = 127.0.0.1/8 192.168.0.0/16 10.0.0.0/8
EOFsudo systemctl enable --now fail2ban
sudo fail2ban-client status sshdStep 8 — Enable Structured SSH Audit Logging
OpenSSH writes to syslog (AUTH facility). Ensure it flows into journald and optionally into a central SIEM:
# Verify SSH logs reach journald
sudo journalctl -u sshd --since "5 minutes ago" -f &
# Test from a second terminal — you should see auth events
ssh -o BatchMode=yes -o ConnectTimeout=2 invaliduser@localhost || trueFor forwarding to a remote syslog or Loki, add to /etc/rsyslog.conf:
sudo tee /etc/rsyslog.d/50-ssh-remote.conf > /dev/null << 'EOF'
# Forward SSH auth logs to central syslog
:programname, isequal, "sshd" @@siem.internal:514
EOF
sudo systemctl restart rsyslogVerification
Confirm the hardened configuration is active:
# 1. Verify only modern algorithms are offered
nmap --script ssh2-enum-algos -p 22 localhost
# 2. Check the active host keys
ssh-keyscan -t ed25519,rsa localhost 2>/dev/null
# 3. Confirm password auth is rejected
ssh -o PasswordAuthentication=yes -o PubkeyAuthentication=no testuser@localhost
# Should receive: "Permission denied (publickey)."
# 4. Test certificate login (from workstation with cert)
ssh -i ~/.ssh/id_ed25519 -i ~/.ssh/id_ed25519-cert.pub dylan@server-ip
# Should succeed with certificate; verify with -v flag
ssh -v -i ~/.ssh/id_ed25519 dylan@server-ip 2>&1 | grep -i "certificate"
# 5. Confirm Fail2ban is watching
sudo fail2ban-client status sshd
# 6. Validate moduli only has strong entries
awk '{print $5}' /etc/ssh/moduli | sort -n | head -1
# Should be 3071 or higherTroubleshooting
Locked out after applying config
Use your out-of-band console. Run sudo sshd -T | grep -E "passwordauthentication|pubkeyauthentication" to confirm values. Check group membership: groups yourusername.
Certificate rejected: "no matching host key type found"
The client must also trust the server's host certificate. Add the CA public key to ~/.ssh/known_hosts as @cert-authority * <ca-pub-key-content>.
"Permission denied (publickey)" even with cert
Run ssh -vvv and look for Offering public key lines. Ensure the certificate principals match your login username. Check server LogLevel VERBOSE output in journalctl -u sshd.
Fail2ban not banning
Check the backend matches: sudo fail2ban-client get sshd backend. On systemd-based systems use backend = systemd. Check sudo journalctl -u fail2ban for parsing errors.
Moduli file empty after filtering Run the full regeneration command in Step 6. Do not reload sshd with an empty moduli file — it will fall back to software DH or refuse connections.
sshd_config.d drop-ins not loading
Verify your main /etc/ssh/sshd_config contains Include /etc/ssh/sshd_config.d/*.conf. Older images may not include this line — add it manually.
Summary
You have transformed a default SSH installation into a defense-in-depth remote access layer:
| Layer | Before | After |
|---|---|---|
| Authentication | Password + keys | Certificate-only, 8h expiry |
| Key exchange | Includes DH-group1 (Logjam) | curve25519 + strong DH only |
| Ciphers | Includes AES-CBC, 3DES | ChaCha20-Poly1305, AES-GCM only |
| Root access | Often enabled | Disabled |
| Brute force | Unlimited attempts | Fail2ban: banned at 3 failures |
| Audit | Basic syslog | VERBOSE journald, SIEM-ready |
| Revocation | Per-server authorized_keys | Rotate CA key, everything revoked |
The certificate model scales cleanly — as your fleet grows you sign new certificates rather than distributing key files. Pair this with a short validity window and you achieve near-zero standing access: engineers request a certificate, do their work, and the access self-destructs.
For the next step, consider integrating the CA signing step into your identity provider (Okta, Authentik, Teleport) so that users authenticate via SSO and receive a scoped certificate automatically — eliminating the manual distribution step entirely.