How to Secure SSH Access: Keys, Fail2ban, and Network Restrictions

Why SSH is a top target for attackers

SSH (Secure Shell) is the standard way to remotely manage Linux and Unix servers. Because it provides full command-line access to a system, it is one of the highest-value targets for attackers.

If you run a server with SSH exposed to the internet, it is being attacked right now. Check your auth log:

# See recent failed SSH login attempts
grep "Failed password" /var/log/auth.log | tail -20

# Count total failed attempts
grep -c "Failed password" /var/log/auth.log

It is common to see thousands or tens of thousands of brute force attempts per day on a publicly exposed SSH port. Automated botnets continuously scan the internet for SSH servers and try common username/password combinations.

Step 1: Disable password authentication

The single most important step. Password authentication is vulnerable to brute force attacks, credential stuffing, and password reuse. SSH key authentication is cryptographically strong and immune to these attacks.

Generate an SSH key pair

On your local machine (not the server):

# Generate an Ed25519 key (recommended -- fast and secure)
ssh-keygen -t ed25519 -C "your_email@yourcompany.com"

# Or RSA 4096 if you need compatibility with older systems
ssh-keygen -t rsa -b 4096 -C "your_email@yourcompany.com"

When prompted, save the key to the default location (~/.ssh/id_ed25519) and set a strong passphrase.

Deploy the public key to the server

# Copy your public key to the server (easiest method)
ssh-copy-id user@your-server-ip

# Or manually:
cat ~/.ssh/id_ed25519.pub | ssh user@your-server-ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

# This should log you in without asking for a password
ssh user@your-server-ip

Disable password authentication

Once you have confirmed key-based login works, edit the SSH daemon configuration:

sudo nano /etc/ssh/sshd_config

Set these directives:

PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
UsePAM no

Restart the SSH service:

sudo systemctl restart sshd

Important: Do not close your current SSH session until you have confirmed you can open a new session with key-based auth. Otherwise, you may lock yourself out.

Never allow direct root login over SSH. Use a regular user and sudo instead:

PermitRootLogin no

If you absolutely need root SSH access (not recommended), at minimum restrict it to key-based authentication:

PermitRootLogin prohibit-password

Step 3: Change the default port

Changing SSH from port 22 to a non-standard port reduces automated scanning noise significantly. This is not a security measure on its own -- a determined attacker will find the port -- but it eliminates the vast majority of bot traffic.

Port 2222

After restarting SSH, connect with:

ssh -p 2222 user@your-server-ip

Warning: Make sure you update your firewall rules to allow the new port before restarting SSH, or you will lock yourself out.

# Allow the new port before restarting SSH
sudo ufw allow 2222/tcp
sudo systemctl restart sshd

# Once confirmed working, block the old port
sudo ufw deny 22/tcp

For more on managing open ports and their security implications, see our guide on open ports security.

Step 4: Install and configure fail2ban

fail2ban monitors log files and bans IP addresses that show signs of brute force attacks. It dynamically adds firewall rules to block offending IPs.

Install

# Debian/Ubuntu
sudo apt update && sudo apt install fail2ban -y

# RHEL/CentOS/Alma
sudo dnf install fail2ban -y

Configure for SSH

Create a local configuration file (never edit the defaults directly):

sudo nano /etc/fail2ban/jail.local

[sshd]
enabled = true
port = ssh
# Use the port you configured, e.g., port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

This bans an IP for 1 hour after 3 failed attempts within 10 minutes.

Start and enable

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# Check status
sudo fail2ban-client status sshd

Verify it is working

# See currently banned IPs
sudo fail2ban-client status sshd

# Unban an IP if needed (e.g., you locked yourself out)
sudo fail2ban-client set sshd unbanip 203.0.113.5

Step 5: Restrict SSH to specific IPs

If your team only connects from known IP addresses or IP ranges, restrict SSH access at the firewall level.

Using UFW (Ubuntu)

# Allow SSH only from your office IP
sudo ufw allow from 203.0.113.0/24 to any port 22 proto tcp

# Allow from a VPN exit IP
sudo ufw allow from 198.51.100.10 to any port 22 proto tcp

# Deny SSH from everywhere else (default deny handles this if enabled)
sudo ufw default deny incoming
sudo ufw enable

Using iptables

# Allow SSH from specific IP
sudo iptables -A INPUT -p tcp -s 203.0.113.0/24 --dport 22 -j ACCEPT

# Drop SSH from everywhere else
sudo iptables -A INPUT -p tcp --dport 22 -j DROP

# Save rules
sudo iptables-save | sudo tee /etc/iptables/rules.v4

Using cloud security groups

If you are running in AWS, Azure, or GCP, configure SSH access in your cloud security group or network security group. Only allow inbound port 22 from specific IP addresses -- never from 0.0.0.0/0.

Step 6: Use a VPN or bastion host

The most secure approach is to not expose SSH to the public internet at all.

VPN approach

Run a VPN server (WireGuard is the modern choice) and only allow SSH from the VPN subnet:

# Allow SSH only from WireGuard subnet
sudo ufw allow from 10.0.0.0/24 to any port 22 proto tcp

Your team connects to the VPN first, then SSHs to servers over the private network.

Bastion host (jump server)

A bastion host is a single hardened server that acts as the gateway to your infrastructure. All SSH access goes through the bastion:

Internet --> Bastion Host --> Internal Servers

Internal servers only allow SSH from the bastion's IP. The bastion itself is heavily monitored and locked down.

Connect through a bastion with SSH ProxyJump:

# Direct jump
ssh -J user@bastion.yourcompany.com user@internal-server

# Or configure in ~/.ssh/config
Host internal-server
    HostName 10.0.1.5
    User admin
    ProxyJump user@bastion.yourcompany.com

Additional hardening

Limit allowed users

Restrict which users can log in via SSH:

AllowUsers deploy admin

Set idle timeout

Disconnect idle sessions to reduce the window of opportunity if a session is left open:

ClientAliveInterval 300
ClientAliveCountMax 2

This disconnects after 10 minutes of inactivity.

Use SSH certificates (advanced)

For larger teams, SSH certificates simplify key management. Instead of distributing public keys to every server, a certificate authority signs user keys, and servers trust the CA.

Enable two-factor authentication

For an extra layer, add TOTP (Google Authenticator) to SSH:

sudo apt install libpam-google-authenticator

This requires both an SSH key and a TOTP code to log in.

Summary: SSH hardening checklist

Also review your broader network exposure -- our guide on open ports security covers other commonly exposed services, and weak TLS cipher suites explains how to harden encrypted connections.

How SurfaceScan helps

SurfaceScan detects SSH services exposed on the internet during every scan -- on the default port 22 and on non-standard ports. It flags password authentication that is still enabled, outdated SSH protocol versions, and SSH running on hosts where it likely should not be publicly accessible. Findings appear in the Network Security section with details on the port, protocol version, and authentication methods detected, helping you verify that your SSH hardening is actually working from the outside.