Securing a VPS Server: A Practical Linux Hardening Guide

A freshly deployed VPS is exposed to the internet within minutes. Automated bots constantly scan for Weak SSH credentials, Open ports, Vulnerable services, Misconfigured Docker containers, Outdated software, Default configurations and other weak spots.

The default installation of most Linux distributions is usable, but not production-ready from a security perspective.

This guide covers practical VPS hardening for Ubuntu/Debian-based systems, including:

• SSH hardening
• Key-based authentication
• Custom SSH configuration
• UFW firewall
• Fail2Ban intrusion prevention
• Kernel/network hardening
• Service minimization
• Auditing and monitoring
• Backup strategy

The goal is not “perfect security.” The goal is reducing attack surface and making compromise significantly harder.

1. Update the System

Before configuring anything else, fully update the server.

sudo apt update && sudo apt upgrade -y
sudo apt autoremove -y

Enable automatic security updates:

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure unattended-upgrades

This helps ensure critical security patches are installed automatically.

2. Create a Non-Root Administrative User

Never use the root account for daily administration.

Create a dedicated user:

adduser myserveruser
usermod -aG sudo myserveruser

Verify sudo access:

su - myserveruser
sudo whoami

Expected output:

root

3. Configure SSH Key Authentication

Password authentication is one of the biggest attack vectors on internet-facing servers.

Use SSH keys instead.

Generate SSH Keys Locally

On your local machine:

ssh-keygen -t ed25519 -C "my-vps-key"

Copy the Public Key to the Server

ssh-copy-id myserveruser@your-server-ip

Test SSH Login

ssh myserveruser@your-server-ip

Do not disable password authentication until SSH key login works successfully.

4. Configure UFW Firewall

Before changing SSH ports or disabling authentication methods, configure the firewall first.

Install UFW

sudo apt install ufw -y

Set Default Policies

sudo ufw default deny incoming
sudo ufw default allow outgoing

Allow SSH Custom Port

We will later move SSH to port 3322.

Allow it now before changing SSH configuration:

sudo ufw allow 3322/tcp

Allow HTTP/HTTPS (If Hosting Websites)

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

Enable UFW

sudo ufw enable

Check Firewall Status

sudo ufw status verbose

Example:

Status: active

To                         Action      From
--                         ------      ----
3322/tcp                   ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
443/tcp                    ALLOW       Anywhere

5. Harden SSH Configuration

Instead of editing the default SSH configuration directly, use:

/etc/ssh/sshd_config.d/custom.conf

Create the file:

sudo nano /etc/ssh/sshd_config.d/custom.conf

Why Use sshd_config.d/custom.conf?

Modern OpenSSH supports modular configuration through:

/etc/ssh/sshd_config.d/

Using a separate custom configuration file is preferred because:

Most distributions already include:

Include /etc/ssh/sshd_config.d/*.conf

inside /etc/ssh/sshd_config.

That means any .conf file placed there is automatically loaded.

6. Recommended Hardened SSH Configuration

Add the following to:

/etc/ssh/sshd_config.d/custom.conf

Port 3322

Protocol 2

PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no

PubkeyAuthentication yes

LoginGraceTime 5m
MaxAuthTries 2
MaxSessions 3

X11Forwarding no

ClientAliveInterval 120
ClientAliveCountMax 2

AllowUsers myserveruser

LogLevel VERBOSE

Explanation of Important SSH Settings

7. Validate SSH Configuration

Always validate before restarting SSH.

sudo sshd -t

If there are no errors:

sudo systemctl restart ssh

Or on some distributions:

sudo systemctl restart sshd

Keep your current SSH session open while testing a new connection.

Test:

ssh -p 3322 myserveruser@your-server-ip

8. Install and Configure Fail2Ban

Fail2Ban automatically bans IP addresses after repeated authentication failures.

Install Fail2Ban

sudo apt install fail2ban -y

Create Local Configuration

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit:

sudo nano /etc/fail2ban/jail.local

Example SSH jail:

[sshd]
enabled = true
port = 3322
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
findtime = 10m
bantime = 1h

Restart Fail2Ban:

sudo systemctl restart fail2ban

Check status:

sudo fail2ban-client status

Check SSH jail:

sudo fail2ban-client status sshd

9. Disable Unnecessary Services

Every running service increases attack surface.

List listening ports:

sudo ss -tulpn

Or:

sudo netstat -tulpn

Disable services you do not need.

Example:

sudo systemctl disable apache2
sudo systemctl stop apache2

10. Secure Shared Memory

Shared memory can sometimes be abused during privilege escalation attacks.

Edit:

sudo nano /etc/fstab

Add:

tmpfs /run/shm tmpfs defaults,noexec,nosuid 0 0

Apply:

sudo mount -o remount /run/shm

11. Apply Basic Kernel and Network Hardening

Edit:

sudo nano /etc/sysctl.conf

Add:

net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.tcp_syncookies = 1

net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0

net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0

Apply:

sudo sysctl -p

12. Install Malware and Rootkit Detection Tools

Install RKHunter

sudo apt install rkhunter -y

Update signatures:

sudo rkhunter --update

Run scan:

sudo rkhunter --check

Install Chkrootkit

sudo apt install chkrootkit -y

Run:

sudo chkrootkit

13. Enable Auditing and Monitoring

Install audit framework:

sudo apt install auditd audispd-plugins -y

Enable:

sudo systemctl enable auditd
sudo systemctl start auditd

Useful commands:

Failed SSH Logins

sudo grep "Failed password" /var/log/auth.log

Login History

last

Active Sessions

w

System Logs

journalctl -xe

14. Secure File Permissions

Check world-writable files:

sudo find / -type f -perm -002 2>/dev/null

Secure SSH permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

15. Secure Docker (If Installed)

Docker can become a major security issue if misconfigured.

Recommendations:

• Avoid exposing Docker daemon publicly
• Use non-root containers
• Limit published ports
• Keep images updated
• Remove unused containers/images
• Use trusted base images
• Scan images regularly

Check running containers:

docker ps

16. Configure Backups

Security also means recoverability.

Minimum recommendations:

• Daily automated backups
• Offsite backup storage
• Snapshot retention
• Database dumps
• Restore testing

Useful tools:

• rsync
• restic
• borgbackup
• Provider snapshots

17. Install Basic Monitoring Tools

Useful utilities:

sudo apt install htop btop ncdu -y

Common checks:

htop
df -h
free -m
uptime

For long-term monitoring consider:

• Prometheus
• Grafana
• Netdata
• Uptime Kuma

Example Hardened VPS Setup

A practical baseline setup might include:

• Ubuntu LTS
• SSH on port 3322
• Root login disabled
• Password authentication disabled
• UFW enabled
• Fail2Ban enabled
• Automatic security updates
• Non-root sudo user
• Docker behind reverse proxy
• HTTPS via Let's Encrypt
• Monitoring enabled
• Regular backups

Common Mistakes

Disabling Password Login Too Early

Always confirm SSH key authentication works before disabling passwords.

Forgetting Firewall Rules

Changing SSH to port 3322 without allowing it in the firewall can lock you out.

Running Everything as Root

Use least privilege whenever possible.

Ignoring Logs

Logs often reveal attacks before they become incidents.

Installing Unnecessary Software

Every package increases maintenance and attack surface.

Conslusion

A VPS connected to the internet is constantly probed by automated bots and scanners. Basic hardening dramatically reduces risk compared to a default installation.

Good security practices include:

• Reducing attack surface
• Using least privilege
• Enforcing key-based authentication
• Keeping systems updated
• Monitoring logs
• Maintaining reliable backups

Security is an ongoing process, not a one-time configuration task.