My VPS security mess

Jun 20, 2025

So it turns out I messed up.

root@ark ~$ cat /etc/ssh/sshd_config | grep PasswordAuth -C 2                                                                     2 ↵

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication no
#PermitEmptyPasswords no

...
#       PermitTTY no
#       ForceCommand cvs server
PasswordAuthentication yes

WAT? I have PasswordAuthentication yes? The first thing I ALWAYS do when I setup a new VPS is to go and disable this.

I have muscle memory of doing vim /etc/ssh/sshd_config, grep for /PasswordAuthetication (using vim /) and uncommenting the line.

And this is where the blunder happened, I might have gone to the “2nd search” result which was #PasswordAuthentication yes instead of the first search result of #PasswordAuthentication no

So turns out I have my server open to raid for approximately 3.5 years:

root@ark ~$ uptime
 05:42:51 up 1386 days,  7:07,  1 user,  load average: 0.02, 0.12, 0.16

Let’s count the pirates

I hope I have fail2ban installed and working. Who really checks if fail2ban works as intended, we just hope it does.

root@ark /var/log$ sudo fail2ban-client status
Status
|- Number of jail:      1
`- Jail list:   sshd
root@ark /var/log$ sudo fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 7
|  |- Total failed:     4501238
|  `- File list:        /var/log/auth.log
`- Actions
   |- Currently banned: 1
   |- Total banned:     591794
   `- Banned IP list:   103.187.147.35

Checking how many BAN actions have occured:

root@ark /var/log$ cat /var/log/fail2ban.log | grep Ban | wc -l
271688

This gives me further information of fail2ban bans:

sudo iptables -L -n

Reading from here: https://serverfault.com/questions/119299/my-var-log-btmp-file-is-huge-what-should-i-do

I also learnt about the lastb command and did a dump like so:

$ lastb -a | wc -l
2464879

and to find total unique IP’s these bad logins came from:

$ lastb -a | awk '{print $10}' | grep -v ^192 | sort | uniq | sed '/^$/d' > bad_login_ips.txt
$ cat bad_login_ips.txt | wc -l
41193

But wait, I don’t want to check which logins failed. I want to check WHO_GOT_IN!??!!?!??!

Luckily that was simple:

$ last -a
root     pts/1        Thu Jun 19 05:14   still logged in    xxx.xxx.xxx.xxx
root     pts/0        Thu Jun 19 05:13 - 05:23  (00:09)     xxx.xxx.xxx.xxx
root     pts/0        Thu Jun 19 04:34 - 04:40  (00:05)     xxx.xxx.xxx.xxx
root     pts/1        Wed Jun 18 12:31 - 12:31  (00:00)     xxx.xxx.xxx.xxx
root     pts/0        Wed Jun 18 08:23 - 12:49  (04:25)     xxx.xxx.xxx.xxx
root     pts/0        Tue Jun 10 11:41 - 11:41  (00:00)     xxx.xxx.xxx.xxx
root     pts/0        Sun May 25 08:44 - 08:52  (00:08)     xxx.xxx.xxx.xxx
root     pts/0        Wed Apr 16 20:28 - 20:33  (00:05)     xxx.xxx.xxx.xxx
root     pts/0        Wed Feb 26 19:17 - 19:18  (00:01)     xxx.xxx.xxx.xxx
root     pts/0        Fri Jan 17 10:43 - 10:44  (00:00)     xxx.xxx.xxx.xxx
root     pts/0        Fri Jan 17 10:43 - 10:43  (00:00)     xxx.xxx.xxx.xxx
...

Those all roughly look like my ip. So I guess I’m safe. No malicious pirate was able to raid my server.

Conclusion

So it turns out keeping PasswordAuthentication yes wasn’t such a big deal after all.

Around 2.5 million bad login attempts were made from ~40,000 unique IP’s and none were succesful. Maybe my root passwords were secure enough that with fail2ban there was no way to brute force it.