Whenever I’m teaching I prefer to use real examples when possible as opposed to contrived ones. This is why in my online training class, Getting to Know Splunk: The Hands-On Administration Guide, there are no sample log files that you are asked to work with. Instead – you capture the actual log files from your training system and work with them. I believe this is a much more valuable teaching tool, since it drives home the point that attackers are constantly trying to break into systems on the Internet.
I was recently teaching some of our new SOC analysts the basics of Splunk (every new Hurricane Labs employee goes through the same training course – they just get the in-person variety), and was really impressed with their analysis of the events logged by their AWS instances.
This got me thinking – we’re already basically creating an SSH honeypot in the lab. Why don’t we add a few more features to make this even more interesting? Which has lead me to this tutorial.
Let’s start by getting some more data, and then we’ll do something with it.
Capturing SSH passwords
It’s one thing to know what users are logging in, and that they’re not successful in doing so, but what if you could also find out what passwords they’re trying?
This was accomplished using the process outlined in this Hacker Noon article: “How I’ve captured all passwords trying to ssh into my server!”
(It should go without saying, but please don’t run this on a system that you care about).
I needed to add a few extra steps from my lab instance to account for tools that weren’t already installed, such as:
I also just ended up moving the system’s existing sshd binary and symlinking the new one to the original location (this is a lab system, after all):
If you’re not using a Ubuntu install, these likely will be a bit different, so your mileage may vary.
Using the data in Splunk
At this point, if you’ve done everything correctly, you’ll see events in /var/log/auth.log that look like this:
And likewise, these results will also be displayed in Splunk when searching the linux_secure sourcetype: