[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]
On 25/11/13 19:42, stinga wrote: > On 25/11/13 16:15, bad apple wrote: >> Let's hope for more details from Stinga when he's finished >> locking everything down again. I love post-hack forensics! > Not sure there will be much! Reinstall is not really an option at > this time. It is pretty locked down but obviously not enough! > > There was no failed attempts from either of the two ip address used > to gain entry. > > Nov 23 00:56:48 sq sshd[28153]: Accepted publickey for root from > 141.8.248.26 port 4596 ssh2 Nov 23 00:56:48 sq sshd[28153]: > pam_unix(sshd:session): session opened for user root by (uid=0) Nov > 23 00:57:11 sq useradd[28257]: new group: name=mark, GID=1211 Nov > 23 00:57:12 sq useradd[28257]: new user: name=mark, UID=1210, > GID=1211, home=/home/mark, shell=/bin/bash Nov 23 00:57:15 sq > passwd[28263]: pam_unix(passwd:chauthtok): password changed for > mark Nov 23 00:57:44 sq sshd[28277]: Accepted password for mark > from 188.77.48.69 port 55870 ssh2 > > I noticed the new user and new user login from logwatch also about > 10 bounced emails to the new user they had created. > > Looks like they got in via a publickey access that was put there > somehow... -rw-r--r-- 1 root root 607 2013-11-02 17:55 > authorized_keys -rw-r--r-- 1 root root 6310 2013-04-30 12:47 > known_hosts God knows when that happened, beginning of the month if > you believe the time-stamp... > > So there were entries in the /root/.ssh/authorized_keys file. Now > removed, along with PubkeyAuthentication no in sshd config > > Shouldn't be able to get in anyway since root is locked down. > > How they got in? 1 - brute force 2 - key log or something else from > a windows box, there is 1 other user with shell access that uses > windows 7 3 - exploitable php script that allowed root access and > the creating of a /root/.ssh/authorized_keys file > > 1 I don't think is likely I have not seen lots of failed logins to > root, over time yes but not in a small time frame, I block using > iptables and sec once a login has failed after 4 attempts in 60 > seconds and I have not seen a huge increase in log for failed root > logins. > > 2 Maybe, a bit difficult to determine, but will probably suggest a > wipe and reinstall of windows 7. This user always su's to do root > stuff. > > 3 Not sure how hard this would be, but leaning towards this being > the access point. The only login I have seen that is not from my ip > is the publickey access. > > What have I done? Made sure ssh access is only allowed to those > users that need it, which was the case anyway. Only allowed access > to root from my ip address since I am the only one that needs > direct root access. (Which was not the case) Removed publickey > access from sshd, now you have to use a password. > > Now just need to monitor and see what happens. Suppose I will have > to look at doing a reinstall, the website is still running and if I > can believe tcpdump/ps binarys etc there is nothing untoward > running. If I can believe the history file for root then nothing > was changed. The history file had the hackers commands interspersed > with mine so unless they where very clever I don't think it was > tampered with. Time to look at upgrading the server maybe and > seeing if I can do a deal with the hosting company to > change/upgrade... > It sounds like you got lucky with this to be honest - it was either a script kiddy who got lucky, or you're dealing with a pro and they're intentionally covering their tracks by feigning incompetence whilst maintaining their APT. Personally, I would still restore/rebuild but if you're not in a situation where you have a good image to flash and versioned backups to rebuild (*why* would you not have these for a production box though?) from I can see why you'd just cross your fingers and pray. Can I also assume that this a bare metal host, and not a VM? Bare metals are harder to rebuild for obvious reasons, which is why it shouldn't be on bare metal in the first place. You're very likely to be correct in your assumption that it was a PHP vulnerability - did you say you also have an un-updated Joomla install on the box? That's suicidal. The other option you haven't considered is good old fashioned SQL injection, especially considering the MySQL password was the same as the root password (WTF?). All in all, looking at the chain of disastrous, basic mistakes that are evident here it's a miracle nobody else got in first, and did a lot more damage. Or maybe they did: if you weren't systematically tripwiring the box's binaries with checksums, looking for a trojaned file across the entire OS is going to be a big problem. You also *really* shouldn't be relying on your logs for anything after someone got root: invoking vi, dropping into editor mode and issuing arbitrary commands is known as a shell escape because it's not logged in the usual way (done right, it's not logged at all) and that's just one of several elementary post-exploit obfuscation techniques I know off the top of my head. There are just so many ridiculous mistakes being made here that if this had happened at one of my clients, I would be advising the boss to discipline/fire outright the admin(s) responsible. SSH as root is NEVER allowable, for any reason, ever. The fact that it's still on and you've reverted to password auth instead of passphrase-protected keys makes my head hurt. Slapping band-aids like IP filtering (because nobody has ever spoofed IPs through infrastructures they've previously rooted...) on the gaping remote-root exploit this server has just suffered is an exercise in pure failure. Now I don't intend to sound mean intentionally, and it sounds like there are a fair few things at play here that you don't have full control over either so it's not like you are entirely to blame but my god man, someone really, really needs to sort their act out. I do genuinely wish you the best of luck in locking the server down and staying unhacked for the future though. You're just not making it easy for yourself. Regards -- The Mailing List for the Devon & Cornwall LUG http://mailman.dclug.org.uk/listinfo/list FAQ: http://www.dcglug.org.uk/listfaq