[{ALLOW view All}]
[{ALLOW edit Authenticated}]

On 16.03.2017 my server was hacked for 2 days and created 632 Gb traffic. Luckily netcup does not charge for traffic.\\
The reason was that I had two security leaks, I guess it was "DirtyCow", but the Struts framework was also vulnerable.

What I did:\\

1. The root password was changed > reset a new via the netcup console\\
2. Updated [struts|https://struts.apache.org/download.cgi#struts25101] libraries in two web applications\\
3. updated Ubuntu to 3.13.0-112 to, see [DigitalOcean|https://www.digitalocean.com/community/tutorials/how-to-protect-your-server-against-the-dirty-cow-linux-vulnerability]
{{{
apt-get update
apt-get upgrade
}}}
Note: if upgrade fails due to disk capacity, delete old update files in /boot\\

4. Analysis, searched for changed files the last two days / last 60 min:
{{{
find . -mtime -2
find . -mmin -60
}}}
5. found user "cloudy_root", removed 
{{{
userdel -r cloudy_root
}}}
6. found changing init.d processes, removed like

{{{
update-rc.d -f lmpjqnxuao remove
}}}
7. found changing executables in /usr/bin, removed, like
{{{
rm lmpjqnxuao 
}}}
8. the processes reinstalled itself, so I blocked traffic, which came from IP 198.*, 69.*, 204.*, 46.*
{{{
netstat -pant

iptables -I INPUT -s 198.0.0.0/8 -j DROP
iptables -I INPUT -s 69.0.0.0/8 -j DROP
iptables -I INPUT -s 204.0.0.0/8 -j DROP
iptables -I INPUT -s 46.0.0.0/8 -j DROP
iptables -I OUTPUT -s 198.0.0.0/8 -j DROP
iptables -I OUTPUT -s 69.0.0.0/8 -j DROP
iptables -I OUTPUT -s 204.0.0.0/8 -j DROP
iptables -I OUTPUT -s 46.0.0.0/8 -j DROP
}}}
9. Finally found cron file, which was responsible for that. It used a lib, so this was suspicious
and it recreated itself again and again with differnt pids
{{{
ps -ef | grep -i libudev.so.6
vi crontab
rm /etc/cron.daily/gcc.sh
}}}
10. Restarted by the way again and again in between

Other useful commands:
{{{
who -a
ls -lat
/etc/init.d/collectl start
collectl -p /var/log/collectl/filename --top 
collectl --top
skill -KILL -u <username>
locate acdnfhruvx
}}}