Seemingly untraceable login

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
chauvetp
Outstanding Member
Outstanding Member
Posts: 350
Joined: Fri Sep 12, 2014 11:28 pm

Seemingly untraceable login

Postby chauvetp » Mon May 23, 2016 3:31 pm

Hello,

We've been having some brute force login attempts (where the attackers appear to be trying 20 or so attempts for a user before moving on). Normally, these are easy to trace for me, but I cannot find the source of these. There is no oip in the log entries, just the ip of one of our Zimbra proxies. I don't know what I'm missing, or how these attempts are even coming through. Any advice on where to look here would be appreciated!


Here's an example below (I've left out the java stack trace for the failed logins). The IPs and usernames have been redacted.

mailbox.log on one of the Zimbra stores:
2016-05-22 13:00:02,046 WARN [qtp1937601231-4651783:https://ZSTORE_IP:7071/service/admin/soap/] [name=USER@OURDOMAIN;ip=PROXY_IP;] account - ldap auth for domain OURDOMAIN failed, fall back to zimbra default auth mechanism
com.zimbra.cs.account.AccountServiceException$AuthFailedServiceException: authentication failed for [USER@OURDOMAIN]
ExceptionId:qtp1937601231-4651783:https://ZSTORE_IP:7071/service/admin/soap/:1463936402046:531d9a48b8676902

2016-05-22 13:00:02,050 INFO [qtp1937601231-4651783:https://ZSTORE_IP:7071/service/admin/soap/] [name=USER@OURDOMAIN;ip=PROXY_IP;] SoapEngine - handler exception: authentication failed for [USER@OURDOMAIN], invalid password
2016-05-22 13:00:02,050 INFO [qtp1937601231-4651783:https://ZSTORE_IP:7071/service/admin/soap/] [name=USER@OURDOMAIN;ip=PROXY_IP;] soap - AuthRequest elapsed=148
2016-05-22 13:00:02,663 WARN [qtp1937601231-4651813:https://ZSTORE_IP:7071/service/admin/soap/] [name=USER@OURDOMAIN;ip=PROXY_IP;] account - ldap auth for domain OURDOMAIN failed, fall back to zimbra default auth mechanism
com.zimbra.cs.account.AccountServiceException$AuthFailedServiceException: authentication failed for [USER@OURDOMAIN]
ExceptionId:qtp1937601231-4651813:https://ZSTORE_IP:7071/service/admin/soap/:1463936402663:531d9a48b8676902

Code:account.AUTH_FAILED

zimbra.log on one of the proxies:
May 22 12:59:59 zproxy2 saslauthd[11500]: auth_zimbra: USER@OURDOMAIN auth failed: authentication failed for [USER@OURDOMAIN]
May 22 12:59:59 zproxy2 saslauthd[11500]: do_auth : auth failure: [user=USER@OURDOMAIN] [service=smtp] [realm=OURDOMAIN] [mech=zimbra] [reason=Unknown]
May 22 13:00:00 zproxy2 saslauthd[11501]: zmpost: url='https://ZSTORE_HOST:7071/service/admin/soap/' returned buffer->data='<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"><soap:Header><context xmlns="urn:zimbra"/></soap:Header><soap:Body><soap:Fault><soap:Code><soap:Value>soap:Sender</soap:Value></soap:Code><soap:Reason><soap:Text>authentication failed for [USER@OURDOMAIN]</soap:Text></soap:Reason><soap:Detail><Error xmlns="urn:zimbra"><Code>account.AUTH_FAILED</Code><Trace>qtp821866309-3692748:https://ZSTORE_IP:7071/service/admin/soap/:1463936400366:71b4000474b86f75</Trace></Error></soap:Detail></soap:Fault></soap:Body></soap:Envelope>', hti->error=''
May 22 13:00:00 zproxy2 saslauthd[11501]: auth_zimbra: USER@OURDOMAIN auth failed: authentication failed for [USER@OURDOMAIN]
May 22 13:00:00 zproxy2 saslauthd[11501]: do_auth : auth failure: [user=USER@OURDOMAIN] [service=smtp] [realm=OURDOMAIN] [mech=zimbra] [reason=Unknown]
May 22 13:00:00 zproxy2 saslauthd[11496]: zmpost: url='https://ZSTORE_HOST:7071/service/admin/soap/' returned buffer->data='<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"><soap:Header><context xmlns="urn:zimbra"/></soap:Header><soap:Body><soap:Fault><soap:Code><soap:Value>soap:Sender</soap:Value></soap:Code><soap:Reason><soap:Text>authentication failed for [USER@OURDOMAIN]</soap:Text></soap:Reason><soap:Detail><Error xmlns="urn:zimbra"><Code>account.AUTH_FAILED</Code><Trace>qtp777206150-5193478:https://ZSTORE_IP:7071/service/admin/soap/:1463936400867:525dc054ceb2ff22</Trace></Error></soap:Detail></soap:Fault></soap:Body></soap:Envelope>', hti->error=''


audit.log on one of the mailbox servers:
2016-05-22 13:00:02,049 WARN [qtp1937601231-4651783:https://ZSTORE_IP:7071/service/admin/soap/] [name=USER@OURDOMAIN;ip=PROXY_IP;] security - cmd=Auth; account=USER@OURDOMAIN; protocol=soap; error=authentication failed for [USER@OURDOMAIN], invalid password;
2016-05-22 13:00:02,665 WARN [qtp1937601231-4651813:https://ZSTORE_IP:7071/service/admin/soap/] [name=USER@OURDOMAIN;ip=PROXY_IP;] security - cmd=Auth; account=USER@OURDOMAIN; protocol=soap; error=authentication failed for [USER@OURDOMAIN], invalid password;



I don't see anything else of use in the nginx logs on the proxies or the maillog file on the MTAs.


liverpoolfcfan
Outstanding Member
Outstanding Member
Posts: 926
Joined: Sat Sep 13, 2014 12:47 am

Re: Seemingly untraceable login

Postby liverpoolfcfan » Mon May 23, 2016 8:18 pm

Check out this document. It should help you. https://wiki.zimbra.com/wiki/Log_Files#Logging_the_Originating_IP
chauvetp
Outstanding Member
Outstanding Member
Posts: 350
Joined: Fri Sep 12, 2014 11:28 pm

Re: Seemingly untraceable login

Postby chauvetp » Mon May 23, 2016 8:47 pm

Hi liverpoolfcfan,

Thanks for the link. That's the odd thing. We almost always have the originating IP.

It's there for every session I can specifically identify as IMAP, ActiveSync, SMTP, and the Zimbra Web Client. Access to the admin port (7071) is only allowed from the other Zimbra servers and a very small number of internal IPs.

To confirm the settings in that link, I've double checked the following:
- the zimbra_http_originating_ip_header is set on all proxy servers.
- all our mail stores have the zimbraMailTrustedIP set with the proxies.

It just seems that these weird connections are somehow bypassing this (or accessing some service that doesn't process/pass along the header).
liverpoolfcfan
Outstanding Member
Outstanding Member
Posts: 926
Joined: Sat Sep 13, 2014 12:47 am

Re: Seemingly untraceable login

Postby liverpoolfcfan » Mon May 23, 2016 9:14 pm

In that case it is probably the smtpd submission port.

Look in the zimbra.log file for one of the failures, then look back a few lines and you should find connection messages similar to this

May 23 12:33:24 mail postfix/submission/smtpd[5884]: connect from mail-vk0-f49.google.com[209.85.213.49]
May 23 12:33:25 mail postfix/submission/smtpd[5884]: Anonymous TLS connection established from mail-vk0-f49.google.com[209.85.213.49]: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
chauvetp
Outstanding Member
Outstanding Member
Posts: 350
Joined: Fri Sep 12, 2014 11:28 pm

Re: Seemingly untraceable login

Postby chauvetp » Tue May 24, 2016 1:43 pm

Thanks! That was what I was missing. It seems a bit difficult to track though. Is there a way to match the saslauthd message with the postfix/smtps/smtpd message? Or is it just a matter of proximity to the logs?

Alternatively, is there a way to log (in /var/log/maillog) the username that failed authentication instead of just the following?

May 22 13:00:00 zproxy1 postfix/smtps/smtpd[871]: warning: SASL authentication failure: Password verification failed
May 22 13:00:00 zproxy1 postfix/smtps/smtpd[871]: warning: unknown[46.161.40.200]: SASL PLAIN authentication failed: authentication failure



Either way, whatever is trying to login appears to be a botnet of some sort due to how much the IPs are changing.

Return to “Administrators”

Who is online

Users browsing this forum: MSN [Bot] and 8 guests