SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

General discussion about Zimbra Desktop.
why_this
Posts: 4
Joined: Mon Jan 19, 2015 4:27 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by why_this »

Hello,

I just got a mail from the German CERT-Bund forwarded by my provider:
[CERT-Bund#2015011528000000]

Sehr geehrte Damen und Herren,

Redis ist eine In-Memory-Datenbank mit einer einfachen Schlüssel-Werte-
Datenstruktur. memcached ist ein Cache-Server zum einfachen Hinterlegen
und Abholen von Daten aus dem Arbeitsspeicher. Redis und memcached werden
häufig in Verbindung mit Web-Applikationen eingesetzt.

CERT-Bund hat von einer externen vertrauenswürdigen Quelle Informationen
zu Redis- und memcached-Installationen in Deutschland erhalten, welche
ohne Zugriffsschutz öffentlich aus dem Internet erreichbar sind.
Dies ermöglicht Angreifern potenziell das Ausspähen von Informationen
auf den betroffenen Systemen wie bspw. Zugangsdaten zu Webapplikationen
oder andere vertrauliche Inhalte.

Nachfolgend senden wir Ihnen eine Liste betroffener Systeme in Ihrem
Netzbereich. Neben der IP-Adresse enthalten die Daten einen Zeitstempel,
zu dem die offenen Systeme identifiziert wurden, sowie die Port- und
Versionsnummer.

Wir möchten Sie bitten, den Sachverhalt zu prüfen und Maßnahmen zur
Absicherung der Systeme zu ergreifen bzw. Ihre Kunden entsprechend
zu informieren.

Diese E-Mail ist mittels PGP digital signiert. Informationen zu dem
verwendeten Schlüssel finden Sie auf unserer Webseite unter:
<https://www.cert-bund.de/reports-sig>

Bitte beachten Sie:
Dies ist eine automatisch generierte Nachricht.
An die Absenderadresse kann nicht geantwortet werden.
Bei Rückfragen wenden Sie sich bitte an <certbund@bsi.bund.de>.

Liste der betroffenen Systeme in Ihrem Netzbereich:

Format: ASN | IP-Adresse | Zeitstempel | Software | Port | Version
 ASN | IP  | 2015-01-09 02:16:54 +0100 | Memcached | 11211 | STAT version 1.4.17

Mit freundlichen Grüßen
Team CERT-Bund

Bundesamt für Sicherheit in der Informationstechnik (BSI)
Referat C21 - CERT-Bund
Godesberger Allee 185-189
D-53175 Bonn



It says, that there is a memcached reachable from the internet on this IP!
I checked, and indeed there was memcached running on Zimbra! Zimbra was installed as Version 7.?? and updated. It is Version 8.6.0 now!
Since it is a standalone server, it seems like memcached was not used. Still, it is a huge security risk to have that open and should be changed!
I was able to disable it using:

zmmemcachedctl stop
zmprov ms `zmhostname` -zimbraServiceEnabled memcached
zmcontrol stop
zmcontrol start


I have a second server running Zimbra and encountered exactly the same problem!
I don't see why memcached should be installed on a standalone instance at all (maybe that was my mistake and I selected it) but in any case, it should definitely NOT listen on a public address!

CarstenK.
Posts: 7
Joined: Fri Aug 22, 2014 5:07 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by CarstenK. »

Hi,
i don't have a solution, but I became this mail - for every of my servers - too (Zimbra 8.6).

In nginx-config IP and Port is configurable.
/opt/zimbra/conf/nginx/includes/nginx.conf.memcache

Is it possible to bind memcache to localhost?
anolting
Posts: 4
Joined: Mon Jan 19, 2015 8:08 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by anolting »

I change servers to 127.0.0.1 in /conf/nginx/includes/nginx.conf.memcache and restarted memcached. Seems to be fixed now.
CarstenK.
Posts: 7
Joined: Fri Aug 22, 2014 5:07 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by CarstenK. »

Ok, i think that this can't fix the problem, because it is only the proxy configuration.

It would be necessary to fix the configuration of memcache.



Can you show the output of:

netstat -tulpen | grep mem
User avatar
tonster
Zimbra Employee
Zimbra Employee
Posts: 313
Joined: Fri Feb 21, 2014 10:14 am
Location: Ypsilanti, MI
ZCS/ZD Version: Release 8.7.0_GA_1659.RHEL6_64_2016

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by tonster »

This is not a Zimbra problem. memcached needs to bind to routable interfaces in order to communicate with other memcached servers in your environment. You should ensure that any service you install is appropriately firewalled to prevent unwanted access. A list of all Zimbra-related ports is available at https://wiki.zimbra.com/wiki/Ports. By default, other ports are also available that should be firewalled or restricted, such as port 7071 (which is the admin console). LDAP is also another common port that should be firewalled off in most environments.
anolting
Posts: 4
Joined: Mon Jan 19, 2015 8:08 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by anolting »

zimbra@s***:~/conf/nginx/includes$ netstat -ap | grep mem*

(Not all processes could be identified, non-owned process info

will not be shown, you would have to be root to see it all.)

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name

tcp 0 0 *:11211 *:* LISTEN 16361/memcached

tcp 0 0 s***.a***.**:11211 s***.a***.**:35185 ESTABLISHED 16361/memcached

tcp 0 0 s***.a***.**:11211 s***.a***.**:35183 ESTABLISHED 16361/memcached

tcp 0 0 s***.a***.**:11211 s***.a***.**:35184 ESTABLISHED 16361/memcached

tcp 0 0 s***.a***.**:11211 s***.a***.**:35186 ESTABLISHED 16361/memcached

tcp6 0 0 [::]:11211 [::]:* LISTEN 16361/memcached

udp 0 0 *:11211 *:* 16361/memcached

udp6 0 0 [::]:11211 [::]:* 16361/memcached

Proto RefCnt Flags Type State I-Node PID/Program name Path

unix 3 [ ] STREAM CONNECTED 16389472 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389482 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389473 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389478 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389469 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389481 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389479 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389475 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389470 16361/memcached

unix 3 [ ] STREAM CONNECTED 16389476 16361/memcached

zimbra@s***:~/conf/nginx/includes$
anolting
Posts: 4
Joined: Mon Jan 19, 2015 8:08 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by anolting »

In case of firewalling your are right. There was a lack of configuration.

Will assess this again.
mo23
Posts: 3
Joined: Mon Jan 19, 2015 9:10 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by mo23 »

Firewalling is one possibility, but why not use configuration parameter "-l 127.0.0.1" given by memcached? this would make the deamon listening localhost and done!



/opt/zimbra/memcached/share/man/man1/memcached.1



-l <ip_addr>

Listen on <ip_addr>; default to INADDR_ANY. This is an important option to

consider as there is no other way to secure the installation. Binding to an

internal or firewalled network interface is suggested.



There is some kind of configfile=${zimbra_home}/conf/${servicename}.conf in /opt/zimbra/bin/zmmemcachedctl ... but no reference on it
User avatar
jorgedlcruz
Zimbra Alumni
Zimbra Alumni
Posts: 2782
Joined: Thu May 22, 2014 4:47 pm

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by jorgedlcruz »

Hi mo23,

If you have Single Server, you can do it if you want, but is not recommended at all if you have Multi Server.



The best practice is protect the port with Firewall, iptables or just a Network Firewall.



Best regards
Jorge de la Cruz https://jorgedelacruz.es
Systems Engineer at Veeam Software https://www.veeam.com/
mo23
Posts: 3
Joined: Mon Jan 19, 2015 9:10 am

SECURITY PROBLEM: Open Memcached in Zimbra 8.6.0_GA_1153

Post by mo23 »

Hi Jorge,

using a multiserver setup you are right ... but this problem was based on single server setups and it would be nice to use given configuration parameter of used services like memcached.



Kind regards

Moritz
Post Reply