Page 1 of 1

MemcachedConnection makes my zimbra server slowly

Posted: Thu May 17, 2018 3:33 am
by hoanglv
Hi,

after i blocked memcache by this guide

https://wiki.zimbra.com/wiki/Blocking_Memcached_Attack

i entered 2 commands:

su - zimbra
/opt/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedBindAddress 127.0.0.1
/opt/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedClientServerList 127.0.0.1

after that my zimbra server log said:

2018-05-17 10:06:40,568 WARN [Memcached IO over {MemcachedConnection to /127.0.0.1:11211}] [] MemcachedConnection - Closing, and reopening {QA sa=/127.0.0.1:11211, #Rops=0, #Wops=43949, #iq=0, topRop=null, topWop=net.spy.memcached.protocol.ascii.StoreOperationImpl@619a3efd, toWrite=0, interested=0}, attempt 8551.

2018-05-17 10:06:53,065 WARN [ImapServer-200] [name=customerservice@mydomain.com;mid=58;ip=xxx.xxx.xxx.xxx;ua=Microsoft Outlook/16.0.4266.1001;] MemcachedConnection - Could not redistribute to another node, retrying primary node for zmImap:3d34c5d7-2711-4545-8440-a264a3f880aa:10674:64533:51742.

my zimbra server is too slow after that

Image

any one has this problem ?

have any solutions ?

thanks so much.

Re: MemcachedConnection makes my zimbra server slowly

Posted: Thu May 31, 2018 8:09 am
by kleversoft
Hello same problem, can someone help us with some solution?

Thank you

Re: MemcachedConnection makes my zimbra server slowly

Posted: Thu May 31, 2018 6:40 pm
by L. Mark Stone
https://wiki.zimbra.com/wiki/Ports

https://wiki.zimbra.com/wiki/Blocking_Memcached_Attack

Essentially, it sounds like you need to configure your firewall so that only Zimbra's intended public-facing ports are available to the public Internet. memcached's port is not one of those ports.

Hope that helps,
Mark

Re: MemcachedConnection makes my zimbra server slowly

Posted: Fri Jun 01, 2018 11:45 am
by kleversoft
Hello, someone can help us with some more info, I can't solve the problem.... thank you very much.


[name=main@xxxx.com;ip=163.172.201.150;oip=80.156.160.86;via=162.156.134.150(nginx/1.2.0-zimbra);ua=iPhone Mail/15E216;] misc - memcached asyncGet timed out after 10000ms
net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: /127.0.0.1:11211
at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:65)
at net.spy.memcached.internal.GetFuture.get(GetFuture.java:37)
at com.zimbra.common.util.memcached.ZimbraMemcachedClient.get(ZimbraMemcachedClient.java:281)
at com.zimbra.common.util.memcached.ZimbraMemcachedClient.get(ZimbraMemcachedClient.java:261)
at com.zimbra.common.util.memcached.MemcachedMap.get(MemcachedMap.java:98)
at com.zimbra.cs.imap.MemcachedImapCache.get(MemcachedImapCache.java:58)
at com.zimbra.cs.imap.ImapSessionManager.deserialize(ImapSessionManager.java:634)
at com.zimbra.cs.imap.ImapSession.reload(ImapSession.java:355)
at com.zimbra.cs.imap.ImapSession.getImapFolder(ImapSession.java:96)
at com.zimbra.cs.imap.ImapSessionManager.duplicateExistingSession(ImapSessionManager.java:390)
at com.zimbra.cs.imap.ImapSessionManager.openFolder(ImapSessionManager.java:285)
at com.zimbra.cs.imap.ImapHandler.setSelectedFolder(ImapHandler.java:1028)
at com.zimbra.cs.imap.ImapHandler.selectFolder(ImapHandler.java:1489)
at com.zimbra.cs.imap.ImapHandler.doSELECT(ImapHandler.java:1453)
at com.zimbra.cs.imap.ImapHandler.executeRequest(ImapHandler.java:722)
at com.zimbra.cs.imap.NioImapHandler.processRequest(NioImapHandler.java:126)
at com.zimbra.cs.imap.NioImapHandler.messageReceived(NioImapHandler.java:63)
at com.zimbra.cs.server.NioHandlerDispatcher.messageReceived(NioHandlerDispatcher.java:95)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:716)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:796)
at org.apache.mina.filter.codec.ProtocolCodecFilter$ProtocolDecoderOutputImpl.flush(ProtocolCodecFilter.java:427)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:245)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:796)
at com.zimbra.cs.server.NioLoggingFilter.messageReceived(NioLoggingFilter.java:62)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:796)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:75)
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:780)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:772)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:714)
at java.lang.Thread.run(Thread.java:745)

Re: MemcachedConnection makes my zimbra server slowly

Posted: Fri Jun 01, 2018 12:29 pm
by phoenix
Single server or a multi-server set-up? If it's a multi server then you've used the wrong commands, read the article again for the multi-server config and revert the changes you've made.

Re: MemcachedConnection makes my zimbra server slowly

Posted: Fri Jun 01, 2018 11:24 pm
by kleversoft
is single server, not memcached installed.

Re: MemcachedConnection makes my zimbra server slowly

Posted: Sat Jun 02, 2018 5:02 am
by phoenix
kleversoft wrote:is single server, not memcached installed.
You've posted this in a thread about memcached causing a problem on a server. You tell us that you've got 'the same problem', now you tell us that it's not installed and you seem surprised that your ZCS server can't connect to it - that's what's causing your problem. You problem is not the same as the original post. Why is memcached not installed and why do you think it benefits you to not have it installed? I'd suggest you install it, are there any other components that you've not installed (or that have been removed)?

Re: MemcachedConnection makes my zimbra server slowly

Posted: Sat Jun 02, 2018 7:36 am
by kleversoft
Hi, I have the same problem, execute this command by mistake

su - zimbra
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedBindAddress 127.0.0.1
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedClientServerList 127.0.0.1

in this zimbra 8.6 I had no memcached installed.

from that moment on zimbra looks for the memcached, and the problem is that I don't know how to deactivate it.

Re: MemcachedConnection makes my zimbra server slowly

Posted: Sat Jun 02, 2018 8:21 am
by phoenix
kleversoft wrote:Hi, I have the same problem,
You do not have the same problem as the o/p, he actually had memcached installed.
kleversoft wrote:execute this command by mistake
Why did you make those changes when you don't have memcached installed, that wasn't a mistake it was a deliberate act ion on your part - what were you trying to achieve?
kleversoft wrote: in this zimbra 8.6 I had no memcached installed.
You should always post details of your current ZCS version by posing the full output of the following command:

Code: Select all

zmcontrol -v
In addition, you should actually give a full description of the problem and not a 'me too' post, it would help us to help you when we know the full facts in advance - we're not mind readers.
kleversoft wrote:from that moment on zimbra looks for the memcached, and the problem is that I don't know how to deactivate it.
My additional question was why didn't you install memcached when you installed ZCS?

The version of ZCS that you're using will be EOL later this year you will need upgrade to the most recent version. In the current version of ZCS the proxy & memcached are required to be installed and I'd suggest you install those now in preparation for an upgrade, see the following article: https://wiki.zimbra.com/wiki/Enabling_Z ... _memcached

It's worth noting that you should not blindly make changes to the ZCS LDAP objects without knowing what (or why) you're doing it. Before making any changes to ZCS you should look at the current contents of an object, make a note of it just in case you have a problem so you can revert to the original settings then you won't be caught in this situation again.

Re: MemcachedConnection makes my zimbra server slowly

Posted: Sat Jun 02, 2018 10:03 pm
by vavai
kleversoft wrote:Hi, I have the same problem, execute this command by mistake

su - zimbra
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedBindAddress 127.0.0.1
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedClientServerList 127.0.0.1

in this zimbra 8.6 I had no memcached installed.

from that moment on zimbra looks for the memcached, and the problem is that I don't know how to deactivate it.
You can check and also disabling memcached port with following command :

Code: Select all

su - zimbra
zmprov gs `zmhostname` zimbraMemcachedBindAddress
zmprov ms `zmhostname` zimbraMemcachedBindAddress ''
zmprov gs `zmhostname` zimbraMemcachedClientServerList
zmprov ms `zmhostname` zimbraMemcachedClientServerList ''