MemcachedConnection makes my zimbra server slowly

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Post Reply
hoanglv
Posts: 3
Joined: Thu May 17, 2018 3:20 am

MemcachedConnection makes my zimbra server slowly

Post by hoanglv »

Hi,

after i blocked memcache by this guide

https://wiki.zimbra.com/wiki/Blocking_Memcached_Attack

i entered 2 commands:

su - zimbra
/opt/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedBindAddress 127.0.0.1
/opt/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedClientServerList 127.0.0.1

after that my zimbra server log said:

2018-05-17 10:06:40,568 WARN [Memcached IO over {MemcachedConnection to /127.0.0.1:11211}] [] MemcachedConnection - Closing, and reopening {QA sa=/127.0.0.1:11211, #Rops=0, #Wops=43949, #iq=0, topRop=null, topWop=net.spy.memcached.protocol.ascii.StoreOperationImpl@619a3efd, toWrite=0, interested=0}, attempt 8551.

2018-05-17 10:06:53,065 WARN [ImapServer-200] [name=customerservice@mydomain.com;mid=58;ip=xxx.xxx.xxx.xxx;ua=Microsoft Outlook/16.0.4266.1001;] MemcachedConnection - Could not redistribute to another node, retrying primary node for zmImap:3d34c5d7-2711-4545-8440-a264a3f880aa:10674:64533:51742.

my zimbra server is too slow after that

Image

any one has this problem ?

have any solutions ?

thanks so much.
kleversoft
Posts: 5
Joined: Fri Sep 29, 2017 11:55 am

Re: MemcachedConnection makes my zimbra server slowly

Post by kleversoft »

Hello same problem, can someone help us with some solution?

Thank you
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2796
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.6 Network Edition
Contact:

Re: MemcachedConnection makes my zimbra server slowly

Post by L. Mark Stone »

https://wiki.zimbra.com/wiki/Ports

https://wiki.zimbra.com/wiki/Blocking_Memcached_Attack

Essentially, it sounds like you need to configure your firewall so that only Zimbra's intended public-facing ports are available to the public Internet. memcached's port is not one of those ports.

Hope that helps,
Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
kleversoft
Posts: 5
Joined: Fri Sep 29, 2017 11:55 am

Re: MemcachedConnection makes my zimbra server slowly

Post by kleversoft »

Hello, someone can help us with some more info, I can't solve the problem.... thank you very much.


[name=main@xxxx.com;ip=163.172.201.150;oip=80.156.160.86;via=162.156.134.150(nginx/1.2.0-zimbra);ua=iPhone Mail/15E216;] misc - memcached asyncGet timed out after 10000ms
net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: /127.0.0.1:11211
at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:65)
at net.spy.memcached.internal.GetFuture.get(GetFuture.java:37)
at com.zimbra.common.util.memcached.ZimbraMemcachedClient.get(ZimbraMemcachedClient.java:281)
at com.zimbra.common.util.memcached.ZimbraMemcachedClient.get(ZimbraMemcachedClient.java:261)
at com.zimbra.common.util.memcached.MemcachedMap.get(MemcachedMap.java:98)
at com.zimbra.cs.imap.MemcachedImapCache.get(MemcachedImapCache.java:58)
at com.zimbra.cs.imap.ImapSessionManager.deserialize(ImapSessionManager.java:634)
at com.zimbra.cs.imap.ImapSession.reload(ImapSession.java:355)
at com.zimbra.cs.imap.ImapSession.getImapFolder(ImapSession.java:96)
at com.zimbra.cs.imap.ImapSessionManager.duplicateExistingSession(ImapSessionManager.java:390)
at com.zimbra.cs.imap.ImapSessionManager.openFolder(ImapSessionManager.java:285)
at com.zimbra.cs.imap.ImapHandler.setSelectedFolder(ImapHandler.java:1028)
at com.zimbra.cs.imap.ImapHandler.selectFolder(ImapHandler.java:1489)
at com.zimbra.cs.imap.ImapHandler.doSELECT(ImapHandler.java:1453)
at com.zimbra.cs.imap.ImapHandler.executeRequest(ImapHandler.java:722)
at com.zimbra.cs.imap.NioImapHandler.processRequest(NioImapHandler.java:126)
at com.zimbra.cs.imap.NioImapHandler.messageReceived(NioImapHandler.java:63)
at com.zimbra.cs.server.NioHandlerDispatcher.messageReceived(NioHandlerDispatcher.java:95)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:716)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:796)
at org.apache.mina.filter.codec.ProtocolCodecFilter$ProtocolDecoderOutputImpl.flush(ProtocolCodecFilter.java:427)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:245)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:796)
at com.zimbra.cs.server.NioLoggingFilter.messageReceived(NioLoggingFilter.java:62)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:796)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:75)
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:780)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:772)
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:714)
at java.lang.Thread.run(Thread.java:745)
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Re: MemcachedConnection makes my zimbra server slowly

Post by phoenix »

Single server or a multi-server set-up? If it's a multi server then you've used the wrong commands, read the article again for the multi-server config and revert the changes you've made.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
kleversoft
Posts: 5
Joined: Fri Sep 29, 2017 11:55 am

Re: MemcachedConnection makes my zimbra server slowly

Post by kleversoft »

is single server, not memcached installed.
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Re: MemcachedConnection makes my zimbra server slowly

Post by phoenix »

kleversoft wrote:is single server, not memcached installed.
You've posted this in a thread about memcached causing a problem on a server. You tell us that you've got 'the same problem', now you tell us that it's not installed and you seem surprised that your ZCS server can't connect to it - that's what's causing your problem. You problem is not the same as the original post. Why is memcached not installed and why do you think it benefits you to not have it installed? I'd suggest you install it, are there any other components that you've not installed (or that have been removed)?
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
kleversoft
Posts: 5
Joined: Fri Sep 29, 2017 11:55 am

Re: MemcachedConnection makes my zimbra server slowly

Post by kleversoft »

Hi, I have the same problem, execute this command by mistake

su - zimbra
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedBindAddress 127.0.0.1
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedClientServerList 127.0.0.1

in this zimbra 8.6 I had no memcached installed.

from that moment on zimbra looks for the memcached, and the problem is that I don't know how to deactivate it.
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Re: MemcachedConnection makes my zimbra server slowly

Post by phoenix »

kleversoft wrote:Hi, I have the same problem,
You do not have the same problem as the o/p, he actually had memcached installed.
kleversoft wrote:execute this command by mistake
Why did you make those changes when you don't have memcached installed, that wasn't a mistake it was a deliberate act ion on your part - what were you trying to achieve?
kleversoft wrote: in this zimbra 8.6 I had no memcached installed.
You should always post details of your current ZCS version by posing the full output of the following command:

Code: Select all

zmcontrol -v
In addition, you should actually give a full description of the problem and not a 'me too' post, it would help us to help you when we know the full facts in advance - we're not mind readers.
kleversoft wrote:from that moment on zimbra looks for the memcached, and the problem is that I don't know how to deactivate it.
My additional question was why didn't you install memcached when you installed ZCS?

The version of ZCS that you're using will be EOL later this year you will need upgrade to the most recent version. In the current version of ZCS the proxy & memcached are required to be installed and I'd suggest you install those now in preparation for an upgrade, see the following article: https://wiki.zimbra.com/wiki/Enabling_Z ... _memcached

It's worth noting that you should not blindly make changes to the ZCS LDAP objects without knowing what (or why) you're doing it. Before making any changes to ZCS you should look at the current contents of an object, make a note of it just in case you have a problem so you can revert to the original settings then you won't be caught in this situation again.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
User avatar
vavai
Advanced member
Advanced member
Posts: 174
Joined: Thu Nov 14, 2013 2:41 pm
Location: Indonesia
ZCS/ZD Version: 0
Contact:

Re: MemcachedConnection makes my zimbra server slowly

Post by vavai »

kleversoft wrote:Hi, I have the same problem, execute this command by mistake

su - zimbra
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedBindAddress 127.0.0.1
/opt/zimbra/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedClientServerList 127.0.0.1

in this zimbra 8.6 I had no memcached installed.

from that moment on zimbra looks for the memcached, and the problem is that I don't know how to deactivate it.
You can check and also disabling memcached port with following command :

Code: Select all

su - zimbra
zmprov gs `zmhostname` zimbraMemcachedBindAddress
zmprov ms `zmhostname` zimbraMemcachedBindAddress ''
zmprov gs `zmhostname` zimbraMemcachedClientServerList
zmprov ms `zmhostname` zimbraMemcachedClientServerList ''
Post Reply