Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Ask questions about your setup or get help installing ZCS server (ZD section below).
User avatar
jorgedlcruz
Zimbra Employee
Zimbra Employee
Posts: 2769
Joined: Thu May 22, 2014 4:47 pm

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby jorgedlcruz » Mon Jan 29, 2018 5:56 pm

brf wrote:
justdave wrote:Been running into this, too, after upgrading from 8.6.x to 8.8.5. Happy to help debug if you tell me what to look for. I've got a small site with very few users so I can take it down on a whim to test stuff.

Support told me that engineering is already working on it, and they didn't seem to need/want any information from me other than my zmdiaglog to confirm that this was the issue I was experiencing.

The official workaround for now is to restart your server frequently...


Please let me know your Support Case, so I can ask to see how far we are from a hotfix or patch.

thank you!


Jorge de la Cruz https://jorgedelacruz.es
Technical Marketing Manager at Zimbra/Synacor https://www.zimbra.com/
brf
Posts: 30
Joined: Fri Sep 12, 2014 11:28 pm
ZCS/ZD Version: 8.8.6_GA_1906.RHEL7_64 NE

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby brf » Tue Jan 30, 2018 12:01 am

jorgedlcruz wrote:
brf wrote:Support told me that engineering is already working on it, and they didn't seem to need/want any information from me other than my zmdiaglog to confirm that this was the issue I was experiencing.

The official workaround for now is to restart your server frequently...


Please let me know your Support Case, so I can ask to see how far we are from a hotfix or patch.

thank you!

#00687508
JakeMS
Posts: 9
Joined: Tue Sep 09, 2014 4:16 pm

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby JakeMS » Tue Jan 30, 2018 7:00 am

I can confirm that this also occurs on CentOS 7 with 8.8.6 OSE :D.

Our server is dropping out every 4 to 6 hours with the same errors as OP. I don't think this is a load issue either. We've only got 14 accounts and the server is a dedicated server running an Intel Xeon E3-1245 V2 and 32GB of ram.

This server has been running just fine with 8.7 up until two days ago when it was upgraded to 8.8 to get the latest security patches. Seems it also got the latest bugs :lol:

For now I've set up a cronjob to reboot Zimbra every 3 hours. :).
_spOOn_
Posts: 1
Joined: Sun Feb 04, 2018 11:42 pm

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby _spOOn_ » Sun Feb 04, 2018 11:44 pm

Hi guys,

We seem to be experiencing the same issue, on CentOS Linux release 7.4.1708, running zcs-8.8.6_GA_1906

Is there any update on this one, as to when a fix might become available?

Cheers,
davidkillingsworth
Advanced member
Advanced member
Posts: 151
Joined: Sat Sep 13, 2014 2:26 am
ZCS/ZD Version: Release 8.8.6.GA.1906.UBUNTU14.64

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby davidkillingsworth » Mon Feb 05, 2018 5:37 am

I'm getting exactly the same issue as the other users.

The server will run for about 48 hours and then the web interface becomes too slow for the users to login and work.

What I am seeing in other discussion posts in this forum is that increasing the following variables works, but nobody says what to change to.

Code: Select all

zimbraHttpNumThreads: 250
zimbraImapNumThreads: 200
zimbraLmtpNumThreads: 20


Please advise if there is a bug report that we can follow, or if there are recommended values for the threads variables listed above that we should be using.
davidkillingsworth
Advanced member
Advanced member
Posts: 151
Joined: Sat Sep 13, 2014 2:26 am
ZCS/ZD Version: Release 8.8.6.GA.1906.UBUNTU14.64

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby davidkillingsworth » Tue Feb 06, 2018 2:56 am

We are an MSP that has been recommending and operating Zimbra servers for some of our customers for the past 7 years or so.

The fact that there is such a serious issue without so much as a bug report that we can follow, a member of the zimbra support team acknowledging this here on the forum, or anyone on Twitter replying, is a damn farce.

We have set a cron job to reboot our server every morning at 6AM in hopes to keep our users from having to experience the extreme slow down each morning until this is fixed.

Putting that kind of a band-aid solution on such a serious problem is not the kind of thing I want to be doing for something as serious as my customer's email servers.

Someone at Zimbra support really really needs to address this.
brf
Posts: 30
Joined: Fri Sep 12, 2014 11:28 pm
ZCS/ZD Version: 8.8.6_GA_1906.RHEL7_64 NE

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby brf » Tue Feb 06, 2018 3:08 am

Rebooting once a day isn't enough. I tried twice a day (7AM and 7PM) and that wasn't even enough, eventually it failed around 6PM one day. Now I'm doing three times a day and seems OK so far. Also, our server isn't even that big, about 60 mailboxes (although I'm not sure if this problem is traffic/usage-based).
User avatar
msquadrat
Advanced member
Advanced member
Posts: 132
Joined: Mon Oct 14, 2013 10:09 am

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby msquadrat » Tue Feb 06, 2018 8:51 am

I have an idea but first some questions about your setup and the issue you see:
  • I might have missed it but can you describe how your Zimbra cluster is set up? Most importantly, where is your LDAP master located?
  • Is the frequency you see by any chance high load every four hours, timed after the last restart of the LDAP service? Probably with low/no issues in off-hours?
  • Do you see high IO load on any host while this happens, especially the LDAP host?
brf
Posts: 30
Joined: Fri Sep 12, 2014 11:28 pm
ZCS/ZD Version: 8.8.6_GA_1906.RHEL7_64 NE

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby brf » Tue Feb 06, 2018 9:28 am

davidkillingsworth, from your description, you might be seeing a different problem than what this thread is about. I don't recall seeing high CPU usage or slowness when this problem happens, what I do see is many users unable to login and access email. In your mailbox.log, do you see tons of "Failed to lock" errors happening when this problem is going on? If not, then I believe your problem is something different.

There is another issue that we did a support case for recently, which does cause high CPU usage. If you're being affected by that one, you should see massive growth of your sync.log whenever the problem is happening with a lot of errors like "sync token too old" and "out of sync". It is caused by Windows Mail clients. The only fix is to have those clients delete and re-create their Windows Mail profile. Zimbra refuses to fix it on their end: https://bugzilla.zimbra.com/show_bug.cgi?id=107746
Unlike the first problem, this problem is not new to Zimbra 8.8.x though.
davidkillingsworth
Advanced member
Advanced member
Posts: 151
Joined: Sat Sep 13, 2014 2:26 am
ZCS/ZD Version: Release 8.8.6.GA.1906.UBUNTU14.64

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby davidkillingsworth » Tue Feb 06, 2018 9:31 am

brf wrote:davidkillingsworth, from your description, you might be seeing a different problem than what this thread is about. I don't recall seeing high CPU usage or slowness when this problem happens, what I do see is many users unable to login and access email. In your mailbox.log, do you see tons of "Failed to lock" errors happening when this problem is going on? If not, then I believe your problem is something different.


Yes, seeing lots of Failed to lock messages in mailbox.log. Mail delivery works fine, it's just that webmail users cannot login, or their functions within the browser are seriously too slow.

brf wrote:There is another issue that we did a support case for recently, which does cause high CPU usage. If you're being affected by that one, you should see massive growth of your sync.log whenever the problem is happening with a lot of errors like "sync token too old" and "out of sync". It is caused by Windows Mail clients. The only fix is to have those clients delete and re-create their Windows Mail profile. Zimbra refuses to fix it on their end: https://bugzilla.zimbra.com/show_bug.cgi?id=107746
Unlike the first problem, this problem is not new to Zimbra 8.8.x though.


We don't use Zimbra mobile sync feature in our environment. I don't think that applies to us. Our sync logs files are empty.

Return to “Installation and Upgrade”

Who is online

Users browsing this forum: No registered users and 5 guests