Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Ask questions about your setup or get help installing ZCS server (ZD section below).
mntwinsfan
Posts: 4
Joined: Tue Jan 02, 2018 12:15 am

Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby mntwinsfan » Tue Jan 02, 2018 12:33 am

I recently upgraded to Zimbra 8.8.5 from 8.7.6. Ever since I did, some users (not all) have been having issues getting into their mailbox from time to time. Through the web client, they get the login page, enter their credentials, and then the browser just spins. On their devices, it will eventually time out. If I restart Zimbra, it works fine for everyone for a while but then starts failing again. Never for all users, just a handful. I am seeing the following errors in the mailbox.log:

Code: Select all

2018-01-01 18:49:28,948 ERROR [LmtpServer-1] [name=user@domain.com;mid=16;ip=nn.nn.nn.nn;] mailbox - Failed to lock mailbox
Write Lock Owner - ImapSSLServer-1 prio=5 id=182 state=TERMINATED
Lock Waiter - Timer-Zimbra daemon prio=5 id=37 state=TIMED_WAITING
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
        at com.zimbra.cs.mailbox.MailboxLock.tryLockWithTimeout(MailboxLock.java:114)
        at com.zimbra.cs.mailbox.MailboxLock.lock(MailboxLock.java:194)
        at com.zimbra.cs.mailbox.Mailbox.lock(Mailbox.java:10411)
        at com.zimbra.cs.imap.ImapListener.unload(ImapListener.java:580)
        at com.zimbra.cs.imap.ImapSessionManager$SessionSerializerTask.run(ImapSessionManager.java:195)
        at java.util.TimerThread.mainLoop(Timer.java:555)
        at java.util.TimerThread.run(Timer.java:505)

com.zimbra.cs.mailbox.MailboxLock$LockFailedException: timeout
        at com.zimbra.cs.mailbox.MailboxLock.lock(MailboxLock.java:211)
        at com.zimbra.cs.mailbox.Mailbox.beginTransaction(Mailbox.java:1759)
        at com.zimbra.cs.mailbox.Mailbox.beginReadTransaction(Mailbox.java:1735)
        at com.zimbra.cs.mailbox.Mailbox.getItemById(Mailbox.java:2864)
        at com.zimbra.cs.mailbox.Mailbox.getItemById(Mailbox.java:2856)
        at com.zimbra.cs.mailbox.Mailbox.getFolderById(Mailbox.java:4092)
        at com.zimbra.cs.filter.IncomingMessageHandler.getDefaultFolderPath(IncomingMessageHandler.java:91)
        at com.zimbra.cs.filter.ZimbraMailAdapter.keep(ZimbraMailAdapter.java:526)
        at com.zimbra.cs.filter.ZimbraMailAdapter.executeActionKeepInternal(ZimbraMailAdapter.java:359)
        at com.zimbra.cs.filter.ZimbraMailAdapter.executeAllActions(ZimbraMailAdapter.java:294)
        at com.zimbra.cs.filter.RuleManager.applyRulesToIncomingMessage(RuleManager.java:440)
        at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliverMessageToLocalMailboxes(ZimbraLmtpBackend.java:615)
        at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver(ZimbraLmtpBackend.java:385)
        at com.zimbra.cs.lmtpserver.LmtpHandler.processMessageData(LmtpHandler.java:445)
        at com.zimbra.cs.lmtpserver.TcpLmtpHandler.continueDATA(TcpLmtpHandler.java:79)
        at com.zimbra.cs.lmtpserver.LmtpHandler.doDATA(LmtpHandler.java:434)
        at com.zimbra.cs.lmtpserver.LmtpHandler.processCommand(LmtpHandler.java:216)
        at com.zimbra.cs.lmtpserver.TcpLmtpHandler.processCommand(TcpLmtpHandler.java:72)
        at com.zimbra.cs.server.ProtocolHandler.processConnection(ProtocolHandler.java:189)
        at com.zimbra.cs.server.ProtocolHandler.run(ProtocolHandler.java:128)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-01-01 18:49:28,948 WARN  [LmtpServer-1] [name=user@domain.com;mid=16;ip=nn.nn.nn.nn;] mailbox - transaction canceled because of lock failure


Searching through these forums, I found some old posts regarding this error message which suggested changing some of the server settings. I did but they had no effect:

Code: Select all

zmlocalconfig -e zimbra_mailbox_lock_timeout=90
zmlocalconfig -e zimbra_mailbox_lock_max_waiting_threads=100
zmlocalconfig -e mailboxd_java_heap_size=4096
zmlocalconfig -e mailboxd_java_heap_memory_percent=25
zmprov ms `zmhostname` zimbraMessageCacheSize 4000
zmprov ms `zmhostname` zimbraImapNumThreads 50
zmprov ms `zmhostname` zimbraHttpNumThreads 600


Does anyone have any further suggestions or know how to fix this problem?


moebius
Posts: 17
Joined: Sat Sep 13, 2014 3:34 am

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby moebius » Mon Jan 08, 2018 5:18 pm

I am also experiencing mailbox locking issues since 8.8.5 and have to restart the mailboxd service to clear them. zmmailboxctl kill does not work, I have to kill the PID. Seems to pop up every 2 or 3 days.
User avatar
maxxer
Advanced member
Advanced member
Posts: 61
Joined: Fri Oct 04, 2013 2:12 am
Contact:

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby maxxer » Mon Jan 08, 2018 5:46 pm

Someone here also suggests increasing http and imap threads.

From other discussions I learnt it could be bond to IMAP usage, especially with Apple clients. Is this your case?
~ YetOpen S.r.l. ~
https://www.yetopen.it
moebius
Posts: 17
Joined: Sat Sep 13, 2014 3:34 am

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby moebius » Tue Jan 09, 2018 5:34 pm

It's certainly possible, but I don't know how to resolve it.
User avatar
maxxer
Advanced member
Advanced member
Posts: 61
Joined: Fri Oct 04, 2013 2:12 am
Contact:

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby maxxer » Tue Jan 09, 2018 8:14 pm

moebius wrote:It's certainly possible, but I don't know how to resolve it.


have you tried this?
~ YetOpen S.r.l. ~
https://www.yetopen.it
moebius
Posts: 17
Joined: Sat Sep 13, 2014 3:34 am

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby moebius » Wed Jan 10, 2018 5:50 pm

Yes, we already have those values increased unfortunately...

Code: Select all

$ zmprov gcf zimbraLmtpNumThreads
zimbraLmtpNumThreads: 40
$ zmprov gs `zmhostname` zimbraImapNumThreads
zimbraImapNumThreads: 500
$ zmprov gs `zmhostname` zimbraHttpNumThreads   
zimbraHttpNumThreads: 500
skurgan
Posts: 1
Joined: Thu Jan 11, 2018 10:50 am

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby skurgan » Thu Jan 11, 2018 11:39 am

Some problem, help.

mntwinsfan wrote:I recently upgraded to Zimbra 8.8.5 from 8.7.6. Ever since I did, some users (not all) have been having issues getting into their mailbox from time to time. Through the web client, they get the login page, enter their credentials, and then the browser just spins. On their devices, it will eventually time out. If I restart Zimbra, it works fine for everyone for a while but then starts failing again. Never for all users, just a handful. I am seeing the following errors in the mailbox.log:

Code: Select all

2018-01-01 18:49:28,948 ERROR [LmtpServer-1] [name=user@domain.com;mid=16;ip=nn.nn.nn.nn;] mailbox - Failed to lock mailbox
Write Lock Owner - ImapSSLServer-1 prio=5 id=182 state=TERMINATED
Lock Waiter - Timer-Zimbra daemon prio=5 id=37 state=TIMED_WAITING
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireNanos(AbstractQueuedSynchronizer.java:934)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1247)
        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1115)
        at com.zimbra.cs.mailbox.MailboxLock.tryLockWithTimeout(MailboxLock.java:114)
        at com.zimbra.cs.mailbox.MailboxLock.lock(MailboxLock.java:194)
        at com.zimbra.cs.mailbox.Mailbox.lock(Mailbox.java:10411)
        at com.zimbra.cs.imap.ImapListener.unload(ImapListener.java:580)
        at com.zimbra.cs.imap.ImapSessionManager$SessionSerializerTask.run(ImapSessionManager.java:195)
        at java.util.TimerThread.mainLoop(Timer.java:555)
        at java.util.TimerThread.run(Timer.java:505)

com.zimbra.cs.mailbox.MailboxLock$LockFailedException: timeout
        at com.zimbra.cs.mailbox.MailboxLock.lock(MailboxLock.java:211)
        at com.zimbra.cs.mailbox.Mailbox.beginTransaction(Mailbox.java:1759)
        at com.zimbra.cs.mailbox.Mailbox.beginReadTransaction(Mailbox.java:1735)
        at com.zimbra.cs.mailbox.Mailbox.getItemById(Mailbox.java:2864)
        at com.zimbra.cs.mailbox.Mailbox.getItemById(Mailbox.java:2856)
        at com.zimbra.cs.mailbox.Mailbox.getFolderById(Mailbox.java:4092)
        at com.zimbra.cs.filter.IncomingMessageHandler.getDefaultFolderPath(IncomingMessageHandler.java:91)
        at com.zimbra.cs.filter.ZimbraMailAdapter.keep(ZimbraMailAdapter.java:526)
        at com.zimbra.cs.filter.ZimbraMailAdapter.executeActionKeepInternal(ZimbraMailAdapter.java:359)
        at com.zimbra.cs.filter.ZimbraMailAdapter.executeAllActions(ZimbraMailAdapter.java:294)
        at com.zimbra.cs.filter.RuleManager.applyRulesToIncomingMessage(RuleManager.java:440)
        at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliverMessageToLocalMailboxes(ZimbraLmtpBackend.java:615)
        at com.zimbra.cs.lmtpserver.ZimbraLmtpBackend.deliver(ZimbraLmtpBackend.java:385)
        at com.zimbra.cs.lmtpserver.LmtpHandler.processMessageData(LmtpHandler.java:445)
        at com.zimbra.cs.lmtpserver.TcpLmtpHandler.continueDATA(TcpLmtpHandler.java:79)
        at com.zimbra.cs.lmtpserver.LmtpHandler.doDATA(LmtpHandler.java:434)
        at com.zimbra.cs.lmtpserver.LmtpHandler.processCommand(LmtpHandler.java:216)
        at com.zimbra.cs.lmtpserver.TcpLmtpHandler.processCommand(TcpLmtpHandler.java:72)
        at com.zimbra.cs.server.ProtocolHandler.processConnection(ProtocolHandler.java:189)
        at com.zimbra.cs.server.ProtocolHandler.run(ProtocolHandler.java:128)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-01-01 18:49:28,948 WARN  [LmtpServer-1] [name=user@domain.com;mid=16;ip=nn.nn.nn.nn;] mailbox - transaction canceled because of lock failure


Searching through these forums, I found some old posts regarding this error message which suggested changing some of the server settings. I did but they had no effect:

Code: Select all

zmlocalconfig -e zimbra_mailbox_lock_timeout=90
zmlocalconfig -e zimbra_mailbox_lock_max_waiting_threads=100
zmlocalconfig -e mailboxd_java_heap_size=4096
zmlocalconfig -e mailboxd_java_heap_memory_percent=25
zmprov ms `zmhostname` zimbraMessageCacheSize 4000
zmprov ms `zmhostname` zimbraImapNumThreads 50
zmprov ms `zmhostname` zimbraHttpNumThreads 600


Does anyone have any further suggestions or know how to fix this problem?
Klug
Elite member
Elite member
Posts: 2251
Joined: Mon Dec 16, 2013 11:35 am
Contact:

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby Klug » Thu Jan 11, 2018 3:58 pm

I'm not sure its a "session number" issue but NIO issue.

Can you check if IMAP NIO is enabled?
See this thread: viewtopic.php?f=15&t=63354
moebius
Posts: 17
Joined: Sat Sep 13, 2014 3:34 am

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby moebius » Thu Jan 11, 2018 6:06 pm

Enabled for me...

Code: Select all

zimbra@svr02-jls:~$ zmlocalconfig|grep nio_imap_enabled
nio_imap_enabled = true
bgszim
Posts: 30
Joined: Sat Sep 13, 2014 1:50 am

Re: Since Upgrading To Zimbra 8.8.5, Have Been Getting Numerous Lock Errors

Postby bgszim » Wed Jan 24, 2018 12:26 pm

Bump?

What is the status of your lock issues. Have you solved your issues (maybe by upgrading to 8.8.6)?

Return to “Installation and Upgrade”

Who is online

Users browsing this forum: No registered users and 11 guests