Zimbra won't shutdown - Stopping zimlet webapp fails

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Ace Suares
Advanced member
Advanced member
Posts: 63
Joined: Thu Aug 07, 2014 7:26 pm

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by Ace Suares »

Confirmed on Release 8.8.8.GA.2009.UBUNTU16.64 UBUNTU16_64 FOSS edition.

pkill -9 zimbra did NOT help.
User avatar
joho
Advanced member
Advanced member
Posts: 74
Joined: Tue Apr 26, 2016 9:24 am
ZCS/ZD Version: Release 8.8.15.GA.4177.UBUNTU20.64

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by joho »

Still here in Release 8.8.12.GA.3794.UBUNTU16.64 UBUNTU16_64 NETWORK edition, Patch 8.8.12_P1 proxy.
User avatar
joho
Advanced member
Advanced member
Posts: 74
Joined: Tue Apr 26, 2016 9:24 am
ZCS/ZD Version: Release 8.8.15.GA.4177.UBUNTU20.64

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by joho »

joho wrote:Still here in Release 8.8.12.GA.3794.UBUNTU16.64 UBUNTU16_64 NETWORK edition, Patch 8.8.12_P1 proxy.
It does (finally) stop/re-start, but it takes *forever* ...

I don't understand why something as critical as this isn't addressed in a patch considering it hits you every single time a restart or shutdown is requested ... :shock:
PaperAdvocate
Posts: 23
Joined: Tue Oct 11, 2016 9:28 pm

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by PaperAdvocate »

Since this is still here I wanted to post what ended up fixing it for me (I've posted about this issue in another thread): viewtopic.php?f=15&t=63221 . Zimbra support had an open ticket with me for a year and never resolved it (I still love Zimbra anyway), for an unrelated reason I had to do a database repair and ran the steps found here: https://wiki.zimbra.com/wiki/Mysql_Crash_Recovery

After that, no more huge delay on shutdown/restart. Hopefully this helps others...
Ace Suares
Advanced member
Advanced member
Posts: 63
Joined: Thu Aug 07, 2014 7:26 pm

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by Ace Suares »

When looking at the log, the amount of 'Modified db pages' stays the same in all Buffer Pools.

This Flush Dirty Pages code does not seem to work all and just sits waiting for the timeout.
User avatar
joho
Advanced member
Advanced member
Posts: 74
Joined: Tue Apr 26, 2016 9:24 am
ZCS/ZD Version: Release 8.8.15.GA.4177.UBUNTU20.64

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by joho »

This works a lot better (for us) in 8.8.15
User avatar
joho
Advanced member
Advanced member
Posts: 74
Joined: Tue Apr 26, 2016 9:24 am
ZCS/ZD Version: Release 8.8.15.GA.4177.UBUNTU20.64

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by joho »

But in the latest patch for 8.8.15 that was released a few days ago, the problem with shutting down zimlet webapp is back :-(
michael.zietlow
Posts: 3
Joined: Thu Feb 11, 2021 12:22 pm

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by michael.zietlow »

psilvahr wrote:Well, i see this solution, but not tested yet:

Jason Waters 2017-06-30 23:49:58 UTC
I got mine fixed! Thanks to Zimbra support. So I did this.

su - Zimbra
zmcontrol stop
Then control-C to to stop the services that don't start
mkdir /opt/zimbra/log/oldpids
mv /opt/zimbra/log/*.pid /opt/zimbra/log/oldpids
zmcontrol start
zmcontrol stop

And mine just worked! Hope it helps others!

Worked for a day on my 8.8.15 version but now zmcontrol stop still hangs on webapp shutdowns.

Kind of sucks but maybe Zimbra9 finally addresses this? I really hate to ctrl-c
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2796
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.6 Network Edition
Contact:

Re: Zimbra won't shutdown - Stopping zimlet webapp fails

Post by L. Mark Stone »

If you feel that the mailboxd shutdown hang is due to MariaDB taking time to write things out to disk, you can do two things to speed this up. Both involve editing ~/conf/my.cnf.

First, use a tool like mysqltuner.pl to make sure that the innodb_buffer_pool_size variable is 110% to 125% of the aggregated sizes of your InnoDB databases. (mysqltuner.pl conveniently prints out this comparison for you in one line.) If your innodb_buffer_pool_size is too small, increase it, being careful to make sure you have adequate RAM to avoid swapping. Note: You can use M and G suffixes to simplify the number. Here's an example entry for a 22GB InnoDB buffer pool:

Code: Select all

innodb_buffer_pool_size        = 22528M
Second, when MariaDB does flush stuff to disk, it performs internal IO throttling, to prevent the flush from monopolizing disk IO. This flush rate is controlled by the innodb_io_capacity variable, which Zimbra does not set. Zimbra therefore uses the default value, which is quite low: only 200 IOPs.

If you have a reasonably fast disk subsystem, you can add this variable to ~/conf/my.cnf and set it to whatever you like. In my experience, on a busy mailbox server, I don't want the disk flushes to consume more than half of the available IOPs. I host on AWS and use gp3 disks, which have 3,000 IOPs, so I add a line to ~/conf/my.cnf like so:

Code: Select all

innodb_io_capacity             = 1500
If MariaDB's InnoDB buffer pool is large enough so that MariaDB doesn't have to thrash RAM and disk to do writes, and if flushes to disk are allowed to happen at a faster IOPs rate, I suspect you will find Zimbra mailbox service restarts will complete more quickly, and, likely you may also see a general performance increase in Zimbra as well.

Hope that helps,
Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
Post Reply