[Solved] Proper way to reboot with Zimbra?

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Post Reply
tonyg
Advanced member
Advanced member
Posts: 51
Joined: Fri Mar 16, 2018 5:25 pm
Location: USA
ZCS/ZD Version: 8.8.12.GA.3794.UBUNTU18.64 FOSS
Contact:

[Solved] Proper way to reboot with Zimbra?

Post by tonyg »

I'm running 8.8.9 over CentOS 7. As I've been getting more familiar with the environment, I break a couple things, services fail to start, I fix them, everything comes back, we're good.
Sometimes I see old .pid files, and I've been wondering if I'm not restarting the server properly. For example, we have at least these three commands available to us:

- shutdown -r now
- init 6
- reboot

Is there one method or another that does a proper "zmcontrol stop" rather than just killing all of the processes? Or is killing the processes acceptable?

Thanks.
Last edited by tonyg on Wed Aug 08, 2018 8:56 pm, edited 1 time in total.
phoenix
Ambassador
Ambassador
Posts: 27262
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Re: Proper way to reboot with Zimbra?

Post by phoenix »

I've never had a problem with ZCS when shutting it down with either of the following:

Code: Select all

systemctl reboot

shutdown -r now
Either of those should shutdown ZCS correctly.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
mcerci
Posts: 1
Joined: Mon Jul 25, 2022 1:49 pm

Re: [Solved] Proper way to reboot with Zimbra?

Post by mcerci »

Hi,

Running Zimbra 8.8.15_P31.1 on centos 7 and centos 8.

As tonyg said if one makes a server shutdown or reboot (with commands such as reboot, init, systemctl reboot) (or sending a shutdown or reboot signal with vmware tools when it is a vm) while zimbra is running, instead of stopping zimbra with "zmcontrol stop" some zimbra processes are just killed.

This is how to see the situation :

1 - If server is shutdown while zimbra is running, it takes approx 10-15 seconds to shutdown. This is abnormal because zmcontrol stop command itself takes about 40 seconds to complete.

2 - If zimbra is shutdown with zmcontrol stop then you see messages in "/var/log/messages" file such as these (but don' t see in direct os shutdown) :
Jun 29 18:36:34 testserver zimbramon[33657]: 33657:info: Stopping services initiated by zmcontrol
Jun 29 18:36:34 testserver zimbramon[33657]: 33657:info: Stopping vmware-ha via zmcontrol
Jun 29 18:36:34 testserver zimbramon[33657]: 33657:info: Stopping zmconfigd via zmcontrol
Jun 29 18:36:34 testserver zmconfigd[5233]: Shutting down. Received signal 15
Jun 29 18:36:35 testserver zimbramon[33657]: 33657:info: Stopping imapd via zmcontrol
Jun 29 18:36:35 testserver zimbramon[33657]: 33657:info: Stopping zimlet via zmcontrol
Jun 29 18:36:46 testserver zmmailboxdmgr[33758]: stop requested


Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping zimbraAdmin via zmcontrol


Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping zimbra via zmcontrol


Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping service via zmcontrol


Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping stats via zmcontrol
Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping mta via zmcontrol


Jun 29 18:36:50 testserver /postfix-script[33962]: stopping the Postfix mail system
Jun 29 18:36:50 testserver postfix/master[11846]: terminating on signal 15
Jun 29 18:36:52 testserver zimbramon[33657]: 33657:info: Stopping spell via zmcontrol
Jun 29 18:36:52 testserver zimbramon[33657]: 33657:info: Stopping snmp via zmcontrol
Jun 29 18:36:53 testserver zimbramon[33657]: 33657:info: Stopping cbpolicyd via zmcontrol
Jun 29 18:36:53 testserver zimbramon[33657]: 33657:info: Stopping archiving via zmcontrol
Jun 29 18:36:53 testserver amavis[11036]: Net::Server: 2020/06/29-18:36:53 Server closing!
Jun 29 18:36:58 testserver amavis-mc[10410]: Master process shutting down


Jun 29 18:37:03 testserver zimbramon[33657]: 33657:info: Stopping opendkim via zmcontrol


Jun 29 18:37:08 testserver zimbramon[33657]: 33657:info: Stopping amavis via zmcontrol
Jun 29 18:37:08 testserver zimbramon[33657]: 33657:info: Stopping antivirus via zmcontrol
Jun 29 18:37:09 testserver clamd[11297]: Pid file removed.
Jun 29 18:37:09 testserver clamd[11297]: --- Stopped at Mon Jun 29 18:37:09 2020
Jun 29 18:37:09 testserver clamd[11297]: Socket file removed.
Jun 29 18:37:10 testserver zimbramon[33657]: 33657:info: Stopping antispam via zmcontrol
Jun 29 18:37:13 testserver zimbramon[33657]: 33657:info: Stopping proxy via zmcontrol
Jun 29 18:37:14 testserver zimbramon[33657]: 33657:info: Stopping memcached via zmcontrol
Jun 29 18:37:15 testserver zimbramon[33657]: 33657:info: Stopping mailbox via zmcontrol


Jun 29 18:37:15 testserver zimbramon[33657]: 33657:info: Stopping convertd via zmcontrol
Jun 29 18:37:15 testserver zimbramon[33657]: 33657:info: Stopping logger via zmcontrol
Jun 29 18:37:17 testserver zimbramon[33657]: 33657:info: Stopping dnscache via zmcontrol
Jun 29 18:37:17 testserver zimbramon[33657]: 33657:info: Stopping ldap via zmcontrol
Jun 29 18:37:17 testserver slapd[4050]: daemon: shutdown requested and initiated.
Jun 29 18:37:17 testserver slapd[4050]: slapd shutdown: waiting for 0 operations/tasks to finish
Jun 29 18:37:17 testserver slapd[4050]: DIGEST-MD5 common mech free
Jun 29 18:37:17 testserver slapd[4050]: DIGEST-MD5 common mech free
Jun 29 18:37:17 testserver slapd[4050]: slapd stopped.
My investigation showed that this seems to be becasuse zimbra processes are out of Cgroup "/system.slice/zimbra.service"
[root@testserver ~]# systemctl status zimbra
Now running the real systemctl command : systemctl status zimbra
â zimbra.service - LSB: Zimbra mail service
Loaded: loaded (/etc/rc.d/init.d/zimbra; generated)
Active: active (exited) since Mon 2022-07-25 17:27:23 +03; 24s ago
Docs: man:systemd-sysv-generator(8)
Process: 1031 ExecStart=/etc/rc.d/init.d/zimbra start (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 178393)
Memory: 0B
CGroup: /system.slice/zimbra.service

Jul 25 17:27:18 testserver.testdomain.com zimbra[1031]: Starting cbpolicyd...Done.
Jul 25 17:27:18 testserver.testdomain.com zimbra[1031]: Starting snmp...Done.
Jul 25 17:27:19 testserver.testdomain.com zimbra[1031]: Starting spell...Done.
Jul 25 17:27:20 testserver.testdomain.com zimbra[1031]: Starting mta...Done.
Jul 25 17:27:21 testserver.testdomain.com zimbra[1031]: Starting stats...Done.
Jul 25 17:27:22 testserver.testdomain.com zimbra[1031]: Starting service webapp...Done.
Jul 25 17:27:22 testserver.testdomain.com zimbra[1031]: Starting zimbra webapp...Done.
Jul 25 17:27:23 testserver.testdomain.com zimbra[1031]: Starting zimbraAdmin webapp...Done.
Jul 25 17:27:23 testserver.testdomain.com zimbra[1031]: Starting zimlet webapp...Done.
Jul 25 17:27:23 testserver.testdomain.com systemd[1]: Started LSB: Zimbra mail service.

[root@testserver ~]# systemd-cgls
...
...
â ââuser-969.slice
â ââuser@969.service
â â ââgvfs-daemon.service
â â â ââ1677 /usr/libexec/gvfsd
â â â ââ1728 /usr/libexec/gvfsd-fuse /run/user/969/gvfs -f -o big_writes
â â ââinit.scope
â â â ââ1307 /usr/lib/systemd/systemd --user
â â â ââ1331 (sd-pam)
â â ââdbus.service
â â ââ1623 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
â ââsession-c2.scope
â ââ 2714 /opt/zimbra/common/libexec/slapd -l LOCAL0 -u zimbra -h ldap://testserver.testdomain.com:389 ldapi:/// -F /opt/zimbra/data/ldap/config
â ââ 2901 /opt/zimbra/common/lib/jvm/java/bin/java -XX:ErrorFile=/opt/zimbra/log -client -Xmx256m -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -Djdk.tls.client.protocols=TLSv1,TLSv1.1,TLSv1.2 -Djava.net.preferIPv4Stack=true -Dzimbra.home>
â ââ 4076 /opt/zimbra/common/sbin/unbound
â ââ 4211 /opt/zimbra/common/bin/swatchdog --config-file=/opt/zimbra/conf/logswatchrc --use-cpan-file-tail --pid-file=/opt/zimbra/log/logswatch.pid --daemon --script-dir=/opt/zimbra/data/tmp --tail-file /var/log/zimbra-stats.log
â ââ 4215 /usr/bin/perl /opt/zimbra/libexec/zmlogger
â ââ 4235 /bin/sh /opt/zimbra/common/bin/mysqld_safe --defaults-file=/opt/zimbra/conf/my.cnf --external-locking --log-error=/opt/zimbra/log/mysql_error.log --malloc-lib=/opt/zimbra/common/lib/libtcmalloc_minimal.so --ledir=/opt/zimb>
â ââ 4476 /opt/zimbra/common/sbin/mysqld --defaults-file=/opt/zimbra/conf/my.cnf --basedir=/opt/zimbra/common --datadir=/opt/zimbra/db/data --plugin-dir=/opt/zimbra/common/lib/plugin --external-locking --log-error=/opt/zimbra/log/my>
As I understood Systemd managed OSs, when shutting down, os tries to stop zimbra.service and other services determined by their service control files as expected. And as paralel sends TERM signal to non service processes (user processes - session scopes). So they are killed. As seen above zimbra processes are in a "session scope"

I think this situation occurs because zimbra processes are run with sudo inside zmcontrol script and the scripts called from zmcontrol itself.

Anyway, i made my own solution. These are the steps:

0 - All the work is done with root user, not zimbra.

1- Open /etc/profile file and add a line before the line "export PATH" :

Code: Select all

PATH=/shutdown_startup:${PATH}
2 - Create directories "/shutdown_startup" and "/etc/vmware-tools/scripts/poweroff-vm-default.d"

Code: Select all

[root@testserver ~]# mkdir /shutdown_startup
[root@testserver ~]# mkdir /etc/vmware-tools/scripts/poweroff-vm-default.d
3 - Upload scripts to "/shutdown_startup" directory

3.1 - /shutdown_startup/init script

Code: Select all

#!/usr/bin/bash

LOG_FILE=/var/log/init_script.log
REAL_INIT_PATH=/usr/sbin

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}

if [ $1 -eq 0 ] || [ $1 -eq 6 ]
then
	echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
	echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
	su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1

#	TIME=`date +%Y%m%d%H%M%S`
#	echo "${TIME} : Now running the command : : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#	su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi

TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real init command : init $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real init command : init $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_INIT_PATH}/init $1 $2 $3 $4 $5 $6 $7 $8 $9

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
3.2 - /shutdown_startup/poweroff script

Code: Select all

#!/usr/bin/bash

LOG_FILE=/var/log/poweroff_script.log
REAL_POWEROFF_PATH=/usr/sbin

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}


echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1

#TIME=`date +%Y%m%d%H%M%S`
#echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1

TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real poweroff command : poweroff $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real poweroff command : poweroff $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_POWEROFF_PATH}/poweroff $1 $2 $3 $4 $5 $6 $7 $8 $9

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
3.3 - /shutdown_startup/reboot script

Code: Select all

#!/usr/bin/bash

LOG_FILE=/var/log/reboot_script.log
REAL_REBOOT_PATH=/usr/sbin

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}


echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1

#TIME=`date +%Y%m%d%H%M%S`
#echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1

TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real reboot command : reboot $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real reboot command : reboot $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_REBOOT_PATH}/reboot $1 $2 $3 $4 $5 $6 $7 $8 $9

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
3.4 - /shutdown_startup/shutdown script

Code: Select all

#!/usr/bin/bash

LOG_FILE=/var/log/shutdown_script.log
REAL_SHUTDOWN_PATH=/usr/sbin

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}

if [ "$1" != "-c" ] || [ "$2" != "-c" ]
then
	echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
	echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
	su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1

#	TIME=`date +%Y%m%d%H%M%S`
#	echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#	su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi

TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real shutdown command : shutdown $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real shutdown command : shutdown $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_SHUTDOWN_PATH}/shutdown $1 $2 $3 $4 $5 $6 $7 $8 $9

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
3.5 - /shutdown_startup/systemctl script

Code: Select all

#!/usr/bin/bash

LOG_FILE=/var/log/systemctl_script.log
REAL_SYSTEMCTL_PATH=/usr/bin

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}

if [ "$1" = "reboot" ] || [ "$2" = "reboot" ] || [ "$1" = "poweroff" ] || [ "$2" = "poweroff" ] || [ "$1" = "shutdown" ] || [ "$2" = "shutdown" ]
then
	echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
	echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
	su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1

#	TIME=`date +%Y%m%d%H%M%S`
#	echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#	su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi

TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real systemctl command : systemctl $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real systemctl command : systemctl $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_SYSTEMCTL_PATH}/systemctl $1 $2 $3 $4 $5 $6 $7 $8 $9

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
3.6 - /shutdown_startup/telinit script

Code: Select all

#!/usr/bin/bash

LOG_FILE=/var/log/telinit_script.log
REAL_TELINIT_PATH=/usr/sbin

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}

if [ $1 -eq 0 ] || [ $1 -eq 6 ]
then
	echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
	echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
	su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1

#	TIME=`date +%Y%m%d%H%M%S`
#	echo "${TIME} : Now running the command : : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#	su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi

TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real telinit command : telinit $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real telinit command : telinit $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_TELINIT_PATH}/telinit $1 $2 $3 $4 $5 $6 $7 $8 $9

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
4 - Upload scripts to "/etc/vmware-tools/scripts/poweroff-vm-default.d" directory

4.1 - /etc/vmware-tools/scripts/poweroff-vm-default.d/shutdown-jobs.sh script (Thx to : https://elatov.github.io/2018/04/run-sc ... are-tools/ )

Code: Select all

#!/usr/bin/bash

LOG_FILE=/var/log/vmware_tools_script.log

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
	
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1

TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}

exit 0
5- Make scripts executable:

Code: Select all

[root@testserver ~]# chmod 744 /shutdown_startup/*
[root@testserver ~]# chmod 744 /etc/vmware-tools/scripts/poweroff-vm-default.d/shutdown-jobs.sh
6 - Reboot server [Don' t forget to run "zmcontrol stop" before rebooting for now :) ]

7- And finally if server is shutdown or rebooted with one of the commands init, shutdown, systemctl, reboot, poweroff, telinit and shutdown or rebooted via vmware tools, first "zmcontrol stop" is run and then the work is done.

8 - Script log files can be looked to see whether they are runned and when they are runned. They are in /var/log directory:
/var/log/init_script.log
/var/log/poweroff_script.log
/var/log/reboot_script.log
/var/log/shutdown_script.log
/var/log/systemctl_script.log
/var/log/telinit_script.log
/var/log/vmware_tools_script.log
Post Reply