I'm running 8.8.9 over CentOS 7. As I've been getting more familiar with the environment, I break a couple things, services fail to start, I fix them, everything comes back, we're good.
Sometimes I see old .pid files, and I've been wondering if I'm not restarting the server properly. For example, we have at least these three commands available to us:
- shutdown -r now
- init 6
- reboot
Is there one method or another that does a proper "zmcontrol stop" rather than just killing all of the processes? Or is killing the processes acceptable?
Thanks.
[Solved] Proper way to reboot with Zimbra?
-
- Advanced member
- Posts: 51
- Joined: Fri Mar 16, 2018 5:25 pm
- Location: USA
- ZCS/ZD Version: 8.8.12.GA.3794.UBUNTU18.64 FOSS
- Contact:
[Solved] Proper way to reboot with Zimbra?
Last edited by tonyg on Wed Aug 08, 2018 8:56 pm, edited 1 time in total.
Re: Proper way to reboot with Zimbra?
I've never had a problem with ZCS when shutting it down with either of the following:
Either of those should shutdown ZCS correctly.
Code: Select all
systemctl reboot
shutdown -r now
Re: [Solved] Proper way to reboot with Zimbra?
Hi,
Running Zimbra 8.8.15_P31.1 on centos 7 and centos 8.
As tonyg said if one makes a server shutdown or reboot (with commands such as reboot, init, systemctl reboot) (or sending a shutdown or reboot signal with vmware tools when it is a vm) while zimbra is running, instead of stopping zimbra with "zmcontrol stop" some zimbra processes are just killed.
This is how to see the situation :
1 - If server is shutdown while zimbra is running, it takes approx 10-15 seconds to shutdown. This is abnormal because zmcontrol stop command itself takes about 40 seconds to complete.
2 - If zimbra is shutdown with zmcontrol stop then you see messages in "/var/log/messages" file such as these (but don' t see in direct os shutdown) :
I think this situation occurs because zimbra processes are run with sudo inside zmcontrol script and the scripts called from zmcontrol itself.
Anyway, i made my own solution. These are the steps:
0 - All the work is done with root user, not zimbra.
1- Open /etc/profile file and add a line before the line "export PATH" :
2 - Create directories "/shutdown_startup" and "/etc/vmware-tools/scripts/poweroff-vm-default.d"
3 - Upload scripts to "/shutdown_startup" directory
3.1 - /shutdown_startup/init script
3.2 - /shutdown_startup/poweroff script
3.3 - /shutdown_startup/reboot script
3.4 - /shutdown_startup/shutdown script
3.5 - /shutdown_startup/systemctl script
3.6 - /shutdown_startup/telinit script
4 - Upload scripts to "/etc/vmware-tools/scripts/poweroff-vm-default.d" directory
4.1 - /etc/vmware-tools/scripts/poweroff-vm-default.d/shutdown-jobs.sh script (Thx to : https://elatov.github.io/2018/04/run-sc ... are-tools/ )
5- Make scripts executable:
6 - Reboot server [Don' t forget to run "zmcontrol stop" before rebooting for now ]
7- And finally if server is shutdown or rebooted with one of the commands init, shutdown, systemctl, reboot, poweroff, telinit and shutdown or rebooted via vmware tools, first "zmcontrol stop" is run and then the work is done.
8 - Script log files can be looked to see whether they are runned and when they are runned. They are in /var/log directory:
/var/log/init_script.log
/var/log/poweroff_script.log
/var/log/reboot_script.log
/var/log/shutdown_script.log
/var/log/systemctl_script.log
/var/log/telinit_script.log
/var/log/vmware_tools_script.log
Running Zimbra 8.8.15_P31.1 on centos 7 and centos 8.
As tonyg said if one makes a server shutdown or reboot (with commands such as reboot, init, systemctl reboot) (or sending a shutdown or reboot signal with vmware tools when it is a vm) while zimbra is running, instead of stopping zimbra with "zmcontrol stop" some zimbra processes are just killed.
This is how to see the situation :
1 - If server is shutdown while zimbra is running, it takes approx 10-15 seconds to shutdown. This is abnormal because zmcontrol stop command itself takes about 40 seconds to complete.
2 - If zimbra is shutdown with zmcontrol stop then you see messages in "/var/log/messages" file such as these (but don' t see in direct os shutdown) :
My investigation showed that this seems to be becasuse zimbra processes are out of Cgroup "/system.slice/zimbra.service"Jun 29 18:36:34 testserver zimbramon[33657]: 33657:info: Stopping services initiated by zmcontrol
Jun 29 18:36:34 testserver zimbramon[33657]: 33657:info: Stopping vmware-ha via zmcontrol
Jun 29 18:36:34 testserver zimbramon[33657]: 33657:info: Stopping zmconfigd via zmcontrol
Jun 29 18:36:34 testserver zmconfigd[5233]: Shutting down. Received signal 15
Jun 29 18:36:35 testserver zimbramon[33657]: 33657:info: Stopping imapd via zmcontrol
Jun 29 18:36:35 testserver zimbramon[33657]: 33657:info: Stopping zimlet via zmcontrol
Jun 29 18:36:46 testserver zmmailboxdmgr[33758]: stop requested
…
…
Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping zimbraAdmin via zmcontrol
…
…
Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping zimbra via zmcontrol
…
…
Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping service via zmcontrol
…
…
Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping stats via zmcontrol
Jun 29 18:36:50 testserver zimbramon[33657]: 33657:info: Stopping mta via zmcontrol
…
…
Jun 29 18:36:50 testserver /postfix-script[33962]: stopping the Postfix mail system
Jun 29 18:36:50 testserver postfix/master[11846]: terminating on signal 15
Jun 29 18:36:52 testserver zimbramon[33657]: 33657:info: Stopping spell via zmcontrol
Jun 29 18:36:52 testserver zimbramon[33657]: 33657:info: Stopping snmp via zmcontrol
Jun 29 18:36:53 testserver zimbramon[33657]: 33657:info: Stopping cbpolicyd via zmcontrol
Jun 29 18:36:53 testserver zimbramon[33657]: 33657:info: Stopping archiving via zmcontrol
Jun 29 18:36:53 testserver amavis[11036]: Net::Server: 2020/06/29-18:36:53 Server closing!
Jun 29 18:36:58 testserver amavis-mc[10410]: Master process shutting down
…
…
Jun 29 18:37:03 testserver zimbramon[33657]: 33657:info: Stopping opendkim via zmcontrol
…
…
Jun 29 18:37:08 testserver zimbramon[33657]: 33657:info: Stopping amavis via zmcontrol
Jun 29 18:37:08 testserver zimbramon[33657]: 33657:info: Stopping antivirus via zmcontrol
Jun 29 18:37:09 testserver clamd[11297]: Pid file removed.
Jun 29 18:37:09 testserver clamd[11297]: --- Stopped at Mon Jun 29 18:37:09 2020
Jun 29 18:37:09 testserver clamd[11297]: Socket file removed.
Jun 29 18:37:10 testserver zimbramon[33657]: 33657:info: Stopping antispam via zmcontrol
Jun 29 18:37:13 testserver zimbramon[33657]: 33657:info: Stopping proxy via zmcontrol
Jun 29 18:37:14 testserver zimbramon[33657]: 33657:info: Stopping memcached via zmcontrol
Jun 29 18:37:15 testserver zimbramon[33657]: 33657:info: Stopping mailbox via zmcontrol
…
…
Jun 29 18:37:15 testserver zimbramon[33657]: 33657:info: Stopping convertd via zmcontrol
Jun 29 18:37:15 testserver zimbramon[33657]: 33657:info: Stopping logger via zmcontrol
Jun 29 18:37:17 testserver zimbramon[33657]: 33657:info: Stopping dnscache via zmcontrol
Jun 29 18:37:17 testserver zimbramon[33657]: 33657:info: Stopping ldap via zmcontrol
Jun 29 18:37:17 testserver slapd[4050]: daemon: shutdown requested and initiated.
Jun 29 18:37:17 testserver slapd[4050]: slapd shutdown: waiting for 0 operations/tasks to finish
Jun 29 18:37:17 testserver slapd[4050]: DIGEST-MD5 common mech free
Jun 29 18:37:17 testserver slapd[4050]: DIGEST-MD5 common mech free
Jun 29 18:37:17 testserver slapd[4050]: slapd stopped.
As I understood Systemd managed OSs, when shutting down, os tries to stop zimbra.service and other services determined by their service control files as expected. And as paralel sends TERM signal to non service processes (user processes - session scopes). So they are killed. As seen above zimbra processes are in a "session scope"[root@testserver ~]# systemctl status zimbra
Now running the real systemctl command : systemctl status zimbra
â zimbra.service - LSB: Zimbra mail service
Loaded: loaded (/etc/rc.d/init.d/zimbra; generated)
Active: active (exited) since Mon 2022-07-25 17:27:23 +03; 24s ago
Docs: man:systemd-sysv-generator(8)
Process: 1031 ExecStart=/etc/rc.d/init.d/zimbra start (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 178393)
Memory: 0B
CGroup: /system.slice/zimbra.service
Jul 25 17:27:18 testserver.testdomain.com zimbra[1031]: Starting cbpolicyd...Done.
Jul 25 17:27:18 testserver.testdomain.com zimbra[1031]: Starting snmp...Done.
Jul 25 17:27:19 testserver.testdomain.com zimbra[1031]: Starting spell...Done.
Jul 25 17:27:20 testserver.testdomain.com zimbra[1031]: Starting mta...Done.
Jul 25 17:27:21 testserver.testdomain.com zimbra[1031]: Starting stats...Done.
Jul 25 17:27:22 testserver.testdomain.com zimbra[1031]: Starting service webapp...Done.
Jul 25 17:27:22 testserver.testdomain.com zimbra[1031]: Starting zimbra webapp...Done.
Jul 25 17:27:23 testserver.testdomain.com zimbra[1031]: Starting zimbraAdmin webapp...Done.
Jul 25 17:27:23 testserver.testdomain.com zimbra[1031]: Starting zimlet webapp...Done.
Jul 25 17:27:23 testserver.testdomain.com systemd[1]: Started LSB: Zimbra mail service.
[root@testserver ~]# systemd-cgls
...
...
â ââuser-969.slice
â ââuser@969.service
â â ââgvfs-daemon.service
â â â ââ1677 /usr/libexec/gvfsd
â â â ââ1728 /usr/libexec/gvfsd-fuse /run/user/969/gvfs -f -o big_writes
â â ââinit.scope
â â â ââ1307 /usr/lib/systemd/systemd --user
â â â ââ1331 (sd-pam)
â â ââdbus.service
â â ââ1623 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
â ââsession-c2.scope
â ââ 2714 /opt/zimbra/common/libexec/slapd -l LOCAL0 -u zimbra -h ldap://testserver.testdomain.com:389 ldapi:/// -F /opt/zimbra/data/ldap/config
â ââ 2901 /opt/zimbra/common/lib/jvm/java/bin/java -XX:ErrorFile=/opt/zimbra/log -client -Xmx256m -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -Djdk.tls.client.protocols=TLSv1,TLSv1.1,TLSv1.2 -Djava.net.preferIPv4Stack=true -Dzimbra.home>
â ââ 4076 /opt/zimbra/common/sbin/unbound
â ââ 4211 /opt/zimbra/common/bin/swatchdog --config-file=/opt/zimbra/conf/logswatchrc --use-cpan-file-tail --pid-file=/opt/zimbra/log/logswatch.pid --daemon --script-dir=/opt/zimbra/data/tmp --tail-file /var/log/zimbra-stats.log
â ââ 4215 /usr/bin/perl /opt/zimbra/libexec/zmlogger
â ââ 4235 /bin/sh /opt/zimbra/common/bin/mysqld_safe --defaults-file=/opt/zimbra/conf/my.cnf --external-locking --log-error=/opt/zimbra/log/mysql_error.log --malloc-lib=/opt/zimbra/common/lib/libtcmalloc_minimal.so --ledir=/opt/zimb>
â ââ 4476 /opt/zimbra/common/sbin/mysqld --defaults-file=/opt/zimbra/conf/my.cnf --basedir=/opt/zimbra/common --datadir=/opt/zimbra/db/data --plugin-dir=/opt/zimbra/common/lib/plugin --external-locking --log-error=/opt/zimbra/log/my>
I think this situation occurs because zimbra processes are run with sudo inside zmcontrol script and the scripts called from zmcontrol itself.
Anyway, i made my own solution. These are the steps:
0 - All the work is done with root user, not zimbra.
1- Open /etc/profile file and add a line before the line "export PATH" :
Code: Select all
PATH=/shutdown_startup:${PATH}
Code: Select all
[root@testserver ~]# mkdir /shutdown_startup
[root@testserver ~]# mkdir /etc/vmware-tools/scripts/poweroff-vm-default.d
3.1 - /shutdown_startup/init script
Code: Select all
#!/usr/bin/bash
LOG_FILE=/var/log/init_script.log
REAL_INIT_PATH=/usr/sbin
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
if [ $1 -eq 0 ] || [ $1 -eq 6 ]
then
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1
# TIME=`date +%Y%m%d%H%M%S`
# echo "${TIME} : Now running the command : : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
# su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi
TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real init command : init $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real init command : init $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_INIT_PATH}/init $1 $2 $3 $4 $5 $6 $7 $8 $9
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
Code: Select all
#!/usr/bin/bash
LOG_FILE=/var/log/poweroff_script.log
REAL_POWEROFF_PATH=/usr/sbin
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1
#TIME=`date +%Y%m%d%H%M%S`
#echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real poweroff command : poweroff $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real poweroff command : poweroff $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_POWEROFF_PATH}/poweroff $1 $2 $3 $4 $5 $6 $7 $8 $9
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
Code: Select all
#!/usr/bin/bash
LOG_FILE=/var/log/reboot_script.log
REAL_REBOOT_PATH=/usr/sbin
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1
#TIME=`date +%Y%m%d%H%M%S`
#echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
#su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real reboot command : reboot $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real reboot command : reboot $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_REBOOT_PATH}/reboot $1 $2 $3 $4 $5 $6 $7 $8 $9
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
Code: Select all
#!/usr/bin/bash
LOG_FILE=/var/log/shutdown_script.log
REAL_SHUTDOWN_PATH=/usr/sbin
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
if [ "$1" != "-c" ] || [ "$2" != "-c" ]
then
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1
# TIME=`date +%Y%m%d%H%M%S`
# echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
# su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi
TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real shutdown command : shutdown $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real shutdown command : shutdown $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_SHUTDOWN_PATH}/shutdown $1 $2 $3 $4 $5 $6 $7 $8 $9
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
Code: Select all
#!/usr/bin/bash
LOG_FILE=/var/log/systemctl_script.log
REAL_SYSTEMCTL_PATH=/usr/bin
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
if [ "$1" = "reboot" ] || [ "$2" = "reboot" ] || [ "$1" = "poweroff" ] || [ "$2" = "poweroff" ] || [ "$1" = "shutdown" ] || [ "$2" = "shutdown" ]
then
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1
# TIME=`date +%Y%m%d%H%M%S`
# echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
# su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi
TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real systemctl command : systemctl $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real systemctl command : systemctl $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_SYSTEMCTL_PATH}/systemctl $1 $2 $3 $4 $5 $6 $7 $8 $9
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
Code: Select all
#!/usr/bin/bash
LOG_FILE=/var/log/telinit_script.log
REAL_TELINIT_PATH=/usr/sbin
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
if [ $1 -eq 0 ] || [ $1 -eq 6 ]
then
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1
# TIME=`date +%Y%m%d%H%M%S`
# echo "${TIME} : Now running the command : : su - zimbra -c \"zmcontrol status\"" >> ${LOG_FILE}
# su - zimbra -c "zmcontrol status" >> ${LOG_FILE} 2>&1
fi
TIME=`date +%Y%m%d%H%M%S`
echo "Now running the real telinit command : telinit $1 $2 $3 $4 $5 $6 $7 $8 $9"
echo "${TIME} : Now running the real telinit command : telinit $1 $2 $3 $4 $5 $6 $7 $8 $9" >> ${LOG_FILE}
${REAL_TELINIT_PATH}/telinit $1 $2 $3 $4 $5 $6 $7 $8 $9
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
4.1 - /etc/vmware-tools/scripts/poweroff-vm-default.d/shutdown-jobs.sh script (Thx to : https://elatov.github.io/2018/04/run-sc ... are-tools/ )
Code: Select all
#!/usr/bin/bash
LOG_FILE=/var/log/vmware_tools_script.log
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script started" >> ${LOG_FILE}
echo "Now running the command : su - zimbra -c \"zmcontrol stop\""
echo "${TIME} : Now running the command : su - zimbra -c \"zmcontrol stop\"" >> ${LOG_FILE}
su - zimbra -c "zmcontrol stop" >> ${LOG_FILE} 2>&1
TIME=`date +%Y%m%d%H%M%S`
echo "${TIME} : Script ended" >> ${LOG_FILE}
exit 0
Code: Select all
[root@testserver ~]# chmod 744 /shutdown_startup/*
[root@testserver ~]# chmod 744 /etc/vmware-tools/scripts/poweroff-vm-default.d/shutdown-jobs.sh
7- And finally if server is shutdown or rebooted with one of the commands init, shutdown, systemctl, reboot, poweroff, telinit and shutdown or rebooted via vmware tools, first "zmcontrol stop" is run and then the work is done.
8 - Script log files can be looked to see whether they are runned and when they are runned. They are in /var/log directory:
/var/log/init_script.log
/var/log/poweroff_script.log
/var/log/reboot_script.log
/var/log/shutdown_script.log
/var/log/systemctl_script.log
/var/log/telinit_script.log
/var/log/vmware_tools_script.log