Updated live sync script

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
16428simonb
Posts: 45
Joined: Fri Sep 12, 2014 9:55 pm

Updated live sync script

Post by 16428simonb »

I have attached to this post some updates to the scripts on the Live Sync wiki page...

Server Live sync - Zimbra :: Wiki
I can't edit the Wiki page right now since the Wiki is undergoing maintenance and not accepting new edits.
The tgz file contains updated scripts plus a Nagios plugin and nrpe configuration. Note that on Ubuntu the plugin would be put in /usr/lib/nagios/plugins/contrib and under Redhat 64-bit it would be under /usr/lib64/nagios/plugins/contrib. Adjust the nrpe config accordingly.
Here is a paste of the changelog...


Version 2.0.0
1) Started changes log :)
2) Constants in upper case, variables in lower case
3) Exclude ldap log changes from inotify watch
4) Nagios plug-in to monitor time since last successful operation
5) Redologs are archived once successfully replayed
6) Redologs are retrieved from incremental backups on Zimbra Network edition
7) Redologs are purged independently after specified time on both servers
8) Magic path names and numbers now defined in constants
9) Log reporting by level and with filtering
10) Script aware of the java redolog replay process
11) convertd runs during redolog replay if enabled so indexing will work
Best regards,

Simon
live_sync.tar.gz
n.sossonko
Advanced member
Advanced member
Posts: 68
Joined: Sat Sep 13, 2014 2:14 am

Updated live sync script

Post by n.sossonko »

Is this up on the Wiki (i.e. is the wiki no up to date)? Has anyone tried this script with 7.2.x and/or 8.0.x?
Thanks.
16428simonb
Posts: 45
Joined: Fri Sep 12, 2014 9:55 pm

Updated live sync script

Post by 16428simonb »

The Wiki is not up to date. I would like to update it but it is frozen for maintenance so I have to post updates here instead. I have some minor updates to the above script that I intend to post up tomorrow.
I know it works fine on 7.2. I haven't tested it on 8.0.x as yet.
16428simonb
Posts: 45
Joined: Fri Sep 12, 2014 9:55 pm

Updated live sync script

Post by 16428simonb »

A minor update added. Use file, live_sync_2.0.1.tar.gz.
Version 2.0.1
1) Preserve mailboxd runing state after sync to allow warmer standby and functioning HSM
2) Don't refuse to start because of stale pid files after a system reboot
live_sync_2.0.1.tar.gz
16428simonb
Posts: 45
Joined: Fri Sep 12, 2014 9:55 pm

Updated live sync script

Post by 16428simonb »

I've made some updates for Zimbra 8. Please do not use earlier versions of this script on Zimbra 8 or it will get very stuck trying to sync the ldap directory.
Also for version 8, with regard to the old setting up instructions, the rsync command used to set up the standby server now has to be aware of sparse files. Otherwise you will end up with 80GB more disk space being used on the standby server compared to the live server.
Old command

rsync -aHz --force --delete live_server:/opt/zimbra/ /opt/zimbra/

New command

rsync -aHz --force --delete --sparse live_server:/opt/zimbra/ /opt/zimbra/
These are the new changes. Many thanks to those who provided feedback to improve the script.
1) Compatible with Zimbra version 8
2) Kill hanging sync_commands processes on live server
3) Rename LDAP_DIR constant to LDAP_TEMP_DIR
4) Make rsync of ldap database aware of sparse files
5) For Zimbra version 8 and above use ldif export/import instead of rsync
live_sync_2.1.0.tar.gz
16428simonb
Posts: 45
Joined: Fri Sep 12, 2014 9:55 pm

Updated live sync script

Post by 16428simonb »

Oops. New version already. There is an egrep that won't work on RHEL systems that has been fixed in the sync_commands script that resulted in the purge time days not being passed.
live_sync_2.1.1.tar.gz
16428simonb
Posts: 45
Joined: Fri Sep 12, 2014 9:55 pm

Updated live sync script

Post by 16428simonb »

The script is now aware of HSM. If use of HSM is detected then mailboxd is kept running on the standby server and sync operations are suspended while the HSM process is active on the standby server.
live_sync.2.1.2.tar.gz
16428simonb
Posts: 45
Joined: Fri Sep 12, 2014 9:55 pm

Updated live sync script

Post by 16428simonb »

If anyone is interested. Here is the /etc/init.d script I use. I know it works on Ubuntu. It /should/ work on Redhat/CentOS too but needs verifying.
One the standby server I disable Zimbra on boot and enable this script on boot.
zimbra_live_sync.tar.gz
15337Raunaq
Advanced member
Advanced member
Posts: 157
Joined: Sat Sep 13, 2014 2:59 am

Updated live sync script

Post by 15337Raunaq »

Tested it...Works for me on 7.2.2
plutocrat
Posts: 19
Joined: Sat Sep 13, 2014 12:56 am

Updated live sync script

Post by plutocrat »

First a huge thankyou to simonb for making this script available. This should really be incorporated into Zimbra, and they should pay him handsomely to do so!
Secondly, please can the Zimbra folk update the Live Sync wiki page to at least point to this forum thread. I wouldn't have found it unless simonb alerted me to it, and I wouldn't have found the version of the script that works on Zimbra 8.
OK. So I got this working on my installation and I had a couple of things which slowed me down, so I thought I'd mention them here in case they help anyone out. Again it would be nice to be able to put these in the wiki, but the page is locked.
1) Hostname. The hostname on live and failover servers should be the SAME. I felt that it should be, but had to check.
2) On the initial rsync, its OK to exclude the /opt/zimbra/backup directory. This is not needed. That saved me about half a terrabyte of bandwidth. You can use rsync's --exclude tag.
3) To check that the failover server is indeed copying over emails and documents as expected, its OK to stop the sync, and start up zimbra. ie

cd /opt/zimbra/live_sync

./live_syncd stop

zmcontrol start

(check Zimbra Email account / Admin panel, and then when happy ....)

zmcontrol stop

./live_syncd start

The sync process will figure out where it is and catch up again.
4) Watch out for file permissions! When I had synced over to the failover server, I tried to start zimbra (zmcontrol start) and got a message about 'zmmailboxctrl is not running'. Not good. In in /opt/zimbra/log/zmmailboxd.out I found some errors claiming that certain files weren't there, and that permission was denied on others. Finally I figured it out. On the old server, jetty was trying to run as UID 1003 and GID 1004. But on the mirror server: zimbra was gid=1001 and uid=1001
~$ id zimbra

uid=1001(zimbra) gid=1001(zimbra) groups=1001(zimbra),4(adm),5(tty),1002(postfix)
So to fix this I did the following on the new mirror server:

usermod -u 1003 zimbra

groupmod -g 1004 zimbra

find / -user 1001 -exec chown -h 1003 {} ;

find / -group 1001 -exec chgrp -h 1004 {} ;

(this takes a looong time)

usermod -g 1004 zimbra
And then zimbra would finally start!
So that's my feedback. In addition I can supply some metrics. On a server with 70 accounts, the server is syncing about 1Gb of data daily (less on weekends). So you might like to use that as a baseline for bandwidth.
Post Reply