Updated live sync script

myarmush · Post by **myarmush** » Wed Feb 18, 2015 11:41 am

Hey guys, (sorry for bumping an older post)
Does this script work with 8.5 or am I missing something?
My sync starts working and processing the redologs but then the PlaybackUtil just hangs.
I cant get a continuous sync to complete.
No errors logged just the last message from the synchronized redologs and then crickets.
In the script it is hanging here (live_syncd:153-155) but I dont think it's the script.
logit 3 "Replaying redologs..."
if ! zmplayredo >> "$LOG_FILE"; then
logit 2 "Replay of redolog failed"
Mailbox.log show the messages being imported to the backup server
2015-02-18 11:30:10,729 INFO [RedoPlayer-40] [] mailop - Adding Message: id=281576, Message-ID=<0.1.0.26.1D04B7A9CC29F56.EABF1@c
olo163.custombriefings.com>, parentId=281256, folderId=2, folderName=Inbox.
2015-02-18 11:30:10,731 INFO [main] [] extensions - Destroying extensions
2015-02-18 11:30:10,731 INFO [main] [] extensions - Destroyed extension zimbra-freebusy-provider: com.zimbra.cs.fb.ZimbraFreeBus
yProviderExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,731 INFO [main] [] extensions - Destroyed extension com_zimbra_cert_manager: com.zimbra.cert.ZimbraCertMgrEx
t@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,806 INFO [main] [] extensions - Destroyed extension hsm: com.zimbra.cs.hsm.HsmInitializer@sun.misc.Launcher$
ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,807 INFO [main] [] extensions - Destroyed extension clamscanner: com.zimbra.clam.ClamScannerExt@sun.misc.Lau
ncher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,807 INFO [main] [] extensions - Destroyed extension voice: com.zimbra.cs.voice.VoiceExtension@sun.misc.Launc
her$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension zimbrasync: com.zimbra.zimbrasync.ZimbraSync@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension archive: com.zimbra.cs.archive.ArchiveInitializer@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension clientUploader: com.zimbra.clientuploader.ZClientUploaderExt@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension com_zimbra_bulkprovision: com.zimbra.bp.ZimbraBulkProvisionExt@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension versioncheck: com.zimbra.cs.versioncheck.VersionCheckExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension convertd: com.zimbra.cs.convert.ConverterExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension : com.zimbra.cs.network.NetworkInitializer@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,808 INFO [main] [] extensions - Destroyed extension nginx-lookup: com.zimbra.cs.nginx.NginxLookupExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,809 INFO [main] [] extensions - Destroyed extension zimbrasamba: com.zimbra.ldaputils.ZimbraLDAPUtilsExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,809 INFO [main] [] extensions - Destroyed extension com_zimbra_xmbxsearch: com.zimbra.xmbxsearch.ZimbraXMbxSearchExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,809 INFO [main] [] mboxmove - Handler at /backup/mboximport shutting down
2015-02-18 11:30:10,809 INFO [main] [] mboxmove - Handler at /backup/mboxexport shutting down
2015-02-18 11:30:10,809 INFO [main] [] extensions - Destroyed extension backup: com.zimbra.cs.backup.BackupExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,809 INFO [main] [] extensions - Destroyed extension license: com.zimbra.cs.network.license.LicenseExtension@sun.misc.Launcher$ExtClassLoader@5fd0d5ae
2015-02-18 11:30:10,842 WARN [main] [] ConfigurationFactory - No configuration found. Configuring ehcache from ehcache-failsafe.xml found in the classpath: jar:file:/opt/zimbra/lib/jars/ehcache-core-2.5.1.jar!/ehcache-failsafe.xml
2015-02-18 11:30:10,971 INFO [main] [] ZimbraHttpConnectionManager - shutting down http client idle connection reaper thread
2015-02-18 11:30:10,971 INFO [main] [] ZimbraHttpConnectionManager - shutting down http client idle connection reaper thread
2015-02-18 11:51:38,049 INFO [LicenseCounter] [] LdapLicenseCounter - Begin account=80,archiving=0
2015-02-18 11:51:38,052 INFO [LicenseCounter] [] LdapLicenseCounter - End account=80,archiving=0,next=3600

Any ideas?

Thanks,
Moe

myarmush · Post by **myarmush** » Thu Feb 19, 2015 4:25 pm

Turns out there is a bug is 8.5 in zmplayredo

https://bugzilla.zimbra.com/show_bug.cgi?id=96058

I'll be upgrading to 8.6 soon.

shockwavecs · Post by **shockwavecs** » Mon May 11, 2015 5:11 pm

did that help? I am on 8.6p1 and it fails here.

myarmush · Post by **myarmush** » Mon May 11, 2015 5:40 pm

I didn't end up upgrading.

I pulled the zmplayredo from the rpm and replaced the 8.5 version(after backing it up).

The scripts all worked after that.

Hth.

shockwavecs · Post by **shockwavecs** » Tue May 12, 2015 10:57 am

nevermind. It works on 8.6.0p1 . I had to add the live_sync.pub key to the /opt/zimbra/.ssh/authorized_keys file on both servers (or on primary before performing the initial rsync)

satish.lx · Post by **satish.lx** » Wed Aug 12, 2015 8:13 am

I am using it on 8.6.0 and it works!!! but what if my primary down for couple hours and i want to use secondary and all my mail will goes to secondary zimbra. but after fixing Primary how my those mail will come back to Primary server which sent to backup when we were down... Can some one explain that process?

sempiterna · Post by **sempiterna** » Tue Jun 21, 2016 8:30 am

I've been running live_sync for about 4 days now, and it is (according to the live_sync log) updating LDAP. I also see the redolog being updated. However, I don't see the redolog being played. The only time it plays the redolog is when the live_sync script is started (right at the beginning).

How often is the script supposed to replay the redolog? Or is replaying something that should be done outside the script?

EDIT: Started working after I restarted zimbra. I thought changing configuration items with zmprov were live immediately, but it seems you have to restart zimbra first for them to take effect.

myzamri · Post by **myzamri** » Fri May 05, 2017 2:24 am

I have followed all the steps in the wiki here : https://wiki.zimbra.com/wiki/Server_Live_sync

I got this error in the log in /opt/zimbra/live_sync/log/live_sync.log:

zimbra@mail:~/live_sync$ tail -f log/live_sync.log
Fri May 5 10:09:00 MYT 2017 :Incremental backups enabled : false
Fri May 5 10:09:00 MYT 2017 :Convertd enabled : false
Fri May 5 10:09:00 MYT 2017 :Starting ldap live sync process
Fri May 5 10:09:00 MYT 2017 :Starting redo log live sync process
Fri May 5 10:09:00 MYT 2017 :Syncing ldap using ldif
Fri May 5 10:09:18 MYT 2017 :Syncing LDAP done
Fri May 5 10:09:18 MYT 2017 :Start new LDAP monitor period
Fri May 5 10:09:18 MYT 2017 :Detecting if HSM used
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
Fri May 5 10:10:14 MYT 2017 :No HSM Detected
Fri May 5 10:10:14 MYT 2017 :Syncing redologs...
Fri May 5 10:10:18 MYT 2017 :Syncing /opt/zimbra/redolog/redo.log
Fri May 5 10:10:48 MYT 2017 :Syncing /opt/zimbra/redolog/redo.log done
Fri May 5 10:10:48 MYT 2017 :Syncing redologs done
Fri May 5 10:10:48 MYT 2017 :Purging redolog directory and archives
Fri May 5 10:10:48 MYT 2017 :Purge redolog directory and archives done
Fri May 5 10:10:50 MYT 2017 :Replaying redologs...
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f9f37260000, 34105589760, 0) failed; error='Cannot allocate memory' (errno=12)
Fri May 5 10:10:50 MYT 2017 :Replay of redolog failed
Fri May 5 10:10:50 MYT 2017 :Replaying redologs done
Fri May 5 10:10:50 MYT 2017 :Purging remote redolog directory
Fri May 5 10:10:51 MYT 2017 :Purging remote redolog directory done
Fri May 5 10:10:51 MYT 2017 :Live streaming redolog
Fri May 5 10:10:56 MYT 2017 :Live streaming redolog established
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
Fri May 5 10:11:25 MYT 2017 :Archive file name not found
Fri May 5 10:11:25 MYT 2017 :Syncing redologs...
Fri May 5 10:11:29 MYT 2017 :Syncing /opt/zimbra/redolog/redo.log
Fri May 5 10:11:31 MYT 2017 :Syncing /opt/zimbra/redolog/redo.log done
Fri May 5 10:11:31 MYT 2017 :Syncing redologs done
Fri May 5 10:11:31 MYT 2017 :Purging redolog directory and archives
Fri May 5 10:11:31 MYT 2017 :Purge redolog directory and archives done
Fri May 5 10:11:32 MYT 2017 :Replaying redologs...
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f17db260000, 34105589760, 0) failed; error='Cannot allocate memory' (errno=12)
Fri May 5 10:11:32 MYT 2017 :Replay of redolog failed
Fri May 5 10:11:32 MYT 2017 :Replaying redologs done
Fri May 5 10:11:32 MYT 2017 :Purging remote redolog directory
Fri May 5 10:11:33 MYT 2017 :Purging remote redolog directory done
Fri May 5 10:11:33 MYT 2017 :Live streaming redolog
Fri May 5 10:11:38 MYT 2017 :Live streaming redolog established
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
Fri May 5 10:19:18 MYT 2017 :Syncing ldap using ldif
Fri May 5 10:19:34 MYT 2017 :Syncing LDAP done
Fri May 5 10:19:34 MYT 2017 :Start new LDAP monitor period
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.

There is an error : Replay of Redolog failed.

Please advice...

myzamri · Post by **myzamri** » Sun May 07, 2017 3:43 pm

It hassomething to with this error: OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f9f37260000, 34105589760, 0) failed; error='Cannot allocate memory' (errno=12)

I increased the memory and the problem solved.

myzamri · Post by **myzamri** » Mon May 08, 2017 1:33 am

The live_sync.log after fixing the error:

Mon May 8 09:18:27 MYT 2017 :Syncing /opt/zimbra/redolog/redo.log
Mon May 8 09:18:28 MYT 2017 :Syncing /opt/zimbra/redolog/redo.log done
Mon May 8 09:18:28 MYT 2017 :Syncing redologs done
Mon May 8 09:18:28 MYT 2017 :Purging redolog directory and archives
Mon May 8 09:18:28 MYT 2017 :Purge redolog directory and archives done
Mon May 8 09:18:29 MYT 2017 :Replaying redologs...
Mon May 8 09:18:39 MYT 2017 :Replaying redologs done
Mon May 8 09:18:39 MYT 2017 :Re-starting Zimbra mailbox service
Mon May 8 09:18:54 MYT 2017 :Purging remote redolog directory
Mon May 8 09:18:55 MYT 2017 :Purging remote redolog directory done
Mon May 8 09:18:55 MYT 2017 :Live streaming redolog
Mon May 8 09:19:00 MYT 2017 :Live streaming redolog established
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.

It all looks good and dandy.

Zimbra Forums