Updated live sync script
-
16428simonb
- Posts: 45
- Joined: Fri Sep 12, 2014 9:55 pm
Updated live sync script
Looks like the Wiki is alive again! I have updated the script on the Wiki page and tried to incorporate as much of the feedback as possible.
-
15342aqeel
- Posts: 11
- Joined: Sat Sep 13, 2014 3:19 am
Updated live sync script
Problem executing script, after 1st rsync i checked my mirror server, & it stoped proxying giving me 404 NOT_FOUND error. cant find logs m new to it so need help.
Updated live sync script
[quote user="15342aqeel"]Problem executing script, after 1st rsync i checked my mirror server, & it stoped proxying giving me 404 NOT_FOUND error. cant find logs m new to it so need help.[/QUOTE]
You didn't do it properly.
On Backup Server:
Run Rsync Command (At bottom of this post)
Primary Server:
STOP zimbra(So no messages can be sent to the mailstore..All incoming mail will queue in your proxy)
On Backup Server:
When Zimbra is Stopped on Primary, do Rsync command again!
When the Rsync is finished:
Primary Server:
Start Zimbra
Backup Server:
service zimbra start
service zimbra status (Does all work?? Yes? Continue)
service zimbra stop
su - zimbra
cd live_sync
./live_syncd start
tail -f log/live_sync.log
It seems like you did only 1 Rsync, and attempted to start Zimbra on backup server..It is incomplete.
Rsync command(You run this command on the backup server):
rsync -aHz --force --delete --sparse live_server:/opt/zimbra/ /opt/zimbra/
Edited this post to make it a little easier to follow.
Hope it helps
You didn't do it properly.
On Backup Server:
Run Rsync Command (At bottom of this post)
Primary Server:
STOP zimbra(So no messages can be sent to the mailstore..All incoming mail will queue in your proxy)
On Backup Server:
When Zimbra is Stopped on Primary, do Rsync command again!
When the Rsync is finished:
Primary Server:
Start Zimbra
Backup Server:
service zimbra start
service zimbra status (Does all work?? Yes? Continue)
service zimbra stop
su - zimbra
cd live_sync
./live_syncd start
tail -f log/live_sync.log
It seems like you did only 1 Rsync, and attempted to start Zimbra on backup server..It is incomplete.
Rsync command(You run this command on the backup server):
rsync -aHz --force --delete --sparse live_server:/opt/zimbra/ /opt/zimbra/
Edited this post to make it a little easier to follow.
Hope it helps
-
15342aqeel
- Posts: 11
- Joined: Sat Sep 13, 2014 3:19 am
Updated live sync script
hello all,
I have gone through the complete process as it is in wiki.zimbra.com/wiki/Server_Live_sync, when i put the command "tail -f log/live_sync.log", i received an error in it. please see below:
***************************************
Wed Oct 30 19:58:47 PKT 2013 :Starting live_syncd Wed Oct 30 19:58:47 PKT 2013 :Incremental backups enabled : false Wed Oct 30 19:58:47 PKT 2013 :Convertd enabled : false Wed Oct 30 19:58:47 PKT 2013 :Starting ldap live sync process Wed Oct 30 19:58:47 PKT 2013 :Starting redo log live sync process Wed Oct 30 19:58:47 PKT 2013 :Syncing ldap using rsync Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
which: no zmhsm in (/opt/zimbra/bin:/opt/zimbra/postfix/sbin:/opt/zimbra/openldap/bin:/opt/zimbra/snmp/bin:/opt/zimbra/rsync/bin:/opt/zimbra/bdb/bin:/opt/zimbra/openssl/bin:/opt/zimbra/java/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin)
Wed Oct 30 19:59:05 PKT 2013 :Syncing LDAP done Wed Oct 30 19:59:05 PKT 2013 :Start new LDAP monitor period Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
Wed Oct 30 19:59:07 PKT 2013 :Detecting if HSM used Error occurred: error while proxying request to target server: HTTP/1.1 404 Not Found Wed Oct 30 19:59:28 PKT 2013 :No HSM Detected Wed Oct 30 19:59:28 PKT 2013 :Syncing redologs...
Wed Oct 30 19:59:28 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131028.183407.141-seq1.log
Wed Oct 30 19:59:28 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131028.190435.724-seq2.log
Wed Oct 30 19:59:29 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131030.143944.706-seq3.log
Wed Oct 30 19:59:29 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131030.144945.391-seq4.log
which: no zmhsm in (/opt/zimbra/bin:/opt/zimbra/postfix/sbin:/opt/zimbra/openldap/bin:/opt/zimbra/snmp/bin:/opt/zimbra/rsync/bin:/opt/zimbra/bdb/bin:/opt/zimbra/openssl/bin:/opt/zimbra/java/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin)
Wed Oct 30 19:59:32 PKT 2013 :Syncing /opt/zimbra/redolog/redo.log Wed Oct 30 19:59:32 PKT 2013 :Syncing /opt/zimbra/redolog/redo.log done Wed Oct 30 19:59:32 PKT 2013 :Syncing redologs done Wed Oct 30 19:59:32 PKT 2013 :Purging redolog directory and archives Wed Oct 30 19:59:32 PKT 2013 :Purge redolog directory and archives done Wed Oct 30 19:59:34 PKT 2013 :Replaying redologs...
Wed Oct 30 19:59:40 PKT 2013 :Replaying redologs done Wed Oct 30 19:59:40 PKT 2013 :Purging remote redolog directory Wed Oct 30 19:59:40 PKT 2013 :Purging remote redolog directory done Wed Oct 30 19:59:40 PKT 2013 :Live streaming redolog Wed Oct 30 19:59:45 PKT 2013 :Live streaming redolog established Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
/opt/zimbra/data/ldap/hdb/db/ CLOSE_WRITE,CLOSE zimbraVirtualHostname.bdb
tail: log/live_sync.log: file truncated
and now i am having same error 404 NOT_FOUND please help
I have gone through the complete process as it is in wiki.zimbra.com/wiki/Server_Live_sync, when i put the command "tail -f log/live_sync.log", i received an error in it. please see below:
***************************************
Wed Oct 30 19:58:47 PKT 2013 :Starting live_syncd Wed Oct 30 19:58:47 PKT 2013 :Incremental backups enabled : false Wed Oct 30 19:58:47 PKT 2013 :Convertd enabled : false Wed Oct 30 19:58:47 PKT 2013 :Starting ldap live sync process Wed Oct 30 19:58:47 PKT 2013 :Starting redo log live sync process Wed Oct 30 19:58:47 PKT 2013 :Syncing ldap using rsync Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
which: no zmhsm in (/opt/zimbra/bin:/opt/zimbra/postfix/sbin:/opt/zimbra/openldap/bin:/opt/zimbra/snmp/bin:/opt/zimbra/rsync/bin:/opt/zimbra/bdb/bin:/opt/zimbra/openssl/bin:/opt/zimbra/java/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin)
Wed Oct 30 19:59:05 PKT 2013 :Syncing LDAP done Wed Oct 30 19:59:05 PKT 2013 :Start new LDAP monitor period Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
Wed Oct 30 19:59:07 PKT 2013 :Detecting if HSM used Error occurred: error while proxying request to target server: HTTP/1.1 404 Not Found Wed Oct 30 19:59:28 PKT 2013 :No HSM Detected Wed Oct 30 19:59:28 PKT 2013 :Syncing redologs...
Wed Oct 30 19:59:28 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131028.183407.141-seq1.log
Wed Oct 30 19:59:28 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131028.190435.724-seq2.log
Wed Oct 30 19:59:29 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131030.143944.706-seq3.log
Wed Oct 30 19:59:29 PKT 2013 :Syncing incremental backup file: /opt/zimbra/redolog/archive/redo-20131030.144945.391-seq4.log
which: no zmhsm in (/opt/zimbra/bin:/opt/zimbra/postfix/sbin:/opt/zimbra/openldap/bin:/opt/zimbra/snmp/bin:/opt/zimbra/rsync/bin:/opt/zimbra/bdb/bin:/opt/zimbra/openssl/bin:/opt/zimbra/java/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin)
Wed Oct 30 19:59:32 PKT 2013 :Syncing /opt/zimbra/redolog/redo.log Wed Oct 30 19:59:32 PKT 2013 :Syncing /opt/zimbra/redolog/redo.log done Wed Oct 30 19:59:32 PKT 2013 :Syncing redologs done Wed Oct 30 19:59:32 PKT 2013 :Purging redolog directory and archives Wed Oct 30 19:59:32 PKT 2013 :Purge redolog directory and archives done Wed Oct 30 19:59:34 PKT 2013 :Replaying redologs...
Wed Oct 30 19:59:40 PKT 2013 :Replaying redologs done Wed Oct 30 19:59:40 PKT 2013 :Purging remote redolog directory Wed Oct 30 19:59:40 PKT 2013 :Purging remote redolog directory done Wed Oct 30 19:59:40 PKT 2013 :Live streaming redolog Wed Oct 30 19:59:45 PKT 2013 :Live streaming redolog established Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
/opt/zimbra/data/ldap/hdb/db/ CLOSE_WRITE,CLOSE zimbraVirtualHostname.bdb
tail: log/live_sync.log: file truncated
and now i am having same error 404 NOT_FOUND please help
Updated live sync script
Hello everybody!
I know this is somewhat an old thread but I can not seem to get this process right. I have re-done the process in a test environment about 3 times so far (maybe 2 I'm tired).
The first time I failed to run some of the steps as root or zimbra, so I completely re-installed ubuntu 12.04 and zimbra to make sure I didn't miss something.
Both times now I have gotten stuck at the same spot.
I am trying to run the live_syncd script for the first time "./live_syncd start" and I receive "permission denied (publickey,password). unable to run commands on remote server"
Now clearly I am doing something wrong, but I am unable to tell where. I have made sure this run through to follow each step with extreme caution and create the SSH key under the zimbra user but alas I am at a loss.
Any ideas/help would be greatly appreciated.
Thank you.
I know this is somewhat an old thread but I can not seem to get this process right. I have re-done the process in a test environment about 3 times so far (maybe 2 I'm tired).
The first time I failed to run some of the steps as root or zimbra, so I completely re-installed ubuntu 12.04 and zimbra to make sure I didn't miss something.
Both times now I have gotten stuck at the same spot.
I am trying to run the live_syncd script for the first time "./live_syncd start" and I receive "permission denied (publickey,password). unable to run commands on remote server"
Now clearly I am doing something wrong, but I am unable to tell where. I have made sure this run through to follow each step with extreme caution and create the SSH key under the zimbra user but alas I am at a loss.
Any ideas/help would be greatly appreciated.
Thank you.
Updated live sync script
This is something related to ssh key generated and related configurations
Sent from my Lenovo S920_ROW using Tapatalk
Sent from my Lenovo S920_ROW using Tapatalk
Updated live sync script
[quote user="picalo"]Hello everybody!
I know this is somewhat an old thread but I can not seem to get this process right. I have re-done the process in a test environment about 3 times so far (maybe 2 I'm tired).
The first time I failed to run some of the steps as root or zimbra, so I completely re-installed ubuntu 12.04 and zimbra to make sure I didn't miss something.
Both times now I have gotten stuck at the same spot.
I am trying to run the live_syncd script for the first time "./live_syncd start" and I receive "permission denied (publickey,password). unable to run commands on remote server"
Now clearly I am doing something wrong, but I am unable to tell where. I have made sure this run through to follow each step with extreme caution and create the SSH key under the zimbra user but alas I am at a loss.
Any ideas/help would be greatly appreciated.
Thank you.[/QUOTE]
Hey man, first, make sure when you're running the live_syncd script, you are user Zimbra.
If you've done that, the solution is pretty easy then.
As user zimbra,
cd /opt/zimbra/.ssh
ssh-keygen -b 4096 -f live_sync
echo "command="/opt/zimbra/live_sync/sync_commands" $( cat live_sync.pub )">>authorized_keys
Running these 3 commands on the Primary server, then performing the 2 initial Rsyncs to the other server will fix your problem.
You could also just do it manually and skip the Rsyn copying the live_sync.pub, live_sync and authorized_keys to the second server.
Hey also make sure the permissions on the keys are properly set. (both owned by zimbra.)
Things keep popping in my head. Is the uid of the Zimbra user the same on both systems? If not, the Zimbra files would be pointing an an owner who doesn't exist on system #2
If you're playing in a VM, you could start SSHD in debug to figure out exactly why it's failing. I believe it's /usr/sbin/sshd -dp22 and run the live_syncd script again from the backup server, the primary server should be able to tell you whats wrong. Anyway I hope one of these things will hit home. If you were on centos or rhel i would say selinux, but ubuntu I am not sure if there is an equivalent
I know this is somewhat an old thread but I can not seem to get this process right. I have re-done the process in a test environment about 3 times so far (maybe 2 I'm tired).
The first time I failed to run some of the steps as root or zimbra, so I completely re-installed ubuntu 12.04 and zimbra to make sure I didn't miss something.
Both times now I have gotten stuck at the same spot.
I am trying to run the live_syncd script for the first time "./live_syncd start" and I receive "permission denied (publickey,password). unable to run commands on remote server"
Now clearly I am doing something wrong, but I am unable to tell where. I have made sure this run through to follow each step with extreme caution and create the SSH key under the zimbra user but alas I am at a loss.
Any ideas/help would be greatly appreciated.
Thank you.[/QUOTE]
Hey man, first, make sure when you're running the live_syncd script, you are user Zimbra.
If you've done that, the solution is pretty easy then.
As user zimbra,
cd /opt/zimbra/.ssh
ssh-keygen -b 4096 -f live_sync
echo "command="/opt/zimbra/live_sync/sync_commands" $( cat live_sync.pub )">>authorized_keys
Running these 3 commands on the Primary server, then performing the 2 initial Rsyncs to the other server will fix your problem.
You could also just do it manually and skip the Rsyn copying the live_sync.pub, live_sync and authorized_keys to the second server.
Hey also make sure the permissions on the keys are properly set. (both owned by zimbra.)
Things keep popping in my head. Is the uid of the Zimbra user the same on both systems? If not, the Zimbra files would be pointing an an owner who doesn't exist on system #2
If you're playing in a VM, you could start SSHD in debug to figure out exactly why it's failing. I believe it's /usr/sbin/sshd -dp22 and run the live_syncd script again from the backup server, the primary server should be able to tell you whats wrong. Anyway I hope one of these things will hit home. If you were on centos or rhel i would say selinux, but ubuntu I am not sure if there is an equivalent
Updated live sync script
[quote user="twokeys"]Hey man, first, make sure when you're running the live_syncd script, you are user Zimbra.
If you've done that, the solution is pretty easy then.
As user zimbra,
cd /opt/zimbra/.ssh
ssh-keygen -b 4096 -f live_sync
echo "command="/opt/zimbra/live_sync/sync_commands" $( cat live_sync.pub )">>authorized_keys
Running these 3 commands on the Primary server, then performing the 2 initial Rsyncs to the other server will fix your problem.
You could also just do it manually and skip the Rsyn copying the live_sync.pub, live_sync and authorized_keys to the second server.
Hey also make sure the permissions on the keys are properly set. (both owned by zimbra.)
Things keep popping in my head. Is the uid of the Zimbra user the same on both systems? If not, the Zimbra files would be pointing an an owner who doesn't exist on system #2
If you're playing in a VM, you could start SSHD in debug to figure out exactly why it's failing. I believe it's /usr/sbin/sshd -dp22 and run the live_syncd script again from the backup server, the primary server should be able to tell you whats wrong. Anyway I hope one of these things will hit home. If you were on centos or rhel i would say selinux, but ubuntu I am not sure if there is an equivalent[/QUOTE]
twokeys -
I made sure I was zimbra user when running the command - same issue
I re-ran the ssh creation, asked to overwrite, did so. then performed the syncs via tutorial - same issue
I have not tried manually pulling those folders but did check their consistency across both systems and they look the same.
i chown'ed both the keys to make sure zimbra was owner - same issue
the uid is the same on both systems
I am not in a vm.
I have re-installed the entire boxes 3 times now to make sure I am not screwing something up. Is there any logs I can pull or something to shed more light on this? I am fairly frustrated but I want to get this working! I really appreciate your help.
If you've done that, the solution is pretty easy then.
As user zimbra,
cd /opt/zimbra/.ssh
ssh-keygen -b 4096 -f live_sync
echo "command="/opt/zimbra/live_sync/sync_commands" $( cat live_sync.pub )">>authorized_keys
Running these 3 commands on the Primary server, then performing the 2 initial Rsyncs to the other server will fix your problem.
You could also just do it manually and skip the Rsyn copying the live_sync.pub, live_sync and authorized_keys to the second server.
Hey also make sure the permissions on the keys are properly set. (both owned by zimbra.)
Things keep popping in my head. Is the uid of the Zimbra user the same on both systems? If not, the Zimbra files would be pointing an an owner who doesn't exist on system #2
If you're playing in a VM, you could start SSHD in debug to figure out exactly why it's failing. I believe it's /usr/sbin/sshd -dp22 and run the live_syncd script again from the backup server, the primary server should be able to tell you whats wrong. Anyway I hope one of these things will hit home. If you were on centos or rhel i would say selinux, but ubuntu I am not sure if there is an equivalent[/QUOTE]
twokeys -
I made sure I was zimbra user when running the command - same issue
I re-ran the ssh creation, asked to overwrite, did so. then performed the syncs via tutorial - same issue
I have not tried manually pulling those folders but did check their consistency across both systems and they look the same.
i chown'ed both the keys to make sure zimbra was owner - same issue
the uid is the same on both systems
I am not in a vm.
I have re-installed the entire boxes 3 times now to make sure I am not screwing something up. Is there any logs I can pull or something to shed more light on this? I am fairly frustrated but I want to get this working! I really appreciate your help.
Updated live sync script
[quote user="umarzuki"]This is something related to ssh key generated and related configurations
Sent from my Lenovo S920_ROW using Tapatalk[/QUOTE]
I would tend to agree but I am not seeing where/why.
Sent from my Lenovo S920_ROW using Tapatalk[/QUOTE]
I would tend to agree but I am not seeing where/why.
Updated live sync script
[quote user="picalo"]twokeys -
I made sure I was zimbra user when running the command - same issue
I re-ran the ssh creation, asked to overwrite, did so. then performed the syncs via tutorial - same issue
I have not tried manually pulling those folders but did check their consistency across both systems and they look the same.
i chown'ed both the keys to make sure zimbra was owner - same issue
the uid is the same on both systems
I am not in a vm.
I have re-installed the entire boxes 3 times now to make sure I am not screwing something up. Is there any logs I can pull or something to shed more light on this? I am fairly frustrated but I want to get this working! I really appreciate your help.[/QUOTE]
This is super strange, really. But I feel like it's an Ubuntu thing for some reason. Anyways let's figure this out. Try this
You're on a development environment so you don't need to worry so much about security (I would assume..) so you don't need to worry too much about restricting the SSH session to that one "/opt/zimbra/live_sync/sync_commands" command.
On the primary server, do a cat /opt/zimbra/.ssh/live_sync.pub copy the contents, and paste it into authorized_keys on secondary server.
Now to test it, from the primary server as Zimbra, run ssh -i /opt/zimbra/.ssh/live_sync zimbra@ip_of_standby
If that doesn't work, I honestly have no more ideas because this is just standard SSH keys that you're having problems with.
I made sure I was zimbra user when running the command - same issue
I re-ran the ssh creation, asked to overwrite, did so. then performed the syncs via tutorial - same issue
I have not tried manually pulling those folders but did check their consistency across both systems and they look the same.
i chown'ed both the keys to make sure zimbra was owner - same issue
the uid is the same on both systems
I am not in a vm.
I have re-installed the entire boxes 3 times now to make sure I am not screwing something up. Is there any logs I can pull or something to shed more light on this? I am fairly frustrated but I want to get this working! I really appreciate your help.[/QUOTE]
This is super strange, really. But I feel like it's an Ubuntu thing for some reason. Anyways let's figure this out. Try this
You're on a development environment so you don't need to worry so much about security (I would assume..) so you don't need to worry too much about restricting the SSH session to that one "/opt/zimbra/live_sync/sync_commands" command.
On the primary server, do a cat /opt/zimbra/.ssh/live_sync.pub copy the contents, and paste it into authorized_keys on secondary server.
Now to test it, from the primary server as Zimbra, run ssh -i /opt/zimbra/.ssh/live_sync zimbra@ip_of_standby
If that doesn't work, I honestly have no more ideas because this is just standard SSH keys that you're having problems with.