[SOLVED] Yet Another Backup Script for Community Version

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
dwmtractor
Outstanding Member
Outstanding Member
Posts: 993
Joined: Fri Sep 12, 2014 10:41 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by dwmtractor »

You guys who care about backup, here is a thread initiated by Zimbra that you really ought to vote on:

Dan">http://www.zimbra.com/forums/announceme ... -foss.html
Dan
cornbread
Advanced member
Advanced member
Posts: 80
Joined: Fri Sep 12, 2014 10:46 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by cornbread »

has anyone fitured out how to modify to go over ssh with a pre-authenticated key? something like ssh -vaz /path/to/backup domain.name:/backup/
dwmtractor
Outstanding Member
Outstanding Member
Posts: 993
Joined: Fri Sep 12, 2014 10:41 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by dwmtractor »

cornbread,
I've noticed your posts on this over the past few days and it occurs to me. . .given that what you're wanting to do is open-source backup, as you have probably realized by now, the effective OSS backup requires you to have Zimbra services stopped. Since you don't want them stopped for longer than necessary, you might really want to break up the stages of your backup anyway. By this I mean do your rsync (or other file copy process) to a local hard drive, as most of these methods do, restart Zimbra, and then (and ONLY then) back it up to the remote location. If you make the backup into a tar/compressed file, then you can use any methodology for ssh that can do ftp over ssh, which might make your scripting easier.
This will be significantly faster in terms of Zimbra downtime since the limiting factor will be HDD throughput rather than your net link throughput.
cheers,
Dan
heinzg
Advanced member
Advanced member
Posts: 83
Joined: Fri Sep 12, 2014 11:16 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by heinzg »

[quote user="dwmtractor"]heinzg,
I'm not enough of a shell script guru to analyze all you have done, but from what I have read of it so far, my hat is off to you for an extremely well-thought-out script. It's great to have the development site you linked, but if I could presume to make a suggestion, it'd be nice if you would post either the script, or at least a brief discussion with link to your development page, on the wiki under the open-source backup section (here) so that when people are reading about backup methodologies, this one is added to the mix. This will make it more likely (1) that those who need it find your script, and (2) that you'll get a wider variety of people testing and providing feedback to you. Kinda what we call a win-win! :D
Kudos for your hard work!
Dan[/QUOTE]
Hi Dan!
Thanx for you suggestion I will put it on the wiki ASAP... promise.
heinzg
heinzg
Advanced member
Advanced member
Posts: 83
Joined: Fri Sep 12, 2014 11:16 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by heinzg »

Hi cornbread!


Sorry I have not gotten back to you until now, vacation so to say.
[QUOTE]How do I do this? I already have the destination set up to do automatic ssh authentication from the machine. Which part of the script would I modify to do rsync via ssh?[/QUOTE]
[QUOTE]has anyone fitured out how to modify to go over ssh with a pre-authenticated key? something like ssh -vaz /path/to/backup domain.name:/backup/[/QUOTE]
I would like to understand what you are wanting to do with doing the the rsync over ssh to be able to offer you the best possible solution for your backup.
I opted not to do rsync over ssh when I started this script as I would have had to split the script into to two parts. Part one on the mail host to sync to a backup to a backup host. And the second part on the backup host to then archive it. I found this to be error prone & complex, as well as resource intensive (network, cpu, overall time).
I am currently testing a version of the script which will, after the creation of the backup archive ssh's (beem) it over to a backup/archive host for save keeping against 2 disks failing on the RAID 5 volume over night (yes I have seen it happen with very expensive SCSI disks made in Hungary in 2002 by a big two letter computer company from the States, who could that be... I remember you! heinzg don't forget that fast :p ). would this be something you could use?
On a slower system that is running my backup script I still have less than 1 min down time for the service and that is when there is as good as no one on it... (I have not had a mail user complain yet ;) ) I find this a good and save compromise for backing up the community edition server.
I would say in advance that if I was to sync to a remote host in my own LAN I would setup a rsync server and not ssh it, or even better use a iSCSI volume mounted on demand on 10Gigabit, then again a simple NFS share would also do, but the network usage, I would only do this kind of stuff on a dedicated backup LAN.... Ah to complex to be safe without support :rolleyes: And when it all goes wrong in production, what is it you want, yes a good old working backup that is fast and simple to restore.
My motto: Simple is good.
please feel free to contact me if there is anything regarding this script I can help you with.
heinzg
cornbread
Advanced member
Advanced member
Posts: 80
Joined: Fri Sep 12, 2014 10:46 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by cornbread »

heinzg,
I think that is the best way to do it.
I thin you are experimenting with exactly what I was envisioning.
local cold backup (small downtime)

restart zimbra, tar, ssh to host:/backup


Thanks again for the great work!
heinzg
Advanced member
Advanced member
Posts: 83
Joined: Fri Sep 12, 2014 11:16 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by heinzg »

Hi there!
I have done a re-write of the script and updated my first post (i.e. see the first post in this thread for more info)

It now has lots of new features like: archive encryption, scp, md5 checksums, installer...
Now also to be found on the zimbra wiki
I would love to hear your feed back on which Distros you are running the script it on.
Cheers

Heinzg
heinzg
Advanced member
Advanced member
Posts: 83
Joined: Fri Sep 12, 2014 11:16 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by heinzg »

Hi again!
JUST A WORD OF CAUTION to those who will run this script with encryption ON
The key file (script default /etc/zmbac/noread) BACK IT UP & KEEP IT SAFE without it your archives are no more than wasted space on the disk and/or tape!!.
The key generated by the script installer is a 48*8=348bit random passphrase, so unless you have a friend at the CIA and lots of time to recover the data, your archive will be lost.
OH one more thing the passphrase should only be known by "the need to know". Your secrets are only as safe as your passphrase is public!
There is also a small flaw in my scripts security you should know about. When "dar" is creating the archive (takes a while) you can SEE THE PASSPHRASE with "top -c" or with "ps -ef" so be sure to have no interactive shell users on the system at the time of backup. :eek:
Now I can sleep better tonight having that of my chest
mmorse
Ambassador
Ambassador
Posts: 6036
Joined: Fri Sep 12, 2014 10:24 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by mmorse »

[quote user="dwmtractor"]Kudos for your hard work![/quote]

Nice scripting - we'll be sure to get ya on the free shirt list :)
jayel
Advanced member
Advanced member
Posts: 85
Joined: Fri Sep 12, 2014 11:15 pm

[SOLVED] Yet Another Backup Script for Community Version

Post by jayel »

Hello,
Nice script.

I am trying to install it but I get an Error when installing.

Instellen van dar (2.2.4-2ubuntu2) ...

Create /etc/zmbac/

install y or n: y

mkdir: created directory `/etc/zmbac'

done
Create noread

install y or n: y

'G'enerate or 'E'nter a secure passphrase

please enter G or E: G


For 'scp' to work, you have to have setup PKI authentication (passwork less login)

Should I try setup this for you?

install y or n: y

/usr/bin/ssh-copy-id: ERROR: No identities found

0

Create ssh ID? y or n: y

Please except defaults, and NO 'passphrase' !!

You must specify a key type (-t).

Usage: ssh-keygen [options]

Options:

-a trials Number of trials for screening DH-GEX moduli.

-B Show bubblebabble digest of key file.

-b bits Number of bits in the key to create.

-C comment Provide new comment.

-c Change comment in private and public key files.

-e Convert OpenSSH to IETF SECSH key file.

-F hostname Find hostname in known hosts file.

-f filename Filename of the key file.

-G file Generate candidates for DH-GEX moduli.

-g Use generic DNS resource record format.

-H Hash names in known_hosts file.

-i Convert IETF SECSH to OpenSSH key file.

-l Show fingerprint of key file.

-M memory Amount of memory (MB) to use for generating DH-GEX moduli.

-N phrase Provide new passphrase.

-P phrase Provide old passphrase.

-p Change passphrase of private key file.

-q Quiet.

-R hostname Remove host from known_hosts file.

-r hostname Print DNS resource record.

-S start Start point (hex) for generating DH-GEX moduli.

-T file Screen candidates for DH-GEX moduli.

-t type Specify type of key to create.

-v Verbose.

-W gen Generator to use for generating DH-GEX moduli.

-y Read private key file and print public key.
copying your ID to 192.168.0.244

please enter your 'root' user password on '192.168.0.244' at the prompt

0

/usr/bin/ssh-copy-id: ERROR: No identities found
Install cronjob to automate the nightly backups

install y or n: n


What does that mean ?

And will my backup file be copied to my remote backup server ?
My server is UBUNTU 6.06.


Again, Nice job
John
Post Reply