How to properly backup Zimbra?

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
mateusscheper
Posts: 17
Joined: Tue May 29, 2018 12:49 pm

How to properly backup Zimbra?

Postby mateusscheper » Fri Jan 11, 2019 3:55 pm

Currently I'm copying via rdiff-backup the Zimbra directory (/opt/zimbra) to another server. But is this a good way to back it up? I'm thinking on using Bacula instead, but I'm not sure.


axslingr
Advanced member
Advanced member
Posts: 152
Joined: Sat Sep 13, 2014 2:20 am
ZCS/ZD Version: Release 8.8.11.GA.3737.UBUNTU16.64

Re: How to properly backup Zimbra?

Postby axslingr » Fri Jan 11, 2019 10:46 pm

Here's a little script I use. Nothing fancy but gets the job done. Modify for your environment and test before using. In the event of disaster, just restore /opt/zimbra and copy data.mdb back to /opt/zimbra/data/ldap/mdb/db and set permissions.

Code: Select all

#!/bin/bash

#Rsync Zimbra while still online...reduces downtime
rsync -avHK --delete --exclude 'data.mdb' /opt/zimbra /some/backup/location/

#Backup ldap database
#This line is necessary after the first database backup because mdb_copy won't delete existing file
mv /some/backup/location/ldap/data.mdb /some/backup/location/ldap/data.mdb.bak

/opt/zimbra/common/bin/mdb_copy /opt/zimbra/data/ldap/mdb/db /some/backup/location/ldap

#Stop Zimbra Services
su - zimbra -c 'zmcontrol stop'

sleep 30

#Rsync again while zimbra services are stopped
rsync -avHK --delete --exclude 'data.mdb' /opt/zimbra /some/backup/location/

#Start zimbra services
su - zimbra -c 'zmcontrol start'

exit
mateusscheper
Posts: 17
Joined: Tue May 29, 2018 12:49 pm

Re: How to properly backup Zimbra?

Postby mateusscheper » Mon Jan 14, 2019 11:04 am

axslingr wrote:Here's a little script I use. Nothing fancy but gets the job done. Modify for your environment and test before using. In the event of disaster, just restore /opt/zimbra and copy data.mdb back to /opt/zimbra/data/ldap/mdb/db and set permissions.

Code: Select all

#!/bin/bash

#Rsync Zimbra while still online...reduces downtime
rsync -avHK --delete --exclude 'data.mdb' /opt/zimbra /some/backup/location/

#Backup ldap database
#This line is necessary after the first database backup because mdb_copy won't delete existing file
mv /some/backup/location/ldap/data.mdb /some/backup/location/ldap/data.mdb.bak

/opt/zimbra/common/bin/mdb_copy /opt/zimbra/data/ldap/mdb/db /some/backup/location/ldap

#Stop Zimbra Services
su - zimbra -c 'zmcontrol stop'

sleep 30

#Rsync again while zimbra services are stopped
rsync -avHK --delete --exclude 'data.mdb' /opt/zimbra /some/backup/location/

#Start zimbra services
su - zimbra -c 'zmcontrol start'

exit


Thank you very much for your help. Your script seems great.

Just to make sure: mdb_copy makes a file with only 9,3M out of 80G. Is this correct? According to this link, it's ok, but I'm not sure how to see my mdb size.

BTW, why do you rsync two times?
spectre.nl
Posts: 2
Joined: Sun Jan 24, 2016 4:22 pm

Re: How to properly backup Zimbra?

Postby spectre.nl » Mon Jan 14, 2019 12:38 pm

Because the open source version lacks the backup I found this on github :

https://github.com/Zmbackup/zmbackup
A reliable software written in Python to help you in your daily task to backup and restore mails and accounts from Zimbra Open Source Email Platform.

works very well
axslingr
Advanced member
Advanced member
Posts: 152
Joined: Sat Sep 13, 2014 2:20 am
ZCS/ZD Version: Release 8.8.11.GA.3737.UBUNTU16.64

Re: How to properly backup Zimbra?

Postby axslingr » Mon Jan 14, 2019 12:57 pm

mateusscheper wrote:
Just to make sure: mdb_copy makes a file with only 9,3M out of 80G. Is this correct? According to this link, it's ok, but I'm not sure how to see my mdb size.



That's correct.

Code: Select all

du -hs /opt/zimbra/data/ldap/mdb/db


This will show you the real size.

BTW, why do you rsync two times?


The first time is while Zimbra is still running, to get the majority of the changes and reduce the time Zimbra is offline during the second rsync.

Lance
halfgaar
Advanced member
Advanced member
Posts: 82
Joined: Sat Sep 13, 2014 12:54 am
Location: Netherlands
ZCS/ZD Version: Release 8.8.12.GA.3794.UBUNTU16.64

Re: How to properly backup Zimbra?

Postby halfgaar » Wed May 08, 2019 8:21 pm

This thread is a bit old, but I think it deserves a continuation.

The Zmbackup tool (actually written in Bash) uses the REST interface for account export, and that has been broken and will remain broken, it seems. See this bug report: REST interface has critical flaws. I wrote a short script using that too, but I recently found it that is useless.

I've been Googling for backup strategies/scripts, and I find it all very rudimentary, and when it comes to disaster recovery, mostly inferior to my virtual machine snapshots.

I'm left with the conclusion that there is no per-account, or even per message, backup method? For instance, if someone deletes some e-mail accidentally, and needs to perform a partial recovery, you just can't?
axslingr
Advanced member
Advanced member
Posts: 152
Joined: Sat Sep 13, 2014 2:20 am
ZCS/ZD Version: Release 8.8.11.GA.3737.UBUNTU16.64

Re: How to properly backup Zimbra?

Postby axslingr » Wed May 08, 2019 9:40 pm

halfgaar wrote:I'm left with the conclusion that there is no per-account, or even per message, backup method? For instance, if someone deletes some e-mail accidentally, and needs to perform a partial recovery, you just can't?

Not with OSS version out of the box. You can get Zimbra Suite Plus or Zextras(both are actually the same).
Lance
paul
Posts: 10
Joined: Fri Apr 12, 2019 12:15 am

Re: How to properly backup Zimbra?

Postby paul » Thu May 09, 2019 11:28 pm

halfgaar wrote:I've been Googling for backup strategies/scripts, and I find it all very rudimentary, and when it comes to disaster recovery, mostly inferior to my virtual machine snapshots.


Can I ask you about this... because I've been struggling with clean backups for Zimbra.
My Zimbra server is running in VirtualBox. I looked at snapshots, but saw a few comments that
snapshots were not a suitable long term backup mechanism.

In the end, I went with an "outside the VM" system: clean shutdown, copy the VM folder, restart.
My script then takes the copy and zips/encrypts it before moving it to a long term storage + rotating.
The server is down for a few minutes (SSDs FTW!) in the early hours - which has been fine.
I've had a few problems that zimbra does not always come back cleanly on reboot - several
times LDAP has not come up quickly and other services will fail, or a patch has come through
and mess up some file permissions (TWICE now).

I'm interested in your script around snapshots because it would reduce the reboots (and potential
startup problems).
- how/when do you clean the snapshots created? Do you do that automatically? How long do you let the chain get?
- how often do you backup/snapshot?
- are you on VMWare, VBox, HyperV?
- do you have a script you can share?

I'm happy to share my script if its of any use.

Thanks!
Paul.
halfgaar
Advanced member
Advanced member
Posts: 82
Joined: Sat Sep 13, 2014 12:54 am
Location: Netherlands
ZCS/ZD Version: Release 8.8.12.GA.3794.UBUNTU16.64

Re: How to properly backup Zimbra?

Postby halfgaar » Fri May 10, 2019 7:27 am

I looked at snapshots, but saw a few comments that
snapshots were not a suitable long term backup mechanism.


Probably when they say it's not a long term solution, they mean live copy-on-write snapshots, and retaining them. That's true, because all write operations then have to be duplicated.

My system is basically what you have, but then without downtime, and more efficient because it doesn't store unused blocks from the file system. I use Xen and my disks are on LVM (logical volume management). With LVM, you can make in instant copy-on-write frozen snapshot of any volume. So, my scripts, in essence:

* Make a snapshot
* Take the first 100 MB and dd it: that will contain partition table and boot loaders.
* use losetup to scan for partitions on it.
* Do a filesystem repair, because it 'looks' like a uncleanly shutdown file system
* Use partclone on a designated partition in the snapshot, to only copy the used blocks to a gzip file
* Delete the snapshot

This is my general VM backup mechanism, which I run daily on this VM. I complement it with daily rdiff-backup from inside the VM, including some slapcat and mysqldump commands.

I could share these scripts, but they are very specific to my situation, and are of no use if you don't use LVM on Linux.
7224jobe
Advanced member
Advanced member
Posts: 142
Joined: Sat Sep 13, 2014 1:55 am
ZCS/ZD Version: 8.6.0_GA_1153.RHEL6_64 Patch 14

Re: How to properly backup Zimbra?

Postby 7224jobe » Fri May 10, 2019 5:41 pm

halfgaar wrote:This thread is a bit old, but I think it deserves a continuation.

The Zmbackup tool (actually written in Bash) uses the REST interface for account export, and that has been broken and will remain broken, it seems. See this bug report: REST interface has critical flaws. I wrote a short script using that too, but I recently found it that is useless.

I've been Googling for backup strategies/scripts, and I find it all very rudimentary, and when it comes to disaster recovery, mostly inferior to my virtual machine snapshots.

I'm left with the conclusion that there is no per-account, or even per message, backup method? For instance, if someone deletes some e-mail accidentally, and needs to perform a partial recovery, you just can't?


Since there are no alternatives at the moment, I'd say that a strategy composed by VM or LVM snapshot + Zmbackup/REST scripts is still an "ok" backup strategy: yes, it is not guaranteed that 100% of mailbox elements are backupped by the scripts, but in my experience I did not have bad experiences using the scripts. During the years I (luckily) did only a small amount of restores and users never complained about missing elements...Moreover, the .tgz files produced by the scripts always have a reasonable size, there are no 10MB backup files of a 1GB mailbox. It is still better than only making/restoring snapshots, that are a quite long task to perform...

Return to “Administrators”

Who is online

Users browsing this forum: vijayr and 15 guests