[SOLVED] Yet Another Backup Script for Community Version

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
blason
Outstanding Member
Outstanding Member
Posts: 265
Joined: Sat Sep 13, 2014 2:13 am

[SOLVED] Yet Another Backup Script for Community Version

Post by blason »

OK - I am facing issue while stopping the services. I belive this was already covered up but my log shows

Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt

Doing a hotsync of /opt/zimbra/ to /opt/Backup/tmp/fakebackup/

Disabling the Zimbra crontab...

Stopping the Zimbra server...


Host mail.goodrichindia.com

Stopping zmconfigd...Done.

Stopping stats...Done.

Stopping mta...Done.

Stopping spell...Done.

Stopping snmp...Done.

Stopping cbpolicyd...Done.

Stopping archiving...Done.

Stopping antivirus...Done.

Stopping antispam...Done.

Stopping imapproxy...Done.

Stopping memcached...Done.

Stopping mailbox...Failed.

Stopping mailboxd...done.

Stopping mysqld...failed.



Stopping logger...Done.

Stopping ldap...Done.

zmcontrol shutdown had an error!
It seems the server was not able to stop the services properly. Does any one have fix for this? This happening almost every night.
Elithrin
Posts: 1
Joined: Sat Sep 13, 2014 3:01 am

[SOLVED] Yet Another Backup Script for Community Version

Post by Elithrin »

Hello
Sorry for my english.
I run the zmbak_v.0.8.sh, but, my zimbra_dars is empty.
root@zimbra:/Backup# ./zmbak_v.0.8.sh -f

============================

jeudi 29 novembre 2012, 13:52:24 (UTC+0100)

Performing FULL backup

============================

full Backup started at: jeudi 29 novembre 2012, 13:52:24 (UTC+0100)

No old backups found

Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt

Doing a hotsync of /opt/zimbra/ to /tmp/fakebackup/

Disabling the Zimbra crontab...

Stopping the Zimbra server...
Host zimbra.evxonline.net

Stopping vmware-ha...Done.

Stopping zmconfigd...Done.

Stopping stats...Done.

Stopping mta...Done.

Stopping spell...Done.

Stopping snmp...Done.

Stopping cbpolicyd...Done.

Stopping archiving...Done.

Stopping antivirus...Done.

Stopping antispam...Done.

Stopping proxy...Done.

Stopping memcached...Done.

Stopping mailbox...Done.

Stopping logger...Done.

Stopping ldap...Done.

Had to kill some left over procs...

Doing a fast cold sync...

Reinstating Zimbra's crontab...

Starting Zimbra...

Host zimbra.evxonline.net

Starting ldap...Done.

Starting zmconfigd...Done.

Starting logger...Done.

Starting mailbox...Done.

Starting memcached...Done.

Starting proxy...Done.

Starting antispam...Done.

Starting antivirus...Done.

Starting snmp...Done.

Starting spell...Failed.

Starting apache...(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:7780

no listening sockets available, shutting down

AH00015: Unable to open logs

failed.


Starting mta...Done.

Starting stats...Done.

There was an error starting Zimbra!

Null message body; hope that's ok
Do you have any idea for this problem ?
Thanks
blason
Outstanding Member
Outstanding Member
Posts: 265
Joined: Sat Sep 13, 2014 2:13 am

[SOLVED] Yet Another Backup Script for Community Version

Post by blason »

[quote user="heinzg"]Hi All
I have written the following script to suit my backup needs (it is actually quite a simple script...). I want to publish it here so you can have a good :D and maybe help improve on it and add functionality to it.
I admit some of the ideas come from existing scripts and ideas from forums.
I will maintain a current working version of the script here: Home*-*Open Source Office Lösungen
The Script Download now available again!
Script features:



Backups with or without strong encryption!

Optional Off-site copying of archives after creation

MD5 checksums for integrity checks of archives

Weekly backup rotation - 1 Full & 6 Diff's per rotation

Email report on Full backup

Email notifications on errors

Backup file lists (attached to weekly full backup report)

Installer & Setup option for quick deployment (install needed software and setup env e.g. ssh pki auth and cronjob helper)



Here is the code up to date code:



#!/bin/bash

## *** Info ***

# USAGE: -h or --help for help & usage.

# -f or --full for Full backup.

# -d or --diff for Diff backup.

# -V or --version for version info.

# --INSTALL for script install and setup.

#

# This is a backup script for the FOSS version of Zimbra mail server.

# The script is free and open source and for use by anyone who can find a use for it.

#

# THIS SCRIPT IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS

# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT

# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR

# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT

# HOLDERS AND/OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,

# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,

# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;

# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED

# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,

# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF

# THE USE OF THIS DOCUMENT, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

#

# What this script does:

# 1. Makes daily off-line backups, at a service downtime of ~
# 2. Weekly backup cycle - 1 full backup & 6 diffs.

# 3. Predefined archive sizes, for writing backups to CD or DVD media...

# 4. Backup archive compression.

# 5. Backup archive encryption.

# 6. Backup archive integrity checks and md5 checksums creation.

# 7. Automated DR - Off-site copy of backup archives via ssh.

# 8. Install and setup function for needed sofrware (Ubuntu Systems only)

# 9. Weekly eMail report & eMail on error - including CC address.

#

# This script makes use of following tools:

# apt-get, cron, dar, dpkg, mailx, md5sum, rsync, ssh, uuencode, wget, zimbra mta.

#

# We have opted to use a pre-sync directory to save on "down time", but this

# causes one to have huge additional space usage.

# But hard drives are cheep today!

#

# What is still to come or needs work on:

# 1. Recover option

# 2. Better documentation

#

#------- CONFIG -------#

# Edit this part of the script to fit your needs.

#
#--- Directories ---#

# Please add the trailing "/" to directories!

ZM_HOME=/opt/zimbra/

SYNC_DIR=/srv/backup/sync/

ARCHIVEDIR=/srv/backup/dars/

TOO_MEDIA_DIR=/srv/backup/burn/
#

#--- PROGRAM OPTIONS ---#

RSYNC_OPTS="-aHK --delete --exclude=*.pid"
#

#--- ARCHIVE NAMES ---#

BACKUPNAME="Zimbra_Backup"

BACKUPTYPE_F="FULL" # name prefix to sort between full and diff backups

BACKUPTYPE_D="DIFF"

BACKUPDATE=`date +%d-%B-%Y`
#

#--- ARCHIVE SIZE ---#

# storage media size

ARCHIVESIZE="4395M"

COMPRESS="9" # valid answers are 1 - 9 ( 9 = best )
CRYPT="yes" # valid answers are "yes" or "no"

PASSDIR=/etc/zmbac/

PASSFILE="noread"
#

#--- EMAIL ADDRESS ---#

EMAIL="admin@localhost"

EMAILCC="admin@remotehost"

LOG="/var/log/zim_backup.log"
#--- SSH REMOTE DR COPY ---#

# This option will secure copy you archives to a remote server via 'scp'

DRCP="yes" # valid answers are "yes" or "no"

SSHUSER="heinzg"

REMOTEHOST="172.16.184.1"

REMOTEDIR="/tmp/"


#--- USE HACKS !?! ---#

# Built in hacks to fix common problems

#Hack to start Stats, even run zmlogprocess if needed

STATHACK="yes" # valid answers are "yes" or "no"


## ~~~~~!!!! SCRIPT RUNTIME !!!!!~~~~~ ##

# Best you don't change anything from here on,

# ONLY EDIT IF YOU KNOW WHAT YOU ARE DOING

#

#
!!! the script is to long to fit into the post!! To view the full script visit my development site link is above.


Here is a sample print out of the log file after a full & diff backup:



Full Backup started at: 23:21
No old backups found
Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt
Doing a hotsync of /opt/zimbra/ to /srv/backup/fakebackup/

sync went ok
Stopping the Zimbra server
Deactivated zimbra's cronjobs
Host mail.igrill.local

Stopping stats...Done

Stopping mta...Done

Stopping spell...Done

Stopping snmp...Done

Stopping archiving...Done

Stopping antivirus...Done

Stopping antispam...Done

Stopping imapproxy...Done

Stopping mailbox...Done

Stopping logger...Done

Stopping ldap...Done
Zimbra server has stopped
kill any left over zimbra procs

Nothing to kill
Doing a fast cold sync...

sync went ok
Reinstated zimbra's cronjobs
Host mail.igrill.local

Starting ldap...Done.

Starting logger...Done.

Starting mailbox...Done.

Starting antispam...Done.

Starting antivirus...Done.

Starting snmp...Done.

Starting spell...Done.

Starting mta...Done.

Starting stats...Done.
Zimbra has started
Service down time was - Hr:0 Min:2 Sec:16
Running A hack... This one to check and start Stats subsystem

Hack not needed Stats seems to be running fine...
All services are running


Writing a full backup called:

12_Zimbra_Backup_30-March-2008_FULL

in: /srv/backup/zimbra_dars/ with file sizes of max: 4395M

Archive is Encrypted


--------------------------------------------

19271 inode(s) saved

with 1115 hard link(s) recorded

0 inode(s) changed at the moment of the backup

0 inode(s) not saved (no inode/file change)

0 inode(s) failed to save (filesystem error)

0 inode(s) ignored (excluded by filters)

0 inode(s) recorded as deleted from reference backup

--------------------------------------------

Total number of inode considered: 19271

--------------------------------------------

EA saved for 0 inode(s)

--------------------------------------------

Archive Created successfully
Create MD5 Checksum for 12_Zimbra_Backup_30-March-2008_FULL.1.dar
MD5 Checksum Created successfully
copy archive to 172.16.184.1 remote directory /tmp/
Create file listing from archive
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

Full Zimbra Backup ended at: 23:42

Backup took Hr:0 Min:21 Sec:27 to complete

:::::::::::::::::Cheers Osoffice for the script:::::::::::::::::::::::


Diff Backup started at: 23:46
Doing a hotsync of /opt/zimbra/ to /srv/backup/fakebackup/
sync went ok
Stopping the Zimbra server
Deactivated zimbra's cronjobs
Host mail.igrill.local

Stopping stats...Done

Stopping mta...Done

Stopping spell...Done

Stopping snmp...Done

Stopping archiving...Done

Stopping antivirus...Done

Stopping antispam...Done

Stopping imapproxy...Done

Stopping mailbox...Done

Stopping logger...Done

Stopping ldap...Done
Zimbra server has stopped
kill any left over zimbra procs

Nothing to kill
Doing a fast cold sync...

sync went ok
Reinstated zimbra's cronjobs
Host mail.igrill.local

Starting ldap...Done.

Starting logger...Done.

Starting mailbox...Done.

Starting antispam...Done.

Starting antivirus...Done.

Starting snmp...Done.

Starting spell...Done.

Starting mta...Done.

Starting stats...Done.
Zimbra has started
Service down time was - Hr:0 Min:2 Sec:15
Running A hack... This one to check and start Stats subsystem

Hack not needed Stats seems to be running fine...
All services are running


Writing a diff backup called:

12_Zimbra_Backup_30-March-2008_DIFF

in: /srv/backup/zimbra_dars/ with file sizes of max: 4395M

Archive is Encrypted

Warning, the archive 12_Zimbra_Backup_30-March-2008_FULL has been encrypted. A wrong key is not possible to detect, it would cause DAR to report the archive as corrupted


--------------------------------------------

154 inode(s) saved

with 1115 hard link(s) recorded

0 inode(s) changed at the moment of the backup

19118 inode(s) not saved (no inode/file change)

0 inode(s) failed to save (filesystem error)

0 inode(s) ignored (excluded by filters)

5 inode(s) recorded as deleted from reference backup

--------------------------------------------

Total number of inode considered: 19277

--------------------------------------------

EA saved for 0 inode(s)

--------------------------------------------

Archive Created successfully
Create MD5 Checksum for 12_Zimbra_Backup_30-March-2008_DIFF.1.dar
MD5 Checksum Created successfully
copy archive to 172.16.184.1 remote directory /tmp/
Create file listing from archive
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

Diff Zimbra Backup ended at: 23:50

Backup took Hr:0 Min:4 Sec:26 to complete

:::::::::::::::::::Cheers Osoffice the script:::::::::::::::::::::::::
I hope this helps someone and have fun!

( Note the service downtime was due to a very weak vmware guest running on my laptop.... So it is faster in real life :cool: )
Cheers

Heinzg[/QUOTE]


This is failing only for Diff Backup however works perfectly fine with Full backup. Any clue why?
blason
Outstanding Member
Outstanding Member
Posts: 265
Joined: Sat Sep 13, 2014 2:13 am

[SOLVED] Yet Another Backup Script for Community Version

Post by blason »

Hi Team,
Just curious to know if the same script has ever been tried with ZCS v8.0? Was the backup and restore successful? Are we safe to use it in production environment?
50asm
Posts: 26
Joined: Sat Sep 13, 2014 2:23 am

[SOLVED] Yet Another Backup Script for Community Version

Post by 50asm »

[quote user="blason"]Hi Team,
Just curious to know if the same script has ever been tried with ZCS v8.0? Was the backup and restore successful? Are we safe to use it in production environment?[/QUOTE]
I was wondering the same thing...
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

[SOLVED] Yet Another Backup Script for Community Version

Post by phoenix »

[quote user="50asm"]I was wondering the same thing...[/QUOTE]Test it and find out, the only way to know if (and how) it works in your environment is to test it in your environment.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
blason
Outstanding Member
Outstanding Member
Posts: 265
Joined: Sat Sep 13, 2014 2:13 am

[SOLVED] Yet Another Backup Script for Community Version

Post by blason »

Hi Guys,
Not sure but this is really making me crazy. when I run script manually that is not through crontab it works fine but if I run it through crontab it fails. Any clue why? I am really running out of ideas now
m0ps
Posts: 9
Joined: Sat Sep 13, 2014 3:00 am

[SOLVED] Yet Another Backup Script for Community Version

Post by m0ps »

[quote user="blason"]Hi Guys,
Not sure but this is really making me crazy. when I run script manually that is not through crontab it works fine but if I run it through crontab it fails. Any clue why? I am really running out of ideas now[/QUOTE]
add to crontab PATH variable like:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
m0ps
Posts: 9
Joined: Sat Sep 13, 2014 3:00 am

[SOLVED] Yet Another Backup Script for Community Version

Post by m0ps »

after update from 8.0.1 to 8.0.2 i got an error with backup report sending. in log i see those message:

[QUOTE]Can't send mail: sendmail process failed with error code 75[/QUOTE]
via shell i can send mail with

[QUOTE]mail -s "Zimbra FULL Backup Log on zimbra.host.com" admin@zimbra.host.com[/QUOTE]
is anybody faced a similar problem?
blason
Outstanding Member
Outstanding Member
Posts: 265
Joined: Sat Sep 13, 2014 2:13 am

[SOLVED] Yet Another Backup Script for Community Version

Post by blason »

You mean like this
SHELL=/bin/bashPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin30 18 * * 1 /bin/bash /opt/scripts/zmbak_v.0.8.sh -f > /var/log/zim_backup.log 2>&1

35 22 * * 2-7 /bin/bash /opt/scripts/zmbak_v.0.8.sh -d >> /var/log/zim_backup.log 2>&1
Post Reply