[SOLVED] Yet Another Backup Script for Community Version

kazooless · Post by **kazooless** » Wed Jan 22, 2014 10:07 am

I recently upgraded from v6 to 8.06 and am having major performance issues. I've narrowed them down to running this backup (which I've been using for years now). I've updated my script to Mr. Twister's (immediately preceding this reply) zmbak_v.0.8.1.sh.txt and it helped a bit. However, when the rsync runs, even on the differential, it is taking an hour now when it used to take 2 to 3 minutes. This morning, after the diff job kicked off at 3am, it is still running 5 hours later and I can't even ssh in or get to the console. The hard drive is churning and I can still ping, so I am afraid to do a hard reset.
I am not sure what to look at next. Is there a switch I can turn on in the script to make the logging more verbose? I am definitely skipping the mdb's in the initial copy.
Thanks,
kazooless
LOG FILE:
[QUOTE]============================

Tue Jan 21 03:00:03 PST 2014

Performing DIFF backup

============================

diff Backup started at: Tue Jan 21 03:00:03 PST 2014

Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt

Doing a hotsync of /opt/zimbra/ to /Backup/fakebackup/

Disabling the Zimbra crontab...

Stopping the Zimbra server...Host mail.domain.com

Stopping vmware-ha...Done.

Stopping zmconfigd...Done.

Stopping stats...Done.

Stopping mta...Done.

Stopping spell...Done.

Stopping snmp...Done.

Stopping cbpolicyd...Done.

Stopping archiving...Done.

Stopping opendkim...Done.

Stopping antivirus...Done.

Stopping antispam...Done.

Stopping proxy...Done.

Stopping memcached...Done.

Stopping mailbox...Failed.

Stopping mailboxd...done.

Stopping mysqld...failed.

Stopping logger...Done.

Stopping ldap...Done.

zmcontrol shutdown had an error!

Trying to stop services again.

Unable to determine enabled services from ldap.

Enabled services read from cache. Service list may be inaccurate.

Doing a fast cold sync...

Copying databases (MDB files)...

data/ldap/mdb/db/ - Remove old... Copy new...

data/ldap/mdb.prev/db/ - Remove old.../bin/rm: cannot remove `/Backup/fakebackup/data/ldap/mdb.prev/db/*': No such file or directory

Copy new...

mdb_env_open failed, error 2 No such file or directory

MDBs copied

Reinstating Zimbra's crontab...

Starting Zimbra...

Host mail.kazules.com

Starting ldap...Done.

Starting zmconfigd...Done.

Starting logger...Done.

Starting mailbox...Done.

Starting antispam...Done.

Starting antivirus...Done.

Starting opendkim...Done.

Starting snmp...Done.

Starting spell...Done.

Starting mta...Done.

Starting stats...Done.

Service down time was - Hr:0 Min:57 Sec:11

Running A hack... This one to check and start Stats subsystem

Hack not needed Stats seems to be running fine...

Services that are not running

antivirus[/QUOTE]

Mr.Twister · Post by **Mr.Twister** » Tue Jan 28, 2014 3:48 am

Hello!
Script has variable MDB_DIR - it contains list of directories with MDB bases.

MDB_DIR="/data/ldap/mdb/db/ /data/ldap/mdb.prev/db/"
Please correct it according to your installation. I think - you can simply remove second item (with mdp.prev).

Mr.Twister · Post by **Mr.Twister** » Tue Jan 28, 2014 3:51 am

New version.
# VERSION: .0.8.2 , proposed 2014-01-28 AVV

#

# - fixed bug with collecting to media
zmbak_v.0.8.2.sh.txt

kazooless · Post by **kazooless** » Tue Jan 28, 2014 5:52 pm

I didn't think that error was meaningful to the problems I am experiencing. I've taken out the "/data/ldap/mdb.prev/db/" directory from my script and will see how it runs tonight. The exclusion of .mdb's from the rsync is included in the script already, yet it seems that the rysync is what is taking forever now (1 hour vs. just a few minutes previously).
Is there a way to increase the verbosity in the log or add time stamps to each line?
Thanks,
kazoo

TheChief79 · Post by **TheChief79** » Wed Jan 29, 2014 6:08 am

Thank you for your backup script. Was looking for something.
I get this error:
Copying databases (MDB files)...

/data/ldap/mdb/db/ - Remove old.../bin/rm: cannot remove `/tmp/fakebackup//data/ldap/mdb/db/*': No such file or directory

Copy new...

/data/ldap/mdb.prev/db/ - Remove old.../bin/rm: cannot remove `/tmp/fakebackup//data/ldap/mdb.prev/db/*': No such file or directory

Copy new...

MDBs copied
Do i have to set the other mdb paths? Both paths exist. But maybe directories are empty while backup?

kkimmell · Post by **kkimmell** » Thu Jan 30, 2014 11:55 am

[quote user="TheChief79"]Thank you for your backup script. Was looking for something.
I get this error:
Copying databases (MDB files)...

/data/ldap/mdb/db/ - Remove old.../bin/rm: cannot remove `/tmp/fakebackup//data/ldap/mdb/db/*': No such file or directory

Copy new...

/data/ldap/mdb.prev/db/ - Remove old.../bin/rm: cannot remove `/tmp/fakebackup//data/ldap/mdb.prev/db/*': No such file or directory

Copy new...

MDBs copied
Do i have to set the other mdb paths? Both paths exist. But maybe directories are empty while backup?[/QUOTE]
Just a guess here but those double //'s might be the issue?

kkimmell · Post by **kkimmell** » Thu Jan 30, 2014 1:59 pm

So now my issue which seems to still be lingering a few pages back is with the dummy mta setup on Ubuntu (or any) server.
How is it that I will be able to install another mta without having an effect on the mta that Zimbra uses? I want to get this backup routine going but am getting stuck on the install when it comes to the mta. The URL referenced in the script is very old and as far as I can tell, the link is no longer working. What are people currently using to get past this?
I am running 8.0.5 but am trying to get my backup routine in place so I can upgrade to 8.0.6.
This more recent link (http://www.zimbra.com/forums/administra ... dummy.html) is saying to just install sendmail but he then references the same older link with the .deb file. Can anyone confirm how they've setup mailx to work on Ubuntu 12.04 or later and with Zimbra 8.0 or later?
Thanks,

KKimmell

kkimmell · Post by **kkimmell** » Thu Jan 30, 2014 2:58 pm

[quote user="kkimmell"]So now my issue which seems to still be lingering a few pages back is with the dummy mta setup on Ubuntu (or any) server.
How is it that I will be able to install another mta without having an effect on the mta that Zimbra uses? I want to get this backup routine going but am getting stuck on the install when it comes to the mta. The URL referenced in the script is very old and as far as I can tell, the link is no longer working. What are people currently using to get past this?
I am running 8.0.5 but am trying to get my backup routine in place so I can upgrade to 8.0.6.
This more recent link (http://www.zimbra.com/forums/administra ... dummy.html) is saying to just install sendmail but he then references the same older link with the .deb file. Can anyone confirm how they've setup mailx to work on Ubuntu 12.04 or later and with Zimbra 8.0 or later?
Thanks,

KKimmell[/QUOTE]
p.s. I'm assuming that we can't use the Zimbra/postfix mta as the Zimbra services are shut down to perform the copy.

kazooless · Post by **kazooless** » Thu Jan 30, 2014 5:31 pm

I was able to get it working again by installing the bsd-mailx suggested when trying to install mailx:
root@mail:/var/log# apt-get install mailx

Reading package lists... Done

Building dependency tree

Reading state information... Done

Package mailx is a virtual package provided by:

mailutils 1:2.1+dfsg1-4ubuntu1

heirloom-mailx 12.4-1.1

bsd-mailx 8.1.2-0.20090911cvs-2ubuntu1

You should explicitly select one to install.

E: Package mailx has no installation candidate
Try: apt-get install bsd-mailx
[quote user="kkimmell"]So now my issue which seems to still be lingering a few pages back is with the dummy mta setup on Ubuntu (or any) server.
How is it that I will be able to install another mta without having an effect on the mta that Zimbra uses? I want to get this backup routine going but am getting stuck on the install when it comes to the mta. The URL referenced in the script is very old and as far as I can tell, the link is no longer working. What are people currently using to get past this?
I am running 8.0.5 but am trying to get my backup routine in place so I can upgrade to 8.0.6.
This more recent link (http://www.zimbra.com/forums/administra ... dummy.html) is saying to just install sendmail but he then references the same older link with the .deb file. Can anyone confirm how they've setup mailx to work on Ubuntu 12.04 or later and with Zimbra 8.0 or later?
Thanks,

KKimmell[/QUOTE]

kkimmell · Post by **kkimmell** » Fri Jan 31, 2014 7:28 am

So an install of bsd-mailx with whatever dependencies Ubuntu grabs will get me a working copy of mailx that in no way effects Zimbra's functionality nor the ability of the server to send and receive emails through Zimbra?
Thanks,

KK