Upgrade from 8.0.6 to 8.8.6

[L. Mark Stone]

You don't say if you are running Network Edition or FOSS, but if you are running Network Edition it is much, much easier to do a side-by-side migration to a totally new server running 8.8.7 and whatever supported OS you want. The BackupNG tools (Zextras, essentially), will have you install an agent on your old server and take a Zextras backup; copy/transport the backup to the new server and restore.

Unlike the old Network Edition backup/restore, the Zextras BackupNG tools restore sharing, distribution lists and all the other bits the Network Edition backup does not.

BackupNG can also be done in an Incremental manner, whereby you provision the mailboxes on the new server to make the server usable and effect your cutover, and then you restore the data. The BackupNG backups work like rsync as well, so you could restore on Friday the backup you took on Thursday, do the cutover, then take an incremental backup and do an incremental restore.

Hope that helps,
Mark

Hi Mark,
I am running the Network Edition at 8.0.9 [Centos6.5 VM on Xen5.5] and want to upgrade to a new Centos7 VM server with Network Edition 8.8.7. (on VMWare ESXi6.0) (Nota Bene: I started with Zimbra 7 back in 2011 and have upgraded a few times on that same server, but have left it alone for a number of years now. Zimbra has just 'worked' flawlessly all this time, but it is now high time to upgrade)

What you have described sounds great, but I am not finding any reference material for this. Can you recommend some good links that do a good job outlining the steps?
(My install is a small corporate install, single server, with only 130 accounts. However accounts are large and I have about 250-300Gigs of message data. I do use HSM to a NAS for older message storage)

TIA
Gary

Hi Gary,

What you want to use is the Incremental Backup/Migration methodology in the github-based Zimbra Admin Guide. Knowing that you've been running Zimbra since version 7.x I would strongly recommend NOT doing an in-place upgrade, and instead use the BackupNG Incremental Restore method to do a side-by-side migration to a new Zimbra server.

Admin Guide Link: https://zimbra.github.io/adminguide/lat ... ith-backup

Hope that helps,
Mark

[gabaker]

Hi Gary,

What you want to use is the Incremental Backup/Migration methodology in the github-based Zimbra Admin Guide. Knowing that you've been running Zimbra since version 7.x I would strongly recommend NOT doing an in-place upgrade, and instead use the BackupNG Incremental Restore method to do a side-by-side migration to a new Zimbra server.

Admin Guide Link: https://zimbra.github.io/adminguide/lat ... ith-backup

Hope that helps,
Mark

Thanks much for the link Mark. Looks very promising. I am about 2 weeks out from making this move. I was wondering one other thing. The guide talks about getting HSM going after the migrations, but I am wondering how my current HSM plays into my migration. Do I need enough storage on the new server to hold all the HSM data that will be part of the backup? It is very confusing to me. I have about 250Gigs in HSM. Do I need to create a primary mail store on the new server to accommodate all this data and then start HSM *after* the migration or is there some way the new server will just place the HSM data into a new HSM share as the data is moved?

Thanks
Gary

[L. Mark Stone]

Hi Gary,

What you want to use is the Incremental Backup/Migration methodology in the github-based Zimbra Admin Guide. Knowing that you've been running Zimbra since version 7.x I would strongly recommend NOT doing an in-place upgrade, and instead use the BackupNG Incremental Restore method to do a side-by-side migration to a new Zimbra server.

Admin Guide Link: https://zimbra.github.io/adminguide/lat ... ith-backup

Hope that helps,
Mark

Thanks much for the link Mark. Looks very promising. I am about 2 weeks out from making this move. I was wondering one other thing. The guide talks about getting HSM going after the migrations, but I am wondering how my current HSM plays into my migration. Do I need enough storage on the new server to hold all the HSM data that will be part of the backup? It is very confusing to me. I have about 250Gigs in HSM. Do I need to create a primary mail store on the new server to accommodate all this data and then start HSM *after* the migration or is there some way the new server will just place the HSM data into a new HSM share as the data is moved?

Thanks
Gary

That's a great question Gary and you are right that the documentation isn't exactly clear on that.

The fact is that storage volume layouts are mailbox-server-specific, so you could have half a dozen primary and HSM volumes on the old source server, but no need to replicate that on the new target server. All the mail blobs on the new target will be deposited to /opt/zimbra/store unless and until you do something about them _on the new server itself_.

Short answer is that your HSM setup on your old server is essentially ignored, though the mail blobs therein are of course migrated.

Hope that helps,
Mark

[gabaker]

Mark,

Just wanted to follow up. I have successfully done the migration of my 8.0.9 Zimbra server from CentOS-6 on a Xen Host to Zimbra 8.8.8 on Centos-7 on a VMware ESXi server Using the Zimbra Admin doc mentioned.
Here are some notes:

1. I installed the Zextras 'Legacy' package on the original server and had the Zimbra Network NG Backup module running on the new server
2. I set the backup directory on the old server as a mounted NFS share from a local NAS
3. While some people have said that the Zextras backup works like rsync, it does not remove deleted items. I had to set the retention to 1-day for deleted items and run a purge right before starting the import on the new server.
4. Right before the import to the new server I ran a "zxsuite backup doSmartScan deep true" on the old server to fixup the data. This took about an hour for 755,000 items in the db.
5. I then stopped zimbra services on the old server to stop mail flow. (My firewall mail proxy then started spooling incoming mail)
6. I mounted the NFS directory containing the old server backup to the new server
7. In doing the import on the new server I started with the standard import command. I found this to be very slow. After 6 hours of import I only had 250,000 of 755,000 items restored. I also noticed that CPU utilization of my 2-vCPU machine was 100% most of the time. So I forced a stop of the import, shutdown the server and added 6 more vCPUs (host has dual 8-core CPUs) and brought the RAM to 20Gigs. Then I started it back up and started the restore again, this time using the new added flag "concurrent_accounts" set to 4. My server then flew through the next 500,000 items in 4 hours. So I would definitely recommend using a much more robust VM setting for the external restore process. After the restore, I set the VM sizing back to what I usually use in production.
8. out of 755,000 items, I only had 11 unrestored items!
9. my question regarding HSM, above is moot. The data is pulled into the backup out of the HSM and is restored to the new server in the default mailstore first. HSM rules would need to be setup after that.
10. On my Network Edition, the NG Mobile activation button seems to not do anything. However following the wiki for activation (https://wiki.zimbra.com/wiki/Zimbra_NG_ ... _NG_Mobile) I pushed the button and then did the CLI Command. Upon Restarting the server the Mobile NG was activated.
11. You must enable the Zextras_Client zimlet on all COS and user account in order to Sync Shared anything (mail, contacts, calendar) on mobile devices. Then the share Sync must be enabled by the user using right-click and selecting Synchronization. Previously, I could sync shared contacts with iOS devices, but this did not work anymore with either version in 8.8 (Zimbra MobileSync or MobileNG)

The migration took about 12 hours total for all the preset work, migration and fixups after the fact. All-in all it was a smooth process and all my users seem quite satisfied. No one has lost anything! One thing to remember is that your internal DNS MX record should also point to your SPAM filter/Firewall whatever you use so that all the various internal systems that send email have a place to go while your main server is down for the migration.

Only a couple of bugaboos remain

A) I am getting two backup failure alerts. One for Chat (which we do not even have enabled as far as I know... we are not licensed for ZimbraTalk). The error is "Table 'zimbra.MESSAGE' doesn't exist... sql : 'SELECT * FROM MESSAGE' "
Full message:

Code: Select all

Operation Backup Chat Server Failed.

Operation Id: 38a9e947-9879-48dc-a46d-6929bb374827
Operation Host: zimbra2.MYDOMAIN.com
Scheduled operation

Network Modules NG Version: 2.8.0
Network Modules NG commit: 772e7329112d3ca56a07cbcec8731b9e59b39a50
ZAL Version: 2.2.1
ZAL commit: e92f292ddefac84a68283b33136bb5ec6a4b98f9
Zimbra version: 8.8.8_GA_2026 20180405163320 20180405-1651 NETWORK

Error : java.sql.SQLSyntaxErrorException: Table 'zimbra.MESSAGE' doesn't exist
Query is:
sql : 'SELECT * FROM MESSAGE'

B) Also getting this error on Backup Server Customizations: "Error : File does not exist /opt/zimbra/conf/templates/abook"
full message:

Code: Select all

 
This is an automated notification from Backup about Backup Server Customizations.

Operation Backup Server Customizations Failed.

Operation Id: 1e3c29fc-874a-49e3-b053-39cb004147eb
Operation Host: zimbra2.MYDOMAIN.com
Scheduled operation

Network Modules NG Version: 2.8.0
Network Modules NG commit: 772e7329112d3ca56a07cbcec8731b9e59b39a50
ZAL Version: 2.2.1
ZAL commit: e92f292ddefac84a68283b33136bb5ec6a4b98f9
Zimbra version: 8.8.8_GA_2026 20180405163320 20180405-1651 NETWORK

Error : File does not exist /opt/zimbra/conf/templates/abook

C) Also, in the Admin Console for Zimbra Network under "General" it shows Admin module "Stopped" but Backup, Chat, Mobile, and HSM are Running. Wondering if this is normal?

I may be submitting a Support request for the above items.

Thanks for your help and answers Mark. I appreciate it. Other than the above, all seems to being going swimmingly with the new server. I am happy with the results.

Cheers!
Gary

[L. Mark Stone]

Hi Gary,

Glad it all went well for you (excluding the minor issues at the end). I would definitely open a Support Case for those. I've seen some of them before, and as I've been doing several migrations recently I don't recall offhand what I did where to fix those, but they are either all fixable or benign.

I have however found that it seems better to "upgrade" to Talk from Chat, even without a Talk license, but the only issue I'm having is that for one client who doesn't want IM, it's not (yet) possible to turn off the IM pane for that customer. I'm working with Zimbra Support on that one.

The Zextras Export/Import process does have the rsync attribute that when run twice it won't create duplicates, but you are right that it won't remove items on the target server that were deleted on the source server after the first export/import. FWIW impasysnc doesn't do that either -- unless you add the "--delete2" switch. But yes, would be nice if the Zextras tool did that.

I have also found like you that the restore is very CPU intensive, but some of that is I/O from the restore writing both to the Zimbra Store and the redologs. If you put either of those on a separate partition you'll see throughput improvements. On XenServer, each VM disk gets one tapdisk process, so tapdisk becomes the bottleneck in Dom0. VMware I'm told has a similar structure. More VM disks = better throughput.

If you don't mind, once you sort out the rest of the minor issues, if you wouldn't mind posting what you did, that would be a helpful reference I am sure. -- At least until 8.8.9 or so comes out!

Alll the best,
Mark

[gabaker]

Zimbra Support has the following response for the Chat backup fail email:

"The fix for this issue has been pushed and will be included in Zimbra 8.8.8 p2, in the meantime please be assured that the backup is running and operational as the failure only affects the Chat backup."

Have not heard about the other two items though.

-Gary

[gabaker]

Response from support:

(1) The "SELECT * FROM MESSAGE" issue has been fixed in Zimbra 8.8.8p2.

(2) The /opt/zimbra/conf/templates/abook issue is a Zimbra permission issue that can be fixed with a chmod a+x of the templates subdirectory.

Code: Select all

chmod a+x /opt/zimbra/conf/templates/

(3) The Admin module must be enabled manually, it's normal for it to be displayed as Stopped if the Legacy Admin module is still being used.

8.8.8 Patch 2 Docs: https://wiki.zimbra.com/wiki/Zimbra_Releases/8.8.8/P2

To enable the NG Admin console, see: https://wiki.zimbra.com/index.php?title ... NG_Modules at the bottom of the page.

Hope this helps anyone else.

Gary