Restore a backup Zimbra Server for testing?

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
bhwong
Advanced member
Advanced member
Posts: 151
Joined: Thu Feb 27, 2014 8:40 pm

Restore a backup Zimbra Server for testing?

Post by bhwong »

My Ubuntu 10 to 12 upgrade has failed because of Unauthenticated packages found. Thus, I would like to restore a backup for testing. In this case, beside changing the hostname and IP address of the recovered Zimbra server, what else do I need to perform to avoid conflict with the production Zimbra server and still able to function?
Brad_C
Advanced member
Advanced member
Posts: 106
Joined: Sat Sep 13, 2014 2:33 am

Restore a backup Zimbra Server for testing?

Post by Brad_C »

Oooooo. One I can answer.



The important thing is that the fqdn on the server resolves to the correct place or some of the things you do on the test server will be applied to the production server across the network. Zimbra uses dns lookups rather than checking /etc/hosts in a number of places, so you must make sure you have a properly configured and working dns to pull this off.



Let me try and explain it (I'm not very good at this).



My production server is fqdn is zimbra.vm.home. It resolves to 192.168.253.198.

When I clone the VM it gets started with a different MAC address and comes up as 192.168.253.196.

The VM runs dnsmasq internally, so zimbra.vm.home *always* resolves to 127.0.0.1.



I run a boot script that checks the MAC address, and if it has the MAC of the test server it changes the servers hostname on the fly from zimbra to ztest (don't need to do this but it makes it easy for me to identify which server I'm into when I ssh in).



It is vitally important that the FQDN resolves to the correct server which is why I use dnamasq to make sure it resolves to 127.0.0.1. Before I did this some changes I made would apply over the network to the production server (because regardless of the IP and host name, the server still thinks it's zimbra.vm.home and that always resolved to 192.168.253.198). So starting and stopping services on the test server would apply to the production server and cause havoc.



Once I made dns resolution work properly with dnsmasq, I can now clone the production VM at any time and do trial upgrades, patches and testing and have everything work properly.



There might be a better way to do it, but doing it this way means I don't have to make *any* config changes to the zimbra install to be able to switch it back and forward between test a& production.



I'm sorry it's not explained very well. I hope you get the gist.
wokie
Posts: 14
Joined: Wed Oct 28, 2015 8:20 am

Restore a backup Zimbra Server for testing?

Post by wokie »

This is not a full answer, but it shows some directions:

After cloning the server and connecting it to the network you'll have two systems with the same ip, hostname and domain.



Please make sure the system has a unique ip, hostname and domain:

- If your domain is example.com, you can introduce a new domain e.g. testexample.com by adding this domain to your DNS server. Don't forget to add MX and other specific mail records (DKIM etc.)

- Get your testerver up without network and configure your network on the console, with Centos you can change those with the nmcli command or by modifying /etc/sysconfig/network-scripts/.

- Change the hostname of your system, Centos uses the hostnamectl command for this

- Reboot the system



You can find all places in the Zimbra configuration where your systemname or ip address is used by running those commands as Zimbra user:

zmprov getallconfig | grep mail.example.com

zmprov getallconfig | grep 1.2.3.4

Please make the necessary changes with zmprov config



Go to the admin console (browse to port 7071 or 9071 on your new server) and login with the admin user

- Choose configure in the left pane

- Choose domains

- Remove the listed production domain

- Add the new testdomain



The above is not tested or verified.



Good luck
wokie
Posts: 14
Joined: Wed Oct 28, 2015 8:20 am

Restore a backup Zimbra Server for testing?

Post by wokie »

One additional remark, besides zmprov, you should check zmlocalconfig as well.
bhwong
Advanced member
Advanced member
Posts: 151
Joined: Thu Feb 27, 2014 8:40 pm

Restore a backup Zimbra Server for testing?

Post by bhwong »

Sound like this is not going to be as simple as duplicating other servers for testing. 

I have tried this before and was shocked that some commands I executed on the trial server also get applied to the production server! Thus, I'm quite worried about doing this again. Some of your replies that this is not tested and verified, and wishing me luck is getting me even more worried! 
Is there an official guide how we can do this safely to ensure that whatever we do will not affect or even get applied to the production server? If I disconnect it from the network and execute zmprov commands and connect it back to the network, it should be safe right?
Brad_C
Advanced member
Advanced member
Posts: 106
Joined: Sat Sep 13, 2014 2:33 am

Restore a backup Zimbra Server for testing?

Post by Brad_C »

[quote user="bhwong"]Sound like this is not going to be as simple as duplicating other servers for testing. [/quote]
If you take the approach of not making any changes to the zimbra configuration at all and deal with it at the DNS & hostname level it is a piece of cake. Install dnsmasq, configure to resolve your production server fqdn to 127.0.0.1, add the test server hostname into /etc/hosts along side the production server host name.
Now when you spin up the clone all you need to do is change the host name and you have a perfect copy of your production environment with *no* changes required to your running zimbra configuration.
[quote user="bhwong"]If I disconnect it from the network and execute zmprov commands and connect it back to the network, it should be safe right?[/quote]
No, it won't work. If you disconnect it from the network then zmprov won't be able to resolve the server address and the command will fail.
Frankly once you start changing the zimbra config it's no longer a clone of your production environment and all bets are off.
bhwong
Advanced member
Advanced member
Posts: 151
Joined: Thu Feb 27, 2014 8:40 pm

Restore a backup Zimbra Server for testing?

Post by bhwong »

I'm not familiar with dnsmasq. Does it isolate the clone from communicating with external network, especially the production Zimbra?

Since I'm not performing any test on Zimbra configuration issues, I think changing the zimbra config to test Ubuntu upgrade issue should be fine right? See this: https://community.zimbra.com/collaborat ... /t/1140260
wokie
Posts: 14
Joined: Wed Oct 28, 2015 8:20 am

Restore a backup Zimbra Server for testing?

Post by wokie »

If the test environment needs to be 100% the same, which I can very well imagine. Then you can think about the following steps:

- Create an test network

- Make sure only traffic from testers and yourself can enter the test network, e.g. fully isolate the network, or only allow access to client access protocols like ssh or rdp.

- Implement related services, like DNS, DHCP, NTP, LDAP/Active Directory in the test network

- Probably implement some test clients in the test network, e.g. a pc or other devices with a browser and a mail client

- Clone the Zimbra machine in the test network



In this scenario it's difficult to test internet connectivity.



If you provide more details about your exact needs, probably the answer can be finetuned further. What do you want to test? What are your options for creating new networks and servers?
Brad_C
Advanced member
Advanced member
Posts: 106
Joined: Sat Sep 13, 2014 2:33 am

Restore a backup Zimbra Server for testing?

Post by Brad_C »

[quote user="bhwong"]I'm not familiar with dnsmasq. Does it isolate the clone from communicating with external network, especially the production Zimbra?
[/quote]

dnsmasq pretends to be a dns server. You install it on the zimbra server and point the zimbra server at it to use as the resolver. The result is any time zimbra tries to talk to itself over the network, dnsmasq makes sure it receives the address 127.0.0.1 rather than the address of your production zimbra server.
It does not exclude any other network communication and the result is your test server thinks it is the production server, and behaves like it is the production server. You can connect to it using any client and perform any tests on it without farting around setting up a separated or segregated network, or changing any configuration in zimbra.
bhwong
Advanced member
Advanced member
Posts: 151
Joined: Thu Feb 27, 2014 8:40 pm

Restore a backup Zimbra Server for testing?

Post by bhwong »

I want to duplicate it so that I can troubleshoot Ubuntu Unauthenticated packages found error that prevent me from upgrading it in order to upgrade Zimbra. See this: http://community.zimbra.com/collaborati ... 01#1588901



I have raised a support case and unfortunately, the tech support suggest that I use rsync, which is not even answering my question! https://wiki.zimbra.com/wiki/ZCS_to_ZCS_rsync_Migration



Not to mention that rsync require me to install a clean Ubuntu server, which defeat the very purpose of duplicating the problem in Ubuntu to troubleshoot, isn't it? sigh.
Post Reply