Recommendations for Virtualizing Zimbra with VMware vSphere 4

Running our Appliance (ZCA), ZCS on VMware, or any other virtual machine software? Post your thoughts here.
Andy.pflaum
Posts: 34
Joined: Fri Sep 12, 2014 9:55 pm

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by Andy.pflaum »

The new Zimbra Wiki article linked below will be of particular interest to Zimbra Partners deploying ZCS in a virtualized manner in a customers' data center, or in the partners' own hosted environment.
The article includes a short list of essential best practices and recommendations to ensure a high performing Zimbra deployment on the VMware's vSphere virtualization platform. Also weÂ’ve provided a list of recommended reference material to build and deploy a vSphere virtualization platform with performance in mind, as well as troubleshooting steps to resolve performance-related issues.
[View:http://wiki.zimbra.com/wiki/Performance ... re_4:940:0]
Klug
Ambassador
Ambassador
Posts: 2741
Joined: Mon Dec 16, 2013 11:35 am
Location: France - Drôme
ZCS/ZD Version: All of them
Contact:

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by Klug »

I have a question about these recommendations.
From what I understood, the "best sized" VM is one 4-vcores and 16 GB of RAM.
Considering nowadays hardware, it's quite easy to get 8-cores or 12-cores servers with 32, 48 or 64 GB (and even more).
From a VMware-ZCS point of view, on a single hardware server with 12 cores and 48 GB of RAM (considering the SAN fits) is it better to:



use it bare metal as a huge mailbox server?

use vSphere with 2 big mailbox servers (6-vcores and 24 GB memory)?

use vSphere with 3 mailbox servers (4-vcores and 16 GB memory, as suggested as "best sized")?

cwindomsr
Posts: 5
Joined: Sat Sep 13, 2014 1:17 am

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by cwindomsr »

When virtualizing, this really all depends on the hardware layout (NUMA nodes)
If the hardware is a 12 core (2x6) then the largest virtual machines would consume a single NUMA node of 6 cores and 24GB of system memory. This would prevent NUMA cross-talk.
In the case of today's more powerful processors, I generally recommend starting with 2 vCPUs and increasing vCPUS if the load is running hot (60 - 70%) during peak times. If the SAN is configured properly, we have seen through testing the ability to get about 8000 users on a 4vCPU/32GB virtual machine running at about 42% user and about 7% sys utilization with a SendMsg latency of about 203ms which is well under the 1000ms max for a good enduser experience.



Hope This Helps,

cwindomsr
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2793
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.6 Network Edition
Contact:

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by L. Mark Stone »

cwindomsr wrote:When virtualizing, this really all depends on the hardware layout (NUMA nodes)
If the hardware is a 12 core (2x6) then the largest virtual machines would consume a single NUMA node of 6 cores and 24GB of system memory. This would prevent NUMA cross-talk.
In the case of today's more powerful processors, I generally recommend starting with 2 vCPUs and increasing vCPUS if the load is running hot (60 - 70%) during peak times. If the SAN is configured properly, we have seen through testing the ability to get about 8000 users on a 4vCPU/32GB virtual machine running at about 42% user and about 7% sys utilization with a SendMsg latency of about 203ms which is well under the 1000ms max for a good enduser experience.



Hope This Helps,

cwindomsr
Presumably that's a mailbox-only server? No MTA, LDAP, Logger, etc., yes?
Thanks,

Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
cwindomsr
Posts: 5
Joined: Sat Sep 13, 2014 1:17 am

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by cwindomsr »

Mark,
You are in fact correct. Follow the appropriate monitoring and data-gathering procedures for determining your organization's messaging requirements. As mentioned in my previous reply, start with a 2vCPU 4GB VM for the LDAP, and MTA VMs. The logger and proxy servers might require less in which case you can reduce the memory and vCPU counts with just a reboot.
Cheers
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2793
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.6 Network Edition
Contact:

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by L. Mark Stone »

cwindomsr wrote:Mark,
You are in fact correct. Follow the appropriate monitoring and data-gathering procedures for determining your organization's messaging requirements. As mentioned in my previous reply, start with a 2vCPU 4GB VM for the LDAP, and MTA VMs. The logger and proxy servers might require less in which case you can reduce the memory and vCPU counts with just a reboot.
Cheers
We have found we can comfortably run MTA-only w/ LDAP-replica servers with 4GB of RAM, 2 cores, and aggressive anti-spam with the only performance tweak comprising 512MB or more allocated to a RAM disk for Amavis's temp directory. Throughput at full load is 18K-30K emails/hour depending on the mix of emails, attachments and the types of CPU cores, and provided that the DNS server can keep up with all of the queries.
Hope that helps,

Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
cwindomsr
Posts: 5
Joined: Sat Sep 13, 2014 1:17 am

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by cwindomsr »

That's great Mark. I will keep that information for furture reference. What is the total number of users in your environment. What is the mix of server vms?
Cheers
User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2793
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.6 Network Edition
Contact:

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by L. Mark Stone »

cwindomsr wrote:That's great Mark. I will keep that information for furture reference. What is the total number of users in your environment. What is the mix of server vms?
Cheers
We have our own hosting farm (we are an HSP) but have built and maintain a number of Zimbra farms and single servers for clients architected to support as few as 50 mailboxes and as many as 10,000. So, our experience comes from a variety of Zimbra environments; physical, virtual, at various release levels and running on various operating systems.
The above MTA stat was from the primary virtual MX on our hosting farm.
Hope that helps,

Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
1415bhwong
Outstanding Member
Outstanding Member
Posts: 498
Joined: Sat Sep 13, 2014 12:03 am

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by 1415bhwong »

FYI, our Zimbra 6 (on Ubuntu 8) is running on 4 cores with 8GB RAM for about 200 mailboxes. Under performance, our CPU usage is merely 12% and Memory 20%. But I'm not sure if these percentage is in reference with it's VM resources or it's host total resources.
cwindomsr
Posts: 5
Joined: Sat Sep 13, 2014 1:17 am

Recommendations for Virtualizing Zimbra with VMware vSphere 4

Post by cwindomsr »

You can obtain the performance metrics for the host using either ESXtop or vCenter. To obtain the performance metrics for the application, you can use the Zimbra performance utilities which use sysstat and other related linux performance utilities. In any case, you have lots of headroom to grown your MBX server.
Hope This Helps,

cwindomsr
Post Reply