I would first like to apologize for the issues triggered by the recent patch release. I know that many people rely on the services offered by our products and when we do not execute well, the impact is felt by customers. While we take efforts to prevent problems, there are times when those efforts are not enough.
Upon releasing Zimbra 8.8.15 patch 32 and 9.0.0 patch 25, we recognized there was a defect which would affect a number of users. Our attempts remove the availability of the patches broke everyone's ability to install the product or update to another patch. This happened to coincide with the release of information on a security issue covered in the previous patch release (8.8.15 patch 31 and 9.0.0 patch 24). Panic over the security issue drove a number of people to attempt to update, only to find out they could not.
Where things stand:
The patches for Zimbra 8.8.15 and 9.0.0 has been made available once again with the blocking issue resolved. The package repository issues have been corrected, fixing the issues with installations and upgrades. Everyone should now be able to install and upgrade without issue.
What we have learned:
- Our quality assurance release gates failed to uncover the issues present in the patch release. While we have identified the specific gaps related to the patch release, we will be undertaking a full evaluation of our processes to ensure a better outcome in the future.
- Our release process did not properly account for pulling back releases due to issues. This has been corrected and we are expanding our resources who have knowledge of the process for redundancy.
I want to personally thank our partners, customers, community members and friends who worked closely with us during this incident. With your help, we were able to quickly identify the major issues and eventually resolve them. Without your support and interaction, things could have been much worse. We will be taking the lessons learned from this incident and implementing adjustments to our processes to ensure we do not repeat the mistakes made here.
We have also separated out our security and patch announcements from our email newsletter. We felt this would be a better way of highlighting security issues which could arise and bringing attention to them without the distraction of other content. We believe this will help you make better decisions around the content of patches and the availability of security releases. You can sign up for the announcements here