After 8.0.8 to 8.5.0 domain certificates per IP are being overridden with mail server certificate

[johnroberts]

Hi All,

We have just upgraded from 8.0.8 to 8.5

Prior to the upgrade everything was working well with the machine having 2 additional IP addresses and the proxy setup to handle https connections for two extra domains. For example:

Main server: mail.example.com (on 10.10.10.1)
IP2: mail.example2.com
IP3: mail.example3.com

These were installed previously using commands:
zmprov cd mail.example2.com zimbraVirtualHostName "mail.example2.com" zimbraVirtualIPAddress "10.10.10.2"
zmprov cd mail.example3.com zimbraVirtualHostName "mail.example3.com" zimbraVirtualIPAddress "10.10.10.3"

And then after the certificates had been installed they were deployed:
/opt/zimbra/libexec/zmdomaincertmgr deploycrts
which correctly Deployed the two additional certificates.

Everything was working well and a visit to:
https://mail.example2.com
would present the correct certificate for 'mail.example2.com'

Since the upgrade to 8.5 however visits to the two other ips as in https://mail.example2.com or https://mail.example3.com get presented with the certificate of the main server and not those setup via the proxy.

I've been looking into this for the past hour or so and can work out why? Can anyone help?

Thanks in advance for any and all replies.

John

[johnroberts]

Hi,
Just to add to the above.
A full 'zmprov gd mail.example2.com' shows the correct complete combined certificate and key of the virtual domain.
Also:
'zmprov gd mail.example2.com | grep Virtual' shows correctly:
zimbraVirtualHostname: mail.example2.com
zimbraVirtualIPAddress: 10.10.10.2
So the certificate, key and IP address are all setup.
When I try:
openssl s_client -connect mail.example2.com:443 (or 995 or 465)
it always returns the certificate of the main server, even though mail.example2.com is a DNS A record linked to 10.10.10.2
In the admin console Configure -> Servers -> mail.example.com -> Proxy
the proxy is activated for Web and Mail.
This is so frustrating.
Can anyone help me debug how the Proxy can be shown to be operating but is not linking to the correct certificate?
Thanks,
John

[johnroberts]

I noticed in the 'nginx.log' log file that before the upgrade it logged, for example:
2014/09/28 23:07:11 [info] 14970#0: *43238 client 100.100.100.100 connected to 10.10.10.10:993
since upgrading to 8.5.0 it now logs:
2014/10/02 15:04:19 [info] 19778#0: *25472 client 100.100.100.100:31398 connected to 0.0.0.0:993
Is the fact that the interface IP it logs since the upgrade is 0.0.0.0 and not 10.10.10.10 a link to why it is not detecting the correct virtual domain and therefore sending the correct certificate?

[kpariani]

This issue has been resolved as part of https://bugzilla.zimbra.com/show_bug.cgi?id=95319 and the fix should be available in 8.5 Patch2

[johnroberts]

Thank you very much for the response.
I don't know if you are linked with Zimbra, but do you know when we might see 8.5 Patch2?
The following files are reportedly affected:
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.mail.imap.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.mail.imaps.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.mail.pop3.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.mail.pop3s.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.mail.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.web.admin.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.web.http.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.web.https.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.web.sso.template
/opt/zimbra/main/ZimbraServer/conf/nginx/nginx.conf.web.template
/opt/zimbra/main/ZimbraServer/src/java/com/zimbra/cs/util/ProxyConfGen.java
Do you know if we manually diff'ed the files pre 8.5 and corrected them, it would resolve the issue prior to any official patch?
As is probably the case elsewhere, this problem is causing us a bit of a headache.
Once again, thank you very much for taking the time to respond.

[kpariani]

Hello,

Yes i am a Zimbra employee & the workaround mentioned in the bug (https://bugzilla.zimbra.com/show_bug.cgi?id=95319#c6 for web proxy & https://bugzilla.zimbra.com/show_bug.cgi?id=95319#c7 for mail) would work for you too as these files would still be there leftover even after the upgrade. Let me know if you run into any issues.



Thanks

[johnroberts]

Ok great thank you.



Unfortunately I can't see comments c6 and c7 on that page, only c1, c9, c10 and c12 so can't see the workarounds.



Do I need to create a Bugzilla login to see them?

[kpariani]

Sorry. I had made those comments public but didn't save the changes. You should now be able to see comment#6 and #7. Also Patch2 is going to be available pretty soon

[johnroberts]

Thank you very much, this fixed the problem. Could I just ask a final question, should we reverse this fix prior to installing the future Patch 2?
For others with the same issue, the exact changes were:-
The files to change are read only so you will need to add write access, as zimbra user, with:
chmod u+w /opt/zimbra/conf/nginx/templates/nginx.conf.web.template
chmod u+w /opt/zimbra/conf/nginx/templates/nginx.conf.mail.template
For web added:
include /opt/zimbra/conf/nginx/includes/nginx.conf.web.https;
to:
/opt/zimbra/conf/nginx/templates/nginx.conf.web.template
For web mail:
include /opt/zimbra/conf/nginx/includes/nginx.conf.mail.imap;
include /opt/zimbra/conf/nginx/includes/nginx.conf.mail.imaps;
include /opt/zimbra/conf/nginx/includes/nginx.conf.mail.pop3;
include /opt/zimbra/conf/nginx/includes/nginx.conf.mail.pop3s;
to:
/opt/zimbra/conf/nginx/templates/nginx.conf.mail.template
(both should be added the the end of the files prior to the last line of curly closed brackets } )
Then restart Zimbra

[kpariani]

Am glad it worked. The Patch will simply overwrite these template files you have modified so you will not have to reverse the fix.