Zimbra letsencrypt manual. It is actual?

[GlooM]

Hello!

To install and update zimbra lets encrypt certs i use this manual:
https://wiki.zimbra.com/wiki/Installing ... ertificate

In paragraph "Build the proper Intermediate CA plus Root CA" we see:
"Let's Encrypt is almost perfect, but during the files the process built, they just add the chain.pem file without the root CA. You must to use the IdenTrust root Certificate and merge it after the chain.pem
https://letsencrypt.org/certs/trustid-x3-root.pem.txt"

But today I read news - https://letsencrypt.org/2019/04/15/tran ... -root.html

"On July 8, 2020, we will change the default intermediate certificate we provide via ACME. Most subscribers don’t need to do anything. Subscribers who support very old TLS/SSL clients may want to manually configure the older intermediate to increase backwards compatibility.

Since Let’s Encrypt launched, our certificates have been trusted by browsers via a cross-signature from another Certificate Authority (CA) named IdenTrust. A cross-signature from IdenTrust was necessary because our own root was not yet widely trusted. It takes time for a new CA to demonstrate that it is trustworthy, then it takes more time for trusted status to propagate via software updates.

Now that our own root, ISRG Root X1, is widely trusted by browsers we’d like to transition our subscribers to using our root directly, without a cross-sign"

If letsencrypt change root cert from IdenTrust to ISRG, Perhaps the instructions for installing the certificate in Zimbra should also be fixed??? Or will everything continue to work correctly according to the old instructions?

[JDunphy]

Can you tell me what this yields on your platform?

Code: Select all

# su - zimbra
% keytool -list -v  -keystore /opt/zimbra/common/etc/java/cacerts -storepass changeit  |grep -i owner |grep -i ISRG
Owner: CN=ISRG Root X1, O=Internet Security Research Group, C=US

Jim

[GlooM]

Can you tell me what this yields on your platform?

Hello Jim!
Thank you for answer.

In my case, I get empty output

[JDunphy]

What version of Zimbra are you running?

If it is empty, I would expect that zmcertmgr could fail on the verify step and most likely will not install after July 8 but I have not tested that theory. That is based on the behavior when the IdentTrust wasn't chained.

I have 3 ideas how we would proceed but none of these are tested nor do I know if I am understanding this enough at present to know what the issue will be for Zimbra.

1) Some acme clients will have a switch to allow you to pull a cross signed certificate once they switch the default to their own ISRG root ca. acme.sh (acme protocol client I use) does not for example although it was discussed last year before they delayed this event for another year. This would be a temporary solution because that IdentTrust cert is expiring next year they use to cross sign.
Ref: https://github.com/acmesh-official/acme.sh/issues/2221

2) We could include the ISRG instead of the IdentTrust in the chain
This could allow those really old Android clients to still work if there are old MUA clients for some users still around.

3) We could add the ISRG directly to the Zimbra keystore given the browsers now have it pre-installed so that Zimbra tools would work. Browsers would work because the ISRG is already loaded in most browsers with the exception of some really old Android clients.

I haven't decided how I want to handle this... My advice is to renew a few days prior to July 8 to give yourself some time should it not verify or install while you sort the process out.

On my test machine, I loaded the ISRG root X1 but I have no idea if I have done this correctly as I have not tested this. I did something like this.

Code: Select all

# su - zimbra
% wget -O /tmp/ISRG-X1.pem 'https://letsencrypt.org/certs/isrgrootx1.pem.txt'
% keytool -list -v  -keystore /opt/zimbra/common/etc/java/cacerts -storepass changeit  |grep -i owner |grep -i ISRG
% keytool -keystore /opt/zimbra/common/etc/java/cacerts -storepass changeit -importcert -alias CACERT -file /tmp/ISRG-X1.pem
% keytool -list -v  -keystore /opt/zimbra/common/etc/java/cacerts -storepass changeit  |grep -i owner |grep -i ISRG
Owner: CN=ISRG Root X1, O=Internet Security Research Group, C=U

If anyone knows or has other ideas, please comment.

Jim

[GlooM]

What version of Zimbra are you running?

A) Release 8.7.11.GA.1854.UBUNTU16.64 UBUNTU16_64 FOSS edition, Patch 8.7.11_P14.

B) Please comment on why there are different paths?

1) First check in: /opt/zimbra/common/lib/jvm/java/jre/lib/security/cacerts
2) Importing the certificate to the storage: /opt/zimbra/common/etc/java/cacerts
3) Check the certificate in storage: /opt/zimbra/common/etc/java/cacerts

Why do I need storage /opt/zimbra/common/lib/jvm/java/jre/lib/security/cacerts ?

C) If the configuration is not changed or the changes do not work. Will verification of certificates that were issued after July 8 stop working or will Letsencrypt certificates that were issued and installed earlier stop working ?

D) Does it make sense to issue a certificate on July 7 that is signed by the IdenTrust root for another 3 months?

[JDunphy]

B) Please comment on why there are different paths?

1) First check in: /opt/zimbra/common/lib/jvm/java/jre/lib/security/cacerts
2) Importing the certificate to the storage: /opt/zimbra/common/etc/java/cacerts
3) Check the certificate in storage: /opt/zimbra/common/etc/java/cacerts

Same file.

Code: Select all

% ls -l /opt/zimbra/common/lib/jvm/java/jre/lib/security/cacerts
lrwxrwxrwx 1 root root 35 Aug 10  2018 /opt/zimbra/common/lib/jvm/java/jre/lib/security/cacerts -> /opt/zimbra/common/etc/java/cacerts

Why do I need storage /opt/zimbra/common/lib/jvm/java/jre/lib/security/cacerts ?

You don't... sloppy cut and paste from me. It was from a document I had saved years ago ... I updated that mistake in my previous post so it's consistent.

C) If the configuration is not changed or the changes do not work. Will verification of certificates that were issued after July 8 stop working or will Letsencrypt certificates that were issued and installed earlier stop working ?
D) Does it make sense to issue a certificate on July 7 that is signed by the IdenTrust root for another 3 months?

Certificates that were previously issued will work until they expire. New certificates will not be cross signed by default after July 8 and that is the behavior I am trying to figure out with Zimbra. I need to get a certificate that is only signed by ISRG X1 root but don't know how to get one without further research.

For me, I don't want any certs to expire on the week of July 8 to allow some time to find a solution should something go wrong with renewals. I was making a general recommendation in case they expire close to July 8. Mine will expire in Aug but it would be really unlucky to have your zimbra letsencrypt certificate expire on July 8 especially if the scripts the community uses could potentially fail to verify and install the new issued certificates.

Jim

[JDunphy]

July 8 has been pushed back to Sept 29, 2020 as of June 11 (today).

https://community.letsencrypt.org/t/tra ... -29/125516

[GlooM]

Thank you again!

Tell me please, if I update Zimbra version 8.7.11 to the latest (8.8.15), its certificate store will already have ISRG root?

[JDunphy]

Tell me please, if I update Zimbra version 8.7.11 to the latest (8.8.15), its certificate store will already have ISRG root?

I am not convinced at this point that would solve anything or if there would be a problem. I am expecting a problem myself once you drop the full chain (ie. IdentTrust) or it wasn't signed by IdentTrust and try to verify.

I was looking at the zmcertmgr verify function this morning when we run zmcertmgr. It calls openssl to perform the work. This is the command that fails when you don't provide a full chain (IdentTrust aka TrustID X3 Root) with the current signed LE certs.

Code: Select all

% /opt/zimbra/common/bin/openssl verify -purpose sslserver -CAfile fullchain.pem mail.example.com.cer

Where fullchain contains 3 certs (mail.example.com.cer, Ident's cer, and letsencrypt cer). That validates perfectly. I ran strace in front of this to observe if it was using /opt/zimbra/common/etc/java/cacerts during the verification. It does not so if cacerts is used, it will be for some java functionality but that is speculation if it will fail.

So at this point, I think there would be 2 options I would test when they no longer sign with the IdentTrust certs by default.

0) Try it without the full chain.... It should fail.
1) Replace DST Root CA X3 with ISRG Root X1 in fullchain.pem ... I think it will verify

Then see if zimbra will work with it... if not, I would add the ISRG root to cacerts and debug from there.

I am going to try and get a signed cert via the ISRG Root X1 and test it on my test server. I would prefer not to wait to Sept 29.

We have lots of time still. It will work with Zimbra but it might take a few whacks at it.

Jim

[GlooM]

As I understand it, the official developers of Zimbra have not commented on this problem in any way? There are no updated instructions for working with Let'sencrypt at the moment?