Certificate/Certbot - best way?

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
kdiamond
Posts: 43
Joined: Mon Apr 12, 2021 12:52 am

Re: Certificate/Certbot - best way?

Post by kdiamond »

Also, I did generate all new certificate files using the .acme scrip with success.

Code: Select all

./acme.sh --issue --dns dns_cf --force --challenge-alias globevoices.eu -d mail.dbstudio.si -d mail.dspsound.si -d mail.xstarbay.com -d mail.globevoices.com
gen.jpg
gen.jpg (35.86 KiB) Viewed 2366 times
But when I try to verify, verification fails.

Code: Select all

zimbra@dbstudio:~/.acme.sh/mail.dbstudio.si$ /opt/zimbra/bin/zmcertmgr verifycrt comm mail.dbstudio.si.key mail.dbstudio.si.cer fullchain.cer
** Verifying 'mail.dbstudio.si.cer' against 'mail.dbstudio.si.key'
Certificate 'mail.dbstudio.si.cer' and private key 'mail.dbstudio.si.key' match.
** Verifying 'mail.dbstudio.si.cer' against 'fullchain.cer'
ERROR: Unable to validate certificate chain: C = US, O = Internet Security Research Group, CN = ISRG Root X1
error 2 at 2 depth lookup: unable to get issuer certificate
error mail.dbstudio.si.cer: verification failed
Thank you
Br,
Dali
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 889
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Certificate/Certbot - best way?

Post by JDunphy »

Best guess is that openssl changed (zmcertmgr uses it) and the workarounds for the expired chain no longer work or acme.sh defaults to a different chain than the deploy script expects for zimbra. I renewed with P33 yesterday so the latest openssl doesn't appear to be a problem for me anyway with the alternative signed chain.

I would verify the following so you have the alternative signed chain for letsencrypt.

Step 1:
Make sure your deploy script is this one and not the older one which was for the default chain (expired root)... You are looking for ISG_X1 in the zimbra.sh code.
Here is the latest deploy script: https://raw.githubusercontent.com/JimDu ... /zimbra.sh

Step 2:
Make sure you have done the following once.

Code: Select all

% ./acme.sh --set-default-ca  --server letsencrypt
% ./acme.sh  --set-default-chain  --preferred-chain  ISRG  --server letsencrypt
Step 3: issue your certificate again. Hopefully you get the alternative chain.

For background information on their 2 signed chains and why openssl can have various reject behaviors for verification with the default chain.
Ref: https://wiki.zimbra.com/wiki/JDunphy-LeChains

For information on this method using the deploy script, etc and acme.sh
Ref: https://wiki.zimbra.com/wiki/JDunphy-Notes

HTH,

Jim
kdiamond
Posts: 43
Joined: Mon Apr 12, 2021 12:52 am

Re: Certificate/Certbot - best way?

Post by kdiamond »

Thank you for your instant reply Jim.

Did the step1 and step2.

Step 3 is the problem because I did it more than 5 times. Now I need to wait 168 hours :(

Br,
Dali
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 889
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Certificate/Certbot - best way?

Post by JDunphy »

If it is signed by the alternative chain, we don't have to do issue again... Just deploy until we can figure out what went wrong. First trying to figure out what chain you have it signed by.

question: Can you verify that ../ISG_X1.pem is non zero file. Take a look inside your .acme.sh directory and verify it is:

Code: Select all

# su - zimbra
% cd .acme.sh
% [zimbra@mail .acme.sh]$ head -2 ISG_X1.pem 
-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
If that checks out... try this

Code: Select all

# su - zimbra
# cd .acme.sh/mail.example.com
% grep -A 1 BEGIN ca.cer.real | tail -5
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
--
-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
Your ca.cer.real should match this. We skip over your cert which is the first entry and the reason for the tail -5

Jim
kdiamond
Posts: 43
Joined: Mon Apr 12, 2021 12:52 am

Re: Certificate/Certbot - best way?

Post by kdiamond »

Hi Jim

ISG_X1.pem is non-zero file. It checks out OK.

ca.cer.real file will contain 4 certificates in a single file.

Code: Select all

zimbra@dbstudio:~/.acme.sh/mail.dbstudio.si$ grep -A 1 BEGIN ca.cer.real | tail
MIIFeTCCBGGgAwIBAgITAPr3+lWtOtKw+Q4fzFFMiMr/bDANBgkqhkiG9w0BAQsF
--
-----BEGIN CERTIFICATE-----
MIIFWzCCA0OgAwIBAgIQTfQrldHumzpMLrM7jRBd1jANBgkqhkiG9w0BAQsFADBm
--
-----BEGIN CERTIFICATE-----
MIIFVDCCBDygAwIBAgIRAO1dW8lt+99NPs1qSY3Rs8cwDQYJKoZIhvcNAQELBQAw
--
-----BEGIN CERTIFICATE-----
MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
zimbra@dbstudio:~/.acme.sh/mail.dbstudio.si$
Is this OK?

Br,
Dali
00a
Posts: 13
Joined: Sun Sep 22, 2019 12:29 pm

Re: Certificate/Certbot - best way?

Post by 00a »

Have been using an automated certbot renewal https://github.com/AJRepo/ZimbraCertDeploy/ which has been working well.

Certbot's renewal goes into /etc/cron.d/

The deploy script goes into /etc/letsencrypt/renewal-hooks/deploy/

and when certbot certificate is renewed certbot calls deploy scripts.
kdiamond
Posts: 43
Joined: Mon Apr 12, 2021 12:52 am

Re: Certificate/Certbot - best way?

Post by kdiamond »

Thank you for your reply 00a.

My setup is not so simple. I have multiple domains in zimbra. I'm generating a certificates using a Cloudflare DNS challenge. Have already forgotten how exactly we did this, but I think all domains have a record pointing to one single Cloudflare domain. Over a year ago Jim was leading me step by step therefore I manage it to work.

Now I don't know how t solve it. In let's encrypt I have a one-week ban as I tried to generate the certificates more than 5 times. I don't know. Maybe even to install a commercial multi-domain certificate.

Br,
Dali
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 889
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Certificate/Certbot - best way?

Post by JDunphy »

kdiamond wrote:In let's encrypt I have a one-week ban as I tried to generate the certificates more than 5 times. I don't know. Maybe even to install a commercial multi-domain certificate.

Br,
Dali
A couple of things.

1) You can get around that ban by adding another name... make something up. tmail.example.com ... Doesn't matter, just another hostname on your domain on your issue line. Do all the same stuff you did for this new one ie) CNAME. See instructions in this thread. That ban I believe says for the same set of names... so we add another. Doesn't matter if you use it. That will get around that ban.

2) It does not appear that you are getting the LE alternative chain from what you are showing. acme.sh will default to zerossl unless you tell it otherwise. It will also pull the default chain when there are more signing chains. I just successful verified a new certificate with acme.sh version 3.05 so it would appear to be your environment and not latest version of acme.sh provided we can get you asking for the LE alternative chain.

A couple of things you can do. Look inside your mail.example.com directory for the per domain/certificate config file. There should be something like this if the first -d has been mail.example.com:

Code: Select all

# su - zimbra
% cd .acme.sh
% grep Le_Preferred_Chain mail.example.com/mail.example.com.conf
Le_Preferred_Chain='__ACME_BASE64__START_SVNSRw==__ACME_BASE64__END_'
% cat ca/acme-v02.api.letsencrypt.org/directory/ca.conf 
ACCOUNT_URL='https://acme-v02.api.letsencrypt.org/acme/acct/xxxxxx'
CA_KEY_HASH='....'
DEFAULT_PREFERRED_CHAIN='ISRG'
There use to be this option that you could specify on the issue but don't remember exact syntax.. something like this perhaps.

Code: Select all

./acme.sh --force --issue --server --preferred-chain "ISRG" --dns dns_cf --challenge-alias domainWeWillAddTxtRecord.com -d mail.example.com -d mail.example.net 
Hints of things that might get you working.

1) you can move your mail.example.com directory out of the way then issue the 2 commands about the alternative and LE chain and then issue/verify that you have the Le_preferred_Chain, etc.
- I have not tested this with version 3.0.5 but that is how I did it in prior versions when we ran into this problem a few years ago.
see this: https://github.com/acmesh-official/acme ... -945993399
2) you can manually put the Le_preferred_Chain option in the per domain certificate file... ie) from the grep above and then reissue certificate.
3) you can force the preferred chain using that option on the issue

About those certificates we saw with the grep BEGIN. The ca.cer.real is from the Zimbra.sh deploy script adding the ISG_X1.pem root to your ca.cer file. The ca.cer is the certificates that signed (intermediates) your certificate and your certificate.

Hope this makes sense. I am looking at the github for acme.sh now to see what has changed that would prevent you from getting the alternative chain given you think you are asking for the alternative for LE CA.

Here is the section for acme.sh documenation: https://github.com/acmesh-official/acme ... default-CA
https://github.com/acmesh-official/acme ... rred-Chain

I am at a loss of why when you issued those 2 server commands that you didn't get the LE alternative chain.

Jim
kdiamond
Posts: 43
Joined: Mon Apr 12, 2021 12:52 am

Re: Certificate/Certbot - best way?

Post by kdiamond »

Thank you for your reply and patience Jim.

It worked!!!

Instead of adding a dummy domain, I removed one domain I didn't really need, to get around the ban. I added a command --preferred-chain "ISRG"

Code: Select all

./acme.sh --issue --preferred-chain "ISRG" --dns dns_cf --force --challenge-alias globevoices.eu -d mail.dbstudio.si -d mail.dspsound.si -d mail.globevoices.com 
after this deploy went smooth.

Code: Select all

./acme.sh --deploy --deploy-hook zimbra -d mail.dbstudio.si -d mail.dspsound.si -d mail.globevoices.com
So is there anything else I need to do in order for the certificates to renew automatically?

I suppose cron from before should continue to work?

Code: Select all

zimbra@dbstudio:~/.acme.sh/mail.dbstudio.si$ crontab -l |grep acme
24 0 * * * "/opt/zimbra/.acme.sh"/acme.sh --cron --home "/opt/zimbra/.acme.sh" > /dev/null
zimbra@dbstudio:~/.acme.sh/mail.dbstudio.si$
Thank you so much!

Br,
Dali
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 889
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Certificate/Certbot - best way?

Post by JDunphy »

kdiamond wrote:
So is there anything else I need to do in order for the certificates to renew automatically?

I suppose cron from before should continue to work?

Code: Select all

zimbra@dbstudio:~/.acme.sh/mail.dbstudio.si$ crontab -l |grep acme
24 0 * * * "/opt/zimbra/.acme.sh"/acme.sh --cron --home "/opt/zimbra/.acme.sh" > /dev/null
zimbra@dbstudio:~/.acme.sh/mail.dbstudio.si$
The cron entry looks good and is enough... double check your certificate/domain conf file to make sure you see that preferred chain I mentioned previously or you will have problems during the automatic renewal. You will also see the installation hook in that file that will be called. So a really good file to look at to give you peace of mind if the automatic renewal will go smoothly.

Generally, once you have done the first --issue followed by your first--deploy that conf file is populated and you are automatic renewal ready. From then on, you can do it in 2 parts again or use the cron entry and it will do both commands automatically for you.

One can test the cron process with the --force (if renewal time is not time yet) using the cron entry you see. Cut/Paste that onto the command line and watch.

Code: Select all

# su - zimbra
% cd .acme.sh
% ./acme.sh --list
% "/opt/zimbra/.acme.sh"/acme.sh --force --cron --home "/opt/zimbra/.acme.sh"
Glad you got it working again.

JIm
Post Reply