System won't renew LE certificate

[NicholasBarnes]

Debian, self-hosted.
I have an interesting problem from a customer who said "Oh yes, we're happy to manage our own system"...
Turns out they hadn't updated it at all since it was installed early last year.
Their LE certificate stopped renewing some time after May 6th this year. For the first time in a very long time, they tried to log in but couldn't because the certificate had expired.
They called me.
I tried to renew the certificate from the command line (/usr/lib/3cxpbx/PbxConfigTool -renew-certificates) - it thought about it for a while and then gave me the prompt back. It hadn't updated the certificates. I changed my clock, logged in and did the LE update. Still no joy renewing. Upgraded to v18. Now when I run the command to renew, I get the prompt back straight away - no thinking about it, just starts and ends immediately.
Did a new installation and recovered the backup from last week. Same problem - renew now starts and stops immediately.
I can't find anything relevant in any log file and the certificate (in /var/lib/3cxpbx/Bin/nginx/conf/Instance1/) hasn't changed (creation/modification date and content).
Does anybody have any idea how I can force a LE cert renewal - or where I can see any debug messages to try to get this resolved?
Many thanks,
Nicholas.

 

[nub]

Are you sure you still have the subdomain?
dig or nslookup the name.3cx.? and see if the reply is from the ip you expect.

 

[NicholasBarnes]

Does anybody have any idea how I can force a LE cert renewal - or where I can see any debug messages to try to get this resolved?

 

[ChrisC_3CX]

Just to first clarify a few things, you said you upgraded to v18, how was this done exactly, did you upgrade via the Management Console or did you redeploy using the 3CX ISO and then restored using a backup?

Please try this, access the Management Console(Use IP instead of FQDN if necessary), go to "Settings >> License "and click on "Refresh License Key Information". Do you get any message whatsoever, what happens exactly?

Also, just to confirm, this PBX is utilizing a 3CX FQDN and not a custom one correct?

 

[NicholasBarnes]

Just to first clarify a few things, you said you upgraded to v18, how was this done exactly, did you upgrade via the Management Console or did you redeploy using the 3CX ISO and then restored using a backup?

I have done both. Initially I set my PC clock back and logged in to the management console using the FQDN (i.e. so the certificate still appeared to be in date) and then applied the upgrade from there. When that didn't work, I did a new install and restored from a backup.

Please try this, access the Management Console(Use IP instead of FQDN if necessary), go to "Settings >> License "and click on "Refresh License Key Information". Do you get any message whatsoever, what happens exactly?

No message - the page refreshes and shows the licence details.

Also, just to confirm, this PBX is utilizing a 3CX FQDN and not a custom one correct?

A custom one in the form 'pbx.companyname.uk'.

Many thanks,
Nicholas.

 

[ChrisC_3CX]

A custom one in the form 'pbx.companyname.uk'.

In that case, 3CX does not manage the certificate meaning it will not automatically renew. You will have to manually renew the certificate or issue a new one for this FQDN.

 

[NicholasBarnes]

In that case, 3CX does not manage the certificate meaning it will not automatically renew. You will have to manually renew the certificate or issue a new one for this FQDN.

So 3CX will get the LE certificate when you do the initial installation (as happened for this server), but it won't renew the LE certificate? That seems a bit strange.

 

[ChrisC_3CX]

Not exactly, if you use a 3CX FQDN such as mycompany.3cx.co.uk, mycompany.3cx.com then 3CX will manage the FQDN's certificate meaning it will be generated upon installation and thereafter automatically renew every three months. If however you use a custom FQDN then you will have to manage the certificate yourself meaning you will need to provide one upon installation and then take care of it's renewal manually regardless of whether you use an LE certificate or not.

 

[NicholasBarnes]

But the installation process took the custom FQDN and got a LE cert automatically - the cert wasn't applied manually.
OK. I take your point, custom FQDN means manual cert - just seems bizarre to me that it did it automatically on installation but won't renew it automatically.

 

[ChrisC_3CX]

But the installation process took the custom FQDN and got a LE cert automatically

That should definitely not happen, could you please PM me the FQDN and the first 8 characters of the License Key so that I can have a quick look?

 

[Evolute IT]

Debian, self-hosted.
I have an interesting problem from a customer who said "Oh yes, we're happy to manage our own system"...
Turns out they hadn't updated it at all since it was installed early last year.
Their LE certificate stopped renewing some time after May 6th this year. For the first time in a very long time, they tried to log in but couldn't because the certificate had expired.
They called me.
I tried to renew the certificate from the command line (/usr/lib/3cxpbx/PbxConfigTool -renew-certificates) - it thought about it for a while and then gave me the prompt back. It hadn't updated the certificates. I changed my clock, logged in and did the LE update. Still no joy renewing. Upgraded to v18. Now when I run the command to renew, I get the prompt back straight away - no thinking about it, just starts and ends immediately.
Did a new installation and recovered the backup from last week. Same problem - renew now starts and stops immediately.
I can't find anything relevant in any log file and the certificate (in /var/lib/3cxpbx/Bin/nginx/conf/Instance1/) hasn't changed (creation/modification date and content).
Does anybody have any idea how I can force a LE cert renewal - or where I can see any debug messages to try to get this resolved?
Many thanks,
Nicholas.

I hope you bill your time accordingly.

Jeez. Customers like that are always a PITA and then blame you for not monitoring them lol

 

[QueBall]

3cx would never had managed a custom domain name SSL certificate out of the box.

But from time to time users have shared shell scripts they hacked together that would manage the SSL certificates separately. Perhaps they installed the original server using one of these user supplied scripts to manage the certificate. It likely broke when letsencrypt old root certificate expired, or the script used an older API they turned off. Or they never used letsencrypt and it was a purchased SSL certificate that simply expired and needs manual intervention to renew and install.

If it was their own script, they likely had been using ACME version 1. It was shut down in June. If their last succesful renewal was around May, that sounds about right.
https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430

So I'm going to confidently guess they used an unsupported script to renew a custom letsencrypt domain. (doesn't mean it wouldn't work, just means 3cx is not responsible for helping you fix it or maintain it)
That script stopped working when they shut down version 1 of their ACME api.
Because nobody documented it, and nobody maintained it you are now in this position.

 

[greychain]

I have a similar situation, well almost.

I use an SNI-Proxy on ports 80 and 443 to connect the two public IP's I have to the correct webservers. Cant always us the same server for all the different projects I work on so SNI-proxy solved this.

However for 3CX, which I have a custom FQDN for (lets see how annoying it is to use a custom FQDN project). When the LE cert tries to renew it fails, so every 3 months I have to flick 443 direct to the 3CX server, manually run certbot then revert 443 back to the SNI proxies.

For all my other LE renewals there is no issue.