SSL Certificate Renewal Failed

[mssit]

I am having difficulty with the SSL Cert renewal with our on-prem system. Our system had been fine renewing consistently for several years now using a 3CX fqdn. But the middle of January we started receiving an email indicating the Cert renewal had failed (FYI, we have been at v18.0.8.939 since September 2023). After reading some of the forum comments, I let it simmer hoping it was possibly a LE issue or that it might renew during the night. As it was reaching 2 weeks out before expiration, I started to follow a couple of the steps I've found throughout the forums. So far, these are the things I've tried:

• changed the DNS to 8.8.8.8
• perform a clean installing using latest Debian ISO (v18.0.9.20) and restore from full backup

I can visit the License Information page within the 3CX Management Console, and some times the License Info page loads within a couple of seconds - so all seems good. Other times, I go to the License Info page, it hangs for almost 30+ seconds and then I receive all these HTTP/SSL Certificate error issues:
HTTPS/SSL error - failed to connect to 3CX Activation server. The HTTPS chain is broken or a non trusted certificate was presented.
Possible causes: Ensure that your firewall or proxy is not performing DPI - Deep packet inspection. Disable SSL inspection from the 3CX Server to activate.3cx.com
Ensure that requests are not proxied.
Check that all operating system updates have been applied.

I also have received an email with the following:
The SSL Certificate renewal for XXXXXX.my3cx.us failed - Max certificate limit Exceeded the maximum number of certificate requests. Limit to 5 certificates per domain per week. The SSL certificate is no longer valid or will expire. This is a sign of multiple active installations using the same FQDN.

I'm not sure how to tell the 3CX system to hold or delay requesting a SSL renew to allow time for a reset.

I'm to the point where I can try to perform a backup without license key and fqdn, provision a new VM, release the current 3CX provided fqdn, and get a new one. Right now, the system is using a temporary self-signed SSL when the restore wasn't able to retrieve/renew the correct one.

 

[Guillaume Bourgeois]

Hello,
First off, are you sure you don't have any other installations using the same license elsewhere?

Next, make sure there are no outbound blocks trafic from your 3CX PBX, which would be ideal.

Ensure you have only one network card ( + loopback ).

Make sure not to install anything else on the same VM as 3CX; only 3CX should be on this VM.

Is it a Linux server or a Windows server?

Is the public IP address displayed on the Dashboard the one that 3CX is using for outbound traffic? Do you have an SD-WAN?

Do you have try reboot ?

 

[mssit]

Thank you, Guillaume.

No, this has been the only installation for years.

We use a Meraki Firewall, and we allow all outbound traffic from the local network.

The 3CX server is a Microsoft Hyper-V VM, provisioned using the custom 3CX Debian ISO (years ago) and tuned to requirements (vCPU, Memory, etc.) per 3CX guidelines and recommendations based on our SC and number of users.

Yes - the public IP address is displayed on the Dashboard. No, we do not utilize SD-WAN.

Made me laugh . . . isn't that exactly what all good IT techs tell someone to do first Yes - 've rebooted the machine multiple times.

I just finished provisioning another Hyper-V VM with v20 version of 3CX (which I believe is Deb12) and will try and restore to this version. We do have the VoIPTools relay installed on our current 3CX server as it provides additional 911 and Emergency Number notification features we needed in our building and for our agencies. By provisioning this latest version of 3CX, it will be exactly as 3CX intends with no additional changes or modifications to the system OS.

Sorry - this has been very frustrating as our 3CX has been extremely stable and reliable since we migrated from Lync/SFB almost 3 years ago.

 

[mssit]

Thank you for your thoughts and time, Guillame. Our 3CX server was able to renew last night.

I did find a firewall rule that was only allowing traffic in specific countries: US, CA, FR, DE, IT, CH, GB, IE, AU. So I'm guessing either the 3CX servers or the Let's Encrypt services were using a connection NOT from one of these countries (which does concern me a bit). What's more - this was a firewall rule that has been in place for years, so why now it became an issue is weird.

Thank you so very much for responding and providing your guidance/help. Merci Beaucoup!!

 

[Guillaume Bourgeois]

Ah, that explains it! Thank you for the follow-up.

It's nice to receive thanks in French!
In French, we respond to thanks with; Soyez le bienvenu.

Country-based firewall rules can sometimes cause issues. It's important to understand that this mechanism validates against a database that holds the geographical location of the IP (GEOIP).

This database needs to be updated over time as IP addresses move; they can change countries, for example.

Your router might have a copy of this database, and it might not be up-to-date, or worse, the database it uses for its checks might not be current.

As a result, the IP is considered to be in a geographical location that is incorrect.

This might be the case here.

Let's Encrypt operates multiple validation servers around the world. To my knowledge, there isn't an official public IP list used by Let's Encrypt.

In any case, don't be concerned about the location of Let's Encrypt's servers, as their reputation is well-established.

There's undoubtedly a higher chance of winning the lottery than being hacked due to renewing an SSL certificate through Let's Encrypt.

I hope this will help you understand a bit more about the reason for the blockage and reassure you about the reliability of 3CX.

3CX is undoubtedly one of the most reliable 100% software-based phone systems on the market. I'm not saying it's perfect, but... is there such a thing as perfect? (Except for my wife, of course )

A+