Multiple times a day the SIP Trunk Loses regisration then succeeds 2 minutes later.

[Nickelpede]

3CX Version: 16.0.655
Windows 10 Pro on local dedicated computer I5 with 8 GB Ram 256GB SSD
IP Phone Make/Model/Firmware (38) Yealink
Provisioning Method: Local
Trunk Provider:Nextiva
Gateway Make/Model: WatchGuard-XTM T50
Has the Firewall Checker passed: YES
Are custom Phone Templates being used: N/A

Since 8/15/2020 The SIP registration fails, then succeeds a few minutes later.
This has happened more than 100 times since 8/15/2020
From 1/1/2020 until 8/14/2020 we had 8 registration failures.

The latest update to 16.0.655 has not resolved the problem


Below are examples of the notifications I get

Date: Fri, 25 Sep 2020 01:09:31 -0400
Trunk L:10002(Nextiva) has changed status to unregistered. This means that no more calls will pass via this trunk. Please check your network connection and the voip provider or other SIP PBX

Date: Fri, 25 Sep 2020 01:09:33 -0400
Registration at Nextiva has failed.
Destination (sip:xxx.xxx.xxx.xxx:5060;lr) is not reachable, DNS error resolving FQDN, or service is not available.

Date: Fri, 25 Sep 2020 01:11:31 -0400
Trunk L:10002(Nextiva) has changed status to registered.

There does not appear to be a pattern in the time that it fails.
Here are some:
Trunk-L:10002(Nextiva)hasbeenchanged 9/23/2020 4:03PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/23/2020 3:28PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/23/2020 8:33PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/23/2020 2:33AM
Trunk-L:10002(Nextiva)hasbeenchanged 9/21/2020 2:37AM
Trunk-L:10002(Nextiva)hasbeenchanged 9/20/2020 9:26AM
Trunk-L:10002(Nextiva)hasbeenchanged 9/19/2020 11:20PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/18/2020 8:08PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/18/2020 5:00AM
Trunk-L:10002(Nextiva)hasbeenchanged 9/18/2020 3:27AM
Trunk-L:10002(Nextiva)hasbeenchanged 9/17/2020 6:18PM *
Trunk-L:10002(Nextiva)hasbeenchanged 9/17/2020 2:12PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/17/2020 5:34AM
Trunk-L:10002(Nextiva)hasbeenchanged 9/16/2020 10:29PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/16/2020 3:24PM
Trunk-L:10002(Nextiva)hasbeenchanged 9/16/2020 11:50AM
Trunk-L:10002(Nextiva)hasbeenchanged 9/16/2020 5:16AM

I installed a Progam called PingPlotter and there were no packet losses during the registration failure.

I eliminated a DNS resolution issue by placing a static IP in the Trunk details in place odf the FQDN.

I have contacted the SIP Provider and over the last 23 days we have been testing, changing and monitoring
They had me change the Re-Registration timeout to 3600 from 60.
The SIP trunk was removed and replaced, and the Authentication ID was changed twice.
Policies were created in the WatchGuard Firewall to 1. to disable SIP-ALG, 2. allow Multicast.
The Trunk IP mode was set to IPV4
The transport protocol was set to UDP
IPV6 was disabled on the computer.

The SIP Provider Tech support said that they monitored the traffic from my IP and the 3CX is sending registration request every 30 seconds.
My packet captures look like it is every 25 seconds.
The SIP Provider says this is excessive and needs to be changed to a longer period of time.

How can I change this?

 

[AWS2P]

What is expected value from your sip provider, when you have it, edit your trunk and set here

 

[Nickelpede]

It is set to 3600

 

[AWS2P]

3600s is the value really expected by your sip provider ?
if this setting is on your trunk and was 60s before and you see no difference for timeout register now it 's set to 3600s, then it seems to me this is something wrong in 3CX.

i've seen same problem on few of my pbxs timeout value set in trunk is not followed, but for now no fix for that.

Perhaps you could try to delete your trunk and reset a new one directly with 3600 an see if it change something.

 

[complex1]

@Nickelpede

Try to use the provider IP instead of his FQDN and see if the registration become steady.

 

[AWS2P]

I eliminated a DNS resolution issue by placing a static IP in the Trunk details in place odf the FQDN.

OP already did this , read his starting post.

 

[leejor]

Were there any changes, or updates to anything, on, or about the date this all began?

 

[Nickelpede]

@Nickelpede

Try to use the provider IP instead of his FQDN and see if the registration become steady.

It is set to a static IP.

 

[Nickelpede]

@leejor
We were using 3CX Version 16, Update 5 FINAL, Build 16.0.5.612
Then my tech support installed 3CX Version 16, Update 5 HOTFIX, Build 16.0.5.619
Then on 8/15 an automatic update to 3CX Version 16 installed, Update 5 Hotfix 2, Build 16.0.5.622.
Problems started.
After scouring the forums and working on it for several days I removed 3cx and re-installed 16.0.5.612 and Hot fix 16.0.5.619.
The re-register failures continued.
I removed 3CX again and Installed just 6.0.5.612.
The registration failures continued.
On 9/1 I started a case with our SIP Provider.
I installed Wireshark and Ping Plotter. Data was collected, firewall rules were double and triple checked.
After 21 days and a half dozen calls the SIP provider said the 3CX app is just making too many requests.

I installed all of the updates and we are using Version 16.0.6.655.
The registration failures continue.

 

[Nickelpede]

It's not a packet loss issue.

Sat, 26 Sep 2020 21:14:11 -0400
Trunk L:10002(Nextiva) has changed status to unregistered. This means that no more calls will pass via this trunk. Please check your network connection and the voip provider or other SIP PBX

 

[YiannisH_3CX]

Hello @Nickelpede

Run a packet capture on the PBX and check the SIP Registration messages. On the Register message from the PBX to the provider you will notice that the PBX is sending the the value that is set in the management console in the Expires header.
You need to check the 200 OK coming from the provider however to see when the registration must be renewed as the answer from the provider defines the re-registration timeout.
What the PBX sends is a suggestion but the final registration period is defined in the 200 OK message from the provider. You will see it in the Contact Header of the 200 OK
Then the PBX will have to renew the registration before it expires. So if the re-registration period is set to 60 seconds the PBX will send a new registration message within those 60 seconds. To be more specific the re-registration will be sent once 90% of the registration elapses. So it the registration is 60 seconds the re-registration will be sent after 54 seconds.

By checking the communication you will able to determine what is going on.

Let us know if you need assistance reading the captures.

 

[Nickelpede]

Hi,
It looks like the provider is sending a expires =30



The sip provider can't complain it's too fast if they set the expire.

 

[YiannisH_3CX]

That would explain why the PBX is trying to re-register within the 30 seconds. I would recommend showing the capture to the provider and see why the set the expiry to 30 if they do not want the PBX to re-register that fast.

I would also run the capture until the trunk goes unregistered to see what exactly is causing the issue. 30 seconds might be short for re-registrations but i do not think that is the issue. Keep it running until the trunk unregisters and check the registration messages. Does the PBX miss the re-registration? Does the provider miss a reply?

 

[Nickelpede]

@YiannisH_3CX

I have captured the packets during a re-registration failure.
And I learned something along the way:
Don't leave Wireshark running along with process monitor and ping plotter for too long.
It used all the available memory and I completely lost remote control and had to drive an hour to my office and power cycle it.
Fortunately it was the weekend, only 2 people were working.


It looks like the SIP Provider misses the request

 

[AWS2P]

Then the PBX will have to renew the registration before it expires. So if the re-registration period is set to 60 seconds the PBX will send a new registration message within those 60 seconds. To be more specific the re-registration will be sent once 90% of the registration elapses. So it the registration is 60 seconds the re-registration will be sent after 54 seconds.

Thanks Yiannis for that explaination , it answer a question I had about when exactly register renew from time set in trunk. Now I can confirm it's a good value I found on my PBX 300s *90% = 270s

 

[YiannisH_3CX]

Yes, wireshark alone can crush a system so combining all of them is not ideal. You can however start wireshark with a filter so it only captures packets to one port for example so the size of the capture is significantly reduced.

It looks like the SIP Provider misses the request

I think you have what you need to speak to your provider again. Keep the captures as they may want to see them.

 

[AWS2P]

It used all the available memory and I completely lost remote control and had to drive an hour to my office and power cycle it.

Perhaps a power plug with remote IP access could be a way to save your driving time
like that kind

 

[Nickelpede]

@aws2p

Thanks for the idea!

 

[Nickelpede]

I contacted our SIP Provider - Nextiva.
They said the problem is with our network.
Specifically the Fiber Optic to Ethernet modem.
(I am not on site so I could not snap a picture of it.)

- Uh we don't have FO Modem. It's a FO to Ethernet media converter.

SIP Provider said that the Firewall UDP timeout needed to be set to 3600.
The default is 180, but I humored him and set it to 3600.
The registration failed twice overnight.

A new SIP Provider will be in place very soon, and at $50 less per month.
The new provider latency is 32ms, while the existing one is 63ms on a good day.

 

[AWS2P]

A new SIP Provider will be in place very soon, and at $50 less per month.
The new provider latency is 32ms, while the existing one is 63ms on a good day.

Well done