Call Drops after update to 16.0.5.619

[jhendrickson]

Data needed:

• 3CX Version
  • Standard Annual 16.0.5.619
• Server OS
  • Windows Server 2016
• Is the 3CX Server Hosted and where?
  • Hosted on cloud datacenter (Computer St. Louis - managed by Me)
• IP Phone Make/Model/Firmware
  • Yealink T46S - 66.85.0.5
• Provisioning Method: Local / VPN / STUN / SBC
  • SBC Provisioned - SBC Version 16.0.390
• Trunk Provider or Gateway Make/Model
  • Intelepeer SIP Trunk Provider
• Has the Firewall Checker passed:
  • YES
• Are custom Phone Templates being used:
  • NO - on any of the phones dropping calls

Since updating to 5.619, I've been getting dropped call complaints. digging through the logs (BLREC and BLDEF with the binary log reader) I'm seeing the ongoing calls continuing after the user indicated that the call no longer exists..

The common thread is that the SBC's are re-registering. A few seconds later I see that the users phone is starting a new call. There isnt a term or bye or even a failed message until around when the extension tries to initiate a new call.

I have the log output, but I'm not clear about the sensitivity of the data therein, so if needed I can provide is via PM or email.

 

[jhendrickson]

I want to point out that this has been a rock solid instacne for some time. I have been running it for about 8-10 months now and this has been the first issue with dropped calls so far.

 

[cobaltit]

So I think you are dealing with coincidence and not cause/effect. Otherwise I imagine there'd be a lot more noise on the forums. Are you certain you didn't update the SBCs as well? And what version did you update from?

 

[jhendrickson]

I did update my SBCs from version 6.0.390 to 6.1.152 beta on accident, then I uninstalled the SBC from each session border device (windows 10 desktops) and deleted the session border controllers from the server, and reinstalled 6.1.390 and configured new relationships to the server.

Initially, this was my suspected cause, because the morning after updating the sbc's to beta it was a train wreck. I had all 6 sites dropping most calls. The logs form the session border controllers appeared to indicate that the connection to the server was failing due to an issue with ssl.

After realizing that I'd installed beta software (i ran the updates at about 2 am, it was a poor choice) i rolled back as indicated before, and this has reduced the instance of dropped calls, but i persist in that the issues with calls dropping are due to a problem with the session border controllers disconnecting and failing to properly reconnect. Each instance of a dropped call I've had reported to me has been punctuated by an SBC renegotiation and re-authentication. I've seen each time that one or two SBCs take over a full second to renegotiate the connection. Each time I can verify the identity of the SBC as the one that the user was on as one with an elongated renegotiation period.

I don't know that its an issue with the software release, but after removing the SBC software, its registry entries and all traces of it from the program files and program data directories, what configuration mishap could possibly persist?

 

[cobaltit]

Any chance you did Windows Updates as part of your maintenance on those SBC boxes? We are almost exclusively Linux for installs, and especially for the SBC so I don't have any feedback for that. Perhaps you can swap out a box with a fresh box, or fresh Windows install, or something like a Pi perhaps just to rule that out?

 

[jhendrickson]

at this point my best bet is to install a prior version of 3cx server (im looking for the prior release, but I dont know that I have it. Anybody have a copy of it on hand?) and restore backups.

 

[AWS2P]

If your sbc's are windows 10, then did you apply WU, if so what is actual windows version V2004 ?
Have you done a complete on/off on these machine not only a reboot?

Cobaltit gave a good idea to test whith another completely fresh device like a PI .

 

[jhendrickson]

All of them are ver 1903 or 1909. I didn't push windows updates to them when I did software updates.
I'm going to reboot a few of them though and see if it magically does some good.

 

[AWS2P]

With windows machines, i already got weird things never ending until I do a power off (reboot alone change nothing) or in worst case needed to reinstall windows from scratch.

 

[jhendrickson]

well, it didnt fix this one i have more drops today.

users indicate that the yealink t46s says the tunnel failed on the display

 

[siyer17]

Updated to latest 16 and my calls were getting dropped as well. i had a different outbound ID for all 3 SIP trunks which i have. Removed the outbound ID and leave it to default and it works now. I am not sure if its a bug.