Solved Suddenly, SBC constantly disconnects and reconnects - no changes to network

[wars]

Hey,

I am pulling my hair out here. I hope someone can point me in the right direction. For the last 2 hours our PBX has basically been offline.

Setup is - 3CX Cloud - Linux SBC is local with Polycom & Yealink handsets. NO network changes. SBC constantly disconnects and reconnects. I am able to SSH to the SBC and can run a constant ping to google.com with no packet loss.

I have checked both the SBC logs by running - 'tail -f /var/log/3cxsbc/3cxsbc.log' and this is what I get - (I've replaced data to make it anonymous)

SIP/2.0 100 Trying
Via: SIP/2.0/UDP 10.70.100.179:5060;branch=z9hG4bK511776126
To: "Warren T" <sip:[email protected]:5060>
From: "Warren T" <sip:[email protected]:5060>;tag=1877268325
Call-ID: [email protected]
CSeq: 1 REGISTER
Content-Length: 0

Also lots of these messages (different IP's for various handsets)

DEBUG | 20220914-182415.501 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.220:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.546 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.8:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.647 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.38:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.693 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.187:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.748 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.38:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.767 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.179:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.770 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.5:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.792 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.187:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.799 | 3CX | RESIP:TRANSPORT | 140586976814848 | Transport.cxx:397 | incoming from: [ V4 10.70.100.188:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.803 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:1119 | Found transport: [ V4 0.0.0.0:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.803 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:918 | Looked up source for destination: [ V4 10.70.100.188:5060 UDP flowKey=12 transportKey=1 ] -> [ V4 10.70.100.173:0 UDP flowKey=12 transportKey=1 ] sent-by=10.70.100.188 sent-port=0
DEBUG | 20220914-182415.803 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:1335 | Transmitting to [ V4 10.70.100.188:5060 UDP flowKey=12 transportKey=1 ] tlsDomain= via [ V4 10.70.100.173:5060 UDP flowKey=12 transportKey=1 ]

Lastly, not sure if this is relevant -

sigcomp id=
DEBUG | 20220914-182415.230 | 3CX | RESIP:TRANSACTION | 140586976814848 | TimerQueue.cxx:50 | Adding timer: Timer J tid=d00ba572B035E5CF ms=32000
DEBUG | 20220914-182415.231 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:1119 | Found transport: [ V4 0.0.0.0:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.231 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:918 | Looked up source for destination: [ V4 10.70.100.5:5060 UDP flowKey=12 transportKey=1 ] -> [ V4 10.70.100.173:0 UDP flowKey=12 transportKey=1 ] sent-by=10.70.100.5 sent-port=0
DEBUG | 20220914-182415.231 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:1335 | Transmitting to [ V4 10.70.100.5:5060 UDP flowKey=12 transportKey=1 ] tlsDomain= via [ V4 10.70.100.173:5060
UDP flowKey=12 transportKey=1 ]

SIP/2.0 407 Proxy Authentication Required
Via: SIP/2.0/UDP 10.70.100.5;branch=z9hG4bKd00ba572B035E5CF
Proxy-Authenticate: Digest nonce="414d535963220e3f07:b3426e6e023d33e6c92b6b9338645c48",algorithm=MD5,realm="3CXPhoneSystem"
To: <sip:[email protected]>;tag=06fc2b19
From: "Andy B"<sip:[email protected]>;tag=7814F1F0-911CE815
Call-ID: b6c7aeddd4767c38f24313d7ecd0573b
CSeq: 1 SUBSCRIBE
Content-Length: 0

sigcomp id=
DEBUG | 20220914-182415.231 | 3CX | RESIP:TRANSACTION | 140586976814848 | TimerQueue.cxx:50 | Adding timer: Timer J tid=2d12c42d1623B68E ms=32000
DEBUG | 20220914-182415.231 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:1119 | Found transport: [ V4 0.0.0.0:5060 UDP flowKey=12 transportKey=1 ]
DEBUG | 20220914-182415.231 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:918 | Looked up source for destination: [ V4 10.70.100.187:5060 UDP flowKey=12 transportKey=1 ] -> [ V4 10.70.100.173:0 UDP flowKey=12 transportKey=1 ] sent-by=10.70.100.187 sent-port=0
DEBUG | 20220914-182415.231 | 3CX | RESIP:TRANSPORT | 140586976814848 | TransportSelector.cxx:1335 | Transmitting to [ V4 10.70.100.187:5060 UDP flowKey=12 transportKey=1 ] tlsDomain= via [ V4 10.70.100.173:5060 UDP flowKey=12 transportKey=1 ]

SIP/2.0 407 Proxy Authentication Required
Via: SIP/2.0/UDP 10.70.100.187;branch=z9hG4bK2d12c42d1623B68E
Proxy-Authenticate: Digest nonce="414d535963220e3f02:816be6fda06758a75f401f0c22f0f9b3",algorithm=MD5,realm="3CXPhoneSystem"
To: <sip:[email protected]>;tag=8e3aaf7e
From: "Rob B"<sip:[email protected]>;tag=29C37EDC-9A5DA399
Call-ID: f5e8e1395b62f0e9994db0e06c8bc235
CSeq: 1 SUBSCRIBE
Content-Length: 0


I've rebooted the SBC, changed from VM to physical, rebooted the PBX, internet connectivity at the site is great, no firewall rule changes etc.

I have recently moved the SBC from a VM to physical box. VM still exists and was working fine for months. I have since reverted back to the VM in case the physical box is causing issues, I am still getting the same disconnect/reconnect constantly.

Whilst writing this post, a different alert has come in from the system, I haven't got any internet issues I'm aware of however maybe, maybe I have.... -

Registration at VoiPTalk 844300000 has failed.
Destination (sip:77.240.48.94:5060;transport=TCP;lr) is not reachable, DNS error resolving FQDN, or service is not available.

Any thoughts? I'm going to get onto our ISP now to see if they have an issue.

 

[NikosT_3CX]

@wars we have checked your machine and also we have made some tests and there is no outbound packet loss from the machine. Yes I would recommend to check this with your ISP in order to ensure that this leg of the ttraffic does not have any packet loss.
If you have any further questions or if there is any way we can assist on this, please let us know.

 

[wars]

@wars we have checked your machine and also we have made some tests and there is no outbound packet loss from the machine. Yes I would recommend to check this with your ISP in order to ensure that this leg of the ttraffic does not have any packet loss.
If you have any further questions or if there is any way we can assist on this, please let us know.

Hi Nikos,

Thanks for looking in to this for me. I've spoken to our ISP and they have confirmed that there hasn't been any outages. Our issue was between 17:00GMT and 20:00GMT. After this time, the SBC's reconnected and stayed up. The PBX emailed me the alert to say it was having DNS issues reaching our SIP provider, this came directly from the PBX so IT was having DNS issues out to the provider, not to us, or using any portion of our network. My suspicions lie at the PBX connectivity.....

 

[NikosT_3CX]

@wars okk now it makes sense since it was yesterday, i thought issue was again this morning again as this post was moved to Hosted category this morning and i got notified
Now for yesterday, yes there was an issue with the network of the Hosting Provider.
We identified this immidiately from our side we we were actively working with them to solve it as soon as possbilt, which happened at the end. Please check here : https://status.digitalocean.com/incidents/yhz9frgy4ngl