SBC reliability

[hood]

Hi Team,

I have 1 site connected to a 3CX PBX located in our data centre that is giving me grief with call drop outs. There are 3 other sites connected via SBCs that aren't experiencing the same issues.

This one site has good comms with a 100/40Mbps connection and an average ping time to the PBX of 4ms along with no observable packet loss. The SBC is running on a Windows 10 Pro MFF PC pretty much dedicated to the task powered by an i5 1.6GHz proc, 8GB RAM and 256GB SSD. PBX is the latest build (updated 2 days ago) and the SBC is also the latest build. The issue pre-dates any recent upgrades, they were performed as troubleshooting steps.

Yesterday the SBC tunnel was dropping ever 5-15 minutes from what I could observe. I'll include a snippet of the 3CXSBC log from around a drop out to see if anyone can decypher it. Last night I went through some basics, and the link appears to be more stable (SBC tunnel staying up for 2 hours) but not as reliable as I would like still. Last night I disabled all NICs except fro the wired interface we are suing, ensured all Windows Defender protection was turned off, uninstalled 3CX SBC, rebooted and reinstalled. SIP ALG is not enabled, and the Windows 10 MFF PC running the SBC has a policy on the firewall to allow any traffic out unfiltered.

Any assistance would be greatly appreciated.

 

[cobaltit]

What's the different in firewalls between the site that is good and the site that isn't? Are all sites running under Windows? For the record I avoid Windows like the plague for SBC and preferably for 3CX itself as well.

As a troubleshooting step you can try changing the SBC from TLS to TCP

 

[JohnS_3CX]

Hi,

If you look at our hardware requirements, you will see that we recommend an i3 from Gen.8 or above.

There is a possibility that your 1.6ghz i5 (which is half the recommended speed) may not be able to keep up during certain moments, and this could be a potential source for trouble.
https://www.3cx.com/docs/recommended-hardware-specifications-for-3cx/

If you wish to keep that machine (even though not recommended), you should at least remove windows from it, and deploy the SBC using our Debian ISO. This will ensure that no resources go to waste on an already underpowered machine, and could improve the situation.
https://www.3cx.com/docs/3cx-tunnel-session-border-controller/#h.9hcxdspxfvz

 

[Alphabetic]

You also mention this machine is "pretty much" dedicated to be the SBC - What else is it doing?

I would boot windows to clean boot starting only the SBC and seeing if its other software causing you problems

 

[Evolute IT]

You also mention this machine is "pretty much" dedicated to be the SBC - What else is it doing?

I would boot windows to clean boot starting only the SBC and seeing if its other software causing you problems

Or use a Linux machine to prevent any Windows stuff from causing issues.

 

[hood]

Thanks all for the replies, I now have further information.

cobaltit, the other sites are basically the same. Same firewall, very similar internet connection, running on the same MFF Windows 10 Pro PC as the SBC. These sites are server less so I cannot run a debian VM SBC.

JohnS_3CX I would have expected this to be more than sufficient for an SBC supporting 4 handsets if it can run on a Raspberry Pi. The tunnel was even dropping after hours where there was no traffic or calls occurring. But thank you for mentioning the recommended specifications, I wasn't aware that they were so high for an SBC. The PBX itself is VM running on a server with 2x Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz processors. Deploying Debian isn't really an option, the site is remote.

kieferschild by pretty much only an SBC, we use it to remote into to access the site if we need to (Web interface for printer, switch, firewall for example) and also running Unifi controller software (not as a service and only running ad-hoc). 99% of the time it is sitting there being an SBC only.

I have another site connected to this same PBX and its SBC uptime is 9 days. And another site / PBX combo that is running the SBC on a Windows Server VM and has 5+ days uptime. Maybe it is the generation of CPU, but I can't see how it can be stressed running only 4 handsets.

 

[JohnS_3CX]

Hi @hood

Ok so first make sure your PBX and SBC are the latest stable versions (you did not mention any versions).
If the problem then persists, edit the SBC settings in your trunks and switch it to TCP rather than TLS, then click "Push Config". Let us know if the behavior changes once the SBC is running in TCP mode.

PS: A Pi might seem underpowered compared to a x86 but in reality it is an entirely different architecture, with the SBC optimized, compiled and tested for that specific platform (not apples to apples basically when compared).

 

[hood]

Thank you JohnS_3CX, I'll make that change now.

Sorry I'm sure I had mentioned the versions, but I didn't. The PBX is running 16.0.7.1078. (I'll push the 16.0.8 update now as well) and the SBCs are all 16.2.24.

I'll switch to TCP and see if that makes any difference.

 

[JohnS_3CX]

First ensure that the updates have been applied, and switch to TCP only if that didn't help

 

[Lee Cramman]

Thanks all for the replies, I now have further information.

cobaltit, the other sites are basically the same. Same firewall, very similar internet connection, running on the same MFF Windows 10 Pro PC as the SBC. These sites are server less so I cannot run a debian VM SBC.

JohnS_3CX I would have expected this to be more than sufficient for an SBC supporting 4 handsets if it can run on a Raspberry Pi. The tunnel was even dropping after hours where there was no traffic or calls occurring. But thank you for mentioning the recommended specifications, I wasn't aware that they were so high for an SBC. The PBX itself is VM running on a server with 2x Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz processors. Deploying Debian isn't really an option, the site is remote.

kieferschild by pretty much only an SBC, we use it to remote into to access the site if we need to (Web interface for printer, switch, firewall for example) and also running Unifi controller software (not as a service and only running ad-hoc). 99% of the time it is sitting there being an SBC only.

I have another site connected to this same PBX and its SBC uptime is 9 days. And another site / PBX combo that is running the SBC on a Windows Server VM and has 5+ days uptime. Maybe it is the generation of CPU, but I can't see how it can be stressed running only 4 handsets.

My eyes were immediately drawn to a few things:

1. You are running other services, including Unifi controller software (so networking is involved). It's not unusual for software like this to do things like install additional driver layers or have background services you aren't aware of that affect network traffic.. Even if this is not the cause, file under 'bad ideas'.

2. That you are running anything else on your SBC is a bad idea. Remoting into Windows can use a significant amount of system resources. I'm sure you wouldn't want your SBC to start having issues every time someone remotes in for support.

3. It can't be overstated how much better use a debian based SBC will make with limited hardware.

4. You're running on Windows. By its nature you could have differing versions of Windows, NIC drivers, security updates, etc. So while you can say that the machines are identical you can only really say that at a hardware level. Another reason to favour Debian.

5. Windows is also very resilient. A good thing but sometimes it ends up chugging along appearing to work when there are, in fact, problems 'under the hood' that are causing services to fail or resources to be gobbled up. It's worth checking the logs for anything unusual.

6. Have you confirmed that there are no transient problems on your WAN link that coincide with these drops? Maybe set a constant ping of your PBX going and capture the results for later analysis.

I know you mention Debian isn't an option but dropping an SBC onto a Raspberry Pi and posting it out to the site is an inexpensive / easy way of deploying when you have limited options, and also a method of ruling in / out the Windows machine as the problem. The costs are probably less than an hour of your troubleshooting time.

 

[ivank]

When running SBC on Windows one option is crucial: the power plan selected must be Performance. I would check this one first. The specs of the machine are good enough for at least 10 phones.
Next, I'd check Windows Event Log around the time of disconnects.

P.S. The log snippet is too short. It states that there were no incoming data over TCP from PBX for 15 seconds.

 

[ivank]

hood, I suggest you read this post as it contains precious info:
https://www.3cx.com/community/threads/windows-sbc-best-practice.70406/