Crackling and Latency only on External Calls

[John Dugan]

I recently switched servers and migrated my 3CX virtual machine to the new server. Around the same time, I started experiencing call quality (crackling) and latency issues. There were none before the hardware switch. However, the issues are only on calls that target or originate to/from external destinations. Any internal calls or pages in my LAN have excellent quality and no latency issues. But if I call an external number (or answer an inbound call), I get extreme latency and quality issues. I am on a 200mbps x 10mbps connection, using Twilio as my SIP provider, and Grandstream GXP2170 handsets with the most recent firmware (1.0.9.135). Currently, my 3CX setup is deployed in my residence where I am piloting it for my business. Appreciate any guidance that the community can provide

 

[eddv123]

my 3CX setup is deployed in my residence where I am piloting it for my business.

Confirm - do you mean you are hosting at your residence for your business on another site ? or is this an on-premise deployment ?

Initially I would say that it sounds like your edge device (WAN) is causing a bottle-neck for traffic perhaps.
what is your Firewall as we have had mixed results with various models.

For external calls (if not already) I would try testing a lower bandwidth (compressed codec) like G729 or iLBC, we use Twilio also and by default the trunk codec is G.711 U-law which if you look at this guide takes up more bandwidth and could be a factor: https://www.3cx.com/blog/docs/bandwidth-utilised-for-voip/

It does not however explain why this issue has only occurred since server move, I would also check the serve to see what differences between the two are and if there is anything running on this new server (3rd party software, Windows firewall, antivirus) which could be effecting.

 

[JohnS_3CX]

Hi John

1. What VM platform did you use?
2. Does the 3CX Firewall Checker pass?

 

[John Dugan]

Hi John

1. What VM platform did you use?
2. Does the 3CX Firewall Checker pass?

1. Using the 3CX Debian 9.2.1 image on VMware 6.7.0 Update 2 (Build 13006603)
2. The 3CX Media Server does not pass. Not sure why. Port forwarding is setup on my Asus RT-AC88U per the docs.

 

[John Dugan]

An update - I ran apt-get update and apt-get upgrade. I am now on 3CX version 16.0.676. The firewall checker passes.

I still have the same quality and latency issues though.

 

[eddv123]

See my points from above, check your server over for the points specified.

The firewall checker "checks" for open ports and SIP ALG but unless something has radically changed recently it would not check for issues related to call quality.

Did you try a codec change also ?

 

[John Dugan]

See my points from above, check your server over for the points specified.

The firewall checker "checks" for open ports and SIP ALG but unless something has radically changed recently it would not check for issues related to call quality.

Did you try a codec change also ?

Neither of those codecs work for me. Per Twilio's docs, only G.711 U-law is supported.

However, I don't believe bandwidth to be the problem. I reliably get 175mbps x 6mbps on my Comcast connection. Regarding the server, As I mentioned above, I'm on the official Debian distro. That's all that I run on that VM. When I moved to the new server, the VM was just migrated over. I was running ESXi 7.5 on my previous server. I really hope it's not some low-level driver issue. Any other suggestions?

 

[John Dugan]

Any chance the issue could stem from the virtual switch setup in VMware? Does 3CX provide documentation on vswitch configuration for VMware (I couldn't find any)?

 

[Zol]

@John Dugan maybe this can help: https://www.3cx.com/docs/installing-microsoft-hyper-v/

 

[JohnS_3CX]

You are having issues with the media server not passing and, the media server is responsible for the audio. Looks like something is affecting the media traffic on the network level that you must investigate further.

You did not tell us though exactly what error the firewall checker presented, looks like the firewall is messing with your traffic though at a first glance

 

[John Dugan]

You are having issues with the media server not passing and, the media server is responsible for the audio. Looks like something is affecting the media traffic on the network level that you must investigate further.

You did not tell us though exactly what error the firewall checker presented, looks like the firewall is messing with your traffic though at a first glance

Unfortunately, I've since closed the browser tab. But I believe the error I was seeing on some Media Server ports was "full cone test failed". I could not tell you what ports. And I no longer experience the error since upgrading to v16.

I'm not sure what the deal is. We use Ubiquiti equipment at work. I am going to go ahead and swap out my Asus RT-AC88U with a USG. I'll update the thread following the change. Very frustrating.

 

[JohnS_3CX]

Hi John,

Ok, lets see hopefully this might change the situation if it is related to the router performance. Remember to reopen your ports on the USG, and under Routing&Firewall disable SIP and H.323 Conntrack Modules to stop SIP ALG. Then run the firewall checker again to see if everything passes.

Note: Avoid upgrading 3CX with apt upgrade in general. This might update certain dependent packages to a version that "breaks" compatibility with the system and could cause instability or failure. We update these packages for you, but selectively according to our tested versions, so if we discover any incompatible updates we block them. The apt upgrade command bypasses this check so we do not recommend it.