Call lag and voicemail recordings glitchy

[Peter Richardson]

Hi all,

Been having 2 issues for the last couple of years which have started to become issues for our clients.

1) the lag on phone calls between an extension and someone on the outside world, be it mobile or landline, is quite bad. It's worse than calling the US from Australia, which I do on a daily basis. Not sure where to start with this one, any ideas?

2) when recording a new voicemail greeting or other prompt like IVR or call queue prompt, we get the occasional pop or crackle in the voice during recording and also sometimes a large delay of 1 to 2 seconds mid speech. This means that for some clients they must re record their new voicemail greeting up to ten times to get one without a pop or a delay. These clients are connected on 100/40 fibre at both ends so it's not internet connection speed related. It happens at times when there is no network traffic at either end. Same ISP at either end. Gigabit switches at both ends. The hosted 3CX is on a VM and shows only half memory usage and almost zero CPU usage. Yealink T48S with latest firmware or a PC with wired headset are the two extensions used to record. Any ideas on where else to look?

Many thanks in advance for any suggestions.

 

[JohnS_3CX]

Hi Peter,

Whats your setup like? Cloud VM with SBC or VPN on 2 user sites?

I would start with a basic traceroute and ping test towards your VM from within the network where the phones are installed to see what the network is like when reaching the PBX.

 

[Peter Richardson]

Hi @JohnS_3CX

We have our own servers in the city that we connect each client to via VPN, so no SBCs.

I just tested ping times from my home (which is on a wireless connection) via our data centre (fibre) to several of our clients (some on fibre, some on wireless) and even the wireless ones didn't go over 100ms. Fibre ones were between 50ms and 70ms, but most of that would be made up from my wireless connection. Either way, still plenty fine for this application, right?

 

[Rhys Haskell]

Hi @JohnS_3CX

We have our own servers in the city that we connect each client to via VPN, so no SBCs.

I just tested ping times from my home (which is on a wireless connection) via our data centre (fibre) to several of our clients (some on fibre, some on wireless) and even the wireless ones didn't go over 100ms. Fibre ones were between 50ms and 70ms, but most of that would be made up from my wireless connection. Either way, still plenty fine for this application, right?

If you go on the 3cx terminal and ping a phone on the customers local network what response time do you get?

 

[Peter Richardson]

@Rhys Haskell between 31 and 32ms on one of the worst performing phones.

 

[JohnS_3CX]

Ok then at this point you should probably move on to some captures to see what the network is doing.

Run a capture on the PBX, and one from the phone device interface and check them in WireShark. You can confirm that the audio stream has been affected by hearing the audio, as well as to see the RTP Stream Analysis that will at least confirm any suspicions of loss or high delta and jitter.

Seems initially like a network issue (since internal calls are ok) so I reckon it's a good place to start looking

 

[Peter Richardson]

Thanks very much @JohnS_3CX !

I did a wireshark capture from within 3CX and also generated the support files if needed. I listened to the audio from within wireshark and I noticed:

1. jittery voice
2. total silence (absence of caller's voice for a period)
3. massive delay (they were talking over one another)

I'm not an expert in wireshark (yet!) but any pointers would be greatly appreciated. I'll review the 3CX training on wireshark fault finding again today also.

 

[JohnS_3CX]

Hi Peter,

The internet speed is more than adequate but we can't know for sure that the packets reach their destination and are on time. So since the following apply:

1. Voicemail greeting has issues (Route: Phone site->PBX)
2. External calls have issues (Route: Phone site->PBX->Provider)
3. Internal calls are OK (Route: Phone site -> Phone site)

and your captures on your PBX end have lost/delayed audio then I guess this evidence points towards a problem in the communication between phone site and PBX as a minimum.

A quick and dirty way to figure out on which end likely has the problem lies is to check if external calls towards the PBX (IVR, listening to music on hold, and leaving a voicemail) are clean and uninterrupted. This would clear the link between your PBX and the provider, and you can now focus on the link from the site to your PBX with additional testing.





Tip: In WireShark you can can also go to Telephony > RTP > RTP Streams, then pick your streams and click Analyze which will show you more detail on loss and delay