One second delay/lag during calls

[nell1000]

Hi, setup is as follows:

3CX PBX connected to Voiptalk SIP trunk via Draytek Vigor 2830n. We have around 30 Cisco SPA504G handsets (FW mixed from 7.4.8.a - 7.5.4 / Codec G711u) connected to 2 x Cisco SG300-28P switches. We also have a SG300-52p switch that connects the data. All on ONE VLAN currently.

The Draytek router is on the latest FW and has QOS enabled to prioritise VOIP, bandwidth limitations are also in place for the data IP's limiting to 2Mbps up/down. The Internet connection is a leased line 16Mbps up & down.

The main issue I'm experiencing is around a one second delay/lag during conversations. Most noticable when multiple users are on the same conference call in the same room. There is also "hesitation" on calls due to the delay.

Ive been racking my brain as to what the potential issues are but must be network based. I have checked the following:
1.) Internal calls carry no lag
2.) Checked the internet connection and no packet loss, jitter hovers around 3ms mark so looks like connection is usitable. Have been monitoring bandwidth usage from the router and never reaches anywhere near the top end.

Thoughts?

Nell

 

[3CXfoxhallsolutions]

Hi Nell
The SPA504G has a network jitter buffer setting that assumes medium to bad (it defaults to 'high') ... So with your low latency I'd change the 'Network Jitter Level:' setting on each active Extension page (Network Settings), and on each phone - to 'low'.

The other thing to look at is how you have your codec's set both on the phones, and on the SIP Trunk interface ... Are these both set to G711u?? If the SIP Trunk is prioritising e.g. G.729 - that could introduce a delay ...

Hopefully that may help ...

 

[nell1000]

Many thanks for the reply, I'll try over the weekend and report as its driving me to insanity! I contacted the sip trunk provider but they merely confirmed what codecs were usable and didn't highlight priority. If trunk/extension codec priorities are not aligned will this cause issues?

Again many thanks!


Bell

 

[3CXfoxhallsolutions]

Shouldn't cause a problem - but if they have told you what codec's they have - then you can set your SIP gateway interface in 3CX to just use one of the codec types - run some test calls for a while, then change to another codec - run some more tests, and see if there is one codec that negotiates the connection faster than another ...

For example, set your SIP Trunk gateway to use only G.711u (i.e. make sure its the only codec in the list), prioritise G.711u on your phones, then sit back and watch ... You could also wireshark the calls and see what codecs are being negotiated and used ... that's a nice way to answer the question quickly.

best regards

 

[lneblett]

I am a little confused about the issue -

You define the problem to be a hesitation or lag between conversational segments. I take this to mean that one side speaks and then the other side, due to the delay, may start with a response as they perceived the pause to be the other side having finished their statement. If long enough, the two parties start to over talk one another or start colliding. It tends to make one think that the need to say "over" at the end of their respective statement is needed in order to provide an affirmative acknowledgement that the other side may now respond.

If this is the case, then it is more commonly referred to as latency. The voice data packets encounter a delay arriving at their destination, but the speech is very understandable as the packets, although delayed, tend to arrive in the same manner in which transmitted.

Jitter is when the packets do not arrive in the same manner. Some arrive sooner than others as it may have to do with network or internet congestion. The speech will be likely understandable, but sounding somewhat broken or stuttered. Jitter buffers capture the packets and introduce small delays in order to try and allow some time for the lagging packets to arrive so that the end user listening is not subjected to the effect.

You indicated that it is most notable on conference calls when mutiple users are in the same room on the same call. I think we could use more clarity on this aspect. Is there a conferencing service setting up the calls? Are internal folks all using their own phones to join or do they listen to a common speaker phone? How many internal and external users participate and is the latency impacted by their number?

Codec and jitter buffer settings need to be set and preferably uniformly in your org., as they will help, but overcoming a 1 second latency is generally beyond their capability.

 

[nell1000]

Thanks for the reply;
@ineblett: it's on both setting up and receiving con calls. An example would be 4 participants in the same office on the same con call setup remotely. They can hear each other locally as well as on the call which is delayed. They are all on individual handsets, not a conference phone.

@3cxfoxhallsolutions: have tried your reccomendations but nothing seemed to help.

I eventually found the culprit. The PBX was setup on a windows 7 vm on hyper-v host. I recreated the PBX on a physical machine and the latency disappeared, so there must be an issue with the vm setup either with the vm itself or the virtual network. Whilst the vm was still operational I monitored resources but the CPU, memory and network utilisation were all pretty minimal so am concluding that it must be a network config issue.

Any wise words appreciated as is your assistance so far.

Many thanks,

Nell

 

[lneblett]

An interesting issue. My guess is that it is not the network. You have indicated 30 phones of which they have been allocated some 14Mbs, which is way more than adequate for all 30 to be on simultaneous calls regardless of the codec employed.

When the 4 in the office are on the conference call and one of the 4 is speaking, do the other 3 experience the delay in hearing the 4th? Or is the delay mostly encountered when the communications is external to internal and/or internal to external?

As an aside, what was the ping time to your provider or test server when you were running jitter and packet loss testing?

Most of the time when I have seen latency, it was with calls that traversed very long distances and usually on analog lines. This is oftentimes observed on fax calls when faxing overseas and you have to setup the fax machine to expect the delays. I also had one case where the route that the call took introduced a delay. We ended up having to change the provider.

My guess is either the conference call service itself or your VM instance not having enough resources. However, with 30 phones, I cannot imagine that you would not frequently have more than 4 calls going at once to external numbers, so why would the delay not be present in this setting as well? You indicated all was well with internal calls, so I assume that the phones negotiated and the system got out of the way leaving the phones to handle the calls. On the external calls, I am guessing that the PBX is providing the voice. I also assume that when you indicate both setting up and receiving, this indicates that the 3CX system is also sometimes used to host the conference calls and the delay is also seen in this regard...correct?

 

[sigma1]

Off the top of my head:

- Snapshot happening?
- Other Guest on the host maxing out your NIC?
- Backup?

 

[nell1000]

@ineblett: when one of the 4 is speaking the delay the others experience is internal to external. It doesn't just happen on con calls but all calls. PBX is not used to host the con calls currently. I do not have the ping times to hand but noticed nothing in the ping tests that caused alarm/suspicion.

@sigma1: nic dedicated to the PBX host, nothing else running on it. No snapshots or backup running.

Thanks,

Nell