Choppy audio Issue

Discussion in '3CX Phone System - General' started by Randika, Sep 15, 2015.

Thread Status:
Not open for further replies.
  1. Randika

    Joined:
    Jan 19, 2015
    Messages:
    22
    Likes Received:
    0
    Hi

    Our company is experiencing a choppy audio issue with internal and external calls. After investigating it further I have managed to trace a bad call (choppy audio) from Wireshark and following are my findings.

    1. When I hear the actual call recording I can hear the jerky audio at the end of the conversation (the external party heard jerky audio from our user)
    2. When I hear the call from Wireshark RTP player I hear clear conversation (from the user's phone to the 3CX server)

    Which suggest me that there is nothing wrong with the packets went from the phone to the 3cx server and somehow 3CX audio mixer failed to deliver the audio properly to the other party. (we use PBX deliver audio option in 3CX)

    Does anybody knows actual cause for this issue ? (I have all the traces with me if any one interested in investigating this)

    Thanks
     
  2. lneblett

    lneblett Well-Known Member

    Joined:
    Sep 7, 2010
    Messages:
    2,083
    Likes Received:
    61
    You can only hear what the Rtp streams are from within the LAN, unless you have mirrored to a WAN. You will likely hear good audio from the phone to 3Cx, but the reverse may not be true. If the router is not able to handle the streams, then you may hear the phone from within the lan just fine, but the caller may not. The caller is listening to the stream as how it was received at his end. So, if the router is having issues feeding data to the Internet and there is no Qos to prioritize outbound or if the network in between 3cx and the caller is congested, then jitter and/or dropped packets are likely at the caller end.
    It may also be that you simply do not have enough internet bandwidth to handle the load of phone and data needs.
     
  3. millsey

    millsey New Member

    Joined:
    Dec 21, 2011
    Messages:
    190
    Likes Received:
    0
    Hi,

    Randika's investigation is for local audio, even between internal extensions. The RTP packets captured by WireShark on the server iteself are all fine, but the call recording done by 3CX, and what the other party hears, is broken audio. This can happen during a call or for the whole time of the call.

    We use VLANS for the general network (which is used for 3CXPhone traffic), a separate VLAN for the phones and a further VLAN for SIP trunk. We cannnot find a pattern to this but we need to resolve ASAP. Everything points to the 3CX audio mixer so the ports it is mixing the audio for, or it is getting overloaded (we are not seeing that one the server) or something like that.

    Any ideas or things to check for appreciated on this problem..

    Millsey
     
  4. lneblett

    lneblett Well-Known Member

    Joined:
    Sep 7, 2010
    Messages:
    2,083
    Likes Received:
    61
    A little confused -
    From you:
    "The RTP packets captured by WireShark on the server iteself are all fine"

    From Randika:
    1. When I hear the actual call recording I can hear the jerky audio at the end of the conversation (the external party heard jerky audio from our user)
    2. When I hear the call from Wireshark RTP player I hear clear conversation (from the user's phone to the 3CX server)

    As he referenced the "external" party specifically, that is what I was addressing. In his description, he gave me the indication that inbound audio from the external party was fine as he indicated "clear conversation", only that of what the external caller heard which is outbound.

    So now that I have a better appreciation, there are a number of questions:
    1. What version of 3CX is in use and what hardware and OS is it running on?
    2. What type of hard drive is used and is recording of calls being done? (there have been some reports of bad audio, but mostly as it pertains to playback from some older SSD devices). CPU loading is also a factor in this instance.
    3. What codec is being used and are all devices set to use the same codec in the same preference, assuming the US, then use G711u, else G711a. BY using the same codec everywhere the CPU is not as burdened trying to transcode every stream.
    4. Are there other applications such as Mail Server, Sql Server or even Anti-virus programs running on the same machine and/or scanning any of the 3CX folders.
    5. Are the WIndow's firewall or other 3rd party firewalls configured correctly or disabled.
    6. Is there more than one NIC running on the server where 3CX resides?
    7. If you remove PBX provides audio from a couple of the extension and have re-invites enabled, can the two internal extensions then have a successful call in both directions?
    8. What type of phones (curiosity)?


    On the surface, it does seem that there is an issue with the system itself. Keep in mind that when the PBX is providing audio, it then becomes the media server (audio streams handler) in both directions. So it is taking in a stream from both sides (phones) and then processing it back out the other side to both phones again. I tend to think that if there is a problem with the media server itself, that both the caller and callee would hear issues. I also tend to think that the software has been in thousands of locations and that if this was an inherent problem with 3CX, there would be far more with the same issue. The baffling part is the internal call issue which is, for the most part, pretty rare. I am assuming the 3CX server is dedicated given that it is a member of all 3 VLANs, but I have to ask about the other to be sure.

    Is this a new install or one that has been around for awhile? If not new, has anything changed? I am not sure why so many VLANS, but I assume you have your reasons.

    You could try and do a direct SIP call from one extension to another and validate the call quality. Of course, by doing this, you eliminate any possible issue associated to the VLANs.

    Sorry for all the questions, but other than the fact that you have jitter on both internal and external calls, the details are pretty sparse.
     
  5. TCF

    TCF

    Joined:
    May 6, 2015
    Messages:
    92
    Likes Received:
    2
    Did you recently update to Version 14? We have had the same issue since doing the upgrade and are still trying to find a solution.
     
  6. Randika

    Joined:
    Jan 19, 2015
    Messages:
    22
    Likes Received:
    0
    Hi lneblett

    Thank you very much for the reply, I have found the answers for your questions.

    1.3CX Phone System Professional Edition version 12.5. (number of simultaneous calls 64)
    Windows server 2008 R2/Intel Xeon 2.13GHz processor/16GB RAM

    2.We record all calls, Disk – HP drive speed 7.2K, member of RAID 1+0

    3. PCMU is used as the preferred codec for everyone and PCMA as the second preferred

    4. Antivirus Program (ESET)/ DHCP server to serve IP only for phones/ and we have our own system which converts wav files to MP3 (lame).It runs as a service in the server and it does the conversion when the wav file is not locked by the system.

    5. Windows firewall is off and we use Cisco ASA to connect to the SIP provider (Node4). (This firewall is dedicated to VoIP traffic - 3cx server).

    6. We have 3 network interface cards in 3cx server. Basically We have Voice and Data VLANS separated. However, we are using 3cx My phone app as well in our user’s machines. Therefore, we need to have the 3cx server in both those VLANs (Data and Voice). In addition, for SIP provider we have separate network card.

    7. Yes I have tested without PBX deliver audio and with “support re-invites” option between two extensions and had a successful call in both directions

    8. Grandstream 2130 and Grandstream 2100

    This is not a new install and we haven’t done any changes recently also this is an intermittent issue, Sometimes we only get 1 bad call in every 100 calls or so.

    I think following information may be also useful

    We have over 300 phones
    2Gbps link between core switch and server NIC

    Thanks
     
  7. TCF

    TCF

    Joined:
    May 6, 2015
    Messages:
    92
    Likes Received:
    2
    For me this issue was caused by so many ports in the 9000-9099 range UDP being opened and transferred between interfaces on my Router/UTM. Once I isolated the open ports to only my phone network the choppiness went away, seems to be a problem created by packet collision.
     
  8. lneblett

    lneblett Well-Known Member

    Joined:
    Sep 7, 2010
    Messages:
    2,083
    Likes Received:
    61
    the port range should present no problem given that you also pointed out that the calls had jitter even on internal calls. The router never came into play. Each call requires 2 ports. As long as 3CX itself is bound to the one NIC with the appropriate IP, that two should be OK. 3CX does no support multiple NICs. It does sound like the multiple paths with the segregation and subsequent member aggregation on the VLANs was/is causing some issue.
     
  9. millsey

    millsey New Member

    Joined:
    Dec 21, 2011
    Messages:
    190
    Likes Received:
    0
    Thanks very much, we do have 2 network card ports aggregated together (HP server teaming one side, Cisco Etherchannel on other side) and a quick check around looks like this is a bad idea even if this is not our issue. I have disabled one of the ports and and we will fully disband this tomorrow.

    Millsey
     
  10. millsey

    millsey New Member

    Joined:
    Dec 21, 2011
    Messages:
    190
    Likes Received:
    0
    Disabling one of the ports in the etherchannel was a bad idea, DHCP stopped working and the phones gradually lost connectivity overnight. We will be properly disbanding the etherchannel in the next day or so and I will report back.
     
  11. Randika

    Joined:
    Jan 19, 2015
    Messages:
    22
    Likes Received:
    0
    Hi All

    Thanks all for your replies

    If 3cx does not support Multiple NIC's , Is there any other way we can configure soft-phones to be in our Data VLAN and the hard phones to be in Voice VLAN (all the soft phones are installed in users machines which are in our Data VLAN)

    To achieve this, the 3cx server need to serve both VLANs and that is why we thought of using multiple Nic's.

    If we implement Inter VLAN routing between two VLANs (Data&Voice) Can we use only one NIC and and use both
    softphones and hardphones

    Any suggestions ?

    Thanks
     
Thread Status:
Not open for further replies.