Call audio drops after 15 minutes, ended calls stay open

[ZorgNed - JDooge]

Since there have been some changes to our firewall/routing (I'm not exactly sure what changes; provisions were made to allow a 2nd internet connection with automatic failover) we have the following weirdness:

• Calls (doesn't matter if incoming or outgoing) will have audio dropped after 15 minutes, but stay active to our side. The other (external) party just has the call drop.
• Calls that are ended by the other (external) party will stay open for our users.

I've read about something like a 'SIP Refresh' or 'SIP Update' that checks every X time if a call is still active, and if not it will drop - but I can't find this in the protocol information, and our SIP trunk provider (Signet.nl) claims no such thing exists.

Internally there are no issues, only with external calls.

I hope to gain (at least read-only) access to the firewalls/routers soon (in maintenance by an external company) so I can have some insight in what is configured there. They are Sonicwall boxes and I've had trouble with that in combination with 3CX before (Sonicwall's smart SIP stuff can be interfering) - but until the recent change all worked without issues.

If someone has a helpful insight - please let me know!

 

[AWS2P]

I think you gave alone where to point --> your firewall
Here some 3CX docs about how to set sonicwall firewall
https://www.3cx.com/docs/sonicwall-firewall-configuration/

 

[ZorgNed - JDooge]

I think you gave alone where to point --> your firewall
Here some 3CX docs about how to set sonicwall firewall
https://www.3cx.com/docs/sonicwall-firewall-configuration/

Thank you! I'll see if I can discuss this with our network administrator when he is back from his holiday next week

To be fair all our clients reside inside the LAN next to the 3CX server, so all the NAT settings should not be required - however the SIP Transformations setting could very much be the key. Thanks!

 

[eddv123]

If you can capture this call using the PCAP feature from 3CX (and the call is not encrypted) then we can most likely see in Wireshark what is causing the drop, see here if you want to capture I would be happy to take a look: https://www.3cx.com/docs/capture-network-traffic/

 

[ZorgNed - JDooge]

If you can capture this call using the PCAP feature from 3CX (and the call is not encrypted) then we can most likely see in Wireshark what is causing the drop, see here if you want to capture I would be happy to take a look: https://www.3cx.com/docs/capture-network-traffic/

Interesting! Thanks, I will experiment with this when I find the time!

 

[JohnS_3CX]

Hi @ZorgNed - JDooge

A few things to consider here:

1. You have to ensure that ports are maintained when a failover event occurs, otherwise the PBX will lose communication with the other side.
2. Does the 2nd WAN have a different IP address? The PBX will not be aware of the failover event and you will probably have calls but no audio (PBX is still asking for audio on the old IP that is now gone offline). Go to Network Settings > External IP Configuration > set to Dynamic Public IP so the system will be able to detect a change whenever it happens.
3. The SIP Update or SIP Refresh you mentioned (sip timer if you want to look it up) is only useful if the provider supports it otherwise it is not really something that can be used. It's a way for the two sides to know if the other end is still there
4. If there are also STUN phones in the mix, it becomes more complicated: you must wait for the FDQN to update (which takes anywhere from 6 mins to 6 hours depending on licence type), then you have to also rely on the DNS server detecting the change at the phone's site, then you also have to rely on the phone device itself to realize that it's server IP has changed (depending on manufacturer implementation)

It's quite a few things, but there are ways to eliminate them and look into them one by one

 

[AWS2P]

is your sip trunk provider a 3CX supported one ?
You are sure your firewall use 2nd internet connection only for failover, not load balancing?

 

[ZorgNed - JDooge]

Hi @JohnS_3CX - thanks for your extensive reply!

1 & 2: The IP does indeed change, but we do expect calls to drop when a failover happens.

3: Ah, perhaps then the provider does not support it. When Googling (verb) I found that this was most likely the cause - but perhaps it is not even used, and the cause is an entirely different one.

4: No STUN phones! In theory we have 3CX (Windows desktop) clients that occasionally connect remotely (for example, when I use my laptop at home) but we don't support this towards our users.

@aws2p It is a supported provider yes (https://www.3cx.com/docs/signet-dutch-sip-trunk/) and it should not be used for load balancing (nor have I seen any indication that it does so far). Sadly I still don't have access to the firewall.

Thanks again for the answers everyone! This has given me some things to try out and check!

Also as more generic information:
We're currently running 3CX Professional Annual 16.0.910 (I'm sure it will auto-update to 16.0.3.676 this weekend) using the Linux installation ISO. It has been running without issues for years (started with 13 or 14 I think, before my time) and this is new behaviour since the firewall changes.

 

[BrenttG]

What vendor is the firewall?

 

[eddv123]

What vendor is the firewall?

See initial post - SonicWall.

I have deployed 3CX with plenty of SonicWalls before, where there was a bit of configuration required they work perfectly fine for me.

 

[BrenttG]

We have seen this problem before on SonicWall installs, there is a setting we had to change to correct it, but I am not remembering it at the second.... We stopped using SonicWalls a while back. But we had these exact same symptoms I believe.

 

[eddv123]

The ones I installed with SonicWall were using VPN's to cloud system so the traffic was not traversing the firewall technically, probably explains it.

 

[BrenttG]

The ones I installed with SonicWall were using VPN's to cloud system so the traffic was not traversing the firewall technically, probably explains it.

That was our quick fix as well putting a VPN tunnel up would bypass the firewall engine and fix the issue number one without actually fixing the underlying cause. we did eventually find the root cause which was a settings change but I cannot for the life of me remember which setting it was that was causing it.

 

[eddv123]

That was our quick fix as well putting a VPN tunnel up would bypass the firewall engine and fix the issue

Believe me we deploy most systems on VPN as long as the customer does not have an issue with us setting up a LAN to LAN connection to their local network.

By-passes a lot of issues and does not provide extension restrictions for our larger partners (like SBC does). I know you can use multiple SBC's but it is not an option I am in favour of.

 

[JohnS_3CX]

Hi @ZorgNed - JDooge

1 & 2 - The calls will be lost during the failover in our current scenario, unfortunately this cannot be avoided because you establish a session with the provider over your first IP, but now this is offline (failover) so the traffic leads nowhere. The only solution here is to hang up and call again (to cause PBX to detect new IP immediately when Dynamic mode is enabled). The firewall must be configured to pass the the same ports when you failover (this depends on you and your firewall config). A way to avoid this altogether is if you have a cloud PBX with static IP, and your office connects to it via SBC or VPN - a much more elegant solution and I would also say more robust.

3 - If it is SIGNET then they support it as I see from their invite, but a wireshark capture can help confirm the reason it fails. They are a registration based 3CX supported provider.

4. If you have a cloud based PBX the clients should not be affected, the static IP and the high availability ensures this even on a non-enterprise license because the DNS entry will always show the same static IP

 

[JohnS_3CX]

The ones I installed with SonicWall were using VPN's to cloud system so the traffic was not traversing the firewall technically, probably explains it.

The solution that @eddv123 describes should be the most elegant one.

If your office WAN1 drops and you failover to WAN2, the VPN will be re-established with a cloud based PBX. The IP change stops being an issue this way, and the firewall rules also stop being an issue.

Meanwhile, the remotely connected windows, mobile or web clients will not be affected at all. This is probably the most robust solution, and IF the office failover happens quickly enough then -hopefully- no calls will be dropped and they might even resume after a small period of silence without the users complaining.

There are no guarantees, but there are solutions that are more robust than others on a per-case basis, depending on your needs and whether you can justify the cost. You can also contact one of our partners of your choice to offer you a bespoke solution based on what equipment/trunk provider/ISP you have, that can minimize the downtime within realistic margins.

 

[ZorgNed - JDooge]

@JohnS_3CX On 1&2: That makes total sense, and to be fair that is also the behaviour we expect (calls dropping on failover). However no actual failover is happening in these cases.

On 3: Ah hm, the person I spoke to from SigNet had no idea what it was, but perhaps I had bad luck with reaching the right person I'll do a capture test after hours if I get the time. Forgot to do it yesterday.

4: I'm thinking the clients should be affected because the connection to the PBX is momentarily gone right? Will they gracefully recover during a failover and how will they handle a poor connection?

However this is not a bad idea - I will take it up with my coworkers to perhaps move the PBX to our cloud platform; we were planning to have a site-to-site VPN with it anyway.

I will still continue the testing because I wonder what is causing the current behaviour. Oddly enough this worked just fine before with the exact same firewall.

@everyone: Thansk!

Edit: To add; I've used Site-to-site VPNs before to get SonicWalls working with 3CX, but that was a more complex setup with a cloud PBX but also multiple sites that had to call eachother internally, all running of TZ400's.

 

[JohnS_3CX]

3. It's not always easy to reach a good support agent, but if you are patient enough you will eventually find one that is familiar enough with SIP to be able to check (or escalate it to their NOC engineer)

4. It really depends on a lot of factors and should also be tested live, there are many IFs in there

Anyway, from the general description of calls dropping at 15 mins probably related to the timers, and I believe the provider should be able to help you on this, or at least point out where the problem is. Do your captures first before you contact them and see what is going on.

 

[AWS2P]

can you check how are your sip settings and eventually test with supports Re-invinte unticked if it is

 

[ZorgNed - JDooge]

@aws2p Thanks, I checked but this option is already unchecked.