Phones not returning to Main ISP after failover...

[Erik Nathe]

Good Morning,

We have been having this issue ever since we switched to a cloud based 3cx solution, hosted on Google's cloud services. We have Frontier as out main ISP with 500/500 and Spectrum as a backup with 100/10. What is happening is that when Frontier goes down, usually for just a moment overnight or early AM, everything switched to Spectrum. Then everything switches back to Frontier...except the phones! Once this happens the phone system also work much less reliably. About 50% of incoming calls don't go through, same for outbound calls. In order to force the phones back over I have to disable the secondary ISP from the gateway then re-enable it... That seems to be the only thing that will work.

Has anyone else dealt with this or have any idea how to resolve this?

 

[leejor]

Does it go down long enough that your FQDN gets updated? My question to Frontier would be,,,,why is your service dropping? Please remedy that.

 

[cobaltit]

@leejor
FQDN shouldn't change as 3CX is in GCE. It's just the phones that aren't failing over.

@Erik Nathe

You don't mention what phones you are using or how they are provisioned or your firewall which all might play a factor in this.

 

[Erik Nathe]

@leejor I definitely am inquiring with Frontier, though since it is only around once a month or so and always at a non critical time, it doesn't really affect anything else but the phones...

@cobaltit They are Fanvil X4's. Provisioning method is Direct SIP (STUN - remote)

Quite honestly I am not seeing anything in the way of firewall settings, I inherited this mess recently... We have a Ubiquiti USG-4pro for our gateway... I will run the firewall checker on 3CX today at noon just to be sure before I go and changing things...

 

[Erik Nathe]

Everything came back clear on the firewall check...

 

[cobaltit]

So STUN phones are supposed to have unique port forwarding ranges per phone. Since the USG line doesn't move the port forwards over to WAN2 on failover and since STUN generally works out of the box I imagine there's zero port forwarding for those phones. While I don't use Fanvil phones we do use the USG-PRO-4 with 2 providers and Yealinks with SBC and failover is not an issue here so. And since using a SBC is the recommended setup for multiple remote phones at a location I'd recommend going that route.

 

[dandenson]

This is NAT. Your router is tracking the connections and keeping them on your backup because it never goes down.

I solve this using 2 VPN tunnels hitting 2 addresses on an EC2 hosted concentrator and running iBGP over the link. You can then run your phones as 'local' and skip all the STUN and NAT issues. When WAN1 fails, BGP moves routing over to WAN2 with a very short delay. I also add in BFD so I lose about 500ms of data and don't even drop a call.

When WAN2 comes back up, BGP migrates the route back over.

Completely seamless. Forget all your NAT/STUN headaches.

 

[Erik Nathe]

Understood... though is it not odd that it is only the phones that stay on the backup? It successfully moves everything else back to the main ISP...

Do you know of any instructions to accomplish you suggestion?

 

[dandenson]

@Erik Nathe the phones are trying to keep their connection alive. Anything that creates a new connection is going to migrate gracefully, but things with keep alives will not.

If you can script on your router platform then you could kill all the connections on WAN2 when WAN1 comes back up but that will drop calls.

Certain phones can have a 're-register' button put on a soft key. That allows for a kind of manual fix.

If you are not super technical with this, you should look at peplink. they have products that make this very simple.

 

[Erik Nathe]

Admittedly phone systems are not quite my forte... Haven't had much experience outside of basic setup and troubleshooting... It seems that re-provisioning, re-registering, rebooting etc. has had no success in the past.. the phones would always come back up on WAN2.

I did find several others having the same issue on their Ubiquiti gateways... Seemingly they only way they can get them to move back over is disabling WAN2 or to clear IP NAT translations...

 

[dandenson]

@Erik Nathe re-registering probably only gets through these 'lazy' NAT engines on TCP. I prefer TCP for SIP because it solves a lot of NAT issues with strange implementations (mikrotik, ubiquiti, sonicwall).

You have to clear the NAT table out and some platforms just don't like to do that without a reboot.

 

[Erik Nathe]

Understood. Thank you all!