Solved 408 Request Timeout on First Call

[jwmilleril]

I am configuring a new SIP trunk which supports TLS with a local provider (not from the supported list). On the first attempt of a call, I get the subject message (Call or Registration to XXX@(Ln.10005@Coeo) failed. 0.0.0.0 replied: 408 Request Timeout; internal) after about 30 seconds. However, after the second attempt to make the call, it works and I now see the session in my firewall. Inbound calls are fine.

A wireshark capture shows that the 3CX server is retransmitting to port 5061 until it fails but I don't see any of that traffic on my firewall.

We have opened the appropriate ports to the provider's IP address.

We also have a TLS SIP trunk setup with Telnyx and that works fine so I am confident our firewall is seutp correctly. But, since we do have the firewall locked down to just the specific IP's for the providers, the Firewall Checker fails.

 

[leejor]

Have you tried removing/bypassing the firewall, as a test, to confirm that that is what is causing the issue?

 

[jwmilleril]

We confirmed our Firewall was dropping the initial packets when it was trying to establish a TLS session. For some reason, the 3cx Server was not setting the TCP SYN so our firewall discarded it. However, if the call was attempted again, the SYN was set and everything flowed normally. I simply deleted the SIP trunk and re-created it using a different template and it works without issue now.

 

[jwmilleril]

We confirmed our Firewall was dropping the initial packets when it was trying to establish a TLS session. For some reason, the 3cx Server was not setting the TCP SYN so our firewall discarded it. However, if the call was attempted again, the SYN was set and everything flowed normally. I simply deleted the SIP trunk and re-created it using a different template and it works without issue now.

Actually I spoke too soon. The problem has resurfaced. It's definitely coming from the 3CX server as the initial attempt to connect to our provider is missing the SYN flag so the firewall discards it. So why would it be fine on the 2nd try?

 

[YiannisH_3CX]

A wireshark capture shows that the 3CX server is retransmitting to port 5061 until it fails but I don't see any of that traffic on my firewall.

So if the 3CX is retransmitting where are the packets going? The capture is done on the NIC level so the PBX is sending traffic so you should see it on your firewall.

Are you using a IP based trunk with no registration?

So why would it be fine on the 2nd try?

Could the PBX be failing over to another trunk on the second call?

It's always better to use a supported provider as all we can do now is speculate.

 

[jwmilleril]

The packet capture showed that our firewall was dropping the first TLS handshake request because there wasn't a SYN prior. After the timeout and on a second call attempt, the SYN packets were properly sent and everything is fine. And as long as we have at least 1 call per hour, the connection stays active. We turned off SYN protection on the firewall and now the provider is dropping it so that's not much of a help.

Yes, it is IP based and I realize you don't officially support it. However, we have another TLS SIP trunk that works just fine with IP based (Telnyx) so I assumed this would work too. No dice, I guess.

 

[YiannisH_3CX]

OK, I see what the issue is. The TCP connection is closed in the firewall but probably the PBX is not receiving a reset for that connection. So for the next call it tries to use the same TCP connection. So after that fails the PBX creates a new TCP connection for the next call which works.
Telnyx is probably sending keep alive packets to keep the connection alive.

 

[jwmilleril]

OK, I see what the issue is. The TCP connection is closed in the firewall but probably the PBX is not receiving a reset for that connection. So for the next call it tries to use the same TCP connection. So after that fails the PBX creates a new TCP connection for the next call which works.
Telnyx is probably sending keep alive packets to keep the connection alive.

what does the keep alive feature in 3cx do? Wouldn't that do the same thing? It doesn't seem to work, though, as I tried that setting.

 

[jwmilleril]

I was able to solve the issue. Our provider's TCP reset was set as the same time as ours and our connection was always aging out instead of getting a reset. I created a custom application in the firewall with a TCP reset of 3660 seconds then tied it to our provider's IP and port 5061. After 1 hour, we got the TCP RST from the SIP provider and I was able to make a call without an problem.

 

[jwmilleril]

OK, I see what the issue is. The TCP connection is closed in the firewall but probably the PBX is not receiving a reset for that connection. So for the next call it tries to use the same TCP connection. So after that fails the PBX creates a new TCP connection for the next call which works.
Telnyx is probably sending keep alive packets to keep the connection alive.

Telnyx is resetting the connection after 10 minutes. Which works fine.

 

[YiannisH_3CX]

Glad you were able to figure it out.