5.0.3752.0 - Stun requests using Private IP/NIC

[wtl]

We trying to test 3CX v5 on a server at SoftLayer.com, where each server has two NICs, one with a private IP for your VLAN, and one with the public IP.

3CX seems to be using the private IP/NIC instead of the public one. How can I change this?

The evidence which leads me to suspect this:

[CM506004]: STUN request to STUN server 192.245.12.229 has timed out; used Transport: 10.4.0.16:5060

10.4.0.16 is our internal IP/NIC used for local traffic only. The STUN request should be going out over the other NIC in the machine. I can't seem to find any configuration option to correct this.

 

[Pentangle]

Does your private VLAN have a gateway anywhere else?

If not, don't include a gateway address in the NIC configuration.

If it does, you need to get the routing metrics correct on your TCP/IP stack.

As with the response to Zensoftware, this is a Windows and routing issue, not a 3CX issue.

Mike.

 

[wtl]

There is no default gateway on the private network NIC - never has been. With that said - any other ideas, or could this be a bug after all since there is no gateway set on the NIC?

 

[archie]

Yes, bug is that there's no default gateway on the NIC.

 

[wtl]

There is no default gateway on the NIC that I DON'T want 3cx to be using - the NIC that is assigned a private IP. The NIC assigned the public IP, the one i DO want 3cx to use, has a default gateway assigned.

The first person who attempted t help said to be sure that I DON'T have a gateway assigned to the private NIC, but now you're saying that's the problem - so I'm confused.

Given my problem - that 3cx is attempting to use the private network interface instead of the public network interface for STUN queries, are you sayin that with a default gateway set on the Private NIC, 3cx will stop trying to send STUN queries over it? Doesn't seem to make sense to me, but I guess I can give it a shot.

 

[archie]

Let's clarify than. 3CX uses windows routing to determine where to send packets. For external communications it tries to resolve external IP on every LAN interface, or uses public IP of interface if present. In your case, as I understood, you have a card with public IP, and LAN card. In this case 3CX shouldn't try to resolve external IP on public NIC, but still will try to resolve on LAN NIC. If LAN doesn't have access to internet - than this attempt will fail. You can also try to remove all STUN servers to disable STUN resolution at all, but I'm not sure if the server will properly work in all cases than. Also, if you adjust your routing table for LAN NIC to have less priority (higher metric) than public NIC - our server will use only public NIC than.

 

[Pentangle]

Yes, bug is that there's no default gateway on the NIC.

That's not a bug, Archie, that's a valid TCP/IP configuration.

 

[Pentangle]

Let's clarify than. 3CX uses windows routing to determine where to send packets. For external communications it tries to resolve external IP on every LAN interface, or uses public IP of interface if present. In your case, as I understood, you have a card with public IP, and LAN card. In this case 3CX shouldn't try to resolve external IP on public NIC, but still will try to resolve on LAN NIC. If LAN doesn't have access to internet - than this attempt will fail. You can also try to remove all STUN servers to disable STUN resolution at all, but I'm not sure if the server will properly work in all cases than. Also, if you adjust your routing table for LAN NIC to have less priority (higher metric) than public NIC - our server will use only public NIC than.

Archie, wtl

The Windows default behaviour is to work with it's routing table. If the STUN requests are being sent out on a 10.x.x.x address and there is no gateway on the NIC with the 10.x.x.x then there is either a manual table entry in the routing table or there's a bug with 3cx and it's binding to specific NICs (unlikely, if it's supposed to sit as a service on top of Windows).

wtl, if you can open a command prompt on the server in question and type "route print" (without the quotes) at the prompt, and then post the results here, we can see what your routing table consists of, and as a result understand why a STUN request to an internet server was being sent out by a NIC containing a private IP address and no gateway.

Cheers,
Mike.

 

[wtl]

Results of "route print" below.

All of the 10.x.x.x routes have the highest metrics.

IPv4 Route Table
===========================================================================
Interface List
0x1 ........................... MS TCP Loopback interface
0x2 ...00 30 48 2c b8 0f ...... Intel(R) PRO/1000 MT Dual Port Network Conn
n #2 - Packet Scheduler Miniport
0x10004 ...00 30 48 2c b8 0e ...... Intel(R) PRO/1000 MT Dual Port Network
ction - Packet Scheduler Miniport
===========================================================================
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0     70.87.128.49     70.87.128.50      1
       10.130.0.0      255.255.0.0      10.130.93.4      10.130.93.4     20
      10.130.93.4  255.255.255.255        127.0.0.1        127.0.0.1     20
   10.255.255.255  255.255.255.255      10.130.93.4      10.130.93.4     20
     70.87.128.48  255.255.255.240     70.87.128.50     70.87.128.50     10
     70.87.128.50  255.255.255.255        127.0.0.1        127.0.0.1     10
   70.255.255.255  255.255.255.255     70.87.128.50     70.87.128.50     10
        127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1      1
        224.0.0.0        240.0.0.0      10.130.93.4      10.130.93.4     20
        224.0.0.0        240.0.0.0     70.87.128.50     70.87.128.50     10
  255.255.255.255  255.255.255.255      10.130.93.4      10.130.93.4      1
  255.255.255.255  255.255.255.255     70.87.128.50     70.87.128.50      1
Default Gateway:      70.87.128.49
===========================================================================
Persistent Routes:
  None

 

[wtl]

Please ignore the above post. I printed the routing table on the wrong server - doh!

I tried to edit the post above, but though I was told the edit was successful, it never showed the new content.

So, here's the correct routing table:

IPv4 Route Table
===========================================================================
Interface List
0x1 ........................... MS TCP Loopback interface
0x2 ...00 30 48 59 77 ed ...... Broadcom NetXtreme Gigabit Ethernet #2 - Pa
Scheduler Miniport
0x10004 ...00 30 48 59 77 ec ...... Broadcom NetXtreme Gigabit Ethernet - P
 Scheduler Miniport
===========================================================================
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      75.126.69.1      75.126.69.4      1
         10.0.0.0        255.0.0.0         10.4.0.1        10.4.0.16      1
         10.4.0.0  255.255.255.192        10.4.0.16        10.4.0.16     20
        10.4.0.16  255.255.255.255        127.0.0.1        127.0.0.1     20
   10.255.255.255  255.255.255.255        10.4.0.16        10.4.0.16     20
      75.126.69.0  255.255.255.240      75.126.69.4      75.126.69.4     20
      75.126.69.4  255.255.255.255        127.0.0.1        127.0.0.1     20
   75.255.255.255  255.255.255.255      75.126.69.4      75.126.69.4     20
   82.201.246.188  255.255.255.255      75.126.69.1      75.126.69.4      1
        127.0.0.0        255.0.0.0        127.0.0.1        127.0.0.1      1
   196.205.135.66  255.255.255.255      75.126.69.1      75.126.69.4      1
  196.205.141.231  255.255.255.255      75.126.69.1      75.126.69.4      1
     208.101.44.0    255.255.255.0   208.101.44.236      75.126.69.4     20
   208.101.44.236  255.255.255.255        127.0.0.1        127.0.0.1     20
   208.101.44.237  255.255.255.255        127.0.0.1        127.0.0.1     20
   208.101.44.238  255.255.255.255        127.0.0.1        127.0.0.1     20
   208.101.44.239  255.255.255.255        127.0.0.1        127.0.0.1     20
   208.101.44.255  255.255.255.255   208.101.44.236      75.126.69.4     20
        224.0.0.0        240.0.0.0        10.4.0.16        10.4.0.16     20
        224.0.0.0        240.0.0.0      75.126.69.4      75.126.69.4     20
  255.255.255.255  255.255.255.255        10.4.0.16        10.4.0.16      1
  255.255.255.255  255.255.255.255      75.126.69.4      75.126.69.4      1
Default Gateway:       75.126.69.1
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
         10.0.0.0        255.0.0.0         10.4.0.1       1

So, I have a 10.0.0.0 route with a low metric, that is persistent. Is that a problem? I would think the system would be able to tell that a request to 64.69.76.23 is not going to be successful over the 10.0.0.0 route.

Or, maybe it's not using that route at all. According to the log, the stun request is using 10.4.0.16 as the transport, and there is a 10.4.0.16 route above.

Oh well, if that helps, great. I'm not knowledgeable enough about TCP/IP routing to know what's going on.

 

[wtl]

If anyone has any ideas here - particular 3cx staff - I'd appreciate it. I was using version 3.x previously without any problems, and am now ready to make a purchase of version 5 to implement call queues, but only if I can get it working, of course.

 

[Henk]

Tell windows what NIC to use as priority.

Network Connections
--> Advanced Settings
----> set the NIC you want as priority to the top of the list.

It might not be the problem, but it gets us on level play field. Let me know how you go.

H.

 

[wtl]

Henk,

Do you mean Network Connections -> Advanced -> Advanced Settings -> Adapters and Bindings -> Connections? (Where you list the adapters in the order in which they're accessed by network services?) If so, The NIC with the public IP is top of the list.

Pinging the stun server uses the correct route, so why doesn't 3cx?

 

[Pentangle]

Sorry for the delay - am still enjoying Christmas here

Your route map looks fine, since you have the default route (0.0.0.0, where all non-specified addresses get sent) as going out via the public IP address.

I think you may have found a bug here, unless as Henk says you can check the NIC bindings (if 3CX has a specific binding presence).

What o/s are you running on your server and i'll try and give you detailed instructions with which to check.

Cheers,
Mike.

 

[wtl]

Thanks Pentangle, and continue enjoying your Christmas. I appreciate the assistance provided by you and others.

The server is running Windows Server 2003 Standard SP1.

I couldn't find any NIC binding configuration options in 3cx. Perhaps they're hiding in a configuration file somewhere, unexposed by the web interface. However, I would hope a stock install of 3cx would not choose to bind to the wrong IP, especially considering the routing is good, and the OS itself knowns exactly how to route a TCP/IP request for the same STUN server that 3cx is having problems with.

 

[wtl]

Sorry, meant "bind to the wrong NIC", not "IP". (Can't edit a message once posted, for some reason.)

 

[Pentangle]

I just checked on my Win2003 server at home which is running 3cx v5. It appears there's no specific carrier/protocol defined by 3cx to be bound to a NIC, and hence assuming you've not previously changed your bindings (and considering you don't know where they are, I doubt it!) then it looks like either:

1) the STUN request goes out through all known interfaces and you're only seeing the failure on the private network whereas there might be a success you're missing??

2) there's a bug in 3cx STUN resolution which means it doesn't follow Windows networking. The issue with this is that if 3cx is written with default tcp/ip libraries etc, then I can't see why this would be the case.

At this point, we'd definitely need to defer to 3cx technical support, but given that we've already troubleshot the network config I would really like them to do some extensive testing on this issue as a result.

Cheers,
Mike.

 

[wtl]

Here's the log from the last time I tried running 3cx, in case it reveals any clues:

07:53:53.045 StunClient::onTestTout [CM506004]: STUN request to STUN server 192.245.12.229 has timed out; used Transport: 10.4.0.16:5060 
07:53:49.982 StunClient::onTestTout [CM506004]: STUN request to STUN server 192.245.12.229 has timed out; used Transport: 10.4.0.16:5060 
07:53:46.920 StunClient::onTestTout [CM506004]: STUN request to STUN server 192.245.12.229 has timed out; used Transport: 10.4.0.16:5060 
07:53:43.857 StunClient::onTestTout [CM506004]: STUN request to STUN server 192.245.12.229 has timed out; used Transport: 10.4.0.16:5060 
07:53:40.982 MediaServerReporting::Service *** Connected to MEDIA:5483/IVRServer at 127.0.0.1:5483 *** 
07:53:40.888 IVRConnected [CM501004]: Connected: IVR Service 
07:53:40.810 StunClient::onTestTout [CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 10.4.0.16:5060 
07:53:39.982 MediaServerConnected [CM112000] Media Server is connected 
07:53:38.092 CallMgr::onAddIPs IP(s) added:[75.126.69.4,10.4.0.16,208.101.44.238,208.101.44.237,208.101.44.236,208.101.44.239] 
07:53:37.763 StunClient::onTestTout [CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 10.4.0.16:5060 
07:53:34.701 StunClient::onTestTout [CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 10.4.0.16:5060 
07:53:31.638 StunClient::onTestTout [CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 10.4.0.16:5060 
07:53:28.748 StunClient::process [CM506002]: Resolved SIP external IP:port (75.126.69.4:5060) on Transport 75.126.69.4:5060 
07:53:28.592 StunClient::onInitTests [CM506001]: STUN request to resolve SIP external IP:port mapping is sent to STUN server 64.69.76.23:3478 over Transport 10.4.0.16:5060 
07:53:28.592 StunClient::onInitTests [CM506001]: STUN request to resolve SIP external IP:port mapping is sent to STUN server 64.69.76.23:3478 over Transport 75.126.69.4:5060 
07:53:28.232 DBA [CM501003]: Connected: Database 
07:53:28.154 CallMgr::findLocalIPs [CM501006]: Default Local IP address: [75.126.69.4, 10.4.0.16]
07:53:28.013 CallCtrl::thread [CM501007]: *** Started Calls Controller thread *** 
07:53:28.013 CallMgr::Initialize [CM501002]: Version: 5.0.3752.0 
07:53:28.013 CallMgr::Initialize [CM501001]: Start 3CX PhoneSystem Call Manager

What's the best way to get 3cx involved in looking at this? Their support services appear to be fee based. Will they perform pre-sales support?

 

[bigsiqc]

I am also getting this record in the log. Other than being annoying is there any problem with using the system? Everything seems to work fine here.

Simon

 

[wtl]

I am also getting this record in the log. Other than being annoying is there any problem with using the system? Everything seems to work fine here.

Simon

Hmm...hadn't considered that maybe these errors aren't a problem overall. Basically, I cannot get a voip line (Vitelity) to register, when I had no trouble at all in Version 3.whatever I was using previously.

The firewall test gives me an Error (13) "The machine is on a public IP. Please check the FAQ for more information." Of course, the FAQ doesn't give any information about this error, and the forums don't really give me much info other than at one point it was a bug in 3cx for systems on public IPs.

I've gone back and tried registering a line again, and the odd thing is that there are no recorded registration attempts in the log at all. The last entries are always the stun timeouts.

I'm about to give up. I'm not an idiot - in fact have been running a successful web hosting company for 6 years now. Version 3 didn't give me any trouble, but I can't make v5 work at all.

To make matters worse, 3cx apparently doesn't provide support unless you pay. I'm ready to shell out several hundred dollars for 3cx v5 if I can get it working, but unless 3cx can provide some sort of pre-sales troubleshooting, they'll lose my business. Who is going to purchase the product if they can't make it work?