Port being changed before reaching PBX

Discussion in '3CX Phone System - General' started by Peter Richardson, Dec 26, 2017.

Thread Status:
Not open for further replies.
  1. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    Hi all, here's a tricky one for you!

    I have been troubleshooting an issue with tech support for a while and it has reached the limits of my knowledge. Tech support have been very helpful, but I'm a little out of my depth. I get the concept, I lack the experience required to fix the issue.

    The phones are connecting on a random port instead of their assigned SIP and RTP ports. This works for a while, but eventually the phones drop their connection and are unable to reconnect to the PBX (they just keep trying). I'm guessing that at this point they are being blocked by the firewall.

    I have opened ports on the firewall on the PBX side (remote)
    I have NATed the ports to the phones on the firewall where the phones are located
    I am using pfSense firewalls at both ends
    I have created the port preservation rules at both ends as per this link:
    https://www.3cx.com/docs/pfsense-firewall/#h.mk14nzhjw26j

    I know that 3CX doesn't support firewalls or any network configuration which is totally understandable because of the many different firewalls on the market, and their countless configuration options, and 3CX's (then implied) liability for your network. I'm just wondering if anyone out there might be able to spot something that I have overlooked or a slightly incorrect rule.

    Let's just talk about one extension, the Yealink DECT cordless phone, this seems to be the one that causes the problem most of the time (it's rarely every any of the other phones that go down, but this is the most important phone to this client).

    The extension is 103 and you can see from the 3CX Activity Log screenshot that it is connecting on port 25342 (am I reading that right?)

    The Yealink interface screenshot shows that it is connecting on port 5060 for SIP, but shouldn't that be 5069 because that's what I have it set to in the PBX? (see attached)

    I have attached images of pfSense rules and NAT for local site and remote PBX site.

    I've run out of ideas, can anyone please suggest a place to start?
     

    Attached Files:

    #1 Peter Richardson, Dec 26, 2017
    Last edited: Dec 26, 2017
  2. Archie Frederic

    Joined:
    Jan 31, 2017
    Messages:
    69
    Likes Received:
    1
    Unless you had auto provisioned the phone, I think you should set the port in the account of yealink phone itself. otherwise you would really experience such problem
     
  3. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    Thanks Archie, yes the phones are on auto-provision.

    But to test the theory, I just logged into the phone and changed the port to 5069 and it immediately dropped and wouldn't connect. So I changed it back to 5060 and it immediately connected.
     
  4. Archie Frederic

    Joined:
    Jan 31, 2017
    Messages:
    69
    Likes Received:
    1
    looks like the error was on the data you had filled on the auto provisioning page which should have been 5060 instead of 5069 which was the one put on the configuration of your phone as a result of provisioning.
     
  5. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    Thanks Archie. Are you saying that there is a mistake in 3CX provisioning for the phone and it wasn't set to 5069? See the screenshot of the provisioning section of the phone, it says 5069, so that should all be okay, right?
     
  6. Archie Frederic

    Joined:
    Jan 31, 2017
    Messages:
    69
    Likes Received:
    1
    Regarding this matter. Please be advised that the port that the PBX used and the port that your phones use must match each other, otherwise they would have problems in communicating with each other. When you say that you had used 5069 as port of the phone, did you mean that you had also used 5069 as port of PBX?
     
  7. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    Have a look at this image of the PBX, it's set to 5069. So if the phones are provisioned, then why are the phones saying 5060? (see the screenshot from original post, the Yealink one).
     

    Attached Files:

  8. Archie Frederic

    Joined:
    Jan 31, 2017
    Messages:
    69
    Likes Received:
    1
    that was the setup for the DECT Phones, but what was the setup for the pbx itself?
     
  9. Archie Frederic

    Joined:
    Jan 31, 2017
    Messages:
    69
    Likes Received:
    1
    to check it, go to network settings then ports. It should match the port that was setup as the sip port.
     
  10. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    5060 and 5090 for tunnel.

    I have been advised by support to have a different sip port for each phone. There are 4 phones at this site so we have used 5066, 5067, 5068 and 5069. We have set up the firewall rules (remote and local) to accommodate this. (see screenshots from original post)
     
  11. Archie Frederic

    Joined:
    Jan 31, 2017
    Messages:
    69
    Likes Received:
    1
    well I'm not sure if that would work, but base on experience having different port from the pbx, you should be able to register on any account, but you would usually encounter call dropping especially if the two phones communicating has different ports with each other.
     
  12. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    Okay. So if all phones are using the same port, 5060, there would be no way to NAT the phones, right? Maybe I've received bad advice to use a different port for each extension? I could see that an installation of say 100 phones would be nearly impossible to implement and maintain if NAT was required for every extension!
     
  13. lneblett

    lneblett Well-Known Member

    Joined:
    Sep 7, 2010
    Messages:
    2,061
    Likes Received:
    56
    Let's step back for a moment and examine the situation a little.
    1. Was SIP ALG disabled?
    1. Are you using a SIP trunk and do calls seem to function correctly - in and out?
    2. Have you run the firewall test and did it pass?
    Assuming the answers to the above are "yes", then the pfsense is passing the NAT/PAT tests and is likely not an issue if the ports are set correctly for NAT,

    Now then, you need to keep in mind that there are two distinct sides to the path - the phones and the PBX. Each is likely protected by a router/firewall and each side expects to see data on certain ports -
    3CX on 5060 UDP, 5001TCP, 10000-10XXXUDP
    Where XXX is the RTP range.
    This covers the forwarding for SIP, Provisioning and RTP.

    The phone on 50XXUDP - SIP, 14XXXUDP - RTP for each phone behind the remote firewall where XX is the unique port number for each phone.

    On the PBX side, 3CX is expecting to receive messages on port 5060 UDP (default). So, in the phones the SIP Server (Host) is correct as shown with 5060. You do not need to forward 5066, 5067, etc. Just 5060.

    For manual provisioning -

    Now then, as you have more than one device behind the remote router, there is the possibility that it is not robust enough to manage the NAT/PAT tables correctly. Some are better than others, but in order to help the router manage things correctly, you need to:
    1 Disable SIP ALG in the remote router.
    2. Inside of each phone' web interface, you will find a setting called LOCAL SIP PORT. This is what needs to be set to the port as defined by 3CX in the provisioning file when using STUN or direct SIP. This will be the port that 3CX uses to communicate back to each phone and because it is unique, will avoid any issues that might arise should the router not be able to manage the multiple NAT.
    3. In the phones, there will also be the RTP ports and these too need to be set to the ports as defined by the 3CX provisioning page for each phone.
    4. In the phone web interface, there is a setting along the lines of NAT IP. This is where you would enter the PUBLIC IP of the remote router. When the phone sends a message to 3CX, this entry tells 3cx where to respond (the remote router PUBLIC IP).
    4. Depending upon your desire, you can-
    A) forward the unique ports SIP and RTP within the remote router to each phone behind its firewall.
    B) If not using A, then in the phone web interface, enable Keep-Alives and use a time interval of not more than 30 seconds. This will cause the phone to send a packet at the defined interval which will create a pin-hole in the router so that responses can be passed.
    5. Within the phone interface there are settings that tell the phone to only accept messages from 3CX. I do not recall what these are 0ff-hand, but they are designed to prevent calls from coming in from other than 3CX so as to prevent "ghost" calls.

    If ports are not seen at 3CX using the ports that have been set-up, then one of the routers is translating same. If however, the SIP trunk questions as asked originally are all YES, then it may be that the remote router is at issue.

    Auto -

    Simply copy the link and paste into the appropriate provisioning setting within the phone and reboot (assumes provision on reboot is enabled). You may still need to use the NAT IP, Keep-alives and/or remote router forwarding and security settings in the phone for ghost call prevention as these settings are not normally set in the default provisioning template.
     
  14. sip.bg

    sip.bg Active Member

    Joined:
    Nov 7, 2016
    Messages:
    704
    Likes Received:
    219
    You better disable SIP ALG on your remote router as well, as for the router in front of the PBX.
    Remote phones may have 5060 as local SIP port, still multiple accounts may use ports 5070, 5080 and so on. During NAT at remote site local SIP port, e.g. 5060 is replaced with random one, which is normal and this is the way it should be. If during NAT port is not NATted, i.e. it remains 5060, you will run in trouble having several phones behind that router. Some routers, especially home/small office models unfortunately don't behave properly, so you may need to adjust manually local SIP ports of the phones to 5062, 5064, 5066, ... and so on.

    The PBX should be kept at port 5060 and external port should be also 5060.

    Under normal circumstances you should not perform any special configuration on remote routers.
    If still running into problems with NAT or routing / firewall as a whole, I would recommend using MikroTik routers, they behave properly and have excellent price/performance ratio.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  15. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    Hi @lneblett thank you kindly for your thorough explanation.

    • We are using pfSense on both ends and pfSense does not have ALG.
    • Yes we are using a SIP trunk and calls do seem to function correctly in and out, the client has not complained of any call related issues. That's not to say that there aren't any, at this stage I think our biggest issues is keeping all phones online.
    • Yes I have run the firewall test many times since this PBX has been set up, and it has never failed since it was set up correctly in the beginning.
    • Yes, on the PBX side we have opened 5060 for SIP, 5001 (for the web interface of 3CX, right?) and 9000 - 9500 for RTP (but not 10000 - 10XXX as suggested, is this okay? I believe we got these port numbers from the initial 3CX setup guide)
    • What port do the phones use for provisioning?
    • Can you please elaborate on this one a little more, please?
    • On the phone side, with the router/firewall, does each phone need to have a unique SIP port and RTP port range? And therefore matching in the PBX? IE - a unique port for SIP and unique port range for RTP is specified for each extension in 3CX and these ports must be opened and port forwarded/NATed on the phone side router/firewall. Is this correct? I think this is where a lot of confusion is happening for me as I have received so many different theories from various people.
    • If each extension needs a unique SIP port and RTP port range, with an installation of say 100 phones, surely this would take an eternity to program, and then imagine the troubleshooting! Is this the normal way that everyone is doing it?
    • Why does each phone have just 1 SIP port but multiple RTP ports? (a port range).
    • I have received several different suggestions on the size of the RTP port range - should it be 5, 10, 12 or 20?
    • I have also read that each phone needs a unique SIP port range of 2, so the first phone should be 5060 - 5061, then the next phone should be 5062 - 5063, is this correct?
    • Are the SIP ports and RTP ports always going to be UDP?

    • Even though all our phones are auto-provisioned, how can I check to see what the current keep-alive interval is?
    • What is the official name for these "Keep-Alive" messages you speak of?
    • When using pfSense and 3CX together (at the PBX side), according to this guide: https://www.3cx.com/docs/pfsense-firewall/#h.mk14nzhjw26j it's important to set up Port Preservation in pfSense, so that pfSense doesn't change the port before it reaches the PBX (from what I understand). Is this also required on the phone side?
    • We are using auto provisioning, but the phones don't seem to re-provision upon reboot. I know this because I have changed some settings in the phone and then rebooted the phone to see if it would erase the setting, but the setting was still in there after reboot, so I'm not sure what's going on here, any ideas?
    • What do you mean by "use the NATP IP"?
    • Can you please elaborate on this a little, especially using the Keep-alives and/or remote router forwarding and security settings in the phone?
    I think this post might be in the running for an award for biggest post ever. Apologies for those who want a quick read but this has been an ongoing issue for about 9 months, so I really appreciate the help of everyone here!

    PS - to anyone replying, please try to be very specific and don't assume I know anything at all! Please avoid the term "remote site" because to me, both sides are remote. I rarely visit a site after install, so the term "remote site" is confusing to me because the phone side and the pbx side are both remote. Perhaps I need my vocabulary / jargon definitions updated?
     
  16. sip.bg

    sip.bg Active Member

    Joined:
    Nov 7, 2016
    Messages:
    704
    Likes Received:
    219
    @peter: You may treat your PBX as "central" cite and sites with phone as "remote", as they are not local to the PBX. This is the point of view of the PBX, not yours, for you all sites may be remote.
    Normally you need to NAT / forward specific ports to your PBX only (5000,5001,5060,5061,5090 tcp; 5060,5090,9000-9500 udp) and nothing at sites with phones. Seems you are complicating unnecessary the setup. This article could be useful: https://www.3cx.com/docs/firewall-checker-client/
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #16 sip.bg, Dec 27, 2017
    Last edited: Dec 27, 2017
  17. YiannisH_3CX

    YiannisH_3CX Support Team
    Staff Member 3CX Support

    Joined:
    May 10, 2016
    Messages:
    4,349
    Likes Received:
    274
    Hello @Peter Richardson

    Let me try and explain the theory behind remote phone provisioning using STUN so you can better understand the way this works and you can better troubleshoot the issue.

    When you select to provision a phone using STUN in the management console you are given the option to select a SIP port and RTP ports. The SIP port should be different for each phone so must the RTP ports. For older Yealink phones 12 ports are required. For newer models you can use 10 that is why you are seeing different answers regarding this.

    After the configuration of the ports you should provision the phones. These ports are included in the provisioning file sent to the phone.

    Now we need an example to demonstrate how these ports are used.
    Lets assume that my phone is correctly configured and provisioned to use port 5065 and RTP ports 14000 - 14011.
    When the phone sends a registration message to the PBX it will contact the PBX on port 5060 which is the SIP port of the PBX but the phone requests a reply to its own SIP port which is 5065. The reply from the PBX will be sent to the phones public IP and specific port.
    In the screenshot below you can see the register message from the phone. Notice that the source port is 5065 and it requests for a reply in the contact to port 5065 and the public IP. (Public IPs and FQDN have been removed for safety reasons).

    2017-12-27_14h14_22.png


    The PBX replies to the requested port:

    2017-12-27_14h17_43.png


    If this port is correctly forwarded on the firewall on the phones side, then the message will reach the phone and the phone is correctly configured.
    If you see the messages arriving to the PBX from a different port than the one specified under the phones provisioning settings, then it can mean that the phones are not correctly provisioned or a device between the phones and the PBX is changing the ports and this is what you need to identify. Are the phones correctly provisioned? You mentioned that the phones are auto provisioned so i will assume that the phones are correctly provisioned. If there is any doubt regarding this then re-provision the phones to make sure. Then you need to identify the device that is changing the ports which is more difficult a different for each firewall.

    A way to start troubleshooting is using wireshark. You can start a capture on the phones web interface (Settings / Configuration / Pcap Feature) and wait a few minutes to capture registration messages. At the same time a capture should be running on the PBX. If you see that the phone is sending port 5065 and the PBX is receiving a random port then your firewall needs to be correctly configured. If the phone is sending a random port then the phone needs to be re-provisioned.

    If you have a site with multiple remote phones then you need a specific SIP port for each phone which needs to be port forwarded to the local IP address of the phone. The same applies for RTP ports. This is why an SBC can help you overcome these issues as all communication passes from the SBC port. Phones are connected to the SBC locally (SBC and phones are on the same LAN) and the SBC is connected to the PBX.

    Each phone uses one port for signalling (SIP port) but requires multiple RTP ports as each stream requires one port so if you are in a conference call or have accept multiple calls enabled you need multiple ports for audio to go through.

    You can find additional information in our academy section found in this link. All material there is very good but you will most interested in the presentation regarding STUN Configuration.
     
    apostolis_3CX likes this.
  18. lneblett

    lneblett Well-Known Member

    Joined:
    Sep 7, 2010
    Messages:
    2,061
    Likes Received:
    56
    Yes, the RTP insteadrange you mention is correct for 3CX. I also do other makes and quoted RTP ports typically used by Asterisk..apologies.
    Yes, you need to do the same for the router at the remote end as well.

    Ideally, at the remote site, each phone will have its own unique set of local SIP and RTP ports. These should be reflected in the 3CX provisioning tab for each extension and these need to match what the phone will have. You can reserve IP at the remote router for the phones and then forward the ports needed for each. If you forward, then keep-alives are not needed. Keep-alives are found in the account, advanced settings and these are used when forwarding is not in place. It effectively does the same thing as forwarding by creating pinholes in the router due to the sent packets.

    If there were a remote site with 100 Phones, it is more likely that a VPN would be in-place or an edge device (SBC) or another PBX.

    NATP IP should read NAT IP
    Go to the phone's Web interface, Network, NAT Manual, Active - set to enable
    In the IP address section, input the public IP of the remote site. This is the NAT IP and tells 3CX where its responses will need to be sent. If this is not possible due to a non-static public IP, then you could enable STUN instead which is in the same section. If you have auto-provisioned the phone, the STUN section may already be populated.

    As far as so called ghost calls, please refer to this -
    http://support.yealink.com/faq/faqInfo?id=559
     
  19. Peter Richardson

    Peter Richardson New Member

    Joined:
    Apr 6, 2017
    Messages:
    149
    Likes Received:
    5
    Thanks @sip.bg that makes much more sense now!

    Thanks @YiannisH_3CX for that detailed explanation.

    I have set 1 port for SIP and 12 ports for RTP for each phone.
    I have forwarded the ports in the router but I am now having major issues with this, all of a sudden. I don't know why but now I can't get anything through the firewall. I have reset the states, rebooted, deleted all the rules and reinstated them, deleted all the NAT rules and reinstated them, I'm about to give up and just tell the customer that we can't give them service anymore. I've downloaded a proper port checking utility and this is definitely the issue. The ports are being blocked and I can't get them to go again.

    Why are we using UDP9000-9500 on the PBX side and 14000-14037 on the phone side for RTP?

    Thanks @lneblett we were using an SBC but it was causing too many problems when the power went off, so we had to get rid of it because nobody could find a way to make it work after the power came back on (reliably). We are using auto-provision so the STUN section was already populated. Should I still use NAT Manual?

    EDIT: OMFG after staring at this screen for the last 8 hours on this problem and feeling rather suicidal, I just checked the activity log and I can see that the phones are now communicating on their correct SIP ports. I don't know how because the port checker is failing everything. How can I check to make sure that the phones are using the correct RTP ports?
     
    #19 Peter Richardson, Dec 28, 2017
    Last edited: Dec 28, 2017
  20. sip.bg

    sip.bg Active Member

    Joined:
    Nov 7, 2016
    Messages:
    704
    Likes Received:
    219
    @peter: Can't you rethink your networking model and use VPNs instead?
    I don't have experience with pfSense, but I'm widely using MikroTik routers to build Layer 2 tunnels (EoIP), equivalent to local LAN or VLAN or Layer 3 tunnels (GRE) to route traffic between sites without NAT. All tunnels use IPsec encryption. I don't have any issues. In some cases I'm using STUN for single or few remote phones per site, but never had to make any special setup of routers (i.e. port forwarding, still mostly MikroTiks) or provisioning the phones. I'm not using SBCs at all.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #20 sip.bg, Dec 28, 2017
    Last edited: Dec 28, 2017
Thread Status:
Not open for further replies.