ID: 12294 Call or Registration has failed 403 forbidden

[BW~Merlin]

So tonight our staff tried to do parent teacher interviews via phone as part of the whole social distancing thing going on with Corona/COVID-19 and we had some issues with staff getting an error message about calls being forbidden (they reported it happening with mobile numbers but that maybe only because most people have mobiles these days rather than a land line). Here is a sample from the event log.


SIP Server/Call Manager ID: 12294

Call or Registration to xxxxxxxxxx@(Ln.10000@AlloyVoice) has failed. 103.214.206.41 replied: 403 Forbidden; from IP:103.214.206.41:5060


I have run the firewall checker (after the event was over) and everything passed. We are running Professional Annual 16.0.4.504 and our SIP provider is AlloyVoice and is listed as a supported provider. We have 16 SIP channels and 24 SIM calls. During the interview process I estimated we had 12 calls max (counting in the active calls section) but can confirm 7 (as viewed from the dashboard).

I had one user report that they could make a call to a mobile and hung up to let me know the call worked only to call back a few seconds later to say that they were getting a forbidden message. During this time I can confirm that users were able to make calls to mobile as viewed from the active calls and we have not prevented users from being able to call mobiles, local or national (international is restricted). Until tonight no-one else has reported this issue so I believe it maybe trigged by the load (even though we were well undercap). The users all have a caller ID set (the same caller ID) that is different from our main trunk ID but is part of our DID range. Most users were using the Yealink T19P E2 phone thought I did get a report that the user with a Yealink T48S also had the issue at one point (firmware is up to date on both models).

I did some searching and found https://www.3cx.com/blog/docs/list-of-events-generated-by-3cx-phone-system/ which suggests that our trunk provider (AlloyVoice) is to blame but after spending and hour and a half on the phone with them they best they could come up with was that because the caller ID was different from the trunk ID that maybe causing the issue.

Any help tracking down what maybe causing this would be great.

 

[eddv123]

4xx is a client side errror - so it is likely to be your configuration on the PBX/trunk.

Firstly confirm the settings from the guide: http://www.alloyvoice.com.au/Files/AlloyVoice-3CX-Configuration-Guide-v1.pdf these guys look to be a supported provider with 3CX but I cannot find an official guide from 3CX https://www.3cx.com/partners/sip-trunks/australia/

Alloy voice are registration based so check your username, password and registrar settings they should of provided, and failing that run a PCAP from 3CX and "refresh registration" on the trunk - the PCAP should provider a little more insight: https://www.3cx.com/docs/capture-network-traffic/

 

[leejor]

It may be the mobile provider (might be just one provider) that is deciding to reject the caller ID being sent as a part of spam call blocking (not working as intended). Your provider should be able to look at one of the calls and tell you whether is is them, or the destination that is rejecting the call.

What you might try, as a test, is to remove the outgoing DID number from one (or more) extensions. This may/should force the use of the "main" number as caller ID. See if that/those extensions continue to have issues.

 

[BW~Merlin]

Alloy voice are registration based so check your username, password and registrar settings they should of provided,

I would assume that if this wasn't setup correctly then ALL calls would be failing not just some calls while the system is under load.

run a PCAP from 3CX and "refresh registration" on the trunk - the PCAP should provider a little more insight:

I can a PCAP, what should I be looking for in it?

It may be the mobile provider (might be just one provider) that is deciding to reject the caller ID being sent as a part of spam call blocking (not working as intended). Your provider should be able to look at one of the calls and tell you whether is is them, or the destination that is rejecting the call.

We will ask the question. I tried again this morning (rounded up a bunch of staff and had them all call their mobiles all at the same time) and only one person got the message about forbidden (it says forbidden on the phone display I found out).

What you might try, as a test, is to remove the outgoing DID number from one (or more) extensions. This may/should force the use of the "main" number as caller ID. See if that/those extensions continue to have issues.

We will try this.

 

[BW~Merlin]

I have made some obersvations and so far with my testing it is looking like I have found what triggers this issue but I don't know the underlying mechanism. From what I have observed if more than five outgoing calls come from the same caller ID any additional calls will fail if they come from that same caller ID.

I have changed the caller ID on some rooms back to the original caller ID (which is different from the main trunk caller ID) and left the other rooms with the main trunk ID (we put all the rooms on the main trunk ID [deleted the caller ID we had set] and noticed the same issue with forbidden) and so far things are working.

Is there somewhere in 3CX that has this setting about outbound limits on the caller ID or is this something we need to take up with AlloyVoice?

 

[eddv123]

I would assume that if this wasn't setup correctly then ALL calls would be failing not just some calls while the system is under load.

Correct, if you are using a registration based provider and the trunk shows red ALL calls will fail (unless you have a redundant link/trunk setup).

I can a PCAP, what should I be looking for in it?

Run the PCAP and try and re-register. Then in the trace you are looking firstly for the REGISTER request from 3CX and then the response that comes back from the public IP of the provider.

 

[leejor]

Obviously best (and easiest), to talk to your provider first, and give them the results of your testing. However, I'm pretty sure that your provider does not have trunks direct to mobile providers, which means the calls pass through one or two "links", (one probably being a PSTN provider) before being passed on to a mobile company. It would be a matter of tracing a failed call (something your provider would have to do), to determine who is rejecting the call, and why.

My suspicion is an incorrect implementation of spam call blocking. As this is relatively new, many companies may still be "tweaking" whatever process they are using.

 

[BW~Merlin]

My suspicion is an incorrect implementation of spam call blocking. As this is relatively new, many companies may still be "tweaking" whatever process they are using.

It seems you are right, after AlloyVoice did a lot of setting changes to our install (I am unsure what has changed so no doubt I will find out the hard way) they finally looked into their up stream provider and indeed came back to us that we had a limit of five calls from the same DID (they did say the main trunk DID didn't have this limit but I have tested it as also being limited to five calls) so we have asked that all DID's have their limit removed.

 

[leejor]

Providers are still working out the "bugs" with the recent requirement to block SPAM calls. Not all have implemented it well.