Twilio - Starting to block outbound unverified callerID when call forwarding?

[Csut12]

Unless I have a misconfigured system, it appears Twilio may have started to block calls outbound using the callerID of the person who called. Started ~2024-04-12 on a long term/established Twilio paid account. The specific error is Twilio's 32204 - CallerID Unverified/Not Owned.

Prior to 2024-04-12 the calls could successfully be forwarded by 3CX to an external phone number [using Twilio trunk] and with the callerID of the incoming call (rather than our owned phone numbers). I believe this is referred to as 'CLIP No Screening' capability. Other threads in the past (~2022) suggested this would only work because Twilio paid account, and other thread suggested this was by default blocked but could be enabled for specific Twilio account on case-by-case basis. In this case it is a paid account and don't recall if the behavior was enabled for our account (but it did work). Checking with Twilio support but don't think was a change got notification on.

The one possible system related change that comes to mind (from a few years ago) is had adjusted SIP trunk - > Outbound Parameters: Remote Party ID - Calling Party : User Part from OriginatorCalledID to OutBoundCallerID. This seemed to allow transferring a call by a receptionist in office to a cellphone that would show the initial inbound callerID to the receiving cellphone (as opposed to office).

Workaround that seems to work for now: Since used this more to forward to a specific number, created an outbound rule that hardcoded outbound callerID to one belonging within Twilio account.

Perhaps this helps someone out there or perhaps someone else is seeing similar. Apologies if this is some bad configuration.

 

[Csut12]

For Twilio trunks, what are the correct/default CallerID values? On my existing instance the current Twilio trunk and a "new" Twilio trunk, it defaults to these CallerID settings:

• From: Custom Field (the extra box appears blank)
• Remote Party ID - Calling Party : Display Name: Default
• P-Asserted Identity : Display Name: Default

I feel like the `From` field might be incorrect, but thought would ask here. V20 Build 1620. Other trunks seem to use the more logical "OutboundCallerID", but I might be misunderstanding.

EDIT: The provider template in settings suggest this is correct, well at least as part of this Build 1620 (V20 Update 0):
<field name="ParameterOut" custom="" parameter="RequestLineURIUser">$CalledNum</field> <field name="ParameterOut" custom="" parameter="RequestLineURIHost">$GWHostPort</field> <field name="ParameterOut" custom="" parameter="ContactUser">$AuthID</field> <field name="ParameterOut" custom="" parameter="ContactHost">$ContactURI</field> <field name="ParameterOut" custom="" parameter="ToUserPart">$CalledNum</field> <field name="ParameterOut" custom="" parameter="ToHostPart">$GWHostPort</field> <field name="ParameterOut" custom="" parameter="FromDisplayName">$CustomField</field> <field name="ParameterOut" custom="" parameter="FromUserPart">$OutboundCallerId</field> <field name="ParameterOut" custom="" parameter="FromHostPart">$GWHostPort</field> <field name="ParameterOut" custom="" parameter="RemotePartyIDCallingPartyUserPart">$OutboundCallerId</field> <field name="ParameterOut" custom="" parameter="RemotePartyIDCallingPartyHostPart">$GWHostPort</field> <field name="ParameterOut" custom="" parameter="P-AssertedIdentityUserPart">$OutboundCallerId</field> <field name="ParameterOut" custom="" parameter="P-AssertedIdentityHostPart">$GWHostPort</field>

Twilio support said the failed/blocked outgoing SIP Invite packet had the 3rd Party phone # in both the `From` and `P-Asserted-Identity` field. They seem to be saying the `From` field needs to be a phone number within our Twilio account and the `P-Asserted-Identity` field can presumably be a non-owned phone number (or vice versa -- not sure exactly which yet).

Prior to V20 I believe there was a place to edit these parameters as needed for a particular trunk. Is that moved/available elsewhere in V20+?

I still don't know how any of this related to the specific April 12th date yet.

 

[Csut12]

Per Twilio Engineering:

"On April 12, 2024 Twilio implemented a functional change intended to prevent a fraud in the form of Caller ID Spoofing. Specifically, Twilio began verifying the P-Asserted-Identity header when present in a SIP INVITE sent to Twilio to initiate a Trunking Termination call.

As the calls in question contained a non-Twilio and non-Validated phone number in the P-Asserted-Identity header, the calls were rejected as having an unverified Caller Identity, as specified in the error returned.

it is recommended to look into implementing a more standard SIP call forwarding solution using the Diversion header to pass the Originating Caller ID as outlined in RFC 5806."

No notification that I saw, but at least sounds like confirmation it is functional change.

Does 3CX support setting/using Diversion header for call forwarding? From this post it sounds needs:

...instead of sending 302 back to provider, 3cx should send another invite that has the following format:
From : original_number ( this can be changed in outbound param)
To : forwarded_number
Diversion: pbx_number - they say this must be present otherwise it will not work.

It seems without it, any attempt to forward a call to external number using Twilio trunk will incorrectly attempt to use No Clip screening type method rather than Diversion header (assuming its not my trunk configuration at fault).

 

[YiannisH_3CX]

Hi @Csut12

The default settings for Twilio will not forward the originator caller ID back to Twilio since the trunk is set up to send the outbound caller ID of the forwarded. Of course what is actually sent depends on how the forwarding is made. If the call comes in through a different trunk for example and you forward it out through Twilio directly then the original trunks caller ID will be used.

Also Diversion header is present in forward calls (only) in V20.

 

[Csut12]

Our scenario is pretty simple: A Call Queue (with a direct DID) has the field `Destination if no answer` = "Forward to Outside Number". Basically if no one picks up forwards to external number.

Would the above scenario add the Diversion header in the forward (by default or needs change)? System is on 20.0 Update 0 (Build 1620 Release). Calls come in and attempt forward out of same, single, Twilio trunk.

 

[YiannisH_3CX]

Would the above scenario add the Diversion header in the forward (by default or needs change)? System is on 20.0 Update 0 (Build 1620 Release). Calls come in and attempt forward out of same, single, Twilio trunk.

Your scenario should work without any changes and the diversion header must be added when the call is forwarded.

 

[Csut12]

Back in ~V16/V18 we did the small edit in the specific Twilio trunk: SIP trunk - > Outbound Parameters: Remote Party ID - Calling Party : User Part from OriginatorCalledID to OutBoundCallerID. Since them we have upgraded to V20 and V20 no longer presents/lists all of the trunk options as before.

I suspect its likely that change is out of alignment with the current 3CX default Twilio template and would like to confirm. Otherwise, the above described scenario would presumably be working in V20 and Twilio.

Is there a way to view an existing created Trunk configuration to see if the existing Trunk parameters match the production template?

 

[Charles_3CX]

You can export the template from the provider settings on the right under the Voice & Chat section. Press the three dots, then export and save. The parameters that have been set will appear in the provider.xml file.

 

[Csut12]

Thanks @Charles_3CX -- I was able download the config template and do a comparison against V20 3CX template.

It appears two values are out of alignment in my existing config:

• ParameterOut: FromUserPart = $OriginatorCallerId (template calls for $OutboundCallerId)
• ParameterOut: P-AssertedIdentityUserPart = $OriginatorCallerId (template calls for $OutboundCallerId)
  

Is there a way to edit this existing trunk values in post-V20 to bring them inline (without deleting/recreating the trunk)? Prior to V20 I could flip these values to expected template values. In this case I can ssh in and edit (please point which file), but if there is webGUI option that is nice too.

As an aside, might make sense for WebUI addition: When puling up an existing trunk, WebUI checks if existing config is out of alignment with latest provider template from 3cx and throws a toast if it is. I suspect template drift will happen as go from V20 to V23,V24 etc etc.

 

[Charles_3CX]

You can make changes to the provider XML, delete the current trunk to re-import, and enter the user data for the trunk, but it would be much easier just to re-create the trunk from the new one with the default template. This way, any updates will be done moving forward, as the template will be the default one.

 

[Csut12]

I've created a new trunk and switched all rules over (tips and steps and feature request suggestions in spoiler tag below). I have kept existing trunk for side by side compare. Also updated to latest 20.0.1.731.

Result of simple call queue forwarding scenario using default Twilio template:
- Diversion header is present
- Diversion header is using the callerID of our owned #. Expected: CallerID of the incoming caller.
- This call was successfully forwarded, albeit with callerID of us rather than outside person calling in.

To test, I activate custom rule to forward a call using old trunk (with its previously mentioned settings). The SIP packet on that call shows following:
- P-Asserted-Identity: The outside person's callerID
- Remote-Party-ID: One of our owned Twilio #s.
- Diversion: One of our owned Twilio #s (different from Remote-Party-ID used, likely from the call queue callerID value).

Using the defaults: it seems the default Twilio template is using our callerID within diversion header on a forwarded queue from a call queue. Is it possible the default template needs to be updated to use different parameter? Is there something else should check? Did I misunderstand how Diversion header value was going to be used?


--==--

Spoiler: Details and Suggestions for recreating Trunk:

Was hoping to avoid this -- manually recreating a trunk means reestablishing all pairings and generates new ID(s) etc.

Some notes going through this:
- Spun a new trunk, leaving existing in place
- Added the DIDs into the new trunk (note: There is import option but not export option on DID page, would be nice to have).
- Set each setting on new trunk, matching what existing trunk was showing.
- Noted the SMS webhook has changed. Has to be changed where we use it (for incoming 3CX SMS messages).
- One-by-one went through each user, department/call queue and add the "new" DID from Trunk2 to each extension. To validate pulled up existing trunk DID page and new trunk DID page and compared the DID assigments as all filled out.
- Went through where we use 3CX SMS webhook and updated to new one.
- Outbound rules: Stepped through each rule and updated it use new trunk
- System-Emergency Numbers: Stepped through each rule and updated it to use new trunk
- System+Advanced: Stepped through each setting pane looking for any additional references to specific trunk.

- Test outbound call
- Monitor/setup up alerts on Twilio for any errors and monitor for outbound issues.
- Test/monitor for call forwarding scenario.

- Its maintenance window so maybe update to latest version of 3CX while at it. Not ideal for some reasons, but staying up to date has its own upsides. From 20.0.0.1620 to 20.0.1.731

In short, it was a mini firedrill to flip to a "new"/old trunk. Not ideal, but keeps those recovery skills up to date.

Niceties/feature requests to add for 3cx around this:
- Trunk page shows warning notification if current template is out of alignment with 3cx shipped template for provider (manually spitting out which fields are out of alignment).
- Trunk DID export option (only import currently exists)
- A new dropdown selector type within extension/call queue: The drop down should show which Trunk a number is associated with, currently can have duplicate DIDs in the list and really have to guess/hope the lower ones are related to new Trunk.
- A non-destructive/recreate method of fixing trunk xml being out of alignment (repairing is nerve racking missed some one-off pairing or ID change).

 

[luke87]

@Csut12 - Did you ever get this to work? I'm having the very same issue!

 

[Csut12]

@Csut12 - Did you ever get this to work? I'm having the very same issue!

Partially - the calls are forwarded successfully but use our owned CallerID rather than the incoming call's CallerID on the forwarded call. I had to recreate our existing Twilio trunk to use the default Twilio template in V20 -- this presumably means will stay more up to date as 3cx updates the template as needed.

My guess to get it fully working is the diversion header potentially needs a slightly different value - my guess change from OutBoundCallerID (default) to OriginatorCalledID. I am not sure if this will be a field that will be editable in the 3CX UI but that presumably will need some dev time to add.

 

[luke87]

@Csut12 - Thank you for the reply! I was able to work with Twilio and had to get approved to allow to send the original caller ID and now working perfectly after I got approved and then I changed "Remote Party ID - Calling Party: User Part" = "OriginatorCallerID"