PSTN Conundrum

[joebocop]

Hi again,

We recently replaced a 1980s-era analog Nortel system with a 3CX PBX. The company needed to retain its 16-line analog MLH, so we opted for a Patton SN4316 device to fill that need.

We have been baffled by an issue since the initial deployment that we are now realizing may not be solvable.

Callers calling into the company over the PSTN (thus hitting the SN4316) sometimes have their DTMF tones incorrectly detected. This occurs about 25% of the time:

• - caller hits the PBX and gets a digital receptionist
  - caller dials their party's 3-digit extension
  - caller is incorrectly routed to a different extension.

The "call-control debug" (obtained by enabling debug call-control detail 5 on the SN4316) reveals the following, for a caller attempting to reach extension '501'

16:55:49 CC > [EP IF_FXO_0/active] Invoke service: User Input: 5 (Duration: 2000)
16:55:49 CC > [EP IF_FXO_0/active] Invoke service: User Input Update (Duration: 70)
16:55:49 CC > [EP IF_FXO_0/active] Invoke service: User Input: 5 (Duration: 2000)
16:55:49 CC > [EP IF_FXO_0/active] Invoke service: User Input Update (Duration: 100)
16:55:51 CC > [EP IF_FXO_0/active] Invoke service: User Input: 0 (Duration: 2000)
16:55:51 CC > [EP IF_FXO_0/active] Invoke service: User Input Update (Duration: 190)
16:55:51 CC > [EP IF_FXO_0/active] Invoke service: User Input: 1 (Duration: 2000)
16:55:51 CC > [EP IF_FXO_0/active] Invoke service: User Input Update (Duration: 190)

In that case, of course, 3CX is given all 4 digits and accepts only the first 3, routing the call to extension '550' rather than '501'.

Just why the SN4316 detects the first DTMF tone as a 70ms instance followed by a 100ms instance is the great mystery. Patton technical support has suggested we

• - try a different SmartNode device (we have; same result)
  - try a different physical PSTN line (we have; same result)
  - try a different PSTN provider (not an option at this location).

Patton technical support has let us know that there is no way of configuring the SN4316 to listen only for tones lasting longer than 80ms, or to treat tones interspaced by less than 50ms as a single tone. I am guessing we are out of luck with Patton hardware with this particular PSTN provider.

So, the questions are:

• - does anyone know of a means of forcing the SN4316 to ignore extremely short DTMF tones?
  - if Patton devices are not an option, is there a reliable alternative?
  - has anyone seen this behaviour? The old Nortel gear did not experience these issues. A neighbouring company uses an Asterisk system with digium PSTN hardware and they have no similar issues with DTMF detection

 

[leejor]

Are these PSTN lines coming from your providers central office over copper, or an ATA box on premise?

The question really is...does the Patton have a problem, and is splitting the first DTMF tone into two , or is your provider actually sending the first tone as two "parts", with some sort of drop-out mid-way through the first tone. Since you have already swapped out the Patton (unless it is a config issue), it points to the PSTN line. However, unless you can '"scope" it and confirm that the tone actually has a drop-out, it's very hard to accuse your provider.

Have you done any testing with different types of sets calling in...say on older analogue set, where DTMF will continue to be sent as long as a button is held down, and a device such as a cordless, or mobile handset, where the tone duration is usually determined by the manufacturer.

 

[joebocop]

Thank you again for weighing-in leejor.

Are these PSTN lines coming from your providers central office over copper, or an ATA box on premise?

Lines are coming from provider CO over copper.

We have 18 such lines coming into the building, and I have tried multiple Patton devices on 4 of those lines, all with the same issue.

The problem occurs on about 20% of calls from land lines, and on about 50% of calls from cell phones. It's awful, I'm getting a ton of heat for it.

Problem did not exist with the old Nortel equipment, on these same lines.

I have altered the PSTN profile on a test Patton device, as shown in the attachment. With these settings, the behaviour is slightly diminished, but callees hear their voice reproduced to them in their own earpiece; not acceptable either.

Am I going to have any better luck with BeroNet device? I believe I am going to lose this contract now anyway, but would like to at least leave them with a functioning system.

 

[leejor]

Am I going to have any better luck with BeroNet device?

The short answer is...maybe. it all depends on what is actually happening. if it were a simple matter of substituting one make of gateway for another, then that is the way to go, even if you have to borrow one to do some testing. It may come down to the combination of central office line equipment (type of switch), and the Patton.
As you've discovered, "playing " with the levels is not always advisable. Echo suppressors can have a hard time overcoming an increase. The fact that all tones were being detected suggests that it is not a "low level" issue.

The fact that this is not happening every time, make it more of a mystery. If it were possible to do a trace on all failing calls, right through the central office switch, it could turn out to be a T1/T3 issue from one particular source/inter-connector/interoffice trunk.

 

[joebocop]

Thanks again for the response!

Unfortunately, the PSTN provider was also a (losing) bidder on the phone system replacement contract, so they are quite content to provide zero assistance in terms of tracing calls, troubleshooting, etc ("our commitment ends at the demarc" is their now famous quote). We are on our own here.

 

[leejor]

I would see if you can borrow (for testing purposes), another gateway. If it works, then consider buying it and selling the Patton if you can't find another solution.

 

[joebocop]

Thanks again, leejor. I think we will get our hands on an alternative gateway and see what effect that has.

Last night I took all the spare Patton gear we've got (an SN4112 FXO and another SN4316 FXO) offsite, and re-created the PBX configuration at another building location with the same PSTN services from the same provider.

My team and I called into the system over 100 times from different sources, and not once could we reproduce the duplicate tone detection issue we are seeing in the main office building.

We are going to troubleshoot those physical lines in the building today. Fingers crossed.

Thanks again.

 

[leejor]

There is the possibility that dialtone, even from the same provider originates from two different types of line equipment. One site may be hosted from a remote such as the Anymedia FAST, Nortel Access Node, Tellabs UMC1000, etc, while the other location is from a Base unit switch such as a GTD-5, DMS-100, or 5ESS. That may be making the difference.

 

[joebocop]

The issue is now resolved.

After moving the entire PSTN service to a different building (!), the problem remained. Thereafter, exasperated, we re-recorded the sound files used on the 3CX digital receptionists, and since THAT time, can no longer reproduce the duplicate DTMF tone issue.

Can anyone account for ANY technical reason that the sound files would cause this behaviour? Patton themselves cannot. We also verified that the old/new sound files are identical in bit rate, frequency, etc.

I am inclined to believe that the problem is not solved given the lack of technical evidence that would account for the symptoms disappearing. Even so, try as we might, we can not produce the issue at this time, and the only change made was the swapping of sound files.

Thanks.

 

[sky_brad]

HELP!

I am presently in the exact same conundrum and am also receiving a lot of heat from the client.

I have sent numerous logs to Patton and 3CX and no one has been able to provide an answer. My set up is precisely the same; Analog copper lines coming in, Patton boxes translating to digital and forwarding to the PBX.

I see the exact same splitting at 70MS. We have debugged the Patton boxes and found that this is happening at the Patton box level, before it even hits the PBX.

Were you able to gather any more information about the cause? Since it is the only resolution I can find, I will definitely have the IVR re-recorded but I would like to know if you have any further information.

Thank you,

Brad

 

[joebocop]

We found that the "new" IVR recording trained this in most cases, but we have not resolved the issue 100%. Still about 2-3% of incoming calls experience this issue. Patton has suggested that it's a problem with our PSTN provider.

We are currently testing a Grandstream FXO device to see if it experiences the same issues.