No outbound/inbound calls

[Matthew Tanis]

A company I do work for started experiencing a problem today, they cannot make outbound or receive inbound calls. When calling out the call terminates with "Seize Failed" message. When calling in the call never connects and you are told "The number you have dialed is not in service." the system has been working for a few years now without issue and the problem just started today with no changes.

I have already contacted the VoIP line provider and they stated they can see the call come through but it terminates due to a second connection attempt from the PBX.

I pulled these logs from when I attempted to call out to my cell.

21-Jul-2017 15:40:55.058 Leg L:6.1[Extn:1122] is terminated: Cause: BYE from PBX
21-Jul-2017 15:40:55.008 Leg L:6.2[Line:10006>>[MYCELL]] is terminated: Cause: 403 Forbidden/INVITE from 172.18.255.1:56772
21-Jul-2017 15:40:55.008 [CM503020]: Call(C:6): Normal call termination. Call originator: Extn:1122. Reason: Forbidden
L:6.1[Extn:1122] failed to reach Line:10006>>[MYCELL], reason Forbidden

Event ID in server event log is, Warning 12294

I'll admit my knowledge is limited, any help is appreciated.

 

[leejor]

[Line:10006>>[MYCELL]] is terminated: Cause: 403 Forbidden/INVITE from 172.18.255.1:56772

I'm going to assume that the IP in the post is that of your provider? Could this be a blacklisted IP? When you see the word forbidden, either you have specified that the call cannot be made, the number format is incorrect, or , for some reason, 3CX has blocked the IP. Did you try restarting the server?

 

[Matthew Tanis]

Correct. See that's what I was thinking, I checked my firewall and there isn't anything blocking it, the 3CX server isn't blocking it, I opened my firewall wide open to the server briefly just to rule that out and it had no effect. I did already try restarting the server, all hardware out actually just to be safe.

The ISP said something about they can see the call connecting to their lines but received a second connection request which was causing it to drop, don't really know what that means or where to start looking, any advice?

 

[leejor]

Problem this severe rarely start unless something has changed, somewhere. Have any changes been made, anywhere, at the time the issue began (it might help if you can locate, in the logs exactly when this started). Does anyone at the company have access to the system, someone that may have done something but doesn't realize that they caused this? Any hardware changes, software updates? Your provider didn't make any changes? Do you have a second provider to test calls over? There are some that will give you free, or trial service, it may help to determine if the issue is limited to your current provider only.

Setting the log level to show more detail might help, especially that second connection your provider mentioned. You may have to employ a tool, such as Wireshark to get to the bottom of what is happening.

It's a bit drastic, depending on how many changes are made to the system on a daily basis, but if all else fails, and you have determined when the problems began, you could try a restore of an older backup.Make a current backup first, and don't do this until you have exhausted all other possibilities, it is a last resort. This also assumes that auto backups have been ongoing.

 

[Matthew Tanis]

Only 3 people have access, and according to the logs no changes have been made on this server in weeks. No updates, nothing.

I do have another phone vendor I work with occasionally that I contacted, and they gave me a sip trunk for testing, I connected it up and when I called I connected no problem. I let my current provider know this and we've been doing troubleshooting all day, but doesn't look like there is any real change to the situation, they don't see any connection from my side, but I keep getting rejected.

What I'm thinking of now is, the phone server is on it's own VM, which we do backups of daily, I'm thinking of restoring it to a time last week before the issues just to see what happens.

 

[leejor]

The fact that another provider works, passing through the same hardware and any firewalls suggests the original provider has a issue.. Did you ever check the IP Blacklist to see if your providers is in there? Something at their end may have changed, but they may not want to admit it, or, may be unaware of it, at least the person you have been dealing with. If you go ahead with the restore of an older backup, do some screen captures of the trunk settings pages to compare with the older backup, if it does begin working again. It would be very interesting to track down exactly what changed.