extension stuck outbound call

[ZenMasta]

A call came in a few minutes ago and for some reason my phone didn't ring. I heard another phone ringing so I had to walk over to pick it up. When I completed the call I wondered why my phone didn't ring as it is in the same queue as the other.

When I checked 3cx console it is showing the my extension is on an outbound call to a number I had dialed earlier...

I can dial out from my phone and you can see in extension status that my line will appear a 2nd time for the duration of this call but as soon as I hang up, it shows only the 1 that is stuck on that outbound call.

I don't have any other devices registering on this same extension so I tried rebooting my phone and that didn't work.
I had at first tried restarting only sip/tunneling but that didn't work.

I didn't know how else to solve this except to restart all services.

 

[leejor]

What type (make/model) is the set? Do you use these elsewhere on the PBX, or is this a "one of"? Any firmware updates available? Have you tried clearing (factory reset?) all settings on the set and re-provisioning?

 

[ZenMasta]

It's a cisco spa504g and I have 3 others on the same lan and 1 remote. We are a small office so I'm usually always first to answer out of our ring group anyway. But people are quick to tell me if something isn't working usually

The firmware on all sets is is 7.5.1a I do believe that is the most current.

I had a problem today where I went to answer a call and the screen just got stuck saying "Answering" but I was never actually connected to the customer Some seconds later... say 30 or so another phone next to me began ringing, and when I asked my colleague about the caller id, it was the same caller. While it was happening, I wrote down the phone number and tried to press the end call button but that didn't work either. When the other station answered the call my phone was still displaying that message so I unplugged the power.
I did check 3cx management console and my extension didn't appear active

Could it be related?

Neither problem has happened more than once (to my knowledge). we're a low call volume office so it doesn't present any major problem but still helpful to know why stuff doesn't work

 

[leejor]

If it is just one phone experiencing the problems, then, it may just be something wrong (hardware/firmware/programming) with the one set.

You could try doing a factory reset, then re-provisioning it. Or...swapping it with one of the other sets, move the programming from one to another to see if the issue moves with the set, or stays with the line.

There is a possibility that the port on a switch that this set is plugged into is flaky (or intermittent/faulty LAN wiring), you could try plugging it into a different port (swapping with another set).

 

[ZenMasta]

I could swap out a phone. It would be hard to say off hand if that's the problem or not since I usually always answer calls first. I would tend to agree more it is the phones problem if after disconnecting the phone my extension still wasn't connected.

It happened again just now. When it said answering none of the buttons on my phone worked so I had to plug the power. But extension status is still showing my extension connected. So I had to disconnect it using the console, just realized I could do that so I didn't have to restart services.

Sucks cause the customer ended up going on hold for a while. But I was able to call them back later to make the sale.

I will swap out a phone and see if this happens again.

 

[ZenMasta]

So I did swap phones (same model, just different unit) It happened this morning on an inbound call.

I attempted to answer a call my phone got stuck so I had to pull the plug. My extension didn't get stuck this time though which is good I guess.

I checked the logs and the server log indicated Lost Call in Queue Sales. I checked the activity log but it had already started to get truncated so I wasn't able to see what was going on.

The customer called back almost immediately while my phone was stuck and another extension picked up the call fine. However the customer got disconnected during the call too but when I checked the activity log there was no mention so maybe the customer accidentally hung up that time.

I posted a log

 

[ZenMasta]

P.S. I swapped the phone back on 7/9 and I believe this is the first time I have encountered the issue since then.

 

[ZenMasta]

Not trying to spam bump, but there is no edit button.
It just happened again, this time there was nothing new in the server event log.

But also this time there were no other calls so the log should contain only this incident.

 

[leejor]

A couple of thing that jump out at me, before even going into an analysis.. The transcoding, are you using the same Codec throughout? And, the odd port numbers. Usually devices tend to make use of ports in the 506X range when on a local network.

I'm not sure if this is an indication of an issue that may be causing the problems you are experiencing, or not.

I'm going to assume that you are auto provisioning the sets. Have you tried a manual provision, of just the basic line features (single line, no extra features), of your set, to see how that goes.

 

[lneblett]

I am interested in the thread, but want to make certain I understand it while you and Leejor work it -

It look to me like a call comes in and likely hits an IVR (805), then based upon the caller's input (or lack thereof) goes into a ring all queue (809) comprised of extensions 100, 101, 102, 103 & 105. 101 answers the call whereupon the calls/ringing to the other extensions are terminated. When you answered at 101, the system reported the call as 5376; yet the original inbound call was 5374. When you answered, you stayed on the line with the call for a little bit, but subsequently terminated the call (5376) some 42 seconds later, however the system continued to report an active call for 5374 after the termination.

Interesting and I wonder what the log showed for a little longer time period. Is it safe to say that there are no other phones, either hard or soft, with the same extensions anywhere else (remote or local)?

 

[ZenMasta]

lneblett

I think you're following about as best as I can explain.

For some additional clarity, when this happened the two times on that day it was different callers. Additionally, I don't think any particular queue is at fault as it has happened on everyone I think. I have wireshark open all the time but by the time I realize its happening it would be too late to RDP and start capturing. Plus I have never been able to reproduce this.

I'm pretty sure there aren't any duplicate registrations. Plus it would show up in the 3cx console under phones and in the logs also right?

So here's a massive Sigh...
Literally as I was typing this, I was looking at the backup trace logs on the server to see if I can copy paste a little more after what I had provided already, then a call came in and the same dang thing happened.

The customer called back and I had to run over to a different station to answer since everyone abandoned me without notice. I was able to ask the customer what they experienced and they said after the MOH it went silent so they hung up and called back.

The erver log also shows
Lost Call in Queue Sales (806) from Caller ID '####'
Queue: 806 [Sales]
caller: ####
but other times this has happened it doesn't indicate lost call.

I also put in a support ticket with nexvortex to see if they can provide any logs for this call.

Maybe you missed was as soon as the event occurs my phone actually locks up (remember that I did a phone swap already though). None of the buttons work so the only solution seems to be pulling the plug (power).So not necessarily that I terminated the call, but I had to pull the plug on my phone... although I did attempt to try and end call but pressing a button on my phone, I doubt the button press would have actually worked since it seemed locked up. Phones are all spa 504g n 7.5.1a firmware. I will note that the back light will go dim after the configured timeout and pressing any key will turn it back on, but nothing else on the screen would change.


ANYWAY, here's log of the most recent event... and it will have a few minutes of activity after the incident too.

 

[leejor]

Did you ever try manually provisioning the phone, just the basic extension, on the set?

Swapping out the set itself sort of proves that it is not the physical phone, as the issue continues, but, this doesn't eliminate the auto provisioning file as the culprit. It is just your extension experiencing this, correct? If so, then that would also point to the file, as the root of the problem.

 

[lneblett]

Thanks. A lot in the log to digest....debating whether I should have coffee or tequila before I try and tackle.....maybe both. In any event, I think Cisco has a later rev of firmware. I am running 7.5.2b on a 509g. Not sure that it will help. I'll take look at the log, and maybe I will see something, but it will be tomorrow am.

 

[ZenMasta]

At the original time I posted I think I was using the most current firmware. But I know about the current revision, just didn't think it was worth updating yet, although I know it would only take a few minutes just with auto provision...

It's only happened on my extension because I answer 90% of the calls so I think it's just coincidental.
I haven't tried provisioning manually yet. But I do have a spare phone I could auto provision on a different extension . Then if it happens again I can reset the phone and manually provision to rule out my extension or a provision template.

Sorry I didn't answer your question about the codec but since these are all auto provision, it's all default to whatever 3cx uses in their templates.

Is there a way to leave wireshark on 24/7 but only save say 10 mins worth of log at a time? This way if/when it happens again I could save the wireshark as well as 3cx logs....

And now that I think about it I'll have to check syslog to see if the phone spit out anything before crashing too, BUT I don't think it did
For what its worth I'm hosting 3cx server on 100mbps box with windows firewall only, no hardware fw.
Office behind pfsense firewall and residential cable 30mbps/4mbps

Phones provisioned automatically and then manually tweaked according to this blog
http://www.3cx.com/blog/voip-howto/remote-extensions/
stun is disabled though.

 

[lneblett]

Zen -

I had asked earlier about remote extensions. Are there any and more specifically any within the ring group? You indicated you are hosting the system, which implies to me that the 3CX system is not co-located with the phones. Is there VPN involved or any tunneling taking place? I am just trying to get my head around your exact configuration. I have to believe that there are others out there using call queues with SPA504G phones all the time; yet you seem to have something odd taking place. Lacking any added info, I am purely guessing that the signalling is getting messed up by your configuration or the Pfsense.

I did not really see much in the logs, so it looks like a wireshark is definitely needed. You can let it run for however long your disk space will suppport. You can then use the telephony tab within wireshar to zero in on a particular call.

 

[ZenMasta]

no vpn no 3cx tunneling.

Almost all phones are in all queues. We mostly just use the queues to help prepare for the next caller, change screens before answering etc.

It just happened to me yesterday, but this time at a remote extension (my home). My home phone is in all queues just logged out... here's what happened.

I sat down at my desk, and saw the BLF's flashing for exts at my office. At the time I actually forgot they were BLF because at the office BLF doesn't work. I pressed one of the BLF to figure out what was going on, and the screen said answering (and the number of some caller) and then my phone froze just as I've described before. Is that even how the BLF is supposed to work, allowing you to intercept a call?

I didn't quite realize the situation until a little later because I was more concerned with trying to call the customer back.

This extension is at my house behind a linksys with custom firmware. So this rules out any blame toward pfsense at my office I would think.

 

[lneblett]

Why does BLF not work at the office? In any event, BLF allows for presence monitoring of extensions. The presence or status is indicated by lights on the phone. Should you have a key set-up for BLF, then it should allow you to see the status of the extenion and if no call (usually a solid light) is then in progress for that extension, then pressing the BLF key should act as a speed dial to that extension. If you see that a call is ringing (usually a blinking light) at that extension, then pressing the key should allow for your extension to intercept and pick-up that call. If the call was answered by the original extension, then the light indication may be red instead of green or it may blink at a different rate or pattern. This indications and how presented are dependent on phone make and model.

You might try the tunneling option to see if any variation. The custom firmware may or may not have a say in how it reacts and I don't know that pfsense is ruled out. The tunnel may just eliminate both issues...if indeed there might be more than one and each side is impacted.

Can you do a wireshark and do the telephony flow? This is a graphical representation of how the datastreams flow between IP addresses without all the detail. It may provide some insight as to why a hang.

 

[ZenMasta]

Seeing as how it took almost a month for this to happen again, it would be insane to leave wireshark on so long.

I may have to try the tunnel at this location sometime.

 

[ZenMasta]

I know probably nothing can be done but just sort of reporting that this happened again today. One inbound call. My phone was using 7.5.1a firmware. Sadly also syslog was not up at the time either to see if the phone logged anything.

I had to unplug power to my phone because it froze. I can't ever say I had any real consistent frequency of this event in the past, but obviously this being almost 6 months since my last incident it is not a significant problem. Just surprised that nobody has seen this. Doubt I'm the only one.

 

[sigma1]

This usually happens on unstable networks where the phone is unable to reach the PBX and vice-versa or when you have a funky switch onsite.