Grandstream GXP and GRP Phones going Offline Daily

[bcarrico]

We have started checking "online" extensions first thing in the morning through the 3CX Management Interface because we have to reboot Grandstream phones so much. They just show go offline. A reboot of the phone fixes the issue every time. We even have the client "factory reset" their phones when they notice it. It has become a real pain. Is this something known between 3CX and Grandstream? Or could it be something on my end? The configuration is mixed between SBC's and VPN. We have very few Direct SIP units. I don't have any of these issues with other phone manufactures.

 

[AWS2P]

As you didn't say, I guess they are up to date for FW and templates, and you are running latest stable V16 ?

 

[bcarrico]

Yes they're on the latest firmware. We keep those up to date as 3CX releases the phone template updates.

But, we're not on a stable release of the 3CX Server. We're on 16.0.5.553. I hope it's as simple as us not being on stable at this point. But, it's been happening long before going to a beta release.

 

[Evolute IT]

Since you're on a Beta, don't expect 3CX support.

That being said, Grandstream phones are tricky. Are you using STUN, VPN and SBC all at the same site?

 

[bcarrico]

Since you're on a Beta, don't expect 3CX support.

That being said, Grandstream phones are tricky. Are you using STUN, VPN and SBC all at the same site?

Yeah I figured there would be no official support. It has just become a frustration at this point.

We have several sites. Most of them are using linux SBC's in the VMware environment onsite. It's not every phone that goes offline, but random phones just drop. We either SSH into them and reboot or cycle the PoE from the switch. Then all is good. Last week, we moved two sites from SBC to VPN to see if there's a difference, but still seeing the same result.

 

[Evolute IT]

Yeah I figured there would be no official support. It has just become a frustration at this point.

We have several sites. Most of them are using linux SBC's in the VMware environment onsite. It's not every phone that goes offline, but random phones just drop. We either SSH into them and reboot or cycle the PoE from the switch. Then all is good. Last week, we moved two sites from SBC to VPN to see if there's a difference, but still seeing the same result.

What models of GXP/GRP and what firmwares on each please.

Try to notice if the phones that are offline have any differences: manufacturing date, firmware version, device revision, power adapter issue, different switches...

A lot can cause this.

 

[bcarrico]

GXP2160 with Firmware 1.0.11.6
GRP2615 with Firmware 1.0.3.6

We are keeping track of what phones are going offline and which we are factory resetting. I'll ask that we get the other details as well. I may need to log a ticket with Grandstream.

 

[Evolute IT]

Yes please do. Also, try to open a thread of GS forums as well. We are a lot there to help.

 

[AlexanderReichert]

I was about to post a question about that today and then I saw this thread.
I have exactly the same issue. GXPXX phones on latest firmware release 1.0.11.6 - 3CX 16.0.4.508 (provided by 3CX support to fix a different issue) but issue also occurred on 16.0.4.504 latest stable release.

all phones are on direct SIP

what I did observe while 3CX shows the phone as unregistered if I go into the phone UI the phone says it is registered.

a re-provision in the phone UI and reboot of the phone will fix it for some time only to re-occure a couple days later.

Frustrating is not an expression for this situation

 

[bcarrico]

I was about to post a question about that today and then I saw this thread.
I have exactly the same issue. GXPXX phones on latest firmware release 1.0.11.6 - 3CX 16.0.4.508 (provided by 3CX support to fix a different issue) but issue also occurred on 16.0.4.504 latest stable release.

all phones are on direct SIP

what I did observe while 3CX shows the phone as unregistered if I go into the phone UI the phone says it is registered.

a re-provision in the phone UI and reboot of the phone will fix it for some time only to re-occure a couple days later.

Frustrating is not an expression for this situation

Glad to know we're not the only one. It's been ongoing for awhile, but it seems like lately it has just reached a breaking point. I logged a ticket with Grandstream Support and they're blaming it on networking issues. I guess we'll just slowly start replacing the Grandstream units with something different.

 

[AlexanderReichert]

ah the good old "it's the network" answer. so tired of this. There were no changes in the network settings in our environment for month so that is really not playing into it. It would be nice to figure this out. If you hear anything else pleas let me know.

We have not the luxury to replace phones here - this is a small non-profit where it happens and they are strapped for money as it is right now.

 

[bcarrico]

ah the good old "it's the network" answer. so tired of this. There were no changes in the network settings in our environment for month so that is really not playing into it. It would be nice to figure this out. If you hear anything else pleas let me know.

We have not the luxury to replace phones here - this is a small non-profit where it happens and they are strapped for money as it is right now.

Oh I agree 100%. I'll be sure and reply back if we find anything out. We don't really want to just replace phones either. But, it's really causing customer sat issues and we don't really want to lose a customer over it.

 

[AlexanderReichert]

also to debunk the network issue: the phone is still reachable via the phone UI. so clearly there is not an issue in the network.

 

[AlexanderReichert]

got some captures after one of our again unregistered and never re-registered. what is baffling me that register expiration on the phone is set to 15 minutes. during the 30 minute capture the phone did not re-register once.

note to the captures: the PBX captures boils it down to only that particular extension 1005
and the phone capture has everything.

 

[Matt Scanzano]

got some captures after one of our again unregistered and never re-registered. what is baffling me that register expiration on the phone is set to 15 minutes. during the 30 minute capture the phone did not re-register once.

note to the captures: the PBX captures boils it down to only that particular extension 1005
and the phone capture has everything.

We had a similar problem with Grandstreams that caused us and our clients a whole heap of trouble. Are the phones getting a "No LAN cable" message on the screen? We also had some phones that would not show the error message but would lose registration completely/designed slightly large. It boiled down to the RJ45 port in the back of the phone being slightly loose. I would be inclined to believe that the patch cable probably has the ability to move around a lot in the RJ45 port. It seems like a weird and unlikely issue, but we had a batch of over 100 Grandstream phones that had this problem, which turned us away from this brand.

A good test would be to reboot/reregister the phone, and then wiggle the patch cable for a bit to see if it loses it's registration (Don't force it).

 

[AlexanderReichert]

@Matt Scanzano appreciate the thought but I doubt this is the issue.
1. These behavior started after a firmware upgrade of the Grandstreams
2. the phones are not moved around at all right now, because people working from home and the phones are not being used.
3. and it is effecting literally every Grandstream phone we have now - not all at the same time but randomly.

and remember: the phone itself still thinks it is registered. 3CX unregistered it however.

 

[Matt Scanzano]

@Matt Scanzano appreciate the thought but I doubt this is the issue.
1. These behavior started after a firmware upgrade of the Grandstreams
2. the phones are not moved around at all right now, because people working from home and the phones are not being used.
3. and it is effecting literally every Grandstream phone we have now - not all at the same time but randomly.

How long have these phones been deployed for? How are the phone's connecting to the PBX. Is the PBX onsite or hosted?

 

[AlexanderReichert]

onsite pbx phones and pbx are on the same VLAN. deployed for over 2 years and this started in February or so after a firmware upgrade of Grandstream pushed down by 3CX.

 

[Matt Scanzano]

Okay, so the good ol' bad batch of phones is out of the question. I checked out your capture files, I didn't see any registration on either of them?

Do you know what firmware version did not have this issue and also have you tried reverting to an older firmware? If you decide to downgrade the firmware, you might have to move/remove the firmware that is currently archived by 3cx under /Instance1/Data/Http/Interface/provisioning/[some random string]/firmware (this could also be named firmware_new)/grandstream/[phone model].bin otherwise it might revert after a reboot. What specific issue was supposed to be fixed on this current firmware release that made you upgrade the phones?

 

[AlexanderReichert]

no I did not try revert to an older firmware version. Mainly because I wasn't quite sure how to do that.

to the question why upgrade: mainly 3CX made me LOL - it loaded down the new firmware - said 3CX detects phone with unsupported firmware - please update.

I checked in the readme of the firmware after these issues started and didn't detect anything specific that would point that it broke something. but I say that with all carefulness - I don't claim to be an expert in phone firmware started with firmware 1.0.11.3