3CXCfgServ01.service failed with result 'protocol', management console won't load, extension won't register, calls failing

[landshrk]

I hit the 10k character limit, so I've moved all the outputs of the various troubleshooting/commands to some paste.bin links. Sorry for the inconvenience.

---

This is an instance of 3CX that I've been running on a dedicated RasPi 4 for a little over a year.

* Raspbian GNU/Linux 10 (buster)
* Raspberry Pi 4 Model B rev 1.1
* Linux kernel: 5.10.60-v7l+

Used the

wget https://downloads-global.3cx.com/downloads/misc/d10pi.zip; sudo bash d10pi.zip

command provided on 3CX's official documentation for installing this on a Raspberry Pi/Linux, and it's been pretty reliable for quite a while. However, I use voip.ms for trunking, and I'm sure most of you are aware of what's been going on with them and the DDoS attack they've been weathering. As such, I haven't messed with my phone much in like a week, just forwarding all inbound calls to my cell phone. My coworker also uses voip.ms for his "home phone" and he and I have been chatting off and on throughout this mess. He informed me today that it's working again, for him, so I figured I'd start working to get my stuff all fixed. Went to log in to the management console, and I get the stupid "server error" message and it won't sign me in. Great. So, here's what I know so far:

https://pastebin.com/BWC2mGzk

Alright, so 3CXCfgServ01.service is failing to start, with the result 'protocol'. A quick google of that brings up... almost nothing. Awesome. There's one thread here, where the user is able to get their instance working after running a Linux update. Granted, their error wasn't exactly the same as mine, but it's still a 3CX service that's failing to start, even when being told to manually. Worth a shot!

Alas, no dice. sudo apt-get update and sudo apt-get upgrade did find plenty of stuff to update, because I've not touched this thing in months, but no dice on a reboot after updating. Still failing to load the 3CXSfgServ01.service, and since that's the only error I'm getting, I'm assuming that that's the issue.

I have re-run the wget script from 3cx.com, again, no change.

Here's the output of

journalctl -xe

, for what it's worth:

https://pastebin.com/b4hvUq6s

And here's the output/results of sudo 3CXStartServices (after a successful 3CXStopServices, of course)

https://pastebin.com/yAbDZUUw

So, any thoughts? I'd like to be able to get this working without having to redo the whole thing, if possible, but everything I'm looking up just keeps taking me in circles, and honestly, I'm getting exasperated. So thanks in advance for any help/advice y'all have, and please let me know if you need any more information, I'm happy to send along whatever you need.

 

[landshrk]

Bump? Would suck to have to get rid of 3CX because of this problem, but I'm forced to shift my phone's registration back to voip.ms directly for now. Would really love some direction here.

 

[NickD_3CX]

Hi!

I'd want to start off by saying that if you have a recent backup, I think by far the easiest solution here would be to use that backup and re-install 3CX.
Even if you do have to redo some configuration, it would probably be faster.

Regarding troubleshooting, I would suggest running "3CXStopServices", then "3CXStartServices" again, then checking the /var/llog/syslog file to see if that sheds any more light.

Also maybe check with df -h that there is enough disk space.

One more place you could check for clues is in the /var/lib/3cxpbx/Instance1/Data/Logs/ , after you do "3CXStopServices", delete all the files in this folder, then do "3CXStartServices" so everything is clean.

 

[landshrk]

Hi!

I'd want to start off by saying that if you have a recent backup, I think by far the easiest solution here would be to use that backup and re-install 3CX.
Even if you do have to redo some configuration, it would probably be faster.

Regarding troubleshooting, I would suggest running "3CXStopServices", then "3CXStartServices" again, then checking the /var/llog/syslog file to see if that sheds any more light.

Also maybe check with df -h that there is enough disk space.

One more place you could check for clues is in the /var/lib/3cxpbx/Instance1/Data/Logs/ , after you do "3CXStopServices", delete all the files in this folder, then do "3CXStartServices" so everything is clean.

Yeah, I ended up just taking an hour and rebuilding the thing. It wasn't too hard, since my setup is pretty simple with just my extension and a straightforward IVR (that I'm not even using anymore). I was hoping to find another solution, since this seems to be a thing that happens not infrequently, and if I were going to have a 3CX instance running in the field, it would be nice to know ways to go about fixing this without having to resort to a wipe and re-install, even from backup.

As for your suggestions on things to look at, there's plenty of disk space. (3CX is the only thing running on this RPi, and it's installed on a 32 GB SD card with only Raspbian and the prereq files/apps for that. Even the backups are sent to another RPi on the network, my little mini-NAS.)

If it happens again, I'll def check out the syslog and /Data/Logs/ folder, see if anything useful is showing up there.

Seems kinda sus that this thing could be working fine for months, and then all of a sudden with no outward warning just fail and fail hard. But hey. It's free, I guess?

Anyway, thanks for the response!