Server error/ not able to login

[nielskool]

Hi all,

I have a server running at the google cloud computing. This morning I got the following mail:

3CX Service notification
Following service(s) were stopped recently:
3CXPhoneSystemMC01

I was unable to login to the 3cx web panel. I had to restart the server. Anyone know why this happend? And which logs I can check for this?

Kind regards,
Niels Kool

//edit
Did find these lines:
3CXConfService.log

06:29:25.930|7faef67fc700| Info|/home/repomaster/workspace/15.5SP4/SPBuild/Sources/Projects/SLDBServ/src/DBServ.cpp(533): SL:Receive failed: socket closed
06:29:27.600|7faef67fc700| Info|/home/repomaster/workspace/15.5SP4/SPBuild/Sources/Projects/SLDBServ/src/DBServ.cpp(533): SL:Remote disconnect: "instance-1:0/ManagementConsoleJS" stopped
06:29:27.600|7faef67fc700| Info|/home/repomaster/workspace/15.5SP4/SPBuild/Sources/Projects/SLDBServ/src/DBServ.cpp(72): instance-1:0/ManagementConsoleJS disconnected

I also see an unusual long list of warnings and errors at that time in the 3CXTunnel.log

 

[eddv123]

Hi Niels,

Obviously it seems one or more of your 3CX services has stopped including the 3CX management console service.

Did this happen randomly? I am not sure what access you have on Google cloud but if running on a Linux instance you could login (If it does it again) and restart the service. At the very least you can see which services have stopped and thus narrow down the search.

 

[nielskool]

I have full control over the linux instance so i just restarted the instance. This happend randomly, didn't change anything in the week before.
But I am more concerned about the reliability. The server has only been running for 1 month now.

 

[YiannisH_3CX]

Hello @nielskool

As @eddv123 mentioned it would be helpful to know exactly which services stop in order to be able to try and determine the cause. There are a number of reason why a service could stop like running out of resources or an OS exception. Keep an eye on the issue and see if this behaviour repeats. If so check which services stop and report back. If the issue persists you could also create a ticket with our support department so they can look into the system logs and determine the cause of the issue.

 

[nielskool]

It happend again a few times. Only the service mentioned in my first post keeps stopping: 3CXPhoneSystemMC01.service (3CX PhoneSystem 01 Management Console).
The other services remain active.

Any thoughts about this behaviour?

 

[eddv123]

Can you specify how many SC licence you have for 3CX and what you are hosting the system on as well as the hardware specifications - I still think this is resource related.

 

[nielskool]

For now it is a Free license running on a virtual machine of google cloud, still for testing purpose. Only when the service is failing the resources spike. I don't know what happens at 7:00 am that makes everything spike that much. A normal call has almost no impact on the system.

Attached are resource usage graphs over the past day. there is 90-100% cpu during that moment when the services fails. In the past week the system didn't even reach 9% cpu load. I don't know what happens at that moment. No automatic backup configured but the disk reaches 56MB/s read traffic, network traffic remains normal.

 

[eddv123]

Are you doing any of the following:

* Storing large amounts of call recordings on the server itself.
* Using a large paging group.
* Using remote phones with STUN method.

I'd like to suggest that you run the following benchmarking tool on your server - Outside of working hours.

http://ssd.userbenchmark.com/

It'll tell you how well most components on your machine are running thus helping you identify whether this is a hardware issue or not.

 

[nielskool]

No recordings are stored on the server.
At this moment there are 16 extensions of which only 5 are ever registered.
3 page groups, largest contains 5 extensions.
No phone uses STUN at the moment.

I will comeback with the benchmark results.