Server goes unresponsive ... where to look first?

[Oliver Broad]

Not an urgent problem as this time I just remoted in and forced a restart, but I'd like to have a response that's more sophisticated than turning it off and on again?

So this morning I found that all the phones were down, but the machine was up and accepting a remote desktop connection. For want of a better plan I forced a restart.

Then I ran into the thing with the delayed service start that means it won't accept log-ins for a while.

After that it wouldn't allow console access over IPv6

Now a couple of hours later it all seems back to normal.


So I suppose myactual question is "given direct access to the server and an unresponsive 3CX management console what steps can I take"

 

[Ritter Technologie]

First have a look in the 3cxsystemservice.log

 

[Alphabetic]

Windows event log leading up to the event.

 

[Alejandro_3CX]

This is a windows installation, the server must be dedicated to the 3CX without any third party software installed, for those 6 users the server should have minimum 2cores CPU and 2GB RAM. Validate that the OS is up to date.

Later you can set the logs to verbose and if the issue reoccurs check the logs from the activity log and the windows event log for system and application.

 

[Oliver Broad]

Device name redacted
Processor Intel(R) Core(TM) i3-3240 CPU @ 3.40GHz 3.40 GHz
Installed RAM 8.00 GB (7.89 GB usable)
Device ID 6C4F4386-EAC3-4910-BA1D-7939D6CB3572
Product ID 00330-80000-00000-AA609
System type 64-bit operating system, x64-based processor
Pen and touch No pen or touch input is available for this display

I think it might have been hit by a big update overnight. I'm really not good at interpreting events in Window Event Viewer but I'm seeing gems like this one:

Level Date and Time Source Event ID Task Category
Error 11/10/2023 04:05:31 PostgreSQL 0 None The operation completed successfully.

Variations on this keep coming up:

Level Date and Time Source Event ID Task Category
Warning 11/10/2023 12:46:39 3CXManagementConsole 0 None "Category: ProductUpdater.Services.UpdateService
EventId: 0

Server returned 'Forbidden' when 'https://downloads-global.3cx.com/downloads/v180/hotfixes/hotfixes.xml' was requested
"
Error 11/10/2023 12:46:39 3CXManagementConsole 0 None "Category: ProductUpdater.Services.DefaultUpdateDefinitionProvider
EventId: 0

Request to https://downloads-global.3cx.com/downloads/v180/hotfixes/hotfixes.xml has failed with message $Server returned 'Forbidden' when 'https://downloads-global.3cx.com/downloads/v180/hotfixes/hotfixes.xml' was requested
"

It also appears to be spammed with events from from Security-SPP

 

[SnakeRiverWebDevelopment]

This might be the problem

 

[Oliver Broad]

I suppose that this is what happens when you put protection against automated downloads on a URL used by your automatic downloader