Windows client loses connection to API when CPU spikes above 40-50%

[cam]

I have installed 3cx on Debian which is on a HyperV VM

The issue we have is when the CPU spikes above 50-60% our 3cx windows clients lose connection until the CPU drops with error:

"3cx phone for windows could not establish a connection to the 3cx phone system web api...."

They will reconnect when the CPU drops back down.

I have checked Debian and the 3cxManagementCo processs is causing the spike

Any idea what could be causing the spike, we are on the latest supported version and only have 16 SIM calls,

This is happening every few minutes and when we have no active calls and no obvious activity on the system.

I have done the following and it has not resolved the issue:

Restarted the 3cx services
Rebooted Debian
Purged calls
Purged chat history

 

[cam]

I have also increased the vCPU and it still drops even though the spike is now above 30%

 

[cam]

Update, I had increased the vcpu to 4, initially there was no change but after 30 minutes it seems to be OK, I will continue to monitor it.

 

[JohnS_3CX]

Hi Cam,

Share some details please as per this post so we can better understand your system.
https://www.3cx.com/community/threads/information-to-provide-when-requesting-help.67558/

Also please state how many 3CX clients are simultaneously connecting to the machine.

 

[cam]

Hi

It is really bad today, users are seeing calls but the softphone keeps losing connection and dropping the call

• 3CX Version, e.g. Standard Annual 16.0.3.676
• Server OS, Debian 9
• Is the 3CX Server Hosted and where? HyperV 2016 server on prem
• IP Phone Make/Model/Firmware: Windows softphone 16.0.1.81
• Provisioning Method: Local
• Trunk Provider or Gateway Gamma
• Has the Firewall Checker passed: YES
• Are custom Phone Templates being used: NO

 

[JohnS_3CX]

Also please state how many 3CX clients are simultaneously connecting to the machine.

 

[cam]

I have rebooted the HyperV host and still having issues, in the attached screenshot the clients disconnected briefly when the cpu went up to nearly 20% so it is pretty bad

 

[cam]

At the moment I have 39 clients connected, this includes phones and windows clients

 

[Zol]

Have you already checked out this link?

https://www.3cx.com/docs/installing-microsoft-hyper-v/#h.cjj20zk71lns

 

[cam]

Hi, I had seen this but I have just realised I may not have a legacy adaptor as I have this message in the settings panel, I have also not got a static mac address,

How should I proceed with setting a static mac and changing the adaptor?.

 

[Zol]

Edit settings on your VM in HyperV and change the Network Adapter Type.
I have VMXNET 3

 

[JohnS_3CX]

Edit settings on your VM in HyperV and change the Network Adapter Type.
I have VMXNET 3

This is in VMware hosts. He is using Microsoft Hyper-V as the host virtualization environment.

 

[JohnS_3CX]

Hi, I had seen this but I have just realised I may not have a legacy adaptor as I have this message in the settings panel, I have also not got a static mac address,

How should I proceed with setting a static mac and changing the adaptor?.

View attachment 13474

You must NOT use the legacy adapter, you are using a Debian 9 guest OS. Keep the adapter you have now.

For the MAC, read all the instructions and do it exactly as the guide describes.

Did you install NTP as per the instructions?

 

[Zol]

This is in VMware hosts. He is using Microsoft Hyper-V as the host virtualization environment.

OMG, I've read wrong. Thanks for the correction

 

[cam]

I did wonder Zol

The ntp is set and the server has the correct time.

With regards to the mac address, will this cause these types of issues?

 

[JohnS_3CX]

Hi Cam,

I can suggest a few things to check, but first run a full backup of your system.

• Hard disk space: ensure it has not run out
• Swap partition: does your Debian have one?
• Activity Log: put it on "Low"
• Chat logs: after the mentioned backup, enable the viewability by clicking in the PBX status box on your dashboard. Purge the logs if there is a very large number and see if this reduces your cpu load
• Any integrations or scripts running?
• Any modifications to the PBX that may affect default performance?
• What else is running on the Host Windows Server? Is the machine dedicated or does it do other tasks too?
• How many vCPUS and RAM is dedicated to the VM?
• What CPU model is in the actual server?

 

[cam]

Sorry another project has taken priority over this,. The server is over 5 years old now so I am considering moving it to a new one. Can I just change the mac address to static and use the mac that is already assigned to it.

Then would I be best moving the VM using hyperv export\import or would it be better using the backup and restore option in 3cx?

thanks

 

[cobaltit]

Export/Import isn't going to change the underlying OS and chances are at 5 years old the OS is no longer supported for v16. Export/import would be my recommendation. And no need to keep the MAC address, just make sure the new VM is configured for a static MAC.

 

[cam]

• Hi Cam,
  
  I can suggest a few things to check, but first run a full backup of your system.
  
  

  • Hard disk space: ensure it has not run out Space is fine
  • Swap partition: does your Debian have one?: Yes but 0 used
  • Activity Log: put it on "Low" - already done
  • Chat logs: after the mentioned backup, enable the viewability by clicking in the PBX status box on your dashboard. Purge the logs if there is a very large number and see if this reduces your cpu load - already done
  • Any integrations or scripts running? - no
  • Any modifications to the PBX that may affect default performance? none
  • What else is running on the Host Windows Server? Is the machine dedicated or does it do other tasks too?, The Hyper-V Server has 2 other vm's a DC and a windows 2016 server running veeam. Both use little resources
  • How many vCPUS and RAM is dedicated to the VM? 4vcpu and 4gb RAM
  • What CPU model is in the actual server? Intel Xenon E5-2403 v2

  [*][*]

I have answered your questions, the server is over 5 years old but has Windows 2016 HyperV server

 

[JohnS_3CX]

Hi Cam,

That is an old and slow CPU and as such the high usage spikes are more indicative of the chip's limited processing power. I don't think you would see such high usage on any machine that exceeds 3 GHz. Having enough users on soft clients will spike up a slower CPU especially on the management console service which does part of the communication with the softclients.

I would recommend moving your machine to something faster since you are migrating anyway, so here are our recently updated hardware recommendations
https://www.3cx.com/docs/recommended-hardware-specifications-for-3cx/