Solved 3CXSystemService01 crashing repeatedly (core-dump status=6/ABRT) – v20 Update 8 on Debian 12

Status
Not open for further replies.

SpreenTech

Premier Customer
Joined
Oct 9, 2016
Messages
12
Reaction score
4
We are seeing repeated hard crashes of 3CXSystemService01.service on 3CX v20 Update 8 (Build 1109).

This is not a reboot, not resource exhaustion, and not an OS issue. The process is aborting and generating core dumps.

Environment​

  • 3CX v20 Update 8 (Build 1109)
  • Debian 12
  • Vultr VPS
  • 6 vCPU
  • 16GB RAM
  • 320GB SSD
  • 48 Simultaneous Calls (Professional)
  • ~278 extensions
  • Multiple SIP trunks (Flowroute, VoxTelesys)
  • SBC deployments across multiple branch offices
  • Active live traffic (automotive dealership group with call centers)

The Failure​

Systemd logs show:

3CXSystemService01.service: Main process exited, code=dumped, status=6/ABRT
3CXSystemService01.service: Failed with result 'core-dump'

Event ID 30035 appears in 3CX:

The following service(s) were interrupted: 3CXSystemService01

The OS remains stable. No OOM. No kernel panic. No CPU spike. No disk issue.

The service runs approximately 60–90 minutes under live traffic and then aborts. It auto-restarts and repeats.

This is clearly an internal application-level crash.

What We Have Confirmed​

  • CPU load is minimal
  • RAM usage stable
  • Swap not under pressure
  • Disk usage ~17%
  • No system reboots
  • No kernel-level faults
  • Core dumps generated
This appears to be a deterministic crash condition triggered by live SIP traffic.

Questions​

  1. Is this a known issue in Update 8?
  2. Has anyone seen SystemService abort under multi-trunk environments?
  3. Could this be related to SIP parsing edge cases?
  4. Any known incompatibilities with certain trunk providers?
  5. Is there a hotfix beyond Build 1109 addressing SystemService crashes?
Before escalating, we’d like to confirm whether this is a known defect in v20 U8.

Given this is a production system with multiple locations and call centers, repeated SystemService aborts are not acceptable behavior.

Looking for direct technical insight.
 

Attachments

We will need to investigate this further, will it be possible to create a support ticket?
Also it seems to happen every 1-2 hours, is there any scheduled activity?
 
I've dumped all my logs, there is no correlating activity or scheduled activity. I wrote a whole post that tagged onto another user and others that had the same issues. I was told by support at the end of that thread to make a new thread with all my issues\findings. Here I am. Seems like there is about 3 threads with the same issues. This all started to happen after last update. Thats also what other users have reported.

Can you roll back an update?
 
Hello @SpreenTech,
Sent you a DM so we learn more as there is no general issue that we are aware of that could lead to such crash.
 
With the help of 3CX support staff seems like we've found the root of the problem. Thank you.

Providing a final update for anyone who may encounter similar behavior.

After further investigation, we identified that our external FTP/SFTP storage target (used for archiving and backups) had reached capacity. The remote storage volume had grown to ~130GB and was fully saturated.

Although the 3CX recording retention settings were configured with a 5GB local cap, the offsite storage location continued accumulating data without effective cleanup on the remote side.

This appears to have created repeated archiving/remote storage failures which coincided with the 3CXSystemService01 abort events.

Since correcting the remote storage issue, we have not observed additional crashes. Hopefully we can set a max remote off-site storage? or automatic pruning behavior?

It appears the storage failure condition was contributing to the abort behavior.

Appreciate the team reviewing the dumps and logs. We’ll continue monitoring, but at this point the system appears stable.
 
@SpreenTech - Thank you so much for your message. Glad things are resolved. We will look into better error handling of such cases....
 
Status
Not open for further replies.

Forum statistics

Threads
112,283
Messages
591,700
Members
165,406
Latest member
ARKC