Recordings and Voicemail saying FULL

[IDS IT]

This inherently started by not having enough space on the disk, so partly self inflicted, but I just can’t seem to get 3CX to now see there’s more space available.

We ran out of disk space, I cleared the oldest recordings directly on our ubuntu lightsail box so I was able to get back into the UI (it kept saying internal server error so I couldn’t even get into the front end), was able to get back into the front end after I’d cleared some out and have migrated the phone system to a new server with more storage (lightsail snapshot -> build from snapshot)

The phone system is back up, it works, the storage even shows in both ubuntu and on the dashboard of 3cx that we have over 400gb of free space, but it just still says full on both. I can temporarily “fix” is by setting the quota to 0GB and it seems to go away for a couple of minutes but it comes back and as a result, all calls won’t record and customers can’t leave voicemails.

Any idea’s would be greatly received without having to do a full restore but I don’t know if that’d even help

 

[IDS IT]

3CX installation _______ is running out of space. (898.9 Mb Space left out of 1000 Mb). As a result of this, recordings, logging and voicemail services are disabled until this problem is resolved. To resolve this issue login to the machine, free up some space or add more disk space and then reboot the server.

This is what I was getting, so I’m assuming there’s some partition here or something that I need to fix but I’ve tried using all the UI tools (optimise database etc etc) and nothing seems to resolve it

 

[TheodorosG_3CX]

From the PBX check also the below options to identify which part is occupied:

 

[IDS IT]

The VmallocTotal looks a bit suspicious but because of how large it is, could be a red herring, but info below

 

[TheodorosG_3CX]

And what is the exact alert notification that you receive as based on the screenshots there is enough disk space.

 

[IDS IT]

we were getting this, so cleared space initially then migrated to a new VM with more storage (Saturday 31st), we're not getting these anymore BUT the system still says "full", if I set the quota to 0GB it momentarily resets to no longer say "full", but give it 10 minutes or so and it's back to saying "full", disabling voice recordings and voicemails when it does.

 

[TheodorosG_3CX]

Would it be possible to contact your Partner and create a support ticket so we can check and advice further?

 

[IDS IT]

I guess I'd need to reach out to a partner first! We've just handled this all ourselves over the 6 years we've had the phone system - will do some research on local 3CX partners and go from there, but if anyone else can advise it'd be appreciated.

Thank you for your help so far.

 

[Evolute IT]

I guess I'd need to reach out to a partner first! We've just handled this all ourselves over the 6 years we've had the phone system - will do some research on local 3CX partners and go from there, but if anyone else can advise it'd be appreciated.

Thank you for your help so far.

I'd be happy to assist with the ticket if needed.

 

[IDS IT]

I'd be happy to assist with the ticket if needed.

I appreciate this thank you - sod's law, I rebuilt the phone system again (I've been poking around in linux making changes over the last few days and wanted to make sure it was "as it was" before I did any of this to not lead anyone down a rabbit hole / cause any further issues) and by some weird miracle, it's resolved itself - it still said "full" after restoring, I reset the quotas, checked all the services were back up and did some testing over a 30 minute period and low and behold, the "FULL" messages have gone, and have stayed gone for almost an hour now.

Here's what it was (1st image) vs here's what it is now (2nd image), just to show I wasn't going mad!

I'll leave it alone for a while and see how it goes, but genuinely stumped on why it would have failed the first time whilst following the same steps.

 

[KyriacosS_3CX]

Out of curiosity, was the second restore from the original backup, or from a backup taken from the first restoration?

 

[IDS IT]

they were both created from the same snapshot in lightsail (so rather than doing a new build -> restore from backup, use a snapshot of existing - build new linux instance from snapshot)

 

[KyriacosS_3CX]

Thanks, very odd then as that should have yielded identical results. Glad it looks sorted now.

 

[ivank]

Every 30 minutes PBX checks the partition where /var/lib/3cxpbx resides, if it is less than MIN_FREE_MB (1000 when absent), this happens. Apparently for some reason it was unable to get the correct free disk space right (logs could help here).