Backup questions & troubleshooting

[Gandres]

Hello there,
I'm having some trouble understanding how I need to migrate my system from one host to another. We are currently using an on-premise server running Debian for 3CX.

In my system, I've set up ~50 user accounts, some auto-attendants (digital receptionists), recorded a couple voicemails, phone greeting messages, ring groups, inbound and outbound rules... Basically it's almost ready to replace our overpriced Cisco UCM (of which we use maybe 10% of its features). I then created a couple backups which are stored on an in-house file server.

I then shut down this system and did a fresh install on a different one. I configured this second system with the same IP address. During the setup phase it asked if I wanted to continue with a fresh installation or restore from a backup. I uploaded each of the backup files I created and each said the file was invalid. I know we're not supposed to extract the zip file, but I figured I'd at least take a look inside to see how it's laid out. I can't seem to extract it or view the contents either. Were the backups corrupted before they even came out of the oven? I know I don't have much data, but they're only 33.7kb each. Surely even very downsampled audio recordings would take up more space than that? (To be clear, I ticked every box on the backup settings).

So because the backup restore didn't work at the intro-screen, I proceeded to set up the new system like I did the original. Once I had access to the admin control panel, I went to the Backup and Restore page, pointed the location at my file server, and started restoring the backup. The process kicks me out, and about 20 minutes later it emails me saying the restoration is complete. I log in and find... nothing. The system is still completely blank. No users, no recordings, no rules, not even any SIP Trunks... (Upon re-reading the email, the restore process completed unsuccessfully. ie, it failed.)

At this point, the only backups I can do are from the VM controller. I didn't mention this before, but I'm running the 3cx PBX Debian install with LibVirt. In order to back up, I have to have a cron script that shuts down the VM and makes a copy/tarball of the virtual hard drive from Install #1. Not the greatest solution but it works.

Please tell me what I'm doing wrong... I'm confused.

 

[JohnS_3CX]

I then created a couple backups which are stored on an in-house file server.

Normally the zips can be opened and you can see the contents. If you were backing up directly to a network drive, it would explain the corruption as 3CX requires block level access to drives it writes to.

Your zip application "Test" button should report the zip is OK on a healthy backup..

 

[Gandres]

If the backup needs block level access, why is SFTP an option?

I was trying to do the backup with that. The permissions appear to be correct. It's using the primary admin account of the backup server, which has RWX permissions on to the folder I use to store backups.

The backup server is also Linux based (as is our local install of 3CX). The backup feature successfully connects to that server over SFTP using the SSH key of that admin account.

 

[JohnS_3CX]

If the backup needs block level access, why is SFTP an option?

You did not specify the transport method in your original post so I was just informing you ahead of time about block level access (it only applies to NAS). Since you are using SFTP do not worry about that requirement!



Ok you mentioned you shut down the original system, so if you still have access to it you can start the machine again and run a quick test so we can troubleshoot:

1. Make a backup to your SFTP again - exactly like before

2. When you get an email that the backup was completed successfully, we have to try two methods to download it:

a) copy it from your SFTP server, using the same method you did before writing your forum post

b) download it directly from your 3CX Management Console using the "Download" button in the Backup page.

Which of the two methods gave you a corrupted file?

 

[Gandres]

I don't actually get those emails... That's a minor thing that I don't care to address presently.
Either way both methods produce a corrupted backup. I can tell even before accessing the file that it is invalid because any backups done to the SFTP server create only a ~33kb file. If I backup to the local disk then the file is 7+mb. I'll reiterate that I can't see any permissions issues and the files are being created on the remote system but just not getting filled with any data.

 

[Gandres]

One more screenshot showing local disk backups with more appropriately sized zips.

 

[JohnS_3CX]

Hi @Gandres

Ok that was the next thing I was going to ask you to do, make a local backup and compare. So since your local backups appear to be healthy, but the SFTP ones are not, the only thing we can say here is that whatever may be going on, it is related to the SFTP session.

Just for comparison, I setup a connection from one 3CX system, to another Linux machine just like you did and performed both small backups of a few megabytes, and large ones of upto 3.5GB and was able to successfully store and retrieve those backups.


I think that whatever the issue is in your case, it is happening on the other end but I cannot assist you further regarding what happens on the destination server - that is something you will have to look further into.

What I can suggest though to act as a troubleshooting aid, is the same test I did basically: try using the 3CX ISO to build a test VM.

1. Deploy the SBC instead of a full PBX (you don't have to use it, its just to complete the setup wizard and get you to the shell)
2. Skip making a key pair for now, use the root credentials instead in your 3CX Backup Location settings of your PBX
3. Try using this as a backup destination and see if the backups are completed
- Optionally create a key to use instead of a password (I also tested this and it works too)


If this works for you, then you will at least have a good system to compare to your existing backup server so you can complete the troubleshooting.

Hope this helps!