FTP Scheduled Backup Fail

[M.Zanetti]

Hello everybody.
we manage several 3CX installation, both on premises and cloud (AWS Lightsail mainly).
All these 3CX have a scheduled backup FTP to our business server, all are using the same account for authentication and have their own backup directory.
Since a couple of months, some 3CX are regularly failing the scheduled backup, but manual backup always works. I tried to change scheduled hours, but nothing changed.
File transfer process works perfectly: on FTP destination appears 3CXScheduledBackup.zip.tmp and renaming it without.tmp extension seems to be ok, but i keep receiving Backup failed notification emails.

All installations are up to date.

Thanks.

 

[NickD_3CX]

Hi!

What 3CX version are you running?
Also could you paste the error you are getting in the email?

 

[AWS2P]

Which FTP server do you use ?

 

[M.Zanetti]

What 3CX version are you running?
Also could you paste the error you are getting in the email?

3CX Version is:16.0.1078

This is the error I'm getting after every failed backup:

Your 3CX backup failed due to the following reason(s):
FluentFTP.FtpCommandException: Internal error renaming the file
at FluentFTP.FtpClient.Rename(String path, String dest)
at _3cxFilesystemImplementations.FTP.FtpFilesystem.Move(String filename1, String filename2)
at BackupRestoreSchedule.BackupJob.Execute(IJobExecutionContext context)

Which FTP server do you use ?

FileZilla on Windows 2012.

This is what I've got after a (successfully) manual backup of the same 3CX:

A backup of your PBX at zinoxlaser.my3cx.it has been successfully completed. No further action is required.
Backup name: gennaio2021.zip
Backup Path: ftp://XXXXXXX/Zinox/gennaio2021.zip

Thank you.

 

[AWS2P]

is it same ftp user used on working backup and failed ones?

 

[M.Zanetti]

is it same ftp user used on working backup and failed ones?

Yes: I haven't changed anything.

 

[NickD_3CX]

Yes, but you said initially "Since a couple of months, some 3CX are regularly failing the scheduled backup".

What does this mean? Some 3CX Servers are failing and some others are not? The ones that are failing, sometimes fail and sometimes not?

 

[M.Zanetti]

Yes, but you said initially "Since a couple of months, some 3CX are regularly failing the scheduled backup".

What does this mean? Some 3CX Servers are failing and some others are not? The ones that are failing, sometimes fail and sometimes not?

There are some servers that keeps failing every daily backup, and there are some others that fail some days.

 

[NickD_3CX]

What happens if you use the FTP User of a PBX that succeeds on another that fails? I am just thinking maybe it is a user rights thing.

Also, so you happen to have all the Backups scheduled and running at the same time? Could it be that your FTP server has a concurrent limit when they all start at the same time?

 

[M.Zanetti]

FTP user is the same for all PBXs: I've got some firewall rules that only allow FTP connection from an IP whitelist.
There shouldn't be connections limits, I've set filezilla to allow unlimited connections for that user.
I also tried to change backup scheduled time, but seems that nothing change.

 

[NickD_3CX]

The only thing I can think to suggest is to check the FTP Server logs in case this gives you some clue as to why some 3CX installs are failing.

 

[M.Zanetti]

I will keep investigating this trouble; thank you all for your help.

 

[M.Zanetti]

After doing some tests I find out that some PBX are able to complete their backup simply changing scheduled time, but there is still one that keeps failing.
The strange thing about this is that scheduled backup size are overly huge: over 500MB when manual backups are about 40MB.
Scheduled backup options are: License key and Custom templates, but i can disable custom template, since there isn't any on this 3CX.
I'm changing this option and scheduling a new backup in a few hours: I'm gonna let you know if there are any news.