3cx backup Failed

[Clinton97]

Hi we have a 3cx v18 update 9 build 31 instance onprem and we are experiencing the below error when backups are run on the system. We have found that when browsing the recordings page and call logs from the management console that the same error "58030: could not read block 4991 in file "base/16387/17310": Input/output error" occurs when browsing to a specific time frame, any thing before or after that time window works fine.

We have removed the recording files off of the system manually and we have purged the call logs, however the same error still occurs when browsing this time window.

Any one else have this issue before and was able to resolve it ?

3cx Backup failed error notificaton

Your 3CX backup failed due to the following reason(s):

System.AggregateException: One or more errors occurred. (NpgsqlTransaction) ---> System.ObjectDisposedException: NpgsqlTransaction ---> Npgsql.PostgresException (0x80004005): 58030: could not read block 4991 in file "base/16387/17310": Input/output error

at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong|233_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)

at Npgsql.NpgsqlRawCopyStream.ReadCore(Int32 count, Boolean async, CancellationToken cancellationToken)

at Npgsql.NpgsqlRawCopyStream.<>c__DisplayClass38_0.<<ReadAsync>g__ReadAsyncInternal|0>d.MoveNext()

--- End of stack trace from previous location ---

at System.IO.Stream.<CopyToAsync>g__Core|29_0(Stream source, Stream destination, Int32 bufferSize, CancellationToken cancellationToken)

at _3cxNetInterfaces.Implementation.Npgsql.NpgsqlDbOperation.BackupTableAsync(String fromTable, Stream outputStream, DbTransaction transactionObject)

Exception data:

Severity: ERROR

SqlState: 58030

MessageText: could not read block 4991 in file "base/16387/17310": Input/output error

File: md.c

Line: 810

Routine: mdread

--- End of inner exception stack trace ---

at Npgsql.NpgsqlTransaction.CheckDisposed()

at Npgsql.NpgsqlTransaction.get_Connection()

at _3cxNetInterfaces.Implementation.Npgsql.NpgsqlDbOperation.CommitAsync(DbTransaction transactionObject)

--- End of inner exception stack trace ---

at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)

at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)

at _3cxNetBackupAndRestore.BackupTasks.BaseDBTask.RunAll(BaseTask ignore)

 

[Alphabetic]

What OS is this?

 

[Clinton97]

OS in use is Debian 10

 

[Alphabetic]

Physical disks? Have you tried to database maintenance from the dashboard?

 

[Clinton97]

Physical disks? Have you tried to database maintenance from the dashboard?

Physical disc/HDD in the onprem server, backups have been functioning normally until a couple days ago when the error started.

Kindly advise on the "database maintenance from the dashboard" which is being referred to. We have purged all of the call logs from the system and the system is on the latest v18 update 9 (build 31) release presently.

We have changed directory for the recordings and the location that recordings are being stored in and the error still remains, whether the backup location is local on the server or being conducted via SFTP to an offsite location.

 

[Alphabetic]

Sorry, thought you were on v20.

have you checked the state of the disks?

You might have to use the fsck command to check the file system is intact.. you may even have to restore.

 

[ivank]

We have changed directory for the recordings and the location that recordings are being stored in and the error still remains, whether the backup location is local on the server or being conducted via SFTP to an offsite location.

The backup task cannot physically read a block of data from the disk, changing the destination will not help at all. Your database is most probably also corrupted because of this.
You need to check your disk for integrity, probably replace it with a new one and restore a valid recent backup.

 

[Clinton97]

Hi all thank you for the help, We conducted an fsck check to ensure disk integrity and check completed successfully. Attempted to run another backup with the same error still occurring, leading to the issue being data base corruption as the files in the time frame the issue occurs in works perfectly fine when browsing to them and downloading the recordings via WinSCP.

We concluded that the issue is data base corruption and we moved the locally stored recordings out from the default location in the 3cxpbx folder to the Debian root to ensure that the recordings are not purged when resetting 3cx. We then conducted a wipe/refresh/clean install of 3cx using the the command line to reset the 3cx instance and not reset the whole OS. Restored latest successful backup and then moved the recordings from the Debian root back into the 3cx default recordings location.

This post can now be marked as solved thanks.