Missing Call Logs & Call Recordings

[KonnectCraft]

Hello everyone,

I have been dealing with an issue with one of my clients with missing call logs & call recordings on a Cloud hosted PBX. I just wanted to see if there's anyone else who's had experience trying to resolve this issue. I've had this issue for about 2 weeks now. Once the system reboots, it works for a brief period but it randomly stops working again. Already been in contact with 3CX support since about 10 days but they haven't been able to get to the bottom of this as of yet.

• Hardware requirements meets/exceeds the install size
• Dedicated AWS LightSail Instance. No other applications or services running beside 3CX
• Have a mix of supported/unsupported SIP Trunk Providers but when the call stops logging/recording, its not even logging internal calls between extensions.
• Latest stable updates installed with no updates pending except for V20
• Call logs missing on 3CX Mobile Apps (Android/iOS/Windows Client/ Management - Call Logs / Recordings) but when analyzing a dump from Wireshark pcap file, you can see all the call logs. It's just not visible on UI/UX of the user apps & management console.

If anyone has any experience dealing with issue similar to this, please advise what resolved your situation.

 

[Guillaume Bourgeois]

Hello,

If the 3CX support team is already involved with the ticket, please refrain from doing anything that might slow them down. If it's taking a while, it's likely because they are conducting an investigation. Have you provided them with the support info .zip file? It's highly unlikely that we can offer a solution they haven't already provided, as they have access to the support info logs, which we do not.

Personally, I have encountered a similar issue before. However, I'm not saying that you're facing the same problem. Symptoms may appear similar but might not stem from the same issue. In my case, it was the database that became corrupted due to not adhering to 3CX support conditions. It wasn't a large system, so I simply exported the user list, started fresh in a new VM with a new setup, and manually reconfigured everything. Only the user list that I had exported as a CSV was re-imported, the rest was set up from scratch.

That being said, I'm not suggesting that you're dealing with the same issue.

Are you sure that no one has attempted to access the 3CX database? When you restart, are you able to perform a complete backup? If so, does the 3CX support team agree with the idea of restoring this backup onto a new, fresh VM?

Before taking any action, please seek advice from the 3CX support team through your ticket. I would not want to interfere with their work.

If you decide to proceed regardless, I recommend cloning the hard drive and running it on a new VM. Make sure to shut down the current VM under investigation by 3CX beforehand.

As I mentioned, any attempt could slow down the process or, worse, exacerbate the situation.

 

[KonnectCraft]

Hello,

If the 3CX support team is already involved with the ticket, please refrain from doing anything that might slow them down. If it's taking a while, it's likely because they are conducting an investigation. Have you provided them with the support info .zip file? It's highly unlikely that we can offer a solution they haven't already provided, as they have access to the support info logs, which we do not.

Personally, I have encountered a similar issue before. However, I'm not saying that you're facing the same problem. Symptoms may appear similar but might not stem from the same issue. In my case, it was the database that became corrupted due to not adhering to 3CX support conditions. It wasn't a large system, so I simply exported the user list, started fresh in a new VM with a new setup, and manually reconfigured everything. Only the user list that I had exported as a CSV was re-imported, the rest was set up from scratch.

That being said, I'm not suggesting that you're dealing with the same issue.

Are you sure that no one has attempted to access the 3CX database? When you restart, are you able to perform a complete backup? If so, does the 3CX support team agree with the idea of restoring this backup onto a new, fresh VM?

Before taking any action, please seek advice from the 3CX support team through your ticket. I would not want to interfere with their work.

If you decide to proceed regardless, I recommend cloning the hard drive and running it on a new VM. Make sure to shut down the current VM under investigation by 3CX beforehand.

As I mentioned, any attempt could slow down the process or, worse, exacerbate the situation.

I;ve already given the support package along with the system owner access to PBX along with root SSH access to the VM about 6 days ago. I am not trying to interfere with the support process but to see if 3CX community has faced similar issue in the past to see if I can assist the support team in any way as my client is constantly reaching out to me and this has become a huge problem to them. They roughly have 100 users affected by this call on their daily reporting stats & call recording compliance.

 

[Guillaume Bourgeois]

So, my suggestion, if 3CX support agrees, is that you perform a full backup of the 3CX system, export the backup, shut down the VM (don’t delete) , create a new VM, and restore your 3CX backup in a new environment.

If the problem persists, it will indicate that there is likely a “non-standard character” causing the issue ?

Let me know, I'm curious.

 

[VoIPTools]

If it was just recordings I would be looking at the recording quota. But no call logs? What does that mean?

Six days is a long time to wait. I would be inclined to backup and restore to a separate VM as suggested by @Guillaume Bourgeois.

However, I too am a little curious whether anyone has been connecting directly to the 3CX PostgreSQL database. Many times people do not realize that simply clicking on a table name in the pgAdmin tool can lock records in the database, and if 3CX cannot write to the database, even briefly, bad things will happen.

Did I miss whether this is V18 or V20? Service Pack number?

 

[KonnectCraft]

That is what is boggling my mind as well. There's no logs on reports, management portal - call logs/reports/recordings, mobile apps, or windows clients. It seems like the calls were never made. V18 with no pending updates except for the V20 upgrade. 3CX Support seems to guess this could be related to CDR. I have disabled the CDR & rebooted the PBX. Will keep an eye out to see if the glitch re-appears again.