compromised 3CX => calls to other countries

[CTK_Fabian]

Hi all,

briefly about us we are a system house and run about 40 3CX-phonesystems at our customers with different installations. ( Windows / Linux / Cloud )

In 5 systems, we had in recent weeks various external calls to foreign numbers ( eg countries in Africa / etc. ) from different extensions.

Currently we have done the following:
- disabled complete console access from external
- changed extension passwords
- changed 3CX tunnel password
- changed root passwords / admin passwords on operating system
- changed backup passwords

According to 3CX support the admin passwords were lost, but we think this is unlikely as it happened to 5 customers almost at the same time.
The passwords were all automatically generated and had at least 20 characters. In our documentation they are protected by MFA and are stored in Keepass databases.

Has anyone had similar experiences ?
Could the admin passwords have been lost before the last security update and are only now being exploited ? ( the security updates were installed promptly )

Many greetings

Translated with www.DeepL.com/Translator (free version)

 

[Alphabetic]

Seems a bit suspicious - Does this mean they logged in as admin, enabled the country codes and made the calls?

What do the audit logs state?

Where do you store your backups?

Are your backups encrypted because if not, you can get the admin password from this.

 

[CTK_Fabian]

Seems a bit suspicious - Does this mean they logged in as admin, enabled the country codes and made the calls?

What do the audit logs state?

Where do you store your backups?

Are your backups encrypted because if not, you can get the admin password from this.

Yes there was a registration at the management, but per system only 1 time. The country codes were probably already activated.

In the audit logs, only the user login to the management console was recorded. ( 197.52.108.227 - ISP:Telecom Egypt - for example )
There were no failed logins or blocked IP address at that time.

The backups are stored locally on the system and the complete system is then backed up again via Veeam. Although a 3CX Cloud phonesystem is also among the affected.

In 2 systems I know that the backup was not encrypted, but then you should have had access to the local system.

All systems were up to date and only the appropriate ports were accessible externally.

How would the attacker have accessed the local system? There were no other signs or symptoms.

Our assumption was that the access data was stolen before the last security update and is now being exploited.

Therefore, here is my question whether more people were affected.

 

[Pierre_3CX]

Could the admin passwords have been lost before the last security update and are only now being exploited ? ( the security updates were installed promptly )

Hi, the vulnerability you are referring to affected Windows installations only so you can rule this out as you mentioned having various environments and different passwords.

I suppose you should look for the common point between these installs, i.e had they a common backup repository? common staff handling them? perhaps one of the machines with the passwords list/documentation was compromised?

 

[Alphabetic]

Really you need to find how it happened, otherwise you could end up with more breaches.

You need to look to impliment the IP filtering for mgmt console, making sure all 3CX are up to date.

You also need to create a timeline of what happened so you get a better picture.

Who made the calls? From what IP did they log in to the mgmt console? did someones welcome email get intercepted?

You should be asking keepass if your data was stolen as I found this:




It cant have been a brute force attack otherwise the IP would have been blacklisted (depending on your settings).

so, someone knew the password.

 

[epodworksnet]

If you regularly close the Keepass app then it is kind of hard-to-get data out without insider knowledge.

I had this happen to me as well. I could never figure out the source of the intrusion. In our case the hackers were taking over extensions that had no physical telephones assigned. Not sure why this was important.

First, it all about trust -- don't trust any of the OSes at this point. I changed all of the VoiceMail PINs to 7 digits random numbers and made sure they were not duplicated between extensions and changed the extension passwords to be longer as well. I also IP restricted Web and Tunnel for some time as I could do that given my install. I also firewalled the SIP IPs from our SIP Trunk Provider to make sure some rando SIP could not connect. Again, some of these limitations may not work depending on your use case.

 

[CTK_Fabian]

Really you need to find how it happened, otherwise you could end up with more breaches.

You need to look to impliment the IP filtering for mgmt console, making sure all 3CX are up to date.

You also need to create a timeline of what happened so you get a better picture.

Who made the calls? From what IP did they log in to the mgmt console? did someones welcome email get intercepted?

You should be asking keepass if your data was stolen as I found this:


View attachment 30683

It cant have been a brute force attack otherwise the IP would have been blacklisted (depending on your settings).

so, someone knew the password.

thanks for your feedback

The console accesses from external are already disabled and we have a sequence of events.

I don't think it is realistic to intercept the welcome mails as they are different 3CX telephony systems.

That the data was stolen from the different Keepass files is also very unlikely. Each client has its own file with its own keyfile, all our clients + servers are protected with Sophos Intercept X Advcanced. We have no other anomalies with other installations or systems. ( in the Keepass files not only the passwords of the 3CX were stored / partly the Keepass files are located in different drives ( security areas ) on our server )

Since our changes nothing has happened so far, we just wanted an answer how this could happen....

 

[CTK_Fabian]

If you regularly close the Keepass app then it is kind of hard-to-get data out without insider knowledge.

I had this happen to me as well. I could never figure out the source of the intrusion. In our case the hackers were taking over extensions that had no physical telephones assigned. Not sure why this was important.

First, it all about trust -- don't trust any of the OSes at this point. I changed all of the VoiceMail PINs to 7 digits random numbers and made sure they were not duplicated between extensions and changed the extension passwords to be longer as well. I also IP restricted Web and Tunnel for some time as I could do that given my install. I also firewalled the SIP IPs from our SIP Trunk Provider to make sure some rando SIP could not connect. Again, some of these limitations may not work depending on your use case.

When did it occur in your case? In the last few weeks or has it been longer?

That it comes from the Keepass databases I also think unlikely...

In our case, several extensions were affected, some of them also directly the SIP trunk.

We have completely changed all passwords of the affected systems. ( also e.g. of the SIP Trunk etc. )
Since we have done this, nothing more has happened. Only I did not want to be satisfied with the answer that we have lost the password somewhere.