V20 Version Upgrades - Top Reasons for Aborting

[Agathocles Prodromou]

3 major reasons and other common issues for your upgrade failing.​

So after releasing beta 1 of V20 and Update 9 we’ve already had quite a number of upgrades - enough to be statistically significant. And the results are interesting. These are the top reasons for an upgrade to not start:

Number #1 - Insufficient core...​

Continue reading the Original Blog Post.​

 

[Les Webb]

Yeah the two core requirement will DOUBLE our cost on the Google Cloud platform. We have 80 VMs currently running with several in the 150+ extension range with single core machines that have ZERO problems and generally run at less than 25 per cent of CPU usage.
What is the reasoning behind the dual core requirement???

 

[N_G]

Well hosted by 3CX smallest instance has two cores, the reasons for the 2 cores are clearly written in the blog post. And 150+ extensions on a single core is crazy, you would be damaging our brand by running it with 1 core.

 

[gorbros]

Well hosted by 3CX smallest instance has two cores, the reasons for the 2 cores are clearly written in the blog post. And 150+ extensions on a single core is crazy, you would be damaging our brand by running it with 1 core.

When will the recommended hardware specs updated? https://www.3cx.com/docs/recommended-hardware-specifications-for-3cx/

 

[SteveITS]

When will the recommended hardware specs updated? https://www.3cx.com/docs/recommended-hardware-specifications-for-3cx/

“Small” is already 2 CPUs…?

https://web.archive.org/web/2023030.../recommended-hardware-specifications-for-3cx/

 

[IT Dept]

Hello,

We have on-premise 3CX instance. All system requirements are covered but still when trying to upgrade from 18.0 (Build 20) to V20, we get the error " Your 3CX Debian OS failed to update, ID: 10012".
In the email that 3CX sends we found out the following:

Fetched 15.0 MB in 4s (3,939 kB/s)
Reading package lists...
[33m====== [11:31:36] Preparation: Upgreadability status seems OK[39m
[33m====== [11:31:36] Preparation: Checking locales[39m
[33m====== [11:31:36] Workaround: Unholding packages to avoid issues during the upgrade[39m
[33m====== [11:31:36] Upgrade Result: *******************************************[39m
{
  "result": "aborted",
  "time": "29-02-2024 11:31:36",
  "title": "File ownership problem",
  "message": "There are files under /var/lib/3cxpbx/Bin/nginx/conf with the wrong user and/or group ownership. Please make sure the files are owned by the user: phonesystem"
}
[31m[11:31] aborted: File ownership problem - 0[39m

We then compared the ownership of all files in that directory with the other 3CX (Version 20.0 Update 0 (Build 1620 Release) we have and everything is the same.

Then we ran "apt-get update && apt-get upgrade" and Debian upgraded its packages successfully, but still we cannot upgrade 3CX - we get the same ownership error.

Could you please advise how we could upgrade without new instance and where to look to resolve the issue?

Thank you in advance.

 

[bitn2]

Hello,

We have on-premise 3CX instance. All system requirements are covered but still when trying to upgrade from 18.0 (Build 20) to V20, we get the error " Your 3CX Debian OS failed to update, ID: 10012".
In the email that 3CX sends we found out the follwoing:

Fetched 15.0 MB in 4s (3,939 kB/s)
Reading package lists...
[33m====== [11:31:36] Preparation: Upgreadability status seems OK[39m
[33m====== [11:31:36] Preparation: Checking locales[39m
[33m====== [11:31:36] Workaround: Unholding packages to avoid issues during the upgrade[39m
[33m====== [11:31:36] Upgrade Result: *******************************************[39m
{
"result": "aborted",
"time": "29-02-2024 11:31:36",
"title": "File ownership problem",
"message": "There are files under /var/lib/3cxpbx/Bin/nginx/conf with the wrong user and/or group ownership. Please make sure the files are owner by the user: phonesystem"
}
[31m[11:31] aborted: File ownership problem - 0[39m

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.


We then compared the ownership of all files in that directory with the other 3CX (Version 20.0 Update 0 (Build 1620 Release) we have and everything is the same.

Then we ran "apt-get update && apt-get upgrade" and Debian upgraded its packages successfully, but still we cannot upgrade 3CX - we get the same ownership error.

Could you please advise how we could upgrade without new instance and where to look to resolve the issue?

Thank you in advance.

Are there any customisations done on your 3cx? To update the pbx by cli isnt a good idea. The fastest way would be now to backup and restore with the newest image.

 

[IT Dept]

Hi bitn2,
No customisations have been made. Just simple conifgs via the GUI.
I would like to find the cause of the error and I believe it will be in positive for the 3CX team as well.
Hopefully they will join with some comments.

Thanks.

 

[bitn2]

The cause is the wrong ownership of some files. If you didnt customised anything by cli, this failure wouldnt pop up. The files must have "phonesystem" as owner. How did you start the update? Over cli or gui?

 

[Agathocles Prodromou]

Hello,

We have on-premise 3CX instance. All system requirements are covered but still when trying to upgrade from 18.0 (Build 20) to V20, we get the error " Your 3CX Debian OS failed to update, ID: 10012".
In the email that 3CX sends we found out the follwoing:

Fetched 15.0 MB in 4s (3,939 kB/s)
Reading package lists...
[33m====== [11:31:36] Preparation: Upgreadability status seems OK[39m
[33m====== [11:31:36] Preparation: Checking locales[39m
[33m====== [11:31:36] Workaround: Unholding packages to avoid issues during the upgrade[39m
[33m====== [11:31:36] Upgrade Result: *******************************************[39m
{
"result": "aborted",
"time": "29-02-2024 11:31:36",
"title": "File ownership problem",
"message": "There are files under /var/lib/3cxpbx/Bin/nginx/conf with the wrong user and/or group ownership. Please make sure the files are owner by the user: phonesystem"
}
[31m[11:31] aborted: File ownership problem - 0[39m

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.


We then compared the ownership of all files in that directory with the other 3CX (Version 20.0 Update 0 (Build 1620 Release) we have and everything is the same.

Then we ran "apt-get update && apt-get upgrade" and Debian upgraded its packages successfully, but still we cannot upgrade 3CX - we get the same ownership error.

Could you please advise how we could upgrade without new instance and where to look to resolve the issue?

Thank you in advance.

Hi,

That is typically a result of manually installing certificates in nginx (for custom FQDNs) and not changing the owner of the certificates to the user phonesystem. (or making changes to other config files).

If you run

ls -lah /var/lib/3cxpbx/Bin/nginx/conf/Instance1/

You will probably see that the certificates are owned by another user. Chown to the correct user and the upgrade will proceed.

 

[IT Dept]

Hi Agathocle,

Thank you big time!

The ownership on the both 3CX instances (v18 and v20) were the same for the certificates. The ROOT user was owner on both 3CX.
I changed the ownership of the certificates to PHONESYSTEM and it all worked like a charm! This info is worth to be added in the Original Blog Post

Thank you once again!

 

[Agathocles Prodromou]

Hi Agathocle,

Thank you big time!

The ownership on the both 3CX instances (v18 and v20) were the same for the certificates. The ROOT user was owner on both 3CX.
I changed the ownership of the certificates to PHONESYSTEM and it all worked like a charm! This info is worth to be added in the Original Blog Post

Thank you once again!

I'm glad I could help! Have a nice rest of the week and enjoy V20! : )

 

[bitn2]

This info is worth to be added in the Original Blog Post

This info is already in the mail you got after the upgrade failure. Fine you solved your problem.

 

[IT Dept]

This info is already in the mail you got after the upgrade failure. Fine you solved your problem.

It is and I read it, but when comparing two 3CX instances, one running V18 and one V20, and see that the ownership is identical you get a bit confused.
Thanks for joining the discussion.

 

[bitn2]

It is and I read it, but when comparing two 3CX instances, one running V18 and one V20, and see that the ownership is identical you get a bit confused.
Thanks for joining the discussion.

No problem, the good thing is you know what to do on the other pbx before you upgrade

 

[IT Dept]

The already upgraded 3CX (running V20) had the same ownership for the certificates like the 3CX which was refusing to upgrade.(running V18). For both, the ROOT user had the ownership of the SSL certificates.

But the already upgraded 3CX (running V20) was first upgraded to V20 RC1 and then upgraded to V20 Final, whereas the problematic 3CX was just upgraded directly to V20 Final.

So now both 3CX are running V20 but one have the ROOT user as owner of the SSL certificates and the other have the PHONESYSTEM user as owner of the SSL certificates.

 

[Colorado VoIP]

simple fix with "sudo chown phonesystemhonesystem 'filename.extension' " (no quotes on anything of course)

edit: no smiley face lol, just a colon & p next to each other

 

[IT Dept]

simple fix with "sudo chown phonesystemhonesystem 'filename.extension' " (no quotes on anything of course)

Yes, that's how it was fixed - following Agathocles' suggestion.
My point was that although one of the 3CX had the wrong ownership of the SSL certificates it was able to upgrade, the other one wasn't able to upgrade having the same ownership of SSL certificates.

 

[bitn2]

Yes, that's how it was fixed - following Agathocles' suggestion.
My point was that although one of the 3CX had the wrong ownership of the SSL certificates it was able to upgrade, the other one wasn't able to upgrade having the same ownership of SSL certificates.

Yeah, that's pretty strange...

 

[Agathocles Prodromou]

Yes, that's how it was fixed - following Agathocles' suggestion.
My point was that although one of the 3CX had the wrong ownership of the SSL certificates it was able to upgrade, the other one wasn't able to upgrade having the same ownership of SSL certificates.

The permission check was recently added to the upgrade script as part of an ongoing process to provide a trouble free migration (to the extent possible). You must have upgraded the first instance prior to the addition of this check.