Upgrade to v18 failed on Debian hosted VM with OVH

[Alphabetic]

On latest build of v16.

This build of 3CX was installed using https://www.3cx.com/community/threads/3cx-manual-installation-instructions-disappeared.76272/

I pushed the button to start the upgrade to Debian 10 v18 and left it.

A few hours later I've come to it and the mgmt console is saying server error.

Rebooted the VM and systemctl 3CX* status is showing no 3CX services.

Is this salvagable?

 

[completeware]

I have the exact same issue on OVH instance. It seems like the 3CX service have been removed, and then the upgrade is stuck in the Debian upgrade.

I am able to SSH into my VM, but it still shows Debian 9 and no 3CX service running after a while. No luck accessing the login page.

Will most likely do a instance restore from within the OVH control panel.

The installation was initially done with the PBXexpress setup.

Sorry this is not of much help at the moment.

 

[Alphabetic]

I just got off the phone with support and they got my 3CX back up and running. Im really sh*t at Debian so i'll give you what notes I managed to take... Note: this is specific to me but may work for you. You do this at your own risk. If you have 3CX Support I would go for this option first.

3CX makes a backup pre upgrade attempt in /var/lib/3cxpbx/instance1/data/backups called rescueBackupUpgrade.zip - take a copy of this.

Check if your web server is still runnnig..

systemctl list-units 3CX* ngin* postgre*

Then try..

nano /etc/apt/sources.list

Comment out everything and prepend

deb http://deb.debian.org/debian/ stretch main
deb-src http://deb.debian.org/debian/ stretch main
deb http://security.debian.org/ stretch/updates main
deb-src http://security.debian.org/ stretch/updates main
deb http://deb.debian.org/debian/ stretch-updates main
deb-src http://deb.debian.org/debian/ stretch-updates main

Save and exit

sudo apt update

Followed by..

nano /etc/apt/sources.list.d/3cxpbx.list

Change to:

#deb http://downloads-global.3cx.com/downloads/debian buster main (v10)
deb http://downloads-global.3cx.com/downloads/debian stretch main

Save and exit

sudo apt update

Reinstall 3CX

apt install 3cxpbx=16.0.8.9

This is what was done to restore my 3CX back - I hope this works for you.

 

[SteveITS]

It seems like the 3CX service have been removed

FWIW, I did one upgrade (self hosted, success) and it sent a 500 KB log file at the end. About 2 minutes in it says, "Upgrade: REACHING POINT OF NO RETURN - STARTING DEBIAN 10 UPGRADE AND REMOVING 3CX" so it does remove/reinstall 3CX. The file emailed was "debian9to10.log" if that helps you find it.

 

[Alphabetic]

FWIW, I did one upgrade (self hosted, success) and it sent a 500 KB log file at the end. About 2 minutes in it says, "Upgrade: REACHING POINT OF NO RETURN - STARTING DEBIAN 10 UPGRADE AND REMOVING 3CX" so it does remove/reinstall 3CX. The file emailed was "debian9to10.log" if that helps you find it.

It actually states this as soon as you click the upgrade button too before you confirm. I just never properly read it.

 

[ChrisAtPentasys]

Because I lost SSH access via Google Cloud Console I decided to move forward with my new instance restored from backup. After I confirmed SSH access via Putty, I ran the upgrade again. Failed again. Was able to use the steps posted by @kieferschild to get back up and running. But now what?

 

[SteveITS]

It actually states this as soon as you click the upgrade button too before you confirm. I just never properly read it.

Ha. I am always so shocked by the "immediate logout without any other notice on screen" that happens*, and guess I forgot in said shock. In fact I am pretty sure the first (minor) update I ever did I thought I was simply kicked out and I went back in to run the upgrade again. I want to say it stopped me, or stopped itself, I can't recall now.

Also FWIW, upgrading our instance took about 9-10 minutes. Not sure how much that depends on database size, disk speed and CPU power.


* vs. say, idiot-proof things like "upgrade in process, don't log in" and maybe "login button disabled during the upgrade"

 

[Alphabetic]

so

Because I lost SSH access via Google Cloud Console I decided to move forward with my new instance restored from backup. After I confirmed SSH access via Putty, I ran the upgrade again. Failed again. Was able to use the steps posted by @kieferschild to get back up and running. But now what?

did you hash out those entries and replace with the ones I specified? Try the upgrade again, check your free disk space too.

 

[ChrisC_3CX]

If having issues with the upgrade, I think the best thing to do is take a backup, move it away from the machine, completely scrap the machine and just deploy a new one straight to Debian 10 and 3CX V18 to which you can restore the backup during installation. To do this, you would have to either use the Hosting Provider's market place, which allows you to restore a backup during installation, or use the 3CX ISO.

Deploying via the Partner Portal will not allow you to restore a backup during installation so using the 3CX v16 backup in this case will not be possible. If you do wish deploy the new machine via this method, you might want to consider first deploying the machine on a local VM (say within hyper-V on a PC) using the latest 3CX ISO, and use the v16 backup during installation. Then, take a backup from the new v18 PBX and proceed with the Partner Portal deployment wizard.

 

[Alphabetic]

If having issues with the upgrade, I think the best thing to do is take a backup, move it away from the machine, completely scrap the machine and just deploy a new one straight to Debian 10 and 3CX V18 to which you can restore the backup during installation. To do this, you would have to either use the Hosting Provider's market place, which allows you to restore a backup during installation, or use the 3CX ISO.

Deploying via the Partner Portal will not allow you to restore a backup during installation so using the 3CX v16 backup in this case will not be possible. If you do wish deploy the new machine via this method, you might want to consider first deploying the machine on a local VM (say within hyper-V on a PC) using the latest 3CX ISO, and use the v16 backup during installation. Then, take a backup from the new v18 PBX and proceed with the Partner Portal deployment wizard.

Chris - The problem is that OVH doesnt let you install with an ISO, so my installs are all done using the commands which were once supported.

Going down the route of using express to create the OVH means I have to turn off MFA on my OVH account and I will also be billed for the additional VPS whilst the previous one exists.

All of these ideas are time consuming and lets not forget the gigabites of recordings that increase backup/restore time too.

For now, i'm going to be modifying the sources.list to the above and attempting the upgrades.

 

[ChrisC_3CX]

I see your point, though, if it is possible to do at some point this would be the route I recommend to keep the instances supported and to avoid similar issues in the future.

 

[Alphabetic]

I see your point, though, if it is possible to do at some point this would be the route I recommend to keep the instances supported and to avoid similar issues in the future.

Would support be interested in the 9to10.log on future failures to see if they can determine a common cause and try to mitigate it or is it (╯°□°）╯︵ ┻━┻ because its "not a supported install"

 

[ChrisC_3CX]

Would support be interested in the 9to10.log on future failures to see if they can determine a common cause and try to mitigate it or is it (╯°□°）╯︵ ┻━┻ because its "not supported"

Although it is indeed not supported and we'd still recommend using the deployment methods mentioned, please do send me the log file via a PM so that I can forward it to the appropriate department for review.

Even if an issue is caused by an unsupported configuration, if it is common across all or most cases and we are aware of the matter, we do tend to evaluate whether there's anything we can do about it or at least try to inform customers.

 

[Gustavo Lopez]

If having issues with the upgrade, I think the best thing to do is take a backup, move it away from the machine, completely scrap the machine and just deploy a new one straight to Debian 10 and 3CX V18 to which you can restore the backup during installation. To do this, you would have to either use the Hosting Provider's market place, which allows you to restore a backup during installation, or use the 3CX ISO.

Deploying via the Partner Portal will not allow you to restore a backup during installation so using the 3CX v16 backup in this case will not be possible. If you do wish deploy the new machine via this method, you might want to consider first deploying the machine on a local VM (say within hyper-V on a PC) using the latest 3CX ISO, and use the v16 backup during installation. Then, take a backup from the new v18 PBX and proceed with the Partner Portal deployment wizard. You can then restore your new v18 backup using the following guide: https://www.3cx.com/docs/manual/pbx-backup-restore-failover/

One question, really is it possible to install V18 and restore the last V16 backup? I am just about to reinstall V16 and it would be better to install V18 at once.

 

[Alphabetic]

One question, really is it possible to install V18 and restore the last V16 backup? I am just about to reinstall V16 and it would be better to install V18 at once.

I installed an OVH VM with Debian 10 and ran through the manual installation again using these commands..

wget -O- http://downloads-global.3cx.com/downloads/3cxpbx/public.key | sudo apt-key add -
echo "deb http://downloads-global.3cx.com/downloads/debian buster main" | sudo tee /etc/apt/sources.list.d/3cxpbx.list
echo "deb http://downloads-global.3cx.com/downloads/debian buster-testing main" | sudo tee /etc/apt/sources.list.d/3cxpbx-testing.list
sudo apt update
sudo apt install net-tools dphys-swapfile
sudo apt install 3cxpbx

I was able to restore my v16 backup and it converted it to v18.

 

[Gustavo Lopez]

How lucky I am! !!! I logged via SSH, ran those commands and the PBX started in V18 with the previous config !!!
All extensions are OK, admin user login is OK, the FQDN and license is OK!!!!!
apt update
apt upgrade
apt install 3cxpbx