502 Bad Gateway

Discussion in '3CX Phone System - General' started by JST, Apr 19, 2017.

Thread Status:
Not open for further replies.
  1. JST

    JST New Member

    Joined:
    Jan 8, 2017
    Messages:
    107
    Likes Received:
    1
    Our system suddenly shows a popup with 502 Bad Gateway and it is no longer possible to logon to the web admin console. The system runs on VMware with one assigned NIC and there has been no change to the system (saw the other posts about having two NICs).

    Any suggestions how we can restore web admin access?
     
  2. YiannisH_3CX

    YiannisH_3CX Support Team
    Staff Member 3CX Support

    Joined:
    May 10, 2016
    Messages:
    6,326
    Likes Received:
    448
    Hello @JST

    Please make sure that all services are running as this sounds like your management console service is down.
    Please check and let us know
     
  3. JST

    JST New Member

    Joined:
    Jan 8, 2017
    Messages:
    107
    Likes Received:
    1
    That makes sense, but how can I do that on Debian?
     
  4. YiannisH_3CX

    YiannisH_3CX Support Team
    Staff Member 3CX Support

    Joined:
    May 10, 2016
    Messages:
    6,326
    Likes Received:
    448
    You need SSH access. Once you have access you can check the services by typing service 3CX* status. If the management console is indeed down you can start it by typing service 3CXPhoneSystemMC01 start
     
  5. JST

    JST New Member

    Joined:
    Jan 8, 2017
    Messages:
    107
    Likes Received:
    1
    I am getting the following output:

    root@pbx:~# service 3CX* status
    ● 3CXQueueManager01.service - 3CX PhoneSystem 01 Queue Manager Server
    Loaded: loaded (/lib/systemd/system/3CXQueueManager01.service; enabled)
    Active: active (running) since Thu 2017-04-20 07:07:06 PDT; 3h 1min ago
    Main PID: 721 (TcxQMsrv)
    CGroup: /system.slice/3CXQueueManager01.service
    └─721 /usr/lib/3cxpbx/TcxQMsrv -p /var/run/phonesystem/3CXQueueMan...

    Apr 20 07:07:06 pbx systemd[1]: Started 3CX PhoneSystem 01 Queue Manager Server.

    ● 3CXTunnel01.service - 3CX PhoneSystem 01 SIP/RTP Tunneling Proxy
    Loaded: loaded (/lib/systemd/system/3CXTunnel01.service; enabled)
    Active: active (running) since Thu 2017-04-20 07:06:57 PDT; 3h 1min ago
    Main PID: 560 (3CXTunnel)
    CGroup: /system.slice/3CXTunnel01.service
    └─560 /usr/sbin/3CXTunnel -n -p /var/run/phonesystem/3CXTunnel01.p...

    Apr 20 07:06:55 pbx systemd[1]: Starting 3CX PhoneSystem 01 SIP/RTP Tunneli.....
    Apr 20 07:06:57 pbx systemd[1]: Started 3CX PhoneSystem 01 SIP/RTP Tunnelin...y.

    ● 3CXPhoneSystem01.service - 3CX PhoneSystem 01 SIP Server
    Loaded: loaded (/lib/systemd/system/3CXPhoneSystem01.service; enabled)
    Active: active (running) since Thu 2017-04-20 07:07:03 PDT; 3h 1min ago
    Main PID: 700 (3CXPhoneSystem)
    CGroup: /system.slice/3CXPhoneSystem01.service
    └─700 /usr/sbin/3CXPhoneSystem -p /var/run/phonesystem/3CXPhoneSys...

    Apr 20 07:07:03 pbx systemd[1]: Started 3CX PhoneSystem 01 SIP Server.

    ● 3CXIVR01.service - 3CX PhoneSystem 01 IVR Server
    Loaded: loaded (/lib/systemd/system/3CXIVR01.service; enabled)
    Active: active (running) since Thu 2017-04-20 07:07:03 PDT; 3h 1min ago
    Main PID: 722 (3CXIVR)
    CGroup: /system.slice/3CXIVR01.service
    └─722 /usr/sbin/3CXIVR -n -p /var/run/phonesystem/3CXIVR01.pid -c ...

    Apr 20 07:07:03 pbx systemd[1]: Started 3CX PhoneSystem 01 IVR Server.

    ● 3CXCfgServ01.service - 3CX PhoneSystem Configuration Service
    Loaded: loaded (/lib/systemd/system/3CXCfgServ01.service; enabled)
    Active: active (running) since Thu 2017-04-20 07:07:02 PDT; 3h 1min ago
    Main PID: 689 (3CXSLDBServ)
    CGroup: /system.slice/3CXCfgServ01.service
    └─689 /usr/sbin/3CXSLDBServ -n -p /var/run/phonesystem/3CXCfgServ0...

    Apr 20 07:07:02 pbx systemd[1]: Started 3CX PhoneSystem Configuration Service.

    ● 3CXPhoneSystemMC01.service - 3CX PhoneSystem 01 Management Console
    Loaded: loaded (/lib/systemd/system/3CXPhoneSystemMC01.service; enabled)
    Active: failed (Result: core-dump) since Thu 2017-04-20 07:07:05 PDT; 3h 1min ago
    Process: 776 ExecStopPost=/bin/rm -f /var/run/phonesystem/management_console01.socket (code=exited, status=0/SUCCESS)
    Process: 774 ExecStopPost=/bin/rm -f /var/run/phonesystem/3CXPhoneSystemMC01.pid (code=exited, status=0/SUCCESS)
    Process: 701 ExecStart=/usr/lib/3cxpbx/ManagementConsoleJS --pid /var/run/phonesystem/3CXPhoneSystemMC01.pid -p /var/lib/3cxpbx/Instance1/Data/Http/webroot (code=dumped, signal=ABRT)
    Main PID: 701 (code=dumped, signal=ABRT)

    Apr 20 07:07:05 pbx systemd[1]: 3CXPhoneSystemMC01.service: main process ex...RT
    Apr 20 07:07:05 pbx systemd[1]: Failed to start 3CX PhoneSystem 01 Manageme...e.
    Apr 20 07:07:05 pbx systemd[1]: Unit 3CXPhoneSystemMC01.service entered fai...e.

    ● 3CXMediaServer.service - 3CX PhoneSystem Media Server
    Loaded: loaded (/lib/systemd/system/3CXMediaServer.service; enabled)
    Active: active (running) since Thu 2017-04-20 07:06:57 PDT; 3h 1min ago
    Main PID: 567 (3CXMediaServer)
    CGroup: /system.slice/3CXMediaServer.service
    └─567 /usr/sbin/3CXMediaServer -n -p /var/run/phonesystem/3CXMedia...

    Apr 20 07:06:57 pbx systemd[1]: Started 3CX PhoneSystem Media Server.

    ● 3CXSystemService01.service - 3CX PhoneSystem 01 System Server
    Loaded: loaded (/lib/systemd/system/3CXSystemService01.service; enabled)
    Active: failed (Result: signal) since Thu 2017-04-20 07:07:06 PDT; 3h 1min ago
    Process: 803 ExecStopPost=/bin/rm -f /var/run/phonesystem/3CXSystemService01.pid (code=exited, status=0/SUCCESS)
    Process: 702 ExecStart=/usr/lib/3cxpbx/3cxSystemService -p /var/run/phonesystem/3CXSystemService01.pid /var/lib/3cxpbx/Instance1/Bin (code=killed, signal=ABRT)
    Main PID: 702 (code=killed, signal=ABRT)

    Apr 20 07:07:05 pbx systemd[1]: Started 3CX PhoneSystem 01 System Server.
    Apr 20 07:07:06 pbx systemd[1]: 3CXSystemService01.service: main process ex...RT
    Apr 20 07:07:06 pbx systemd[1]: Unit 3CXSystemService01.service entered fai...e.

    ● 3CXEventNotificationManager.service - 3CX Event Notification Manager
    Loaded: loaded (/lib/systemd/system/3CXEventNotificationManager.service; enabled)
    Active: active (running) since Thu 2017-04-20 07:07:02 PDT; 3h 1min ago
    Main PID: 568 (CloudServicesWa)
    CGroup: /system.slice/3CXEventNotificationManager.service
    └─568 /usr/lib/3cxpbx/CloudServicesWatcher -p /var/run/phonesystem...

    Apr 20 07:07:02 pbx systemd[1]: Started 3CX Event Notification Manager.
    Hint: Some lines were ellipsized, use -l to show in full.


    It seems that two services aren't working properly:
    3CX Phone System 01 Management Console
    3CX Phone System 01 System Server

    A restart fails with the message shown above.

    I am really puzzled how this can happen when no changes were applied to the system.

    What is our best recovery option now?
     
  6. YiannisH_3CX

    YiannisH_3CX Support Team
    Staff Member 3CX Support

    Joined:
    May 10, 2016
    Messages:
    6,326
    Likes Received:
    448
    I have sent you a p.m.
     
  7. Barnabis

    Joined:
    May 28, 2017
    Messages:
    2
    Likes Received:
    0
    Hello,

    I ve the same problem.

    i have increased the timeout in systemd and two services didn't start. I try to reboot the server but nothing ... Below the result of the command when I try to start 3CXPhoneSystemMC01

    May 28 14:19:51 sl0001 systemd[1]: Starting 3CX PhoneSystem Configuration Service...
    -- Subject: Unit 3CXCfgServ01.service has begun with start-up
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    --
    -- Unit 3CXCfgServ01.service has begun starting up.
    May 28 14:20:16 sl0001 systemd[1]: Started 3CX PhoneSystem Configuration Service.
    -- Subject: Unit 3CXCfgServ01.service has finished start-up
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    --
    -- Unit 3CXCfgServ01.service has finished starting up.
    --
    -- The start-up result is done.
    May 28 14:20:16 sl0001 systemd[1]: Starting 3CX PhoneSystem 01 Management Console...
    -- Subject: Unit 3CXPhoneSystemMC01.service has begun with start-up
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    --
    -- Unit 3CXPhoneSystemMC01.service has begun starting up.
    May 28 14:22:16 sl0001 systemd[1]: 3CXPhoneSystemMC01.service start operation timed out. Terminating.
    May 28 14:22:16 sl0001 systemd[1]: Failed to start 3CX PhoneSystem 01 Management Console.
    -- Subject: Unit 3CXPhoneSystemMC01.service has failed
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    --
    -- Unit 3CXPhoneSystemMC01.service has failed.
    --
    -- The result is failed.
    May 28 14:22:16 sl0001 systemd[1]: Unit 3CXPhoneSystemMC01.service entered failed state.

    When I launch the command with straw (see above) the response is continuously :

    Connecting to the PBX... Check config.json!

    Can you help me ?

    Best regards,

    Olivier
     
  8. JST

    JST New Member

    Joined:
    Jan 8, 2017
    Messages:
    107
    Likes Received:
    1
    Hi Olivier,

    In my case, support did a remote session to my system and found that there was an issue with license activation. Unfortunately, they were unable to resolve that and so I ended up with a re-installation. While I didn't have a configuration backup, my home only has 4 extensions and so I didn't loose too much configuration.

    Lesson learned: Use auto backup on a daily basis!

    So, if you have a working backup, it might be faster to do a re-installation because I don't think that support will be reading your question before Monday. If you want to wait on them, you should either way be present during the remote session (European time) or setup VPN/SSH access (that's what I did).

    Good luck!

    Cheers,
    Jens
     
Thread Status:
Not open for further replies.