Runaway 3CXVoiceBoxManager service

Discussion in '3CX Phone System - General' started by paragondie, Oct 10, 2007.

  1. paragondie

    Joined:
    Jun 25, 2007
    Messages:
    52
    Likes Received:
    0
    When we first turned on the 3CX system the email notifications were working fine but after a few days the system locked up. Investigation showed that 3CXVoiceBoxManager.exe was using 100% of the processor and the trace log filled the hard drive. After shutting off the service we did some testing and found that there were two voicemails that would trigger this behavior. We could find nothing unusual about these voice mails. In fact at least one of them had been on the server for two days before we had the problem. We deleted the two voice mails and restarted the service. A few hours later the same thing happened. We currently have the service disabled. The trace.log file contains these lines at the point the system runs amuck:

    09:02:54.125|0598|.\TasksProcessor.cpp(753):CTaskSendNotify::SendNotify(), Notification for the 'from 02 Oct 2007 12:00, caller 20004' message ('138' extension) has been sent.
    09:02:54.125|0598|.\PhpWrapper.cpp(381):pHPWrapper::Execute(), Process 0x344 (D:\Program Files\3CX PhoneSystem\Bin\..\Bin\Php\php-win.exe) created
    09:02:54.125|0598|.\PhpWrapper.cpp(382):pHPWrapper::Execute(), Writing 326 bytes of input data
    09:02:54.125|0598|.\PhpWrapper.cpp(394):pHPWrapper::Execute(), Input data written for 0x344 process
    09:02:54.125|0598|.\PhpWrapper.cpp(406):pHPWrapper::Execute(), Waiting for finishing of 0x344 process, timeout is 5000 ms
    09:02:54.313|0598|.\PhpWrapper.cpp(412):pHPWrapper::Execute(), Process 0x344 finished
    09:02:54.313|0598|.\PhpWrapper.cpp(431):pHPWrapper::Execute(), Waiting for data from process
    09:02:54.375|0598|.\PhpWrapper.cpp(433):pHPWrapper::Execute(), Data from process gotten
    09:02:54.547|05a0|.\VoiceBoxScanner.cpp(235):CVoiceBoxScanner::WatchFolders(), Error! Exception detected
    09:02:54.547|05a0|.\VBoxAgent.cpp(55):VBoxAgent::WatchingThread::Run(), Error of voicemail folders watching
    09:02:54.547|05a0|.\VoiceBoxScanner.cpp(235):CVoiceBoxScanner::WatchFolders(), Error! Exception detected
    09:02:54.547|05a0|.\VBoxAgent.cpp(55):VBoxAgent::WatchingThread::Run(), Error of voicemail folders watching
    09:02:54.547|05a0|.\VoiceBoxScanner.cpp(235):CVoiceBoxScanner::WatchFolders(), Error! Exception detected
    09:02:54.547|05a0|.\VBoxAgent.cpp(55):VBoxAgent::WatchingThread::Run(), Error of voicemail folders watching
    09:02:54.547|05a0|.\VoiceBoxScanner.cpp(235):CVoiceBoxScanner::WatchFolders(), Error! Exception detected
    09:02:54.547|05a0|.\VBoxAgent.cpp(55):VBoxAgent::WatchingThread::Run(), Error of voicemail folders watching

    These lasts two lines repeat continually and very rapidly causing the log file to quickly consume gigabytes of space.

    Has anyone else had this problem and has anyone found a solution?
     
  2. miraportuga

    miraportuga Member

    Joined:
    Aug 7, 2007
    Messages:
    297
    Likes Received:
    0
    Seems to be an software problem, reinstall ? :|
     
  3. paragondie

    Joined:
    Jun 25, 2007
    Messages:
    52
    Likes Received:
    0
    Did that ...

    Actually we just reinstalled the software on a new box running Virtual Iron for a virtual server, same result.
     
  4. Mirzab

    Mirzab Member

    Joined:
    Jul 22, 2007
    Messages:
    400
    Likes Received:
    0
    Would be nice if you could supply additional info regarding specs and OS of first system and current virtual. Was first install virtual also?

    Since voicemail is cause and stored as media files I would speculate your Anti-Virus software may be the culprit. Exempt 3CX folder and outbound email for testing.
     
  5. paragondie

    Joined:
    Jun 25, 2007
    Messages:
    52
    Likes Received:
    0
    First install: Dell Precision 530 (1.7GHz Xeon w/2GB RAM) running Windows XP Pro SP2
    Second install: Same system/same operating system
    Third install: Dell PowerEdge 4600 (2.4GHz Xeon w/2GB RAM) running Windows 2003 Server R2 Enterprise (trial) SP2
    Fourth install: Virtual Iron VM (Dual 2.66GHz Xeon w/4GB RAM) running Windows 2003 Server R2 Enterprise (full) SP2

    Installs one and two included anti virus. Installs three and four did not.
     
  6. LORD ORION

    Joined:
    Jul 23, 2007
    Messages:
    39
    Likes Received:
    0
    What about sharing and security on the folder?

    Did you install 3CX using admin priviledges?

    If you are not logged into the account that you used to install 3CX, does the user have permission to acces the folder? Can enough concurrent users be connected when considering sharing?

    This happens on other software when a temp file is created during recording and then finalaized and written to another folder.
     
  7. paragondie

    Joined:
    Jun 25, 2007
    Messages:
    52
    Likes Received:
    0
    3CX was installed using the Administrator account. Permissions on those folders are set by the 3CX voice mail system as it creates them. No sharing is turned on for the system. The users only have access to their voice mail through the voice mail system.

    As mentioned above, if we delete the voice mail that's there the system will work fine for a while but eventually fails again. We thought it might be the quantity of voice mail but we narrowed it down to a specific voice mail message that crashed it the first time. If we moved that file out of the voice mail folders the service would start back up fine. If we moved all other voice mail and put that one file back the service would jump immediately to 100% CPU as soon as it was started so it's not an issue with the quantity of voice mail or voice mail folders.
     
  8. LORD ORION

    Joined:
    Jul 23, 2007
    Messages:
    39
    Likes Received:
    0
    Agreed,

    But I have seen it more then once on other software recording to a temp file fail to finalize and write, connect concurrently and get stuck forever.

    eg: SQL write BLOB or Virtual Machine timing problems in telephony apps, which is why I mentioned it. Typically it is sovled by allowing as many concurrent connections to the folder as you have potential simultaneous extensions in use, and or by simply allowing everyone write access to the folder in security to verify that is the problem. (especially true in MSS)

    What is interesting is that the system gets gummed up when you copy the file back over. :)

    I wonder what is so special about it.
    eg: Not finalized? Corrupted?
     
  9. paragondie

    Joined:
    Jun 25, 2007
    Messages:
    52
    Likes Received:
    0
    Have done some more testing. Turns out it's the number of folders that's the problem. It doesn't matter if there are messages in the folder or not. If there are more than 63 folders in the Extensions folder then the service begins generating errors and takes over whatever processor it's running on. The only solution at that point is to stop the service, reduce the number of folders to 63 or less and restart the service. Is there a limit to the number of voice mailboxes that 3CX supports? We could find no such limit in the documentation. We have nearly 100 at this point but we only run into this problem once the actual folders are created. As a work around we are running a script that deletes empty folders periodically but it's only a matter of time before there are more than 63 folders that actually have voice mail in them.
     
  10. RobLloyd

    RobLloyd Member

    Joined:
    Oct 13, 2006
    Messages:
    488
    Likes Received:
    0
    That sounds like a bug. Good catch on that.
     
  11. Alexander

    Joined:
    Nov 10, 2006
    Messages:
    73
    Likes Received:
    0
    Hi to All !

    If somebody wants to test patch please send me (alexander@3cx.com) a message. I'll send the patch as a response.

    Thanks
    Alexander
     
  12. paragondie

    Joined:
    Jun 25, 2007
    Messages:
    52
    Likes Received:
    0
    Is this patch included in v4 that has just been released or will we need the fix to load on top of v4?
     
  13. Alexander

    Joined:
    Nov 10, 2006
    Messages:
    73
    Likes Received:
    0
    Hi !

    This patch is included to version 4. But for now version 4 exists only as alpha. So if somebody wants to postpone installation of version 4, then he can use patch.

    Thanks
    Alexander
     
  14. paragondie

    Joined:
    Jun 25, 2007
    Messages:
    52
    Likes Received:
    0
    Thank you Alexander. We are going to load alpha on a test box and do some testing.
     

Share This Page