Voicemail Issues

[SeanBnd]

I setup a new 3CX phone system a couple weeks ago and everything seems to be working great except this new issue I have never dealt with before.

Here is what the issue is:

Someone calls an extension, via being transferred to (external call) or internally, and leaves a voice message.
The owner of the extension is now able to access the voicemail in three different ways: Email attachment, softphone (cell or computer), and their desk phone.
Email and softphone complete voicemail recording is there.
Desk phone voicemail, blank.

Looking over the system log we found the following error:

2018/10/07 20:22:50.330|1880|0003|Erro|VMMonitor: Error parsing voicemail info file 'C:\ProgramData\3CX\Instance1\Data\Ivr\Voicemail\Extensions\2532\vmail_************(info removed for privacy).xml'
2018/10/07 20:22:51.333|1880|0003|Excpt|System.Xml.XmlException: ' ' is an unexpected token. The expected token is ';'. Line 3, position 103.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos)
   at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars)
   at System.Xml.XmlTextReaderImpl.FinishPartialValue()
   at System.Xml.XmlTextReaderImpl.get_Value()
   at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r)
   at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r, LoadOptions o)
   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(Stream stream, LoadOptions options)
   at VBMutils.VoiceMailInfo.LoadInfo()

I'm still gathering more details on the issue with the in-house IT the company has but I figured I would throw it out here for any input while I investigate more.

They use Yealink T48s

Thanks!

 

[JayDub]

This sounds like a provisioning related error as the Voicemail function itself is working but only the IP Phone is having the problem. Have you attempted to re-provision the affected devices and do you recall making any advanced configuration changes in the Yealink Web Interface?

 

[SeanBnd]

As far as I know no setting changes have been made to the actual phone via the phone's web interface. We are going to reset the phone tied to one of the extensions having the issue and start with a fresh provision to see if the issue persists.

 

[JayDub]

That is what I would recommend. I work with Yealink devices day in and out and have never had any issues like that come up. I personally have a T48S on my desk right now.

 

[cobaltit]

Not sure what @JayDub is looking at but that doesn't look like a provisioning issue at all. That looks like a corrupted XML file. A provisioning issue would be something not making it into the phone which that error doesn't seem to point at.

Is this specific to that one voicemail message, the one extension, or any extension on the system?

 

[SeanBnd]

I'm currently investigating that now @cobaltit. From whatI have gathered so far it's effect multiple user extensions.

I also just found out that a user had an email come in today but was timestamped for Friday at 11:07am. Looking at the logs I had in the original post it seems this issue is tied to it and not the original issue. So two different issue both with voicemail.

2018/10/07 20:22:50.330|1880|0003|Erro|VMMonitor: Error parsing voicemail info file 'C:\ProgramData\3CX\Instance1\Data\Ivr\Voicemail\Extensions\2532\vmail_************(info removed for privacy).xml'
2018/10/07 20:22:51.333|1880|0003|Excpt|System.Xml.XmlException: ' ' is an unexpected token. The expected token is ';'. Line 3, position 103.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos)
   at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars)
   at System.Xml.XmlTextReaderImpl.FinishPartialValue()
   at System.Xml.XmlTextReaderImpl.get_Value()
   at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r)
   at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r, LoadOptions o)
   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(Stream stream, LoadOptions options)
   at VBMutils.VoiceMailInfo.LoadInfo()
2018/10/07 20:22:51.536|1880|0003|Erro|VMMonitor: Giving up, Voicemail vmail_2578_2585_20180924154637 to 2585 Could not be delivered for 72 hours.
2018/10/07 20:23:51.755|1880|0003|Erro|VMMonitor: Error parsing voicemail info file 'C:\ProgramData\3CX\Instance1\Data\Ivr\Voicemail\Extensions\2532\vmail_************(info removed for privacy).xml'
2018/10/07 20:23:51.756|1880|0003|Excpt|System.Xml.XmlException: ' ' is an unexpected token. The expected token is ';'. Line 3, position 103.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos)
   at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars)
   at System.Xml.XmlTextReaderImpl.FinishPartialValue()
   at System.Xml.XmlTextReaderImpl.get_Value()
   at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r)
   at System.Xml.Linq.XContainer.ReadContentFrom(XmlReader r, LoadOptions o)
   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(Stream stream, LoadOptions options)

 

[cobaltit]

I've never seen the XML get corrupted before so that's kinda worrisome. Is this a baremetal or VM install? It could be some sort of storage issue but I'd imagine you'd have other problems as well. I'm leaning towards you having some sort of weird naming convention for your extension(s) or how incoming callerid is formatted that is freaking the system out when it parses the XML file as well. I guess find a voicemail that works and one that doesn't and open up both of the XML files as a basic sanity check. Storage corruption would likely look like complete garbage while a simple parse error because of unexpected characters/formatting would look mostly correct. The errors seem to give you a specific location to look for the offender.

 

[SeanBnd]

I did confirm that the onsite IT changed some settings in the inbound call ID setup for the trunk. I changed them back to the default settings and have had no reports of the issue still happening. Looks to be that was the cause. If anything changes I will post it here. Thanks for the help!

 

[cobaltit]

It would be great if you can send that file to 3CX or to a partner who can submit it to 3CX. If the inbound settings were technically correct but the results caused parsing errors in the voicemail XML file then 3CX needs to know so they can fix it. You can probably PM it to @YiannisH_3CX .