Ghost Voicemails

[jwmilleril]

All of our users recently received a voicmail light and an indicator on the Yealink phone that they have a voicemail message but there isn't one there. I checked in the web client too and nothing. We have the indicator lights set to VM's only. I've rebooted the server and reprovisioned a few of the phones just to try to get rid of the ghost message with no luck. If I turn off the indicator light in the phone's provisioning, the light goes away but it still says I have a message on the screen. Any suggestions?

 

[ITMgr2527]

• 3CX Self Hosted - version 20.0.7.1057
• Debian 12
• Yealink T54W and T58W
• Local provisioning
• Flowroute Trunk
• Passed FW checker
• No custom template

We're seeing the same thing, but specifically for extensions with "Send voicemail as attachment and delete from mbox". If I swap voicemail handling to leave the VM on the server, ghost VM notification stays (if it existed prior to change, even before a new VM comes in). If I call and leave a new VM in the mailbox, the notification goes away on the phone after deleting only the new voicemail. However, if I swap back to email-and-delete, the problem recurs with subsequent VMs and ghost VM notifications persist.

I've reprovisioned, rebooted, and replaced a physical phone. Less than 500MB of 50GB voicemail storage is used. Nothing exists in "/var/lib/3cxpbx/Instance1/Data/Ivr/Voicemail/Extensions/####" folder.

After the email-and-delete extension gets a VM, the desk phone receives the following snippet from a SIP Notify:

Message Body
Messages-Waiting: yes\r\n
voice-message: 1/0\r\n

Maybe a flag isn't being cleared after the voicemail is emailed and deleted, OR the notify is simply sent before the VM is deleted? From everything I've seen, voicemails are not being lost. I've received every voicemail left while testing the ghost VM notification issue. Seems to be a simple bug around timing or clearing the notification.

 

[jwmilleril]

I verified that it's the same for us. We have everyone set to "send to email and delete" so this must've been the last update that was applied. This is obviously a bug that needs to be addressed.

 

[jwmilleril]

We are also self-hosted on a windows server on the same version as you, 10.0 Update 7 Build 1057 using Yealink T54W, T53 and T57. The behavior is the same on all of those phones but not in the 3cX app on smart phones.

 

[paulodagraca]

Hello,

Did you also check, if there is new firmware for the Yealink IP Phones ?
There have been some updates, however you can be on the new firmware already.
Just asking.

Paulo

 

[jwmilleril]

Hello,

Did you also check, if there is new firmware for the Yealink IP Phones ?
There have been some updates, however you can be on the new firmware already.
Just asking.

Paulo

We are on the latest supported firmware for all the phones.

 

[JohnS_3CX]

Hi,

I will send you a private message to check a couple of things.

 

[NexusIT]

We are also having this issue since upgrading to update 7.

 

[JohnS_3CX]

Hi all,

1. Are you all using transcription by any chance? And if so, when you receive the email with the attachment, is the transcription included or missing?

2. Are the voicemails direct to the extension or is it perhaps forwarded to this extension from a queue, ring group, IVR etc?

3. Do you notice it also on other brands (if you have them), not just Yealink ?

 

[ITMgr2527]

Hi all,

1. Are you all using transcription by any chance? And if so, when you receive the email with the attachment, is the transcription included or missing?

2. Are the voicemails direct to the extension or is it perhaps forwarded to this extension from a queue, ring group, IVR etc?

3. Do you notice it also on other brands (if you have them), not just Yealink ?

1. No transcription on our system.
2. Voicemails are direct to extension, not a Q, RG, IVR, etc.
3. We only have Yealink so I have no insight on this question.

 

[mic03167]

We have the same issue here. We have 3500 extensions and it is happening over a range of phones from new Yealink to really old Polycom.

 

[NexusIT]

Hi all,

1. Are you all using transcription by any chance? And if so, when you receive the email with the attachment, is the transcription included or missing?

2. Are the voicemails direct to the extension or is it perhaps forwarded to this extension from a queue, ring group, IVR etc?

3. Do you notice it also on other brands (if you have them), not just Yealink ?

1. We are not using transcriptions.
2. They are being forwarded to email and deleted.
3. We only have Yealink phones.

 

[mic03167]

1. We are not using transcriptions.
2. We forward and delete almost all of our 3500 extensions.
3. We have Yealink, and Polycom phones. I would say we have at least 10 to 15 different phones.

The issue is a backend issue with 3cx setting the phone flag to voicemail waiting. You can clear it temporarily if you do the following:

1. Disable the forward and delete for an extension.
2. Call and leave a voicemail.
3. Go into the extension and delete the voicemail.

This will clear the notification. But if you reenable the forward/delete then call and leave another voicemail it will turn it back on.

This behavior did not happen in Update 6. It started with update 7. We updated over the weekend and Monday we started getting calls from users when they came in to work. We are a school district with 30+ sites and about 3500 extensions and 2000+ users.

It does not happen to a phone until a voicemail it left at the extension. We have been able to reproduce the issue over and over on our site.

I have an email into our partner but will share this thread with them.

 

[AndreasPo_3CX]

Hi everyone,

Can you confirm the following:

1. What SMTP are you using? (3CX, M365,Google,Custom)
2. If Custom, is it local or remote?
3. Is SSL/TLS enabled?

 

[ivank]

This behavior did not happen in Update 6. It started with update 7. We updated over the weekend and Monday we started getting calls from users when they came in to work. We are a school district with 30+ sites and about 3500 extensions and 2000+ users.

It does not happen to a phone until a voicemail it left at the extension. We have been able to reproduce the issue over and over on our site.

I have an email into our partner but will share this thread with them.

Could you please check if adding parameter VMAIL_NOTIFY_DELAY equal to 0 in Advanced > Parameters resolves the issue?

 

[mic03167]

Hi everyone,

Can you confirm the following:

1. What SMTP are you using? (3CX, M365,Google,Custom)
2. If Custom, is it local or remote?
3. Is SSL/TLS enabled?

We are using M365.

 

[mic03167]

Could you please check if adding parameter VMAIL_NOTIFY_DELAY equal to 0 in Advanced > Parameters resolves the issue?

So yes and no. I just did a bunch of testing and it does fix the issue on new calls but does not fix any extension that already has it set. To fix the extensions that are still showing the indicator you need to leave that user a voicemail to initiate the update on the extension. I tested it on about 5 extensions and it worked on all 5.

For now I will send out a message in teams that anyone that has the blinky light (polycom phones blink and chirp) to have someone leave them a voicemail message.

Let me know what else you need done on our end and if this is going to be the fix or if this is just temporary.

 

[mic03167]

We have at least 100+ extensions now that seem to be fixed by the VMAIL_NOTIFY_DELAY parameter. They do need to receive a new voicemail after the parm is added to clear it on their phone.

 

[ITMgr2527]

So yes and no. I just did a bunch of testing and it does fix the issue on new calls but does not fix any extension that already has it set. To fix the extensions that are still showing the indicator you need to leave that user a voicemail to initiate the update on the extension. I tested it on about 5 extensions and it worked on all 5.

For now I will send out a message in teams that anyone that has the blinky light (polycom phones blink and chirp) to have someone leave them a voicemail message.

Let me know what else you need done on our end and if this is going to be the fix or if this is just temporary.

Based on my conversations with support, the fix is anticipated to be included in Update 8 and anyone impacted should just use the VMAIL_NOTIFY_DELAY parameter in the meantime.

 

[mic03167]

Has anyone tested the new Hotfix and do we need to remove the parameter when we apply the hotfix?