The first thing I would suspect (and don't take this lightly please) is lack of resources.
Lack of resources can degrade audio like your description.
1) CPU: Often multiple VMs may share resources on the same server (shared cores VS dedicated cores), and one machine can "steal" resources from another.
2) Network access: if the network access has issues due to the virtual NIC or the actual NIC on the host, you will have choppy audio as the very least of your issues, or robotic voice. Bandwidth starvation and lack of QoS can also make matters worse.
So these above would be a very good starting point to start investigating. Think of ways to test against your existing server (ie. by moving a problematic VM to an idle server or a dedicated machine and compare performance)