Has anyone seen this? Gemini Speech Sensitivity?

UCMUserAZ

Customer
Joined
Dec 19, 2023
Messages
156
Reaction score
248
I have been testing a few gemini call scripts with 3cx this week. Very powerful and awesome I have to say. It seems the speech sensitivity once connected with Gemini is really touchy. Testing without speakerphone it works really good. When on speakerphone (iOS latest; 17pro) Gemini can interrupt sometimes. I tested with just a simple greeting and it sometimes repeats. If I put speakerphone on with the mic muted, it works 100%. Not sure if the iphone mic is picking up the speakerphone audio or what or an iphone issue. I don't think this is anything on the 3cx side, but just how Gemini is listening. I see gemini has some realtime_input_config settings around: https://aistudio.google.com/docs/live-api/capabilities
  • activityHandling (NO_INTERRUPTION)
  • automaticActivityDetection.startOfSpeechSensitivity
  • automaticActivityDetection.endOfSpeechSensitivity
  • automaticActivityDetection.silenceDurationMs
I do not think 3cx exposes these settings through StartGeminiVoiceSessionAsync() or global parameters. Do you think it would be possible to add support for passing realtimeInputConfig when initializing the Gemini Live session? Only reason I ask, I know some of our callers use speakerphone and we would hear this feedback from them on the Ai voice agent. Not sure if Grok or OpenAi also have this kind of sensitivity.

If anyone out there has a gemini call script, I would be curious if speakerphone produces the same? It is just odd where if I mute the mic on speakerphone it works perfect so would not think anything in my prompt.

Thanks in advance!
 
The mute test is a good clue.

Please clarify the call route; is the iOS making an incoming call through the PSTN network? Is it caling as an extension? Can we have an explicit description of the call route end-to-end?

Try the same greeting at low and high speaker volume, then repeat it on a second (different model, ideally not iOS) phone using the same call route and Gemini model.

If the interruptions get more frequent as the volume goes up, that would point toward speaker audio feeding back into the mic.

If different model phones give different experiences, that adds hints towards noise cancellation settings.
 
  • Like
Reactions: KyriacosS_3CX

Forum statistics

Threads
112,545
Messages
593,127
Members
165,752
Latest member
f8bar