OpenAI AI agents, transcription model

[rlg]

Hi,

From v20u9 to v20u10, you changed the transcription model used in AI agents from Whisper to gpt-realtime-whisper.

Moving cost from $0.006 /minute to $0.017 /minute :
https://developers.openai.com/api/docs/pricing#audio-tokens

Which can have an impact with high-volume installations.

Does gpt-realtime-whisper really improves over Whipser ?
Whisper seems to already give really good results in terms of transcription for AI agents.

Could it perhaps be possible to have a dropdown in Realtime API Configuration panel to choose the Transcription model, as we already do for the Realtime and Text ones ?

Thank you very much !

 

[KyriacosS_3CX]

Hello,

OpenAI based AI Agents use gpt-realtime in U9 too. (Hosted) whisper is for "offline" transcription, not AI agents that respond live, hence the "realtime" model. Between u9 and u10, for AI agents, the change is the gpt-realtime versions that are available to choose from, inline with OpenAI model releases and retirements.

 

[rlg]

Hello @KyriacosS_3CX,

Thank you very much for your reply.

Well, look at the OpenAI interface below, for an API key only used by a v20u9 AI Receptionist.
And then especially the use of whispter-1 model.

Hence my question.

Thank you again !

Usage > Responses and Chat Completions :



Usage > Audio Transcriptions :

 

[rlg]

On the contrary, with v20u10 :

Usage > Audio Transcriptions :



Hence my primary assumption.

Thank you again !

 

[rlg]

I confirm my first assumption is good, based on cost calculation from the OpenAI usage portal.
v20u9 uses Whisper as the transcription model for AI agents.
While v20u10 uses gpt-realtime-whisper.
Moving cost from $0.006 /minute to $0.017 /minute.
Which will have a huge impact.

 

[nikolascx]

Hi @rlg,

Yes, you are correct – thank you for highlighting this.

We changed the transcription model to address quality issues with our AI Agents. Whisper worked well for transcribing completed recordings, but it was not the best fit for our live transcription requirements.

After investigating these issues with OpenAI, they recommended moving to gpt-realtime-whisper, a streaming speech-to-text model designed to provide low-latency, incremental transcripts from live audio. We saw a significant improvement after making that change.

You are also correct about the increased transcription cost. We made this change to improve quality, but we recognise its impact on high-volume installations.

In Update 11, we plan to move to gpt-live-transcribe, OpenAI’s recommended model for new realtime transcription integrations. It costs the same as gpt-realtime-whisper: $0.017 per minute of audio. We are also doing these changes because we need to support additional features we are working on.. More on this later on.

We are also testing different model combinations to assess overall call costs. In our current tests, using the latest recommended combinations, a five-minute AI Agent call costs approximately $0.40 in total, including transcription. This is an indicative result; actual costs vary with the models selected and the conversation.

Please let me know if you have additional questions and at your disposal for anything you need.

Thanks so much and God bless

 

[rlg]

Hi @nikolascx,

Thank you very much for your very detailed reply, really appreciated !

That makes sense.
I agree that these AI Agents should first focus on quality, with the risk otherwise to put callers off...
Then we'll analyze costs with customers.

Grok / grok-voice-3.0 may help here in terms of costs, but still if quality is at the expected level.

Thank you again !

 

[nikolascx]

From our results Grok-voice-3.0 is the same cost. Same as Grok v2 and equal to openai.
openai And Grok (although they charge differently - one per tokens and the other per duration) they end up with the same bill.

a 5 min call with openai and Grok were both $0.40
(Grok was 0.42 cents on some tests and 0.39 on some others)

Using openai MINI slashes the price of the 5 min call to $0.16!! We are definitely keeping that one in there. We think that the quality of this model will improve.

We will be publishing some statistics soon via the blog and we will also be making some youtibe content.. Stay tuned..

 

[rlg]

Your feedback is very interesting @nikolascx.

Going through this KB, we can find this speech to speech AI models comparison.
The Cost per Hour of Input Audio graph tends to show that Grok Voice 3.0 ($4.80) is much cheaper than OpenAI 2.1 models ($10.75 / $11.31), Mini variants being a little bit cheaper though ($3.45 / $4.60).
Which leaded to my assumption above that Grok may help in terms of costs.

But you results are way different.
Though of course much more closer to what we should expect using the AI agents...

 

[nikolascx]

The prices you quoted look very strange - from where did you get them?
You need to check the official pages

https://developers.openai.com/api/docs/pricing


https://docs.x.ai/developers/pricing


Now ... when you want to calculate, you can use the following standard (32 tokens per second)
So its more difficult to compare because grok (more predictable) charges per hour whilst openai charges per M tokens and you have inbound and outbound.

If you want to calculate you need to assume the following for openai.
600 tokens per minute INPUT
1200 tokens per minute OUTPUT
so total 1800. I would put it at 2k per minute to be sure.

Openai is cheaper on paper but from our tests, they amount to be the same.
Openai could be cheaper for shorter calls, but then will be more expensive than Grok for longer calls.

And this is for audio only. What about tool calls? Additional models used? Its hard to really calculate the price.
You need to take calls and then make averages.

We will post a blog post with expenses soon. Bear with us - currently busy funalizing a release

Keep us posted.

 

[rlg]

Thank you again @nikolascx for your detailed reply.

The prices you quoted look very strange - from where did you get them?

From this speech to speech AI models comparison.

 

[nikolascx]

You cannot rely on https://artificialanalysis.ai/ for costs
This site is good to tell you the best or top coding models, benchmarks, most used models, tests etc.
You can use it - I refer to this many times - but not for costs. Also audio costs are more complicated than text or coding costs.. Because multiple models are called in an audio conversation.

 

[rlg]

Understood, perfectly clear.
Many thanks for your valuable feedback, and good luck with next release !