Summary with OpenAI, sometimes there isn't one

[noord]

Hello,

We have been using the new OpenAI-features extensively.
All conversations are always transcribed a few seconds / minutes after the call ends.
Usually, a sentiment and summary is also available around that same time.

But, I have noticed that a transcription and sentiment are always filled out, but a summary isn't always.

Is it possible OpenAI is just unable to summarize some calls, for example personal conversations with a lot of topics in one call?
Or is there something wrong with OpenAI at that time?
I have noticed this happening on multiple instances. Sometimes, one or two calls don't have a summary, and the next call does again.

Has anyone seen this as well? Or any idea what the conditions are for OpenAI not to be able to write a summary?

 

[Evolute IT]

Could you be running out of tokens per minutes (TPMs) in your limits? That could explain this.

 

[noord]

Could you be running out of tokens per minutes (TPMs) in your limits? That could explain this.

Thanks for the suggestion!
I took a look at that, but it doesn't seems that's the issue. OpenAI reports very few API calls and TPM in the logs
Also, it doesn't seem to matter if a call is long or short, it just stopped summarizing.
The 3CX instance is still in pre-deployment, so it's not used heavily.



As you can see, the bottom three calls have been summarized and then it just stopped for no apparant reason.
The transcription still works fine though.

 

[ivank]

Hello, try filtering 3CXSystemService.log by "SPEECH" and most probably you'll see some error message there.

 

[ivank]

Also try "OPENAI".

 

[noord]

Also try "OPENAI".

Thanks for the suggestions.
The log files contains no errors regarding speech or openai

It just says things like this:

2024/09/09 14:41:09.126|0052|Info| [Npgsql.Command] Command execution completed (duration=1ms): select * from recordings join recording_participant on fkid_recordings=recordings.id_recording where id_recording=$1 order by id_participant 2024/09/09 14:41:09.156|0052|Info| [System.Net.Http.HttpClient.Forge.OpenAI.LogicalHandler] Start processing HTTP request POST https://api.openai.com/v1/audio/transcriptions 2024/09/09 14:41:09.156|0052|Info| [System.Net.Http.HttpClient.Forge.OpenAI.ClientHandler] Sending HTTP request POST https://api.openai.com/v1/audio/transcriptions 2024/09/09 14:41:12.732|0027|Info| [System.Net.Http.HttpClient.Forge.OpenAI.ClientHandler] Received HTTP response headers after 3575.6055ms - 200 2024/09/09 14:41:12.732|0027|Info| [System.Net.Http.HttpClient.Forge.OpenAI.LogicalHandler] End processing HTTP request after 3575.9989ms - 200 2024/09/09 14:41:12.758|0027|Info| [Npgsql.Command] Command execution completed (duration=2ms): update recordings set transcription=$1, summary=$2, sentiment_score=$3, result=1 where id_recording=$4


You can see the call from 14:40 being transcribed in the above log-file and screen shot.
Everything seems fine, it's just that sentiment and summary aren't generated.
There aren't many calls on 3CX yet, because it's still in pre-deployment. OpenAI doesn't report any rate limits triggering or anything.

I also rebooted 3CX and that didn't help either.

If you have any further suggestions, that would be really helpful!

 

[ivank]

It's a too small snippet to tell. Could you send me a link to the full log file in PM?

 

[noord]

I have sent the full logs to IvanK.
I didn't see any errors in the logs, but Ivan is still checking.

Does anyone else have the summaries working on 3CX v20 with a Pro-license?
They did work for me until they stopped working for no apparent reason.
Transcription is working fine, it's just the sentiment and summary that don't work anymore.
I then updated to U3 beta but that didn't fix it either.