3CX Integration in our custom AI model

[janjanpalz]

Hello. This is my first time posting here, hopefully someone has an answer to my questions.

Our company has created an AI model that can answer complex questions with regards insurance (we are using OpenAI as our LLM but we provided our own knowledgebases). We already deployed this same model in our own e-commerce (in a form of chatbot) and whatsapp official number (so if a customer replies to our official whatsapp number, the AI bot will takeover the conversation).

Now, we are planning to deploy the same model to voice but I can't seem to find answers online.

Use Case:
1. Customer calls our hotline
2. AI Bot answers the call
3. Customer asks question (the way I imagine this in my head is to convert speech to text)
4. AI Bot understands the question (same text should be run in our AI model)
5. AI Bot gives response (AI model gives response in text format. Convert text to speech then stream in call)

Also, our AI Engineer suggested for me search about audio-stream support for 3CX (to get inputs from customer and give outputs to customer) but I can't seem to find any answers online.

If anyone is currently doing a project of the same, please do reach out to me.

 

[edossantos]

You will need a CFD app for this. Is your bot expecting text or the voice stream? If you need to send text to your AI bot, then you should do the following:
1) Use the Voice Input component to convert the user speech into text. Check this:
https://www.3cx.com/docs/cfd-google-cloud-speech/
https://www.3cx.com/docs/manual/cfd-components/#h.lgzfai7rclby

2) Once you have the response from Google Cloud STT, you use the Web Service REST component to send this text to your API, and get a text response back.

3) Once you have the response text, you use text to speech (Google Cloud or Amazon Polly) to convert this text to a WAV file, and then play that WAV file. Check this:
https://www.3cx.com/docs/cfd-text-to-speech-audio-prompt/

4) You will need to iterate through the previous steps, so surround everything with a Loop component.

 

[Evolute IT]

If you need assistance on this project, feel free to DM us. That would be a neat project to work on!

 

[VoIPTools]

I think the hope/expectation was that we could stream the audio to their AI. I am not aware of a way to do this with 3CX. Perhaps, you could create an application that registers like a phone and have this custom SIP phone stream the audio to/from 3CX.

 

[edossantos]

In case of needing the audio stream, there is no way with a CFD app. As mentioned by Matthew, the only way would be creating a SIP client, and behave as a phone.