Hardware for local AI

[nikolascx]

Hi @nikolascx & @Kevin Attard Compagno,
One last question regarding the 3CX AI Server, does it also allow to run the AI agents, especially the AI receptionist ?
If not, is it perhaps planned ?
Would allow to avoid relying on external AI services...
Thank you very much !

Hi @rlg
Why dont you use the programmable extension?
https://www.3cx.com/docs/programmable-extensions/

Clone the github repo, configure the config file,
yarn install
yarn start: openai or yarn start:gemini

Let me know how it goes. We are going to release some videos shortly.

 

[rlg]

Hi @nikolascx,

Thank you for your proposal, looks really promising !
I was however rather thinking about moving speech recognition, reasoning and speech generation to the 3CX AI Server directly.
So that this could be done locally, without having to rely on a cloud AI service such as OpenAI.
Still thinking about customers' data / sovereignty, availability / uptime, costs control etc...

Thank you again !

 

[NikosT_3CX]

@rlg for costs control, please take a look on this blog : https://www.3cx.com/blog/docs/cloud-vs-local-ai-server/

 

[rlg]

@NikosT_3CX thank you for this, I already planned a local 3CX AI Server for v20u10, for transcriptions.
I'm now talking about the other AI part, agents (speech recognition, reasoning and speech generation), which is currently handled by 3CX through OpenAI.
I then share the idea to have it handled by the 3CX AI Server, without knowing if it's feasible or not (@Kevin Attard Compagno gave some hints in his previous post).

 

[nikolascx]

Hi @nikolascx,

Thank you for your proposal, looks really promising !
I was however rather thinking about moving speech recognition, reasoning and speech generation to the 3CX AI Server directly.
So that this could be done locally, without having to rely on a cloud AI service such as OpenAI.
Still thinking about customers' data / sovereignty, availability / uptime, costs control etc...

Thank you again !

In a couple of years time, yes definitely.
But currently, the world is not ready for this.

To do this, there should be a local realtime model (does not exist... yet)
And with a pipeline, you will need GPU Power. So each 3CX you support, needs to get a GPU. How realistic is that?

To put it in perspective, if you have a 4090, plus all the work required to do the Audio pipeline, plus a local model, then you could only make 4-6 sim calls.

Let's ride the wave for now. We are watching attentively, btw what you mention is actually my second dream..

On the bright side, My primary 1.5 year old dream, is coming very very soon ... God willing