3CX WebMeeting vs Zoom Performance

[alexdb]

Wanted to share my findings with 3CX WebMeeting vs Zoom . Like many others, I was receiving internal complaints regarding 3CX WebMeetings. Rather than rely on anecdotal evidence from end users I setup a test environment to determine if there was any qualitative difference.

I setup 1 to 5-way video conferences using the following hardware mixture:

Test setup:
1x Dell Latitude 5590: Intel i5-8250U, 16GB RAM, Windows 10 Pro 1909 64-bit, SSD, Microsoft Edge 80, 3x monitors (Zoom window on Dell 27” 2560x1440)
1x Surface Book 2: Intel i5-7300U, 8GB RAM, Windows 10 Pro 1903 64-bit, SSD, Microsoft Edge 80, Built-in monitor (13” 3000x2000)
1x Macbook Pro 13" (Mid-2010): , 2.66GHz Core 2 Duo, 4GM RAM, GeForce 320M, Mac OS High Sierra, Microsoft Edge 80, Built-in monitor ( 13.3” 1280x800)
1x iPhone 8
1x iPhone 6s Plus

Zoom app was installed on all devices.
Zoom and 3CX video quality settings left as default (as they would be for 99% of users).
3CX was utilised with Microsoft Edge for laptops and the 3CX WebMeeting app for iPhones.
3CX MCU: as020-syd.3cx.net

The test was done on an 50/20Mbps internet connection.

Test Methodology:
(Performed for Zoom and 3CX)
1) Start a meeting with the Dell 5590
2) Add the Macbook Pro to the meeting
3) Add the Surface Book 2 to the meeting
4) Add the iPhone 8 to the meeting
5) Add the iPhone 6s Plus to the meeting

For each step above, record CPU, GPU and bandwidth usage on each laptop (not phones) that has joined the meeting.

Results:

Quantitative:
Zoom used anywhere from 10-40% less CPU resources for each meeting for the Dell 5590 and Surface Book 2
Zoom used 5-20% less GPU resources for each meeting for the Dell 5590 and Surface Book 2
Zoom used about 1Mbps less bandwidth for each meeting on all laptops (did not measure phones)
Macbook pro was using 100% CPU usage with 3CX, about 30-40% CPU usage with Zoom.

Macbook pro was unable to keep up with a 2-way meeting when using 3CX. When sitting at the macbook Pro, video latency from the laptops would steadily increase to 30 seconds before "resetting" the video back to ~1 second latency, then latency would climb again. Strangely, the video latency "climb" did not seem to occur from the iphone video feeds.The Dell 5590 and Surface book 2 did not have these issues.

None of this happened with Zoom where latency was consistent on all devices.

Qualitative:
Video image quality was "good enough" for zoom meetings. When sitting at any device, video latency from other devices was almost identical. Sound was always synchronised with video. Very usable meeting

Video image quality was variable for 3CX meetings, latency from from each device varied (Macbook had the most, followed by iphone 6s). Sound was sometimes unsynchronized with video.

Discussion:
From my (limited) testing, Zoom seems to provide a better video conferencing experience than 3CX WebMeetings. Hopefully the development time spent on 3CX WebMeeting can address some of the performance - I'm not sure if this is out of the developers hands due to WebRTC?

I do understand that 3CX is a phone system first (which it excels at) and video conferencing second. I certainly wouldn't have bothered testing this if the need of video conferencing wasn't so important at the moment.

I also appreciate the price difference between Zoom and 3CX, but I think at the moment given the reliance on remote working, we are willing to pay extra for Zoom.

 

[JohnS_3CX]

Hi @alexdb

While respecting that different clients will need different solutions today, the product was built around WebRTC technologies which allows us to keep it open and free.

Older pre-broadwell and skylake era machines are not equipped with hardware encoders/decoders so they are forced to use the cpu entirely to accomplish the task of encoding/decoding the video, meaning they will lag behind. While this is not desirable, it mostly affects machines with chips older than 5 years. Mobile devices are a bit better at handling video than older PCs: they are "weaker" to begin with so they tend to have good hardware encoder/decoders to make up for the difference. Plus, they get replaced sooner than PCs would so the majority of them is already up to date with modern technologies.

I would consider anything older than that as a best-effort device. Future plans may change this behavior, but as newer machines replace older ones, the pool of best-effort devices will keep getting smaller while VP8 capable devices are taking over the market.

TL;DR is, older devices will not be supported any time soon, especially as newer more capable devices phase the old ones out.

 

[alexdb]

Thanks John,

I actually realised I've made an error with my conclusions, the percentages above are in absolute terms (e.g. 15% CPU vs 40% CPU usage for Zoom vs 3CX).

Basically, Zoom uses anywhere from half to a third of CPU and GPU resources compared to 3CX - on each laptop tested. I agree older devices (especially 10 year old macbooks) aren't very relevant today, but across the board resource usage of 3CX is way higher than Zoom.

 

[Jeff Harlan]

@alexdb Yeah we are in the same boat. I love 3CX as a phone system and if all the stars line up you can have a good 3cx meeting but our staff of 50 is screaming at top volume about video, audio, or sharing not working. I have experienced it myself in trying 3cx meeting and one person not able to hear anything we jump straight into ms teams and it works fine. Works great for most internal meetings but it gets tough when customers try to join. And forget the audio bridge while in a web meeting. I want to love 3cx meetings but staff may make it impossible.

 

[noord]

I have experienced it myself in trying 3cx meeting and one person not able to hear anything we jump straight into ms teams and it works fine.

Regarding this issue, could you perhaps please elaborate on this in my topic regarding audio issues? It's here: https://www.3cx.com/community/threads/webmeeting-audio-not-working-currently.72164/

Thanks!

 

[ITE-Chris]

Just tried a 25 person 3CX webmeeting and it was unsuccessful. Lots of users getting kicked out, people coming and going - could not get the meeting started. Ended up sending a Zoom link out to get everyone together. Will be hard for me to suggest this going forward.

 

[aberry]

Having far fewer issues here with 3cx compared to zoom. zoom is lighter on the cpu / network connection, but has lower quality and much lower frame rate - especially in larger groups. Also, "Macbook Pro" doesn't mean much - Retina MBP, Touch Bar MBP? Makes a big difference, those retina macbooks are dinosaurs and going to be slow.

 

[alexdb]

Having far fewer issues here with 3cx compared to zoom. zoom is lighter on the cpu / network connection, but has lower quality and much lower frame rate - especially in larger groups. Also, "Macbook Pro" doesn't mean much - Retina MBP, Touch Bar MBP? Makes a big difference, those retina macbooks are dinosaurs and going to be slow.

The specifications are in the post (I know it's long so don't blame you).

Performance is perfectly OK with Zoom on the Macbook, however is unusable with 3CX. I would not really care about that as it's 10 years old, but for the fact that it is emblematic of a wider issue with 3CX WebMeetings (or WebRTC).

 

[ITE-Chris]

something else is definitely going on when the same users having issues move from 3CX to Zoom and Zoom performs perfectly - same computers, same network connections - only the service has changed.

 

[fergaloim]

I would agree with all Alexdb has said above.

I dont think its WebRTC as such, possibly to do with 3CXs implementation of it.
Jitsi (a free webrtc platform) does not have the same issues as 3CX.

Alexdb - it would be interesting to run your test on 3CX vs Jitsi?

 

[Juan Marriott]

I would agree with all Alexdb has said above.

I dont think its WebRTC as such, possibly to do with 3CXs implementation of it.
Jitsi (a free webrtc platform) does not have the same issues as 3CX.

Alexdb - it would be interesting to run your test on 3CX vs Jitsi?

I can’t wait for the on premise version of 3CX web meeting....
I am also very happy with 3CX; but looking for video integration in the same app.
we are also using zoom...

 

[nitrox]

I have done the same post days before where I was complaining the same situation. Zoom vs 3cx webmeeting. It is difficult to discuss to customers who wants to speak remotely like if they were in the office and they are able to do it via zoom and not with 3cx webmeeting . I really don’t understand why 3cx doesn’t say anything about the evidence we (all) have discovered during this days. Sometimes the truth is not wrong . It is enough to say: guys we already know the limitations you are discovering and we are going to fix in “days/months” next release”. When 3cx doesn’t recognize the limitation I am thinking that I have to change service because the problem cannot be solved if the software house doesn’t recognize that there is a problem.

 

[Juan Marriott]

I would agree with all Alexdb has said above.

I dont think its WebRTC as such, possibly to do with 3CXs implementation of it.
Jitsi (a free webrtc platform) does not have the same issues as 3CX.

Alexdb - it would be interesting to run your test on 3CX vs Jitsi?

Jitsi is open source... maybe 3CX should look at some way of integration... I have tested, and works fine but I am looking to have it all on one app...

 

[argenvoip]

Permission to comment
I like 3cx as ip-pbx
but right now Jitsi proves to be superior to 3cx webmeeting
Jitsi is very easy to use for the participants and very easy to install on the servers themselves, as well as being free
We all hope 3cx webmeeting solves your problems
(Sorry my bad English, I use a translator)

 

[Juan Marriott]

Permission to comment
I like 3cx as ip-pbx
but right now Jitsi proves to be superior to 3cx webmeeting
Jitsi is very easy to use for the participants and very easy to install on the servers themselves, as well as being free
We all hope 3cx webmeeting solves your problems
(Sorry my bad English, I use a translator)

Hello @argenvoip, I was very tempted to install Jitsi too; but
1. Every transition takes time for the average user in the company.
2. 3CX is working hard to get things working right, and off course the on-premise MCU.
3. I really prefer to have unified communication in one app. Integration.

I was testing to use an iPad with AnyDesk as a controller to replace the zoom room...

 

[Mike Hammett]

Why haven't we had a substantive response from 3CX?

 

[Justin Scott]

I’d like to add to this if I may...

Over the past few weeks I’ve been using 3CX web meetings for my child’s basketball team training sessions.

At times, the link just will not work for all participants - 4 out of 5 people will be able to log in, but 1 will log in and not be able to see or hear anything. They’re present in the meeting screen (we can see them, their camera and microphone are enabled) but they cannot participate at all. This is not just on one device, but across any - IOS, Android or web on a Mac.

Furthermore, there are times when I will try to log in using the link for external participants and I’ll get a 503 error. For a period of time, I am simply not able to log on to the meeting then, as if by magic, it will work again.

I put 3CX up as the platform for a board meeting I had a couple of months back, and people had trouble getting on it left right and centre. Dropouts, bad audio, bad video. They tried Zoom the next month, and everyone got on no problem.

My daughter’s gymnastics class has been using Zoom, and it just works. Every single time.

If the guys at 3CX need any logs or anything to troubleshoot these issues, let me know and I’ll log a ticket or whatever is required next time it happens.