Blog

Exploring Video Codec Options for WebRTC Applications

Business owners across the globe have leveraged WebRTC APIs to build websites and applications that enable real-time communication, using audio, video, and/or and data. To establish communication, the two devices just need to comply with a mutually understood codec. This article reviews the required codecs for implementation as well as other codecs that browsers support for WebRTC.

***

WebRTC video codecs – a quick overview!

Once upon a time, WebRTC was easy with the availability of VP8, Opus, and G.711. (G.711 is an audio codec and there is no reason to use it with better and comprehensive options available). Later on, H.264 joined the list as a mandatory video codec. Owing to all this, all was well in the world of WebRTC.

Then, in 2016, Google introduced VP9 in Chrome 48 as an optional codec. Mozilla followed the move and added VP9 to Firefox. Microsoft got it for “free” by switching Edge to Chromium. Apple? All we know is that Apple added HEVC as an optional codec of their own to WebRTC and that VP9 will come to Safari soon.

Apart from these, there’s AV1 – the next-gen best video codec. In Chrome 90, Google adopts the AV1 codec for optimized video conferencing with WebRTC. This new codec is likely to improve compression efficiency and reduce bandwidth while also improving video quality. 

Video codecs support across different  WebRTC browsers

Summing up, 

• VP8 and H.264 are ubiquitous across browsers. 
• VP9 has been adopted but it is not widespread. It is likely to come to Safari.
• HEVC enables Apple.
• AV1 is a new development.

Let’s discuss the performance of these video codecs in WebRTC… 

1. VP8 or H.264 for WebRTC applications

The VP8 vs H.264 debate has been making rounds for a long time. 

Are there any actual differences between the two? Not really. Both produce videos of similar quality for the same bitrate. Having said that, there are some nuances:

• With Google not using H.264 in WebRTC, VP8 is the more maintained video codec between the two. 
• H.264 has started supporting simulcasts in Chrome after many years.
• VP8 has no hardware acceleration, so in some cases, it demands more computing power.
• H.264 has hardware acceleration. 
• VP8 has temporal scalability. H.264 does not.
• H.264 decoder is better than the one in VP8.

So, which one to use? We believe that both are equally good and you make a decision based on your preference.

2. VP9 in WebRTC (vs VP8/H.264)

To start with, Google uses VP9. VP9 is close to VP8 in its CPU use. That is why Google is using it in Google Meet. VP9 also supports SVC.

3. HEVC in WebRTC

HEVC might be a good option when you have calls taking place between Apple devices.

4. AV1

AV1 is not there in terms of performance. AV1 takes twice the computing power to encode or decode vis-a-vis other video codecs.

Don’t write off AV1 just yet. Here are some compelling reasons to choose AV1

• For handling specific scenarios like very low bitrate: CPU isn’t the bottleneck but bandwidth.
• When you are decoding and the encoder is in the cloud.
• AV1 is reportedly good at decoding thumbnails.

***

There exists a multi-codec WebRTC world… 

WebRTC started with no options. But now, we have got 4-5 video codecs to choose from.

Almost all video codecs are promising: VP9, AV1, and even HEVC. But, they all come with some challenges – mainly the processing power and availability across browsers.

A new approach is to use more than a single video codec within the same session.

Here are some suggestions that can be explored:

• Supporting higher complexity codecs on 1:1 calls and switching to other video codecs if and when there are more than two participants.
• Switching to a higher complexity codec when low bitrates.
• Decoding in as many codecs as possible parallelly, and then deciding what the encoder should send based on its CPU capabilities
• Using multiple video codecs in simulcast (AV1 with very low bitrate and next to it use VP8 or VP9 at higher bitrates).

Is it worth it to go for multi-codec?  It is when enhancing video quality in the application is important. Venturing into the multi-video codec gives that final push for ultimate results. Opt for it once you have finished pursuing all simpler optimization options.

RTCWeb.in is here to support you in your development initiatives. Be it implementing video codes and APIs, using Agora SDKs, or writing code from scratch, you can contact RTCWeb.in for anything.