videocalling
glossary.backToGlossary

Codec

技术

Software that compresses and decompresses audio/video data for efficient transmission

What is a Codec?

A codec (compressor-decompressor or coder-decoder) is software or hardware that compresses raw audio and video data into a smaller format for transmission and then decompresses it for playback. Think of it as a translator that takes the massive amount of data from your camera and microphone, squeezes it down to something your internet connection can handle, and then expands it back out on the other end so the recipient sees and hears you clearly.

Without codecs, video calling would be impossible for most users. Raw, uncompressed 1080p video at 30fps requires roughly 1.5 Gbps (gigabits per second) of bandwidth. Even fiber internet connections typically max out at 100-1000 Mbps upload. Codecs compress this to 1-3 Mbps for high-quality video calls—a reduction of 500-1500× while maintaining visual quality that looks nearly identical to the original.

In WebRTC, codec selection is critical. The codec determines your call quality, bandwidth requirements, CPU usage, battery consumption, and whether the call even works across different browsers and devices. As of 2025, the WebRTC codec landscape has matured significantly with clear winners for most use cases.

How Codecs Work

Video codecs exploit redundancy and human perception limitations:

  • Spatial compression: Within each frame, large areas are often similar. The codec stores patterns once and references them elsewhere
  • Temporal compression: Consecutive frames are usually very similar. The codec stores a full frame occasionally (keyframe) and only the changes (delta frames) for subsequent frames
  • Perceptual compression: Human eyes are less sensitive to certain details (like exact color in dark areas). Codecs discard information you wouldn't notice anyway
  • Mathematical transforms: Techniques like DCT (Discrete Cosine Transform) convert image data into frequency space where it compresses more efficiently

The encoder (compression) side is computationally expensive. Modern codecs can adjust encoding complexity on-the-fly—when CPU is available, they produce smaller files with better quality. When CPU is constrained (battery low, many apps running), they use faster, simpler encoding that produces slightly larger files or lower quality.

The decoder (decompression) side is typically much faster. This asymmetry is intentional—one person encodes, but potentially many people decode (in a broadcast or meeting), so it's better to burden the encoder.

Mandatory WebRTC Codecs

RFC 7742 specifies that all WebRTC-compatible browsers MUST support certain codecs to ensure interoperability:

Video: VP8 and H.264

Every WebRTC browser must support VP8 and H.264 Constrained Baseline profile. This guarantees that any two WebRTC clients can establish a video call using at least one common codec, even if they support different sets of advanced codecs.

Audio: Opus and G.711

Every WebRTC browser must support Opus and G.711 (PCMA and PCMU formats). Opus is the preferred choice—it's superior in quality and efficiency. G.711 exists mainly for compatibility with legacy VoIP systems.

Video Codecs in Detail

VP8 (2008)

VP8 is Google's open, royalty-free video codec released in 2010 and mandated by WebRTC. It's the baseline codec that every WebRTC implementation supports.

Pros:

  • 100% free—no licensing fees, no patent concerns
  • Universal browser support
  • Low CPU usage for encoding and decoding
  • Excellent for low-latency real-time communication
  • Well-optimized in all browser implementations

Cons:

  • Compression efficiency is dated—requires more bandwidth than newer codecs for similar quality
  • Minimal hardware acceleration support (most encoding/decoding is software-based)
  • Not ideal for very low bandwidth scenarios

Best for: When compatibility is critical and bandwidth is adequate. Use VP8 as a fallback when newer codecs aren't available.

H.264/AVC (2003)

H.264 (also called AVC) is the most widely deployed video codec in history. It's used in Blu-ray, streaming services, smartphones, and nearly every device with a screen.

Pros:

  • Excellent compression efficiency—better than VP8, competitive with VP9 at typical bitrates
  • Near-universal hardware acceleration (encoding and decoding)
  • Very mature, well-optimized implementations
  • Low power consumption due to hardware support
  • Mandatory in WebRTC, ensuring compatibility

Cons:

  • Licensing fees (though browsers handle this, not app developers)
  • Not as efficient as VP9 or AV1 at very low bitrates
  • WebRTC only requires the Constrained Baseline profile, which is less efficient than Main/High profiles

Best for: Mobile devices (battery savings from hardware acceleration), applications where power efficiency matters, interoperability with non-WebRTC systems.

VP9 (2013)

VP9 is Google's successor to VP8, offering significantly better compression while remaining royalty-free.

Pros:

  • 50% bitrate reduction compared to VP8 for similar quality
  • Competitive with H.264 High profile, sometimes better
  • Completely free and open
  • Excellent for low bandwidth scenarios
  • Growing hardware decoder support
  • Supported in all major browsers (Chrome, Firefox, Edge)

Cons:

  • Higher CPU usage for encoding (1.5-2× VP8)
  • Limited hardware encoder support (more common for decoding)
  • Not supported in Safari (Apple prefers H.264/H.265)
  • Slightly higher latency than VP8 due to encoding complexity

Best for: Desktop video calling where bandwidth is constrained but CPU is available. Screen sharing benefits greatly from VP9's efficiency.

AV1 (2018)

AV1 is the newest royalty-free codec from the Alliance for Open Media, designed to replace VP9 with even better compression.

Pros:

  • 30% better compression than VP9, 50% better than H.264
  • Exceptional quality at very low bitrates
  • Completely free and open
  • Future-proof—hardware support is growing rapidly
  • Excellent for screen sharing with fine text

Cons:

  • Encoding is 3-5× more CPU-intensive than VP9
  • Not practical for real-time encoding on most devices without hardware acceleration
  • Hardware encoder support is still emerging as of 2025
  • Not yet widely adopted in WebRTC applications
  • Higher latency due to encoding complexity

Best for: Broadcasting, recording, screen sharing on powerful machines. In 2025, AV1 is primarily used for one-way streams rather than interactive video calls. Expect broader adoption as hardware support improves.

H.265/HEVC (2013)

H.265 (High Efficiency Video Coding) is the successor to H.264, offering double the compression efficiency.

Pros:

  • 50% bitrate reduction compared to H.264 for similar quality
  • Excellent hardware support (75% Windows, 99% macOS, 86% Android, 90% iOS)
  • Low power consumption with hardware acceleration
  • Superior quality per bitrate
  • Now supported in Chrome 136+, Safari

Cons:

  • Complex licensing situation (multiple patent pools)
  • Not supported in Firefox as of 2025
  • Software encoding is very CPU-intensive
  • Browser support relies on hardware decoders to avoid licensing fees

Best for: Safari-based applications, high-quality calls on devices with hardware HEVC support, bandwidth-sensitive environments. Chrome 136+ enables H.265 when hardware support is detected.

Audio Codecs

Opus (Mandatory, Preferred)

Opus is WebRTC's preferred audio codec and for good reason—it's technically superior to all alternatives.

Key features:

  • Handles both voice and music exceptionally well
  • Adaptive bitrate: 6 kbps (narrow-band voice) to 510 kbps (full-band stereo)
  • Low latency (5-66.5ms algorithmic delay)
  • Royalty-free
  • Better quality than AAC, MP3 at any bitrate
  • Resilient to packet loss

Typical usage: 32-64 kbps for voice calls (mono), 64-128 kbps for music or high-quality voice (stereo).

If your application needs to transmit audio and Opus is available (which it always is in WebRTC), use Opus. There's no reason to use anything else.

G.711 (PCMA/PCMU)

G.711 is an ancient codec (1972) using simple companding to compress 16-bit PCM audio to 8-bit, resulting in 64 kbps bitrate.

It's only in WebRTC for compatibility with legacy VoIP systems. Never choose G.711 over Opus for WebRTC-to-WebRTC calls—Opus delivers better quality at lower bitrate and CPU usage.

Codec Negotiation in WebRTC

During the signaling phase, peers exchange SDP offers and answers listing supported codecs in preference order. The answering peer selects a codec that both sides support, typically choosing the first mutually supported codec from the offerer's list.

For example:

  1. Peer A offers: VP9, VP8, H.264
  2. Peer B supports: H.264, VP8
  3. Result: VP8 is selected (first match in A's preference order)

You can manipulate this preference order via SDP munging (modifying the SDP before sending) to prioritize certain codecs. For instance, if you want to prefer H.264 over VP8, reorder the codec list in the SDP.

Practical Recommendations (2025)

For General Video Calling

Prefer H.264 on devices with hardware support (most mobile devices, modern laptops). Use VP8 as fallback for universal compatibility. Consider VP9 for desktop applications where CPU is available and bandwidth is constrained.

For Mobile Applications

Always prefer H.264. Hardware acceleration saves battery significantly—20-30% reduction in power consumption compared to software VP8. On newer devices with H.265 support, H.265 offers even better efficiency.

For Screen Sharing

VP9 excels at screen sharing, especially for content with text and sharp edges. AV1 is even better if the encoder has sufficient CPU. H.265 also performs exceptionally well for screen sharing with low CPU usage when hardware-accelerated.

For Broadcasting/Streaming

Use the most efficient codec your infrastructure supports: AV1 > VP9 > H.265 > H.264. Since broadcasting involves one encoder and many decoders, investing CPU in encoding pays off.

For Low Bandwidth

VP9 or AV1 provide the best quality at very low bitrates (sub-500 kbps). H.264 and VP8 degrade noticeably below 300 kbps.

Bandwidth Consumption

Typical bitrates for 720p 30fps video:

  • VP8: 1.5-2.5 Mbps
  • H.264: 1.0-2.0 Mbps
  • VP9: 0.75-1.5 Mbps
  • H.265: 0.5-1.0 Mbps
  • AV1: 0.5-1.0 Mbps

Audio (Opus): 32-64 kbps for voice, 64-128 kbps for music

These are guidelines. Actual bitrate varies based on content complexity (talking heads vs. fast motion), encoder settings, and adaptive bitrate adjustments.

The Future: Hardware Acceleration

The codec landscape in 2025 is increasingly shaped by hardware support. As of 2025:

  • H.264: Universal hardware support (encode and decode)
  • H.265: Widespread hardware decode (75-99% across platforms), growing encoder support
  • VP9: Common hardware decode, rare encoder support
  • AV1: Growing hardware decode in newer devices (Intel Arc, AMD RDNA 3, Apple M3+), encoder support just emerging
  • VP8: Still primarily software

Expect AV1 adoption to accelerate dramatically as hardware encoders become standard in consumer devices over the next 2-3 years. By 2027-2028, AV1 may rival H.264 in deployment.

The Bottom Line

Codecs are the invisible workforce making WebRTC practical. Without them, video calling would require internet speeds available to almost no one. With them, high-quality video calls work on modest connections, even mobile networks.

In 2025, the codec choice matters more for optimization than basic functionality. VP8 and H.264 guarantee compatibility. VP9 and H.265 offer better efficiency where supported. AV1 is the future but not yet practical for real-time encoding on most devices. And Opus has definitively won the audio codec war—there's no competition.

For most applications, a simple strategy works: prefer H.264 on mobile (hardware acceleration), prefer VP9 on desktop when bandwidth is limited, and always use Opus for audio. This covers 95% of use cases with excellent results.

References

相关术语

WebRTCBandwidthLatency
glossary.backToGlossary