Video, HTML5 canvas and the codec cavalcade

Browser support is again a new frontier. Video codec are a central issue. A variety of codecs are being offered for HTML5. Some codecs may be the subject of patent litigation.

Video is central to new HTML5-based browsers, and it is not without controversy. First, a variety of codecs are being offered. Second, such codecs are often the subject of patent litigation.

How did we get here? A browser rendering video and audio in a HTML5 canvas has to interpret a compressed data stream, expanding on the fly, into screen pixel colors and synchronized audio chip output. An enormous amount of effort has been expended on figuring out how to capture the essence of video frames and audio streams in a form compact enough to transmit at practical net speeds. The general term for hardware and/or software that can perform compression or expansion of a data stream is "codec" - for COder/DECoder. It is the codec that makes the modern media world of DVDs, music downloads, and video in the Web browser possible.

There are a wide variety of requirements for codecs depending on the available bandwidth and the degree to which various kinds of loss are tolerable on reproduction. A given codec technology may exist in a variety of implementations with adjustable parameters for various requirements. Codec technology has evolved rapidly under the pressure of commercial opportunity.

The most prominent group working on codecs has been MPEG, the Moving Picture Experts Group, which has made many recommendations which have become recognized by standards organizations such as the ISO (International Organization for Standardization) and the IEC (International Electrotechnical Commission.) However, many alternatives have been created by smaller organizations, some for commercial purposes and some as open-source projects.

Video/Audio containers

Since the data streams for video and audio codecs are so different, playback requires coordination by means of a container format to provide synchronization. A container may also provide for meta-data such as sub-titles. Two container formats you will be hearing a lot about are:

  • Ogg a standard maintained by the Xiph.Org foundation and believed to be free from software patent conflicts. Xiph.Org has released reference implementations for both audio and video playback.
  • WebM a project supported by multiple industry heavy hitters to create a royalty-free audio and video compression format for HTML5 canvas use. A WebM container uses Vorbis for audio and VP8 for video.

Just to keep things confusing, people may use "codec" to refer to both the encoding/decoding standard and to actual software or hardware implementations. Here are some of the terms you may see in use.

  • H.264 Also known as MPEG-4 part 10 or Advanced Video Coding (AVC), but also includes audio as part of the standard. A considerable improvement on earlier MPEG compression formats, it is used on Blu-ray discs, and a variety of web video plug-ins such as Adobe Flash.
  • VP8 A video compression format acquired by Google when it purchased On2. The codec has been released with a patent-free license.
  • Theora Also known as Ogg Theora, a free video compression format. It was derived from a propriety format developed by On2 Technologies, the corporation recently acquired by Google.
  • VP8 Google released this open video compression format and a reference implementation after acquiring On2 Technologies. It is used in the WebM container for the video stream.
  • Vorbis Also known as Ogg Vorbis, is a free audio compression standard. In addition to the Ogg container, it is commonly used for sound tracks in video games and streaming audio sources due to the lack of any patent claims.

The MPEG (Motion Picture Experts Group) standards for compression and expansion of audio and video started early efforts to cram video data into a limited bandwidth. The various MPEG-* standards define how a compressed data stream is decoded, with implementation of the much more complex encoding process left up to developers. Starting in 1993, various improvements are now managed by the ISO/IEC jointly. There are so many patents involved in actual implementation of MPEG related encoders and decoder programs that a company known as MPEG-LA pioneered the idea of "pools" of patent rights to provide one-stop licensing and royalty payment management. (MPEG-LA has no formal affiliation with the MPEG industry group.) MPEG-LA has claimed that some nominally open source codecs infringe on unspecified patents in the pool. Recent news items suggest they are targeting Google's VP8 codec and are actively soliciting patent holders for possible infringements.

Current browser support

This is a rapidly changing area as browser vendors jockey for position in the HTML5 race. I ran my collection of browsers through the test at html5test.com for support of video codecs in the HTML5 canvas element. Here is what I found in order of current browser use statistics from January stats by w3counter.

  • Microsoft Internet Explorer version 8 - no support.
  • Firefox 3.6.13 supports Ogg Theora only.
  • Google Chrome 9.0 supports H.264, Ogg Theora and WebM.
  • Apple Safari 5.0 supports H.264 only.
  • Opera 11.01 supports Ogg Theora, WebM.

Note that IE version 9 will have support for H.264 but you will not be able to use it on Windows XP. Firefox 4 and Opera 10 versions currently in beta are expected to support WebM. Google is reported to have stated that it will remove H.264 support from Chrome in the future and push WebM, while Microsoft has announced it will supply a H.264 plugin for Chrome. There are also plug-ins for various browsers which the user can install to support more video formats in a canvas.

All the industry heavy hitters can see that HTML5 will probably set the pattern for web reproduction of audio and video for years to come and that this will have a huge impact on market share. It is not hard to predict that the current squabbling over patent rights and royalties will continue, keeping lawyers and courts busy while content providers wonder if it will ever settle down.

Additional HTML5 Resources

HTML5 working draft of Jan 18, 2011 entitled "A vocabulary and associated APIs for HTML and XHTML" [W3C]

A valuable Wikipedia summary of various MPEG related standards. [Wikipedia]

The Theora open source compression home site. [theora.org]

Wikipedia summary of the squabble over use of Ogg Theora and Ogg Vorbis in HTML5. [Wikipedia]

The open web media project for WebM. [WebM Project]

Apple's WebKit open source project. [WebKit.org]

This was first published in March 2011
This Content Component encountered an error

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchSoftwareQuality

SearchCloudApplications

SearchAWS

TheServerSide

SearchWinDevelopment

Close