The Continuing Evolution of Video Codecs Tue, 07/01/2014 - 6:39Pm
Total Page:16
File Type:pdf, Size:1020Kb
The Continuing Evolution of Video Codecs Tue, 07/01/2014 - 6:39pm Stephen Wright, Software Architect, Openwave Mobility Today's top stories for broadband professionals - Sign up now! The current dominant codec for Internet video is H.264, but there are influential companies dedicated to alternatives. With the advent of 4K (aka Ultra HD), new codecs will be useful. H.265 (aka HEVC) and VP9 could split the market, and another approach from Mozilla, called Daala, may prove to be a wild card. When discussing delivery of video over the Internet, three aspects of the format are important protocol, container, and codec. Protocol and container are discussion topics in their own right. In this article we will concentrate on emerging video codecs, and their potential impact on mobile networks. Media delivery has undergone continual evolution, driven by technological advances and consumer demand for higher quality and lower cost. Audio has developed from the low fidelity Phonograph through many generations of physical media and digital formats. Video has followed the same trend. Each step change in quality or cost point is often accompanied by a ‘format war’, for example VHS versus Betamax or Blu-Ray versus HD- DVD. Often there is a clear victor who dominates the market segment, and not always the superior format. State of Play H.264, otherwise known as Advanced Video Codec (AVC), remains the most widely used video codec. The only real challenger to its crown within this generation of video codecs has been VP8, which is used in the WebM format created by Google. VP8 was developed by On2 Technologies, which was acquired by Google in February 2010. With WebM, Google attempted to exploit the chink in the AVC armor: that it is a patented technology. WebM is a combination of the VP8 video codec, OGG Vorbis audio, and the Matroska container format, all of which are open source (though VP8 makes use of some patented technologies). Another exploitable opportunity was the developing HTML5 standard. Traditionally, internet video has either been played by a “break out player” outside of the browser context, or by a plugin. The dominant technology in the browser plugin space has been Adobe (Shockwave) Flash. HTML5 introduced the ‘video’ tag that allows the browser to play video natively, embedded in the web page, without the need for a plugin player. Crucially, HTML does NOT mandate supported video formats, and so each browser vendor is free to choose. The vendor’s choice is influenced by factors such as the desire to promote open source (Firefox), or having a stake in a particular technology (Chrome). Data traffic (red) is growing at an extraordinary rate. Source: Ericsson AVC became a pragmatic choice, given the breadth of content and device support. Eventually, all of the major browser vendors introduced support for AVC. Importantly, at the time of writing, support for VP8 and VP9 on iOS Safari is a definite ‘no’. Apple has consistently supported AVC while ignoring VP8, and this is another important factor given the popularity of iDevices. The failure of WebM/VP8 to make a significant impact within the current generation of video codecs can largely be attributed to the following: AVC was already very well established, with mature codec implementations. Content providers had already built a huge catalogue of AVC content, with the tools and workflows in place to do so. Hardware acceleration for AVC decoding is commonplace. This is particularly important for mobile devices, as hardware decoding implementations drain the battery less than pure software implementations using a general purpose CPU. Apple refused to add VP8 support to their devices, ensuring that to provide adequate coverage of devices the content provider would need to supply an AVC variant. Why then also provide a VP8 alternative? AVC licensing is administered by MPEG-LA. In a nutshell, there are two enforcement points for the license. A license required to implement and distribute the codec based on the standard, and another to distribute ‘for-pay’ content. Content that is free to users has no license to pay until “at least” 2015. Will we see MPEG-LA maintain this stance on ‘free’ content, or will we see royalty payments introduced? Note that it is not the content producer who is liable for royalty payments, but rather the ‘apparent provider’. As one of the largest “free” video providers on the Internet, these issues are the main drivers behind Google’s (and therefore YouTube’s) VP8/VP9 initiatives. VP8 itself uses patented techniques in video compression, that are owned by the MPEG-LA patent pool. In order to maintain the position that users would be free of royalty payments, Google licensed the technology from MPEG-LA in March 2013. The Next Generation Network demands of streaming high quality video, in conjunction with the availability of mass market higher resolution devices, has created the need for a new generation of video codecs to provide higher quality video at lower bit rates. Most commercial content is now produced at 1080p, and many smartphones are now capable of recording at that resolution. Ultra-HD resolution content such as 4K video is emerging, with Television leading the way. Netflix has introduced a 4k video service, with content limited to recent Hollywood blockbusters and popular series such as “House of Cards”. The library can only grow over time. Ultra-HD formats at 4k or 8k resolution makes sense for very large screens, such as cinema. However in the home, maths tells us that at a viewing distance of 10 feet – the human eye cannot distinguish individual pixels on a 720p screen. This means that there is no gain in viewing quality for any resolution above 720p. 1080p is now commonplace, and at a viewing distance of 10 feet the screen would need to be at least 77 inches diagonal before the viewer could distinguish individual pixels. In the short term, Ultra-HD television will be a niche market. Displays are still very expensive and with limited available content, it remains to be seen if Ultra-HD Television will become truly mass market. Counterintuitively, 4k+ resolution may have more of an impact on the video experience on laptops, mobile devices, and tablets due to the closer proximity of the viewer to the screen. 4k laptops are already available, with 10.1” tablet devices in the works. Increasing video resolution brings unavoidable costs into the ecosystem: For the content provider o Higher processing cost to encode o Higher storage requirements for Video on Demand (VoD) applications o Higher bandwidth required to serve the content For the consumer o Higher processing cost to decode, which can affect battery life o Higher bandwidth required to play the content without stalling and buffering o More bytes transferred, which impacts mobile data plans For the network o Higher bandwidth demands o User dissatisfaction if video cannot be played smoothly The increased processing cost can be absorbed by the continual evolution of hardware to provide greater processing capabilities. Processing will eventually be supported by hardware accelerated chipsets for that codec. For VoD services, cloud instances of video encoders can be spun up to encode high volumes of video data in a short time. This is a tactic used by YouTube when a new format is introduced. The more difficult issue is the effect on the network due to the bandwidth required to sustain playback of the content at higher bit rates. This has necessitated a new generation of video codec, to provide high quality video at a lower bit rate. The two major players in this space are the next generation of the AVC and VP8 codecs, High Efficiency Video Codec (HEVC, otherwise known as H.265) and VP9 respectively. Both codecs produce content at the same perceptible quality level as their predecessors, but at a lower bit rate. The trade-off is that both require more processing to do so. There are many published comparisons available online however the codec implementations are immature, with development ongoing, and both encoding speed and quality constantly improving. Each of the comparisons therefore is comparing two particular codec implementations at a point in time. One example study can be found here. A summary table from the study is reproduced below. For each column, the ‘-‘ entry is the reference, and the % values are how much the bit rate needed to be increased or decreased in order to maintain the same quality (measured using PSNR). Video Encoder HM-10.0 (HEVC) x264 VP9 HM-10.0 - -39.3% -43.3% x264 66.4% - -6.2% VP9 79.4% 8.4% - The conclusion of the study states: “The typical encoding times of the VP9 encoder are around 130 times higher than those measured for the x264 encoder. On the other hand, when compared to the H.265/MPEG HEVC reference encoder implementation, the VP9 encoding times are lower by a factor of 7.35, on average. According to the experimental results, the coding efficiency of VP9 was shown to be inferior to both H.264/MPEG AVC and H.265/MPEG HEVC with an average bit rate overhead at the same objective quality of 8.4% and 79.4%, respectively.” In a nutshell, for coding efficiency HEVC > AVC > VP9. For encoding speed AVC > VP9 > HEVC. These are early codec implementations, and these profiles will change over time. On paper, HEVC has 10 times the encoding complexity of AVC and twice decoding complexity. This means that modern laptop and high end mobile devices area already capable of decoding in software. The cost is a heavy battery drain, which can be addressed with hardware acceleration. YouTube are currently providing VP9 content as a DASH variant.