Towards Video on the Web with HTML5 François Daoust 1, Philipp Hoschka 1, Charalampos Z. Patrikakis 2, Rui S. Cruz 3, Mário S. Nunes 3, David Salama Osborne 4 1W3C/ERCIM, Sophia-Antipolis, France 2School of Electrical and Computer Engineering, NTUA, Athens, Greece 3IST/INESC-ID/INOV, Lisbon, Portugal 4 Atos Research & Innovation - Atos Origin, Madrid, Spain E-mail: [email protected], [email protected] [email protected] [email protected], [email protected] [email protected] Abstract: Motivated by the revolution in the media to fully realize the potential of video on the Web as well industry brought by recent developments in video access as to allow the use of Web technologies on TV and other and sharing, this paper investigates the future of internet devices that stream video. video. We present new web standards such as HTML5, This paper is structured as follows: In Section 2, we give promising unified, simple and platform independent more background on the situation for video on the Web. access to video files and streams, as well as novel Section 3 focuses on video delivery techniques on the techniques for adaptive video coding. We also analyze Web. Section 4 looks at the future. how these techniques can be used both over traditional client server and novel P2P based media distribution 2 BACKGROUND models. This analysis is followed by a description of the remaining challenges for making video a true citizen of 2.1 Before HTML5 the Web. In the absence of a standard way to include video in an HTML page, video on the Web has been mostly the Keywords: HTML5, video, codec, streaming, adaptive, prerogative of browser-specific implementations or third- P2P, scalable coding party plug-ins such as the VLC plug-in [3], Apple QuickTime or Adobe Flash Player. These plug-ins are 1 INTRODUCTION activated through the use of the <object> tag in HTML. Following the boom of peer to peer systems that have Relying on third-party plug-ins to render the video works taken the lion’s share of Internet traffic, Internet video is fine in a variety of use cases. It does come with quickly becoming the main source of traffic on the net. drawbacks though, because the video is rendered in a According to CISCO [1], by 2014, global online video black box from the Web browser's perspective. will approach 57 percent of consumer Internet traffic (up from 40 percent in 2010). The Web is already preparing Thus, CSS cannot be used to style the video, or to apply to embrace these developments, paving the way for new transformations. SVG cannot be used to apply masks and standards. filters on the video. In short, visual effects such as the ones that appear in Figure 1 cannot be achieved using HTML5 [2] is the core specification that shapes the next regular Web technologies. open Web platform. Although HTML5 is not a Web standard yet, latest versions of Web browsers have started to implement key parts of HTML5. In particular; most desktop Web browsers now support the <video> tag, a key initiative in HTML5 to integrate video on the Web. The <video> tag opens the possibility to manipulate video within a regular Web page. Innovative user interfaces that mix video and other content can now be created without having to resort to dedicated plug-ins. As of today, the <video> tag does not mandate support for specific codecs and streaming technologies though. Standardization work around these technologies is needed Corresponding author: François Daoust, W3C/ERCIM, 2004 route des Lucioles 06410 Sophia-Antipolis, France, +33 1 77 21 16 85, [email protected] The HTML5 specification also defines a standard API to access and control the video from JavaScript. This new API opens up the possibilities in terms of the user interface, as everything may now be done through regular HTML, SVG, CSS, and Javascript. Figure 3 shows a video player entirely developed in SVG and JavaScript by Philippe le Hégaret in W3C [4]. The text that appears at a specific time on top of the video are regular HTML paragraphs controlled through JavaScript and positioned through CSS. Figure 1: Example of a visual effect using standard Web technology hard to achieve when video is rendered via video plug-in There is no standard way either to control the “black box” through JavaScript, and thus no way to change the look and feel of the video playback interface from within the rest of the page. Finally, these plug-ins are rarely available on devices that are not "regular" desktop computers, e.g. on mobile devices. 2.2 The <video> Tag to the Rescue The <video> tag introduced by HTML5 alleviates these Figure 3: Implementation of a video player interface in SVG problems. Since video becomes a regular tag such as <p> or <div>, it is directly integrated within the rest of the One of the advantages of sticking to Web technologies for Web page. CSS may thus be used to style and transform networked media content such as video is that they were the video box, as shown in Figure 2. designed with accessibility in mind. Rich user interfaces can be made accessible easily, following the Web Content Accessibility Guidelines (WCAG) [5] and Accessible Rich Internet Application (ARIA) Authoring Practices [6]. 2.3 Codecs and Containers The HTML5 specification does not restrict the list of video formats and codecs that a browser may support, leaving the door open for future formats. The <video> tag features a fallback mechanism whereby alternative “sources” can be specified, each using a different format or codec. Should the Web browser not support the first format, it will try to use the second one, then the third one, and so on. Figure 2: A <video> tag transformed through CSS in Firefox Corresponding author: François Daoust, W3C/ERCIM, 2004 route des Lucioles 06410 Sophia-Antipolis, France, +33 1 77 21 16 85, [email protected] 3 STREAMING VIDEO ON THE WEB <video id="movie" width="640" height="480"> <source src="video.mp4" type='video/mp4; 3.1 HTTP Progressive Delivery codecs="avc1.42E01E, mp4a.40.2"' /> <source src="video.webm" type='video/webm; The easiest way to deliver video on the Web is to use codecs="vp8, vorbis"' /> HTTP progressive delivery, in other words to serve video <p>The video is available as <a href="video.mp4">H.264 in an as any other content on the Web. Progressive delivery is MP4 container</a>, or as <a href="video.webm">VP8 in a WebM container</a>.</p> supported by all Web browsers. </video> Using regular HTTP has three main advantages: • A regular HTTP Web server may be used. Figure 4: Fallback mechanism of the <video> tag in HTML5 • Video delivery is transparent for firewalls. Other As of today, HTML5 does not mandate support for any transport protocols usually require changing the specific codec and format container either. The situation settings on firewalls to let the content pass. among primary desktop browsers is currently – or will be • Existing solutions to improve caching can be used in a near future – as follows: with video as well. In particular, Content Delivery • H.264 in an MP4 container: supported by Internet Networks (CDN) provided by companies such as Explorer 9.0, Safari, and Google Chrome. Support for Akamai may be used to distribute video content more H.264 usually means support for the Baseline profile. efficiently to a potentially large audience. In particular, this profile is the only one supported on Note that HTTP-based video delivery may actually mobile devices such as iPhone or Android. require caching solutions and CDN support for videos • Theora in an Ogg container: supported by Firefox, with massive audience, to avoid a meltdown effect Google Chrome, and Opera triggered by the parallel streaming of the same video to a • VP8 in a WebM container: supported by Firefox, huge number of clients (HTTP delivery is “unicast”). Google Chrome, and Opera HTTP progressive delivery is well suited for short video As things stand, there will be no way to target all HTML5 clips where the user does not need to be able to jump capable browsers with one version of a video. Delivering forward and backward in the video. Jumping forward in video on the Web will today require at least two versions: the video requires that the client has received the video up to the targeted position. This is not practical in the case of 1) one version that uses the H.264 Baseline profile in an longer video clips or movies. MP4 container Furthermore, there is no guarantee that the delivery rate 2) one version that uses VP8 in a WebM container, or will remain consistent over time. If the network Theora in an Ogg container. throughput varies over time during the delivery of the Microsoft announced [7] that support for additional video, playback will eventually stop. codecs could be added to Internet Explorer, provided they Newer "intelligent" approaches to stream video on the are installed on the operating system. Apple has not Web that go beyond progressive delivery are being provided information on its plans. It is impossible to add invented. Most of these methods still rely on HTTP support for additional codecs in Firefox that would be though. Other protocols, e.g. DCCP or RTP/RTSP, will supported in the <video> tag. New codecs in Firefox can probably play a role in video delivery but are not only be added via the <object> tag and plug-ins, which envisioned as the main streaming methods for the has the “black box” disadvantages already described.