DOCSLIB.ORG

Summary and Outlook

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Summary and Outlook A P P E N D I X ■ ■ ■ Summary and Outlook It has been a long journey—almost 300 pages about a couple of new elements in HTML. Who would have thought there was this much to learn. However, realistically, we are only at the start of what will be possible with audio and video in the coming years. Right now we are only seeing the most basic functionality of multimedia implemented on the Web. Once the technology stabilizes, publishers and users will follow, and with them businesses, and further requirements and technologies will be developed. However, for now, we have a fairly complete overview of existing functionalities. Before giving you a final summary of everything that we have analyzed in this book, let us mention two more areas of active development. A.1 Outlook Two further topics deserve a brief mention: metadata and quality of service metrics. Both of these topics sound rather ordinary, but they enable functionalities that are quite amazing. A.1.1 Metadata API The W3C has a Media Annotations Working Group1. This group has been chartered to create “an ontology and API designed to facilitate cross-community data integration of information related to media objects in the Web, such as video, audio, and images”. In other words: part of the work of the Media Annotations Working Group is to come up with a standardized API to expose and exchange metadata of audio and video resources. The aim behind this is to facilitate interoperability in search and annotation. In the Audio API chapter we have already come across something related: a means to extract key information about the encoding parameters from an audio resource through the properties audio.mozChannels, audio.mozSampleRate, and audio.mozFrameBufferLength. The API that the Media Annotations Working Group is proposing is a bit more generic and higher level. The proposal is to introduce a new Object into HTML which describes a media resource. Without going into too much detail, the Object introduces functions to expose a list of properties. Examples are media resource identifiers, information about the creation of the resource, about the type of content, content rights, distribution channels, and ultimately also the technical properties such as framesize, codec, framerate, sampling-rate, and number of channels. While the proposal is still a bit rough around the edges and could be simplified, the work certainly identifies a list of interesting properties about a media resource that is often carried by the media 1 See http://www.w3.org/2008/WebVideo/Annotations/ 297 APPENDIX ■ SUMMARY AND OUTLOOK resource itself. In that respect, it aligns with some of the requests from archival organizations and the media industry, including the captioning industry, to make such information available through an API. Interesting new applications are possible when such information is made available. An example application is the open source Popcorn.js semantic video demo2. Popcorn.js is a JavaScript library that connects a video, its metadata, and its captions dynamically with related content from all over the Web. It basically creates a mash-up that changes over time as the video content and its captions change. Figure A–1 has a screenshot of a piece of content annotated and displayed with popcorn.js. Figure A–1. A screenshot of a video mashup example using popcorn.js A.1.2 Quality of Service API A collection of statistics about the playback quality of media elements will be added to the media element in the near future. It is interesting to get concrete metrics to monitor the quality of service (QoS) that a user perceives, for benchmarking and to help sites determine the bitrate at which their streaming should be started. We would have used this functionality in measuring the effectiveness of Web Workers in Chapter 7 had it been available. Even more importantly, if there are continuously statistics available about the QoS, a JavaScript developer can use these to implement adaptive HTTP streaming. 2 See http://webmademovies.etherworks.ca/popcorndemo/ 298 APPENDIX ■ SUMMARY AND OUTLOOK We have come across adaptive HTTP streaming already in Chapter 2 in the context of protocols for media delivery. We mentioned that Apple, Microsoft, and Adobe offer solutions for MPEG-4, but that no solutions yet exist for other formats. Once playback statistics exist in all browsers, it will be possible to implement adaptive HTTP streaming for any format in JavaScript. This is also preferable over an immediate implementation of support for a particular manifest format in browsers—even though Apple has obviously already done that with Safari and m3u8. It is the format to support for delivery to the iPhone and iPad. So, what are the statistics that are under discussion for a QoS API? Mozilla has an experimental implementation of mozDownloadRate and mozDecodeRate3 for the HTMLMediaElement API. These respectively capture the rate at which a resource is being downloaded in bytes per second, and the rate at which it is being decoded in bytes per second. Further, there are additional statistics for video called mozDecodedFrames, mozDroppedFrames, and mozDisplayedFrames, which respectively count the number of decoded, dropped, and displayed frames for a media resource. These allow identification of a bottleneck either as a network or CPU issue. Note that Adobe has a much more extensive interface for Flash4. A slightly different set of QoS metrics for use in adaptive HTTP streaming is suggested in the WHATWG wiki5: • downloadRate: The current server-client bandwidth (read-only) • videoBitrate: The current video bitrate (read-only) • droppedFrames: The total number of frames dropped for this playback session (read-only) • decodedFrames: The total number of frames decoded for this playback session (read-only) • height: The current height of the video element (already exists) • videoHeight: The current height of the videofile (already exists) • width: The current width of the video element (already exists) • videoWidth: The current width of the videofile (already exists) These would also allow identification of the current bitrate that a video has achieved, which can be compared to the requested one and can help make a decision to switch to a higher or lower bitrate stream. We can be sure that we will see adaptive HTTP streaming implementations shortly after, when such an API has entered the specification and is supported in browsers. This concludes the discussion of HTML5 media technologies under active development. A.2 Summary of the Book In this book we have taken a technical tour of HTML5 <audio> and <video>. The Introduction told the story behind the formats and technologies that we have arrived at today and, in particular, explained why we don't have a single baseline codec for either <audio> or <video>. This is obviously a poor situation for content providers, but the technology has been developed around it and there are means to deal with this. Ultimately, the availability of browser plugins—such as Adobe 3 See http://www.bluishcoder.co.nz/2010/08/24/experimental-playback-statistics-for-html-video-audio.html 4 See http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetStreamInfo.html 5 See http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics 299 APPENDIX ■ SUMMARY AND OUTLOOK Flash, VLC, and Cortado for Ogg Theora—can help a content provider deliver only a single format without excluding audiences on browsers that do not support that format natively. The Introductory Chapters In the Audio and Video Elements chapter we had out first contact with creating and publishing audio and video content through the <audio> and <video> elements. We dug deep into the new markup defined for <audio>, <video>, and <source>, including all their content attributes. We took a brief look at open source transcoding tools that are used to get our content into a consistent format for publishing. We briefly explained how to publish the content to a Web server and how it is delivered over HTTP. We concluded the chapter with a comparison of user interfaces to the media elements between browsers, in particular paying attention to the support of accessibility in implemented player controls. Chapter 3 on CSS3 Styling saw us push the boundaries of how to present audio and video content online. Simply by being native elements in HTML, <audio> and <video> are able to make use of the amazing new functionalities of CSS3 including transitions, transforms, and animations. We also identified some short-comings for video in CSS3, in particular that reflections cannot be achieved through CSS alone, that the marquee property is too restricted to create a video scroller, and that video cannot be used as a background to a web page. However, we were able to experiment with some amazing new displays for video collections—one about a pile of smart phones where you can basically pick one up and watch the video on it, and one with video playing on the faces of a 3D spinning cube. The JavaScript API chapter saw us dig deep into the internal workings of the <audio>, <video>, and <source> elements. The interface of these media elements is very rich and provides a web developer with much control. It is possible to set and read the content attribute values through this interface. Particular features of the underlying media resources are exposed, such as their intrinsic width and height. It is also possible to monitor the stats and control the playback functionalities of the resources. We concluded this chapter with an implementation of a video player with our own custom controls that make use of many of the JavaScript API attributes, states, events, and methods. At this point, we reached quite a sophisticated understanding of the HTML5 media elements and their workings. These first four chapters provided a rather complete introduction.
Load more

Recommended publications
  • The Interplay of Compile-Time and Run-Time Options for Performance Prediction Luc Lesoil, Mathieu Acher, Xhevahire Tërnava, Arnaud Blouin, Jean-Marc Jézéquel
    The Interplay of Compile-time and Run-time Options for Performance Prediction Luc Lesoil, Mathieu Acher, Xhevahire Tërnava, Arnaud Blouin, Jean-Marc Jézéquel To cite this version: Luc Lesoil, Mathieu Acher, Xhevahire Tërnava, Arnaud Blouin, Jean-Marc Jézéquel. The Interplay of Compile-time and Run-time Options for Performance Prediction. SPLC 2021 - 25th ACM Inter- national Systems and Software Product Line Conference - Volume A, Sep 2021, Leicester, United Kingdom. pp.1-12, 10.1145/3461001.3471149. hal-03286127 HAL Id: hal-03286127 https://hal.archives-ouvertes.fr/hal-03286127 Submitted on 15 Jul 2021 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. The Interplay of Compile-time and Run-time Options for Performance Prediction Luc Lesoil, Mathieu Acher, Xhevahire Tërnava, Arnaud Blouin, Jean-Marc Jézéquel Univ Rennes, INSA Rennes, CNRS, Inria, IRISA Rennes, France [email protected] ABSTRACT Both compile-time and run-time options can be configured to reach Many software projects are configurable through compile-time op- specific functional and performance goals. tions (e.g., using ./configure) and also through run-time options (e.g., Existing studies consider either compile-time or run-time op- command-line parameters, fed to the software at execution time).
    [Show full text]
  • Screen Capture Tools to Record Online Tutorials This Document Is Made to Explain How to Use Ffmpeg and Quicktime to Record Mini Tutorials on Your Own Computer
    Screen capture tools to record online tutorials This document is made to explain how to use ffmpeg and QuickTime to record mini tutorials on your own computer. FFmpeg is a cross-platform tool available for Windows, Linux and Mac. Installation and use process depends on your operating system. This info is taken from (Bellard 2016). Quicktime Player is natively installed on most of Mac computers. This tutorial focuses on Linux and Mac. Table of content 1. Introduction.......................................................................................................................................1 2. Linux.................................................................................................................................................1 2.1. FFmpeg......................................................................................................................................1 2.1.1. installation for Linux..........................................................................................................1 2.1.1.1. Add necessary components........................................................................................1 2.1.2. Screen recording with FFmpeg..........................................................................................2 2.1.2.1. List devices to know which one to record..................................................................2 2.1.2.2. Record screen and audio from your computer...........................................................3 2.2. Kazam........................................................................................................................................4
    [Show full text]
  • Arxiv:2007.15943V1 [Cs.SE] 31 Jul 2020
    MUZZ: Thread-aware Grey-box Fuzzing for Effective Bug Hunting in Multithreaded Programs Hongxu Chen§† Shengjian Guo‡ Yinxing Xue§∗ Yulei Sui¶ Cen Zhang† Yuekang Li† Haijun Wang# Yang Liu† †Nanyang Technological University ‡Baidu Security ¶University of Technology Sydney §University of Science and Technology of China #Ant Financial Services Group Abstract software performance. A typical computing paradigm of mul- tithreaded programs is to accept a set of inputs, distribute Grey-box fuzz testing has revealed thousands of vulner- computing jobs to threads, and orchestrate their progress ac- abilities in real-world software owing to its lightweight cordingly. Compared to sequential programs, however, multi- instrumentation, fast coverage feedback, and dynamic adjust- threaded programs are more prone to severe software faults. ing strategies. However, directly applying grey-box fuzzing On the one hand, the non-deterministic thread-interleavings to input-dependent multithreaded programs can be extremely give rise to concurrency-bugs like data-races, deadlocks, inefficient. In practice, multithreading-relevant bugs are usu- etc [32]. These bugs may cause the program to end up with ab- ally buried in the sophisticated program flows. Meanwhile, normal results or unexpected hangs. On the other hand, bugs existing grey-box fuzzing techniques do not stress thread- that appear under specific inputs and interleavings may lead interleavings that affect execution states in multithreaded pro- to concurrency-vulnerabilities [5, 30], resulting in memory grams. Therefore, mainstream grey-box fuzzers cannot ade- corruptions, information leakage, etc. quately test problematic segments in multithreaded software, although they might obtain high code coverage statistics. There exist a line of works on detecting bugs and vulner- To this end, we propose MUZZ, a new grey-box fuzzing abilities in multithreaded programs.
    [Show full text]
  • Efficient Multi-Codec Support for OTT Services: HEVC/H.265 And/Or AV1?
    Efficient Multi-Codec Support for OTT Services: HEVC/H.265 and/or AV1? Christian Timmerer†,‡, Martin Smole‡, and Christopher Mueller‡ ‡Bitmovin Inc., †Alpen-Adria-Universität San Francisco, CA, USA and Klagenfurt, Austria, EU ‡{firstname.lastname}@bitmovin.com, †{firstname.lastname}@itec.aau.at Abstract – The success of HTTP adaptive streaming is un- multiple versions (e.g., different resolutions and bitrates) and disputed and technical standards begin to converge to com- each version is divided into predefined pieces of a few sec- mon formats reducing market fragmentation. However, other onds (typically 2-10s). A client first receives a manifest de- obstacles appear in form of multiple video codecs to be sup- scribing the available content on a server, and then, the client ported in the future, which calls for an efficient multi-codec requests pieces based on its context (e.g., observed available support for over-the-top services. In this paper, we review the bandwidth, buffer status, decoding capabilities). Thus, it is state of the art of HTTP adaptive streaming formats with re- able to adapt the media presentation in a dynamic, adaptive spect to new services and video codecs from a deployment way. perspective. Our findings reveal that multi-codec support is The existing different formats use slightly different ter- inevitable for a successful deployment of today's and future minology. Adopting DASH terminology, the versions are re- services and applications. ferred to as representations and pieces are called segments, which we will use henceforth. The major differences between INTRODUCTION these formats are shown in Table 1. We note a strong differ- entiation in the manifest format and it is expected that both Today's over-the-top (OTT) services account for more than MPEG's media presentation description (MPD) and HLS's 70 percent of the internet traffic and this number is expected playlist (m3u8) will coexist at least for some time.
    [Show full text]
  • User Manual 19HFL5014W Contents
    User Manual 19HFL5014W Contents 1 TV Tour 3 13 Help and Support 119 1.1 Professional Mode 3 13.1 Troubleshooting 119 13.2 Online Help 120 2 Setting Up 4 13.3 Support and Repair 120 2.1 Read Safety 4 2.2 TV Stand and Wall Mounting 4 14 Safety and Care 122 2.3 Tips on Placement 4 14.1 Safety 122 2.4 Power Cable 4 14.2 Screen Care 123 2.5 Antenna Cable 4 14.3 Radiation Exposure Statement 123 3 Arm mounting 6 15 Terms of Use 124 3.1 Handle 6 15.1 Terms of Use - TV 124 3.2 Arm mounting 6 16 Copyrights 125 4 Keys on TV 7 16.1 HDMI 125 16.2 Dolby Audio 125 5 Switching On and Off 8 16.3 DTS-HD (italics) 125 5.1 On or Standby 8 16.4 Wi-Fi Alliance 125 16.5 Kensington 125 6 Specifications 9 16.6 Other Trademarks 125 6.1 Environmental 9 6.2 Operating System 9 17 Disclaimer regarding services and/or software offered by third parties 126 6.3 Display Type 9 6.4 Display Input Resolution 9 Index 127 6.5 Connectivity 9 6.6 Dimensions and Weights 10 6.7 Sound 10 7 Connect Devices 11 7.1 Connect Devices 11 7.2 Receiver - Set-Top Box 12 7.3 Blu-ray Disc Player 12 7.4 Headphones 12 7.5 Game Console 13 7.6 USB Flash Drive 13 7.7 Computer 13 8 Videos, Photos and Music 15 8.1 From a USB Connection 15 8.2 Play your Videos 15 8.3 View your Photos 15 8.4 Play your Music 16 9 Games 18 9.1 Play a Game 18 10 Professional Menu App 19 10.1 About the Professional Menu App 19 10.2 Open the Professional Menu App 19 10.3 TV Channels 19 10.4 Games 19 10.5 Professional Settings 20 10.6 Google Account 20 11 Android TV Home Screen 22 11.1 About the Android TV Home Screen 22 11.2 Open the Android TV Home Screen 22 11.3 Android TV Settings 22 11.4 Connect your Android TV 25 11.5 Channels 27 11.6 Channel Installation 27 11.7 Internet 29 11.8 Software 29 12 Open Source Software 31 12.1 Open Source License 31 2 1 TV Tour 1.1 Professional Mode What you can do In Professional Mode ON, you can have access to a large number of expert settings that enable advanced control of the TV’s state or to add additional functions.
    [Show full text]
  • Ffmpeg Codecs Documentation Table of Contents
    FFmpeg Codecs Documentation Table of Contents 1 Description 2 Codec Options 3 Decoders 4 Video Decoders 4.1 hevc 4.2 rawvideo 4.2.1 Options 5 Audio Decoders 5.1 ac3 5.1.1 AC-3 Decoder Options 5.2 flac 5.2.1 FLAC Decoder options 5.3 ffwavesynth 5.4 libcelt 5.5 libgsm 5.6 libilbc 5.6.1 Options 5.7 libopencore-amrnb 5.8 libopencore-amrwb 5.9 libopus 6 Subtitles Decoders 6.1 dvbsub 6.1.1 Options 6.2 dvdsub 6.2.1 Options 6.3 libzvbi-teletext 6.3.1 Options 7 Encoders 8 Audio Encoders 8.1 aac 8.1.1 Options 8.2 ac3 and ac3_fixed 8.2.1 AC-3 Metadata 8.2.1.1 Metadata Control Options 8.2.1.2 Downmix Levels 8.2.1.3 Audio Production Information 8.2.1.4 Other Metadata Options 8.2.2 Extended Bitstream Information 8.2.2.1 Extended Bitstream Information - Part 1 8.2.2.2 Extended Bitstream Information - Part 2 8.2.3 Other AC-3 Encoding Options 8.2.4 Floating-Point-Only AC-3 Encoding Options 8.3 flac 8.3.1 Options 8.4 opus 8.4.1 Options 8.5 libfdk_aac 8.5.1 Options 8.5.2 Examples 8.6 libmp3lame 8.6.1 Options 8.7 libopencore-amrnb 8.7.1 Options 8.8 libopus 8.8.1 Option Mapping 8.9 libshine 8.9.1 Options 8.10 libtwolame 8.10.1 Options 8.11 libvo-amrwbenc 8.11.1 Options 8.12 libvorbis 8.12.1 Options 8.13 libwavpack 8.13.1 Options 8.14 mjpeg 8.14.1 Options 8.15 wavpack 8.15.1 Options 8.15.1.1 Shared options 8.15.1.2 Private options 9 Video Encoders 9.1 Hap 9.1.1 Options 9.2 jpeg2000 9.2.1 Options 9.3 libkvazaar 9.3.1 Options 9.4 libopenh264 9.4.1 Options 9.5 libtheora 9.5.1 Options 9.5.2 Examples 9.6 libvpx 9.6.1 Options 9.7 libwebp 9.7.1 Pixel Format 9.7.2 Options 9.8 libx264, libx264rgb 9.8.1 Supported Pixel Formats 9.8.2 Options 9.9 libx265 9.9.1 Options 9.10 libxvid 9.10.1 Options 9.11 mpeg2 9.11.1 Options 9.12 png 9.12.1 Private options 9.13 ProRes 9.13.1 Private Options for prores-ks 9.13.2 Speed considerations 9.14 QSV encoders 9.15 snow 9.15.1 Options 9.16 vc2 9.16.1 Options 10 Subtitles Encoders 10.1 dvdsub 10.1.1 Options 11 See Also 12 Authors 1 Description# TOC This document describes the codecs (decoders and encoders) provided by the libavcodec library.
    [Show full text]
  • Is the Linux Desktop Less Secure Than Windows 10? Or How Super Mario Music Can Own Your System
    IS THE LINUX DESKTOP LESS SECURE THAN WINDOWS 10? OR HOW SUPER MARIO MUSIC CAN OWN YOUR SYSTEM. Hanno Böck https://hboeck.de 1 This was too easy . It should not be possible to find a serious memory corruption vulnerability in the default Linux desktop attack surface with just a few minutes of looking. Although it’ s hard to say it, this is not the kind of situation that occurs with a latest Windows 10 default install. Is it possible that Linux desktop security has rotted? (Chris Evans) 2 NINTENDO SOUND FILES (1) Exploit against Gstreamer in Ubuntu 12.04 (LTS). Thumbnail parser. 3 NINTENDO SOUND FILES (2) NSF players are mini-emulators - the attacker can execute code in an emulator. Easier to bypass modern exploit mitigation techniques. 4 FIX The fix is to delete the affected NSF gstreamer plugin. No problem: Ubuntu shipped two different NSF player plugins. 5 FLIC EXPLOIT 6 AUTOMATIC DOWNLOADS Some browsers automatically download files to ~/Downloads. Any webpage can create files on your filesystem. (Chrome/Chromium, Epiphany, ... - not Linux specific) 7 TRACKER GNOME Desktop search tool automatically indexes all new files in a user's home - including ~/Downloads. 8 REACTION FROM TRACKER DEVELOPER Furthermore, the GStreamer guys were extremely fast in fixing it. You could claim that other libraries used for metadata extraction are just as insecure, but that'd really be bugs in these libraries to fix. (Carlos Garnacho) 9 TRACKER PARSERS (1) Gstreamer, ffmpeg, flac, totem-pl-parser, tiff, libvorbis, taglib, libpng, libexif, giflib, libjpeg-turbo, libosinfo, poppler, libxml2, exempi, libgxps, ghostscript, libitpcdata 10 TRACKER PARSERS (2) If you can exploit any of them you can exploit many Linux desktop users from the web without user interaction.
    [Show full text]
  • Neural Rate Control for Video Encoding Using Imitation Learning
    Neural Rate Control for Video Encoding using Imitation Learning Hongzi Mao†? Chenjie Gu† Miaosen Wang† Angie Chen† Nevena Lazic† Nir Levine† Derek Pang* Rene Claus* Marisabel Hechtman* Ching-Han Chiang* Cheng Chen* Jingning Han* †DeepMind *Google ?MIT CSAIL Abstract module manages the distribution of available bandwidth, usually constrained by the network condition. It directly In modern video encoders, rate control is a critical com- determines how many bits to spend to encode each video ponent and has been heavily engineered. It decides how frame, by assigning a quantization parameter (QP). The many bits to spend to encode each frame, in order to opti- goal of rate control is to maximize the encoding efficiency mize the rate-distortion trade-off over all video frames. This (often measured by the Bjontegaard delta rate) as well as is a challenging constrained planning problem because of maintaining the bitrate under a user-specified target bitrate. the complex dependency among decisions for different video Rate control is a constrained planning problem and can frames and the bitrate constraint defined at the end of the be formulated as a Partially Observable Markov Decision episode. Process. For a particular frame, the QP assignment decision We formulate the rate control problem as a Partially depends on (1) the frame’s spatial and temporal complexity, Observable Markov Decision Process (POMDP), and ap- (2) the previously encoded frames this frame refers to, and ply imitation learning to learn a neural rate control pol- (3) the complexity of future frames. In addition, the bitrate icy. We demonstrate that by learning from optimal video constraint imposes an episodic constraint on the planning encoding trajectories obtained through evolution strategies, problem.
    [Show full text]
  • An Analysis of VP8, a New Video Codec for the Web Sean Cassidy
    Rochester Institute of Technology RIT Scholar Works Theses Thesis/Dissertation Collections 11-1-2011 An Analysis of VP8, a new video codec for the web Sean Cassidy Follow this and additional works at: http://scholarworks.rit.edu/theses Recommended Citation Cassidy, Sean, "An Analysis of VP8, a new video codec for the web" (2011). Thesis. Rochester Institute of Technology. Accessed from This Thesis is brought to you for free and open access by the Thesis/Dissertation Collections at RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. An Analysis of VP8, a New Video Codec for the Web by Sean A. Cassidy A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Engineering Supervised by Professor Dr. Andres Kwasinski Department of Computer Engineering Rochester Institute of Technology Rochester, NY November, 2011 Approved by: Dr. Andres Kwasinski R.I.T. Dept. of Computer Engineering Dr. Marcin Łukowiak R.I.T. Dept. of Computer Engineering Dr. Muhammad Shaaban R.I.T. Dept. of Computer Engineering Thesis Release Permission Form Rochester Institute of Technology Kate Gleason College of Engineering Title: An Analysis of VP8, a New Video Codec for the Web I, Sean A. Cassidy, hereby grant permission to the Wallace Memorial Library to repro- duce my thesis in whole or part. Sean A. Cassidy Date i Acknowledgements I would like to thank my thesis advisors, Dr. Andres Kwasinski, Dr. Marcin Łukowiak, and Dr. Muhammad Shaaban for their suggestions, advice, and support.
    [Show full text]
  • Video - Dive Into HTML5
    Video - Dive Into HTML5 You are here: Home ‣ Dive Into HTML5 ‣ Video on the Web ❧ Diving In nyone who has visited YouTube.com in the past four years knows that you can embed video in a web page. But prior to HTML5, there was no standards- based way to do this. Virtually all the video you’ve ever watched “on the web” has been funneled through a third-party plugin — maybe QuickTime, maybe RealPlayer, maybe Flash. (YouTube uses Flash.) These plugins integrate with your browser well enough that you may not even be aware that you’re using them. That is, until you try to watch a video on a platform that doesn’t support that plugin. HTML5 defines a standard way to embed video in a web page, using a <video> element. Support for the <video> element is still evolving, which is a polite way of saying it doesn’t work yet. At least, it doesn’t work everywhere. But don’t despair! There are alternatives and fallbacks and options galore. <video> element support IE Firefox Safari Chrome Opera iPhone Android 9.0+ 3.5+ 3.0+ 3.0+ 10.5+ 1.0+ 2.0+ But support for the <video> element itself is really only a small part of the story. Before we can talk about HTML5 video, you first need to understand a little about video itself. (If you know about video already, you can skip ahead to What Works on the Web.) ❧ http://diveintohtml5.org/video.html (1 of 50) [6/8/2011 6:36:23 PM] Video - Dive Into HTML5 Video Containers You may think of video files as “AVI files” or “MP4 files.” In reality, “AVI” and “MP4″ are just container formats.
    [Show full text]
  • View the Slides
    Retrofitting Fine Grain Isolation in the Firefox Renderer Shravan Narayan, Craig Disselkoen, Tal Garfinkel, Nathan Froyd, Eric Rahm, Sorin Lerner, Hovav Shacham, Deian Stefan 1 We use browsers for everything Email, IM, conferences, meetings etc. 2 Third-party libraries make this possible Browsers cannot implement every feature from scratch Browsers use third-party libraries • Used to render audio, video, images etc. Large number of supported formats and libraries • Images – JPEG, PNG, GIF, SVG, BMP, APNG, ICO, TIFF, WebP • Video – H.264, VP8, VP9, Theora • Audio – MP3, WAV, AAC, Vorbis, FLAC, Opus 3 Bugs in libraries can compromise browsers (2018 Pwn2Own) 4 How do browsers deal with bugs in libraries? Traditionally: Coarse-grain renderer isolation • Goal: protect system from browser compromise • Isolates the renderer (code handling untrusted HTML, images, JavaScript) More recently: Site isolation • Goal: protect one site from another • Isolates different sites from each other • E.g., *.google.com is isolated from *.zoom.us 5 Why Site Isolation is not enough Real sites rely on cross origin resources • 93% of sites load cross-origin media • Lots of cross origin jpegs • Bug in libjpeg ⇒ renderer compromise Attacker may be able to host untrusted content on same origin • Malicious media on Google Drive ⇒ compromised renderer • Allows access victim’s Drive files 6 We need fine grain isolation evil.com libjpeg libvpx libogg Firefox Isolate/sandbox media libraries like libjpeg • Bugs in libjpeg should not compromise the rest of Firefox 7 We know how to do this! 1. Pick an isolation/sandboxing mechanism • Process isolation Browser Sandbox Sandbox Sandbox • In-process: Native Client, WebAssembly memory memory memory memory libjpeg libvpx libogg 2.
    [Show full text]
  • Encoding Parameters Prediction for Convex Hull Video Encoding
    Encoding Parameters Prediction for Convex Hull Video Encoding Ping-Hao Wu∗, Volodymyr Kondratenko y, Gaurang Chaudhari z, and Ioannis Katsavounidis z Facebook Inc., 1 Hacker Way, Menlo Park, CA 94025 Email: [email protected]∗, [email protected], [email protected], [email protected] Abstract—Fast encoding parameter selection technique have BD-rate loss. It uses a faster encoder or the same encoder been proposed in the past. Leveraging the power of convex hull with faster speed settings to perform analysis on the video video encoding framework, an encoder with a faster speed setting, sequence first, to determine the optimal encoding parameters or faster encoder such as hardware encoder, can be used to determine the optimal encoding parameters. It has been shown to be used with a slower encoder with better compression that one can speed up 3 to 5 times, while still achieve 20-40% efficiency. Although this fast selection technique can achieve BD-rate savings, compared to the traditional fixed-QP encodings. similar savings with a fraction of the computations, limitations Such approach presents two problems. First, although the are still present. First of all, the few percent loss in BD-rate is speedup is impressive, there is still ∼3% loss in BD-rate. not negligible in the case of higher-quality contents, where a Secondly, the previous approach only works with encoders implementing standards that have the same quantization scheme much higher target quality and/or target bitrate is often used. and QP range, such as between VP9 and AV1, and not in the Secondly, the quantization scheme of the fast encoder, used scenario where one might want to use a much faster H.264 for analysis, and that of the slow encoder, used for producing encoder, to determine and predict the encoding parameters for the actual deliverable encodings, need to be the same in order VP9 or AV1 encoder.
    [Show full text]