Encoding for Multiple Screen Delivery Jan Ozer www.streaminglearningcenter.com @janozer [email protected]/ 276-238-9135 Questions Most information derived from 0For more information, check out the book http://bit.ly/Ozer_multi Published, April 2013 Copyright © 2013 Jan Ozer, All Rights Reserved Agenda Terms and terminology Producing H.264 Finding the optimal video configuration Introduction to adaptive streaming Choosing a delivery technology Producing for adaptive streaming If you see red, pay attention Copyright © 2013 Jan Ozer, All Rights Reserved Terms and Terminology Codec/compression Bandwidth Resolution Data rate Bitrate control Container formats, program and transport streams Copyright © 2013 Jan Ozer, All Rights Reserved Compression/Codec Compression Used to shrink the size of video/audio still images Common codecs Video - H.264/AVC, H.265/HEVC, WebM, MPEG-2 Audio - MP3, AAC, WMV Still image - JPG, PNG, GIF Codecs - all of the above Any technology that COmpresses in the studio, then DECompresses in the field Copyright © 2013 Jan Ozer, All Rights Reserved Why is Compression Important? To achieve the target data rate, you have to compress Compression is “lossy,” the more you compress, the more you lose This is immutable Copyright © 2013 Jan Ozer, All Rights Reserved About Bandwidth What is bandwidth? Viewer’s connection speed Why is it important? Controls your viewer’s ability to retrieve and play video smoothly Higher delivery bandwidths mean higher data rates, which means better quality Copyright © 2013 Jan Ozer, All Rights Reserved Bandwidth - Where Are We? Viewer’s connection speed to the Internet Average download speeds per Akamai State of the Internet: US - 7,611 kbps Canada - 6,607 kbps Mexico - 2,941 kbps UK - 6,470 kbps France - 4,804 kbps So why does CNN max out at 1,200 kbps? Copyright © 2013 Jan Ozer, All Rights Reserved Paradigm Shift Modem connection speeds (28.8/56 kbps) Make the video fit Broadband Make it good enough Make it affordable CNN is at 1,200 kbps; not to fit pipes, but because it’s “good enough” and affordable Cellular Make it fit Copyright © 2013 Jan Ozer, All Rights Reserved About Resolution What is it? Actual pixels in the file Copyright © 2013 Jan Ozer, All Rights Reserved Resolutions That We Know and Love Acquisition Distribution DV - 720x480 DVD - 720x480 720p - 1280x720 Streaming - 1080p or 1080i/p - 1920x1080 smaller iPod/iPhone/iPad - 320x240 - 1080p Copyright © 2013 Jan Ozer, All Rights Reserved Why is Resolution Important Most video starts life at 720p or higher Streamed at 320x180 to 1080p Resolution is key quality factor At a set bit rate, increasing resolution degrades quality because you must compress more pixels Can’t say that 1,200 kbps is “adequate” without knowing resolution For 640x360x25, it’s adequate For 1080p, it’s not Copyright © 2013 Jan Ozer, All Rights Reserved What is Data Rate? Amount of data per second of video Largely determines video quality At static resolution and frame rate, increasing bit rate increases quality Largely determines bandwidth costs Copyright © 2013 Jan Ozer, All Rights Reserved Constant vs Variable Bit Rate 800 kbps VBR 500 kbps CBR 300 kbps Low Motion Moderate Motion Low Moderate High Motion Motion Motion Copyright © 2013 Jan Ozer, All Rights Reserved VBR vs. CBR Copyright © 2013 Jan Ozer, All Rights Reserved Constant vs Variable Bit Rate Constant Bit Rate (CBR) One bit rate applied to entire video, irrespective of content Pros: Computationally easy Fast - one pass will do it Cons: Doesn't optimize quality Copyright © 2013 Jan Ozer, All Rights Reserved Constant vs Variable Bit Rate Variable Bit Rate (VBR) Dynamic bit rate matches motion in video Pros: Best quality Cons: Need two or more passes Can produce stream with significant variability Copyright © 2013 Jan Ozer, All Rights Reserved How do I Produce the Best Quality CBR? Use 2-pass CBR when available Scans file (like VBR), but packs data into a consistent stream Best of both worlds when available 1-pass of live or draft work Copyright © 2013 Jan Ozer, All Rights Reserved How Do I Produce the Optimal VBR File? 2 passes or more Use “Constrained” Constrains to data rate to specified max Set Target and Max/Min Overall target – 500 kbps Max/Peak bit rate – how high rate can go when varying Rule of thumb is 1.5 - 2X of target If minimum setting, use .5x Copyright © 2013 Jan Ozer, All Rights Reserved When Should I Use VBR/CBR? VBR CBR Constrained VBR (usually CBR for live, particularly 2X) for most streaming when limited outbound applications bandwidth Broadband delivery can CBR or constrained VBR handle spikes for adaptive streaming Constrained VBR for (more later) virtually all progressive delivery Constrained VBR for most cellular, though not universal Copyright © 2013 Jan Ozer, All Rights Reserved Container Formats, Transport Streams, Program Streams Container format: A meta-file format whose specification describes how data and metadata are stored Examples: MPG, MP4, MOV, F4V, WMV, FLV Transport stream: Transport stream specifies a container format encapsulating packetized elementary streams, with error correction and stream synchronization features for maintaining transmission integrity when the signal is degraded (e.g. - broadcast) Examples: .ts, Copyright © 2013 Jan Ozer, All Rights Reserved Why Do We Care? When producing for single file or adaptive delivery, you must choose the right container format/transport stream for your application Streaming Flash - MP4, FLV, F4V, MOV - H.264 HTML5 - MP4 - H.264, WebM - VP8 iDevices - MP4, MOV - H.264 Adaptive - HTTP Live Streaming - .ts Copyright © 2013 Jan Ozer, All Rights Reserved Questions on Terms and Terminology? Should be 9:25 Download presentation handouts At www.streaminglearningcenter.com Copyright © 2013 Jan Ozer, All Rights Reserved Producing H.264 Introduction to H.264 H.264 encoding parameters Copyright © 2013 Jan Ozer, All Rights Reserved Introduction to H.264 The MPEG-4 specification The MPEG-4 codec MPEG-4 spec audio options The H.264 codec H.264 Basics Preload http://www.doceo.com/rethink2/Main.html Copyright © 2013 Jan Ozer, All Rights Reserved MPEG-4 Specification Introduced in 1998 by ISO/IEC Moving Picture Experts Group 28 “Parts” within specification Part 2 - MPEG video codec (the MPEG-4 codec) Part 3 - MPEG-4 audio (AAC, etc) Part 10 - Advanced Video Coding (AVC/H.264) Part 14 - container format (MP4) Copyright © 2013 Jan Ozer, All Rights Reserved The MPEG-4 Video Codec Used only for low power devices: Two profiles: Simple Profile - very low power, low bandwidth applications Advanced Simple Profile - Check specs on target devices Simple plus: Will refer to MPEG-4 encoding Support for "MPEG"-style only in mobile segments quantization Never use for computer Support for B pictures (a.k.a. B-frames) playback Motion compensation Copyright © 2013 Jan Ozer, All Rights Reserved MPEG-4 Audio AAC-Low Complexity (AAC-LC) The most basic and most broadly compatible In my tests, indistinguishable from HE AAC/HE AACv2 High Efficiency AAC (2003) Also called AAC+ and aacPlus High Efficiency AACv2 (2006) Also called enhanced AAC+, aacPlus v2 and eAAC+ Copyright © 2013 Jan Ozer, All Rights Reserved MPEG-4 Audio Summary Recommendations Usage dictated by specs of target device When producing for multiple devices, use lowest common denominator When in doubt, choose AAC-LC Most compatible on the playback side Universally available on MPEG-4/H.264 encoders Copyright © 2013 Jan Ozer, All Rights Reserved What is H.264? Adapted by ISO and ITU Telephony/cellular TV - consumer electronics Computer electronics Only codec adopted by top three streaming providers (Apple, Adobe, Microsoft) Streamcrest Associates http://www.streamcrest.com/SDF%20Final1.pdf Copyright © 2013 Jan Ozer, All Rights Reserved What's MPEG-4/H.264 Cost? For free Internet video (e.g. no subscription or pay per view), free in perpetuity Thank you Google (WebM) Still technically an obligation to sign a license, but there are no teeth and no motivation to enforce For subscription or PPV, there may be a royalty obligation Check www.mpeg-la.com Copyright © 2013 Jan Ozer, All Rights Reserved The Many Wrappers of H.264 .MP4 - official MPEG-4 wrapper .M4V - Apple’s variant for iTunes and devices .MOV - H.264 file for editing or QuickTime delivery .F4V - H.264 for Flash .3GP - (not shown) - phone .MPG - H.264 in MPEG-2 transport stream Copyright © 2013 Jan Ozer, All Rights Reserved H.264 Encoding Parameters The basics I, B and P-frames Search-related options Miscellaneous options Copyright © 2013 Jan Ozer, All Rights Reserved H.264 Encoding - Basics Profiles and Levels Entropy encoding Copyright © 2013 Jan Ozer, All Rights Reserved What are H.264 Profiles? “Define a set of coding tools or algorithms that can be used in generating a bitstream” Copyright © 2013 Jan Ozer, All Rights Reserved Profiles and Playback Compatibility Profile Pre- Android Android iPad 1 iPhone 4- iPad 2/3 Flash/ iPhone 4 3.0 Hardware 5 HTML5 Software Baseline Yes Yes Yes Yes Yes Yes Yes Main No No ? Yes Yes Yes Yes High No No ? No Yes Yes Yes Using the wrong profile results in a file incompatible with your target Need to know profile of target device and choose appropriate profile Some interesting decisions to make when encoding for multiple device delivery Copyright © 2013 Jan Ozer, All Rights Reserved Quality-Related Expectations Baseline