H.264 Encoding Parameters

Encoding for Multiple Screen Delivery

Jan Ozer www.streaminglearningcenter.com @janozer [email protected]/ 276-238-9135

Questions

 Most information derived from 0For more information, check out the book  http://bit.ly/Ozer_multi  Published, April 2013

 Terms and terminology  Producing H.264  Finding the optimal video configuration  Introduction to adaptive streaming  Choosing a delivery technology  Producing for adaptive streaming  If you see red, pay attention

Terms and Terminology

 Codec/compression  Bandwidth  Resolution  Data rate  Bitrate control  Container formats, program and transport streams

 Compression  Used to shrink the size of video/audio still images  Common codecs  Video - H.264/AVC, H.265/HEVC, WebM, MPEG-2  Audio - MP3, AAC, WMV  Still image - JPG, PNG, GIF  Codecs - all of the above

 Any technology that COmpresses in the studio, then DECompresses in the field

Why is Compression Important?

 To achieve the target data rate, you have to compress  Compression is “lossy,” the more you compress, the more you lose  This is immutable

 What is bandwidth?  Viewer’s connection speed  Why is it important?  Controls your viewer’s ability to retrieve and play video smoothly  Higher delivery bandwidths mean higher data rates, which means better quality

Bandwidth - Where Are We?

 Viewer’s connection speed to the Internet  Average download speeds per Akamai State of the Internet:  US - 7,611 kbps  Canada - 6,607 kbps  Mexico - 2,941 kbps  UK - 6,470 kbps  France - 4,804 kbps  So why does CNN max out at 1,200 kbps?

 Modem connection speeds (28.8/56 kbps)  Make the video fit  Broadband  Make it good enough  Make it affordable  CNN is at 1,200 kbps; not to fit pipes, but because it’s “good enough” and affordable  Cellular  Make it fit

About Resolution

 What is it?  Actual pixels in the file

Acquisition Distribution  DV - 720x480  DVD - 720x480  720p - 1280x720  Streaming - 1080p or  1080i/p - 1920x1080 smaller  iPod/iPhone/iPad - 320x240 - 1080p

Why is Resolution Important

 Most video starts life at 720p or higher  Streamed at 320x180 to 1080p  Resolution is key quality factor  At a set bit rate, increasing resolution degrades quality because you must compress more pixels  Can’t say that 1,200 kbps is “adequate” without knowing resolution  For 640x360x25, it’s adequate  For 1080p, it’s not

 Amount of data per second of video  Largely determines video quality  At static resolution and frame rate, increasing bit rate increases quality  Largely determines bandwidth costs

Constant vs Variable Bit Rate 800 kbps VBR

500 kbps CBR

300 kbps

Low Motion Moderate Motion Low Moderate High Motion Motion Motion

Constant vs Variable Bit Rate

 Constant Bit Rate (CBR)  One bit rate applied to entire video, irrespective of content  Pros:  Computationally easy  Fast - one pass will do it  Cons: Doesn't optimize quality

 Variable Bit Rate (VBR)  Dynamic bit rate matches motion in video  Pros: Best quality  Cons:  Need two or more passes  Can produce stream with significant variability

How do I Produce the Best Quality CBR?

 Use 2-pass CBR when available  Scans file (like VBR), but packs data into a consistent stream  Best of both worlds when available  1-pass of live or draft work

Copyright © 2013 Jan Ozer, All Rights Reserved How Do I Produce the Optimal VBR File?  2 passes or more  Use “Constrained”  Constrains to data rate to specified max  Set Target and Max/Min  Overall target – 500 kbps  Max/Peak bit rate – how high rate can go when varying  Rule of thumb is 1.5 - 2X of target  If minimum setting, use .5x

When Should I Use VBR/CBR?

VBR CBR  Constrained VBR (usually  CBR for live, particularly 2X) for most streaming when limited outbound applications bandwidth  Broadband delivery can  CBR or constrained VBR handle spikes for adaptive streaming  Constrained VBR for (more later) virtually all progressive delivery  Constrained VBR for most cellular, though not universal

 Container format:  A meta-file format whose specification describes how data and metadata are stored  Examples: MPG, MP4, MOV, F4V, WMV, FLV  Transport stream:  Transport stream specifies a container format encapsulating packetized elementary streams, with error correction and stream synchronization features for maintaining transmission integrity when the signal is degraded (e.g. - broadcast)  Examples: .ts,

Why Do We Care?

 When producing for single file or adaptive delivery, you must choose the right container format/transport stream for your application  Streaming  Flash - MP4, FLV, F4V, MOV - H.264  HTML5 - MP4 - H.264, WebM - VP8  iDevices - MP4, MOV - H.264  Adaptive - HTTP Live Streaming - .ts

 Should be 9:25

Download presentation handouts At www.streaminglearningcenter.com

Producing H.264

 Introduction to H.264  H.264 encoding parameters

 The MPEG-4 specification  The MPEG-4 codec  MPEG-4 spec audio options  The H.264 codec  H.264 Basics

Preload http://www.doceo.com/rethink2/Main.html

MPEG-4 Specification

 Introduced in 1998 by ISO/IEC Moving Picture Experts Group  28 “Parts” within specification  Part 2 - MPEG video codec (the MPEG-4 codec)  Part 3 - MPEG-4 audio (AAC, etc)  Part 10 - Advanced Video Coding (AVC/H.264)  Part 14 - container format (MP4)

 Used only for low power devices: Two profiles:  Simple Profile - very low power, low bandwidth applications  Advanced Simple Profile -  Check specs on target devices Simple plus:  Will refer to MPEG-4 encoding  Support for "MPEG"-style only in mobile segments quantization  Never use for computer  Support for B pictures (a.k.a. B-frames) playback  Motion compensation

MPEG-4 Audio

 AAC-Low Complexity (AAC-LC)  The most basic and most broadly compatible  In my tests, indistinguishable from HE AAC/HE AACv2  High Efficiency AAC (2003)  Also called AAC+ and aacPlus  High Efficiency AACv2 (2006)  Also called enhanced AAC+, aacPlus v2 and eAAC+

 Recommendations  Usage dictated by specs of target device  When producing for multiple devices, use lowest common denominator  When in doubt, choose AAC-LC  Most compatible on the playback side  Universally available on MPEG-4/H.264 encoders

What is H.264?

 Adapted by ISO and ITU  Telephony/cellular  TV - consumer electronics  Computer electronics  Only codec adopted by top three streaming providers (Apple, Adobe, Microsoft) Streamcrest Associates http://www.streamcrest.com/SDF%20Final1.pdf

 For free Internet video (e.g. no subscription or pay per view), free in perpetuity  Thank you Google (WebM)  Still technically an obligation to sign a license, but there are no teeth and no motivation to enforce  For subscription or PPV, there may be a royalty obligation  Check www.mpeg-la.com

The Many Wrappers of H.264

 .MP4 - official MPEG-4 wrapper  .M4V - Apple’s variant for iTunes and devices  .MOV - H.264 file for editing or QuickTime delivery  .F4V - H.264 for Flash  .3GP - (not shown) - phone  .MPG - H.264 in MPEG-2 transport stream

 The basics  I, B and P-frames  Search-related options  Miscellaneous options

H.264 Encoding - Basics

 Profiles and Levels  Entropy encoding

 “Define a set of coding tools or algorithms that can be used in generating a bitstream”

Profiles and Playback Compatibility

Profile Pre- Android Android iPad 1 iPhone 4- iPad 2/3 Flash/ iPhone 4 3.0 Hardware 5 HTML5 Software Baseline Yes Yes Yes Yes Yes Yes Yes Main No No ? Yes Yes Yes Yes High No No ? No Yes Yes Yes

 Using the wrong profile results in a file incompatible with your target  Need to know profile of target device and choose appropriate profile  Some interesting decisions to make when encoding for multiple device delivery

 Baseline profile - fewest advanced algorithms:  Lowest quality stream  Easiest to encode and particularly decode  High profile – most advanced algorithms  Highest quality stream  Hardest to encode/decode

Why Do Profiles Exist?

 Meeting point for hardware vendor and video producer  Example: Original video capable iPod is baseline only: why?  Lower power (cheaper) CPU  Less power consumption  Video producer: to produce that will play on iPod, use Baseline profile

 Don’t exceed profile of target device  Exclusively a concern with mobile  Computers and OTT devices can play High profile (any level)

Required Profiles for iDevices

Original iPod iPod nano/ iPhone 4 iPhone 4S IPhone 5/ (to-5g) classic/ iPod /iPod touch 4/ New iPad touch/iPhone iPad 1&2 to V4 Video codec H.264 H.264 H.264 H.264 H.264 Profile/level Baseline to Baseline to Main to Level High to Level High to Level Level 1.3 Level 3.0 3.1 4.1 4.1 Max video data rate 768 kbps 2.5 Mbps 14 Mbps 50 Mbps 62.5 mbps for level Max video resolution 320x240 640x480 720p 1080p 108p Frame rate 30 fps 30 fps 30 fps 30 fps 30 fps Audio codec AAC-LC AAC-LC AAC-LC AAC-LC AAC-LC Max audio data rate 160 kbps 160 kbps 160 kbps 160 kbps 160 kbps Audio params 48 kHz, stereo 48 kHz, stereo 48 kHz, stereo 48 kHz, stereo 48 kHz, stereo  Four playback classes  Must configure video appropriately to play on target

 These recommendations are least common denominator software-only playback  Most devices probably more capable (no table because too many manufacturers)  For 100% compatibility, use these

Required Profiles for Windows 8 Phones

 H.264-encoded video in MP4 container  Files s/not exceed params for 7x27a CPU.

http://msdn.microsoft.com/en-us/library/windowsphone/develop/ff462087%28v=vs.105%29.aspx

Profile Pre- Android Android iPad 1 iPhone Win 8 iPad PCs/ OTT iPhone 3.0 Hardwar 4-5 Phone 2/3 note 4 Software e books Baseline Yes Yes Yes Yes Yes Yes Yes Yes Yes Main No No ? Yes Yes Yes Yes Yes Yes High No No ? No Yes Yes Yes Yes Yes

 Using the wrong profile produces a file incompatible with your target  Need to know profile of target device and choose appropriate profile

Big Decision #1

 One set of files – all Baseline – that plays everywhere?  Cheapest, easiest  Separate files:  Baseline – Very old iOS and Android  Main – Old iOS and Android  High – new iOS, computers and OTT  Optimal quality, but more encoding, storage and administrative

 Consulting project – questions  “Is it worth creating two sets of files, one for mobile (with Baseline and Main), the other for Desktop (High)?  “How much difference quality?  Ran comparisons at aggressive data rates:  720p@800 kbps (Burberry is at 3 mbps)  640x480 @ 500 kbps (CNN is at 1200 kbps)  640x360 @ 240 kbps

Does Profile Matter?

 http://www.streaminglearningcenter.com/articles/h264-in-a-mobile-world-adios-to- the-main-and-high-profiles.html  http://www.doceo.com/rethink/Main.html  http://www.doceo.com/rethink2/Main.html

http://bit.ly/h264profiles

The Bottom Line

 At 640x360@240 kbps:  Some noticeable differences appear at these extreme encoding parameters

 Key takeaway:  The quality difference may be less than you would expect  Two sets of files, one for mobile, one for desktop/OTT?  Test quality difference at most extreme rates and see if the quality difference warrants it  This client used all Baseline

What are H.264 Levels?

 “Constrains key parameters in the bitstream”

 Critical not to exceed level of target device  Devices won’t load file encoded beyond max supported level  So, make sure encoding parameters don’t exceed level (most templates do this)

H.264 Levels – Desktops

 Flash/HTML5 players will load and attempt to play all profiles/all levels  Exceeding a certain level won’t kick the file out like devices  If resolution too large, file won’t play smoothly  Up to 848x480 will play smoothly on most computers  720p may not play on some older/slower computers . Don’t use 720p and higher unless you’re offering adaptive (or otherwise multiple) streams

 CABAC (Context-adaptive binary arithmetic coding)  More efficient (e.g. better quality), but harder to decode  Main/High profile only  CAVLC (Context-adaptive variable-length coding)  Less efficient, easier to decode

What Would YouTube Do?

 High Profile  CABAC

 Caveats:  Presented differently by each encoding tool  Will only cover most important and most common parameters

What are I, B and P Frames?

 I-Frame - encoded without reference to other frames (also called Key Frames)  P - looks backward to I and P frames (predictive)

 B - looks forward and backward to previous I and P frames (Bi-directional interpolated)  No frames refer to B-Frame (most of the time)

 Least efficient – so largest (which is bad)  But, key frames enhance interactivity  All playback starts on a key frame  When seeking to random frame, must start playback at key frame  Maximum interval should be 5-10 seconds  Key frames "reset" quality:  Useful at scene changes  Enable natural key frames or key frames at scene changes

Configuring Key Frames

 Rules change completely when encoding a group of files for adaptive streaming  Shorter key frame interval  Disable scene change detection

 Main/High profiles only  The most “efficient” frame  So improves quality (comparisons to come)  Hardest to decode  Decoder has to have all referenced frames in memory to decode B-frame  Frame usually delivered out of order, which also complicates playback

Typical B-Frame Encoding Parameters

 Number is number of B frames between I and P-Frames; (IBBBPBBBPBBBPBBBP)  3 is recommended  Reference frames (both P and B-frames)  Number of frames searched for redundancies  5 is recommended setting

 YouTube does NOT use B- frames with many H.264 files it produces  Though they are used with some encodes  My tests in Episode with 0, 1, 3 and 5 B-frames  640x480@468 kbps  1280x720@800 kbps Showed very, very little difference (and then only in HD clip)

HD Comparisons

Do B-Frames Matter?

 3 B-frames/5 reference frames is a good general purpose recommendation  Don’t spend a lot of time worrying about “optimal” B-frame settings, because for most videos, it probably wouldn’t make that much of a difference

Advanced Frame Options

 IDR frames  Two kinds of key frames  IDR key frame – no frame can reference frame before key frame  Non-IDR key frames – B- and P-frames can reference frames before key frame  All key frames should be IDR key frames in streaming files

 Adaptive B-frames – adapt frame type to improve quality  Always enable  Reference B-pictures – Allows P-frames to reference B-frames  Enable for optimal quality  Pyramid B-frame – allows B-frames to reference other B-frames  Enable for optimal quality

Search Related Options

 Searching for redundancies; multiple factors  Search shape (8x8/16x16) – size of shape used for searching (smaller is more accurate  Sub-pixel mode – (full/half/quarter pixel) – smaller is more accurate  Fast - trades encoding speed for quality

 Divides frame into segments to speed encoding  Assign a CPU to each slice  But: Search for redundancies occurs within each slice  If lots of inter-slice motion, encoder won’t find these redundancies  Using slices can reduce quality  Set to lowest value (either 0 or 1)

Reflections on the Utility of H.264 Tweaking

Less is more  Many high-end hardware encoders offer minimal H.264 config options  Elemental (right)  Envivio  Make most configuration decisions for you

 Profile matters very little  B-frames matters very little

Where to Focus Your Attention?

 Get the config right  Rez/data rate/key frames  Choose the right H.264 codec (x264 > MC > Apple)  You’ve captured 98% of potential quality

As data rates Tweaking Relevance increase, the impact of H.264- related tweaking goes down

Low Data Rate High

Synthesis

 Focus heavily on finding the optimal configuration for your video  Don’t mess up anything else  De-interlacing  Aspect ratio  Bitrate control  Then, if you have the time, worry about tweaking to get the last 1-5%

 Should be 10:20

Download presentation handouts At www.streaminglearningcenter.com

Finding the Optimal Configuration

 Understanding bits/pixel  Applying bits/per pixel

 Which of these firms would you take video- related configuration advice from?  Kind of tough to compare, isn’t it?

Why Bits/Pixel is Relevant

 This adds some data, but how do they really compare compression-wise?

 How about now?  Accenture’s bits/pixel is almost 7 times higher

Bits Per Pixel  How much data applied to each pixel in video file  To calculate:  Data rate/(fps*pixels)  Or, get MediaInfo  Calculates for you, mediainfo.sourceforge.net/en plus lots of other file details

 In general  At 640x360 or smaller, values above .2 almost clearly a waste  CNN is about .121 for low motion  ESPN maxes at .201, but that’s for sports  Values drop as videos get larger (codecs work more efficiently at higher resolutions)

What’s the Magic Number?

Corporate - Conservative

Corporate – Aggressive

Bottom Line

 You should know the bits/pixel for each video that you produce  If too low – quality suffers  If too high, may not stream smoothly and bandwidth costs (if a factor) may be unnecessarily high

Questions on Bits Per Pixel

 Should be 10:35

Download presentation handouts At www.streaminglearningcenter.com

 What it is  How the various alternatives work  RTMP (server-based)  HTTP (no server)  ID existing technologies  Transmuxing

What is Adaptive Streaming?

 One stream in, multiple streams out  Streams switched transparently to adapt to factors like:  Changing delivery bandwidths (avoid hard stops)  CPU utilization at client (avoid frame drops)

 Match stream quality to connection and playback platform used by viewer

Mobile

Desktop

OTT

Technology Overview

 Two types of systems  Server-based (Flash, RTMP)  HTTP  HTTP Live Streaming (HLS)  Smooth Streaming  HTTP-based Dynamic Streaming (HDS)  Dynamic Adaptive Streaming over HTTP (DASH)

1. Uses standard 3. Player monitors: MP4/F4V files - Buffer as encoded (no - CPU utilization chunking) 4. Requests 2. Persistent different stream connection when needed between player 5. Server sends and server different stream

Issues with RTMP

 Server required:  May increase cost (because servers cost $$)  May limit scalability (if not enough servers for all viewers)  RTMP packets can get blocked by Firewalls  RTMP packets do not take advantage of HTTP caching mechanisms  Less efficient for popular videos  Some CDNs are dropping support for RTMP

 Don’t require a streaming server  Better scalability  Lower cost  Ubiquitous CDN support  Use HTTP protocol so:  No firewall issues  Will leverage standard HTTP caching structures  More efficient for popular videos

Bottom Line

 RTMP-based Dynamic Streaming has been most popular technology  Market is moving towards HTTP

 Encoder creates:  Player  Chunked video files  Retrieves index  Index file (M3U8)  Monitors buffer  Sends to HTTP Server  Changes streams as necessary

The Manifest File Structure

 All HTTP Technologies work the same way  Create chunked data files (or segments within longer file)  Create index files (also called manifest)  Placed on HTTP server  Player side  Monitors playback buffer and (sometimes) CPU use  Changes streams as necessary  Uses index files to find the right files

Adobe HTTP Dynamic Streaming (HDS)

.F4F Media file chunks Into MP4 Packager .F4M Manifest files that tell player where to find file chunks

 Player driven system  Player monitors heuristics  Retrieves different stream as necessary

Chunked content file (ismv)

Dynamic Adaptive Streaming over HTTP (DASH)

 Like all HTTP-based technologies, it has  Fragmented video chunks (or single file with segments)  Manifest files  Supports multiple codecs, container formats and manifest file formats

 Lots of smoke, but no real fire  No native support on any platform  Plug-ins available on most platforms  Microsoft/Google working on browser support

Where is DASH?

 DASH committee working on H.264-only sub- spec that Adobe/MS will endorse  Apple/Google (Android) have not announced support  Royalty picture not cleared  DASH for computers/mobile is very much a work in process  Likely will find native support in OTT platforms first (because closed system), but HLS is very entrenched

 Several pre-fab video players (JW Player) are adding HLS support  (As you’ll see) this makes HLS available on computers, mobile and OTT  If these become pervasive, HLS could become the dominant format over DASH

Transmuxing Technologies

 Perspective: most producers serve multiple clients  Desktop - Flash (or Silverlight or DASH) AND  Mobile - HLS for iOS/Android (maybe)  OTT - HLS  In the past, that meant two separate encoding and delivery workflows  Separate encoder for each target  Now, multiple streaming that:  Input one set of streams, and  “Transmux” to create second set for different target

 Options  Technology providers - Wowza, Real, Microsoft, Adobe

Transmuxing

 Not re-encoding  “re-wrapping” file into a different container format  Creating necessary manifest files  Delivering files to HTTP server  All in real time

 Key point: If serving multiple targets, you must produce using lowest common denominator H.264 encoding parameters  Must be all baseline for streams targeted for mobile  If so, can use one set of files to deliver to all three groups  Bottom line: If serving multiple targets, must consider transmuxing

Questions on ABR?

 Should be 11:10

Download presentation handouts at www.streaminglearningcenter.com

 Single video file, no DRM, live or captioning  HTML5 with Flash fallback should work for browser-based playback and OTT  If you need adaptive, live, DRM or captions, choosing a path is much more challenging

Distributing Single Files

 Desktop  Flash/HTML5/  Mobile  100% playback of HTML5/H.264  OTT  Each service has own delivery structure, typically MP4 files are recommended

Flash Player Playback Specifications

Video codec H.264 ONLY, not MPEG-4 Profiles Baseline, Main, High Audio codec AAC, AAC-LC, HE-AAC Container formats MP4 (preferred), F4V, M4V, M4A, MOV, 3GP

Understanding the HTML5 Playback Environment

 HTML5 introduction  HTML5 penetration - desktop  HTML5 codec support

 HTML5’s key benefit  Death to plug-ins everywhere (Particularly Flash)  How it works  Instead of obtaining decoders for H.264 and other codecs in plug-ins  Browsers supply players and decoders  Use

HTML5 Operation

 For video tag to work: 1. Browser must be HTML5 compatible 2. Browser must supply appropriate decoder  Two key markets  Desktop  Mobile

Not HTML5 Comp+

 So, at best, HTML5 is 69% of the market on desktops

http://www.netmarketshare.com/browser-market-share.aspx?qprid=2&qpcustomd=0

HTML5 Codec Support

 H.264 Camp – IE, Safari, Chrome  Google announced plans to remove H.264 1/2011, hasn’t yet  WebM camp  Native – Firefox, Chrome, Opera  Via plug-ins – IE 9/Safari http://diveintohtml5.info/video.html

Flash Player Playback Specifications

Video codec H.264 ONLY, not MPEG-4 Profiles Baseline, Main, High Audio codec AAC, AAC-LC, HE-AAC, HE-AACv2 Container formats MP4 (otherwise, may call a different player)

Desktop Analysis

 HTML5 can access 69% of desktops (maximum)  To access, you must produce two file versions  H.264 – Chrome, Safari, IE9  Chrome leaving? In 1/2011, Google said Chrome would drop H.264; it hasn’t yet  Firefox coming? Mozilla adding support for Android Firefox 17, so H.264 may be coming to Firefox soon (at least of OSs with H.264 support, which doesn’t include Windows XP)  Web M – Firefox post 3.6, Opera post 10.6, Chrome

 HTML5 - 100% support in iOS, Android, Windows 8 phones and tablets  H.264 - 100% support, though, as seen, profiles and levels differ

Distributing to OTT

 Channel schema varies by device (HTML5 isn’t relevant)  All devices play H.264 video up to (at least) 1080p, High Profile  Any video configured for mobile/computers should play on OTT

 HTML5 with Flash Fallback  HTML5 to all mobile devices  HTML5 to all computers running HTML5 compatible browsers  Fallback to Flash for non-HTML5 compatible browsers  Can encode WebM & H.264 for optimal support  Encode H.264 only and have Mozilla/Opera play Flash

What is Flash Fallback?

Video tag

MP4 WebM

If all else fails, play Flash

From Sorenson Squeeze 9

 When you need:  Adaptive streaming  Live streaming  DRM  Closed captioning

Targeting Desktops

 Requirements may extend beyond just video  DRM, captions (public)  Multicast/P2P (intranets)

 Flash or  HLS  Use JW Player in website  So long as Flash is installed, will play HLS streams  One set of streams for desktop, iOS, OTT

http://bit.ly/Wijering

Targeting Mobile

* Android has issues playing back HLS - bit.ly/Rayburn_android

 iOS – HLS  Android  Universal coverage – AIR application  HLS – 56% penetration and aspect ratio issues  Windows Phone 8  Smooth

Targeting OTT

 Flash is non-existent  HLS is ubiquitous and generally the preferred technology  Can implement smooth in most platforms if preferred

 Library that sits above Flash Player  Adds HLS (this summer) DASH (end of year)  Includes SDKs for iOS/Android  iOS SDK will not include DASH  Android SDK includes HLS stack that avoids HLS playback issues on that platform  OTT support for both DASH/HLS by end of year  Even with Primetime (licensing fees apply), DASH will not be available on all platforms

Summary

 HLS is the only technology available on every target platform  Today, DASH is not available on any platform  Even though DASH is technically superior to HLS, it’s hampered by:  The lack of player support  The potential for royalties  Seeming lack of interest from Apple and Google (for Android, not Chrome)

 Should be 11:35

Download presentation handouts at www.streaminglearningcenter.com

Producing Adaptive Streams

 Basic configuration options  Number of streams  Resolutions/data rates  H.264 parameters  Encoding adaptive streams  Key frame settings  Bitrate control  Audio settings

 One set of streams for all devices?  Customize streams for each class

What We Know  Most producers customize  Turner Broadcasting – NBA League Pass

 Big sites tend to customize  Large network in the US  Separate streams for web, mobile, OTT  Movie site in US  Ditto, though some crossover  Smaller sites  Tend to use one set of files; all baseline

Why the Same Set of Streams?

 Simplifies encoding (encode fewer streams)  Can transmux  Create one set for Mobile  Transmux to Flash/OTT

 There is no right or wrong way  I recommend:  Attempt to find one set of streams that satisfies all using Baseline profile  Test to see if Main or High delivers noticeably higher quality  If so, customize for different targets  If not, go with one set of streams

Mobile First

 Slowest connection, lowest quality  Configure for best service to mobile  Try to configure at same resolutions as low end computer targets

 At least one stream for each window size in web site (HBO)  Try to use same configuration as mobile

Then High End Mobile, Desktop and OTT

 Full screen viewing on all devices  Highest quality streams that you can afford

 At least one stream for each playback window  More streams required for HD than SD  SD – usually 3, 4 maximum  HD – up to 11  More for entertainment than education/business  Entertainment – about the experience  Business – it’s about making sure the viewer can watch the stream  More for subscription than general entertainment  Provide more options when viewer is paying

What Resolution?

 Never encode at larger than source  Scaling upwards degrades quality  Resolution should match playback window whenever possible  Optimal quality and playback efficiency  If you can’t match playback window, it’s better to scale up to larger resolutions than to scale down to smaller  More efficient playback (GPU assisted)  More efficient bandwidth consumption

 Considerations  Must be sufficiently far apart to:  Avoid constant switching  Make a meaningful quality difference  Will vary with stream size  Smaller gaps at lower bandwidths  Larger gaps at higher bandwidths

What Data Rates?

 MTV Schema

 Now we know stream count, resolution and data rate  How do we customize encoding for adaptive?  Key frame settings  Bit rate control  Audio parameters

Key Frame Interval

 Why important - RTMP Flash – stream switches occur at key frame  Key frame location must be identical in all streams  Use same interval  Disable scene change detection  Typically shorter (2-5 seconds) to enable more responsive switching

 Why important – HTTP technologies - key frame must be first frame of every chunk  Key frame interval must divide evenly into chunk duration  If 9-sec. chunks, use key frame interval of 3 sec  As with RTMP, key frame location must be identical in all streams, so:  Use same interval  Disable scene change detection

VBR vs. CBR

 Most conservative - CBR  Or constrained VBR  Max data rate 125-150% of target

 Most conservative-same parameters for all files  Popping can occur if audio parameters change  But, does’t optimize experience at higher bit rates

In General - Audio Parameters

 If you do switch audio parameters  Switch from stereo to mono at same per channel sampling rate and bit rate  From 64 kbps/44 kHz/16-bit/mono  To 128 kbps/44 kHz/16-bit/stereo  Test to ensure no artifacts when switching streams

 Abhinav Kapoor, “Live dynamic streaming with Flash Media Server 3.5,” (adobe.ly/kapoorlivefms)  Maxim Levkov, “Video encoding and transcoding recommendations for HTTP Dynamic Streaming on the Flash Platform,” (adobe.ly/Levkovhttp)  Larry Bouthillier, “How to do Dynamic Streaming with Flash Media Server,” website (bit.ly/fmsdynamic)  “Tutorial: On-demand HTTP Dynamic Streaming,” Adobe website (bit.ly/ondemanddynamic)  “Encoding Guidelines Dynamic Streaming for Flash over HTTP,” Akamai website (bit.ly/akamaiwhitepaper) .

HTTP Flash Resources

 Best sources:  Maxim Levkov - Adobe - very detailed recommendations  adobe.ly/Levkovhttp  Akamai - White paper:”Encoding Guidelines for Dynamic Streaming for Flash over HTTP”  bit.ly/akamaiwhitepaper

 Questions?  Should be 11:55

Download presentation handouts at www.streaminglearningcenter.com