AWS Media Services - Enabling Advanced Media Workflows

SessionD AWS Media Services - Enabling Advanced Media Workflows

Andrew Thornton GM AWS Elemental APAC

SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Traditional industry chain IS RIGID & YET FRAGILE

QualityShift

MobilityAgility Resiliency Personal

ReliabilityQuality ScalabilityReliable

”

Image courtesy of Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Bob_Iger_crop_2.jpg SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 5 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark AWS Media Services

AWS AWS Amazon S3 Amazon EC2 AWS Elemental Amazon Elastic Amazon Amazon AWS AWS Elemental Storage Gateway Direct Connect MediaConvert Transcoder Polly Comprehend Step Functions MediaTailor

AWS S3 Transfer Amazon Glacier Amazon VPC AWS Elemental Amazon Amazon Amazon Amazon SQS Amazon Snowball Acceleration MediaLive RDS Machine Learning Translate CloudFront

AWS AWS Elemental Amazon EBS Lambda AWS Elemental Amazon Amazon Amazon Amazon SNS AWS WAF Import/ Export MediaConnect MediaPackage DynamoDB Lex Transcribe

Amazon EFS AWS Elemental Amazon Amazon Route 53 MediaStore Rekognition CloudSearch

Ingest/Create Store Process Deliver

Andrew Thornton GM AWS Elemental APAC

• Quality optimisation & lower costs with intelligent encoding. QVBR

• Reducing Latency

• Personalised Ad insertion

• Examples of the role of Machine Learning, AI & IoT in Media workloads • Recognition • Advanced Meta data tagging • Recommendation • ML IoT & Analytics • Live Subtitling with translation 12 SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Intelligent Encoding QVBR

9 8 Bitrate over Time 7 6 5 4 v2.11 3 v2.10 2 1 0 mbps

Very hard Very hard Moderate Moderate Very hard

Information contained in a video is natively VBR Different content and scenes carry varying amounts of information in the video frames

• A content-conscious version of VBR bitrate control

• QVBR analyses every macroblock, frame, and scene in the video source, and automatically allocates bits to address information differences:

• Higher bitrate use during complex video segments, and Lower bitrate use during less complex video segments.

• The result: an enhanced version of variable bitrate control that minimizes wasted bits and scales video bitrate to maintain consistent video quality.

14 SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What are customers saying? • Transcode videos from car test drives • Initial tests have shown a 50% savings in storage • Comparison was with ffmpeg on EC2

• Transcode education and online learning content • Tests showed over a 50% decrease in storage and delivery costs • Win from Akamai encoding

• Transcode short form social content • Tests on a sample of 100 videos showed a 20% decrease in output file size • Comparison was with Amazon Elastic Transcoder

What are the trade off’s in order to reduce end-to-end latency?

• Video quality / video bitrate Short segments mean shorter GOPs, and this impacts video quality, or increases bitrates

• Quality of experience—risk of buffering Players have less buffer stored up, so less time to calculate and execute bitrate switches to avoid buffering

• Extra features—monetization and content protection Adding monetization and/or encryption becomes more challenging as they need time to complete interactions with ad decision servers or key exchanges

LEGACY LATENCY RANGE LOW LATENCY RANGE ULTRA-LOW LATENCY RANGE SUB-SEC

VOICE

LIVE AUCTIONS Typical Broadcast Broadcast INTERACTIVE GAMING APPLICATIONS Latency GAMBLING, BETTING BUSINESS BUSINESS USE CASES USE CASES

LIVE SPORTS

10s segments BROADCAST LATENCY – DTT, DTH, CABLE

6s segments

2s segments

1s segments TECHNOLOGY TECHNOLOGY DISTRIBUTION DISTRIBUTION CHUNKED CMAF segments 1s-6s OTT STREAMING OTT STREAMING WITH HLS & DASH WITH HLS WITH HLS & DASH & HLS WITH WebRTC 45+ 30 20 10 8 6 4 2 1 200 seconds seconds seconds seconds seconds seconds seconds seconds second ms Latencies that are achievable at scale ususinging mainstream CDN streaming technologies

AWS Elemental AWS Elemental Amazon Live MediaStore CloudFront Global Content ABR Encoding and Media-Optimized Delivery Network Players Live Feed Packaging Storage and Origin

Capture Encoding Ingest Delivery Last Mile Player

CAPTURE ENCODING INGEST DELIVERY LAST MILE PLAYER 0.39 SECONDS 1.48 SECONDS 0.27 SECONDS 0.17 SECONDS 0.28 SECONDS 2.55 SECONDS

7.58% 28.79% 5.25% 3.33% 5.44% 49.61%

Sub-3 second latency is possible under ideal conditions Network performance is the key; this won’t work unless 5.14SECONDS the network itself is fast and reliable

Client-side Server-side Burnt-in 2007 2012 advertising 2017 advertising static ads insertion Insertion

Deliver Provide Improve personalised broadcast- tracking of ad ads to viewers grade ad views experience

Amazon S3 AWS Elemental MediaTailor Dynamic / Managed Transcoding

Manifest Amazon AWS Elemental AWS Elemental cache AWS Elemental MediaLive MediaPackage MediaTailor CloudFront Content Delivery Network Live Video Origination and Manifest Processing JIT Packaging Manipulation Devices

Video with SCTE 35/104 Conditioned manifest with ad markers

Ad Decision Server

DFP, Freewheel or other ads . EasilyImproveIncrease deliver theviewing accuracy targeted experiences of ads ad to view any reporting platform

Personalized Seamless Ads on Live Video

Live stream NFL Thursday Night Football with Problem Statement personalized ads to millions of Prime subscribers Low-latency workflow required for minimal time-behind-live performance while maintaining maximum quality, uptime, and distribution

Use of AWS • AWS Elemental MediaTailor, AWS Elemental Server, Amazon S3 for ad encoding and live ad insertion • AWS Elemental encoders functions as a playout server into CloudFront

Business Benefits • Deploy user-centric features like viewer-selectable audio tracks and content personalization • Live stream availability on 600+ device types

Amazon Transcribe Subtitling and captioning service Leverage Complementary ML Services New set of ML services allows companies to Amazon Translate A 文 Multi-language translations easily caption text, translate that text into multiple languages, and create an audio Amazon Polly track with those new spoken languages Text-to-speech Object recognition also simplifies detection Amazon Rekognition of specific people, objects, or brands to Image recognition & analysis mark them in the video Amazon Lex Natural Language Understanding (NLU) & Automatic Speech Recognition (ASR)

Media Metadata Automate the creation of rich metadata index (object, scene, activity, faces etc.), extracted from audio Tagging visual content and integrated into digital and media asset management systems AWS SERVICES TO BUILD USE CASE: AMAZON REKOGNITION, AMAZON SAGEMAKER, AMAZON TRANSCRIBE, AMAZON COMPREHEND

Identify objects and emotion in content that enables users to integrate personalized ads into Ad Personalization subscriber video streams

AWS SERVICES TO BUILD USE CASE: AMAZON SPARK/EMR, AMAZON SAGEMAKER, AWS MEDIATAILOR

Closed Captioning Automated captions, transcription and translation of audio content AWS SERVICES TO BUILD USE CASE: AMAZON TRANSCRIBE, AMAZON TRANSLATE, AMAZON COMPREHEND

Detect potentially inappropriate content to avoid compliance issues in global markets, and to Automated increase brand safety for advertisers Compliance Marking AWS SERVICES TO BUILD USE CASE: AMAZON REKOGNITION, AMAZON TRANSCRIBE, AMAZON SAGEMAKER

Player/ Actor Identify and path player activity in the field or identify the precise timecodes when actors enter and identification & leave a scene, both visually and in spoken dialog (off-screen voice) activity pathing AWS SERVICES TO BUILD USE CASE: AMAZON REKOGNITION, AMAZON TRANSCRIBE

Language Translate transcripts and metadata. Improve localization workflows and search experience

Translation AWS SERVICES TO BUILD USE CASE: AMAZON TRANSLATE

Content Analysis disparate data and enable the consumer to make personalized content choices and the business to predict behavior Recommendations AWS SERVICES TO BUILD USE CASE: AMAZON SAGEMAKER

Detect and pixelate faces captured incidentally to preserve the privacy of non-persons of interest in Automated news feeds and security footage Redaction AWS SERVICES TO BUILD USE CASE: AMAZON REKOGNITION, AMAZON TRANSCRIBE, AMAZON SAGEMAKER

SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 27 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Sports Highlights and Player Activity Pathing

SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 28 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Introducing Amazon Personalize

Real-time personalization and recommendation, based on the machine learning technology used by Amazon.com

Generates personalized recommendations for your users via API calls . State-of-the-art deep learning algorithms that out-perform traditional methods

. Auto-ML capabilities that automate the entire process from data ingestion to inference

. Real-time personalized recommendations – Ability to ingest activity/clickstream data and generate recommendations, in real time, based on the session/context of the use

With thousands of movies and TV shows selection, it is a challenge to surface the most relevant programs to our viewers. To enable faster and more efficient content discovery, we want to deliver the highest quality content recommendations that are tailored to individuals' interests and preferences. Amazon Personalize enables us to easily and quickly implement a recommendation engine. Despite having no prior experience in AI/ML, it only took us 3 days to understand the service, create a first data set, and generate a recommendation. With Amazon Personalize, we will be able to provide every consumer with an unique viewing experience.

" Daniel Muller, Head of Cloud Infrastructure - Spuul

Background • Formula 1 is the worlds premier auto racing championship. • Since the first gran prix in 1950, Formula 1 engages with 500 million fans watching 21 global races annually.

Challenge 120 car telemetry sensors create 3Gb of data per second, how can Formula 1 bring analytics and insights to car data that drives advances in the sport and an deeper fan experience delivered by their next generation OTT/VOD platform. https://aws.amazon.com/f1insights/

Use of AWS Results • AWS Kinesis streams data into Amazon S3 in real-time • Deeper fan experience • Amazon Rekognition, Amazon SageMaker, & Amazon Transcribe analyze race data • Monetize data and direct to consumer video platform • AWS Media Services powers next generation video platform

SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. At the NFL, we are more efficient in the development of metrics for the Next Gen Stats offering by leveraging AWS technology. Machine Learning and other computations that could take months to refine now take weeks or days, allowing us to engage, inform and excite the fan in new and unique ways. — Matt Swensson, Vice President, Emerging Products and Technology at the NFL

SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 33 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Subtitling - Live Automated Multi-Language

. Real-time subtitling solution for live streaming video

. Combines Amazon Transcribe, Amazon Translate

. Serverless architecture

. Automatically generates multi-language subtitles for live streaming videos.

SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Media Solutions & Starter Kits Automatically provision the services necessary for building common media use cases on AWS

Edit in the Cloud - Deploy a highly available architecture for cloud video editing Content Production VFX Station in the Cloud - Deploy a Microsoft Windows-based visual effects environment

Automated Video Metadata Generation - Extract a rich layer of metadata from your content

Media Supply Chain Media to Cloud Migration - Set up a serverless ingest workflow to move video assets to the cloud

Media Services Application Mapper - Deploy a AWS Media Services visualization/status workflow

Live Streaming - Utilize a Solution to worldwide deliver broadcast-grade live content

Distribution Live Subtitling and Translation - Generate multi-language subtitles for live OTT streaming

Video on Demand Streaming - Create a workflow for broadcast and multiscreen VOD delivery

AIML and Analytics Automated Video Metadata Generation - Extract a rich layer of metadata from your content

Go to aws.amazon.com/media/solutions SUMMIT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 37 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Voice of the customer

PAC 12 Media services deployment

https://youtu.be/Jb_bg5GyrrM