Digital TV – The Software Components

SIBGRAPI 2007 – Tutorial October 7-10, 2007 Belo Horizonte, Minas Gerais, Brazil

Author Contact Information

 Alisson Sol Microsoft Research Cambridge [email protected] http://research.microsoft.com/~asol/

 Paulo Sérgio Pinto Microsoft TV [email protected]

Disclaimers

The views and opinions offered by individuals in this presentation do not necessarily represent the views of Microsoft. This work is intended for information and education only and is not guaranteed by Microsoft as to accuracy, completeness, nor any trading result.

All referenced company and product names are trademarks, registered trademarks or copyrights of their respective holders.

What Consumers Like About TV

• Clearer Picture And Sound: 78% SIBGRAPI 2007 • Variety of TV Channels: 72% • Electronic Programming Guide: 44% Digital TV: The Software Components • DVR: 38% Demonstration & Motivation • Digital Music Channels (Audio Only): 34% • VOD: 32% Alisson Sol • Picture-In-Picture: 23% Paulo Sérgio Pinto • Metadata (Info about directors, etc.): 22% • Sports Premium Channels: 20% • Interactive TV Services: 13%

1 2

Learning From Customers The Next Generation

• Computer: “Do what I want!” • Beloit College’s Mindset List – • : “Entertain me!” The Class Of 2011 (Born 1989) 1. What Berlin wall? 9. Nelson Mandela has always been free... • Distance 14. Music has always been “unplugged.” – Computer: 2 feet (~0.6m) 33. U2 has always been more than a spy plane. 47. High definition television has always been available. – Television: 10 feet (3m) 66. The World Wide Web has been an online tool since they were born. • What About Those Born on 1999? 3 4

Interactivity Television Would Have To Change

• Notes • This Generation (3 Top Features) – Nintendo – Clearer picture and sound – 360 – Variety of TV channels – Xbox Live – – YouTube.com • The Next Generation – Interactivity

5 6

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 2

Digital TV Myths Set Top Box

• Analog TV Systems Cannot Add Data • STB (Set Top Box) – Closed captions, ... • IRD (Integrated Receiver Decoder) • Digital TV Transmission Is “Digital” • PT: URD (Unidade Receptora Decodificadora) – The signal is still an electromagnetic wave, now encoding “digital” information • Old TVs Have To Be Replaced – You can use a digital-to-analog converter to get digitally encoded signal to a traditional TV

7 8

User Experience Video Erosion Of Terrestrial TV Revenue

• Notes • Competing TV Transmission Media – DVB (Digital Video Broadcasting) – Satellite, Cable, Broadband – Scanning and configuring • DVR – EPG (Electronic Programming Guide) – 53% of DVR users skip commercials (Source: Jupiter Research) – Interactivity (local) • Alternative Ads Destination – DVR (Digital Video Recorder) – Internet, Games, etc. • “Digital Convergence” – Content available in computers and devices 9 10

Storage Convenience Motivations Summary

• High Definition Content Formats • Customer Features – HD DVD: 30GB – More Channels – Blu-Ray: 50GB – Better picture and sound • TV Content (If all in HD) – Interactivity – Blu-Ray 2-hours movie = 50GB (Max) • Revenue Model – 1 day = 1200GB – 1 year = 427TB • Industry Pressure • At $1/GB: 1 year ~ $437,000 • Compare to 4380 2-hour tapes...

11 12

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 3

Agenda

• Browse Terminology SIBGRAPI 2007 – Define acronyms that will be used, but are mostly Digital TV: The Software Components out of scope Terminology – Have broad view before going deeper into specific areas of interest Alisson Sol Paulo Sérgio Pinto

13 14

Video Frame Definitions Initial B&W TV System

• Frame • Scanning: Progressive – Single image from video • VBS: Video Baseband Signal • Field – – Half of the frame from – White level • Video scanning – Synchronization – Progressive x Interlaced • Per line • Per frame (retrace) • Interlacing: getting fields from frame • Deinterlacing: getting frame from fields 15 16

Video Signal Line STB and Modulation

White level • Different Technologies – AM: – FM: – QAM: Quadrature Amplitude Modulation – QPSK: Quadrature Phase Shift Keying Black level – VSB: Vestigial Sideband Modulation – DQPSK: Differential Quadrature Phase Shift Keying – COFDM: Coded Orthogonal Frequency-Division Multiplexing Horizontal Visible part – BST-OSDM: Band Segmented Transmission Orthogonal suppression Frequency Division Multiplexing Horizontal Line duration sync 17 18

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 4

Modulation Technologies Color TV Channels Amplitude Frequency Channel Width 0 1 0 1

Video Audio

Phase Modulation Quadrature Phase Shift Keying 0 1 00 01 10 11 Chroma

Central Frequency

19 20

Analog Color TV Systems Video Resolution

• NTSC (National Television Systems Committee) • Lines And Scanning • Common Resolutions – , – DVD: 720x480 – Chroma: 3.579MHz – , – HDTV: 1280x720 (16:9) – QAM (Quadrature amplitude modulation) – 720i, – HDTV: 1920x1080 (16:9) – – • , DCI ( SECAM (Séquentiel couleur avec mémoire) • Initiatives): 4096x2160 – Same color information for two consecutive lines – 4:3 – RED Digital Cinema: 4520x2540 – 16:9 • PAL (Phase Alteration Line) – UHDV (Ultra High • Conversion Definition Video): – Inversion of carrier every second line – Upscaling 7680×4320 – PAL M: 525 lines/59.94 Hz, subcarrier 3.576 MHz – Downscaling

21 22

Transmission of Video Viable Digital TV Transmission

Technology Bandwidth (Mbps) • Source Coding 640x480x24 bits * 24fps 168.75 – Maximize compression and minimize information Wired Cable (100BaseT) 100 loss during video digitization 802.11b/g/n 11/54/248 • Channel Coding xDSL 8 – Limit bandwidth utilization and minimize effect of errors during transmission WiMax (802.16) 70

Bandwidth is maximum from standards 23 24

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 5

Source Coding Channel Coding

• JPEG: Joint Photographic Experts Group • Stream Multiplexing • DCT: Discrete Cosine Transform • Scrambling • MPEG: Moving Picture Experts Group • CAM (Conditional Access Messages) – MPEG-1: Video/Audio (MPEG-1, layer 3 = MP3) • FEC (Forward Error Correction) – MPEG-2: Program Streams – MPEG-4: AV + objects, HD – MPEG-7: XML metadata – MPEG-21: Multimedia framework 25 26

Digital TV System Return Channel Digital TV Systems

Services, Applications & Content Services, Applications & Content • DVB-S: Digital Video Broadcasting – Satellite • DVB-C: Digital Video Broadcasting – Cable Audio Video Data Source Signal Decoding Encoder Encoder Encoder Audio Video Middleware • DVB-T: Digital Video Broadcasting – Terrestrial Decoder Decoder Source Signal Encoding • ATSC: Advanced Television Systems Committee • ISDB: Integrated Services Digital Broadcasting Transport Layer Transport Layer • SBTVD: Sistema Brasileiro de Televisão Digital Transmission, Modulation and Reception, Demodulation and Channel Encoding Channel Decoding Broadcasting Access Terminal 27 28

Connections Content Protection

• RCA connector (Composite Cable) • HDCP (High-bandwidth Digital Content • S-Video Protection) • Component Cable • Region Coding • SCART (Syndicat des Constructeurs d'Appareils – DVD (Digital Video Disc) Region Codes Radiorécepteurs et Téléviseurs) • 0 (All), 1 to 6 (7 future, 8 international) – Blu-Ray Region Codes • HDMI (High-Definition Multimedia Interface) • Regions A, B, C • DVI (Digital Visual Interface) – HD DVD • Currently Region Free 29 30

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 6

Audio Interactivity

• • Middleware - Application Environments – DASE (Digital TV Applications Software Environment) – 3.0, 4.0, 5.1, 6.1, 7.1, 10.2, 22.2 – BML (Broadcast Markup Language) • 3D Audio Effect – MHP (Multimedia Home Platform) – GEM (Globally Executable MHP) – OCAP (OpenCable Applications Platform) – Ginga • Profiles – Enhanced Broadcast – Interactive Broadcast – Internet Access 31 32

Recording Mobility

• DVR (Digital Video Recorder) • DVB-H (Digital Video Broadcasting – – PVR (Personal Video Recorder) Handheld) – TiVo • DVB-IPDC (DVB specifications for IP • EPG (Electronic Programming Guide) Datacasting) • Tuners • GPRS (General Packet Radio Service) – Dual Tuners • EDGE (Enhanced Data rates for GSM • VOD (Video On Demand) Evolution) • 3GPP (3rd Generation Partnership Project) 33 34

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 7

Agenda

• Historic Reasons for Different TV Systems SIBGRAPI 2007 • Digital TV Systems Factors and Choices Digital TV: The Software Components International Scenario And Digital TV Systems

Alisson Sol Paulo Sérgio Pinto

35 36

World Electrical/Electronic Issues Terrestrial TV System Factors

• Differences • Frame/Field Rate – Power outlet • Color Information – Voltage • Channel Regulatory Issues – Frequency • Standard Committees – Analog TV systems – “Regions”: DVD regions, Blu-Ray regions, Internet content regions

37 38

Electromagnetic Spectrum Analog Color TV Systems

• NTSC (National Television Systems Committee) – Chroma: 3.579MHz – QAM (Quadrature amplitude modulation) • SECAM (Séquentiel couleur à mémoire) – Same color information for two consecutive lines • PAL (Phase Alteration Line) – Inversion of carrier every second line – PAL M: 525 lines/59.94 Hz, subcarrier 3.576 MHz 39 40

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 8

Digital TV Systems Factors ATSC

• Current Installed Base • Advanced Television Systems Committee – Compatibility • Main Usage: North America – Transition Phase • Channel Bandwidth: 6 MHz • Economical and Political Factors • Modulation: 8-VSB, 16-VSB • Bit rate: Up to ~19Megabits/second

41 42

DVB-T ISDB-T

• Digital Video Broadcasting • Integrated Services Digital Broadcasting • Main Usage: Europe • Main Usage: Japan • Channel Bandwidth: 5, 6, 7 or 8 MHz • Channel Bandwidth: 6 MHz • Modulation: COFDM, QPSK, 16-QAM, 64-QAM • Modulation: BST-OSDM, QPSK, DQPSK, • Bit rate: Up to ~32Megabits/second 16-QAM, 64-QAM – 31.668Mbps in 8MHz channel using 64-QAM • Bit rate: Up to ~19Megabits/second (62-QAM)

43 44

SBTVD-T Digital TV System Encoders Return Channel

Services, Applications & Content Services, Applications & Content • Sistema Brasileiro de Televisão Digital • Main usage: Brazil Audio Video Data Source Signal Decoding • Channel Bandwidth: 6 MHz Encoder Encoder Encoder Audio Video Middleware Decoder Decoder • Modulation: BST-OSDM, QPSK, DQPSK, Source Signal Encoding 16-QAM, 64-QAM Transport Layer Transport Layer • Bit rate: Up to ~19Megabits/second (62-QAM) Transmission, Modulation and Reception, Demodulation and Channel Encoding Channel Decoding Broadcasting Access Terminal 45 46

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 9

Digital TV Systems Choices Switchover

System Video Coded Middleware • Simulcast ATSC MPEG-2 DASE, ACAP • Analog Turn Off DVB-T MPEG-2 MHEG, MHP, SMS – Luxembourg: September 1, 2006 – Netherlands: December 11,2006 ISDB-T MPEG-2 ARIB BML – Finland: September 1, 2007 SBTVD-T MPEG-4 (H.264) Ginga • Population Guidance – Schedule – Example: Japan (ISDB) analog terrestrial broadcast ends on July 24, 2011 47 48

Technologies Out Of Scope

• But Worth A Brief Citation – Satellite TV Systems – Cable TV Systems – Terrestrial Return Channel

49

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 10

Agenda

• Video Technologies Review SIBGRAPI 2007 • Encoding For the Bandwidth Profile Digital TV: The Software Components • Video Applications Encoding, Streaming and Video Applications

Alisson Sol Paulo Sérgio Pinto

50 51

Digital Images Video Digitization

• Raw Format • Digitization in the Analog Lines • Encoding • Video Signal in 4:2:2 YCbCr format – RLE: Run Length Encoding • Other formats • JPEG – Color Quantization • RGB: Red, Green, Blue • YCbCr: Luminance, Blue, Chrominance Red – DCT (Discrete Cosine Transform)

52 53

Video Broadcasting MPEG Video Coding

Digital Video • Picture Types – Intra pictures: all information for reconstruction Multiplexing Filtering and Up-conversion FEC (scrambling) modulation + amplification – Predicted pictures: motion from previous I or P – Bidirectional pictures: bidirectional interpolation

Digital • Movement Estimator Broadcasting Video – Prediction – Motion estimation – Compensation Source coding Channel coding and multiplexing and modulation Broadcasting 54 55

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 11

MPEG Video Layers MPEG-2 Broadcasting

Sequence • Levels (Resolution) • Profiles Group of pictures – Low: 360x288 – Simple: No B pictures – – ... I ... I ... I Main: 720x576 Main: I, P, B – High-1440: 1440x1152 – Scalable • Spatial Picture Block – High: 1920x1152 • SNR

Slice Macroblock – High: HDTV

(8 x 8 pixels) (4 blocks) (n macroblocks) 56 57

Transport Streams Transmission/Reception Chain

Digital • Video, Audio, Data Video • Transport Packet Multiplexing Filtering and Up-conversion FEC • Scrambling and Conditional Access (scrambling) modulation + amplification

Digital Broadcasting Video

Digital (Descrambling) Amplification, FEC Demodulation Video Demultiplexing down-conversion

58 59

SBTVD Reference Model Return Channel Aerial Transmission

Services, Applications & Content Services, Applications & Content • Error Factors • BET (Bit Error Rate) Audio Video Data Source Signal Decoding Encoder Encoder Encoder Audio Video Middleware • Super Channel Decoder Decoder Source Signal Encoding • Error Correction

Transport Layer Transport Layer

Transmission, Modulation and Reception, Demodulation and Channel Encoding Channel Decoding Broadcasting Access Terminal 60 61

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 12 IPTV Streaming Over IP Subscription TV Services

• IPTV Live Media • Broadband Internet IP Network • Routing Schemes On-Demand TV via – Anycast Media Operator Operator Access Network STB – Broadcast Headend • What IPTV is not: • What IPTV is: – Multicast • Video streaming over the Internet • Competitive TV services over managed IP • Watching TV on your PC networks – Unicast • Best-efforts video services • Broadcast television • Based on unproven business models • All forms of on-demand • Electronic program guide • Connected entertainment 62 63

Video On Demand Video Database Applications

• Content Storage • Metadata Search • VOD on Private Networks – Actors, directors, producers... • Push Video On Demand – Keywords – • Content Protection Script, director comments – Rating • Metadata Issues – Mistakes – Globalization/Localization

64 65

Internet Applications Peer To Peer Systems

• Content Distribution • Concepts – Video streaming – Seeds – Movie download – Registrar • Content Management • Algorithms • Content Distribution Network • Live Content Distribution – Mirroring – P2P (Peer-To-Peer)

66 67

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 13

Agenda

• Interactive Television Standards SIBGRAPI 2007 • International Scenario Digital TV: The Software Components • SBTVD Middleware Overview Programmability And Interactivity

Alisson Sol Paulo Sérgio Pinto

68 69

TV Video Audience Safety Taxonomy and Usage

• “TV Safe” • Rigid Taxonomy – Colors – eTV (enhanced TV): features depend on content – Action area and data delivered to the STB – Title area • Personalized TV: user can change the TV appearance – Movement speed – iTV (interactive TV): there is a return channel to • London 2012 motion graphics causing people epileptic the content provider seizures... • • Video Games User’s point-of-view – Action-packed video games have warnings... – If the user can influence, it is “interactive” – Consider video games (few use a return channel) 70 71

Digital TV System Return Channel Software Layers

Services, Applications & Content Services, Applications & Content Set Top Box PC

Application API Applications Applications API Audio Video Data Source Signal Decoding Middleware API Middleware Windows Encoder Encoder Encoder Audio Video Middleware Decoder Decoder Source Signal Encoding Platform API Adapter Layer MS-DOS

Transport Layer Transport Layer Drivers RTOS BIOS

Transmission, Modulation and Reception, Demodulation and Hardware Hardware Channel Encoding Channel Decoding Broadcasting Access Terminal 72 73

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 14

Middleware Choices GEM (Globally Executable MHP)

• ATSC – DASE: DTV Applications Software Environment GEM – OCAP: OpenCable Application Platform – ACAP: Advanced Common Application Platform • DVB – MHP: Multimedia Home Platform ACAP MHP BML • ISDB – ARIB: Association of Radio Industries and Businesses – BML: Broadcast Mark-up Language (ARIB STD-B23) 74 75

Environments And “Standards” Harmonization Execution Presentation Environment Environment

DVB-H DVB (MHP) DVB-J DVB-HTML ACAP MHP OCAP DASE BML J2ME ATSC ACAP-J ACAP-X GEM ESG ARIB ARIB-AE BML Software Environment ITU ITU J.202 ITU J.201 OCAP data GEM DVB-T/S/C ATSC ISDB DVB-H broadcasting ITU J.200 Transmission System Architecture 76 77

GEM MHP XLets Demo

• Package javax.tv.xlet • Xlet Applications – public interface javax.tv.xlet.Xlet { • XHTML Applications public void initXlet(XletContext ctx) throws XletStateChangeException; public void startXlet() throws XletStateChangeException; • Microsoft Mediaroom ADK Simulator public void pauseXlet(); public void destroyXlet(boolean unconditional) throws XletStateChangeException; } – public interface XletContext { public static final String ARGS = "javax.tv.xlet.args"; public void notifyDestroyed(); public void notifyPaused(); public void resumeRequest(); public Object getXletProperty(String key); } 78 79

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 15

Getting Applications to STB Middleware Engineering

• Data Broadcasting • Updates • DSM-CC (Digital Storage Media - Command – Service channels and Control) • Software Deployment • Data Carousel – Channel stream • Application Environments • Security Issues – Native Application Environment – Java Application Environment – XHTML Application Environment 80 81

SBTVD Middleware Enhanced TV Applications

• Ginga • Multi Screen News – Ginga-CC (Common Core) • Traffic Cameras – Ginga-J • Angle From Sports Show – Ginga-NCL – Soccer/Football • Ferramentas • Traditional view, reverse angle, goalkeeper – Composer – Car Race – Scripting language: Lua • Pilot view, choice from car • Enhanced Ads

82 83

Return Channel Interactive Applications

• Cable or Broadband • Voting – Wired Connection – Audience influencing show • Satellite or Terrestrial • Public Services – Mobile Phone Message – Internet-like applications • SIM card directly into STB • Utility bills, public service scheduling – Internet • Educational shows – Traditional • Phone

84 85

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 16

Perception and Reality

• Customers Report Higher Satisfaction, But... SIBGRAPI 2007 – Most digital broadcasts are not in high definition Digital TV: The Software Components – Most digital broadcasts have less resolution than Complimentary Issues traditional analog broadcasts – Depending on user location, the digital signal may Alisson Sol be too weak Paulo Sérgio Pinto

86 87

UK Ofcom Communications Market Switchover to Digital TV

• Office of Communications Report 2007 • Almost Always Slower Than Expected – Converging Communication: >50% household – Only Complete in Small Countries have broadband • Regulatory Agencies Work – HDTV: 1.7% households – Consumer Education – success sparked interest into Freesat – Coupons • Freesat to be based on one-off fee; to start on 2008 • EUA: 2 U$40.00 coupons/house that relies on terrestrial – Quiz TV down (using telephone as return channel) • Coupon to get STB for digital-to-analog conversion • March/2007: 1000 hours/week • July/2007: 90 hours/week 88 89

Content Protection Privacy

• In Some Countries, the STB Will Use DRM • STB Identification Diminishes Privacy – DRM (Digital Rights Management) will limit ability – Allow viewing habits to be tracked to copy the digital content – Happening already in cable systems • Tagging Digital Broadcasts • Enhanced/Interactive Advertisement – Lower resolution than normal – Inserting special frames

90 91

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 17

Enhanced Advertising Accessibility

• Ad based on target for current show and... • Current Interactivity: Mostly Visual – ...time since switched channel... • Issue Not Only For Disabled People – ...shows recorded in digital recorder... – “Red button” confusion in DVRs – ...viewer gender... – Text too small • Ad pop-up when fast forwarding show – Misunderstanding of “setup issues” • “Extended Ad” • Wrong cables may prevent real high definition • Black areas around picture – Push extended ad to local storage • Buffer loss when switching channel (involuntarily – Offer extended ad during “normal ad” happens when pressing wrong key) 92 93

HD Content Current Developments: Standards

• Blu-Ray versus HD DVD • DVB-T Now 10 Years Old – Several HDTVs will downscale resolution • DVB-CPCM (DVB Content Protection & Copy • Not able to really show 1080p or 1080i Management)

94 95

Mobile Devices Mobile Specific Technologies

• Terrestrial Broadcast to Mobile • WAP (Wireless Application Protocol) – DVB-H (Digital Video Broadcasting – Handheld) – New XML based language was created – DMB (Digital Multimedia Broadcasting) • WML (Wireless Markup Language) – ISDB-T (Integrated Services Digital Broadcasting) – Some sites had WAP versions – MediaFlo – Several explications for failure, but mostly: mobiles simply got to “desktop level” • Digital TV Development For Mobiles – Other than lower resolution, little should be different for developers at authoring level 96 97

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 18 Tutorial References

Books  : MPEG-1, MPEG-2 and Principles of the DVB System, 2nd Edition Herve Benoit Focal Press, 2002  Understanding Digital Television: An Introduction to DVB Systems with Satellite, Cable, Broadband and Terrestrial TV Distribution Lars-Ingemar Lundstrom Focal Press, 2006  HDTV and the Transition to Digital Broadcasting: Understanding New Television Technologies Philip J. Cianci Focal Press, 2007  Digital Interactive TV and Metadata: Future Broadcast Multimedia Artur Lugmayr, Samuli Niiranen, Seppo Kalli Springer-Verlag New York, 2004  Interactive TV Standards: A Guide to MHP, OCAP, and JavaTV Steven Morris, Anthony Smith-Chaigneau Focal Press, 2005  Mobile TV: DVB-H, DMB, 3G Systems and Rich Media Applications Amitabh Kumar Focal Press, 2007

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 19 Online References

 ARIB: http://www.arib.or.jp/english/  ATSC: http://www.atsc.org  Digital UK: http://www.digitaluk.co.uk/  DVB Glossary: http://www.dvb.org/technology/dvb_glossary/  DVB: http://www.dvb.org/  ISDB-T: http://www.dibeg.org/techp/isdb/isdbt.htm  MHP: http://www.mhp.org  Microsoft Mediaroom: http://www.microsoftmediaroom.com/  Middleware Ginga: http://www.ginga.org.br/  Motion Picture Experts Group: http://www.chiariglione.org/mpeg/  SBTVD: http://sbtvd.cpqd.com.br/  YouTube: http://www.youtube.com/

SIBGRAPI 2007 – Tutorial: Digital TV – The Software Components 20