Video Demystified

A Handbook for the Digital Engineer Fifth Edition

by Keith Jack

AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO ELSEVIER Newnes is an imprint of Elsevier Newnes Contents

About the Author xix chapteri • introduction 1

Contents 3 Standards Organizations 5

Chapter 2 • Introduction to Video 6

Analog vs. Digital 6 Video Data 6 Digital Video 7 Video Timing 7 Video Resolution 9 Standard-Definition 9 Enhanced-Definition 9 High-Definition 11 Audio and Video Compression 11 Application Block Diagrams 11 DVD Players 11 Digital Media Adapters 12 Digital Television Set-Top Boxes 12

v vi Contents chapter 3 • Color Spaces 15

RGB Color Space 15 sRGB 16 scRGB 17 YUV Color Space 17 YIQ Color Space 18 YCbCr Color Space 19 RGB-YCbCr Equations: SDTV 19 RGB-YCbCr Equations: HDTV 20 4:4:4 YCbCr Format 21 4:2:2 YCbCr Format 22 4:1:1 YCbCr Format 22 4:2:0 YCbCr Format 22 xvYCC Color Space 26 PhotoYCC Color Space 26 HSI, HLS, and HSV Color Spaces 27 Chromaticity Diagram 28 Non-RGB Color Space Considerations 32 Gamma Correction 34 Constant Luminance Problem 36 References 36 chapter 4 • Video Signals Overview 37

Digital Component Video Background 37 Coding Ranges 37 480i and 480p Systems 39 576i and 576p Systems 48 720p Systems 56 1080i and 1080p Systems 59 Other Video Systems 64 References 67

/ Contents vii chapter 5 • Analog Video Interfaces 68

S-Video Interface 68 SCART Interface 69 SDTVRGB Interface 71 HDTV RGB Interface 75 Constrained Image 77 SDTV YPbPr Interface 77 VBI Data for 480p Systems 82 VBI Data for 576p Systems 87 HDTV YPbPr Interface 90 VBI Data for 720p Systems 94 VBI Data for 1080i Systems 96 Constrained Image 97 D-Connector Interface 97 Other Pro-Video Analog Interfaces 100 VGA Interface 100 References 100 chapter 6 • Digital Video Interfaces 106

Pro-Video Component Interfaces 106 Parallel Interfaces 110 Serial Interfaces 124 Pro-Video Composite Interfaces 129 Pro-Video Transport Interfaces 143 Serial Data Transport Interface (SDTI) 143 High Data-Rate Serial Data Transport Interface (HD-SDTI) 144 IC Component Interfaces 149 BT.601 Video Interface 149 Video Module Interface (VMI) 154 BT.656 Interface 156 Zoomed Video Port (ZV Port) 157 Video Interface Port (VIP) 158 Consumer Component Interfaces 162 Digital Visual Interface (DVI) 162 High-Definition Multimedia Interface (HDMI) 167 Digital Fiat Panel (DFP) Interface 168 viii Contents

Open LVDS Display Interface (OpenLDI) 170 Gigabit Video Interface (GVIF) 172 Consumer Transport Interfaces 174 USB 2.0 174 Ethernet 174 IEEE 1394 174 References 189

Chapter 7 • Digital Video Processing 192

Rounding Considerations 193 Truncation , 193 Conventional Rounding 193 Error Feedback Rounding 193 Dynamic Rounding 193 SDTV-HDTVYCbCrTransforms 194 SDTVto HDTV 194 HDTVto SDTV 194 4:4:4 to 4:2:2 YCbCr Conversion 195 Display Enhancement 198 Brightness, Contrast, Saturation (Color), and Hue (Tint) 198 Color Transient Improvement .' 200 Luma Transient Improvement 200 Sharpness 200 Blue Stretch 202 Green Enhancement 202 Dynamic Contrast 202 Color Correction 203 Color Temperature Correction 203 Video Mixing and Graphics Overlay 204 Luma and Chroma Keying 211 Luminance Keying 211 Chroma Keying 214 Superblack and Luma Keying 222 Contents ix

Video Scaling 223 Pixel Dropping and Duplication 224 Linear Interpolation 224 Anti-Aliased Resampling 224 Display Scaling Examples 227 Scan Rate Conversion 227 Frame or Field Dropping and Duplicating 232 Temporal Interpolation 234 2:2 Pulldown 240 3:2 Pulldown 240 3:3 Pulldown 241 24:1 Pulldown 241 Noninterlaced-to-Interlaced Conversion 241 Scan Line Decimation 241 Vertical Filtering 243 Interlaced-to-Noninterlaced Conversion 243 Video Mode: Intra-Field Processing 243 Video Mode: Inter-Field Processing 245 Film Mode 247 Frequency Response Considerations 247 DCT-Based Compression 248 Fixed Pixel Display Considerations 252 Expanded Color Reproduction 252 Detail Correction 252 Non-uniform Quantization 252 Scaling and Deinterlacing 253 References 256 chapter 8 • NTSC, PAL, and SECAM Overview 257

NTSC Overview 257 Luminance Information 257 Color Information 257 Color Modulation 258 Composite Video Generation 259 Color Subcarrier Frequency 261 NTSC Standards 261 RF Modulation 265 X Contents

Analog Channel Assignments 268 Luminance Equation Derivation 279 PAL Overview 280 Luminance Information 280 Color Information 281 Color Modulation 281 Composite Video Generation 281 PALStandards 285 RF Modulation 285 Analog Channel Assignments 295 Luminance Equation Derivation 295 PALplus 300 SECAM Overview .' 303 Luminance Information 303 Color Information 303 Color Modulation 304 Composite Video Generation 304 SECAM Standards 304 Luminance Equation Derivation 307 Video Test Signals 312 VBI Data 337 Timecode 337 CEA-608 Closed Captioning 346 Widescreen Signaling and CGMS 369 Teletext 374 AMOL (Automated Measurement of Lineups) 381 Raw VBI Data 381 Sliced VBI Data 381 Enhanced Television Programming 384 References 385 Contents xi chapter 9 • NTSC and PAL Digital Encoding and Decoding 388

NTSC and PAL Encoding 389 2x Oversampling 389 Color Space Conversion 389 Luminance (Y) Processing 393 Color Difference Processing 396 Analog Composite Video 404 Color Subcarrier Generation 407 Horizontal and Vertical Timing 411 Clean Encoding 415 Bandwidth-Limited Edge Generation 416 Level Limiting 417 Encoder Video Parameters 417 Genlocking Support 421 Alpha Channel Support 422 NTSC and PAL Digital Decoding 422 Digitizing the Analog Video 422 Y/C Separation 425 Color Difference Processing 425 Luminance (Y) Processing 429 User Adjustments 432 Color Space Conversion 434 Genlocking 436 Video Timing Generation 444 Auto-Detection of Video Signal Type 446 Y/C Separation-Techniques 446 Alpha Channel Support 458 Decoder Video Parameters 461 References 465 xii Contents chapterio • H.261 and H.263 466

H.261 466 Video Coding Layer 466 Video Bitstream 472 Still Image Transmission 481 H.263 481 Video Coding Layer 482 Video Bitstream 484 Optional H.263 Modes 505 Profiles 512 References 514 chapter ii • Consumer DV 515

Audio 517 Video 521 Digital Interfaces 534 IEEE 1394 535 SDTI 535 100 Mbps DV Differences 536 HDVFormat 536 AVCHD Format 536 References 537

Chapter 12 • MPEG-1 539

MPEG vs. JPEG 539 Quality Issues 540 Audio Overview 541 Video Coding Layer 542 Interlaced Video 543 Encode Preprocessing 543 Coded Frame Types 543 Motion Compensation 545 I Frames 546 Contents xiii

P Frames 548 B Frames 549 D Frames 550 Video Bitstream 551 Video Sequence 551 Sequence Header 551 Group of Pictures (GOP) Layer 555 Picture Layer 556 Slice Layer 557 Macroblock (MB) Layer 558 Block Layer 562 System Bitstream 570 ISO/IEC 11172 Layer 570 Pack Layer 570 System Header 571 Packet Layer 573 Video Decoding 575 Real-World Issues 576 References 576

Chapterl3 • MPEG-2 577

Audio Overview 578 Video Overview 578 Levels 578 Profiles 578 Scalability 584 Transport and Program Streams 584 Video Coding Layer 585 YCbCr Color Space 585 Coded Picture Types 585 Motion Compensation 586 Macroblocks 587 I Pictures 587 P Pictures 590 B Pictures 591 xiv Contents

Video Bitstream 591 Video Sequence 593 Sequence Header 593 User Data 596 Sequence Extension 596 Sequence Display Extension 598 Sequence Scalable Extension 601 Group of Pictures (GOP) Layer 603 Picture Layer 604 Content Description Data 605 Picture Coding Extension 611 Quant Matrix Extension 614 Picture Display Extension 616 Picture Temporal Scalable Extension 617 Picture Spatial Scalable Extension 618 Copyright Extension 619 Camera Parameters Extension 620 ITU-T ext. D Extension 620 Slice Layer 620 Macroblock Layer 621 Block Layer 622 Motion Compensation 642 PES Packet 647 Program Stream 656 Pack Layer 657 System Header 657 Program Stream Map (PSM) 659 Program Stream Directory 661 Transport Stream 661 Packet Layer 661 Adaptation Field 663 Program Specific Information (PSI) 666 Program Association Table (PAT) 668 Program Map Table (PMT) 670 Transport Stream Description Table (TSDT) 671 Conditional Access Table (CAT) 672 Network Information Table (NIT) 673 IPMP Control Information Table (ICIT) 673 Contents XV

Intellectual Property Management and Protection (IPMP) 674 MPEG-4.2 Video over MPEG-2 Transport Streams 674 MPEG-4.10 (H.264) Video over MPEG-2 Transport Streams 674 SMPTE 421M (VC-1) Video over MPEG-2 Transport Streams 675 MPEG-2 PMT/PSM Descriptors 675 MPEG-4 PMT/PSM Descriptors 689 APJB PMT Descriptors 692 ATSC PMT Descriptors 695 DVB PMT Descriptors 698 OpenCable PMT Descriptors 704 Closed Captioning 706 VBI Standard 712 Teletext 717 Active Format Description (AFD) 718 Subtitles 720 Enhanced Television Programming 725 Data Broadcasting 727 Decoder Considerations 732 References 737

ChaPteri4 • MPEG-4 and H.264 738

Audio Overview 739 General Audio Object Types 739 Speech Object Types 740 Synthesized Speech Object Types 740 Synthesized Audio Object Types 740 Visual Overview 741 YCbCr Color Space 741 Visual Objects 741 MPEG-4 Part 2 Natural Visual Object Types 741 MPEG-4 Part 2 Natural Visual Profiles 743 Graphics Overview 747 Visual Layers 747 Visual Object Sequence (VS) 747 Video Object (VO) 747 Video Object Layer (VOL) 747 Group of Video Object Plane (GOV) 749 Video Object Plane (VOP) 749 xvi Contents

Object Description Framework 749 Object Descriptor (OD) Stream 749 Object Content Information (OCI) 751 Intellectual Property Management and Protection (IPMP) 751 Scene Description 751 BIFS 751 Synchronization of Elementary Streams 753 Sync Layer 753 DMIF Application Interface 754 Multiplexing of Elementary Streams 754 FlexMux 755 MPEG-4 Over MPEG-2 755 MP4 File Format 755 Intellectual Property Management and Protection (IPMP) 755 MPEG-4 Part 10 (H.264) Video 756 Profiles and Levels 756 Supplemental Enhancement Information (SEI) Messages 758 Video Coding Layer 759 Network Abstraction Layer (NAL) 762 References 763 chapter 15 • ATSC Digital Television 764

Video Capability 766 Audio Capability 766 Program and System Information Protocol (PSIP) 768 Required Tables 768 Optional Tables 768 Descriptors 770 E-VSB 772 Data Broadcasting 773 Application Block Diagrams 774 References 777 Contents xvii chapter 16 • OpenCable™ Digital Television 778

Video Capability 780 Audio Capability 780 In-Band System Information (SI) 780 Required Tables 781 Optional Tables 782 Descriptors 784 Out-of-Band System Information (SI) 786 Tables 786 Descriptors 788 In-Band Data Broadcasting 790 Data Service Announcements 790 Service Description Framework (SDF) 791 Conditional Access 791 Related Technologies 792 Application Block Diagrams 792 References 795 chapter 17 • DVB Digital Television 796

Video Capability 798 Audio Capability 798 System Information (SI) 798 Required Tables 798 Optional Tables 799 Descriptors 804 Data Broadcasting 808 Conditional Access 808 Application Block Diagrams 810 References 810 xviii Contents chapter 18 • ISDB Digital Television 812

ISDB-S (Satellite) 813 ISDB-C (Cable) 813 ISDB-T (Terrestrial) 814 Video Capability 814 Audio Capability 814 Still Picture Capability 814 Graphics Capability 814 System Information (SI) 816 Tables 816 Descriptors 817 Captioning 825 Data Broadcasting 825 Application Block Diagrams 826 References 826

Chapter 19 • IPTV 827

Considerations 827 Multicasting 828 RTSP-Based Solutions 828 RTSP 828 RTP 830 RTCP 833 RSVP 834 ISMA 834 Broadcast over IP 835 Conditional Access (DRM) 835 References 835 chapter 20 • Giossary 837

Index 889