Contents i Demystified

Video Demystified ii Contents Contents iii

Video Demystified

A Handbook for the Digital Engineer Fifth Edition

by Keith Jack

AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO Newnes is an imprint of Elsevier iv Contents

Newnes is an imprint of Elsevier 30 Corporate Drive, Suite 400, Burlington, MA 01803, USA Linacre House, Jordan Hill, Oxford OX2 8DP, UK

Copyright © 2007, Elsevier Inc. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher.

Permissions may be sought directly from Elsevier's Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, E-mail: [email protected]. You may also complete your request online via the Elsevier homepage (http://elsevier.com), by selecting “Support & Contact” then “Copyright and Permission” and then “Obtaining Permissions.”

Recognizing the importance of preserving what has been written, Elsevier prints its books on acid-free paper whenever possible.

Library of Congress Cataloging-in-Publication Data

(Application submitted.)

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

ISBN: 978-0-7506-8395-1

For information on all Newnes publications visit our Web site at www.books.elsevier.com

07 08 09 10 11 10 9 8 7 6 5 4 3 2 1

Printed in the United States of America Contents v Contents

Contents

About the Author xix

Chapter 1 • Introduction 1

Contents ...... 3 Standards Organizations...... 5

Chapter 2 • Introduction to Video 6

Analog vs. Digital ...... 6 Video Data ...... 6 ...... 7 Video Timing ...... 7 Video Resolution ...... 9 Standard-Definition ...... 9 Enhanced-Definition ...... 9 High-Definition ...... 11 Audio and Video Compression ...... 11 Application Block Diagrams ...... 11 DVD Players ...... 11 Digital Media Adapters ...... 12 Digital Set-Top Boxes ...... 12

v vi Contents

Chapter 3 • Color Spaces 15

RGB ...... 15 sRGB ...... 16 scRGB ...... 17 YUV Color Space ...... 17 YIQ Color Space ...... 18 YCbCr Color Space ...... 19 RGB-YCbCr Equations: SDTV ...... 19 RGB-YCbCr Equations: HDTV ...... 20 4:4:4 YCbCr Format ...... 21 4:2:2 YCbCr Format ...... 22 4:1:1 YCbCr Format ...... 22 4:2:0 YCbCr Format ...... 22 xvYCC Color Space ...... 26 PhotoYCC Color Space ...... 26 HSI, HLS, and HSV Color Spaces ...... 27 Chromaticity Diagram ...... 28 Non-RGB Color Space Considerations ...... 32 Gamma Correction ...... 34 Constant Luminance Problem ...... 36 References ...... 36

Chapter 4 • Video Signals Overview 37

Digital Background ...... 37 Coding Ranges ...... 37 and Systems ...... 39 and Systems ...... 48 Systems ...... 56 and Systems ...... 59 Other Video Systems ...... 64 References ...... 67 Contents vii

Chapter 5 • Analog Video Interfaces 68

S-Video Interface ...... 68 SCART Interface ...... 69 SDTV RGB Interface ...... 71 HDTV RGB Interface ...... 75 Constrained Image ...... 77 SDTV YPbPr Interface ...... 77 VBI Data for 480p Systems ...... 82 VBI Data for 576p Systems ...... 87 HDTV YPbPr Interface ...... 90 VBI Data for 720p Systems ...... 94 VBI Data for 1080i Systems ...... 96 Constrained Image ...... 97 D-Connector Interface ...... 97 Other Pro-Video Analog Interfaces ...... 100 VGA Interface ...... 100 References ...... 100

Chapter 6 • Digital Video Interfaces 106

Pro-Video Component Interfaces ...... 106 Parallel Interfaces ...... 110 Serial Interfaces ...... 124 Pro-Video Composite Interfaces ...... 129 Pro-Video Transport Interfaces ...... 143 Serial Data Transport Interface (SDTI) ...... 143 High Data-Rate Serial Data Transport Interface (HD-SDTI) ...... 144 IC Component Interfaces ...... 149 BT.601 Video Interface ...... 149 Video Module Interface (VMI) ...... 154 BT.656 Interface ...... 156 Zoomed Video Port (ZV Port) ...... 157 Video Interface Port (VIP) ...... 158 Consumer Component Interfaces ...... 162 (DVI) ...... 162 High-Definition Multimedia Interface (HDMI) ...... 167 Digital Flat Panel (DFP) Interface ...... 168 viii Contents

Open LVDS Display Interface (OpenLDI) ...... 170 Gigabit Video Interface (GVIF) ...... 172 Consumer Transport Interfaces ...... 174 USB 2.0 ...... 174 Ethernet ...... 174 IEEE 1394 ...... 174 References ...... 189

Chapter 7 • Digital 192

Rounding Considerations ...... 193 Truncation ...... 193 Conventional Rounding ...... 193 Error Feedback Rounding ...... 193 Dynamic Rounding ...... 193 SDTV-HDTV YCbCr Transforms ...... 194 SDTV to HDTV ...... 194 HDTV to SDTV ...... 194 4:4:4 to 4:2:2 YCbCr Conversion ...... 195 Display Enhancement ...... 198 Brightness, Contrast, Saturation (Color), and Hue (Tint) ...... 198 Color Transient Improvement ...... 200 Luma Transient Improvement ...... 200 Sharpness ...... 200 Blue Stretch ...... 202 Green Enhancement ...... 202 Dynamic Contrast ...... 202 Color Correction ...... 203 Color Temperature Correction ...... 203 Video Mixing and Graphics Overlay ...... 204 Luma and Chroma Keying ...... 211 Luminance Keying ...... 211 Chroma Keying ...... 214 Superblack and Luma Keying ...... 222 Contents ix

Video Scaling ...... 223 Pixel Dropping and Duplication ...... 224 Linear Interpolation ...... 224 Anti-Aliased Resampling ...... 224 Display Scaling Examples ...... 227 Scan Rate Conversion ...... 227 Frame or Field Dropping and Duplicating ...... 232 Temporal Interpolation ...... 234 2:2 Pulldown ...... 240 3:2 Pulldown ...... 240 3:3 Pulldown ...... 241 24:1 Pulldown ...... 241 Noninterlaced-to-Interlaced Conversion ...... 241 Scan Line Decimation ...... 241 Vertical Filtering ...... 243 Interlaced-to-Noninterlaced Conversion ...... 243 Video Mode: Intra-Field Processing ...... 243 Video Mode: Inter-Field Processing ...... 245 Film Mode ...... 247 Frequency Response Considerations ...... 247 DCT-Based Compression ...... 248 Fixed Pixel Display Considerations ...... 252 Expanded Color Reproduction ...... 252 Detail Correction ...... 252 Non-uniform Quantization ...... 252 Scaling and Deinterlacing ...... 253 References ...... 256

Chapter 8 • NTSC, PAL, and SECAM Overview 257

NTSC Overview ...... 257 Luminance Information ...... 257 Color Information ...... 257 Color Modulation ...... 258 Generation ...... 259 Color Subcarrier Frequency ...... 261 NTSC Standards ...... 261 RF Modulation ...... 265 x Contents

Analog Channel Assignments ...... 268 Luminance Equation Derivation ...... 279 PAL Overview ...... 280 Luminance Information ...... 280 Color Information ...... 281 Color Modulation ...... 281 Composite Video Generation ...... 281 PAL Standards ...... 285 RF Modulation ...... 285 Analog Channel Assignments ...... 295 Luminance Equation Derivation ...... 295 PALplus ...... 300 SECAM Overview ...... 303 Luminance Information ...... 303 Color Information ...... 303 Color Modulation ...... 304 Composite Video Generation ...... 304 SECAM Standards ...... 304 Luminance Equation Derivation ...... 307 Video Test Signals ...... 312 VBI Data ...... 337 Timecode ...... 337 CEA-608 ...... 346 Signaling and CGMS ...... 369 ...... 374 AMOL (Automated Measurement of Lineups) ...... 381 Raw VBI Data ...... 381 Sliced VBI Data ...... 381 Enhanced Television Programming ...... 384 References ...... 385 Contents xi

Chapter 9 • NTSC and PAL Digital Encoding and Decoding 388

NTSC and PAL Encoding ...... 389 2× Oversampling ...... 389 Color Space Conversion ...... 389 Luminance (Y) Processing ...... 393 Color Difference Processing ...... 396 Analog Composite Video ...... 404 Color Subcarrier Generation ...... 407 Horizontal and Vertical Timing ...... 411 Clean Encoding ...... 415 Bandwidth-Limited Edge Generation ...... 416 Level Limiting ...... 417 Encoder Video Parameters ...... 417 Genlocking Support ...... 421 Alpha Channel Support ...... 422 NTSC and PAL Digital Decoding ...... 422 Digitizing the Analog Video ...... 422 Y/C Separation ...... 425 Color Difference Processing ...... 425 Luminance (Y) Processing ...... 429 User Adjustments ...... 432 Color Space Conversion ...... 434 Genlocking ...... 436 Video Timing Generation ...... 444 Auto-Detection of Video Signal Type ...... 446 Y/C Separation Techniques ...... 446 Alpha Channel Support ...... 458 Decoder Video Parameters ...... 461 References ...... 465 xii Contents

Chapter 10 • H.261 and H.263 466

H.261 ...... 466 Video Coding Layer ...... 466 Video Bitstream ...... 472 Still Image Transmission ...... 481 H.263 ...... 481 Video Coding Layer ...... 482 Video Bitstream ...... 484 Optional H.263 Modes ...... 505 Profiles ...... 512 References ...... 514

Chapter 11 • Consumer DV 515

Audio ...... 517 Video ...... 521 Digital Interfaces ...... 534 IEEE 1394 ...... 535 SDTI ...... 535 100 Mbps DV Differences ...... 536 HDV Format ...... 536 AVCHD Format ...... 536 References ...... 537

Chapter 12 • MPEG-1 539

MPEG vs. JPEG ...... 539 Quality Issues ...... 540 Audio Overview ...... 541 Video Coding Layer ...... 542 ...... 543 Encode Preprocessing ...... 543 Coded Frame Types ...... 543 Motion Compensation ...... 545 I Frames ...... 546 Contents xiii

P Frames ...... 548 B Frames ...... 549 D Frames ...... 550 Video Bitstream ...... 551 Video Sequence ...... 551 Sequence Header ...... 551 Group of Pictures (GOP) Layer ...... 555 Picture Layer ...... 556 Slice Layer ...... 557 Macroblock (MB) Layer ...... 558 Block Layer ...... 562 System Bitstream ...... 570 ISO/IEC 11172 Layer ...... 570 Pack Layer ...... 570 System Header ...... 571 Packet Layer ...... 573 Video Decoding ...... 575 Real-World Issues ...... 576 References ...... 576

Chapter 13 • MPEG-2 577

Audio Overview ...... 578 Video Overview ...... 578 Levels ...... 578 Profiles ...... 578 Scalability ...... 584 Transport and Program Streams ...... 584 Video Coding Layer ...... 585 YCbCr Color Space ...... 585 Coded Picture Types ...... 585 Motion Compensation ...... 586 Macroblocks ...... 587 I Pictures ...... 587 P Pictures ...... 590 B Pictures ...... 591 xiv Contents

Video Bitstream ...... 591 Video Sequence ...... 593 Sequence Header ...... 593 User Data ...... 596 Sequence Extension ...... 596 Sequence Display Extension ...... 598 Sequence Scalable Extension ...... 601 Group of Pictures (GOP) Layer ...... 603 Picture Layer ...... 604 Content Description Data ...... 605 Picture Coding Extension ...... 611 Quant Matrix Extension ...... 614 Picture Display Extension ...... 616 Picture Temporal Scalable Extension ...... 617 Picture Spatial Scalable Extension ...... 618 Copyright Extension ...... 619 Camera Parameters Extension ...... 620 ITU-T ext. D Extension ...... 620 Slice Layer ...... 620 Macroblock Layer ...... 621 Block Layer ...... 622 Motion Compensation ...... 642 PES Packet ...... 647 Program Stream ...... 656 Pack Layer ...... 657 System Header ...... 657 Program Stream Map (PSM) ...... 659 Program Stream Directory ...... 661 Transport Stream ...... 661 Packet Layer ...... 661 Adaptation Field ...... 663 Program Specific Information (PSI) ...... 666 Program Association Table (PAT) ...... 668 Program Map Table (PMT) ...... 670 Transport Stream Description Table (TSDT) ...... 671 Conditional Access Table (CAT) ...... 672 Network Information Table (NIT) ...... 673 IPMP Control Information Table (ICIT) ...... 673 Contents xv

Intellectual Property Management and Protection (IPMP) ...... 674 MPEG-4.2 Video over MPEG-2 Transport Streams ...... 674 MPEG-4.10 (H.264) Video over MPEG-2 Transport Streams ...... 674 SMPTE 421M (VC-1) Video over MPEG-2 Transport Streams ...... 675 MPEG-2 PMT/PSM Descriptors ...... 675 MPEG-4 PMT/PSM Descriptors ...... 689 ARIB PMT Descriptors ...... 692 ATSC PMT Descriptors ...... 695 DVB PMT Descriptors ...... 698 OpenCable PMT Descriptors ...... 704 Closed Captioning ...... 706 VBI Standard ...... 712 Teletext ...... 717 Active Format Description (AFD) ...... 718 Subtitles ...... 720 Enhanced Television Programming ...... 725 Data Broadcasting ...... 727 Decoder Considerations ...... 732 References ...... 737

Chapter 14 • MPEG-4 and H.264 738

Audio Overview ...... 739 General Audio Object Types ...... 739 Speech Object Types ...... 740 Synthesized Speech Object Types ...... 740 Synthesized Audio Object Types ...... 740 Visual Overview ...... 741 YCbCr Color Space ...... 741 Visual Objects ...... 741 MPEG-4 Part 2 Natural Visual Object Types ...... 741 MPEG-4 Part 2 Natural Visual Profiles ...... 743 Graphics Overview ...... 747 Visual Layers ...... 747 Visual Object Sequence (VS) ...... 747 Video Object (VO) ...... 747 Video Object Layer (VOL) ...... 747 Group of Video Object Plane (GOV) ...... 749 Video Object Plane (VOP) ...... 749 xvi Contents

Object Description Framework ...... 749 Object Descriptor (OD) Stream ...... 749 Object Content Information (OCI) ...... 751 Intellectual Property Management and Protection (IPMP) ...... 751 Scene Description ...... 751 BIFS ...... 751 Synchronization of Elementary Streams ...... 753 Sync Layer ...... 753 DMIF Application Interface ...... 754 Multiplexing of Elementary Streams ...... 754 FlexMux ...... 755 MPEG-4 Over MPEG-2 ...... 755 MP4 File Format ...... 755 Intellectual Property Management and Protection (IPMP) ...... 755 MPEG-4 Part 10 (H.264) Video ...... 756 Profiles and Levels ...... 756 Supplemental Enhancement Information (SEI) Messages ...... 758 Video Coding Layer ...... 759 Network Abstraction Layer (NAL) ...... 762 References ...... 763

Chapter 15 • ATSC 764

Video Capability ...... 766 Audio Capability ...... 766 Program and System Information Protocol (PSIP) ...... 768 Required Tables ...... 768 Optional Tables ...... 768 Descriptors ...... 770 E-VSB ...... 772 Data Broadcasting ...... 773 Application Block Diagrams ...... 774 References ...... 777 Contents xvii

Chapter 16 • OpenCable™ Digital Television 778

Video Capability ...... 780 Audio Capability ...... 780 In-Band System Information (SI) ...... 780 Required Tables ...... 781 Optional Tables ...... 782 Descriptors ...... 784 Out-of-Band System Information (SI) ...... 786 Tables ...... 786 Descriptors ...... 788 In-Band Data Broadcasting ...... 790 Data Service Announcements ...... 790 Service Description Framework (SDF) ...... 791 Conditional Access ...... 791 Related Technologies ...... 792 Application Block Diagrams ...... 792 References ...... 795

Chapter 17 • DVB Digital Television 796

Video Capability ...... 798 Audio Capability ...... 798 System Information (SI) ...... 798 Required Tables ...... 798 Optional Tables ...... 799 Descriptors ...... 804 Data Broadcasting ...... 808 Conditional Access ...... 808 Application Block Diagrams ...... 810 References ...... 810 xviii Contents

Chapter 18 • ISDB Digital Television 812

ISDB-S (Satellite) ...... 813 ISDB-C (Cable) ...... 813 ISDB-T (Terrestrial) ...... 814 Video Capability ...... 814 Audio Capability ...... 814 Still Picture Capability ...... 814 Graphics Capability ...... 814 System Information (SI) ...... 816 Tables ...... 816 Descriptors ...... 817 Captioning ...... 825 Data Broadcasting ...... 825 Application Block Diagrams ...... 826 References ...... 826

Chapter 19 • IPTV 827

Considerations ...... 827 Multicasting ...... 828 RTSP-Based Solutions ...... 828 RTSP ...... 828 RTP ...... 830 RTCP ...... 833 RSVP ...... 834 ISMA ...... 834 Broadcast over IP ...... 835 Conditional Access (DRM) ...... 835 References ...... 835

Chapter 20 • Glossary 837

Index 889 Contents xix About the Author

About the Author

Keith Jack is Director of Product Marketing at Sigma Designs. Sigma Designs develops and markets high-performance, highly-integrated System-on-a-Chip (SoC) semiconductors for IPTV Set-top Boxes, Blu-ray and HD DVD Players/Recorders, HDTVs, Digital Media Adapters, and Por- table Media Players. Prior to joining Sigma Designs, Mr. Jack held various marketing and chip design positions at Harris Semiconductor, Brooktree, and Rockwell International. He has been involved in over 40 multimedia chips for the consumer market.

I dedicate this book to my wife Gabriela, and my two sons Ethan and Andy, all who have brought tremendous joy into my life.

xix