Digital Video and HDTV Algorithms and Interfaces The Morgan Kaufmann Series in Computer Graphics and Geometric Modeling Series Editor: Brian A. Barsky, University of California, Berkeley Digital Video and HDTV Algorithms Andrew Glassner’s Notebook: and Interfaces Recreational Computer Graphics Charles Poynton Andrew S. Glassner Texturing & Modeling: A Procedural Warping and Morphing of Graphical Approach, Third Edition Objects David S. Ebert, F. Kenton Musgrave, Jonas Gomes, Lucia Darsa, Bruno Darwyn Peachey, Ken Perlin, and Costa, and Luiz Velho Steven Worley Jim Blinn's Corner: Dirty Pixels Geometric Tools for Computer Graphics Jim Blinn Philip Schneider and David Eberly Rendering with Radiance: The Art Understanding Virtual Reality: and Science of Lighting Visualization Interface, Application, and Design Greg Ward Larson and Rob Shakes- William Sherman and Alan Craig peare Jim Blinn's Corner: Notation, Introduction to Implicit Surfaces Notation, Notation Edited by Jules Bloomenthal Jim Blinn Jim Blinn’s Corner: A Trip Down Level of Detail for 3D Graphics the Graphics Pipeline David Luebke, Martin Reddy, Jim Blinn Jonathan D. Cohen, Amitabh Varshney, Benjamin Watson, and Interactive Curves and Surfaces: Robert Huebner A Multimedia Tutorial on CAGD Alyn Rockwood and Peter Chambers Pyramid Algorithms: A Dynamic Programming Approach to Curves and Wavelets for Computer Graphics: Surfaces for Geometric Modeling Theory and Applications Ron Goldman Eric J. Stollnitz, Tony D. DeRose, and David H. Salesin Non-Photorealistic Computer Graphics: Modeling, Rendering, and Animation Principles of Digital Image Synthesis Thomas Strothotte and Stefan Andrew S. Glassner Schlechtweg Radiosity & Global Illumination Curves and Surfaces for CAGD: François X. Sillion and Claude Puech A Practical Guide, Fifth Edition Knotty: A B-Spline Visualization Gerald Farin Program Subdivision Methods for Geometric Jonathan Yen Design: A Constructive Approach User Interface Management Systems: Joe Warren and Henrik Weimer Models and Algorithms Computer Animation: Algorithms Dan R. Olsen, Jr. and Techniques Making Them Move: Mechanics, Rick Parent Control, and Animation of Articulated The Computer Animator’s Technical Figures Handbook Edited by Norman I. Badler, Brian A. Lynn Pocock and Judson Rosebush Barsky, and David Zeltzer Advanced RenderMan: Creating CGI Geometric and Solid Modeling: for Motion Pictures An Introduction Anthony A. Apodaca and Larry Gritz Christoph M. Hoffmann Curves and Surfaces in Geometric An Introduction to Splines for Use Modeling: Theory and Algorithms in Computer Graphics and Geometric Jean Gallier Modeling Richard H. Bartels, John C. Beatty, and Brian A. Barsky Publishing Director: Diane Cerra Publishing Services Manager: Edward Wade Production Editor: Howard Severson Design, illustration, and composition: Charles Poynton Editorial Coordinator: Mona Buehler Cover Design: Frances Baca Copyeditor: Robert Fiske Proofreader: Sarah Burgundy Printer: The Maple-Vail Book Manufacturing Group Cover images: Close-up of woman/Eyewire; Circuit connection/Artville; Medical prespectives/Picture Quest Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. Morgan Kaufmann Publishers An imprint of Elsevier Science 340 Pine Street, Sixth Floor San Francisco, CA 94104-3205 www.mkp.com Copyright 2003 by Elsevier Science (USA). All rights reserved. Printed in the United States of America 2007 2006 2005 2004 2003 5 4 3 2 1 No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means – electronic, mechanical, photocopying, scanning or otherwise – without prior written permission of the Publisher. Library of Congress Control Number: 2002115312 ISBN: 1-55860-792-7 This book is printed on acid-free paper. Dedication In memory of my mother, Marjorie Johnston 1926—2002 and to Quinn and Georgia, and the new family tree Digital Video and HDTV Algorithms and Interfaces Charles Poynton Part 1 Introduction 1 Raster images 3 2 Quantization 17 3 Brightness and contrast controls 25 4 Raster images in computing 31 5 Image structure 43 6 Raster scanning 51 7 Resolution 65 8 Constant luminance 75 9 Rendering intent 81 10 Introduction to luma and chroma 87 11 Introduction to component SDTV 95 12 Introduction to composite NTSC and PAL 103 13 Introduction to HDTV 111 14 Introduction to video compression 117 15 Digital video interfaces 127 1 Raster images 1 This chapter introduces the basic features of the pixel array. I explain how the pixel array is digitized from the image plane, how pixel values are related to brightness and color, and why most imaging systems use pixel values that are nonlinearly related to light intensity. Imaging In human vision, the three-dimensional world is imaged by the lens of the eye onto the retina, which is popu- lated with photoreceptor cells that respond to light having wavelengths ranging from about 400 nm to 700 nm. In video and in film, we build a camera having a lens and a photosensitive device, to mimic how the world is perceived by vision. Although the shape of the retina is roughly a section of a sphere, it is topologi- cally two dimensional. In a camera, for practical reasons, we employ a flat image plane, sketched in Figure 1.1 below, instead of a section of a sphere. Image science concerns analyzing the continuous distribution of optical power that is incident on the image plane. Figure 1.1 Scene, lens, image plane 3 Figure 1.2 Aspect ratio of video, Widescreen SDTV, HDTV, and film are compared. Video Video HDTV Aspect ratio is properly written image 4:3 16:9 width:height (not height:width). 1.33:1 1.78:1 Film 35 mm still film image 3:2 Cinema film Cinema film 1.5:1 1.85:1 2.39:1 Aspect ratio Schubin, Mark, “Searching for the Aspect ratio is simply the ratio of an image’s width to its Perfect Aspect Ratio,” in SMPTE height. Standard aspect ratios for film and video are Journal 105 (8): 460–478 (Aug. sketched, to scale, in Figure 1.2 above. Conventional 1996). The 1.85:1 aspect ratio is achieved with a spherical lens (as standard-definition television (SDTV) has an aspect ratio opposed to the aspherical lens of 4:3. Widescreen refers to an aspect ratio wider than used for anamorphic images). 4:3. Widescreen television and high-definition televi- sion (HDTV) have an aspect ratio of 16:9. Cinema film commonly uses 1.85:1 (“flat,” or “spherical”). In Europe and Asia, 1.66:1 is usually used. The 2.39:1 ratio for cinema film is To obtain 2.39:1 aspect ratio (“Cinemascope,” or collo- recent; formerly, 2.35:1 was used. quially, “scope”), film is typically shot with an aspher- The term anamorphic in video usually ical lens that squeezes the horizontal dimension of the refers to a 16:9 widescreen variant of a base video standard, where the image by a factor of two. The projector is equipped horizontal dimension of the 16:9 with a similar lens, to restore the horizontal dimension image is transmitted in the same of the projected image. The lens and the technique are time interval as the 4:3 aspect ratio called anamorphic. In principle, an anamorphic lens can standard. See page 99. have any ratio; in practice, a ratio of two is ubiquitous. Film can be transferred to 4:3 video by cropping the sides of the frame, at the expense of losing some picture content. Pan-and-scan, sketched in Figure 1.3 opposite, refers to choosing, on a scene-by-scene basis during film transfer, the 4:3 region to be maintained. Many directors and producers prefer their films not to be altered by cropping, so many movies on VHS and DVD are released in letterbox format, sketched in Figure 1.4 opposite. In letterbox format, the entire film image is maintained, and the top and bottom of the 4:3 frame are unused. (Either gray or black is displayed.) 4 DIGITAL VIDEO AND HDTV ALGORITHMS AND INTERFACES 4:3 16:9 4:3 16:9 4:3 16:9 Figure 1.3 Pan-and-scan Figure 1.4 Letterbox Figure 1.5 Pillarbox format crops the width of widescreen format fits widescreen (sometimes called sidebar) fits material – here, 16:9 – for material – here, 16:9 – to narrow-aspect-ratio material a 4:3 aspect ratio display. the width of a 4:3 display. to the height of a 16:9 display. With the advent of widescreen consumer television receivers, it is becoming common to see 4:3 material displayed on widescreen displays in pillarbox format, in Figure 1.5. The full height of the display is used, and the left and right of the widescreen frame are blanked. Digitization Signals captured from the physical world are translated into digital form by digitization, which involves two processes, sketched in Figure 1.6 overleaf. A signal is digitized by subjecting it to both sampling (in time or space) and quantization (in amplitude). The operations may take place in either order, though sampling usually precedes quantization. Quantization assigns an integer to signal amplitude at an instant of time or a point in space, as I will explain in Quantization, on page 17. 1-D sampling A continuous one-dimensional function of time, such as sound pressure of an audio signal, is sampled through forming a series of discrete values, each of which is a function of the distribution of intensity across a small interval of time. Uniform sampling, where the time intervals are of equal duration, is nearly always used. Details will be presented in Filtering and sampling, on page 141.