_________________________________________________________________ SMPTE 421M-2006 SMPTE STANDARD VC-1 Compressed Video Bitstream Format and Decoding Process _________________________________________________________________ Intellectual property notice Copyright 2003-2006 THE SOCIETY OF MOTION PICTURE AND TELEVISION ENGINEERS 3 Barker Ave. White Plains, NY 10601 +1 914 761 1100 Fax +1 914 761-3115 E-mail [email protected] Web http://www.smpte.org The user’s attention is called to the possibility that compliance with this document may require use of inventions covered by patent rights. By publication of this document, no position is taken with respect to the validity of these claims or of any patent rights in connection therewith. The patent holders have, however, filed statements of willingness to grant a license under these rights on fair, reasonable and nondiscriminatory terms and conditions to applicants desiring to obtain such a license. Contact information may be obtained from the SMPTE. No representation or warranty is made or implied that these are the only licenses that may be required to avoid infringement in the use of this document. © SMPTE 2003-2006 – All rights reserved Approved 24-February-2006 i Foreword SMPTE (the Society of Motion Picture and Television Engineers) is an internationally-recognized standards developing organization. Headquartered and incorporated in the United States of America, SMPTE has members in over 80 countries on six continents. SMPTE’s Engineering Documents, including Standards, Recommended Practices and Engineering Guidelines, are prepared by SMPTE’s Technology Committees. Participation in these Committees is open to all with a bona fide interest in their work. SMPTE cooperates closely with other standards-developing organizations, including ISO, IEC and ITU. SMPTE Engineering Documents are drafted in accordance with the rules given in Part XIII of its Administrative practices. This Standard 421M was prepared by Technology Committee C24. ii © SMPTE 2003-2006 – All rights reserved Introduction This document was prepared for the primary purpose of documenting the bitstream format and decoding process used in the VC-1 video decoder. It defines the bitstream syntax, semantics and constraints for compressed video bitstreams and describes the complete process required to decode them. VC-1, as defined in this document, consists of three profiles: Simple, Main, and Advanced. Simple and Main profile were originally developed for use in lower-bit-rate networked computing environments. As such, certain assumptions were made regarding the display environment (e.g. square pixel aspect ratio) to improve compression efficiency. The Advanced Profile adds extensive in-band metadata support to allow for optimized experience on a wide range of display devices. © SMPTE 2003-2006 – All rights reserved Approved 24-February-2006 iii SMPTE 421M SMPTE Standard: VC-1 Compressed Video Bitstream Format and Decoding Process Table of Contents TABLE OF CONTENTS IV TABLE OF FIGURES X TABLE OF TABLES XV 1 SCOPE 1 2 NORMATIVE REFERENCES 1 3 OVERVIEW 1 3.1 SYNTAX OVERVIEW (INFORMATIVE) 2 3.2 DECODING PROCESS OVERVIEW 4 3.3 ENCODING PROCESS OVERVIEW (INFORMATIVE) 6 3.4 DOCUMENT STRUCTURE (INFORMATIVE) 7 4 NOTATION 8 4.1 CONFORMANCE NOTATION 8 4.2 ARITHMETIC OPERATORS 9 4.3 LOGICAL OPERATORS 10 4.4 RELATIONAL OPERATORS 10 4.5 BITWISE OPERATORS 10 4.6 ASSIGNMENT 11 4.7 PRECEDENCE ORDER OF OPERATORS 11 4.8 MNEMONICS 11 4.9 PSEUDO-CODE OPERATIONS 11 4.10 BITSTREAM PARSING OPERATIONS 12 4.11 FUNCTION DEFINITIONS 13 4.12 DEFINITION OF TERMINOLOGY 14 4.13 INTERMEDIATE VARIABLES 20 4.14 ACRONYM DEFINITIONS 21 4.15 GUIDE TO INTERPRETING SYNTAX DIAGRAMS AND SYNTAX ELEMENTS 21 5 PICTURE SAMPLING AND OVERALL BITSTREAM STRUCTURE 22 5.1 INTRODUCTION 22 5.2 PROGRESSIVE CODING MODE 22 5.2.1 Input/output Format 22 5.2.2 Hierarchical Elements 22 5.3 INTERLACE CODING MODE 23 5.3.1 Input/Output Format for 4:2:0 Interlace 23 5.4 FRAME ORDERING 24 5.5 CONSTRAINTS 25 5.5.1 Minimum and maximum frame sizes 25 5.5.2 Maximum size of compressed bits 25 © 2006 SMPTE iv SMPTE 421M 5.5.3 Bitstream Construction Constraints 26 6 SEQUENCE AND ENTRY-POINT BITSTREAM SYNTAX AND SEMANTICS 26 6.1 SEQUENCE-LEVEL SYNTAX AND SEMANTICS 26 6.1.1 Profile (PROFILE)(2 bits) 30 6.1.2 Level (LEVEL)(3 bits) 30 6.1.3 Color-Difference Format (COLORDIFF_FORMAT) (2 bits) 30 6.1.4 Post processing Indicators 30 6.1.5 Post processing Flag (POSTPROCFLAG) (1 bit) 32 6.1.6 Maximum Horizontal Size of Picture (MAX_CODED_WIDTH)(12 bits) 32 6.1.7 Maximum Vertical Size of Picture (MAX_CODED_HEIGHT)(12 bits) 32 6.1.8 Pull down Flag (PULLDOWN) (1 bit) 32 6.1.9 Interlace Content (INTERLACE) (1 bit) 32 6.1.10 Frame Counter Flag (TFCNTRFLAG) (1 bit) 32 6.1.11 Frame Interpolation Flag (FINTERPFLAG)(1 bit) 32 6.1.12 Reserved Advanced Profile Flag (RESERVED)(1 bit) 32 6.1.13 Progressive Segmented Frame (PSF)(1 bit) 32 6.1.14 Display Extension Flag (DISPLAY_EXT) (1 bit) 32 6.1.15 Hypothetical Reference Decoder Indicator Flag (HRD_PARAM_FLAG)(1 bit) 39 6.2 ENTRY-POINT HEADER SYNTAX AND SEMANTICS 40 6.2.1 Broken Link Flag (BROKEN_LINK) (1 bit) 43 6.2.2 Closed Entry Point (CLOSED_ENTRY) (1 bit) 43 6.2.3 Pan Scan Present Flag (PANSCAN_FLAG) (1 bit) 43 6.2.4 Reference Frame Distance Flag (REFDIST_FLAG) (1 bit) 43 6.2.5 Loop Filter Flag (LOOPFILTER) (1 bit) 43 6.2.6 Fast UV Motion Compensation Flag (FASTUVMC) (1 bit) 43 6.2.7 Extended Motion Vector Flag (EXTENDED_MV)(1 bit) 43 6.2.8 Macroblock Quantization Flag (DQUANT)(2 bit) 43 6.2.9 Variable Sized Transform Flag (VSTRANSFORM)(1 bit) 43 6.2.10 Overlapped Transform Flag (OVERLAP) (1 bit) 44 6.2.11 Quantizer Specifier (QUANTIZER) (2 bits) 44 6.2.12 HRD Buffer Fullness (HRD_FULLNESS)(Variable Size) 44 6.2.13 Coded Size Flag (CODED_SIZE_FLAG) (1 bit) 44 6.2.14 Extended Differential Motion Vector Range Flag (EXTENDED_DMV)(1 bit) 45 6.2.15 Range Mapping Luma Flag (RANGE_MAPY_FLAG)(1 bit) 45 6.2.16 Range Mapping Color-Difference Flag (RANGE_MAPUV_FLAG)(1 bit) 45 7 PROGRESSIVE BITSTREAM SYNTAX AND SEMANTICS 46 7.1 PICTURE-LEVEL SYNTAX AND SEMANTICS 46 7.1.1 Picture layer 93 7.1.2 Slice Layer 105 7.1.3 Macroblock Layer 106 7.1.4 Block Layer 111 7.2 BITPLANE CODING SYNTAX 119 7.2.1 Invert Flag (INVERT) (1-bit) 119 7.2.2 Coding Mode (IMODE) (variable) 119 7.2.3 Bitplane Coding Bits (DATABITS) (variable) 120 8 PROGRESSIVE DECODING PROCESS 120 8.1 PROGRESSIVE I FRAME PICTURE DECODING 120 8.1.1 Progressive I Frame Picture Layer Decode 120 8.1.2 Macroblock Layer Decode 122 8.1.3 Block Layer Decode 123 8.2 PROGRESSIVE BI FRAME PICTURE DECODING 136 8.2.1 BFRACTION following picture type (main profile only) 136 8.2.2 No picture resolution index (RESPIC) 136 8.3 PROGRESSIVE P FRAME PICTURE DECODING 136 8.3.1 Skipped Frame Pictures 136 © 2006 SMPTE v SMPTE 421M 8.3.2 Out-of-bounds Reference Pixels 137 8.3.3 P Picture Types 137 8.3.4 P Picture Layer Decode 138 8.3.5 Macroblock Layer Decoding 140 8.3.6 Block Layer Decode 153 8.3.7 Rounding Control (RND) 164 8.3.8 Intensity Compensation 165 8.4 PROGRESSIVE B FRAME PICTURE DECODING 166 8.4.1 Skipped Anchor Frames 166 8.4.2 Out-of-bounds Reference Pixels 166 8.4.3 Progressive B Frame Picture Types 166 8.4.4 Progressive B Frame Picture Layer Decode 166 8.4.5 B Frame Macroblock Layer Decode 168 8.4.6 B Block Layer Decode 174 8.5 OVERLAPPED TRANSFORM 175 8.5.1 Overlap Smoothing in Main and Simple Profiles 176 8.5.2 Overlap Smoothing in Advanced Profile 177 8.6 IN-LOOP DEBLOCK FILTERING 178 8.6.1 I Picture In-loop Deblocking 178 8.6.2 P Picture In-loop Deblocking 179 8.6.3 B Picture In-loop Deblocking 181 8.6.4 Filter Operation 181 8.7 BITPLANE CODING 189 8.7.1 INVERT 190 8.7.2 IMODE 190 8.7.3 DATABITS 190 8.8 SYNC MARKERS (SIMPLE AND MAIN PROFILES ONLY) 195 8.9 PAN SCAN 197 8.9.1 Number of Pan Scan Windows 197 8.9.2 Pan Scan Parameters 198 8.9.3 Pan Scan Restrictions 198 9 INTERLACE BITSTREAM SYNTAX AND SEMANTICS 199 9.1 PICTURE-LEVEL SYNTAX AND SEMANTICS 199 9.1.1 Picture layer 240 9.1.2 Slice Layer 249 9.1.3 Macroblock Layer 249 9.1.4 Block Layer Syntax Elements 252 10 INTERLACE DECODING PROCESS 253 10.1 INTERLACE FIELD I PICTURE DECODING 253 10.1.1 Macroblock Layer Decode 253 10.1.2 Block Layer Decode 253 10.2 INTERLACE BI FIELD DECODING 254 10.3 INTERLACE FIELD P PICTURE DECODING 254 10.3.1 Handling of Top-Field First (TFF) 254 10.3.2 Out-of-bounds Reference Pixels 254 10.3.3 Reference Pictures 255 10.3.4 P Picture Types 258 10.3.5 Macroblock Layer Decode 258 10.3.6 Block Layer Decode 283 10.3.7 Rounding Control 284 10.3.8 Intensity Compensation 284 10.4 INTERLACE FIELD B PICTURE DECODING 285 10.4.1 Handling of TFF 286 10.4.2 Out-of-bounds Reference Pixels 286 10.4.3 Reference Pictures 287 © 2006 SMPTE vi SMPTE 421M 10.4.4 B Picture Types 287 10.4.5 B Macroblock Layer Decode 287 10.4.6 MV Prediction in B fields 291 10.4.7 Block Layer Decode 295 10.5 INTERLACE FRAME I PICTURE DECODING 295 10.5.1 Macroblock Layer Decode 295 10.5.2 Block Decode 295 10.6 INTERLACE BI FRAME DECODING 296 10.7 INTERLACE FRAME P PICTURE DECODING 296 10.7.1 Skipped Frames 296 10.7.2 Out-of-bounds Reference Pixels 297 10.7.3 Macroblock Layer Decode 297 10.7.4 Block Layer Decode 316 10.8 INTERLACE FRAME B PICTURE DECODING 316 10.8.1 Skipped Anchor Frames 317 10.8.2 Out-of-bounds Reference Pixels 317 10.8.3 BFRACTION 317 10.8.4 Bitplane coding of direct mode 317 10.8.5 4MVSWITCH and 4MVBPTAB 317 10.8.6 B Macroblock Layer Decode 317 10.8.7 B Block Layer Decode 321 10.9 OVERLAPPED TRANSFORM 322 10.9.1 Overlap Smoothing for Interlace Field Pictures