DirectX Video Acceleration Specification for H.264/MPEG-4 AVC Multiview Video Coding (MVC), Including the Stereo High Profile Gary J. Sullivan and Yongjun Wu Microsoft Corporation March 2011 Applies to: DirectX Video Acceleration Summary: Defines extensions to DirectX Video Acceleration (DXVA) to support Multiview Video Coding (MVC) according to the H.264/AVC specification of MVC. Support of the H.264/AVC Stereo High Profile is an important special case. This document describes high-level design concepts and specific MVC extensions to the DXVA interfaces and data structures for H.264/AVC decoding. Only off-host VLD profiles for H.264/AVC-based MVC decoding are specified. The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place or event is intended or should be inferred. Microsoft does not make any representation or warranty regarding specifications in this document or any product or item developed based on these specifications. Microsoft disclaims all express and implied warranties, including but not limited to the implied warranties or merchantability, fitness for a particular purpose and freedom from infringement. Without limiting the generality of the foregoing, Microsoft does not make any warranty of any kind that any item developed based on these specifications, or any portion of a specification, will not infringe any copyright, patent, trade secret or other intellectual property right of any person or entity in any country. It is your responsibility to seek licenses for such intellectual property rights where appropriate. Microsoft shall not be liable for any damages arising out of or in connection with the use of these specifications, including liability for lost profit, business interruption, or any other damages whatsoever. Some states do not allow the exclusion or limitation of liability or consequential or incidental damages; the above limitation may not apply to you. © 2011 Microsoft Corporation. All rights reserved. Microsoft, MS-DOS, Windows, Windows Media, Windows NT, Windows Server, Windows Vista, Active Directory, ActiveSync, ActiveX, Direct3D, DirectDraw, DirectInput, DirectMusic, DirectPlay, DirectShow, DirectSound, DirectX, Expression, FrontPage, HighMAT, Internet Explorer, JScript, Microsoft Press, MSN, NetShow, Outlook, PlaysForSure logo, PowerPoint, SideShow, Visual Basic, Visual C++, Visual InterDev, Visual J++, Visual Studio, WebTV, Win32, and Win32s are either registered trademarks or trademarks of Microsoft Corporation in the U.S.A. and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners. © 2011 Microsoft Corporation. All rights reserved. Contents Contents ................................................................................................................................... 3 1. Introduction .......................................................................................................................... 4 2. Normative references ...................................................................................................... 4 2.1 The H.264/AVC standard .............................................................................................. 4 2.2 The DXVA Specification for Ordinary H.264/AVC Bitstream Decoding ......................... 4 3. Overview of the MVC Extensions of H.264/AVC .................................................................. 4 3.1 Basic Design Structure .................................................................................................. 4 3.2 The Multiview High Profile ............................................................................................. 5 3.3 The Stereo High Profile ................................................................................................. 5 4. Basic DXVA design approach .............................................................................................. 5 4.1 Consistency with DXVA design for ordinary H.264/AVC bitstream decoding ................ 5 4.2 Support for Multiview High Profile versus Stereo High Profile ....................................... 5 4.3 Basic Handling of Frame Buffers ................................................................................... 6 4.4 Differences In Relation To Ordinary H.264/AVC DXVA Decoding ................................ 6 4.5 Support for Off-Host VLD Operation Only ..................................................................... 6 4.6 No Planned Support for Film-Grain Synthesis ............................................................... 7 5. DXVA Extensions for MVC ................................................................................................... 7 5.1 Extensions to DXVA Decoding Operations ................................................................... 7 5.2 Extensions to Configuration Parameters ....................................................................... 7 5.3 Extensions to the DXVA_PicEntry_H264 Structure ....................................................... 8 5.4 Extensions to the Picture Parameters Data Structure ................................................... 8 5.5 Extensions to Slice Control Data Structure ................................................................. 13 5.6 Extensions to Restricted Mode Profiles ....................................................................... 13 5.6.1 DXVA_ModeH264_VLD_Stereo_High_Progressive Profile ................................ 13 5.6.2 DXVA_ModeH264_VLD_Stereo_High Profile ..................................................... 14 5.6.3 DXVA_ModeH264_VLD_Multiview_High Profile ................................................. 15 Annex A Compatibility with H.264/AVC Decoding and Display .............................................. 16 A.1 Host Conversion from H.264/AVC-based MVC to H.264/AVC .................................... 16 A.2 Single-View Display with H.264/AVC-based MVC ...................................................... 16 A.3 Single-View H.264/AVC Decoding with MVC DXVA ................................................... 17 For More Information.............................................................................................................. 17 1. Introduction This document defines extensions to DirectX® Video Acceleration (DXVA) to support decoding of the Multiview Video Coding (MVC) extensions to the H.264/MPEG-4 Advanced Video Coding (AVC) specification. This includes support of the H.264/AVC Stereo High Profile as an important special case. This document is currently in draft form. In this document, the term "ordinary H.264/AVC bitstream" refers to an H.264/AVC bitstream that does not use the SVC and MVC extensions that are specified in Annexes G and H of the H.264/AVC specification. This document assumes that the reader is familiar with the H.264/AVC standard specification and its MVC extensions in Annex H, and the DXVA specification for ordinary H.264/AVC bitstream decoding. 2. Normative references 2.1 The H.264/AVC standard The Multiview Video Coding (MVC) extensions to the H.264/AVC standard (referred to hereafter as MVC) are specified in Annex H of ITU-T Rec. H.264, ISO/IEC 14496-10: Advanced video coding for generic audiovisual services. The standard is available at http://www.itu.int/rec/T-REC-H.264. Unless otherwise specified, this document refers to the edition approved by ITU-T in March 2010, which is the version given in the URL. 2.2 The DXVA Specification for Ordinary H.264/AVC Bitstream Decoding The DXVA specification for ordinary H.264/AVC bitstream decoding is found in the Microsoft publication DirectX Video Acceleration Specification for H.264/AVC Decoding," available at http://go.microsoft.com/fwlink/?LinkId=107379. Unless otherwise specified, this document refers to the edition that includes updates as of December 2010. 3. Overview of the MVC Extensions of H.264/AVC This section provides a general overview of the MVC extensions. 3.1 Basic Design Structure The MVC design is specified primarily in Annex H of the H.264/MPEG-4 Advanced Video