Entropy Coding in HEVC The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation Sze, Vivienne, and Detlev Marpe. “Entropy Coding in HEVC.” High Efficiency Video Coding (HEVC) (2014): 209–274. As Published http://dx.doi.org/10.1007/978-3-319-06895-4_8 Publisher Springer-Verlag Version Author's final manuscript Citable link http://hdl.handle.net/1721.1/100315 Terms of Use Creative Commons Attribution-Noncommercial-Share Alike Detailed Terms http://creativecommons.org/licenses/by-nc-sa/4.0/ Entropy Coding in HEVC Vivienne Sze and Detlev Marpe Abstract Context-Based Adaptive Binary Arithmetic Coding (CABAC) is a method of entropy coding first introduced in H.264/AVC and now used in the latest High Ef- ficiency Video Coding (HEVC) standard. While it provides high coding efficiency, the data dependencies in H.264/AVC CABAC make it challenging to parallelize and thus limit its throughput. Accordingly, during the standardization of entropy coding for HEVC, both aspects of coding efficiency and throughput were considered. This chapter describes the functionality and design methodology behind CABAC entropy coding in HEVC. 1 Introduction Context-Based Adaptive Binary Arithmetic Coding (CABAC) [51] is a form of en- tropy coding used in H.264/AVC [3] and also in HEVC [5]. Entropy coding is a loss- less compression scheme that uses the statistical properties to compress data such that the number of bits used to represent the data is logarithmically proportional to the probability of the data. For instance, when compressing a string of characters, frequently used characters are each represented by a few bits, while infrequently used characters are each represented by many bits. From Shannon’s information theory [72], when the compressed data is represented in bits f0,1g, the optimal av- erage code length for a character with probability p is −log2 p. Entropy coding is performed at the last stage of video encoding (and first stage of video decoding), after the video signal has been reduced to a series of syntax elements. Syntax elements describe how the video signal can be reconstructed at Vivienne Sze Massachusetts Institute of Technology (MIT) e-mail: [email protected] Detlev Marpe Fraunhofer Institute for Telecommunications Heinrich Hertz Institute (HHI) e-mail: [email protected] 1 2 Vivienne Sze and Detlev Marpe the decoder. This includes the method of prediction (e.g. spatial or temporal predic- tion) along with its associated prediction parameters as well as the prediction error signal, also referred to as the residual signal. Note that in HEVC only the syntax elements belonging to the slice segment data are CABAC encoded. All other high level syntax elements are coded either with zero-order Exp-Golomb codes or fixed- pattern bit strings. Tab. 1 shows the syntax elements that are encoded with CABAC in HEVC and H.264/AVC. For HEVC, these syntax elements describe properties of the coding tree unit (CTU), prediction unit (PU), and transform unit (TU), while for H.264/AVC, the equivalent syntax elements have been grouped together along the same categories in Tab. 1. For a CTU, the related syntax elements describe the block partitioning of the CTU into coding units (CU), whether the CU is intra-picture (i.e. spatially) predicted or inter-picture (i.e., temporally) predicted, the quantization pa- rameters of the CU, and the type (edge or band) and offsets for sample adaptive offset (SAO) in-loop filtering performed on the CTU. For a PU, the syntax elements describe the intra prediction mode or the motion data. For a TU, the syntax elements describe the residual signal in terms of frequency position, sign and magnitude of the quantized transform coefficients. This chapter describes how CABAC entropy coding has evolved from H.264/AVC to HEVC. While high coding efficiency is important for reducing the transmission and storage cost of video, processing speed and area cost also need to be considered in the development of HEVC in order to handle the demand for higher resolutions and frame rates in future video coding systems. Accordingly, both coding efficiency and throughput improvement tools are discussed. Sect. 2 provides an overview of CABAC entropy coding. Sect. 3 explains the design considerations and techniques used to address both coding efficiency and throughput requirements. Sect. 4, Sect. 5, Sect. 6 and Sect. 7 describe how these techniques were applied to coding tree unit coding, prediction unit coding, transform unit coding and context initialization, re- spectively. Sect. 8 compares the coding efficiency, throughput and memory require- ments of HEVC and H.264/AVC for both common conditions and worst case con- ditions. 2 CABAC Overview The CABAC algorithm was originally developed within the joint H.264/AVC stan- dardization process of ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Moving Picture Experts Group (MPEG). In a first preliminary version, the new entropy-coding method of CABAC was introduced as a standard contribution [45] to the ITU-T VCEG meeting in January 2001. CABAC was adopted as one of two alternative methods of entropy coding within the H.264/AVC standard. The other method specified in H.264/AVC was a low-complexity entropy-coding technique based on the usage of context-adaptively switched sets of variable-length codes, so- called Context-Adaptive Variable-Length Coding (CAVLC). Compared to CABAC, CAVLC offers reduced implementation cost at the price of lower compression ef- Entropy Coding in HEVC 3 Table 1: CABAC coded syntax elements in HEVC and H.264/AVC. HEVC H.264/AVC Coding Block and split cu flag, pred mode flag, part mode, Prediction Block pcm flag, cu transquant bypass flag, mb type, sub mb type, mb skip flag, Structure, and cu skip flag, cu qp delta abs, mb qp delta, end of slice flag, Quantization cu qp delta sign flag, mb field decoding flag Coding Tree Unit Parameters end of slice segment flag, (CTU) and [Sect. 4.1–4.4] end of sub stream one bit Coding Unit (CU) sao merge left flag, sao merge up flag, Sample Adaptive [Sect. 4] sao type idx luma, sao type idx chroma, Offset (SAO) sao offset abs, sao offset sign, n/a Parameters sao band position, sao eo class luma, [Sect. 4.5] sao eo class chroma prev intra4x4 pred mode flag, Intra Prediction prev intra luma pred flag, mpm idx, prev intra8x8 pred mode flag, Mode Data rem intra luma pred mode, rem intra4x4 pred mode, Prediction Unit [Sect. 5.2] intra chroma pred mode rem intra8x8 pred mode, (PU) intra chroma pred mode [Sect. 5] merge flag, merge idx, inter pred idc, Motion Data ref idx l0, ref idx l1, abs mvd greater0 flag, ref idx l0, ref idx l1, mvd l0, [Sect. 5.1] abs mvd greater1 flag, abs mvd minus2, mvd l1 mvd sign flag, mvp l0 flag, mvp l1 flag rqt root cbf, split transform flag, cbf luma, cbf cb, cbf cr, transform skip flag, coded block flag, last sig coeff x prefix, coded block pattern, Transform Unit Residual Data last sig coeff y prefix, transform size 8x8 flag, (TU) [Sect. 6.1, 6.2, 6.3, last sig coeff x suffix, significant coeff flag, [Sect. 6] 6.4, 6.5] last sig coeff y suffix, coded sub block flag, last significant coeff flag, sig coeff flag, coeff abs level greater1 flag, coeff abs level minus1, coeff abs level greater2 flag, coeff sign flag coeff abs level remaining, coeff sign flag 4 Vivienne Sze and Detlev Marpe ficiency. Typically, the bit-rate overhead for CAVLC relative to CABAC is in the range of 10–16% for standard definition (SD) interlaced material, encoded at Main Profile, and 15–22% for high definition (HD) 1080p material, encoded at High Pro- file, both measured at the same objective video quality and for the case that all other used coding tools within the corresponding H.264/AVC Profile remain the same [51, 50]. CABAC became also part of the first HEVC test model HM1.0 [55] together with the so-called low-complexity entropy coding (LCEC) as a follow-up of CAVLC. Later, during the HEVC standardization process, it turned out that the discrimina- tion between high efficiency and low complexity was no longer necessary. Thus, CABAC in its improved form, both with respect to throughput speed and compres- sion efficiency, became the single entropy coding method of the HEVC standard. The basic design of CABAC involves the key elements of binarization, context modeling, and binary arithmetic coding. These elements are illustrated as the main algorithmic building blocks of the CABAC encoding block diagram in Fig. 1. Bi- narization maps the syntax elements to binary symbols (bins). Context modeling estimates the probability of each non-bypassed (i.e., regular coded) bin based on some specific context. Finally, binary arithmetic coding compresses the bins to bits according to the estimated probability. Fig. 1: CABAC block diagram (from the encoder perspective): Binarization, context modeling (including probability estimation and assignment), and binary arithmetic coding. In red: Potential throughput bottlenecks, as further discussed from the de- coder perspective in Sect. 3.2. Entropy Coding in HEVC 5 2.1 Binarization The coding strategy of CABAC is based on the finding that a very efficient cod- ing of non-binary syntax-element values in a hybrid block-based video coder, like components of motion vector differences or transform-coefficient level values, can be achieved by employing a binarization scheme as a kind of preprocessing unit for the subsequent stages of context modeling and arithmetic coding. In general, a bi- narization scheme defines a unique mapping of syntax element values to sequences of binary symbols, so-called bins, which can also be interpreted in terms of a bi- nary code tree.