ECE 5578 Multimedia Communication

Lec 15 - HEVC Rate Control

Guest Lecturer: Li Li

Dept of CSEE, UMKC Office: FH560E, Email: [email protected], Ph: x 2346. http://l.web.umkc.edu/lizhu

slides created with WPS Office Linux and EqualX LaTex equation editor

Z. Li, ECE 5578 Multimedia Communciation, 2020 Outline

1 Background

2 Related Work

3 Proposed Algorithm

4 Experimental Results

5 Conclusion Rate control introduction

 Assuming the bandwidth is 2Mbps, the encode parameters should be adjusted to reach the bandwidth

Loss of data > 2 2 X

Unable to make full use of the < 2 bandwidth  How to reach the bandwidth accurately and provide the best quality ?

min D s.t. R £ Rt paras

Optimal Rate Control Rate control applications

 Rate control can be used in various scenarios

CBR: Constant Bitrate VBR:

wire wireless

Rate control

storage

ABR: Rate distortion optimization

 Rate distortion optimization

min D s.t. R £ Rt Û min D + l(Rt )R paras paras

λ determines the optimization target

λ Background

 Rate control  Non-normative part of video coding standards  Quite widely used: of great importance

 Considering its importance , rate control algorithm is always included in the reference software of standards

 MPEG2: TM5

 H.263: VM8

 H.264: Q-domain rate control

 HEVC: λ-domain rate control (JCTVC K0103, M0036) Outline

1 Background

2 Related Work

3 Proposed Algorithm

4 Experimental Results

5 Conclusion Related work

 Q-domain rate control algorithm  Yaqin Zhang：Second-order R-Q model  The improvement of RMSE in model prediction diminishes as the degree exceeds two. R = aQ-1 + bQ-2 Order RMSE 1 104.382 2 39.433 3 29.057  Q-domain extension in HEVC (pixel-domain) MAD MAD bpp = a´ + b´ QP QP2 Q/QP is the key factor to determine the bitrate Related work

 ρ-domain rate control algorithm  Zhihai He: linear R-ρ model  ρ: the percentage of non zero coefficients after transform and quantization  The coding of a picture is much more closely related to the percentage of zeros ρ than to the quantization parameter q R = q ×(1- r )

(R- (R- ρ) Q)

ρ is the key factor to determine the bitrate Related work

 Existing problems  does not exist a one-to-one correspondence between R and Q/ρ, mode and motion have significant influence on the bitrate and distortion  Q/ρ is unable to determine the non-residue bits  “chicken and egg” dilemma Outline

1 Background

2 Related Work

3 Proposed Algorithm

4 Experimental Results

5 Conclusion λ Domain R-D analysis (1)

 λ domain R-D analysis  Use the Lagrange multiplier λ to characterize the R-D relationship  There exists a one-to-one correspondence between R and λ, QP/mode/motion should be determined through RDO  λ can determine both the residue and non-residue bits  λ is a continuous value

λ λ Domain R-D analysis (2)

 Verification of the λ domain R-D analysis  QP and λ change the same extent, the bitrate changes much more significantly when λ changes. λ domain R-D analysis (3)

 R-D model selection

D(R) = Ce-KR D(R) = CR -K

 R2 means the coefficient of determination which reflects the degree of fitting between the estimated and actual values R2 = 1- SSE / SST λ domain R-D analysis (4)

 R-λ model derivation

-K ¶D -K -1 b b1 D(R) = CR l = - = CKR = aR R = a1l ¶R

BasketballDrill RaceHorsesC

0.3 0.8 -0.704 bpp = 0.8698λ-0.592 bpp = 2.9066λ 0.25 0.7 0.6 0.2 0.5 0.15 b 0.4 b p p p p 0.1 0.3 0.2 0.05 0.1 0 0 0 50 100 150 200 250 0 50 100 150 200 250 lambda value lambda value λ domain R-D analysis (5)

 D-λ model derivation

-K -K b1 b2 D = CR = C (a1l ) = a2l Rate control process

 General rate control process

Bit allocation Reach target bitrate

Assign bits to each unit Achieve the target bitrate ( t h e unit can be a GOP, a for each unit picture or a BU) 1. Determine encoder 1. GOP level bit parameters for each allocation picture

2. Picture level bit 2. Determine encoder allocation parameters for each BU 3. Basic unit level bit allocation 3. Complete encoding Bit allocation (1)

 GOP level bit allocation  Generally, the target bits of a GOP is determined by the current network conditions and buffer condition B F ´SW -V T = ´N G SW G  V is the buffer size, if the buffer fullness is high, the current GOP should be assigned less number of bits  SW: the bits should be kept equal to the target bitrate within the size of the sliding window

 B: bandwidth; F: ; NG: GOP size

f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12

GOP GOP GOP Bit allocation (2)

 First picture bit allocation  Three related factors  Intra frame complexity: the bits the intra frame will cost under a certain quality – The larger the frame complexity, the larger the intra bits ratio

– SATDI to represent the intra frame complexity 7 7

– SATDI is the sum of the coefficients after SATDI = åå hi ,j - h0,0 i=0 j=0 Hadamard transform to 8x8 original block  The influence of the intra frame to the subsequent frames – SATDB to represent the influence of the intra frame to the subsequent frames – SATDB is the sum of the coefficients after 7 7 SATDB = åå ri, j Hadamard transform to 8x8 residue block after ME i=0 j=0  Target bitrate: The larger the target bitrate, the smaller the intra bits ratio b c d bitsRatio = a × SATDI × SATDB ×TBpp

Intra Intra Intra bits bits bits ratio ratio ratio

SATDI SATDB targetBitrate Bit allocation (3)

 Inter picture level bit allocation  Equal bit allocation: Low delay applications  Hierarchical bit allocation: based on the λ-domain R-D analysis and fundamental RDO theory N S NG

min DP s.t. RP £ RG l l ...l å i å i P1 P 2 P Ns i=1 i=1 NS

Ns NG ¶ D å Pi ¶R min D + l R i=1 j å Pi å Pi + l = 0 lP1lP 2 ...lP Ns i=1 i=1 ¶lP ¶lj j N NS NS S ¶ D ¶ D ¶ DP å Pi å Pi å i ¶DP ¶R i=1 × j + l j = 0 i=1 =1+ i= j+1 =1+q ¶D ¶l ¶l ¶D ¶D Pj Pj j j Pj Pj b l q Pi Pi Pj R a l = Pi Pi Pi = b Pj lP qP RP a l j i j Pj Pj Bit allocation (4)

 Basic Unit level bit allocation  Based on the λ-domain R-D analysis and fundamental RDO theory

NB NB min D s.t. R £ R å Bi å Bi P l l ...l NB NB B1 B 2 BN i=1 i=1 B ¶ D ¶ R NB NB å Bi å Bi i=1 + l i=1 = min DB + l RB 0 l l ...l å i å i B B 2 B ¶lB ¶lB 1 NB i=1 i=1 j j l = l The λ between different BUs Bi B j should be equal to each other b R a l Bi Bi Bi Bi Content-related bits ratio = b B j between different BUs RB a l j B j B j Achieving of the target bitrate

 Encoding parameter determination  Lambda determination l = a Rb  QP determination  The optimal method – Determine the QP through RDO based multiple QP optimization – Quite complex process  Sub-optimal method – Find a simple relationship between λ and QP

QP = C1 ln l + C2

C1 = 4.2005;C2 =13.7122 R-λ model parameter updating (1)

 Intra picture consideration (1)  Intra picture is of great importance for the whole sequence  The model parameters α and β should be carefully determined l b » - 2 .0 a = adjust bpp-2.0 R-λ model parameter updating (2)

 Intra picture consideration (2)

2.6729 -2.0 l = (0.0064× SATDI )×bpp R-λ model parameter updating (3)

 Inter picture consideration  Minimize the squared error between the real used λ and the calculated λ according to the model

2 2 b e = (lnlreal - lnlmodel ) lmodel = a ×bppreal

2 2 2 min e = min(lnlreal - lnlmodel ) = min( lnlreal -a - b ln bppreal ) a ,b a ,b a ,b

 Model updating formula (the updating speed δα, δβ determines the updating speed, and relates to bpp)

anew = aold + da (ln lreal - ln lmodel ) ×aold

bnew = bold +db (ln lreal - ln lmodel ) ×ln(bppreal ) Outline

1 Background

2 Related Work

3 Proposed Algorithm

4 Experimental Results

5 Conclusion Experimental results

 Bitrate accuracy  Bitrate error Average Bitrate Error LD RA R-Q model 0.16% 0.78% R-λ model 0.10% 0.22%

 Per frame bit cost Equal bit allocation Hierarchical bit allocation Experimental results

 Objective quality  Low delay: average 0.55dB, maximum 1.81dB improvement  Random access: average 1.81dB, maximum 3.77dB improvement Experimental results

 Subjective quality

R-Q R-λ Outline

1 Background

2 Related Work

3 Proposed Algorithm

4 Experimental Results

5 Conclusion Conclusion

 We develop a complete comprehensive λ-domain R-D analysis framework, which can fully reflect the inherent R-D characteristics governed by the video content.  We propose both picture-level and BU-level optimal bit allocation algorithms based on λ-domain R-D analysis framework.  We propose λ-domain rate control algorithm based on λ-domain R-D analysis framework, which achieve the target bitrates more accurately with significant R-D performance gain.  The proposed rate control and bit allocation algorithm have been adopted by HEVC and integrated into the HEVC reference software. References

1. Bin Li, Houqiang Li, Li Li, Jinlei Zhang, “Rate Control for High Efficiency Video Coding via λ domain Model”, IEEE TIP 23(9) 2014.

2. Li Li, Bin Li, Houqiang Li, Chang Wen Chen, “λ Domain Optimal Bit Allocation Algorithm for High Efficiency Video Coding”, IEEE TCSVT (In press)

3. Li Li, Bin Li, Dong Liu, Houqiang Li, “λ-Domain Rate Control Algorithm for HEVC Scalable Extension”, IEEE TMM (In press)

4. Bin Li, Houqiang Li, Li Li, Jinlei Zhang, “Rate control by R-lambda model for HEVC”, JCTVC-K0103.

5. Bin Li, Houqiang Li, Li Li, “Adaptive bit allocation for R-lambda rate control for HEVC”, JCTVC-M0036. Thank you!

Z. Li, ECE 5578 Multimedia Communciation, 2020