Master's Thesis
Total Page:16
File Type:pdf, Size:1020Kb
MASTER'S THESIS Compression of High Dynamic Range Video Simon Ekström 2015 Master of Science in Engineering Technology Computer Science and Engineering Luleå University of Technology Department of Computer Science, Electrical and Space Engineering Abstract For a long time the main interest in the TV-industry has been in increasing the resolution of the video. However, we are getting to a point where there is little benefit in increasing it even further. New technologies are quickly rising as a result of this and High Dynamic Range (HDR) video is one of these. The goal of HDR video is to provide a greater range of luminosity to the end-consumer. MPEG (Moving Picture Experts Group) wants to know if there is a potential for improvements to the HEVC (High Efficiency Video Coding) standard, specifically for HDR video, and in early 2015 the group issued a Call for Evidence (CfE) to find evidence whether improvements can be made to the existing video coding standard. This work presents the implementation and analysis of three different ideas for suggestions: bit shifting at the coding unit level, histogram- based color value mapping, and modifications to the existing Sample Adaptive Offset (SAO) in-loop filter in HEVC. Out of the three suggestions, the histogram-based color value mapping is shown to provide significant improvements to the coding efficiency, both objectively and subjectively. The thesis concludes the work with a discussion and possible directions for future work. Acknowledgements I am very grateful for the opportunity to perform my thesis work at Ericsson Research's Visual Technology unit in Kista and I would like to thank all the people there that have assisted me throughout this work. I would like to especially thank my external super- visor at Ericsson, Martin Pettersson, for assisting me and providing me with valuable feedback throughout the whole process. I would also like to thank my supervisor at Lule˚aUniversity of Technology, Anders Landstr¨om,for showing an interest in the work and providing valuable guidance. Finally I would like to thank family and friends, both new and old, for all the support I have been given during the work and especially the move to a new town. ii Contents 1 Introduction 1 1.1 Background . .2 1.2 Purpose . .2 1.3 Delimitations . .3 1.4 Related Work . .3 1.5 Contribution . .4 2 Theory 6 2.1 High Dynamic Range . .6 2.1.1 Transfer Functions . .7 2.1.1.1 Philips TF . .8 2.1.1.2 PQ-TF . .9 2.2 Color Models . 10 2.2.1 RGB . 10 2.2.2 YCbCr . 11 2.3 Color Spaces . 12 2.3.1 CIE 1931 . 12 2.3.2 CIELAB . 12 2.3.3 Wide Color Gamut . 14 2.3.4 BT.709 . 14 2.3.5 BT.2020 . 15 2.3.6 DCI P3 . 16 2.4 Chroma Subsampling . 16 2.4.1 4:4:4 to 4:2:0 . 17 2.4.2 4:2:0 to 4:4:4 . 18 2.5 File Formats . 19 2.5.1 EXR . 19 2.5.2 TIFF . 20 2.6 Video Coding . 20 2.6.1 Encoder . 21 2.6.2 Decoder . 22 2.6.3 Rate-Distortion Optimization . 22 2.6.4 Video Coding Artifacts . 23 2.6.5 HEVC Standard . 25 2.6.5.1 Quantization Parameter . 27 2.6.5.2 Coding Tree Units . 27 2.6.5.3 Deblocking Filter . 28 iii Contents iv 2.6.5.4 Sample Adaptive Offset . 29 2.6.5.5 Profiles . 30 2.7 Quality Measurement . 31 2.7.1 PSNR . 31 2.7.2 tPSNR . 32 2.7.3 CIEDE2000 . 32 2.7.4 mPSNR . 35 2.7.5 Bjøntegaard-Delta Bit-Rate Measurements . 36 3 Method 38 3.1 Test Sequences . 38 3.2 Processing Chain . 40 3.2.1 Preprocessing . 41 3.2.2 Postprocessing . 41 3.2.3 Anchor Settings . 42 3.2.4 Conversion of TIFF Input Files . 42 3.3 Evaluation . 43 3.3.1 Objective Evaluation . 43 3.3.2 Subjective Evaluation . 44 3.4 HEVC Profile Tests . 44 3.5 Bitshifting at the CU Level . 45 3.5.1 Variation 1 . 47 3.5.2 Variation 2 . 47 3.6 Histogram Based Color Value Mapping . 48 3.6.1 Preprocessing . 50 3.6.2 Postprocessing . 52 3.6.3 Parameters . 52 3.7 SAO XYZ . 53 4 Results 57 4.1 HEVC Profile Tests . 57 4.1.1 Main-RExt, 12 bits, 4:2:0 . 57 4.1.2 Main-RExt, 10 bits, 4:4:4 . 59 4.1.3 Main-RExt, 12 bits, 4:4:4 . 59 4.1.4 Main-RExt, QP offsets . 60 4.2 Bitshifting at the CU Level . 60 4.2.1 Variation 1 . 61 4.2.2 Variation 2 . 61 4.3 Histogram Based Color Value Mapping . 62 4.3.1 Objective Results . 62 4.3.2 Subjective Results . 63 4.4 SAO XYZ . 65 5 Discussion 66 5.1 Reflections . 66 5.2 Conclusions . 67 5.3 Future Work . 67 Contents v 5.3.1 Bitshifting at the CU Level . 68 5.3.2 Histogram Based Color Value Mapping . 68 5.3.3 SAO XYZ . 69 Chapter 1 Introduction The TV industry is developing quickly. Until now the main interest has been in in- creasing the resolution of the video content. Ultra HDTV (High Definition Television) provides a number of improvements, including increased resolution, higher frame rate, and an improved color space. However, we are currently approaching the point where there is little benefit in increasing the resolution for ordinary TV sets. Therefore, there is a rising interest in other technologies which can be used to increase the visual expe- rience. One technology introduced is High Dynamic Range (HDR) video and content providers such as Amazon are already providing HDR content [1]. The goal of HDR video is to provide a greater range of luminosity to the end consumer. Today's television systems only provides Standard Dynamic Range (SDR), which is a dynamic range of about 1000:1 (ratio between brightest and darkest brightness), with a luminosity between 0:1 to 100 candela per square metre (cd=m2). As an example, the sky near the horizon during noon a clear day has a luminance level of approximately 10 000 cd=m2. HDR is defined as a dynamic range greater than 65 536:1. HDR video, however, may require changes throughout the video chain, affecting every- thing from video capturing to the TV sets that are meant to display the video. The content providers want to produce and distribute content that actually utilizes this new feature, but they also want to be able to distribute it as efficiently as possible. This may require new tools that are specific for compression of HDR video. This thesis will present three main ideas for improvements to existing tools which im- proves the coding of HDR video: • Bit depth shifting at the CU level, • Histogram based color value mapping, 1 Chapter 1. Introduction 2 • Sample adaptive offset in XYZ domain. These ideas will be described in detail and analyzed throughout the report. Chapter 1 will provide an introduction to the work, providing background, purpose and delimitations. Chapter 2 provides the theory necessary to get an understanding of the presented ideas. It will cover areas such as HDR, color theory, and video coding. Chapter 3 presents the three approaches, describing them in detail. This chapter also provides an overview of the method used for evaluation of the ideas. Chapter 4 will present the results for each of the ideas and also a brief analysis of the results. The last chapter, chapter 5, provides a discussion of the result, including a more detailed analysis of the results and overall conclusions. 1.1 Background HEVC (High Efficiency Video Coding) is a video compression standard [2] and version 3 was approved in April 2015 [3]. MPEG (Moving Picture Experts Group) issued a Call for Evidence (CfE) for HDR and WCG (Wide Color Gamut) video coding in the spring 2015. This process has a clear purpose: MPEG wants to explore if the coding efficiency and/or the functionality of the HEVC standard can be improved for HDR. The introduction of HDR video presents a number of challenges not previously consid- ered. Many of the methods used for compressing ordinary video may not work as well for HDR video, both in terms of quality and compression rate. There is an interest in how to efficiently represent the colors to fit the given number of bits per pixel, as is how to do the coding as efficiently as possible. The former is typically performed in the pre- and post-processing stages of the processing chain. Standard video is usually represented in the sRGB color space, which gives a more efficient use of the pixels compared to just storing them in a linear color space. However, the sRGB gamut, i.e. the complete subset of colors which can be represented within the.