“PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page i — #1 Perceptually-Inspired Gamut Mapping for Display and Projection Technologies Syed Waqas Zamir DOCTORAL THESIS UPF / YEAR 2017 Thesis Advisors: Marcelo Bertalm´ıo and Javier Vazquez-Corral Department of Information and Communication Technologies “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page ii — #2 “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page iii — #3 To my parents, grandmother, and Khazina iii “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page iv — #4 “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page v — #5 Acknowledgments First and foremost, I would like to express my utmost gratitude to my super- visors Marcelo Bertalm´ıo and Javier Vazquez´ Corral for their encouragement, continuous support and guidance throughout all these long years of research. Without their invaluable experience and mentoring this work would not have been possible. I would like to thank all the participants of the psychophysical experiments. Special thanks to Stephan Cattan, from Deluxe (Spain), for providing us with test images and for all his help and suggestions. I would like to express my gratitude to Barco N.V. for their invaluable help, without which this work would not have been possible. Many thanks and cheers go to my dear wife Khazina for her endless love, encouragement and unconditional support even when she had to endure practical and emotional discomfort several times during my PhD. I am very grateful to Raquel Gil for being an awesome friend and always willing to help whether it was at work, or fighting against Bureaucracy in public offices. For all the favors I am greatly in your debt — forever. Thanks to my very kind (and annoyingly polite) friend and colleague Gabriela Ghimpeteanu for always hearing me out patiently. A very special thanks to my very special friend Praveen Cyriac for invaluable discussions, help and support throughout my PhD. I am also very thankful to my many other friends and colleagues: Zulqarnain Rashid, Thomas Batard, David Kane, Jihyun Kim, Antoine Grimaldi, Adnan Butt, Umar Iqbal, Hashim Shahid, Sadegh Mohammadi, Kamruddin Nur, Raul´ Parada, Aamir Saleem, Yasuko Sugito and Jose Maria Rojano. Very special thanks to the most amazing and friendly secretaries: Jana Sa- frankova, Lydia Garcia, Joana Soria, Joana Clotet, Venessa Jimenez, Magda Castellnou, Ruth Temporal, Montserrat Serrano, and Lluis Bosch. Words cannot express how thankful I am to my parents for all their uncondi- tional support, love, help, encouragement and hard work in getting me where I am today. Thanks to my other family members: my brothers Sohail and Haroon, my sister Sahar, my grandparents especially my late grandmother Sughra Bibi (whom I miss dearly). v “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page vi — #6 “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page vii — #7 Abstract The cinema and television industries are continuously working in the development of image features that can provide a better visual experience to viewers; these image attributes include large spatial resolution, high temporal resolution (frame rate), greater contrast, and recently, with emerging display technologies, much wider color gamut. The gamut of a device is the set of colors that this device is capable of reproducing. Gamut Mapping Algorithms (GMAs) transform colors of the original content to the color palette of the display device with the simultaneous goals of (a) reproducing content accurately while preserving the artistic intent of the original content’s creator and (b) exploiting the full color rendering potential of the target display device. There are two types of gamut mapping: Gamut Reduction (GR) and Gamut Extension (GE). GR involves the transformation of colors from a larger source gamut to a smaller destination gamut. Whereas in GE, colors are mapped from a smaller source gamut to a larger destination gamut. In this thesis we propose three spatial Gamut Reduction Algorithms (GRAs) and four spatial Gamut Extension Algorithms (GEAs). These methods comply with some basic global and local perceptual properties of human vision, producing state of the art results that appear natural and are perceptually faithful to the original material. Moreover, we present a psychophysical evaluation of GEAs specifically for cinema using a digital cinema projector under cinematic (low ambient light) conditions; to the best of our knowledge this is the first evaluation of this kind reported in the literature. We also show how currently available image quality metrics, when applied to the gamut extension problem, provide results that do not correlate well with users’ choices. vii “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page viii — #8 Resum Les industries´ de cinema i televisio´ estan treballant cont´ınuament en el desenvolu- pament de diferents caracter´ıstiques de la imatge que puguin proporcionar una millor experiencia` visual per als espectadors; aquests atributs d’imatge inclouen la resolucio´ espacial, la resolucio´ temporal (fotogrames per segon), major contrast i, recentment, amb les noves tecnologies de visualitzacio´ emergents, una gamma de colors (gamut) molt mes´ ampli. El gamut d’un dispositiu es´ el conjunt de colors que aquest dispositiu es´ capa de reproduir. Els algoritmes de modificacio´ de gamut (GMA, de les seves sigles en angles)´ transformen els colors del contingut original a la paleta de color del dispositiu de visualitzacio´ amb els objectius de (a) reproduir el contingut amb precisio´ preservant al mateix temps la intencio´ art´ıstica del creador del contingut original i (b) utilitzar tot el gamut de color del dispositiu de visualitzacio.´ Hi ha dos tipus d’algoritmes de modificacio´ de gamut: Reduccio´ de Gamut (GR) i Extensio´ de Gamut (GE). GR implica la transformacio´ dels colors d’un gamut d’origen mes´ gran a un gamut de destinacio´ mes´ petit. Mentre que a GE, els colors s’assignen d’un gamut d’origen petit a un gamut de destinacio´ mes´ gran. En aquesta tesi es proposen tres algoritmes de Reduccio´ de Gamut (GRAs) i quatre algoritmes d’extensio´ de Gamut (GEAs). Aquests metodes` compleixen amb algunes propietats perceptives globals i locals basiques` de la visio´ humana, produint resultats que son´ estat de l’art, i que son´ naturals i perceptualment fidels al material original. D’altra banda, es presenta una avaluacio´ psicof´ısica del problema d’extensio´ de Gamut espec´ıficament dissenyada per a cinema utilitzant un projector de cinema digital en condicions cinematiques` (baixa llum ambiental); aquest estudi creiem que es´ el primer del seu tipus a la literatura. Tambe´ mostrem com les metriques` de qualitat d’imatge disponibles actualment proporcionen resultats que no es correlacionen be´ amb l’eleccio´ dels usuaris quan s’apliquen al problema d’Extensio´ de Gamut. viii “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page ix — #9 Contents Nomenclature xv List of Figures xxiv List of Tables xxvi 1 INTRODUCTION 1 1.1 Main Contributions . 5 1.1.1 Publications . 8 1.2 Thesis Outline . 9 2 FUNDAMENTALS OF VISION AND BASIC COLOR SCIENCE 11 2.1 Light . 11 2.2 Human Eye Anatomy, Biology, and Vision Science . 13 2.2.1 The Eye Optics . 13 2.2.2 The Retina . 14 2.2.3 Thalamus and Visual Cortex . 20 2.3 Characteristics of Human Vision . 20 2.3.1 Effects of Ambient Illumination on Perception . 20 Visual Adaptation . 20 Hunt Effect and Stevens Effect . 23 Bezold-Brucke¨ (B-B) Effect . 23 ix “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page x — #10 2.3.2 Local Behavior of Visual System . 24 Simultaneous Contrast . 24 Mach Bands . 24 Chromatic Induction . 25 Helmholtz-Kohlrausch Effect . 26 2.3.3 Human Color Constancy . 26 Retinex Theory of Color Vision . 27 2.4 Colorimetry . 30 2.4.1 Color Matching Experiment . 30 2.4.2 The First Standard Color Spaces . 32 CIE xyY Color Space and CIE Chromaticity Diagram . 33 2.4.3 Perceptually-uniform Color Spaces . 35 Limitations of CIELAB, and CIELUV . 38 2.4.4 HSV Color Space . 38 2.4.5 Color Temperature and Different Reference White Points 38 3 CAMERAS, DISPLAYS, AND COLOR GAMUTS: THE NEED FOR GAMUT MAPPING 41 3.1 Color Image Processing Pipeline of a Digital Camera . 41 3.1.1 Lens, Aperture, Shutter, and Sensor . 42 3.1.2 Preprocessing . 44 3.1.3 White Balance . 45 3.1.4 Demosaicking . 45 3.1.5 In-camera Color Transformations . 46 3.1.6 Gamma Correction . 47 3.1.7 Post Processing . 49 3.1.8 Compression . 50 3.2 Color Gamuts . 50 3.2.1 Pointer’s Gamut . 52 3.2.2 Broadcast Standard: BT.709 . 53 x “PhD˙Thesis˙RevM2” — 2017/4/4 — 16:20 — page xi — #11 3.2.3 sRGB system . 56 3.2.4 Cinema Standard: DCI-P3 . 57 3.2.5 Next-Generation Broadcast Standards: BT.2020 and BT.2100 59 3.3 Significant Gamut Variation Among Displays . 60 3.3.1 What Makes the Gamut of a Display Wide? . 60 3.3.2 Cathode Ray Tube (CRT) . 61 3.3.3 Liquid Crystal Display (LCD) . 63 3.3.4 Organic Light Emitting Diode (OLED) . 65 3.3.5 Quantum Dots Technology . 65 3.4 Digital Projectors . 67 3.4.1 Digital Light Processing (DLP) . 68 Color in DLP Systems . 69 3.4.2 Light Sources in Projectors . 70 Lamps . 70 Laser RGB . 70 Laser Phosphor . 71 Advantages of Laser projectors over Lamp Projectors . 71 3.5 Color Transforms . 72 3.5.1 Transformations between RGB and CIE XYZ . 72 3.5.2 Transformations among RGB Systems . 73 Chromatic Adaptation . 74 3.6 The Need for Gamut Mapping . 75 4 LITERATURE SURVEY 77 4.1 Gamut Reduction Algorithms (GRAs) . 77 4.1.1 Global GRAs . 77 4.1.2 Local GRAs . 80 4.2 Gamut Extension Algorithms (GEAs) .