VELI-TAPANI PELTOKETO Benchmarking of Mobile Phone Cameras ACTA WASAENSIA 352 COMPUTER SCIENCE 16 TELECOMMUNICATION ENGINEERING Reviewers Professor Farag Sallabi University of United Arab Emirates Department of Information Technology P.O. Box 15551 Al Ain, UAE Vice President, Dr. Lasse Eriksson Cargotec Oyj P.O. Box 61 FI-00501 Helsinki, Finland III Julkaisija Julkaisupäivämäärä Vaasan yliopisto Elokuu 2016 Tekijä(t) Julkaisun tyyppi Veli-Tapani Peltoketo Artikkeliväitöskirja Julkaisusarjan nimi, osan numero Acta Wasaensia, 352 Yhteystiedot ISBN Vaasan yliopisto 978-952-476-684-5 (painettu) Teknillinen tiedekunta 978-952-476-685-2 (verkkoaineisto) Tietotekniikan yksikkö ISSN PL 700 0355-2667 (Acta Wasaensia 352, painettu) 65101 VAASA 2323-9123 (Acta Wasaensia 352, verkkoaineisto) 1455-7339 (Acta Wasaensia. Tietotekniikka 16, painettu) 2342-0693 (Acta Wasaensia. Tietotekniikka 16, verkkoaineisto) Sivumäärä Kieli 168 Englanti Julkaisun nimike Matkapuhelinkameroiden suorituskykymittaus ja vertailu Tiivistelmä Matkapuhelinten käytön kasvu on korostanut matkapuhelimien kameroiden kuvalaadun merkitystä. Viime vuosina on julkaistu useita uusia kuvalaatuun liittyviä standardeja ja vanhoja standardeja päivitetään jatkuvasti. Standardit määrittelevät kuvalaadun mittaukset kuitenkin vain tietyille ominaisuuksille ja yleisiä mittareita koko kameran laadulle ei ole olemassa. Tässä väitöskirjassa tutkitaan ja määrittellään vaatimuksia ja haasteita, jotka liittyvät matkapuhelin- kameroiden suorituskykymittauksiin ja vertailuun. Väitöskirjassa luodaan laaja katsaus digitaalisen kuvauksen laatuun ja tekijöihin, jotka heikentävät kuvan laatua. Samalla työssä paneudutaan virheisiin, joita tavataan digitaalisessa kuvauksessa. Väitöskirja käsittelee myös kameroissa esiintyviä viiveitä ja kameroiden nopeutta. Lisäksi kameroiden laadun ja suorituskyvyn mittauksessa käytettäviä mittareita ja metodeita esitellään kattavasti. Työ sisältää viisi aiemmin julkaistua artikkelia, jotka yhdessä väitöskirjan muun tekstin kanssa muodostavat tutkimuskokonaisuuden. Väitöskirja sisältää pohdintaa erilaisista kuvalaadun ja suorituskyvyn mittareista sekä miten niitä voidaan käyttää kameroiden vertailussa. Työ esittelee myös haasteet, jotka liittyvät useiden ominaisuuksien yhdistämiseen yhdeksi suoritus- kykyä mittaavaksi arvoksi kameroiden vertailun helpottamiseksi. Lisäksi väitös- kirjassa käsitellään erilaisia ympäristömuuttujia ja niiden vaikutusta suoritus- kykymittaukseen. Tutkimuksen tuloksena väitöskirja esittelee uuden suorituskykymittausmetodin matkapuhelimien kameroille. Erittäin tärkeä osa työn tulosta on myös suoritus- kykymittaukseen liittyvien vaatimusten ja haasteiden määritys. Asiasanat Suorituskykymittaus, matkapuhelinkamera, kuvanlaatu. V Publisher Date of publication University of Vaasa August 2016 Author(s) Type of publication Veli-Tapani Peltoketo Selection of articles Name and number of series Acta Wasaensia, 352 Contact information ISBN University of Vaasa 978-952-476-684-5 (print) Faculty of Technology 978-952-476-685-2 (online) Department of Computer ISSN Science 0355-2667 (Acta Wasaensia 352, print) P.O. Box 700 2323-9123 (Acta Wasaensia 352, online) FI–65101 VAASA 1455-7339 (Acta Wasaensia. Computer FINLAND Science 16, print) 2342-0693 (Acta Wasaensia. Computer Science 16, online) Number Language of pages 168 English Title of publication Benchmarking of Mobile Phone Cameras Abstract Great success of the mobile phone industry has highlighted image quality of mobile phone cameras. Several new standards are published during recent years and old standards are continuously updated. Nevertheless, each standard measures and validates a certain image quality feature or artefact and there is a lack of generic metrics to validate a whole camera system. This thesis investigates and defines the requirements and challenges of benchmarking when mobile phone cameras are compared. This thesis contains a comprehensive introduction of image quality factors, image quality distortions and artefacts, which are common in digital imaging. In addition, performance issues like delays and slowness of camera systems are investigated. Correspondingly, quality metrics and methods, which are used to validate image quality and performance of digital cameras are described. This work includes five previously published articles, which are an essential part of the thesis work. This thesis includes considerations of different image quality and performance metrics and how they should be used in benchmarking. The challenges of a single number benchmarking score is discussed. The environmental factors, like lightness are evaluated, too and their influence to benchmarking is discussed. The outcome of the research is a novel benchmarking method for mobile phone cameras which includes both quality and performance metrics. Even more important this thesis highlights the requirements and challenges of a mobile phone camera benchmarking. Keywords Benchmarking, mobile phone cameras, image quality, performance. VII PREFACE It is contradictory to look back the time period which I have spent in University of Vaasa. Contradictory, because the time from the autumn 2009 is at the same time very short but contains so many exciting and rewarding moments. Moments, when I have been tested myself and qualified, how far I can force and motivate myself. In 2009 I started to study again, after 16 years in the working life. From the very first steps, the spirit and extremely beautiful environment of Vaasa University fascinated and convinced me that I am the right place to learn more and try something new. When I graduated as a master of science in 2011, it was quite obvious to ask a permission to continue my studies as a postgraduate student. However, a university, how beautiful it could be, is just a bunch of empty buildings without inspiring people. I would like to thank all the personnel and student colleagues of Vaasa University because they made the University of Vaasa a good place to be and study. Particularly, I am very grateful to Professor Mohammed Elmusrati, who has supported, advised, and guided me during the doctoral work. I have had a privilege to work and study at the same time. The twofold role has given me perspective to focus to the essential areas in the studies and research. At the same time, my role in the working life has enabled to use learned things and research results at once in real products and vice versa, validate the research methods using real data got from the product development. Therefore I have a great pleasure to thank the whole personnel of Sofica Ltd. The atmosphere of the company encouraged me to target higher than I would have never expected. Especially I would like to thank Mr. Marko Nurro, who partially forced but mostly motivated me to face new challenges. Without the exciting and rewarding moments in demanding customer meetings and negotiations, I would not have been ready to present my research and papers in several conferences. I would also thank my colleagues in Nokia Technology. I have worked in Nokia Technology and particularly in Digital Media group, when I have been writing my doctoral thesis. Again, I have learned new facts of image quality and new approaches to tackle challenging problems. I am also grateful to Mr. Graham Soundy, who has shown extremely patience to review and correct my English, which is – I afraid – far from the beautiful British English. VIII Every manuscript requires a proper review. Professor Farag Sallabi and Doctor Lasse Ericsson made a valuable work to review and comment my thesis. Especially I want to thank Doctor Lasse Ericsson, whose review was extremely comprehensive and denoted a great commitment to pre-examination work as well as great understanding of the research area. Finally, it is difficult to highlight enough the importance of the most loved ones. The pages of this thesis would not be enough to express my gratefulness and love which I feel to my wife Arja and my sons Matias, Tuomas and Mikael. All in all, the time from year 2009 has been very fun, which is the main motivator to make something new. Nurmo, Finland, 1st June, 2016 Veli-Tapani Peltoketo IX Contents PREFACE ........................................................................................................... VII 1 INTRODUCTION ........................................................................................... 1 1.1 Background and motivation .................................................................. 1 1.2 Objectives and contributions ................................................................ 2 1.3 Methods ................................................................................................ 3 1.4 Structure of thesis ................................................................................. 4 2 PRINCIPLES OF MODERN MOBILE PHONE CAMERA .......................... 5 2.1 Glance at history of photography .......................................................... 5 2.1.1 Film era ................................................................................ 5 2.1.2 Digitalization ........................................................................ 6 2.1.3 From CCD to CMOS ........................................................... 7 2.1.4 Mobile phone cameras ......................................................... 8 2.2 Generic structure of mobile phone camera ........................................... 9 2.2.1 Image sensor ...................................................................... 10 2.2.2