This content has been downloaded from IOPscience. Please scroll down to see the full text.

Download details:

IP Address: 170.106.202.8 This content was downloaded on 25/09/2021 at 11:50

Please note that terms and conditions apply. Physics of Digital (Second Edition) IOP Series in Emerging Technologies in Optics and Photonics

Series Editor R Barry Johnson a Senior Research Professor at Alabama A&M University, has been involved for over 50 years in lens design, optical systems design, electro-optical systems engineering, and photonics. He has been a faculty member at three academic institutions engaged in optics education and research, employed by a number of companies, and provided consulting services.

Dr Johnson is an IOP Fellow, SPIE Fellow and Life Member, OSA Fellow, and was the 1987 President of SPIE. He serves on the editorial board of Infrared Physics & Technology and Advances in Optical Technologies. Dr Johnson has been awarded many patents, has published numerous papers and several books and book chapters, and was awarded the 2012 OSA/SPIE Joseph W Goodman Book Writing Award for Lens Design Fundamentals, Second Edition. He is a perennial co-chair of the annual SPIE Current Developments in Lens Design and Optical Engineering Conference.

Foreword Until the 1960s, the field of optics was primarily concentrated in the classical areas of photography, cameras, binoculars, telescopes, spectrometers, colorimeters, radio- meters, etc. In the late 1960s, optics began to blossom with the advent of new types of infrared detectors, liquid crystal displays (LCD), light emitting diodes (LED), charge coupled devices (CCD), lasers, holography, fiber optics, new optical materials, advances in optical and mechanical fabrication, new optical design programs, and many more technologies. With the development of the LED, LCD, CCD and other electo-optical devices, the term ‘photonics’ came into vogue in the 1980s to describe the science of using light in development of new technologies and the performance of a myriad of applications. Today, optics and photonics are truly pervasive throughout society and new technologies are continuing to emerge. The objective of this series is to provide students, researchers, and those who enjoy self-teaching with a wide- ranging collection of books that each focus on a relevant topic in technologies and application of optics and photonics. These books will provide knowledge to prepare the reader to be better able to participate in these exciting areas now and in the future. The title of this series is Emerging Technologies in Optics and Photonics where ‘emerging’ is taken to mean ‘coming into existence,’‘coming into maturity,’ and ‘coming into prominence.’ IOP Publishing and I hope that you find this Series of significant value to you and your career. Physics of (Second Edition)

D A Rowlands

IOP Publishing, Bristol, UK ª IOP Publishing Ltd 2020

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the publisher, or as expressly permitted by law or under terms agreed with the appropriate rights organization. Multiple copying is permitted in accordance with the terms of licences issued by the Copyright Licensing Agency, the Copyright Clearance Centre and other reproduction rights organizations.

Permission to make use of IOP Publishing content other than as set out above may be sought at [email protected].

D A Rowlands has asserted his right to be identified as the author of this work in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.

ISBN 978-0-7503-2558-5 (ebook) ISBN 978-0-7503-2559-2 (print) ISBN 978-0-7503-2560-8 (myPrint) ISBN 978-0-7503-2561-5 (mobi)

DOI 10.1088/978-0-7503-2558-5

Version: 20201001

IOP ebooks

British Library Cataloguing-in-Publication Data: A catalogue record for this book is available from the British Library.

Published by IOP Publishing, wholly owned by The Institute of Physics, London

IOP Publishing, Temple Circus, Temple Way, Bristol, BS1 6HG, UK

US Office: IOP Publishing, Inc., 190 North Independence Mall West, Suite 601, Philadelphia, PA 19106, USA For my parents, Ann and Gareth

Contents

Preface xv Author biography xvi Abbreviations xvii

1 Photographic optics 1-1 1.1 Optical image formation 1-2 1.1.1 Refraction 1-2 1.1.2 Lens design 1-3 1.1.3 Paraxial imaging 1-4 1.1.4 Gaussian optics 1-6 1.1.5 Compound lenses: ynu raytrace 1-7 1.1.6 Principal planes 1-9 1.1.7 Gaussian conjugate equation 1-11 1.1.8 Thick and thin lenses 1-12 1.1.9 Focal length 1-14 1.1.10 Magnification 1-18 1.1.11 Lens aberrations 1-19 1.2 Focusing 1-20 1.2.1 Unit focusing 1-21 1.2.2 Internal focusing 1-23 1.2.3 Single lens reflex cameras 1-24 1.2.4 Phase-detect autofocus 1-25 1.3 Framing 1-27 1.3.1 Entrance and exit pupils 1-27 1.3.2 Chief rays 1-28 1.3.3 Pupil magnification 1-30 1.3.4 Angular field of view formula 1-32 1.3.5 Focus breathing 1-35 1.3.6 Focal length multiplier 1-36 1.3.7 Perspective 1-37 1.3.8 Keystone distortion 1-39 1.4 Depth of field 1-40 1.4.1 1-40 1.4.2 Depth of field formulae 1-42

vii Physics of Digital Photography (Second Edition)

1.4.3 Depth of field control 1-44 1.4.4 Hyperfocal distance 1-44 1.4.5 Focus and recompose limits 1-46 1.4.6 Bokeh 1-48 1.5 Photometric 1-51 1.5.1 Photometry 1-51 1.5.2 Flux emitted into a cone 1-53 1.5.3 Relative 1-56 1.5.4 f-number 1-58 1.5.5 Working f-number 1-59 1.5.6 f-stop 1-61 1.5.7 Natural vignetting 1-62 1.5.8 Camera equation 1-64 1.5.9 Shutters 1-65 1.5.10 f-number for aplanatic lenses 1-68 References 1-71

2 Digital output and exposure strategy 2-1 2.1 Raw data 2-2 2.1.1 Sensor response 2-2 2.1.2 Colour 2-3 2.1.3 Dynamic range transfer 2-5 2.2 Digital output levels 2-6 2.2.1 Bit depth reduction 2-7 2.2.2 Posterisation 2-8 2.2.3 Lightness 2-9 2.2.4 Gamma encoding 2-9 2.2.5 Gamma decoding 2-11 2.3 Image dynamic range 2-13 2.3.1 Gamma curves 2-14 2.3.2 Tone curves 2-15 2.3.3 Raw headroom 2-17 2.3.4 Shadow and highlight dynamic range 2-17 2.4 Histograms 2-18 2.4.1 Luminance histograms 2-19 2.4.2 Image histograms 2-19

viii Physics of Digital Photography (Second Edition)

2.5 Average photometry 2-20 2.5.1 Reflected light meter equation 2-21 2.5.2 Proportionality constant 2-22 2.5.3 Photographic constant 2-23 2.5.4 Hand-held meter calibration constant 2-24 2.5.5 Average scene luminance 2-24 2.5.6 Exposure value 2-25 2.6 Exposure index 2-26 2.6.1 ISO speed 2-27 2.6.2 Standard output sensitivity 2-30 2.6.3 Recommended exposure index 2-32 2.6.4 Extended highlights 2-32 2.7 Advanced metering 2-34 2.7.1 Exposure compensation 2-35 2.7.2 In-camera metering modes 2-35 2.7.3 Incident light metering 2-35 2.8 Exposure modes 2-36 2.8.1 Aperture priority 2-36 2.8.2 Shutter priority 2-37 2.8.3 Program mode 2-37 2.8.4 Manual mode 2-38 2.9 Photographic lighting 2-38 2.9.1 Sunrise and sunset 2-40 2.10 Neutral density filters 2-41 2.10.1 Graduated neutral density filters 2-42 2.11 Polarizing filters 2-44 2.11.1 Malus’ law 2-45 2.11.2 Surface reflections 2-46 2.11.3 Blue skies 2-47 2.11.4 Circular polarizing filters 2-48 2.12 High dynamic range 2-49 2.12.1 High dynamic range imaging 2-50 2.12.2 Tone mapping 2-52 2.13 Image display 2-54 2.13.1 Luma 2-54 2.13.2 Display luminance 2-55 2.13.3 Display dynamic range 2-56 References 2-57

ix Physics of Digital Photography (Second Edition)

3 Raw data model 3-1 3.1 Linear systems theory 3-2 3.1.1 Radiometry 3-3 3.1.2 Ideal optical image 3-5 3.1.3 Point spread function (PSF) 3-5 3.1.4 Linear shift invariance 3-6 3.1.5 Convolution: derivation 3-7 3.1.6 Convolution: examples 3-9 3.1.7 Optical transfer function 3-11 3.1.8 Modulation transfer function (MTF) 3-13 3.1.9 Phase transfer function 3-15 3.1.10 Model camera system 3-15 3.2 Optics 3-17 3.2.1 Wave optics 3-17 3.2.2 Huygens–Fresnel principle 3-20 3.2.3 Aperture diffraction PSF 3-21 3.2.4 Circular aperture: airy disk 3-27 3.2.5 Aperture diffraction MTF 3-29 3.2.6 Aberrations: wavefront error 3-31 3.3 Sensor 3-33 3.3.1 Spatial averaging 3-34 3.3.2 Detector-aperture PSF 3-35 3.3.3 Sampling 3-36 3.3.4 Detector-aperture MTF 3-37 3.4 Optical low-pass filter 3-39 3.4.1 Function sampling 3-39 3.4.2 Replicated spectra 3-40 3.4.3 Reconstruction 3-41 3.4.4 Aliasing 3-42 3.4.5 Sensor Nyquist frequency 3-43 3.4.6 Pre-filtering 3-44 3.4.7 Four-spot filter PSF 3-45 3.4.8 Four-spot filter MTF 3-46 3.5 Sampled convolved image 3-47 3.5.1 Model camera system PSF 3-48 3.5.2 Model camera system MTF 3-49

x Physics of Digital Photography (Second Edition)

3.6 Charge signal 3-50 3.6.1 Sampled spectral exposure 3-50 3.6.2 Photoelements 3-51 3.6.3 Colour filter array 3-52 3.6.4 Camera response functions 3-53 3.6.5 Polychromatic PSF and MTF 3-56 3.6.6 Charge detection 3-57 3.7 Analog-to-digital conversion 3-58 3.7.1 Programmable ISO gain 3-58 3.7.2 Digital numbers 3-59 3.7.3 Conversion factor 3-60 3.7.4 Bias offset 3-61 3.8 Noise 3-61 3.8.1 Photon shot noise 3-62 3.8.2 Read noise 3-62 3.8.3 Dark current shot noise 3-63 3.8.4 Noise power 3-64 3.8.5 Fixed pattern noise 3-64 3.9 Noise measurement 3-65 3.9.1 Conversion factor measurement 3-65 3.9.2 Read noise measurement 3-67 3.9.3 Noise models 3-68 References 3-71

4 Raw conversion 4-1 4.1 Reference colour spaces 4-2 4.1.1 Theory of colour 4-3 4.1.2 Eye cone response functions 4-4 4.1.3 Colour-matching functions 4-5 4.1.4 Units 4-7 4.1.5 Standard luminosity function 4-8 4.1.6 CIE RGB colour space 4-9 4.1.7 rg chromaticity diagram 4-10 4.1.8 CIE XYZ colour space 4-11 4.1.9 xy chromaticity diagram 4-14 4.1.10 Absolute colourimetry 4-16 4.1.11 Relative colourimetry 4-16 4.1.12 Reference white 4-17

xi Physics of Digital Photography (Second Edition)

4.2 Illumination 4-18 4.2.1 Colour temperature 4-18 4.2.2 Correlated colour temperature 4-18 4.2.3 White point 4-20 4.2.4 Standard illuminants 4-20 4.3 Camera raw space 4-21 4.3.1 Raw channels 4-21 4.3.2 Colour demosaicing 4-22 4.3.3 Raw pixel vectors 4-24 4.3.4 Camera raw space primaries 4-24 4.3.5 Camera raw space reference white 4-25 4.4 Camera colour characterisation 4-25 4.4.1 Luther–Ives condition 4-26 4.4.2 Raw to CIE XYZ 4-26 4.4.3 Colour difference: CIE LAB 4-29 4.4.4 Transformation matrix normalisation 4-29 4.5 Output-referred colour spaces 4-30 4.5.1 sRGB colour space: linear form 4-32 4.5.2 CIE XYZ D65 to sRGB D65 4-33 4.5.3 Raw D65 to sRGB D65 4-34 4.6 White balance 4-35 4.6.1 Adopted white 4-37 4.6.2 Chromatic adaptation transforms 4-38 4.6.3 Raw channel multipliers 4-41 4.7 Strategy 1: transformation matrices + CAT 4-42 4.8 Strategy 2: raw channel multipliers + rotation matrix 4-43 4.8.1 Traditional digital cameras 4-44 4.8.2 dcraw 4-46 4.9 Adobe DNG 4-49 4.9.1 Method 1: transformation matrix + CAT 4-49 4.9.2 Method 2: raw channel multipliers + forward matrix 4-52 4.10 sRGB colour space: nonlinear form 4-53 4.10.1 sRGB digital output levels 4-53 4.10.2 sRGB colour cube 4-55 4.11 Raw processing workflow 4-56 4.11.1 Colour management 4-56 4.11.2 Maximal colour strategy 4-57

xii Physics of Digital Photography (Second Edition)

4.11.3 16-bit TIFF files 4-58 4.11.4 Adobe Photoshop colour settings 4-59 4.11.5 Image resizing 4-63 References 4-65

5 Camera image quality 5-1 5.1 Cross-format comparisons 5-2 5.1.1 Equivalence and image quality 5-4 5.1.2 Generalised equivalence theory 5-6 5.1.3 Proof of equivalence theory 5-9 5.2 Perceived resolution 5-20 5.2.1 Observer resolving power 5-20 5.2.2 Standard viewing conditions 5-21 5.2.3 Circle of confusion: standard value 5-22 5.2.4 Circle of confusion: custom value 5-23 5.2.5 Circle of confusion: derivation 5-23 5.2.6 Depth of focus 5-25 5.3 Lens MTF 5-25 5.3.1 Lens MTF: standard viewing conditions 5-27 5.3.2 Lens MTF: lens resolving power 5-30 5.3.3 Cross-format comparisons 5-33 5.3.4 Limitations of lens MTF 5-34 5.4 Camera system MTF 5-35 5.4.1 Cross-format comparisons 5-35 5.5 Camera system resolving power 5-36 5.5.1 Model camera system 5-36 5.6 Perceived image sharpness 5-39 5.6.1 MTF50 5-40 5.6.2 Example: pixel count 5-40 5.6.3 Subjective quality factor 5-41 5.7 Image resampling 5-44 5.7.1 Upsampling 5-44 5.7.2 Downsampling 5-48 5.8 Signal-to-noise ratio (SNR) 5-49 5.8.1 SNR and ISO setting 5-50 5.8.2 SNR: output-referred units 5-50 5.8.3 SNR: input-referred units 5-52

xiii Physics of Digital Photography (Second Edition)

5.8.4 ISO invariance 5-53 5.8.5 SNR and pixel count 5-54 5.8.6 SNR per unit area 5-55 5.8.7 SNR: cross-format comparisons 5-56 5.8.8 Raw ISO values 5-57 5.9 Raw dynamic range 5-58 5.9.1 Raw dynamic range per photosite 5-58 5.9.2 Sensor dynamic range 5-59 5.9.3 Perceivable dynamic range 5-60 5.10 Practical strategies 5-62 5.10.1 Object resolution 5-63 5.10.2 Diffraction softening 5-65 5.10.3 Non-destructive noise reduction 5-67 5.10.4 Exposing to the right (ETTR) 5-69 5.10.5 ETTR: variable exposure 5-69 5.10.6 ETTR: fixed exposure 5-70 References 5-71

Index 6-1

xiv Preface

The aim of this book is to provide a theoretical overview of the photographic imaging chain. It is intended for use by both graduate students and established researchers as a link between imaging science and photographic practice. It should also be useful for photographers who have a graduate-level technical background. Chapter 1 titled ‘Photographic Optics’ describes the formation of an optical image by a compound photographic lens. Topics discussed include focusing, framing, perspective and depth of field. The final section derives the photometric exposure distribution formed at the sensor plane of the camera. Chapter 2 titled ‘Digital Output and Exposure Strategy’ discusses the strategy for generating useful digital output in response to the photometric exposure distribution at the sensor plane derived in chapter 1. In modern digital photography, the standard exposure strategy defined by the CIPA DC-004 and ISO 12232 standards is based upon the output JPEG image obtained from the camera. Consequently, this chapter begins by discussing the nature of the digital output levels of a digital image. The theory of the standard exposure strategy is subsequently developed in detail. For a typical scene, the aim is to produce a JPEG image that has the correct mid- tone lightness. The later sections discuss practical exposure strategy and cover topics such as photographic lighting, exposure modes and advanced metering, photo- graphic filters and high dynamic range imaging. Chapter 3 titled ‘Raw Data Model’ uses linear systems theory to develop a model of the raw data produced by a camera. The aim is to illustrate how the nature and quality of the raw data is affected by phenomena such as aliasing and noise, along with various blurring phenomena such as diffraction. A derivation of the charge signal is included, along with a model of the analog-to-digital conversion process. Chapter 4 titled ‘Raw Conversion’ describes the main steps involved in converting the raw data into a viewable output colour image. The majority of the chapter is devoted to colour conversion of the raw data. Unlike conventional academic treatments, this chapter describes colour conversion and white-balancing strategies that are used by digital cameras in practice. Chapter 5 titled ‘Camera Image Quality’ discusses the theory behind camera and lens image quality metrics. Unlike conventional treatments, this chapter demon- strates how such metrics should be applied and interpreted when comparing camera systems that have different sensor pixel counts or are based on different sensor formats. Practical strategies for maximising the full image quality potential of a camera system are also discussed. In this second edition, the chapter structure of the first edition has been preserved but the material has been reorganised and extensively rewritten. New material has been added, typographical errors have been corrected, the figures have been improved and a detailed index has been included. I would like to thank Prof. R Barry Johnson for useful discussions. Andy Rowlands, June 2020

xv Author biography

D A Rowlands

Andy Rowlands gained a first-class degree in Mathematics and Physics and a PhD in Theoretical Condensed Matter Physics from the University of Warwick, UK. He was subsequently awarded a Fellowship in Theoretical Physics from the Engineering and Physical Sciences Research Council (EPSRC), which he held at the University of Bristol, UK, and this was followed by research positions at Lawrence Livermore National Laboratory, USA, Tongji University in Shanghai, China, and the University of Cambridge, UK. Andyʼs combined interests in physics and photography inspired the writing of this book. His photographic work, much of which features China, can be viewed at http://www.andyrowlands.com.

xvi Abbreviations

1D one dimension 2D two dimensions ADC analog-to-digital converter ADU analog-to-digital unit AF autofocus AFoV angular field of view AHD Adaptive Homogeneity-Directed APEX Additive System of Photographic Exposure APS-C Advanced Photo System Type-C AS aperture stop ATF aberration transfer function Av aperture value AW adopted white BSI backside illumination Bv brightness value CAT chromatic adaptation transform CCD charge-coupled device CCE charge collection efficiency CCT correlated colour temperature CDS correlated double sampling CFA colour filter array CIE Commission Internationale de l’Eclairage CIPA Camera and Imaging Products Association CMM colour-matching module CMOS complimentary metal-oxide semiconductor CoC circle of confusion CRT cathode ray tube CSF contrast sensitivity function CTF contrast transfer function DCNU dark current non-uniformity DN data number DNG digital negative DoF depth of field DOL digital output level DR dynamic range DSC/SMI digital still camera sensitivity metameric index D-SLR digital single-lens reflex DSNU dark signal non-uniformity EC exposure compensation EP entrance pupil ESF edge spread function ETTR expose to the right Ev exposure value EW entrance window EXIF Exchangeable Image File FF fill factor FFP front focal plane

xvii Physics of Digital Photography (Second Edition)

FoV field of view FP focal plane FPN fixed pattern noise FS field stop FT Fourier transform FWC full-well capacity GND graduated neutral density HDR high dynamic range HVS human visual system ICC International Color Consortium IP image plane IQ image quality ISO International Organization for Standardization Iv incident light value JPEG Joint Photographic Experts Group LCD liquid crystal display LDR low dynamic range LENR long-exposure noise reduction LRS least resolvable separation LSI linear shift-invariant LUT look-up table MOS metal oxide semiconductor MTF modulation transfer function NA numerical aperture ND neutral density OA optical axis OECF opto-electronic conversion function OLPF optical low-pass filter OP object plane OPD optical path difference OTF optical transfer function PCS profile connection space PDAF phase-detect autofocus PDR photographic dynamic range PGA programmable gain amplifier PPG Patterned Pixel Grouping PRNU pixel response non-uniformity PSF point spread function PTF phase transfer function REI recommended exposure index RMS root mean square QE quantum efficiency RA relative aperture REI recommended exposure index RFP rear focal plane RI relative illumination RP resolving power SA spherical aberration SQF subjective quality factor SLR single-lens reflex

xviii Physics of Digital Photography (Second Edition)

SNR signal-to-noise ratio SOS standard output sensitivity SP sensor plane SPD spectral power distribution SQF subjective quality factor Sv speed value TIFF Tagged Image File Format TMO tone-mapping operator TTL through-the-lens Tv time value UCS uniform chromaticity scale USM unsharp mask VNG Variable Number of Gradients WB white balance XP exit pupil XW exit window

xix