High-Level Numerical Simulations of Noise in CCD and CMOS
Total Page:16
File Type:pdf, Size:1020Kb
High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial Mikhail Konnik∗ and James Welsh Faculty of Engineering and Built Environment, University of Newcastle, Australia Abstract The photosensor models differ by the generality and the scope of the noise sources being included. High- In many applications, such as development and test- level simulators [5] are used for the evaluation of how ing of image processing algorithms, it is often neces- different noise sources influence image quality [7], while sary to simulate images containing realistic noise from low-level simulator models specific aspects of the solid- solid-state photosensors. A high-level model of CCD sate physics of a sensor [8]. The thoroughness of noise and CMOS photosensors based on a literature review modelling differs as well; for example, in some models is formulated in this paper. The model includes photo- of the photosensor, the dark current shot noise is de- response non-uniformity, photon shot noise, dark cur- scribed by its mean value [9, 10], while other models use rent Fixed Pattern Noise, dark current shot noise, off- more sophisticated statistical description of noise [11]. set Fixed Pattern Noise, source follower noise, sense Sophisticated software integrated suits such as the Im- node reset noise, and quantisation noise. The model age Systems Evaluation Toolkit [12, 7, 13] (ISET) were also includes voltage-to-voltage, voltage-to-electrons, developed recently. and analogue-to-digital converter non-linearities. The formulated model can be used to create synthetic im- ages for testing and validation of image processing al- Models of CMOS photosensors: Both high- gorithms in the presence of realistic images noise. An level [7, 2, 3, 5, 14, 15] and low-level [16, 17] numerical example of the simulated CMOS photosensor and a models of CMOS photosensors are reported in the liter- comparison with a custom-made CMOS hardware sen- ature. Low-level simulators like Medici [18], which are sor is presented. Procedures for characterisation from able to model the 2-D distributions of potential and car- both light and dark noises are described. Experimen- rier concentrations, are used for simulation of solid-state tal results that confirm the validity of the numerical physics in CMOS photosensors [19]. Some of the papers model are provided. The paper addresses the issue of deal with the numerical modelling of colour images that the lack of comprehensive high-level photosensor mod- allow end users to see the effects of the noise on the im- els that enable engineers to simulate realistic effects of ages [12]. Other papers focus on the specific types of noise on the images obtained from solid-state photosen- noise; for example, the analysis of temporal noise in sors. CMOS active pixels sensors can be found in [20], an autoregressive model of fixed-pattern noise (FPN) was presented in [21] along with other FPN models [22, 23]. 1 Introduction High-level models (e.g., [24, 25]) often assume that the ADC, sense node and source follower are completely lin- Many modern measuring devices use CMOS or CCD ear, which, as we will show in this paper, is not always solid-state photosensors to convert light into a digital the case. signal. Due to imperfections of photosensors such a con- version is not ideal and leads to noise in the measured arXiv:1412.4031v1 [astro-ph.IM] 11 Dec 2014 Models of CCD photosensors: The models of signal. Therefore, one can either estimate and reduce CCD photosensors [25, 26] and their noise parame- the impact of noise from the sensor, or simulate and ters estimation [26] are generally simpler than mod- predict the impact of noise on the images quality. The els of CMOS sensors. The model by Flory [27] de- problem of noise estimation has been addressed by the scribes the noise of the sensor electronics as a combi- EMVA1288 standard [1]. The high-level simulation of nation of shot noise and amplifiers noise. The CCD noise in photosensors, however, is still an area of ac- camera noise model by Cox [4] includes photon and tive research. The main problem is that photosensors electronic shot noise, dark-current noise and readout are affected by many different sources of noise, some of noise; however the model suggests that noise is station- which cannot be modelled adequately using only Gaus- ary. Healey and Kondepudy [28] model camera noise sian noise. with offset fixed-pattern noise (FPN), photon and dark- Different numerical models of CMOS [2, 3] and current shot noise, read noise and photo response non- CCD [4, 5, 6] sensors are reported in the literature. uniformity (PRNU). An adequate model was proposed ∗Corresponding author. Emails: [email protected] by Costantini and S¨usstrunk [5], where noises was cat- or [email protected] egorised in four main types: photon shot noise, Photo High-level noise simulations in CCD/CMOS page 1 of 21. M. Konnik and J.Welsh Response Non Uniformity, dark current shot noise and – Correlated double sampling ..................5 read noise. 2.3. From Voltage to Digital Numbers (including Dif- ferential and Integral Linearity Errors) .........5 The purpose of the article: This article provides a literature review of the noise models that are used in the 3. Simulation Methodology .........................6 simulations of CCD and CMOS photosensors and give 3.1. Model: From Photon to Charge: ............6 a working numerical model of a photosensor. The aim – simulating photon shot noise and PRNU ....7 of this article is to summarise the advances in numer- – converting photons to electrons ..............7 ical simulations of photosensors and provide a MAT- – simulating dark signal, dark current FPN and LAB implementation of a CCD/CMOS sensor model, dark current shot noise ......................7 which has been demonstrated to adequately describe – simulating Source follower noise .............8 noise properties of a hardware CMOS photosensor. 3.2. Model: From Charge to Voltage: ........... 8 The experimental results for a hardware 5T CMOS – simulating sense node reset noise (kTC noise) photosensor are provided in the article as validation and offset FPN ..............................9 for the developed numerical simulator. The article ad- – converting electrons to voltage ...............9 dresses the issue of the lack of high-level models of pho- – simulating V/e− and V/V gain non-linearity 10 tosensors. It is shown that the dark noise may have more complicated structure than is typically assumed 3.3. Model: From Voltage to Digital Numbers . 10 and therefore requires a more sophisticated statistical – simulating ADC non-linearity ..............10 description. The formulated high-level model of a pho- – simulating quantisation noise ...............10 tosensor allows the simulation of realistic noise effects 4. Experimental validation of the photosensor model 10 on the images and can be useful for testing of image 4.1. Radiometric function ......................11 processing algorithms. 4.2. Photon Transfer Curve ....................11 Software implementation: The MATLAB- 4.3. Signal-To-Noise Ratio . .12 based software implementation of the high-level 4.4. Photo Response Non-Uniformity ...........13 CCD/CMOS photosensor model described in this article is freely available on the website 4.5. Noise Spectrogram .........................14 https://code.google.com/p/highlevelsensorsim/ 4.6. Dark signal performance for different integration and also on email request. The software simulator times: .....................................15 written as a series of functions that call the corre- – Short integration time ......................16 sponding sub-functions sequentially to add the noise – Long integration time ......................17 with specified parameters. Two example test files 5. Conclusion .....................................18 are supplied that run: (1) a simple sensor model, which is completely linear and where all noise are Gaussian, and (2) an advanced model, which has 2 Noise sources in solid-state V/V and V/e non-linearities, LogNormal noise models and other specific noises. Both test files use the same photosensors code: adjusting the parameters in the test file, a user can turn the noise sources on or off and trigger Many noise sources contribute to the resulting noise non-linearities. The MATLAB code has a thorough image that is produced by photosensors. Noise sources documentation, both as comments inside the m-files can be broadly classified as either fixed-pattern (time- and rendered for web view [29] in a browser (see /help invariant) or temporal (time-variant) noise. Fixed- directory and index.html file inside). pattern noise refers to any spatial pattern that does not change significantly from frame to frame. Temporal Contents of the article noise, on the other hand, changes from one frame to the next. These noise sources will be described in detail in 1. Introduction .....................................1 the following subsections. 2. Noise sources in solid-state photosensors: ........2 2.1. From Photon to Charge: ................... 2 2.1 From Photon to Charge – photon shot noise and PRNU ................3 The ability of a semiconductor to produce electrons – dark signal, dark current FPN and dark current from incident photons is referred to as quantum effi- shot noise ...................................3 ciency (QE). Quantum efficiency can be supplied by – Source follower noise (Johnson noise, Flicker the sensor manufacturer in the form of spectral respon- (1/f) noise, and Random Telegraph Noise) ..4 sivity as a function of wavelength, or measured empiri- 2.2. From Charge to Voltage: ...................4 cally. For the sake of simplicty, we assume that the light – Sense node reset noise (kTC noise) ..........4 is monochromatic, and therefore we need the spectral – Offset FPN .................................4 response of the sensor at one wavelength. – Gain non-linearities (V/e− gain and V/V gain The charge collected in each pixel of a sensor array is non-linearity) ...............................5 converted to voltage by a sense capacitor and a source- High-level noise simulations in CCD/CMOS page 2 of 21.