Noname manuscript No. (will be inserted by the editor) Poisson noise reduction with non-local PCA Joseph Salmon · Zachary Harmany · Charles-Alban Deledalle · Rebecca Willett Received: date / Accepted: date Abstract Photon-limited imaging arises when the ing methods. The results reveal that, despite its concep- number of photons collected by a sensor array is small tual simplicity, Poisson PCA-based denoising appears relative to the number of detector elements. Photon to be highly competitive in very low light regimes. limitations are an important concern for many appli- Keywords Image denoising · PCA · Gradient cations such as spectral imaging, night vision, nuclear methods · Newton's method · Signal representations medicine, and astronomy. Typically a Poisson distri- bution is used to model these observations, and the inherent heteroscedasticity of the data combined with 1 Introduction, model, and notation standard noise removal methods yields significant arti- facts. This paper introduces a novel denoising algorithm In a broad range of imaging applications, observations for photon-limited images which combines elements of correspond to counts of photons hitting a detector ar- dictionary learning and sparse patch-based representa- ray, and these counts can be very small. For instance, in tions of images. The method employs both an adapta- night vision, infrared, and certain astronomical imaging tion of Principal Component Analysis (PCA) for Pois- systems, there is a limited amount of available light. son noise and recently developed sparsity-regularized Photon limitations can even arise in well-lit environ- convex optimization algorithms for photon-limited im- ments when using a spectral imager which character- ages. A comprehensive empirical evaluation of the pro- izes the wavelength of each received photon. The spec- posed method helps characterize the performance of tral imager produces a three-dimensional data cube, this approach relative to other state-of-the-art denois- where each voxel in this cube represents the light in- tensity at a corresponding spatial location and wave- Joseph Salmon length. As the spectral resolution of these systems Department LTCI, CNRS UMR 5141, Telecom Paristech increases, the number of available photons for each Paris, France E-mail: [email protected] spectral band decreases. Photon-limited imaging algo- arXiv:1206.0338v4 [cs.CV] 28 Apr 2014 rithms are designed to estimate the underlying spa- Zachary Harmany Department of Electrical and Computer Engineering tial or spatio-spectral intensity underlying the observed University of Wisconsin-Madison photon counts. Madison, Wisconsin, USA There exists a rich literature on image estimation E-mail: [email protected] or denoising methods, and a wide variety of effective Rebecca Willett tools. The photon-limited image estimation problem is Department of Electrical and Computer Engineering particularly challenging because the limited number of Duke University Durham, NC, USA. available photons introduces intensity-dependent Pois- E-mail: [email protected] son statistics which require specialized algorithms and Charles-Alban Deledalle analysis for optimal performance. Challenges associated IMB, CNRS-Universit´eBordeaux 1 with low photon count data are often circumvented Talence, France in hardware by designing systems which aggregate E-mail: [email protected] photons into fixed bins across space and wavelength 2 Joseph Salmon et al. (i.e., creating low-resolution cameras). If the bins are of the exponential family, and propose an optimiza- large enough, the resulting low spatial and spectral tion formulation for matrix factorization. Section 3 pro- resolution cannot be overcome. High-resolution obser- vides an algorithm to iteratively compute the solution vations, in contrast, exhibit significant non-Gaussian of our minimization problem. In Section 5, an impor- noise since each pixel is generally either one or zero tant clustering step is introduced both to improve the (corresponding to whether or not a photon is counted performance and the computational complexity of our by the detector), and conventional algorithms which ne- algorithm. Algorithmic details and experiments are re- glect the effects of photon noise will fail. Simply trans- ported in Section 6 and 7, and we conclude in Section forming Poisson data to produce data with approximate 8. Gaussian noise (via, for instance, the variance stabiliz- ing Anscombe transform [2,31] or Fisz transform [17, 18]) can be effective when the number photon counts 1.2 Problem formulation are uniformly high [5,46]. However, when photon counts are very low these approaches may suffer, as shown later For an integer M > 0, the set f1; : : :; Mg is denoted in this paper. 1;M . For i 2 1;M , let yi be the observed pixel val- J K J K This paper demonstrates how advances in low- ues obtained through an image acquisition device. We dimensional modeling and sparse Poisson intensity re- consider each yi to be an independent random Poisson construction algorithms can lead to significant gains in variable whose mean fi ≥ 0 is the underlying intensity photon-limited (spectral) image accuracy at the reso- value to be estimated. Explicitly, the discrete Poisson lution limit. The proposed method combines Poisson probability of each yi is Principal Component Analysis (Poisson-PCA { a spe- f yi cial case of the Exponential-PCA [10,40]) and sparse (y jf ) = i e−fi ; (1) P i i y ! Poisson intensity estimation methods [20] in a non-local i estimation framework. We detail the targeted opti- where 0! is understood to be 1 and 00 to be 1. mization problem which incorporates the heteroscedas- A crucial property of natural images is their ability tic nature of the observations and present results im- to be accurately represented using a concatenation of proving upon state-of-the-art methods when the noise patches, each of which is a simple linear combination of level is particularly high. We coin our method Pois- a small number of representative atoms. One interpre- son Non-Local Principal Component Analysis (Poisson tation of this property is that the patch representation NLPCA). exploits self-similarity present in many images, as de- Since the introduction of non-local methods for im- scribed in AWGN settings [11,30,12].p Let Ypdenote the age denoising [8], these methods have proved to out- M × N matrix of all the vectorized N × N overlap- perform previously considered approaches [1,11,30,12] ping patches (neglecting border issues) extracted from (extensive comparisons of recent denoising method can the noisy image, and let F be defined similarly for the be found for Gaussian noise in [21,26]). Our work is in- true underlying intensity. Thus Yi;j is the jth pixel in spired by recent methods combining PCA with patch- the ith patch. based approaches [33,47,15] for the Additive White Many methods have been proposed to represent the Gaussian Noise (AWGN) model, with natural exten- collection of patches in a low dimensional space in the sions to spectral imaging [13]. A major difference be- same spirit as PCA. We use the framework considered tween these approaches and our method is that we di- in [10,40], that deals with data well-approximated by rectly handle the Poisson structure of the noise, without random variables drawn from exponential family distri- any \Gaussianization" of the data. Since our method butions. In particular, we use Poisson-PCA, which we does not use a quadratic data fidelity term, the singu- briefly introduce here before giving more details in the lar value decomposition (SVD) cannot be used to solve next section. With Poisson-PCA, one aims to approxi- the minimization. Our direct approach is particularly mate F by: relevant when the image suffers from a high noise level (i.e., low photon emissions). Fi;j ≈ exp([UV ]i;j) 8(i; j) 2 1;M × 1;N ; (2) J K J K where 1.1 Organization of the paper { U is the M × ` matrix of coefficients; { V is the ` × N matrix representing the dictionary In Section 1.2, we describe the mathematical frame- components or axis. The rows of V represents the work. In Section 2, we recall relevant basic properties dictionary elements; and Poisson noise reduction with non-local PCA 3 { exp(UV ) is the element-wise exponentiation of UV : The parameter θ 2 Θ is called the natural parameter exp [UV ]i;j := exp(UV ) i;j. of P(φ), and the set Θ is called the natural parameter space. The function Φ is called the log partition func- The approximation in (2) is different than the approxi- tion. We denote by [·] the expectation w.r.t. p : mation model used in similar methods based on AWGN, Eθ θ where typically one assumes Fi;j ≈ [UV ]i;j (that is, Z Eθ[g(X)] = g(y) (exp(hθjφ(y)i) − Φ(θ)) dν(y): without exponentiation). Our exponential model allows X us to circumvent challenging issues related to the non- negativity of F and thus facilitates significantly faster Example 1 Assume the data are independent (not nec- algorithms. essarily identically distributed) Gaussian random vari- 2 The goal is to compute an estimate of the form (2) ables with means µi and (known) variances σ . Then n from the noisy patches Y . We assume that this approx- the parameters are: 8y 2 R ; φ(y) = y, Φ(θ) = Pn 2 2 2 2 imation is accurate for ` M, whereby restricting the i=1 θi =2σ and rΦ(θ) = (θ1/σ ; ··· ; θn/σ ) and ν is n rank ` acts to regularize the solution. In the following the Lebesgue measure on R (cf. [34] for more details on section we elaborate on this low-dimensional represen- the Gaussian distribution, possibly with non-diagonal tation. covariance matrix). Example 2 For Poisson distributed data (not necessar- 2 Exponential family and matrix factorization ily identically distributed), the parameters are the fol- n lowing: 8y 2 R ; φ(y) = y, and Φ(θ) = hexp(θ)j1ni = Pn θi We present here the general case of matrix factorization i=1 e , where exp is the component-wise exponential for an exponential family, though in practice we only function: use this framework for the Poisson and Gaussian cases.