JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1 Deconvolved Image Restoration from Autocorrelations Daniele Ancora* and Andrea Bassi Abstract—Recovering a signal from auto-correlations or, equiv- problem: methods based on alternating projections, protocols alently, retrieving the phase linked to a given Fourier modulus, based on optimization and iterative approaches inspired dif- is a wide-spread problem in imaging. This problem has been ferent inversion strategies. Although the field is in continuous tackled in a number of experimental situations, from optical microscopy to adaptive astronomy, making use of assumptions progress, Shechtman et al. provides an extensive review on based on constraints and prior information about the recovered the topic [11]. object. In a similar fashion, deconvolution is another common Measurements relying on the estimation of the object’s auto- problem in imaging, in particular within the optical community, correlation might be corrupted from blurring, which is given allowing high-resolution reconstruction of blurred images. Here by the limited bandwidth of the detection system or limited we address the mixed problem of performing the auto-correlation inversion while, at the same time, deconvolving its current by diffraction. In this case, two consecutive inverse problems estimation. To this end, we propose an I-divergence optimization, (deconvolution and phase retrieval) should be solved. Here, driving our formalism into a widely used iterative scheme, instead, we propose an iterative procedure that allows one to inspired by Bayesian-based approaches. We demonstrate the obtain a deconvolved image from auto-correlation measure- method recovering the signal from blurred auto-correlations, ments, solving both problems at the same time. We ground further analysing the cases of blurred objects and band-limited Fourier measurements. the procedure on the I-divergence minimization, similarly to Richardson-Lucy [1], [2] deconvolution algorithm and in Index Terms—Deconvolution, phase retrieval, computational close analogy with Schultz-Snyder approach [12]. For this imaging, auto-correlation inversion, deblurring, inverse problem. purpose, it is useful to rearrange the auto-correlation into a convolutional form. This, renders the problem similar to a deconvolution task in which the target-object is blurred by a I. INTRODUCTION kernel that depends on itself. Inspired by blind deconvolution ECONVOLUTION is an image processing technique strategies [13], we discuss how neglecting the kernel’s de- D commonly used in a number of computer vision and pendence on the object does not alter reconstruction abilities. optics applications [1], [2], including astronomy [3] and Our approach is similarly developed to established methods in optical microscopy [4]. The goal of image deconvolution the field of signal processing. This is a useful design, since is to restore super-resolved features of an object [5] from the reconstruction can be further improved by adding total- images acquired with diffraction-limited optics or aberrated variation regularization [14] or deconvolution with unknown wavefronts. A different computational imaging technique deals kernels [13]. In the following section, we start introducing the with the estimation of an object’s auto-correlation. This is an general problem-framework, discussing the context of applica- increasingly studied problem in the fields of not-conventional bility and different measurement-scenarios. In sec. III, we in- imaging and far-field diffraction imaging. Applications based troduce the iterative algorithm that solves both auto-correlation on auto-correlation imaging can be found in, but are not inversion and deconvolution problem. Sec.IV presents the limited to, image detection through a turbulent medium (e.g. reconstruction results and examines the deconvolution ability the atmosphere) [6], lens-less imaging [7], hidden imaging of the technique. Conclusions and potential implementation arXiv:2007.02029v1 [eess.IV] 4 Jul 2020 [8] and tomography [9]. In order to form a reconstruction are discussed in the last part of the article, in sec.V. of the object acquired in one of these conditions, typically, mds one needs to invert the signal’s auto-correlation. to reconstruct June 23, 2020 the signal that generated it. Since the auto-correlation forms a Fourier pair with the modulus of the Fourier transform of the object, the image reconstruction process is often referred II. PROBLEM STATEMENT to as the Phase-Retrieval (PR) problem. Furthermore, given First of all, let’s define the formalism that we will use that it is not possible to directly measure the phase with throughout the manuscript. We will make use of integral camera-sensors, PR compensates for that trying to find the formulation, where every equation may be transported into its phase connected with the Fourier modulus of the underlined discrete equivalent by replacing the integral with a summation object [10]. There are several approaches to solve this inverse over the integrated variables. For the sake of notation, we use D. Ancora is with with the Department of Physics, Politecnico di Milano, one-dimensional functions of the spatial variable x, eventually 20133 Milan, Italy (*email to: [email protected]). shifted by the quantity ξ due to convolution or correlation A. Bassi is with with the Department of Physics, Politecnico di Milano, operation. The formalism can be extended to any dimension- 20133 Milan, Italy and with Istituto di Fotonica e Nanotecnologie, Consiglio Nazionale delle Ricerche, Piazza Leonardo da Vinci 32, 20133 Milan, Italy ality, and we therefore drop any explicit variable dependence. Manuscript received June XX, 2020; revised June XX, 2020. The operator Ff:g indicates the Fourier transform and · JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 2 the element-wise product. We indicate the cross-correlation A) B) C) between two generic functions f (x) and g (x) as: Z f ? g = f (x)g (ξ + x) dx = F−1Fffg · Ffgg (1) consequently the auto-correlation of the function f with itself χ = o o χµ = χ + is f ? f. The convolution is defined as: H ∗ H Z D) E) F) f ∗ g = f (x) g (ξ − x) dx = F−1Fffg · Ffgg (2) and the auto-convolution as f ∗ f. We are interested in the reconstruction of an unknown object o. Its auto-correlation is χ0 = (o h + 0) defined as: µ ∗ o h (o h + 0) χ = o ? o = F−1kFfogk2 : (3) ∗ The identity on the right hand side of eq.3, is given by the power spectral theorem. Throughout the text we will make use of the greek-pedix µ, which indicates the blurred quantity with added noise, simulating an experimental measurement. Fig. 1.0.0 Panel A)0.2 Noise-free0.4 test image 0.6o of a satellite0.8 [18] used1.0 to test Normalized intensity (a. u. ) To avoid confusion, from now on we make use of the word the reconstruction protocol (size 256 × 256 px). B) Gaussian kernel H with σ = 2 px ”measurement” when referring to a simulated effect that may standard deviation used to blur the auto-correlation. C) Ideal object’s auto-correlation χ = o ? o. D) Auto-correlation χµ corrupted by be measured in a real imaging experiment. In the present paper Poissonian noise " and blurred by H, reproducing the quantitity to which we we address the case in which the auto-correlation is blurred, have access with standard measurements. All the images are peak-normalized RdBu a blurred object is used to compute the auto-correlation and and displayed using a diverging color-map ( ). the measurement is performed in a band-limited Fourier space. Although slightly different, it can be seen that these problems moment, we consider " as a generic form of additive noise. are equivalent and can be solved with the same iterative Approaching this problem would require to deconvolve the approach. auto-correlation first (i.e. using Richardson-Lucy [1], [2]) and then to find the signal having that (de-blurred) auto-correlation A. Blurred Auto-correlation (e.g. by using Fienup iterative phase retrieval [10]), or vice versa. In some applications, we have access to a statistically com- Instead, here we want to approach both problems simulta- puted estimate of the auto-correlation. Looking through turbu- neously. To do this, we reduce the formula above to the con- lent atmosphere in astronomy [6], through scattering slabs or volutional form o ∗ K, where the object of interest o is subject behind corners [8], [15] and performing hidden tomography to the blurring kernel K. Let’s rearrange the correlations and [9] are a few examples. The light emerging from turbulent convolutions in a convenient form: environment, in these conditions, has undergone unpredictable scattering events and when detected resembles a random χ ∗ H = (o ? o) ∗ H = o ? (o ∗ H) arrangement of intensity distribution. It has been proven that = o ? (H ∗ o) = (o ? H) ∗ o under isoplanatic conditions, known also as ”memory effect” = o ∗ (o ? H) ≡ o ∗ K; (5) regime [16], the auto-correlation of these patterns shares the same object’s auto-correlation. Thus, the turbidity acts as an where we used the fact that convolution between two functions opaque auto-correlation lens. In order to estimate the auto- f and g permutes, f ∗g = g∗f, and the correlation-convolution correlation of the hidden object, we can process the patterns identity (f ? g) ∗ h = f ? (g ∗ h). produced by the light propagation through turbid environment. In this way, we have introduced a new blurring kernel that Since the auto-correlation is typically averaged through several depends on the object itself K [o] = o ? H. The convolution of detections, each of which affected by blurring, the presence the object with K gives the measured auto-correlation under of an effective point-spread function (PSF) blurs the final the effect of a blurring factor H. estimate of the auto-correlation. Thus, we have access to a measurement of the auto-correlation given by: B.