LOW RANK FOURIER PTYCHOGRAPHY Zhengyu Chen, Gauri Jagatap, Seyedehsara Nayer, Chinmay Hegde, Namrata Vaswani Iowa State University ABSTRACT reconstruction problem is nascent, and several important questions remain unanswered. Rigorous guarantees of correctness of the meth- In this paper, we introduce a principled algorithmic approach for ods proposed in [8, 10] are not yet available. More worrisome is the Fourier ptychographic imaging of dynamic, time-varying targets. To issue of sample complexity of image reconstruction, since a basic the best of our knowledge, this setting has not been explicitly ad- requirement in all existing methods is that the number of measure- dressed in the ptychography literature. We argue that such a setting ments must exceed the resolution of the image. However, this re- is very natural, and that our methods provide an important first step quirement can be particularly challenging when imaging a dynamic towards helping reduce the sample complexity (and hence acquisi- scene involving a moving target; for a video sequence with q images tion time) of imaging dynamic scenes to managaeble levels. With each with resolution n, without using any structural prior assump- significantly reduced acquisition times per image, it is conceivable tions, the number of observations must be at least Ω(nq), which can that dynamic ptychographic imaging of fast changing scenes indeeed quickly become prohibitive. becomes practical in the near future. Index Terms— Phase retrieval, ptychography, low rank 1.2. Our Contributions 1. INTRODUCTION In this paper, we introduce a principled algorithmic approach for Fourier ptychographic imaging of dynamic, time-varying targets. To 1.1. Motivation the best of our knowledge, this setting has not been explicitly ad- dressed in the ptychography literature. However, we argue that such In recent years, the classical problem of phase retrieval has attracted a setting is very natural, and that our methods provide an important renewed interest in the signal and image processing community. The first step towards alleviating the aforementioned issues of sample phase retrieval problem involves reconstructing a length-n discrete- complexity that can arise in Fourier ptychography. time signal (or image) given noisy observations of the magnitudes The high level idea is that if the dynamics of the scene are suf- of its discrete Fourier transform (DFT) coefficients. A generalized ficiently slow, then the underlying video signal can be well-modeled version of phase retrieval studies a similar reconstruction problem by a low-rank matrix. This modeling assumption has been success- where the DFT is replaced by a generic linear measurement opera- fully employed in a variety of video acquisition, compression, and tor. A series of recent breakthrough results [1,2,3, 24,4] have intro- enhancement applications [11, 12, 13, 14, 15]. Specifically, if we duced principled and provably accurate algorithms for generalized n reshape each image in the video sequence as a vector xk 2 R and n×q phase retrieval, provided the measurement operator is constructed stack up q consecutive frames into a matrix X 2 R , then X is by sampling vectors from certain families of multivariate probability (approximately) rank-r, with r min(n; q). distributions. The crux of this paper is to demonstrate how we can effectively Phase retrieval algorithms enable a variety of imaging appli- leverage such low-rank structure in order to enable better image cations ranging from X-ray crystallography and biomedical imag- reconstruction (specifically, one that surpasses the na¨ıve approach ing [5,6,7]. A related imaging technique is known as Fourier of reconstructing each image frame by frame using an existing ptychography, which can be used for super-resolving images ob- method). To this end, we advocate a new Fourier ptychographic tained in microscopic imaging systems. The high level approach is to approach that consists of two key ingredients: capture multiple snapshots of a target scene using a programmable coherent illumination source coupled with a system involving two 1. We develop two novel “under-sampling” strategies that can con- lenses, and reconstruct a high-resolution image of the target scene siderably reduce the sample complexity of video Fourier pty- via (generalized) phase retrieval. One way to engineer multiple snap- chography. Moreover, these strategies can be implemented in shots of a scene is to fix the position of the illumination source, and common Fourier ptychographic imaging setups (such as [9, 10]) either let the camera aperture undergo spatial translations [8], or con- in a straightforward fashion. struct an array of fixed cameras, each of which captures a specific 2. We couple these under-sampling strategies with a new image re- portion of the Fourier spectrum of the desired high-resolution im- construction algorithm which fully exploits the underlying low- age. Zheng et al. [9] have demonstrated that using such a system, rank structure of the target video sequence. Moreover, we con- one can image beyond the diffraction-limit of the objective lens in firm the advantages of this algorithm via a number of simulation a microscope. Recently, Holloway et al. have demonstrated similar, experiments. very promising results in the context of long-distance sub-diffraction Our algorithm builds upon those introduced in our recent previ- imaging [10]. ous work on low rank phase retrieval [16]. While that paper analyzes While the above results indicate the considerable promise of a similar setup for leveraging low-rank structure for (generalized) Fourier ptychography, an algorithmic understanding of the image phase retrieval, we only considered special families of measurement This work is supported in parts by the National Science Foundation un- operators (i.i.d. Gaussian measurements [1] and coded-diffraction der the grants CCF-1526870, CCF-1566281 and IIP-1632116 patterns (CDP) [17]). We expand the utility of our previous work 1 to the Fourier ptychographic measurement setup, hence demonstrat- −1 ing a real-world application of our approach for the first time. Ai : x F Pi◦ F Mi y^i Our reconstruction algorithm involves a non-convex, iterative estimation procedure; hence, initializing the algorithm properly is crucial. It turns out that the initialization procedure in [16] is not par- y^i j · j yi ticularly effective (since they are specially tailored to the Gaussian or the CDP case). Instead, we devise a novel initialization mech- anism for our reconstruction algorithm, and justify it conceptually > −1 as well as in experiments. We experimentally demonstrate that our Ai : y^i Mi F Pi◦ F x^i new modified reconstruction algorithm (that we call Low-Rank Pty- chography, or LRPtych), compares very favorably in terms of sample Fig. 1. Sequence of operations defined by A . Here the green box in- complexity when contrasted with existing “single-frame” methods, i dicates the extra sub-sampling step and i = [N] denotes the camera such as the Iterative Error Reduction Algorithm (IERA) of [10]. index. This paper focuses on Fourier ptychographic acquisition of dy- namic scenes that are well approximated as forming a low-rank ma- trix. In a companion paper [18], we develop algorithms for Fourier 2.2. Mathematical model ptychography for static scenes that obey intra-frame modeling as- sumptions such as sparsity and/or structured sparsity. While the un- We now represent the optical setup mathematically. Consider a (high dersampling strategies in both cases are similar, the associated re- resolution) spatial light field at time instant k, denoted by xk. We de- construction algorithms are very different. See [18] for further de- fine the video matrix X, composed of frames xk for k 2 [1; : : : ; q]: tails. n×q X := [x1; x2;:::; xq]; X 2 R : 1.3. Prior Work We assume that the dynamics of the video are sufficiently slow, There exists a rich body of literature on phase retrieval, dating back and hence the rank of matrix is X is no greater than r, with r min(n; q). For each video frame xk, the ptychographic measure- to the optical imaging work of [19, 20, 21, 22]. th There is also a large (and newer) body of work on low-rank ments corresponding to the i camera location takes the following models for video acquisition and analysis [11, 12, 13]. Our recent form: work [16] is the first to leverage low-rank models for (generalized) y = jA (x )j: phase retrieval. However, the algorithms introduced in that work are i;k i;k k designed to succeed in the case of i.i.d. Gaussian measurements, and Here, we introduce the operator Ai;k, where the index i 2 [1;:::;N] the applicability of that technique to other families of measurements corresponds to different camera positions, and the index k = has not been explored. 1; 2; : : : ; q indicates the time stamp. Collectively, all measurements On the other hand, the literature on Fourier ptychography has of a single frame xk can be expressed in terms of the cumulative mainly focused on achieving improved resolution of the recon- measurement vector yk, defined as follows: structed images [8,9, 23], and relatively little attention has been 2 3 given to the issue of (potentially) large measurement complexity. jA1;k(xk)j As discussed above, this issue is particularly acute in the case 6 jA2;k(xk)j 7 y = 6 7 : of dynamic (video) imaging. Below, we discuss a ptychographic k 6 . 7 reconstruction framework that integrates low-rank models to consid- 4 . 5 erably reduce measurement rates, at little or no reduction of image jAN;k((xk)j reconstruction quality. In terms of various stages of the data acquisition process, the opera- tor Ai;k can be expressed as: 2. DATA ACQUISITION SETUP −1 Ai;k(·) = Mi;kF Pi ◦ F(·): (1) 2.1. Optical setup where F and F −1 represent the Fourier and inverse Fourier trans- We describe the standard Fourier ptychographic measurement pro- th forms, and Pi is a pupil mask corresponding to the i camera.