The Geometry of the Phase Retrieval Problem Charles L

The Geometry of the Phase Retrieval Problem Charles L. Epstein The Geometry of the Phase Retrieval Problem Introduction Operator Splitting Methods in Data Analysis Workshop Discrete Problem

Torus Geometry

Reconstruction Charles L. Epstein Algorithms

Numerical Examples CCM at the Flatiron Institute University of Pennsylvania Other Conditions History of CDI March 20, 2019 Bibliography Acknowledgment

The Geometry of the Phase Retrieval Problem Charles L. Epstein I want to thank Christian and Patrick for inviting me to speak. Introduction What I will discuss today is somewhat unconventional, ongoing Discrete Problem work, at the intersection of pure mathematics, numerical analysis

Torus Geometry and mathematical physics.

Reconstruction Algorithms This work was done jointly with Alex Barnett, Leslie Greengard, Numerical and Jeremy Magland and supported by the Centers for Examples Computational Biology and Mathematics at the Flatiron Institute Other Conditions of the Simons Foundation, New York, NY. History of CDI

Bibliography Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography High Resolution Imaging

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction Today I will speak about inverse problems that arise in Coherent Discrete Problem Diffraction Imaging. This is a technique that uses very high Torus Geometry energy, monochromatic light, either electrons or x-rays, to form Reconstruction high resolution images of animate and inanimate materials. The Algorithms goal is to achieve resolutions in the 1–10nm range. Numerical Examples

Other Conditions

History of CDI

Bibliography What is Measured?

The Geometry of the Phase Retrieval Problem Charles L. The photons used to illuminate the samples are in the 0.1-10KeV Epstein range; the light is typically assumed to be monochromatic, Introduction produced either by a laser or as synchrotron radiation. The object Discrete being imaged is typically many wavelengths across and the Problem measurement is made very far from the object and light source, Torus Geometry

Reconstruction hence in the far field (the Fraunhofer regime). As is well known, Algorithms if .x/ describes the density of the object, then the leading order Numerical Examples term in the far field expansion of the scattered radiation is .k/; : Other proportional to the square of the Fourier transform, b of At Conditions such small wavelengths, one can only measure the intensity of the History of CDI 2 scattered field, b.k/ ; and not its phase. Bibliography j j Examples, I

The Geometry of the Phase Retrieval Here are computed far ﬁeld diffraction patterns produced by (a) a Problem semi-circle, (b) an equilateral triangle. Charles L. Epstein

Introduction

Discrete Problem

Torus Geometry

Reconstruction Algorithms

Numerical Examples

Other Conditions

History of CDI

Bibliography Examples, II FROZEN-HYDRATED YEAST SPORES RECONSTRUCTION

The Geometry of the Phase Here is an actual diffraction• E. Lima, patternPhD Thesis, produced Stony Brook by frozen-hydrated University Retrieval Problem yeast spore at 520• eV.Unstained Courtesy frozen-hydrated E. Lima PhD yeast thesis, spore, Stony at 520 Brook eV Charles L. U, from [8]. Epstein • First CDI reconstruction of a frozen hydrated biological object

Introduction

Discrete Problem

Torus Geometry

Reconstruction Algorithms

Numerical Examples

Other Conditions

History of CDI

Bibliography

Signal extends to 25 nm Brightness - amplitude half-period Hue - phase ESRF Lecture Series on Coherent X-rays and their Applications, Lecture 7, Malcolm Howells What is Needed?

The Geometry .x/ of the Phase Of course we would like to reconstruct from this Retrieval Problem measurement, but we need to, at least, estimate the unmeasured Charles L. phase of b.k/: This is called the phase retrieval problem. Epstein It also arises in x-ray crystallography, but is really quite a different Introduction problem for a periodic structure. In the context of a compact Discrete object, the possibility of solving this problem was ﬁrst suggested Problem

Torus Geometry by D. Sayre in 1952. He saw it as a consequence of Nyquist’s

Reconstruction sampling theorem! See [1]. Algorithms For this problem to be solvable, even in principle, some auxiliary Numerical Examples information is required, such as an estimate for the support of the Other object, or knowledge that the function .x/ is non-negative and an Conditions estimate on the support of its autocorrelation function. History of CDI

Bibliography Many attempts have been made to solve this problem, but with rather limited success. Today we’ll discuss the underlying reasons for the difﬁculty of this problem...which may suggest better approaches. Outline of this Lecture

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction We will try to cover to following material: Discrete Problem The Discrete Phase Retrieval Problem (a simpler model) Torus Geometry Geometry of the Amplitude Torus Reconstruction Algorithms Reconstruction Algorithms Numerical What We Learn from Numerical Examples Examples

Other Conditions

History of CDI

Bibliography Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography A Simpler Model Problem

The Geometry of the Phase Retrieval Problem Charles L. The phase retrieval problem encountered in coherent diffraction Epstein imaging is fraught with many practical difﬁculties. For the Introduction remainder of the talk we focus on a simpler model problem, which Discrete Problem already proves very difﬁcult to solve. We call this the discrete,

Torus Geometry classical phase retrieval problem. We imagine that the unknowns

Reconstruction are samples of an object on a ﬁnite uniform grid: Algorithms

Numerical xj .j1x; j2x/; where j J: (1) Examples D 2 Other Conditions Here J Œn1 N1 Œn2 N2; is a rectangular grid. We call D W W History of CDI such collections of data indexed by J images. Bibliography The Measurements

The Geometry of the Phase In our model problem, the measured data are the magnitudes of Retrieval Problem the DFT of these samples: Charles L. Epstein aj xj j bJ ; (2) f D j O j W 2 g Introduction where bJ is a set of sample frequencies, with J bJ : If Discrete d j j D j j Problem J Œ0 N 1 ; then D W Torus Geometry X 2ij k N Reconstruction xj xke : (3) Algorithms O D k J Numerical 2 Examples We let RJ RJ denote the measurement map: Other M W ! Conditions C

History of CDI .x/ . xj /j J : (4) M D j O j 2 Bibliography There are important differences between these model measurements and the samples of a continuum Fourier transform that would actually be collected. The Support Condition, I

The Geometry of the Phase As noted above we need to have auxiliary information to be able Retrieval Problem to solve the phase retrieval problem. We imagine that the support Charles L. x;S ; Epstein of the unknown image x is contained in a rectangular subset R J with the side lengths of R at most half those of J: In the Introduction literature it is often said that we “oversample,” but really we just Discrete Problem need to sample in k-space on a ﬁne enough grid for the data set to Torus Geometry contain information about the support of x: Reconstruction Algorithms Our auxiliary information will be an estimate for Sx: This is a Numerical subset S with Sx S R J: We are therefore looking for an Examples image x so that xj 0 for j S; and Other D … Conditions History of CDI xj aj for all j bJ: (5) j O j D 2 Bibliography This problem usually does not have a unique solution, but generically, the non-uniqueness is of a simple sort. Trivial Associates

The Geometry of the Phase Retrieval Problem x J; Charles L. If is an image indexed by then we think of it as being Epstein periodic in Z2; with j J representing a single period cell. We f 2 g can then translate an image by any lattice vector k Z2; deﬁning Introduction 2 Discrete Problem .k/ d xj xj k: (6) Torus Geometry D Reconstruction Algorithms We can also deﬁne the inverted image by setting

Numerical Examples xj x j : (7) Other L D Conditions

History of CDI Translation and inversion lead to images with the same magnitude

Bibliography Fourier data; these are called the trivial associates of x: Uniqueness

The Geometry of the Phase Retrieval Hence there are always many images with the same magnitude Problem R; Charles L. Fourier data, and support in a translate of but there is a Epstein theorem, due to Monson Hayes (see [5]) which states that

Introduction generically this is the only form of non-uniqueness one

Discrete encounters: with sufficiently fine sampling in the Fourier domain, Problem the only images with the given magnitude Fourier data are the Torus Geometry trivial associates of a given image with this data. Reconstruction Algorithms Let Sx S R; as above; the support condition S defines a Numerical J Examples linear subspace of R ; Other Conditions J BS x R xj 0 if j S : (8) History of CDI D f 2 W D … g Bibliography 1 In all cases . .x// BS is a finite set, and generically it M M \ consists of trivial associates of x: The Support Condition, II

The Geometry of the Phase Retrieval Problem The better our estimate, S; for the support is, the fewer trivial Charles L. associates there are, with the given magnitude Fourier data, that Epstein will also satisfy the condition that Sx S: One might expect that Introduction if S Sx; then the phase retrieval problem should be “easy,” but Discrete D Problem that, in fact, is not usually the case. Torus Geometry For numerical experiments we often quantify our knowledge of Reconstruction Algorithms the support of x in terms of p-pixel neighborhoods. The p-pixel 2 Numerical neighborhood of a set U Z is obtained as the union of all Examples translates of U by integer vectors .j1; j2/ lying in the square with Other Conditions vertices at . p; p/: ˙ ˙ History of CDI Before turning our attention to methods for solving the phase Bibliography retrieval problem, we consider its underlying geometry. The Magnitude Torus

The Geometry of the Phase Retrieval Problem Charles L. As we’ve assumed that the image is real, the phase retrieval Epstein problem takes place in the ambient space RJ ;

Introduction The magnitude Fourier data of x aj j bJ ; deﬁnes a torus, W f W 2 g Discrete which we call the magnitude torus deﬁned by x Problem W Torus Geometry J Aa y R yj aj for all j bJ : (9) Reconstruction D f 2 W j O j D 2 g Algorithms

Numerical The cardinality of J is typically in the hundreds of thousands, or Examples millions. For a small, low resolution image Other J Conditions J 4096 64 64: The dimension of Aa is about j2j ; and (in j j D D J History of CDI 2d) dim BS is about j4j : Bibliography Geometric Statement of the Phase Retrieval Problem

The Geometry of the Phase Retrieval Problem Geometrically the phase retrieval problem is the problem of Charles L. Epstein finding points in the intersection Aa BS : \ Introduction Alternately we can imagine trying to find a local inverse for the Discrete Problem measurement map BS defined above. Torus Geometry M Reconstruction A very important difference between our model problem and the Algorithms

Numerical actual problem in CDI (with ﬁnitely many samples of the Examples continuum Fourier transform) is that the model problem has Other Conditions exact solutions, but the actual problem does not: A function

History of CDI with only ﬁnitely many non-zero Fourier coefﬁcients cannot be

Bibliography either compactly supported or non-negative. Other Phase Retrieval Problems, I

The Geometry of the Phase Retrieval Generalized phase retrieval has enjoyed a recent resurgence of Problem Charles L. interest, with work of Candes, Daubechies, et al. The problems Epstein that these authors have considered are usually quite different from

Introduction the classical phase retrieval problem. Discrete Brieﬂy, one begins with a Hilbert space . ; ; ; / and a frame Problem H h i ˚ 'n n for : The measurements at one’s disposal are Torus Geometry D f W 2 Ig H Reconstruction the magnitudes of the frame coefﬁcients: Algorithms

Numerical ˚ .x/ . x;'n /n I : (10) Examples M D jh ij 2 Other Conditions The phase retrieval problem is then to recover x; up to a global History of CDI phase (or sign) from the measurements ˚ .x/: From the M Bibliography deﬁnition of a frame it follows that ˚ .x/ ˚ .x/ 2 C1 x y H: kM M k Ä k k Other Phase Retrieval Problems, II

The Geometry If dim < and complex, (real) then generically, for a of the Phase H 1 H Retrieval frame, ˚; with > 4 dim 4; ( > 2 dim 2) the map Problem jIj H C jIj H C x ˚ .x/ is injective and the generalized phase retrieval Charles L. 7! M Epstein problem it deﬁnes is, in principle, solvable. Introduction In this case one can show that the inverse satisﬁes a Lipschitz Discrete estimate: there exists a 0 < C < so that Problem 1 Torus Geometry i˛ inf x e y H C ˚ .x/ ˚ .x/ 2: (11) Reconstruction ˛ k k Ä kM M k Algorithms Numerical The problem we consider is quite different in that the Examples measurements .x/ never determine a unique solution. The Other M Conditions question about the Lipschitz estimate is quite interesting, and History of CDI rather subtle. Bibliography Many real physical measurements can only be modeled by the classical Fourier measurements, and the generalized theory simply does not apply. Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography Finding Intersections of Submanifolds

The Geometry of the Phase The underlying problem in phase retrieval is that of ﬁnding the Retrieval A; B Problem intersection between two subsets, of a vector space, where we know that A B consists of a ﬁnite set of points. In most Charles L. \ Epstein analyses of this class of problems, it is assumed that the sets A Introduction and B are convex and meet transversally; that is, A and B are Discrete convex submanifolds and if x A B; then the tangent spaces Problem 2 \ meet only at x itself: Torus Geometry Reconstruction TxA TxB x : (12) Algorithms \ D f g Numerical Examples

Other Transversality is more or less necessary for the linearized Conditions problem, that of locating TxA TxB; to provide a good model \ History of CDI for problem of locating points in A B: Bibliography \ The magnitude torus is never convex and, as we see below, the transversality assumption almost always fails in the classical phase retrieval problem. The Tangent Space

The Geometry of the Phase A surprising feature of the classical problem is the extent to which Retrieval Problem we can explicitly describe the tangent space to a magnitude torus. Charles L. Let y Aa and let y be the DFT of y: The following vectors Epstein 2 O contain a spanning set for the ﬁber of Ty Aa W Introduction 1 h i Discrete tj iyj ej iyj ej for j J: (13) Problem D F O O 0 2 Torus Geometry Here j is the “conjugate index” to j ; for which reality of y Reconstruction 0 Algorithms implies that Numerical y y : (14) Examples O j 0 D O j Other Each non-zero vector appears twice, and we let J1 J be chosen Conditions so that History of CDI ( ) Bibliography tj p j J1 : (15) 2 xj W 2 j O j is a basis for the ﬁber of Ty Aa: Computing with this Basis

The Geometry of the Phase Retrieval Problem Charles L. In all cases, a basis for the ﬁber of the tangent space can be Epstein described (in the Fourier domain) by the list of the indices in J1 Introduction paired with J1 complex numbers of modulus 1. Notice that an Discrete j j Problem orthonormal basis for a subspace of this dimension generically requires a matrix with dimensions J1 J : Hence these Torus Geometry j j j j Reconstruction subspaces admit a highly compressed representation, which Algorithms requires only the FFT to access and use. Numerical Examples In fact, there is a different description of the tangent space that is Other Conditions even more useful for analyzing the relationship of Ty Aa to BS ; History of CDI when y Aa BS : 2 \ Bibliography 0 0 A Second Basis for Ty Aa and Ny Aa

The Geometry of the Phase Retrieval Problem We state this as a lemma: Charles L. Epstein Lemma 2 Let y Aa and v N ; then the vectors Introduction 2 2 Discrete .v/ .v/ . v/ 0 .v/ .v/ . v/ 0 Problem y y T Aa and y y N Aa: D 2 y D C 2 y Torus Geometry (16) Reconstruction If xj 0; for any j J; then these sets contain spanning sets Algorithms O ¤ 2 Numerical for these vector spaces. Examples .v/ Other Recall that xj xj v; is just a translate of x: This allows us to Conditions D examine the intersection Ty Aa BS in detail. If Sy S, and v History of CDI \ is a vector so that both S .v/ S and S . v/ S; Then the y y Bibliography .v/ 0 tangent vector T Aa BS : 2 y \ The dim Ty Aa BS \ p

The Geometry of the Phase Retrieval Problem Suppose that Sp is the p-pixel neighborhood of Sx; for some Charles L. Epstein image x; with small support.

Introduction It turns out that the dimension of the “numerical” intersection Discrete Ty Aa BSp depends on both the smoothness of the image and Problem \ the size of the neighborhood. A simple combinatorial argument Torus Geometry

Reconstruction shows that Algorithms dim Ty Aa BSp 2p.p 1/: (17) Numerical \ C Examples This does not depend on the ambient dimension, i.e. J : For a Other j j Conditions piece-wise constant image this gives the exact dimension of the intersection (again independently of J ). As the image becomes History of CDI j j Bibliography smoother this becomes a lower bound. Test Images

The Geometry of the Phase We examine these intersections for “small” examples. Retrieval Problem Test images are obtained as samples of random collections of Charles L. disks, which have been smoothed by a Gaussian whose width is Epstein determined by a parameter 0 k: Ä Introduction

k = 0 k = 2 Discrete Problem 20 20 40 40 Torus Geometry 60 60 Reconstruction 80 80 Algorithms 100 100 120 120 Numerical 20 40 60 80 100 120 20 40 60 80 100 120 Examples

k = 4 k = 6 Other Conditions 20 20 40 40 History of CDI 60 60 Bibliography 80 80 100 100

120 120 20 40 60 80 100 120 20 40 60 80 100 120 Support Conditions

The Geometry x of the Phase As noted above, we quantify our knowledge of the support of in Retrieval p S p Problem terms of -pixel neighborhoods. Let p be the -pixel Charles L. neighborhood of Sx: This ﬁgure shows S1;S2;S3;S4 for the Epstein k 1 case in the previous slide. D Introduction l = 1 l = 2 Discrete Problem 20 20 40 40

Torus Geometry 60 60 Reconstruction 80 80 Algorithms 100 100

Numerical 120 120 Examples 20 40 60 80 100 120 20 40 60 80 100 120

Other l = 3 l = 4 Conditions 20 20

History of CDI 40 40 Bibliography 60 60 80 80

100 100

120 120 20 40 60 80 100 120 20 40 60 80 100 120 Table of Dimensions

The Geometry of the Phase Retrieval In the table below we show the dimensions of the “numerical” Problem T B k p: Charles L. intersections xAa Sp for various values of and These \ 7 Epstein are subspaces where the angle is less than 10 radians. The ambient dimension is 4096, and dim Ty Aa 2046: Introduction D Discrete Problem diff\supp p 1 p 2 p 3 p 4 Torus Geometry D D D D k 0 4 12 24 40 Reconstruction D Algorithms k 1 12 24 40 60 D Numerical k 2 48 72 100 132 Examples D Other k 3 96 128 164 204 Conditions D History of CDI In the table the dimensions are always positive, indeed it is very Bibliography rare for this dimension to be zero! That is: the intersection Aa BS is not usually transversal. \ Numerical Versus Exact Intersections

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction For the smoother examples (k > 0) it is not clear if the Discrete Problem dimensions of the exact intersections are increasing, or if there are Torus Geometry subspaces, whose dimension increases with k; where BS and Reconstruction Ty Aa make a very shallow angle. Indeed, it seems likely that the Algorithms latter possibility is the correct interpretation. Numerical Examples

Other Conditions

History of CDI

Bibliography Why Non-transversality Matters, I

The Geometry of the Phase Retrieval Problem Charles L. Epstein Non-transversality renders the problem of ﬁnding points in Aa BS ill-conditioned in a neighborhood of y Suppose that Introduction \ W Discrete y Aa BS and v is a unit vector belonging to Ty Aa BS : Problem 2 \ \ Because this direction is tangent to both submanifolds it is clear Torus Geometry that, y tv BS and for small t; Reconstruction C 2 Algorithms 2 Numerical dist.y tv; Aa/ t Examples C / (18) dist.y tv; Aa B/ t: Other C \ Conditions

History of CDI

Bibliography Why Non-transversality Matters, II

The Geometry of the Phase Retrieval Problem Charles L. This shows that if our numerical computations are accurate to Epstein Mach; then the set of vectors Introduction

Discrete y tv t pMach and v Ty Aa Ty BS with v 1 Problem f C W j j Ä 2 \ k k D g Torus Geometry (19)

Reconstruction all lie in Aa BS to accuracy Mach: Algorithms \

Numerical Because these intersections are non-transversal, at machine Examples precision, there will be many equally valid “solutions,” which Other Conditions actually lie quite far from a true intersection point. History of CDI This is entirely independent of the algorithm. Bibliography The Lipschitz Lower Bound, I

The Geometry of the Phase As noted above, the map BS is always locally invertible on Retrieval M Problem its range. But note: Charles L. Epstein Theorem Suppose that x satisﬁes xj 0; for all j J; and Sx S R; Introduction O ¤ 2 with R at most half the size of J: There exists > 0 and a ﬁnite C Discrete Problem so that Torus Geometry x y 2 C .x/ .y/ 2 (20) Reconstruction k k Ä kM M k Algorithms for y BS ; with x y 2 < ; if and only if Numerical 2 k k Examples TxAa BS x : (21) Other Conditions \ D f g

History of CDI That is the local inverse is Lipschitz if and only if the intersection Bibliography Aa BS is transversal at x: The “if” part follows from the \ observations on the previous two slides. The “only if” is a little harder to prove. The Lipschitz Lower Bound, II

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction The failure to have a Lipschitz lower bound shows that the local Discrete inverse of cannot be Lipschitz continuous. This means that the Problem M solution to this problem can be expected to have great sensitivity Torus Geometry

Reconstruction to noise and measurement errors. These geometric considerations Algorithms also have a signiﬁcant impact on the performance of standard Numerical Examples algorithms.

Other Conditions

History of CDI

Bibliography Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography Alternating Projection, I

The Geometry of the Phase Retrieval Problem The most classical approach to ﬁnding points in A BS is called Charles L. \ Epstein alternating projection. In Functional Analysis it was introduced

Introduction by von Neumann, and, in image reconstruction, by Gerchberg and J Discrete Saxton, see [2]. We let PBs R BS be the orthogonal Problem J W ! projection, and PA R A be the closest point map. This map Torus Geometry W ! is deﬁned on the complement of a union of codimension 2 linear Reconstruction Algorithms subspaces. Numerical Examples The idea for alternating projection is very simple. We choose a i Other random collection of phases e j j bJ and use the inverse Conditions f .0/ W 21 g ij DFT to ﬁnd the initial image x .Œaj e /; which is a History of CDI D F Bibliography random point on the magnitude torus. Alternating Projection, II

The Geometry of the Phase Retrieval Problem Charles L. Epstein We define the alternating projection sequence Introduction .n 1/ .n/ Discrete x P PB .x /: (22) Problem C D A ı s Torus Geometry It is clear that any point x A BS is a fixed point of this Reconstruction 2 \ Algorithms iteration, and it was hoped for many years that these iterates Numerical Examples would converge to such a fixed point. In fact AP has many

Other attracting ﬁxed points that do not come from intersection points. Conditions

History of CDI

Bibliography What Alternating Projection Really Does

The Geometry of the Phase Retrieval Problem Charles L. These additional ﬁxed points are local minima of the map from Epstein A BS Œ0; / deﬁned by ! 1 Introduction

Discrete dAB .x; y/ x y 2 (The Euclidean distance): (23) Problem D k k Torus Geometry A priori, it is not obvious that any such points exist, but this is Reconstruction Algorithms quickly clariﬁed by numerical experiments.

Numerical Examples The next slide shows the results of running AP for 10,000 iterates

Other with 25 different randomly chosen starting points. The Conditions 256 256-image is piecewise constant, with a support History of CDI neighborhood of 3-pixels. Bibliography Experiments with AP

The Geometry of the Phase Retrieval Problem

Charles L. 50 Epstein

100 Introduction

Discrete 150 Problem

200 Torus Geometry

Reconstruction 250 Algorithms 50 100 150 200 250

Numerical Examples (a) The target for the ﬂow.

Other Figure: In (b), blue is the log10 of the distance to A BS3 ; the cyan Conditions .n/ \ .n/ curves are plots of the “residual:” log10 x PBS .x / ; and red is History of CDI k 3 k the distance between successive iterates of the algorithm, Bibliography log x.n/ x.n 1/ : 10 k C k What AP Tells Us about Geometry

The Geometry of the Phase Retrieval Problem On the previous slide we saw the results of starting the Charles L. AP-algorithm at 25 random initial point on the magnitude torus. It Epstein is quite apparent, from the behavior of the red curves, which show Introduction log x.n/ x.n 1/ ; that the iterates of the AP-algorithms 10 k C k Discrete converge. Indeed the sum P x.n/ x.n 1/ < : Problem n k C k 1 Torus Geometry The limit points, however, do not lie on A BS : A careful Reconstruction \ Algorithms examination of these limits shows that they are all different, which Numerical gives strong evidence that dAB has many local minima, where Examples dAB .xA; yB / 0: Other ¤ Conditions Contrary to what is stated in the many places, Alternating History of CDI Projection does not stagnate, it just converges to limit points that Bibliography are far from A BS : \ Stagnation versus Convergence

The Geometry of the Phase Retrieval Problem We use these terms a little differently from most people working Charles L. in this ﬁeld. Epstein Deﬁnition Introduction .n/ A sequence of points x converges to x if Discrete .n/ f g Problem limn x x 0: Torus Geometry !1 k k D

Reconstruction Algorithms Deﬁnition .n/ Numerical A sequence of points x stagnates if the sequence remains in a Examples f g .n/ .n 1/ bounded set D; but the successive differences x x C do Other k k Conditions not converge to zero. History of CDI

Bibliography These aren’t all the possibilities, but these are the outcomes we have observed, experimentally, in the phase retrieval problem. Difference Maps, I

The Geometry of the Phase Retrieval Problem Charles L. It has been known for a long time that algorithms based on the Epstein AP-map do not work very well. A different class of maps, which Introduction we call difference maps, were introduced to correct this problem. Discrete The ﬁrst such algorithms were proposed by Fienup, using a map Problem

Torus Geometry given by

Reconstruction Algorithms ˇ DBA x x PB Œ.1 ˇ/PA.x/ x ˇPA.x/: (24) Numerical D 7! C C Examples Here ˇ .0; 1: This is call the “hybrid input-output method;” Other 2 Conditions there were later additions by Elser, Miao, etc. See [3], [4], [6], [7]. History of CDI There are many variants, which all behave similarly. Bibliography Difference Maps, II

The Geometry of the Phase Retrieval Problem Charles L. A representative case is given by the map: Epstein

Introduction DAB .x/ x PA RB .x/ PB .x/; (25) Discrete D C ı Problem 1 which, in the notation of the previous slide, is D : Here RB is Torus Geometry AB the “reﬂection around B” deﬁned by Reconstruction Algorithms Numerical RB .x/ 2PB .x/ x: (26) Examples D Other Conditions If B is a linear subspace, then RB is just the usual orthogonal

History of CDI reﬂection with ﬁxed point set equal to B:

Bibliography The Fixed Point Set for Difference Maps

The Geometry of the Phase Retrieval Problem A easy calculation shows that x is a fixed point for D if and Charles L. AB Epstein only if PA RB .x / PB .x /: (27) Introduction ı D Discrete The fixed points do not necessarily belong to A BS ; but once a Problem \ Torus Geometry fixed point is found, then the point

Reconstruction Algorithms x PA RB .x/ PB .x/ Numerical D ı D Examples

Other automatically does lie on the intersection. Conditions

History of CDI Thus we would like to understand the ﬁxed point sets of

Bibliography difference maps. The Center Manifold, I

The Geometry of the Phase x B ; Retrieval Suppose that A S then we let Problem 2 \ 1 1 Charles L. LA PA .x/ and LB PB .x/: (28) Epstein D D

Introduction The set of fixed points that “find” x consists of Discrete Problem x 1 AB RB .LA/ LB : (29) Torus Geometry C D \ Reconstruction Algorithms We call this the center manifold defined by x: The set Numerical LB BS?; and, at least near enough to x; Examples D Other LB Nx A: Conditions D History of CDI x : Bibliography This is the fiber of the normal bundle to the torus at ˇ ˇ At least near to A B the maps D ;D all have the same \ AB BA center manifolds. The Center Manifold, II

1 The Geometry BS is a linear subspace, and R RB ; acts as the identity map of the Phase B D Retrieval on BS and as Id on B : Hence we easily compute that Problem S? Charles L. x Epstein RB .Nx A B?/: (30) CAB D \ S Introduction N B The set x A S? is an afﬁne space consisting of vectors, Discrete \ Problem x v; where v is orthogonal to the tangent space to the torus at C c Torus Geometry x; and supported in S : Reconstruction Algorithms This means that, in its initial steps, an algorithm based on iterating Numerical a difference map is searching for a neighborhood of a subset of Examples dimension Other Conditions x dim AB dim Nx A BS? History of CDI C D \ Bibliography J dim Tx A dim BS dim Tx A BS ; D j j C \ which is usually about J =4: The dimension of the center j j manifold increases along with dim Tx A BS Š \ Other Invariant Sets for Difference Maps

The Geometry of the Phase Retrieval Problem Charles L. While it is true that the ﬁxed points of difference maps are always Epstein related to points in A Bs; there are other invariant sets that are \ Introduction not. For example, if there exists points x1 A and x2 Bs; with Discrete 2 2 Problem x1 x2; which are critical for dABS ; the distance between A ¤ J and BS ; then x x Nx A B (as afﬁne subspaces of R ) is Torus Geometry C 1 2 D 1 \ S? Reconstruction non-empty. Many such critical points exist. Algorithms

Numerical If x1x2 A Bs ; then x1x2 is an invariant set without a Examples C \ \ D; C ﬁxed point. In examples, we have seen that this sort of set may be Other Conditions dynamically attracting. The map DAB translates by a ﬁxed vector History of CDI along such an invariant set. Bibliography The Linear Case, I

The Geometry of the Phase Retrieval Problem Charles L. Epstein To have somewhat better intuition for the behavior of these

Introduction algorithms near to their ﬁxed point sets it pays to consider how

Discrete they behave if we replace A;BS with arbitrary linear subspaces of Problem RN ; which we denote by A; B; with similar dimensions Torus Geometry Suppose that A B F; and let A0 A F ?;B0 B F ? Reconstruction \ D D \ D \ Algorithms and C .A B/ : It easy to see that C is the center manifold. D C ? Numerical U;V;W;Z A ;B ;F;C Examples If we let denote orthonormal bases for 0 0

Other respectively, then we can represent the action of DAB on Conditions x U x1 V x2 W x3 Zx4: History of CDI D C C C

Bibliography The Linear Case, II J R A0 B0 F C D ˚ ˚ ˚ The Geometry of the Phase We have the formula Retrieval 0Ä 1 Problem 0 1 x1 AH 0 0 Charles L. B x2 C Epstein DAB .x1; x2; x3; x4/ @ 0 Id 0 A B C ; (31) D @ x3 A Introduction 0 0 Id x4 Discrete Problem where H V t U; and Torus Geometry D Â Ã Â t t ÃÂ Ã Reconstruction x1 2H HH x1 Algorithms AH : (32) x2 ! H 0 x2 Numerical Examples One can show that the map AH is a contraction, and therefore Other Conditions n lim D .x1; x2; x3; x4/ .0; 0; x3; x4/: (33) History of CDI n AB D !1 Bibliography That is: transverse to the F C -directions the map is contracting, ˚ but in the F C -directions the map acts as the identity, and is ˚ therefore only neutrally stable. The Linear Case, III

The Geometry of the Phase Retrieval Problem Charles L. For completeness we observe that, in the linear case, the AP-map Epstein in the A0 B0 F C -decomposition is given by ˚ ˚ ˚ Introduction 0 0 0 0 01 Discrete t Problem BHHH 0 0C PB PA B C : (34) Torus Geometry ı D @ 0 0 Id 0A Reconstruction Algorithms 0 0 0 0

Numerical Examples Near enough to intersection points these algorithms should behave Other similarly, but practically speaking iterates of neither the Conditions AP-algorithm, nor difference maps converge to intersection History of CDI

Bibliography points. Linear to Non-linear, I

The Geometry of the Phase Retrieval Problem Charles L. Epstein Even if F 0 ; the analysis above shows that, in the linear case, D f g Introduction the iterates converge to a point on the center manifold, C: Near Discrete A B the map is only neutrally stable in these directions; Problem \ Torus Geometry convergence is therefore a non-linear phenomenon. The rate of

Reconstruction convergence in the remaining directions is determined by the Algorithms singular values of H: These are all less than 1; and equal the Numerical Examples cosines of angles between vectors in A0 and B0: The smaller Other these angles, the slower this iteration converges. Conditions

History of CDI

Bibliography Linear to Non-linear, II

The Geometry of the Phase Retrieval Problem Charles L. Epstein In the case of interest A A; which is non-linear. Under the best Introduction D Discrete of circumstances, the iterates of a difference map converge to a Problem point on the center manifold which is quite distant from A BS : \ Torus Geometry So it’s not clear if this linear analysis is applicable to the Reconstruction Algorithms non-linear case. It is surprisingly informative.

Numerical To get some intuition we consideration a very low dimensional Examples example. Other Conditions

History of CDI

Bibliography Low Dimensional Examples, I

The Geometry of the Phase We consider two low dimensional examples. The ﬁrst is a circle Retrieval A; xy Problem and a line. The circle, is the unit circle in the -plane, and the line B .1; 0; 0/ .0; cos ; sin / : We show two Charles L. D f C g Epstein possibilities below

Introduction

Discrete Problem

Torus Geometry

Reconstruction Algorithms

Numerical Examples

Other Conditions

History of CDI

Bibliography (a) Center manifold is (b) Center manifold is x-axis. xz-plane. Figure: A circle and a line. Low Dimensional Examples, II

The Geometry 10 of the Phase In these examples we run DAB until the error reaches 10 : Retrieval Problem Observe the how the lengths of the trajectories depends on the Charles L. angle. Epstein

Introduction

Discrete Problem

Torus Geometry

Reconstruction Algorithms

Numerical Examples

Other Conditions

History of CDI

Bibliography Low Dimensional Examples, II

The Geometry of the Phase Retrieval Problem The iteration is the following map: Charles L. Epstein 0 1 0 1 x1 1 x1=r x1 C Á Introduction B 2 cos 2 x3 sin 2 C @x2A x2 sin r 2 ; (35) Discrete 7! @ C A Problem x3 2 sin 2 2 x2 r 1 2 x3 cos Torus Geometry C

Reconstruction q 2 2 Algorithms where r x x : The x1-coordinate is monotonically D 1 C 2 Numerical x Examples increasing and converges to a value larger than 1. The 2 coordinate rapidly approaches zero, independently of whereas, Other I Conditions for small ; after the initial iterates, the x3-coordinate is History of CDI 2 dominated by the x3 cos term. This is the linearization at the Bibliography point of intersection. Low Dimensional Examples, IV

The Geometry of the Phase Retrieval Problem We see that the number of iterates required to get 10-digits of Charles L. Epstein accuracy grows quickly as the angle decreases, until the angle reaches 0, when the number drops from 200,000 to 30. The Introduction iterates always converge to a point on the center manifold quite Discrete Problem distant from A B: Nonetheless the linearized analysis at A B \ \ Torus Geometry does a very good job predicting the behavior of the algorithm Reconstruction except when 0: This is essentially because when 0; both Algorithms D ¤ y z Numerical and need to converge to zero, and this process is governed by Examples the linearized analysis at A B: Other \ Conditions When 0; then only y needs to converge to zero, and this is D History of CDI governed by non-linear effects that essentially “choose” a point on Bibliography the center manifold where this coordinate decreases geometrically. Low Dimensional Examples, V

The Geometry of the Phase We now consider the effects of almost intersections, and nearby Retrieval Problem intersections. Trajectories are ordered blue to red. Charles L. Epstein

Introduction

Discrete Problem

Torus Geometry

Reconstruction Algorithms

Numerical Examples

Other Conditions

History of CDI

Bibliography

Near miss. Non-trans. int. 2 nearby ints. Trans. int. Low Dimensional Examples, VI

The Geometry of the Phase The first images shows a near miss. There is a center Retrieval Problem manifold defined by the closest approach of A to B, which is Charles L. attracting, and sends the trajectories spiraling out to infinity. Epstein The second example is a tangent intersection. Once again, Introduction the single coordinate that needs to converge to zero is Discrete Problem governed by a non-linear process that finds a point on the Torus Geometry 2-dimensional center manifold near which this coordinate Reconstruction goes quickly to zero. Algorithms

Numerical The third example is the result of two very nearby Examples intersection points. The center manifold deﬁned by the local Other Conditions maximum of the dAB between these points is attracting and, History of CDI again, translates the iterates out to inﬁnity. Bibliography Finally a transversal intersection is attracting, but the angle between the two tangent spaces is very small, leading to slow spiraling convergence. Low Dimensional Examples, VI

The Geometry of the Phase Retrieval Problem Despite the fact that the iterates converge to points on center Charles L. manifolds far from A B; the linearized analysis of the Epstein \ iteration in the non-common tangent directions remains Introduction qualitatively correct. Discrete Problem Critical points of the function dAB deﬁne generalized center Torus Geometry manifolds that can be attracting sets for the difference map. Reconstruction Algorithms These do not have to be local minima. Numerical Convergence in the common tangent directions is non-linear, Examples can Other and governed by the location on the center manifold. It Conditions occur very rapidly. History of CDI Difference maps have attracting basins that are not deﬁned Bibliography by intersection points. Lessons from Low Dimensional Examples

The Geometry of the Phase Retrieval Problem The linearization of the torus is the tangent space at x; denoted Charles L. T : Epstein x A As we have seen, it is often the case that dim Tx A BS > 0 this subspace is the analogue of the Introduction \ I F -directions above. Hence there are directions transverse to the Discrete Problem center manifold at A BS where the difference map is only \ Torus Geometry neutrally stable. Convergence in these directions is a non-linear Reconstruction Algorithms phenomenon. Numerical There can also be large dimensional subspaces where these vector Examples spaces make very small, but non-zero angles. The behavior of the Other Conditions iterations in these directions is accurately predicted by the linear History of CDI analysis at exact intersection points, even though these points are Bibliography quite far from A BS : \ What Low-d Examples Tell Us, I

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction The low dimensional examples suggest the sorts of phenomena Discrete we should expect to see in the phase retrieval problem. These Problem examples certainly suggest that small, non-zero angles between Torus Geometry T B Reconstruction subspaces of x A and S can lead to very slow convergence, or Algorithms even stagnation, far from the center manifolds deﬁned by points in Numerical A BS : Examples \ Other Conditions

History of CDI

Bibliography What Low-d Examples Tell Us, II

The Geometry of the Phase T B Retrieval When there are subspaces x A and S that make very small, Problem non-zero angles, the iteration of a difference map displays two Charles L. Epstein phases:

Introduction 1 In the ﬁrst phase the maps behave as if the small angles were

Discrete in fact zero. The interates converge toward a Problem higher-dimensional pseudo-center manifold, but not quite Torus Geometry what would result from these angles being zero. The Reconstruction Algorithms reconstruction error remains “large.” Numerical 2 Examples In the second phase the coefﬁcients from the directions that

Other make very small non-zero angles might tend very slowly to Conditions zero, as predicted by the linearization at the exact History of CDI intersection point. Bibliography Because of the existence of multiple nearby attracting basins other more complicated things also happen. Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography How Do We Measure Success?

The Geometry of the Phase Retrieval Problem Charles L. Epstein In the real phase retrieval there is only one reasonable measure of Introduction success; it is the data error: Discrete Problem .n/ .n/ P .x / PB .x / : (36) Torus Geometry k A S k Reconstruction Algorithms Once this falls below a certain threshold then one has, to all Numerical Examples intents and purposes, solved the problem. But do solutions in this

Other sense correspond to accurate reconstructions? Conditions

History of CDI

Bibliography Guide to the Plots

The Geometry of the Phase Retrieval In the plots below, the red curves are the log10 of the residuals Problem Charles L. .n/ .n/ .n 1/ .n/ PB .x / P RB .x / x x Epstein k Sp A ı Sp k D k C kI Introduction .n/ .n/ The point r PBSp .x / is our best guess for the Discrete D Problem reconstruction, given x.n/ this residual gives an upper bound for I Torus Geometry the data error Reconstruction Algorithms P .x.n// P R .x.n// .r.n// a : Numerical BSp A BSp (37) Examples k ı k kM k Other x.v/ r.n/; Conditions If is the trivial associate of the target image closest to

History of CDI then the blue curves show the log10 of the distance

Bibliography r.n/ x.v/ : (38) k k The Effect of the Support Condition

The Geometry In this slide we show the effect on the behavior of a difference of the Phase .n/ Retrieval dist.r ; A BSp /; indicating approach along a common tangent direction. Once algorithm of\ loosening the support condition.n/ with a piecewise Problem this occurs, there is very little further change in dist.r ; A BS /: \ p Charles L. constant image. N = 128, s-nbdh = 0, shift = [0, 0] N = 128, s-nbdh = 1, shift = [-1, 0] N = 128, s-nbdh = 2, shift = [-1, 0] pos = 0, supp = 1, k = 0, ALG= ALG B IO pos = 0, supp = 1, k = 0, ALG= ALG B IO A H pos = 0, supp = 1, k = 0, ALG= ALG B IO A H 1 A H 1 Epstein 10 101 10

100 100

100 10-1 Introduction 10-1 10-2 -1 -2 10 10 -3 Discrete 10 -4 10-3 10 Problem 10-2 10-5 10-4

10-6 10-3 Torus Geometry 10-5 10-7

-6 -8 10 10 10-4 Reconstruction 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Algorithms D p 0: D p 1: D p 2: (a) ABS0 with (b) ABS1 with (c) ABS2 with N = 128, s-nbdh = 3, shift = [-1, 0]D N = 128, s-nbdh = 4, shift = [-3, -2] D N = 128, s-nbdh = 5, shift = [-3, -3] D pos = 0, supp = 1, k = 0, ALG= ALG B IO pos = 0, supp = 1, k = 0, ALG= ALG B IO pos = 0, supp = 1, k = 0, ALG= ALG B IO A H A H 1 A H Numerical 10 101 101 Examples

100 0 0 Other 10 10 Conditions 10-1 10-1 10-1 History of CDI 10-2

10-2 10-2 Bibliography 10-3

-3 -3 10-4 10 10 0 500 1000 1500 2000 2500 3000 3500 4000 4500 50000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

(d) D B with p 3: (e) D B with p 4: (f) D B with p 5: A S3 D A S4 D A S5 D

Figure 10: The dependence of the geometry near intersection points in A BS \ p on the size of support neighborhood is revealed by the behavior of iterates under a difference algorithm. The images are 256 256: When p 0; 1 the trajectory ⇥ D does not appear to lie along a common tangent direction. If p 2; 3; 4; 5 then D almost as soon as the trajectory falls into the attracting basin it follows a tangent direction.

Example 6. These examples shown in Figure 11 are similar to the previous ones except that we vary the smoothness for a 1-pixel support neighborhood. The red and blue curves are as in example 5. As before, by comparing the dist.r.n/; A .n/ .n 1/ \ BS / to x x we can easily discern if the trajectory is approaching along 1 k k a path close to a direction in Tx A TBS ; or if the path is essentially transversal ⇤⇤ \ 1 to both. As we see in the plots below, this depends on the smoothness of the example. It is quite apparent in the trajectories corresponding to k 1; 2; 3; 4; 5 D that x.n/ x.n 1/ is very close to the square, or higher power, of dist.r.n/; A k k \ BS1 /; indicating approach along a common tangent direction. Once this occurs, .n/ there is again very little further reduction in dist.r ; A BS /: \ 1

46 The Effect of Less Precise Support Information

The Geometry .n/ of the Phase Once the support condition reaches p 2; the iterates, x of Retrieval D f g Problem DABSp stagnate and approach the center manifold so that the Charles L. reconstructions r.n/ are approaching x along a trajectory Epstein f g T B : lying largely in x A Sp Examining exactly which trivial Introduction \ associate the r.n/ are approaching, we can see in each of these Discrete f g Problem cases the dimension of this intersection is positive. In the p 3 D Torus Geometry case we see that Reconstruction Algorithms .n/ .n/ .n/ .v/ m PBSp .x / PA RBSp .x / r x ; (39) Numerical k ı k / k k Examples

Other for an 2 < m: This is not because the iterates have found a Conditions .v/ direction in BSp that makes higher order contact with A at x ; History of CDI but rather because Bibliography

dim T .v/ A BS dim N .v/ A BS : (40) x \ x \ and the surrounding discussion. If the target image is piecewise constant, then the theory in Section 3.1 and numerical examples in Example 1 suggest that only

the exact intersection between Tx0 A and BS produces small angles, moreover the sym dim Tx A BS equals the cardinality of the set : This in turn should govern 0 \ Tx0;S the conditioning of the problem near this intersection point. In this example we examine the trajectories of the iterates of the difference algorithm based on the map D B for a single 256 256; piecewise-constant image A S2 ⇥ with two different random starting points. We let x0 denote the reference image centered in the 2-pixel neighborhood S2: In Figure 13[a] the iterates are in the at- .2;2/ sym tracting basin deﬁned by the x0 : At this translate we have that .2;2/ 0 jTx0 ;S2 jD and so Lemma 3 implies that

dim Tx.2;2/ A BS2 0: (121) 0 \ D The iterates are clearly converging geometrically to the center manifold deﬁned by this exact intersection point. In Figure 13[b] the iterates are in the attracting .1; 1/ sym basin deﬁned by the x0 : At this translate we have that .1; 1/ 4 and jTx0 ;S2 jD so Lemma 3 implies that

dim Tx.1; 1/ A BS 4: (122) 0 \ D The iterates have clearly stagnated, and the trajectory appears to lie along directions

Twoin Tx Starting.1; 1/ A BS2 : Points 0 \

it = 1 it = 5000 it = 1 it = 5000 400 12000 400 800

10000 300 300 600 The Geometry 8000 of the Phase 200 6000 200 400 In this slide4000 we show 5000 iterates from two runs of DABS on a Retrieval 100 100 200 2 Problem 2000 0 0 0 0 piecewise-1 0 constant1 -5 0 image.5 -1 0 1 -0.1 0 0.1 #10-7 Charles L. seed = 121, N = 128, s-nbdh = 2, seed = 121, N = 128, s-nbdh = 2, pos = 0, supp = 1, k = 0, bkgd = 0, s/n = Inf, noise-model = 1 pos = 0, supp = 1, k = 0, bkgd = 0, s/n = Inf, noise-model = 1 2 Epstein 105 10 Residuals Residuals Errors Errors

0 100 10 Introduction 10-5 10-2 Discrete 10-10 10-4 Problem 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

Torus Geometry D D (a) Errors and residuals for ABS2 with (b) Errors and residuals for ABS2 with k 0; p 2; where the iterates lie in k 0; p 2; where the iterates lie in Reconstruction D D D D Algorithms an attracting basin with dim Tx.2;2/ A an attracting basin with dim Tx.1; 1/ A 0 \ 0 \ BS 0: BS 4: Numerical 2 D 2 D Examples Figure 13: An illustration of how the convergence properties of the difference al- Other D ; T .v/ B : Conditions gorithm ABS2 depend on the dimension of the dim x A S2 The plot on the left shows geometric convergence,0 \ that on the History of CDI right shows either very slow convergence, or stagnation. In the Bibliography 49 phase retrieval problem, non-transversality alone can lead to very slow convergence. The Effect of Smoothness

The Geometry of the Phase In this slide we show the effects of varying the level of Retrieval Problem smoothness of the image on the behavior of a difference Charles L. algorithm. In all cases we use a 1-pixel support neighborhood. seed = 20, N = 128, s-nbdh = 1, seed = 20, N = 128, s-nbdh = 1, seed = 20, N = 128, s-nbdh = 1, pos = 0, supp = 1, k = 0, ALG= ALG B IO pos = 0, supp = 1, k = 2, ALG= ALG B IO A H pos = 0, supp = 1, k = 1, ALG= ALG B IO A H 1 A H 1 Epstein 10 101 10 Residuals Residuals Residuals Errors Errors Errors

100

100 100 Introduction 10-1 10-1 10-1 10-2

Discrete 10-3 10-2 10-2 Problem 10-4

-3 10 10-3 Torus Geometry 10-5

-4 -4 10-6 10 10 0 500 1000 1500 2000 2500 3000 3500 4000 0 500 1000 1500 2000 2500 3000 3500 4000 0 500 1000 1500 2000 2500 3000 3500 4000 Reconstruction D k D k D k Algorithms (a) ABS1 with (b) ABS1 with (c) ABS1 with 0; p 1: D 1; p 1: D 2;p 1: D

D seed = 20, N = 128, s-nbdh = 1, D seed = 20, N = 128, s-nbdh = 1, D seed = 20, N = 128, s-nbdh = 1, Numerical pos = 0, supp = 1, k = 3, ALG= ALG B IO pos = 0, supp = 1, k = 5, ALG= ALG B IO 1 A H pos = 0, supp = 1, k = 4, ALG= ALG B IO A H 10 1 A H 1 10 10 Residuals Residuals Residuals Errors Errors Examples Errors

0 0 10 100 10 Other

-1 -1 Conditions 10 10-1 10

-2 -2 History of CDI 10 10-2 10

-3 -3 Bibliography 10 10 10-3

-4 -4 10 10 10-4 0 500 1000 1500 2000 2500 3000 3500 4000 0 500 1000 1500 2000 2500 3000 3500 4000 0 500 1000 1500 2000 2500 3000 3500 4000 D k D k D k (d) ABS1 with (e) ABS1 with (f) ABS1 with 3; p 1: D 4;p 1: D 5; p 1: D D D D

Figure 11: The dependence of the geometry near intersection points in A BS \ 1 on the size of support neighborhood is revealed by the behavior of iterates under a difference algorithm. The image is 256 256; and we use a 1-pixel support ⇥ neighborhood. When k 0 the trajectory does not appear to lie along a common D tangent direction, If k 1; 2; 3; 4; 5 then almost as soon as the trajectory falls into D the attracting basin it follows a tangent direction.

Example 7. Figure 12 shows the behavior of algorithms based on DABSp ; for sev- eral different values of k and p: For each example we show the error trajectories for 100 random initial conditions. The red and blue curves depict the same quantities as in the previous two examples. In Figure 12[a], with k 0; p 1; both the red and blue curves are converging D D linearly to zero. In this case the intersection between A and BS1 is very nearly transversal. In Figure 12[b], with k 0; p 3; some of the trajectories are D D converging linearly to zero, whereas others display the stagnation seen in all the trajectories in Figures 12[c,d], where k 4;p 1; 3: The fact that the same D D iteration, with different starting points, can produce trajectories indicating either convergence, or stagnation is a reﬂection of the differences in the geometry among different points in A BS ; see Example 8. \ p When the iterates stagnate we see that the values along the red curves are, at

47 What this Slide Shows

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction Once the image is no longer piecewise constant, i.e. k 1; the .n/ Discrete iterates, x of DABS stagnate and approach the center Problem f g 1 manifold so that the reconstructions r.n/ are approaching x Torus Geometry f g along a trajectory lying largely in Tx A BS ; and the large Reconstruction \ 1 Algorithms dimensional subspace of the tangent space that makes a very small Numerical Examples angle with BS1 :

Other Conditions

History of CDI

Bibliography it = 1 it = 10000 it = 25000 it = 50000 it = 100000 it = 150000 it = 200000 it = 250000 400 2500 2000 1500 2000 4000 2000 1500

2000 300 1500 1500 3000 1500 1000 1000 1500 200 1000 1000 2000 1000 1000 500 500 100 500 500 1000 500 500

0 0 0 0 0 0 0 0 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 seed = 121, N = 128, s-nbdh = 1, pos = 0, supp = 1, k = 2, bkgd = 0, s/n = Inf, noise-model = 1 105 Residuals Errors 100

Stagnation10-5

10-10 0 0.5 1 1.5 2 2.5 105 The Geometry of the Phase (a) Retrieval it = 1 it = 10000 it = 25000 it = 50000 it = 100000 it = 150000 it = 200000 it = 250000 Problem 400 1000 4000 6000 4000 5000 6000 6000 In this slide800 we show 250,000 iterates of D4000 on a Charles L. 300 3000 3000 ABS1 4000 4000 4000 Epstein 600 3000 256 200 256-image with2000 p 1; k 20002: It seems very plausible that 400 2000 D2000 D 2000 2000 100 1000 1000 Introduction this iteration200 has stagnated! 1000 0 0 0 0 0 0 0 0 Discrete -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 Problem seed = 111, N = 128, s-nbdh = 1, pos = 0, supp = 1, k = 2, bkgd = 0, s/n = Inf, noise-model = 1 105 Torus Geometry Residuals Errors 0 Reconstruction 10 Algorithms 10-5 Numerical Examples 10-10 0 0.5 1 1.5 2 2.5 5 Other #10 Conditions History of CDI (b) Bibliography it = 1 it# =10 100004 it# =10 250004 it = 50000 it = 100000 it = 150000 it = 200000 it = 250000 400 4 2 6000 800 800 800 800

300 3 1.5 600 600 600 600 4000 200 2 1 400 400 400 400 2000 100 1 0.5 200 200 200 200

0 0 0 0 0 0 0 0 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 seed = 118, N = 128, s-nbdh = 1, pos = 0, supp = 1, k = 2, bkgd = 0, s/n = Inf, noise-model = 1 105 Residuals Errors 100

10-5

10-10 0 0.5 1 1.5 2 2.5 #105

(c)

D Figure 15: The behavior of 250,000 iterates of the algorithm based on ABS1 on a 256 256-image showing histograms of the phase error. ⇥

53 What is Stagnation?

The Geometry of the Phase Retrieval It is a remarkable fact that the iterates of these algorithms fall into Problem Charles L. attracting basins in which they remain, without converging for a Epstein very large number of iterates, e.g. 5 106: In fact the sizes of the successive differences, x.n 1/ x.n/ ; are essentially constant, Introduction k C k Discrete and large enough that these iterates could easily have wandered Problem far out of the attracting basin. But they don’t! Torus Geometry .n 1/ .n/ Reconstruction The bulk of the differences x C x lie in the Algorithms F C -directions deﬁned by x0; and in directions where Tx0 A Numerical ˚ Examples and BS make a very small angle. It may be that the iterates are Other trapped in some sort of “strange attractor” lying in a small Conditions neighborhood of x0 : History of CDI CABS Bibliography We examine this possibility by looking at low-dimensional projections of these trajectories. A Low dimensional Projection, I

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction

Discrete Problem

Torus Geometry

Reconstruction Algorithms

Numerical Examples (a) 3-principal components (b) 3-principal components Other of iterates 1,500,000 to of reconstructions 1,500,000 Conditions 6,500,000. to 6,500,000. History of CDI Bibliography Figure: An example with k 0; p 3; with closest exact intersection D D F Œ0;0: N0 A Low dimensional Projection, II

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction

Discrete Problem

Torus Geometry

Reconstruction Algorithms

Numerical Examples (b) 3-principal components Other of reconstructions 1,500,000 Conditions (a) 3-principal components of iterates 1,500,000 to to 6,500,000. History of CDI 6,500,000. Bibliography Figure: Example with k 2; p 1; with closest exact intersection D D F Œ0;0: N0 Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography Uniqueness and the Positivity Condition

The Geometry of the Phase Retrieval There are circumstances where it is reasonable to assume that the Problem unknown image is non-negative. Clearly non-negativity alone is Charles L. Epstein not a strong enough constraint to imply uniqueness up to trivial associates. The measured data xj sufﬁces to determine the Introduction fj O jg Discrete autocorrelation image Problem X Torus Geometry x ? xj xkxk j : (41) D Reconstruction k Algorithms Numerical 2 Examples This is because x ? xj xj : We have shown that if Sx?x is j j D j O j Other small enough and x is known to be non-negative, then we can Conditions prove that Sx is also small. Without the sign constraint this is History of CDI false. We can then conclude that the set of images with the given Bibliography data .x/ is ﬁnite and generically consists of trivial associates of x: M 1 Using the Positivity Constraint

The Geometry of the Phase Retrieval Problem Charles L. If we set B x 0 xj for j J ; then we can looks for Epstein C D f W Ä 2 g points in the intersection Aa B : Assuming that Sx?x is small Introduction \ C enough, there will be finitely many of these points on the @B : Discrete C Problem The boundary of the orthant is not a smooth manifold, but is Torus Geometry instead a stratified space, stratified by the number of vanishing Reconstruction coordinates. In general, the intersections we seek lie in a stratum Algorithms of high codimension. These strata are much more convex than a Numerical Examples linear subspace, so it is reasonable to expect that the intersections Other Conditions have a better chance of being transversal. But what do we mean

History of CDI by that?

Bibliography Transversality of the Positivity Constraint

The Geometry @B of the Phase While is not smooth, it is a union of orthants in Euclidean Retrieval C Problem spaces, so it is piecewise linear. With that in mind, it makes sense Charles L. to measure transversality of intersections by examining Epstein TxAa @B : (42) Introduction \ C

Discrete Problem To analyze this intersection it turns out to be useful to study the Torus Geometry ` T : Reconstruction 1-norm as a function on xAa Algorithms If x is a non-negative image, then x 1 x0; moreover the Numerical k k D O Examples `1-norm on Aa assumes its minimum value at images that are 1 Other either non-negative or non-positive. If B is the `1-ball with Conditions r radius r x 1; then History of CDI D k k 1 Bibliography Aa B Aa Br : (43) \ C D \ 1 There is a fast algorithm for projection onto @Br and so this leads to different algorithms for ﬁnding these intersections. The `1-norm on TxAa

The Geometry of the Phase Retrieval Problem The understand the transversality of the intersections in (43) we Charles L. need to study the `1-norm on TxAa; where x is a non-negative Epstein image. Introduction The `1-norm always assumes its minimum value at x TxAa; 2 Discrete however, this minimum may not be strict. If it is not, then there is Problem a convex set where the `1-norm assumes the value x 1 in Torus Geometry W k k I Reconstruction fact: Algorithms 1 TxAa @B TxAa Br : (44) Numerical W D \ C D \ Examples In light of this, algorithms based on PB can be expected to work Other C P 1 : Conditions about as well as those based on Br Experimentally this is true, History of CDI but, being non-linear, individual runs can be quite different. As

Bibliography expected, the dim grows much more slowly with the W smoothness of x than dim TxAa BS : \ Suggestions for Improvement

The Geometry of the Phase Retrieval Problem This analysis suggests that finding methods to more effectively Charles L. recover the unmeasured phase information will require a different Epstein experimental protocol. Simply changing the algorithm is unlikely Introduction to produce substantial improvements. We have considered two Discrete Problem possible modifications that lead to better reconstructions. The first

Torus Geometry involves cutting a soft object off along a known hard edge. The

Reconstruction second involves adding a known hard object exterior to the object Algorithms we are trying to image; we call this external holography. Numerical Examples It is important to have accurate information about the shape of the Other edge along which the object is cut, or of the hard external object. Conditions This is needed to break the inﬁnitesimal symmetry that leads to History of CDI non-transversal intersections. In the next two slides we show Bibliography examples of these approaches indicating how well they work. Using a Sharp Cut-off

The Geometry of the Phase Soft edge

Retrieval 2000 Problem 10 10 1000 Charles L. 20 20

Epstein 30 30 0 40 40 -16 -14 -12 -10 -8 -6 -4 -2 0

Introduction 50 50 Circular hard edge

Discrete 60 60 2000 Problem 10 20 30 40 50 60 20 40 60 (a) The image 1000 Torus Geometry in [a] cut-off 0 Reconstruction -16 -14 -12 -10 -8 -6 -4 -2 0 Algorithms along a circular Asymmetric hard edge (b) Histograms comparing log -errors with Numerical edge. 2000 10 Examples and without a hard-edge. 1000 Other

Conditions Figure: k 4-image cut-off0 along a circular edge, known to 1-pixel. D -16 -14 -12 -10 -8 -6 -4 -2 0 History of CDI Bibliography The errors are computing by randomly choosing 400 runs out of a pool of 20,000 trials, and then using the minimum residual image as the reconstruction. External Holography

The Geometry of the Phase Supp-mask Ref-Image Recon Retrieval Problem 50 50 50

Charles L. 100 100 100 Epstein 150 150 150

Introduction 200 200 200

Discrete 250 250 250 Problem 50 100 150 200 250 50 100 150 200 250 50 100 150 200 250

Torus Geometry Errors and Data-errors 102 Reconstruction Algorithms 100 Numerical -2 Examples 10

Other 10-4 Conditions 10-6 History of CDI 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 104 Bibliography D Figure: Improvement of convergence properties of AaBS1 by addition of a hard external object. The original object is k 6; and the shape of D the external object is known to 1-pixel. Thanks!

The Geometry of the Phase Retrieval Problem Charles L. Epstein

Introduction Thanks for your attention! Discrete And thanks to Flatiron Institute of the Simons Foundation for Problem supporting this research. Torus Geometry

Reconstruction Algorithms

Numerical Examples

Other Conditions

History of CDI

Bibliography transparent Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography transparent Short History of Coherent Diffraction Imaging

The Geometry of the Phase This slide courtesy Malcolm Howells, ESRF Retrieval http://www.esrf.eu/ﬁles/live/sites/www/ﬁles/events/conferences/Tutorials/slideslecture7.pdf Problem Charles L. Epstein COHERENT X-RAY DIFFRACTION IMAGING: HISTORY Introduction

Discrete Sayre (1952) - Fundamentals of sampling the wave amplitude and wave intensity Problem Gerchberg and Saxton (1972) - First phase-retrieval algorithm successful on test data Torus Geometry Sayre (1980) - Idea to do "crystallography" with non-periodic objects (i. e. attempt phase retrieval) and exploit the cross-section advantage of soft x-rays Reconstruction Algorithms Sayre, Yun, Chapman, Miao, Kirz (1980’s and 1990’s) - development of the experimental technique

Numerical Fienup (1978-) - Development of practical phase-retrieval algorithms including use of the combination Examples of support constraint and oversampling of the amplitude pattern Miao, Charalambous, Kirz and Sayre (1999) - ﬁrst demonstration of 2-D CXDM using a Fienup-style Other algorithm at 0.73 keV x-ray energy, 75 nm resolution Conditions Miao et al (2000) - imaging of a ﬁxed biological sample in 2-D at 30 nm resolution History of CDI Miao et al (2001) - improved resolution in 2-D: 7 nm Bibliography achievement of 3-D with moderate resolution: 55 nm Robinson et al (2001-3) - Application to microcrystals and defects - 3D reconstruction - hardest x-rays ALS group 2002-5 - reconstruction without use of other micoscopes - 3D reconstructions with many (up to 280) views and 10 nm resolution

ESRF Lecture Series on Coherent X-rays and their Applications, Lecture 7, Malcolm Howells transparent Outline

The Geometry of the Phase Retrieval Problem 1 Introduction Charles L. Epstein 2 The Discrete Phase Retrieval Problem

Introduction 3 Geometry of the Torus Discrete Problem

Torus Geometry 4 Reconstruction Algorithms

Reconstruction Algorithms 5 Numerical Examples in Phase Retrieval Numerical Examples 6 Other Auxiliary Conditions Other Conditions

History of CDI 7 Short History of CDI

Bibliography 8 Bibliography transparent Some References

The Geometry of the Phase Retrieval Problem 1 D.SAYRE, Some implications of a theorem due to Shannon, Acta Crystallogr., vol. 5, p. 843, 1952.

Charles L. 2 R. W. GERCHBERGAND W. O. SAXTON, Practical algorithm for determination of phase from image and Epstein diffraction plane pictures, Optik, vol. 35, pp. 237-246, 1972.

3 J.R.FIENUP, Reconstruction of an object from the modulus of its Fourier transform, Opt. Lett, vol. 3, pp. Introduction 27-29, 1978.

Discrete 4 J.R.FIENUP, Phase retrieval algorithms: a comparison, 1982, Applied Optics. 21 (15): 2758âA¸S2769.˘ Problem 5 M.H.HAYES, The reconstruction of a multidimensional sequence from the phase or magnitude of its Fourier Torus Geometry transform, IEEE Trans. Accoust., Speech Signal Process., vol. ASP-30, no. 2, pp. 140-154, Apr. 1982.

Reconstruction 6 V. ELSER, Solution of the crystallographic phase problem by iterated projections, Acta Crystallogr. A, vol. 59, Algorithms pp. 201-209, 2003.

Numerical 7 JIANWEI MIAO,RICHARD L.SANDBERG, AND CHANGYONG SONG, Coherent X-ray Diffraction Imaging, Examples IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS, DOI:10.1109/JSTQE.2011.2157306, 2011. Contains a comprehensive bibliography on CXDI. Other Conditions 8 MALCOLM HOWELLS, Principles and Practice of Coherent X-ray Diffraction Imaging, lecture notes, http://www.esrf.eu/ﬁles/live/sites/www/ﬁles/events/conferences/ Tutorials/slideslecture7.pdf, 2014. History of CDI 9 A.BARNETT,C.L.EPSTEIN,L.GREENGARD, J.F. MAGLAND, Geometry of the Classical Phase Retrieval Bibliography Problem, arXiv:1808.10747v1 [math.NA], 2018, 26pp.