1 Distributed optimization for nonrigid nano-tomography Viktor Nikitin, Vincent De Andrade, Azat Slyamov, Benjamin J. Gould, Yuepeng Zhang, Vandana Sampathkumar, Narayanan Kasthuri, Doga˘ Gursoy,¨ Member, IEEE, Francesco De Carlo Abstract—Resolution level and reconstruction quality in nano- with tomographic acquisitions are complex, unpractical and computed tomography (nano-CT) are in part limited by the impose stringent experimental constraints ruling out most of stability of microscopes, because the magnitude of mechanical in situ experiments [8]–[10]. vibrations during scanning becomes comparable to the imaging resolution, and the ability of the samples to resist radiation To address sample deformation due to radiation damage, a induced deformations during data acquisition. In such cases, number of time-resolved methods have been adopted during there is no incentive in recovering the sample state at different the last two decades. They demonstrate significant quality time steps like in time-resolved reconstruction methods, but improvement for time-evolving samples compared to the con- instead the goal is to retrieve a single reconstruction at the ventional approach, however, most of them are based on highest possible spatial resolution and without any imaging artifacts. Here we propose a distributed optimization solver additional a priori knowledge about the sample structure and for tomographic imaging of samples at the nanoscale. Our motion, and require demanding computational resources for approach solves the tomography problem jointly with projec- reconstructing experimental data in a reasonable time. For tion data alignment, nonrigid sample deformation correction, instance, in [11] we proposed a multi-GPU implementation and regularization. Projection data consistency is regulated by of a method for suppressing motion artifacts by using time- dense optical flow estimated by Farneback’s algorithm, leading to sharp sample reconstructions with less artifacts. Synthetic domain decomposition of functions with a basis chosen with data tests show robustness of the method to Poisson and low- respect to the motion structure. The method operates with a frequency background noise. We accelerated the solver on multi- low number of decomposition coefficients that can be used GPU systems and validated the method on three nano-imaging to determine the object state continuously in time. Next, the experimental data sets. approach proposed in [12], [13] is based on estimating local Index Terms—Tomographic reconstruction, ADMM, nonrigid structural correlations over multiple time frames and finding alignment, deformation estimation, optical flow inner object edges which remain constant in time, followed by the patched-based regularization according to the object I. INTRODUCTION structure. Also, there are methods built upon the concept Current X-ray imaging instruments demonstrate sub-100 nm of compressed sensing, which employs sparsity promoting resolution level in a wide range of fields, including biology, algorithms, where the prior knowledge is given in terms of medicine, geology, and material sciences [1]–[7]. To reach spatial-temporal total variation regularization for detecting this or even higher resolution levels one needs to address sharp object changes [14], [15]. various instrument component limitations such as mechanical Compared to the time-resolved methods above, the use of instabilities from scanner, detector, and optic components. At deformation vector fields (DVFs) is an even more common the same time, the structure of a sample may change during X- approach to model the sample evolution at discrete time ray data acquisition, leading to inconsistent tomographic data, intervals during scanning. The displacement between volumes further increasing the complexity of the sample reconstruction. obtained from two independent sets of projections covering arXiv:2008.03375v2 [cs.CE] 28 Feb 2021 ◦ This is caused not only by the controlled environmental 180 intervals is generally recovered with the Digital Volume conditions for in situ study of dynamic processes, but also Correlation (DVC) [16]. This works only if the deformation by the uncontrolled sample deformation due, for example, is slow compared to the scan speed, which is the case of to radiation damage. Severe radiation dose deposition may some fast synchrotron imaging applications [17]–[19]. Faster induce not only sample deformation but also ultimate de- acquisition can be achieved by decreasing exposure times struction of features. Cryogenic systems like cryojet in most and the number of projection angles, however, DVC in this cases protect the sample from being destroyed, however, they case suffers from noise and limited angle artifacts. In medical only partially reduce sample deformation while the cold air imaging applications, such as the study of heart or lungs, stream induces vibrations. On the other hand, in vacuum the displacement problem is easier to handle because the cryostages typically maintain samples at lower temperatures motion is often periodic and standard regrouping of projections than cryostreams, more efficiently preventing sample deforma- yields reconstruction for different phases of the cycle [20], tion without adding vibrations. But such systems compatible [21]. However, the quality of reconstruction still suffers from the angular undersampling for each sample phase, so as the V. Nikitin, V. De Andrade, B. J. Gould, Y. Zhang, D. Gursoy,¨ F. De Carlo applicability is limited to a concrete deformation process. are with Argonne National Laboratory, USA A. Slyamov is with the Technical University of Denmark, Denmark Time-resolved DVC-based algorithms have also been proposed V. Sampathkumar and N. Kasthuri are with the Unviersity of Chicago, USA to deal with continuous sample deformation during data acqui- 2 sition. The method proposed in [22] uses the sensitivity maps parameters including initial and final DVF smoothness levels as the basis functions for representing DVFs, establishing a in spatial and temporal variables, regularizer for deformation connection between the acquired images and the unknown ma- evolution as in [24]; weights κ1; κ2; κ3; κ4 for minimization terial parameters. The DVC then operates only with these basis terms as in [26]. However, there is no general guidance on functions, allowing for faster deformation estimation. Direct how to pick these parameters. material parameter identification provides strong regulariza- In most experiments there is no requirement to recon- tion ensuring optimal accuracy and noise robustness. Another struct the object state at different time steps as proposed method [23] is based on the existence of the template sample by computationally demanding time-resolved reconstruction volume recovered from high-quality undeformed projections. methods, but instead the goal is to have a single reconstruction Further sample motion is recovered by using only a few of high quality and resolution. These experiments include projections per deformation. This approach can be efficiently tomography of biological and medical samples, where X-ray used in dynamic imaging of samples with manual changing dose deposition causes uncontrolled radiation or thermally of temperature or pressure conditions, however, it will fail if induced deformation [5], [24], [28], [29]. Another application some deformation happens during acquisition of the template. area is tomography of geological samples affected by exter- nal cooling or heating systems, where the sample structure Several advanced methods have been recently proposed changes depending on the rotation angle and corresponding to address the above issues [24]–[26]. We see two general distance to the external device [30], [31]. Even mechanical limitations in using these methods for nano-CT applications limitations of the rotation stage, such as inaccurate roll/pitch with typically large-scale datasets. First, these methods are angles alignment or vibrations, in fact, create a time-resolved time-consuming even though they are implemented on GPUs. tomography problem where only one state needs to be recov- Because it is not feasible to store and process a large number ered. of DVFs for modeling rapid object changes, the temporal The consistency of projections for different angles can be resolution for the DVF estimation is chosen coarsely (e.g., one improved by directly translating projection images horizon- or several time frames for each 180 degrees rotation) [24], tally or vertically before the tomographic reconstruction [5], [27], so as the spatial resolution is decreased by applying [32]–[35], or by using iterative re-projection methods based binning (e.g., 4 × 4) to get faster and more robust DVF esti- on a joint estimation of alignment errors and the object mation [24]. Similarly, the authors in [25] proposing the usage function [36]–[38]. These methods work especially well for of B-splines for DVF estimation directly highlight that down- the interlaced angular scanning protocol, in which a full sampling can be carried out to perform GPU computations but tomographic scan is acquired with multiple sample rotations at the cost of some loss in the displacement field’s accuracy. where projection angles are mod 2π different and uniformly They also employ linear interpolation functions to estimate cover the interval [0; 2π) [26], [39]. With this protocol, a DVFs for time stamps corresponding to the acquisition of each significant time gap between acquisition of two nearby angles, projection,