DOCSLIB.ORG

Image Restoration Using Total Variation Regularized Deep Image Prior

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Image Restoration Using Total Variation Regularized Deep Image Prior IMAGE RESTORATION USING TOTAL VARIATION REGULARIZED DEEP IMAGE PRIOR Jiaming Liu1, Yu Sun2, Xiaojian Xu2, and Ulugbek S. Kamilov1;2 1 Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO 63130 2 Department of Computer Science and Engineering, Washington University in St. Louis, St. Louis, MO 63130 ABSTRACT In the past decade, sparsity-driven regularization has led to signifi- cant improvements in image reconstruction. Traditional regularizers, such as total variation (TV), rely on analytical models of sparsity. However, increasingly the field is moving towards trainable mod- els, inspired from deep learning. Deep image prior (DIP) is a recent regularization framework that uses a convolutional neural network (CNN) architecture without data-driven training. This paper extends the DIP framework by combining it with the traditional TV regular- ization. We show that the inclusion of TV leads to considerable per- formance gains when tested on several traditional restoration tasks Original Corrupt CBM3D 20.93dB such as image denoising and deblurring. Index Terms— Image reconstruction, image restoration, deep learning, deep image prior, total variation regularization. 1. INTRODUCTION Image reconstruction is one of the most widely studied problems in computational imaging. Since the problem is often ill-posed, the process is traditionally regularized by constraining the solutions to be consistent with our prior knowledge about the image. Some tradi- tional imaging priors include nonnegativity, transform-domain spar- DIP 20.91dB DIP-TV 21.35dB sity, and self-similarity [1–4]. Recently, however, the attention in the field has been shifting towards new imaging formulations based on deep learning [5]. Fig. 1: Comparison of DIP-TV against several standard algorithms The most common deep-learning approach is based on an end- for image denoising. DIP-TV achieves the best SNR performances to-end training of a convolutional neural network (CNN) for repro- on Monarch with AWGN of σ = 65. The combination of the CNN ducing the desired image from its noisy measurements [6–10]. A and TV priors preserve homogeneity of the background as well as popular alternative considers training a CNN as an image denoiser the texture, highlighted by rectangles drawn inside the images. and using it within an iterative reconstruction algorithms [11–14]. However, recently, it was also shown that a CNN can by itself reg- ularize image reconstruction without data-driven training [15]. This 2. BACKGROUND deep image prior (DIP) framework naturally regularizes reconstruc- tion by optimizing the weights of a CNN for it to synthesize the Consider the restoration as a linear inverse problem measurements from a given random input vector. The intuition be- y =Hx+e; (1) hind DIP is that natural images can be well represented by CNNs, N which is not the case for the random noise and certain other image where the goal is to reconstruct an unknown image x2R from M M×N degradations. DIP was shown to achieve remarkable performance on the measurements y 2R . Here, H2R is a degradation ma- M a number of image reconstruction tasks [15, 16]. trix and e2R corresponds to the measurement noise, which is as- 2 In this paper, we propose to further improve DIP by combin- sumed to be additive white Gaussian (AWGN) of variance σ . ing an implicit CNN regularization with an explicit TV penalty. The As practical inverse problems are often ill-posed, it is common idea of our DIP-TV approach is simple: by including an additional to regularize the task by constraining the solution according some TV term into the objective function, we restrict the solutions synthe- prior knowledge. In practice, the reconstruction often relies on the sized by CNN to those that are piecewise smooth. We experimentally regularized least-squares formulation show that our DIP-TV method outperforms the traditional formula- ∗ 2 x =argmin ky −Hxk`2 +λρ(x) (2) tions of DIP and TV, and performs on a par with other state-of-the-art x image restoration methods such as BM3D [17] and IRCNN [12]. where the data-fidelity term ensures the consistency with measure- This material is based upon work supported by the National Science ments, and regularizer ρ constrains the solution to the desired image Foundation under Grant No. 1813910. class. The parameter λ>0 controls the strength of regularization. 978-1-5386-4658-8/18/$31.00 ©2019 IEEE 7715 ICASSP 2019 128 128 128 Skip Connections 128 128 128 128 128 128 Fig. 2: The set of 14 grayscale images used in experiments. Total variation (TV) is one of the most widely used image priors that promotes sparsity in image in image gradients [18]. It has been 128 128 Leaky ReLU BN u[i] shown to be effective in a number of applications [19–21]. The 3x3 conv. 1x1 conv. d[i] `1-based anisotropic TV is given by N X Fig. 3: CNN architecture [15] used in this paper. The architecture ρTV (x) , j[D1x]nj+j[D2x]nj; (3) is based on the well-known U-net with skip connections between i=1 the down layers and up layers. Two different kernel sizes are noted under each convolutional layer, and the number of filters is illustrated where D1 and D2 denote the finite difference operation along the first and second dimension of a two-dimensional (2D) image with above each block. The variable ns[i] denotes the number of feature appropriate boundary conditions. maps at ith skip layer, and the features in other layers correspond to Currently, deep learning achieves the state-of-the-art perfor- 128. mance for different image restoration problems [22–24]. The core idea is to train a CNN via the following optimization Optimization in (6) is similar to training of a CNN and one can rely ∗ Θ =argmin L(fΘ(y);x); (4) on any standard optimization algorithms. Θ Figure 3 illustrates the CNN architecture we used in this paper, where fΘ(·) represents the CNN parametrized by Θ. L denotes which was adapted from [15]. In particular, the popular U-net ar- the loss function. In practice, (4) can be effectively optimized using chitecture [26] is modified such that the skip connections contain the family of stochastic gradient descend (SGD) methods, such as a convolutional layer. The decoder uses a down-sampling and up- adaptive moment estimation (ADAM) [25]. sampling based scaling-expanding structure, which makes the effec- Recently, Ulyanov et al. [15] proposed to use CNN-based meth- tive receptive field of the network increase as the input goes deeper ods in an alternative way. They discovered that the architecture of into the network [27]. Besides, the skip connection enables the later deep CNN models is well-suited for representing natural images, but layers to reconstruct the feature maps with both local details and not random noise. With a random input vector, CNN can reproduce global texture. Here, the input z can be initialized with a fixed 3D the clear image without supervised training on a large dataset. In the tensor with 32 feature maps and of the same spatial size as x filled context of image restoration, the associated optimization for DIP can with uniform noise. The proposed framework can deal with both be formulated as grayscale and color images, where for color images anisotropic TV jointly regularizes all three channels. ∗ 2 Θ =argmin ky −HfΘ(z)k`2 ; Θ (5) ∗ such that x = fΘ∗ (z): 4. EXPERIMENTS N We now present the experimental results on image denoising and where z2R denotes the random input vector. The CNN generator is initialized with random variables Θ, and these variables are iter- deblurring. We consider 14 gray scale images and 8 standard color atively optimized so that the output of the network is as close to the images (256×256 and 512×512) from set12, set14, and BSD68 as target measurement as possible. our testing images. The gray scale images are shown in Figure 2, while color images are: Monarch, Parrots, House, Lena, Peppers, Baby, and Jet. 3. PROPOSED METHOD The goal of DIP-TV is to use the TV regularization to improve the 4.1. Image Denoising basic DIP approach. We first consider the optimization problem shown in (2) and the objective function of DIP in (5). One can In this subsection, we analyze the performance of DIP-TV method 2 for image denoising problems. The CNN architecture in Figure 3 find that the ky −HfΘ(z)k`2 term in (5) actually corresponds to the is used for both color and grayscale images, with ns[i]=4 for each data-fidelity term in (3) by replacing fΘ(z) with an unknown image output. Thus, we can consider replacing (5) with an optimization skip layers. All algorithmic hyperparameters were optimized in problem each experiment for the best signal-to-noise ratio (SNR) perfor- mance with respect to the ground truth test image. Both DIP-TV ∗ 2 and DIP were set to run 5000 optimization step. We use the average Θ =argmin ky −HfΘ(z)k`2 +λρTV(fΘ(z)) ; Θ (6) SNR to denote the SNR values averaged over the associated set of ∗ such that x = fΘ∗ (z): test images. 7716 Original Corrupt (SNR: 5.00dB) EPLL (SNR: 21.20dB) BM3D (SNR: 21.22dB) TV (SNR: 20.60dB) DIP (SNR: 20.51dB) DIP-TV (SNR: 21.57dB) Original Corrupt (SNR: 15.00dB) EPLL (SNR: 27.21dB) BM3D (SNR: 27.26dB) TV (SNR: 25.49dB) DIP (SNR: 26.89dB) DIP-TV (SNR: 27.33dB) Baselines TV / DIP / DIP-TV Fig. 4: Image denoising results on Tower and Jet obtained by EPLL, BM3D, TV-FISTA, DIP, and DIP-TV. The first and second columns display the original images and corrupted images, respectively.
Load more

Recommended publications
  • NTGAN: Learning Blind Image Denoising Without Clean Reference
    RUI ZHAO, DANIEL P.K. LUN, KIN-MAN LAM: NTGAN 1 NTGAN: Learning Blind Image Denoising without Clean Reference Rui Zhao Department of Electronic and [email protected] Information Engineering, Daniel P.K. Lun The Hong Kong Polytechnic University, [email protected] Hong Kong, China Kin-Man Lam [email protected] Abstract Recent studies on learning-based image denoising have achieved promising perfor- mance on various noise reduction tasks. Most of these deep denoisers are trained ei- ther under the supervision of clean references, or unsupervised on synthetic noise. The assumption with the synthetic noise leads to poor generalization when facing real pho- tographs. To address this issue, we propose a novel deep unsupervised image-denoising method by regarding the noise reduction task as a special case of the noise transference task. Learning noise transference enables the network to acquire the denoising ability by only observing the corrupted samples. The results on real-world denoising benchmarks demonstrate that our proposed method achieves state-of-the-art performance on remov- ing realistic noises, making it a potential solution to practical noise reduction problems. 1 Introduction Convolutional neural networks (CNNs) have been widely studied over the past decade in various computer vision applications, including image classification [13], object detection [29], and image restoration [35]. In spite of high effectiveness, CNN models always require a large-scale dataset with reliable reference for training. However, in terms of low-level vision tasks, such as image denoising and image super-resolution, the reliable reference data is generally hard to access, which makes these tasks more challenging and difficult.
    [Show full text]
  • High-Quality Self-Supervised Deep Image Denoising
    High-Quality Self-Supervised Deep Image Denoising Samuli Laine Tero Karras Jaakko Lehtinen Timo Aila NVIDIA∗ NVIDIA NVIDIA, Aalto University NVIDIA Abstract We describe a novel method for training high-quality image denoising models based on unorganized collections of corrupted images. The training does not need access to clean reference images, or explicit pairs of corrupted images, and can thus be applied in situations where such data is unacceptably expensive or im- possible to acquire. We build on a recent technique that removes the need for reference data by employing networks with a “blind spot” in the receptive field, and significantly improve two key aspects: image quality and training efficiency. Our result quality is on par with state-of-the-art neural network denoisers in the case of i.i.d. additive Gaussian noise, and not far behind with Poisson and impulse noise. We also successfully handle cases where parameters of the noise model are variable and/or unknown in both training and evaluation data. 1 Introduction Denoising, the removal of noise from images, is a major application of deep learning. Several architectures have been proposed for general-purpose image restoration tasks, e.g., U-Nets [23], hierarchical residual networks [20], and residual dense networks [31]. Traditionally, the models are trained in a supervised fashion with corrupted images as inputs and clean images as targets, so that the network learns to remove the corruption. Lehtinen et al. [17] introduced NOISE2NOISE training, where pairs of corrupted images are used as training data. They observe that when certain statistical conditions are met, a network faced with the impossible task of mapping corrupted images to corrupted images learns, loosely speaking, to output the “average” image.
    [Show full text]
  • Learning Deep Image Priors for Blind Image Denoising
    Learning Deep Image Priors for Blind Image Denoising Xianxu Hou 1 Hongming Luo 1 Jingxin Liu 1 Bolei Xu 1 Ke Sun 2 Yuanhao Gong 1 Bozhi Liu 1 Guoping Qiu 1,3 1 College of Information Engineering and Guangdong Key Lab for Intelligent Information Processing, Shenzhen University, China 2 School of Computer Science, The University of Nottingham Ningbo China 3 School of Computer Science, The University of Nottingham, UK Abstract ods comes from the effective image prior modeling over the input images [5, 14, 20]. State of the art model-based meth- Image denoising is the process of removing noise from ods such as BM3D [12] and WNNM [20] can be further noisy images, which is an image domain transferring task, extended to remove unknown noises. However, there are a i.e., from a single or several noise level domains to a photo- few drawbacks of these methods. First, these methods usu- realistic domain. In this paper, we propose an effective im- ally involve a complex and time-consuming optimization age denoising method by learning two image priors from process in the testing stage. Second, the image priors em- the perspective of domain alignment. We tackle the domain ployed in most of these approaches are hand-crafted, such alignment on two levels. 1) the feature-level prior is to learn as nonlocal self-similarity and gradients, which are mainly domain-invariant features for corrupted images with differ- based on the internal information of the input image without ent level noise; 2) the pixel-level prior is used to push the any external information.
    [Show full text]
  • Layered Neural Rendering for Retiming People in Video
    Layered Neural Rendering for Retiming People in Video ERIKA LU, University of Oxford, Google Research FORRESTER COLE, Google Research TALI DEKEL, Google Research WEIDI XIE, University of Oxford ANDREW ZISSERMAN, University of Oxford DAVID SALESIN, Google Research WILLIAM T. FREEMAN, Google Research MICHAEL RUBINSTEIN, Google Research Input Video Frame t Frame t1 (child I jumps) Frame t2 (child II jumps) Frame t3 (child III jumps) Our Retiming Result Our Output Layers Frame t1 (all stand) Frame t2 (all jump) Frame t3 (all in water) Fig. 1. Making all children jump into the pool together — in post-processing! In the original video (left, top) each child is jumping into the poolata different time. In our computationally retimed video (left, bottom), the jumps of children I and III are time-aligned with that of child II, such thattheyalljump together into the pool (notice that child II remains unchanged in the input and output frames). In this paper, we present a method to produce this and other people retiming effects in natural, ordinary videos. Our method is based on a novel deep neural network that learns a layered decomposition oftheinput video (right). Our model not only disentangles the motions of people in different layers, but can also capture the various scene elements thatare correlated with those people (e.g., water splashes as the children hit the water, shadows, reflections). When people are retimed, those related elements get automatically retimed with them, which allows us to create realistic and faithful re-renderings of the video for a variety of retiming effects. The full input and retimed sequences are available in the supplementary video.
    [Show full text]
  • Deep Image Prior for Undersampling High-Speed Photoacoustic Microscopy
    Photoacoustics 22 (2021) 100266 Contents lists available at ScienceDirect Photoacoustics journal homepage: www.elsevier.com/locate/pacs Deep image prior for undersampling high-speed photoacoustic microscopy Tri Vu a,*, Anthony DiSpirito III a, Daiwei Li a, Zixuan Wang c, Xiaoyi Zhu a, Maomao Chen a, Laiming Jiang d, Dong Zhang b, Jianwen Luo b, Yu Shrike Zhang c, Qifa Zhou d, Roarke Horstmeyer e, Junjie Yao a a Photoacoustic Imaging Lab, Duke University, Durham, NC, 27708, USA b Department of Biomedical Engineering, Tsinghua University, Beijing, 100084, China c Division of Engineering in Medicine, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Cambridge, MA, 02139, USA d Department of Biomedical Engineering and USC Roski Eye Institute, University of Southern California, Los Angeles, CA, 90089, USA e Computational Optics Lab, Duke University, Durham, NC, 27708, USA ARTICLE INFO ABSTRACT Keywords: Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound. However, limited Convolutional neural network by the laser’s repetition rate, state-of-the-art high-speed PAM technology often sacrificesspatial sampling density Deep image prior (i.e., undersampling) for increased imaging speed over a large field-of-view. Deep learning (DL) methods have Deep learning recently been used to improve sparsely sampled PAM images; however, these methods often require time- High-speed imaging consuming pre-training and large training dataset with ground truth. Here, we propose the use of deep image Photoacoustic microscopy Raster scanning prior (DIP) to improve the image quality of undersampled PAM images. Unlike other DL approaches, DIP requires Undersampling neither pre-training nor fully-sampled ground truth, enabling its flexible and fast implementation on various imaging targets.
    [Show full text]
  • Image Deconvolution with Deep Image Prior: Final Report
    Image deconvolution with deep image prior: Final Report Abstract tion kernel is given (Sezan & Tekalp, 1990), i.e. recovering Image deconvolution has always been a hard in- X with knowing K in Equation 1. This problem is ill-posed, verse problem because of its mathematically ill- because simply applying the inverse of the convolution op- −1 posed property (Chan & Wong, 1998). A recent eration on degraded image B with kernel K, i.e. B∗ K, −1 work (Ulyanov et al., 2018) proposed deep im- gives an inverted noise term E∗ K, which dominates the age prior (DIP) which uses neural net structures solution (Hansen et al., 2006). to represent image prior information. This work Blind deconvolution: In reality, we can hardly obtain the mainly focuses on the task of image deconvolu- detailed kernel information, in which case the deconvolu- tion, using DIP to express image prior informa- tion problem is formulated in a blind setting (Kundur & tion and ref ne the domain of its objectives. It Hatzinakos, 1996). More concisely, blind deconvolution is proposes new energy functions for kernel-known to recover X without knowing K. This task is much more and blind deconvolution respectively in terms of challenging than it is under non-blind settings, because the DIP, and uses hourglass ConvNet (Newell et al., observed information becomes less and the domains of the 2016) as the DIP structures to represent sharpness variables become larger (Chan & Wong, 1998). or other higher level priors of natural images, as well as the prior in degradation kernels. From the In image deconvolution, prior information on unknown results on 6 standard test images, we f rst prove images and kernels (in blind settings) can signif cantly im- that DIP with ConvNet structure is strongly ca- prove the deconvolved results.
    [Show full text]
  • PROCEEDINGS of the ICA CONGRESS (Onl the ICA PROCEEDINGS OF
    ine) - ISSN 2415-1599 ISSN ine) - PROCEEDINGS OF THE ICA CONGRESS (onl THE ICA PROCEEDINGS OF Page intentionaly left blank 22nd International Congress on Acoustics ICA 2016 PROCEEDINGS Editors: Federico Miyara Ernesto Accolti Vivian Pasch Nilda Vechiatti X Congreso Iberoamericano de Acústica XIV Congreso Argentino de Acústica XXVI Encontro da Sociedade Brasileira de Acústica 22nd International Congress on Acoustics ICA 2016 : Proceedings / Federico Miyara ... [et al.] ; compilado por Federico Miyara ; Ernesto Accolti. - 1a ed . - Gonnet : Asociación de Acústicos Argentinos, 2016. Libro digital, PDF Archivo Digital: descarga y online ISBN 978-987-24713-6-1 1. Acústica. 2. Acústica Arquitectónica. 3. Electroacústica. I. Miyara, Federico II. Miyara, Federico, comp. III. Accolti, Ernesto, comp. CDD 690.22 ISSN 2415-1599 ISBN 978-987-24713-6-1 © Asociación de Acústicos Argentinos Hecho el depósito que marca la ley 11.723 Disclaimer: The material, information, results, opinions, and/or views in this publication, as well as the claim for authorship and originality, are the sole responsibility of the respective author(s) of each paper, not the International Commission for Acoustics, the Federación Iberoamaricana de Acústica, the Asociación de Acústicos Argentinos or any of their employees, members, authorities, or editors. Except for the cases in which it is expressly stated, the papers have not been subject to peer review. The editors have attempted to accomplish a uniform presentation for all papers and the authors have been given the opportunity
    [Show full text]
  • Controlling Neural Networks Via Energy Dissipation
    Controlling Neural Networks via Energy Dissipation Michael Moeller Thomas Mollenhoff¨ Daniel Cremers University of Siegen TU Munich TU Munich [email protected] [email protected] [email protected] Abstract and computing its Radon transform, respectively. Mathe- matically, the above problems can be phrased as linear in- The last decade has shown a tremendous success in verse problems in which one tries to recover the desired solving various computer vision problems with the help of quantity uˆ from measurements f that arise from applying deep learning techniques. Lately, many works have demon- an application-dependent linear operator A to the unknown strated that learning-based approaches with suitable net- and contain additive noise ξ: work architectures even exhibit superior performance for the solution of (ill-posed) image reconstruction problems f = Auˆ + ξ. (1) such as deblurring, super-resolution, or medical image re- Unfortunately, most practically relevant inverse problems construction. The drawback of purely learning-based meth- are ill-posed, meaning that equation (1) either does not de- ods, however, is that they cannot provide provable guaran- termine uˆ uniquely even if ξ = 0, or tiny amounts of noise tees for the trained network to follow a given data formation ξ can alter the naive prediction of uˆ significantly. These process during inference. In this work we propose energy phenomena have been well-investigated from a mathemat- dissipating networks that iteratively compute a descent di- ical perspective with regularization methods being the tool rection with respect to a given cost function or energy at the to still obtain provably stable reconstructions.
    [Show full text]
  • 2007 Registration Document
    2007 REGISTRATION DOCUMENT (www.renault.com) REGISTRATION DOCUMENT REGISTRATION 2007 Photos cre dits: cover: Thomas Von Salomon - p. 3 : R. Kalvar - p. 4, 8, 22, 30 : BLM Studio, S. de Bourgies S. BLM Studio, 30 : 22, 8, 4, Kalvar - p. R. 3 : Salomon - p. Von Thomas cover: dits: Photos cre 2007 REGISTRATION DOCUMENT INCLUDING THE MANAGEMENT REPORT APPROVED BY THE BOARD OF DIRECTORS ON FEBRUARY 12, 2008 This Registration Document is on line on the website www .renault.com (French and English versions) and on the AMF website www .amf- france.org (French version only). TABLE OF CONTENTS 0 1 05 RENAULT AND THE GROUP 5 RENAULT AND ITS SHAREHOLDERS 157 1.1 Presentation of Renault and the Group 6 5.1 General information 158 1.2 Risk factors 24 5.2 General information about Renault’s share capital 160 1.3 The Renault-Nissan Alliance 25 5.3 Market for Renault shares 163 5.4 Investor relations policy 167 02 MANAGEMENT REPORT 43 06 2.1 Earnings report 44 MIXED GENERAL MEETING 2.2 Research and development 62 OF APRIL 29, 2008: PRESENTATION 2.3 Risk management 66 OF THE RESOLUTIONS 171 The Board first of all proposes the adoption of eleven resolutions by the Ordinary General Meeting 172 Next, six resolutions are within the powers of 03 the Extraordinary General Meeting 174 SUSTAINABLE DEVELOPMENT 79 Finally, the Board proposes the adoption of two resolutions by the Ordinary General Meeting 176 3.1 Employee-relations performance 80 3.2 Environmental performance 94 3.3 Social performance 109 3.4 Table of objectives (employee relations, environmental
    [Show full text]
  • Denoising and Regularization Via Exploiting the Structural Bias of Convolutional Generators
    Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel∗ and Mahdi Soltanolkotabiy ∗Dept. of Electrical and Computer Engineering, Technical University of Munich yDept. of Electrical and Computer Engineering, University of Southern California October 31, 2019 Abstract Convolutional Neural Networks (CNNs) have emerged as highly successful tools for image generation, recovery, and restoration. This success is often attributed to large amounts of training data. However, recent experimental findings challenge this view and instead suggest that a major contributing factor to this success is that convolutional networks impose strong prior assumptions about natural images. A surprising experiment that highlights this architec- tural bias towards natural images is that one can remove noise and corruptions from a natural image without using any training data, by simply fitting (via gradient descent) a randomly initialized, over-parameterized convolutional generator to the single corrupted image. While this over-parameterized network can fit the corrupted image perfectly, surprisingly after a few iterations of gradient descent one obtains the uncorrupted image. This intriguing phenomena enables state-of-the-art CNN-based denoising and regularization of linear inverse problems such as compressive sensing. In this paper we take a step towards demystifying this experimental phenomena by attributing this effect to particular architectural choices of convolutional net- works, namely convolutions with fixed interpolating filters. We then formally characterize the dynamics of fitting a two layer convolutional generator to a noisy signal and prove that early- stopped gradient descent denoises/regularizes. This results relies on showing that convolutional generators fit the structured part of an image significantly faster than the corrupted portion.
    [Show full text]
  • Deep Geometric Prior for Surface Reconstruction
    Deep Geometric Prior for Surface Reconstruction Francis Williams1, Teseo Schneider1, Claudio Silva 1, Denis Zorin1, Joan Bruna1, and Daniele Panozzo1 1New York University [email protected], [email protected], [email protected], [email protected], [email protected], [email protected] Abstract retains critical surface features and approximates the sam- pled surface well, is a pervasive and challenging problem. The reconstruction of a discrete surface from a point Different approaches have been proposed, mostly cloud is a fundamental geometry processing problem that grouped into several categories: (1) using the points to de- has been studied for decades, with many methods devel- fine a volumetric scalar function whose 0 level-set corre- oped. We propose the use of a deep neural network as a sponds to the desired surface, (2) attempt to ”connect the geometric prior for surface reconstruction. Specifically, we dots” in a globally consistent way to create a mesh, (3) fit a overfit a neural network representing a local chart parame- set of primitive shapes so that the boundary of their union terization to part of an input point cloud using the Wasser- is close to the point cloud, and (4) fit a set of patches to the stein distance as a measure of approximation. By jointly point cloud approximating the surface. fitting many such networks to overlapping parts of the point cloud, while enforcing a consistency condition, we compute We propose a novel method, based, on the one hand, on a manifold atlas. By sampling this atlas, we can produce a constructing a manifold atlas commonly used in differen- dense reconstruction of the surface approximating the input tial geometry to define a surface, and, on the other hand, on cloud.
    [Show full text]
  • Deep Learning at the Edge for Channel Estimation in Beyond-5G Massive MIMO Mauro Belgiovine, Kunal Sankhe, Carlos Bocanegra, Debashri Roy, and Kaushik R
    EDGE INTELLIGENCE FOR BEYOND 5G NETWORKS Deep Learning at the Edge for Channel Estimation in Beyond-5G Massive MIMO Mauro Belgiovine, Kunal Sankhe, Carlos Bocanegra, Debashri Roy, and Kaushik R. Chowdhury BSTRACT large antenna arrays. This article is motivated by A our desire to decouple the scale of deployment Massive multiple-input multiple-output with the limits of classical processing, especially as (mMIMO) is a critical component in upcoming 5G it pertains to the task of understanding the channel wireless deployment as an enabler for high data between a given antenna-receiver antenna-element rate communications. mMIMO is effective when pair for millimeter-wave (mmWave) communi- each corresponding antenna pair of the respec- cation. We accomplish this via training a deep tive transmitter-receiver arrays experiences an inde- learning (DL) architecture that offers the ability to pendent channel. While increasing the number of produce a robust and high fidelity channel matrix antenna elements increases the achievable data between the mobile user and the mMIMO BS in rate, at the same time computing the channel state a single forward pass. Since the overhead of the information (CSI) becomes prohibitively expen- DL-based channel estimation becomes irrespective sive. In this article, we propose to use deep learn- of the size of the antenna array, we believe this ing via a multi-layer perceptron architecture that approach will enable a fundamental leap toward exceeds the performance of traditional CSI pro- beyond 5G (B5G) standards where thousands of cessing methods like least square (LS) and linear coordinated antennas will become the new norm. minimum mean square error (LMMSE) estimation, Emerging B5G networks are envisioned to support thus leading to a beyond fifth generation (B5G) edge computing, which will enable rapid optimiza- networking paradigm wherein machine learning tion and reconfiguration of the network architec- fully drives networking optimization.
    [Show full text]