Sparse Sensing for Statistical Inference
Total Page:16
File Type:pdf, Size:1020Kb
Full text available at: http://dx.doi.org/10.1561/2000000069 Sparse Sensing for Statistical Inference Sundeep Prabhakar Chepuri Faculty of Electrical Engineering, Mathematics, and Computer Science Delft University of Technology The Netherlands [email protected] Geert Leus Faculty of Electrical Engineering, Mathematics, and Computer Science Delft University of Technology The Netherlands [email protected] Boston — Delft Full text available at: http://dx.doi.org/10.1561/2000000069 Foundations and Trends R in Signal Processing Published, sold and distributed by: now Publishers Inc. PO Box 1024 Hanover, MA 02339 United States Tel. +1-781-985-4510 www.nowpublishers.com [email protected] Outside North America: now Publishers Inc. PO Box 179 2600 AD Delft The Netherlands Tel. +31-6-51115274 The preferred citation for this publication is S.P. Chepuri and G. Leus. Sparse Sensing for Statistical Inference. Foundations and Trends R in Signal Processing, vol. 9, no. 3-4, pp. 233–386, 2015. R This Foundations and Trends issue was typeset in LATEX using a class file designed by Neal Parikh. Printed on acid-free paper. ISBN: 978-1-68083-236-5 c 2016 S.P. Chepuri and G. Leus All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording or otherwise, without prior written permission of the publishers. Photocopying. In the USA: This journal is registered at the Copyright Clearance Cen- ter, Inc., 222 Rosewood Drive, Danvers, MA 01923. Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by now Publishers Inc for users registered with the Copyright Clearance Center (CCC). The ‘services’ for users can be found on the internet at: www.copyright.com For those organizations that have been granted a photocopy license, a separate system of payment has been arranged. Authorization does not extend to other kinds of copy- ing, such as that for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. In the rest of the world: Permission to pho- tocopy must be obtained from the copyright owner. Please apply to now Publishers Inc., PO Box 1024, Hanover, MA 02339, USA; Tel. +1 781 871 0245; www.nowpublishers.com; [email protected] now Publishers Inc. has an exclusive license to publish this material worldwide. Permission to use this content must be obtained from the copyright license holder. Please apply to now Publishers, PO Box 179, 2600 AD Delft, The Netherlands, www.nowpublishers.com; e-mail: [email protected] Full text available at: http://dx.doi.org/10.1561/2000000069 Foundations and Trends R in Signal Processing Volume 9, Issue 3-4, 2015 Editorial Board Editor-in-Chief Yonina Eldar Technion - Israel Institute of Technology Israel Editors Robert M. Gray Sheila Hemami Vincent Poor Founding Editor-in-Chief Northeastern University Princeton University Stanford University Lina Karam Anna Scaglione Pao-Chi Chang Arizona State U UC Davis NCU, Taiwan Nick Kingsbury Mihaela van der Shaar Pamela Cosman University of Cambridge UCLA UC San Diego Alex Kot Nicholas D. Sidiropoulos Michelle Effros NTU, Singapore TU Crete Caltech Jelena Kovacevic Michael Unser Yariv Ephraim CMU EPFL GMU Geert Leus P. P. Vaidyanathan Alfonso Farina TU Delft Caltech Selex ES Jia Li Ami Wiesel Sadaoki Furui Penn State Hebrew U Tokyo Tech Henrique Malvar Min Wu Georgios Giannakis Microsoft Research University of Maryland University of Minnesota B.S. Manjunath Josiane Zerubia Vivek Goyal UC Santa Barbara INRIA Boston University Urbashi Mitra Sinan Gunturk USC Courant Institute Björn Ottersten Christine Guillemot KTH Stockholm INRIA Thrasos Pappas Robert W. Heath, Jr. Northwestern University UT Austin Full text available at: http://dx.doi.org/10.1561/2000000069 Editorial Scope Topics Foundations and Trends R in Signal Processing publishes survey and tutorial articles in the following topics: • Adaptive signal processing • Signal decompositions, subband • Audio signal processing and transform methods, sparse representations • Biological and biomedical signal • processing Signal processing for communications • Complexity in signal processing • Signal processing for security and • Digital signal processing forensic analysis, biometric signal • Distributed and network signal processing processing • Signal quantization, sampling, • Image and video processing analog-to-digital conversion, • Linear and nonlinear filtering coding and compression • Multidimensional signal • Signal reconstruction, processing digital-to-analog conversion, • Multimodal signal processing enhancement, decoding and inverse problems • Multirate signal processing • Speech/audio/image/video • Multiresolution signal processing compression • Nonlinear signal processing • Speech and spoken language • Randomized algorithms in signal processing processing • Statistical/machine learning • Sensor and multiple source signal processing, source separation • Statistical signal processing Information for Librarians Foundations and Trends R in Signal Processing, 2015, Volume 9, 4 issues. ISSN paper version 1932-8346. ISSN online version 1932-8354. Also available as a combined paper and online subscription. Full text available at: http://dx.doi.org/10.1561/2000000069 Foundations and Trends R in Signal Processing Vol. 9, No. 3-4 (2015) 233–386 c 2016 S.P. Chepuri and G. Leus DOI: 10.1561/2000000069 Sparse Sensing for Statistical Inference Sundeep Prabhakar Chepuri Faculty of Electrical Engineering, Mathematics, and Computer Science Delft University of Technology The Netherlands [email protected] Geert Leus Faculty of Electrical Engineering, Mathematics, and Computer Science Delft University of Technology The Netherlands [email protected] Full text available at: http://dx.doi.org/10.1561/2000000069 Contents 1 Introduction 2 1.1 Pervasive sensors and data deluge . 2 1.2 Outline............................ 6 2 Sparse Sensing 10 2.1 Introduction . 10 2.2 Sparse sensing model . 11 2.3 Applications . 14 2.4 Benefits ........................... 15 2.5 Sparse sensing versus compressive sensing . 16 2.6 Sparse sensing design . 17 2.7 Relationtodesignofexperiments. 23 2.8 Discussion .......................... 23 3 Sparse Sensing for Estimation 25 3.1 Introduction . 25 3.2 Sensing nonlinear observations . 28 3.3 f(w) forestimation ..................... 28 3.4 Problemstatement ..................... 33 3.5 Solvers............................ 35 3.6 Numerical example: sensor placement . 48 ii Full text available at: http://dx.doi.org/10.1561/2000000069 iii 3.7 Dependent observations . 54 3.8 Discussion .......................... 59 3.A Performance thresholds . 59 3.B Projected Newton’s method . 61 3.C Poweriterations ....................... 62 4 Sparse Sensing for Filtering 63 4.1 Introduction . 63 4.2 Sensing time-varying observations . 66 4.3 f(w) forfiltering ...................... 67 4.4 Problemstatement ..................... 69 4.5 Solvers............................ 71 4.6 Numerical example: sensor scheduling . 73 4.7 Dependent observations . 77 4.8 Structuredsignals ...................... 79 4.9 Discussion .......................... 85 4.A ExtendedKalmanfilter . 85 5 Sparse Sensing for Detection 87 5.1 Introduction . 87 5.2 Sensing conditionally distributed observations . 89 5.3 Problemstatement ..................... 90 5.4 f(w) fordetection...................... 91 5.5 Solvers............................ 98 5.6 Illustrativeexamples. 101 5.7 Dependent observations . 107 5.8 Discussion .......................... 118 5.A ProofofProposition5.1. 119 5.B Upper bound on Pm ..................... 120 5.C ProofofProposition5.2. 121 5.D Expressions for f0(w) and f1(w) .............. 121 6 Continuous Sparse Sensing 124 6.1 Introduction . 124 6.2 Sensitivity to gridding . 125 6.3 Sensing model based on binning . 128 Full text available at: http://dx.doi.org/10.1561/2000000069 iv 6.4 Risk function for continuous sparse sensing . 130 6.5 Solver ............................ 130 6.6 Discussion .......................... 135 7 Outlook 136 7.1 Concluding remarks . 136 7.2 Directions for future research . 138 Glossary 143 References 147 Full text available at: http://dx.doi.org/10.1561/2000000069 Abstract In today’s society, we are flooded with massive volumes of data in the order of a billion gigabytes on a daily basis from pervasive sensors. It is becoming increasingly challenging to sense, store, transport, or process (i.e., for inference) the acquired data. To alleviate these problems, it is evident that there is an urgent need to significantly reduce the sensing cost (i.e., the number of expensive sensors) as well as the related mem- ory and bandwidth requirements by developing unconventional sensing mechanisms to extract as much information as possible yet collecting fewer data. The aim of this monograph is therefore to develop theory and al- gorithms for smart data reduction. We develop a data reduction tool called sparse sensing, which consists of a deterministic and structured sensing function (guided by a sparse vector) that is optimally designed to achieve a desired inference performance with the reduced number of data samples. We develop sparse sensing mechanisms, convex pro- grams, and greedy algorithms to efficiently design sparse sensing func- tions, where we assume that the data is not yet available and the model information is perfectly known. Sparse sensing offers a number