CONN CTIONS DEPARTMENT of ELECTRICAL & COMPUTER ENGINEERING A
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Arxiv:1805.06576V5 [Stat.ML] 11 Nov 2018
Mad Max: Affine Spline Insights into Deep Learning RANDALL BALESTRIERO, RICHARD BARANIUK Rice University, Houston, Texas, USA [email protected], [email protected] We build a rigorous bridge between deep networks (DNs) and approximation theory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max- affine spline operators (MASOs), which provide a powerful portal through which to view and analyze their inner workings. For instance, conditioned on the input signal, the output of a MASO DN can be written as a simple affine transformation of the input. This implies that a DN constructs a set of signal- dependent, class-specific templates against which the signal is compared via a simple inner product; we explore the links to the classical theory of optimal classification via matched filters and the effects of data memorization. Going further, we propose a simple penalty term that can be added to the cost function of any DN learning algorithm to force the templates to be orthogonal with each other; this leads to significantly improved classification performance and reduced overfitting with no change to the DN architecture. The spline partition of the input signal space that is implicitly induced by a MASO directly links DNs to the theory of vector quantization (VQ) and K-means clustering, which opens up new geometric avenue to study how DNs organize signals in a hierarchical fashion. To validate the utility of the VQ interpretation, we develop and validate a new distance metric for signals and images that quantifies the difference between their VQ encodings. -
Asymptotic Analysis of Complex LASSO Via Complex Approximate Message Passing (CAMP) Arian Maleki, Laura Anitori, Zai Yang, and Richard Baraniuk, Fellow, IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. .., NO. .., .. 1 Asymptotic Analysis of Complex LASSO via Complex Approximate Message Passing (CAMP) Arian Maleki, Laura Anitori, Zai Yang, and Richard Baraniuk, Fellow, IEEE Abstract—Recovering a sparse signal from an undersampled This algorithm is an extension of the AMP algorithm [3], set of random linear measurements is the main problem of [11]. However, the extension of AMP and its analysis from interest in compressed sensing. In this paper, we consider the the real to the complex setting is not trivial; although CAMP case where both the signal and the measurements are complex- shares some interesting features with AMP, it is substantially valued. We study the popular recovery method of `1-regularized least squares or LASSO. While several studies have shown that more challenging to establish the characteristics of CAMP. LASSO provides desirable solutions under certain conditions, the Furthermore, some important features of CAMP are specific to precise asymptotic performance of this algorithm in the complex complex-valued signals and the relevant optimization problem. setting is not yet known. In this paper, we extend the approximate Note that the extension of the Bayesian-AMP algorithm to message passing (AMP) algorithm to solve the complex-valued LASSO problem and obtain the complex approximate message complex-valued signals has been considered elsewhere [12], passing algorithm (CAMP). We then generalize the state evolution [13] and is not the main focus of this work. framework recently introduced for the analysis of AMP to the In the next section, we briefly review some of the existing complex setting. Using the state evolution, we derive accurate algorithms for sparse signal recovery in the real-valued setting formulas for the phase transition and noise sensitivity of both LASSO and CAMP. -
Download The
2018 Year in Review CONNECTOR News from the MIT Department of Electrical Engineering and Computer Science Year in Review: 2018 Connector The MIT Stephen A. Schwarzman College of Rising Stars, an academic-careers workshop Computing — established with a gift from the for women, brought 76 of the world’s top EECS co-founder, CEO, and chairman of Blackstone postdocs and grad students to MIT to hear — is scheduled to open in the fall of 2019. faculty talks, network with each other, and Photo: Courtesy of Blackstone. present their research. Photo: Gretchen Ertl CONTENTS 1 A Letter from the Department Head FACULTY FOCUS FEATURES 47 Faculty Awards 4 MIT Reshapes Itself to Shape the Future: EECS Leadership Update Asu Ozdaglar Introducing the MIT Stephen A. Schwarzman College of Computing 53 Asu Ozdaglar, Department Head Department Head 8 Artificial Intelligence in Action: Introducing the 54 Associate Department Heads MIT-IBM Watson AI Lab 55 Education and Undergraduate Officers Saman Amarasinghe 10 EECS Professor Antonio Torralba Appointed to 56 Faculty Research Innovation Fellowships (FRIFs) Associate Department Head Direct MIT Quest for Intelligence 57 Professorships 12 Summit Explores Pioneering Approaches for AI Nancy Lynch and Digital Technology for Health Care 62 Faculty Promotions Associate Department Head, 14 SuperUROP: Coding, Thinking, Sharing, 64 Tenured Faculty Building Strategic Directions 66 New Faculty 16 SuperUROP: CS+HASS Scholarship Program Debuts with Nine Projects 69 Remembering Professor Alan McWhorter, 1930-2018 Joel Voldman -
2014 Honor Roll of Donors
2014 Honor roll of Donors | 1 When we encourage a promising young person with a bright future, invest in a teacher or a technologist with a solution to a well-defined problem, or support a team that is prepared to tackle a significant challenge with energy, optimism and talent — a world of possibility unfolds. There is nothing more rewarding than turning ideas into action and making a difference in the world; and there is no organization more capable of delivering tangible, impactful results than IEEE. As the philanthropic arm of IEEE, the IEEE foundation inspires the generosity of donors to energize IEEE programs that improve access to technology, enhance technological literacy, and support technical education and the IEEE professional community. The IEEE foundation, a tax-exempt 501(c)(3) organization in the United states, fulfills its purpose by • soliciting and managing donations • recognizing the generosity of our donors • awarding grants to IEEE grassroots projects of strategic importance • supporting high impact signature Programs • serving as a steward of donations that empower bright minds, recognize innovation, preserve the history of technology, and improve the human condition. With your support, and the support of donors worldwide, the IEEE foundation strives to be a leader in transforming lives through the power of technology and education. Table of Contents Donor Profiles 1 Leadership Perspective 19 Students and Young Professionals 21 Nokia Bell Labs 2 Year in Review 20 IEEE Circle of Honor 29 Roberto Padovani 4 Grants Program 22 IEEE Goldsmith Legacy League 31 Thomas E. McDermott 6 IEEE PES Scholarship Plus Initiative 24 IEEE Heritage Circle 33 Gary Hoffman 8 IEEE Smart Village 26 Leadership Donors 35 Francis J. -
Michel Foucault Ronald C Kessler Graham Colditz Sigmund Freud
ANK RESEARCHER ORGANIZATION 1 Michel Foucault Collège de France 2 Ronald C Kessler Harvard University 3 Graham Colditz Washington University in St Louis 4 Sigmund Freud University of Vienna Brigham and Women's Hospital 5 JoAnn E Manson Harvard Medical School 6 Shizuo Akira Osaka University Centre de Sociologie Européenne; 7 Pierre Bourdieu Collège de France Massachusetts Institute of Technology 8 Robert Langer MIT 9 Eric Lander Broad Institute Harvard MIT 10 Bert Vogelstein Johns Hopkins University Brigham and Women's Hospital 11 Eugene Braunwald Harvard Medical School Ecole Polytechnique Fédérale de 12 Michael Graetzel Lausanne 13 Frank B Hu Harvard University 14 Yi Hwa Liu Yale University 15 M A Caligiuri City of Hope National Medical Center 16 Gordon Guyatt McMaster University 17 Salim Yusuf McMaster University 18 Michael Karin University of California San Diego Yale University; Howard Hughes 19 Richard A Flavell Medical Institute 20 T W Robbins University of Cambridge 21 Zhong Lin Wang Georgia Institute of Technology 22 Martín Heidegger Universität Freiburg 23 Paul M Ridker Harvard Medical School 24 Daniel Levy National Institutes of Health NIH 25 Guido Kroemer INSERM 26 Steven A Rosenberg National Institutes of Health NIH Max Planck Institute of Biochemistry; 27 Matthias Mann University of Copenhagen 28 Karl Friston University College London Howard Hughes Medical Institute; Duke 29 Robert J Lefkowitz University 30 Douglas G Altman Oxford University 31 Eric Topol Scripps Research Institute 32 Michael Rutter King's College London 33 -
Signal Processing with Compressive Measurements
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Signal Processing with Compressive Measurements Mark Davenport, Petros Boufounos, Michael Wakin, Richard Baraniuk TR2010-002 February 2010 Abstract The recently introduced theory of compressive sensing enables the recovery of sparse or com- pressive signals from a small set of nonadaptive, linear measurements. If properly chosen, the number of measurements can be much smaller than the number of Nyquist-rate samples. In- terestingly, it has been shown that random projections are a near-optimal measurement scheme. This has inspired the design of hardware systems that directly implement random measurement protocols. However, despite the intense focus of the community on signal recovery, many (if not most) signal processing problems do not require full signal recovery. In this paper, we take some first steps in the direction of solving inference problems - such as detection, classifica- tion, or estimation - and filtering problems using only compressive measurements and without ever reconstructing the signals involved. We provide theoretical bounds along with experimental results. Journal of Special Topics in Signal Processing This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. -
2010 IEEE International Conference on Acoustics, Speech, and Signal Processing
2010 IEEE International Conference on Acoustics, Speech, and Signal Processing Proceedings March 14–19, 2010 Sheraton Dallas Hotel Dallas, Texas, U.S.A. Sponsored by The Institute of Electrical and Electronics Engineers Signal Processing Society IEEE Catalog Number: CFP10ICA ISBN: 978-1-4244-4296-6 Copyright ©2010 by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Copyright and Reprint Permission: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the limit of U.S. copyright law for private use of patrons those articles in this volume that carry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923. For other copying, reprint or republication permission, write to IEEE Copyrights Manager, IEEE Operations Center, 445 Hoes Lane, P.O. Box 1331, Piscataway, NJ 08855-1331. All rights reserved. Copyright ©2010 by the Institute of Electrical and Electronics Engineers, Inc. The papers in this book comprise the proceedings of the meeting mentioned on the cover and title page. They reflect the authors’ opinions and, in the interests of timely dissemination, are published as presented and without change. Their inclusion in this publication does not necessarily constitute endorsement by the editors, the IEEE Signal Processing Society, or the Institute of Electrical and Electronics Engineers, Inc. IEEE is committed to the principle that all persons shall have equal access to programs, facilities, services, and employment without regard to personal characteristics not related to ability, performance, or qualifications as determined by IEEE policy and/or applicable laws. -
Compressive Sensing 1 Scope 2 Relevance 3 Prerequisites 4
Compressive Sensing Richard Baraniuk Rice University [Lecture Notes in IEEE Signal Processing Magazine] Volume 24, July 2007 1 Scope The Shannon/Nyquist sampling theorem tells us that in order to not lose information when uni- formly sampling a signal we must sample at least two times faster than its bandwidth. In many applications, including digital image and video cameras, the Nyquist rate can be so high that we end up with too many samples and must compress in order to store or transmit them. In other ap- plications, including imaging systems (medical scanners, radars) and high-speed analog-to-digital converters, increasing the sampling rate or density beyond the current state-of-the-art is very ex- pensive. In this lecture, we will learn about a new technique that tackles these issues using compressive sensing [1, 2]. We will replace the conventional sampling and reconstruction operations with a more general linear measurement scheme coupled with an optimization in order to acquire certain kinds of signals at a rate significantly below Nyquist. 2 Relevance The ideas presented here can be used to illustrate the links between data acquisition, linear algebra, basis expansions, inverse problems, compression, dimensionality reduction, and optimization in a variety of courses, from undergraduate or graduate digital signal processing to statistics and applied mathematics. 3 Prerequisites The prerequisites required for understanding and teaching this material are linear algebra, basic optimization, and basic probability. 4 Problem Statement Nyquist-rate sampling completely describes a signal by exploiting its bandlimitedness. Our goal is to reduce the number of measurements required to completely describe a signal by exploiting its compressibility.