IEEE Information Theory Society Newsletter
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Bernard M. Oliver Oral History Interview
http://oac.cdlib.org/findaid/ark:/13030/kt658038wn Online items available Bernard M. Oliver Oral History Interview Daniel Hartwig Stanford University. Libraries.Department of Special Collections and University Archives Stanford, California November 2010 Copyright © 2015 The Board of Trustees of the Leland Stanford Junior University. All rights reserved. Note This encoded finding aid is compliant with Stanford EAD Best Practice Guidelines, Version 1.0. Bernard M. Oliver Oral History SCM0111 1 Interview Overview Call Number: SCM0111 Creator: Oliver, Bernard M., 1916- Title: Bernard M. Oliver oral history interview Dates: 1985-1986 Physical Description: 0.02 Linear feet (1 folder) Summary: Transcript of an interview conducted by Arthur L. Norberg covering Oliver's early life, his education, and work experiences at Bell Laboratories and Hewlett-Packard. Subjects include television research, radar, information theory, organizational climate and objectives at both companies, Hewlett-Packard's associations with Stanford University, and Oliver's association with William Hewlett and David Packard. Language(s): The materials are in English. Repository: Department of Special Collections and University Archives Green Library 557 Escondido Mall Stanford, CA 94305-6064 Email: [email protected] Phone: (650) 725-1022 URL: http://library.stanford.edu/spc Information about Access Reproduction is prohibited. Ownership & Copyright All requests to reproduce, publish, quote from, or otherwise use collection materials must be submitted in writing to the Head of Special Collections and University Archives, Stanford University Libraries, Stanford, California 94304-6064. Consent is given on behalf of Special Collections as the owner of the physical items and is not intended to include or imply permission from the copyright owner. -
Group Testing
Group Testing Amit Kumar Sinhababu∗ and Vikraman Choudhuryy Department of Computer Science and Engineering, Indian Institute of Technology Kanpur April 21, 2013 1 Motivation Out original motivation in this project was to study \coding theory in data streaming", which has two aspects. • Applications of theory correcting codes to efficiently solve problems in the model of data streaming. • Solving coding theory problems in the model of data streaming. For ex- ample, \Can one recognize a Reed-Solomon codeword in one-pass using only poly-log space?" [1] As we started, we were directed to a related combinatorial problem, \Group testing", which is important on its own, having connections with \Compressed Sensing", \Data Streaming", \Coding Theory", \Expanders", \Derandomiza- tion". This project report surveys some of these interesting connections. 2 Group Testing The group testing problem is to identify the set of \positives" (\defectives", or \infected", or 1) from a large set of population/items, using as few tests as possible. ∗[email protected] [email protected] 1 2.1 Definition There is an unknown stream x 2 f0; 1gn with at most d ones in it. We are allowed to test any subset S of the indices. The answer to the test tells whether xi = 0 for all i 2 S, or not (at least one xi = 1). The objective is to design as few tests as possible (t tests) such that x can be identified as fast as possible. Group testing strategies can be either adaptive or non-adaptive. A group testing algorithm is non-adaptive if all tests must be specified without knowing the outcome of other tests. -
DRASIC Distributed Recurrent Autoencoder for Scalable
DRASIC: Distributed Recurrent Autoencoder for Scalable Image Compression Enmao Diao∗, Jie Dingy, and Vahid Tarokh∗ ∗Duke University yUniversity of Minnesota-Twin Cities Durham, NC, 27701, USA Minneapolis, MN 55455, USA [email protected] [email protected] [email protected] Abstract We propose a new architecture for distributed image compression from a group of distributed data sources. The work is motivated by practical needs of data-driven codec design, low power con- sumption, robustness, and data privacy. The proposed architecture, which we refer to as Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC), is able to train distributed encoders and one joint decoder on correlated data sources. Its compression capability is much bet- ter than the method of training codecs separately. Meanwhile, the performance of our distributed system with 10 distributed sources is only within 2 dB peak signal-to-noise ratio (PSNR) of the performance of a single codec trained with all data sources. We experiment distributed sources with different correlations and show how our data-driven methodology well matches the Slepian- Wolf Theorem in Distributed Source Coding (DSC). To the best of our knowledge, this is the first data-driven DSC framework for general distributed code design with deep learning. 1 Introduction It has been shown by a variety of previous works that deep neural networks (DNN) can achieve comparable results as classical image compression techniques [1–9]. Most of these methods are based on autoencoder networks and quantization of bottleneck representa- tions. These models usually rely on entropy codec to further compress codes. Moreover, to achieve different compression rates it is unavoidable to train multiple models with different regularization parameters separately, which is often computationally intensive. -
James Massey Memorial Service
On June 16, 2013, JaImens L. MRassey, peassmed aweaym at hibs hormea in Cnopcenheagen, Denmark. In recognition of Jim’s great service to and love for Notre Dame, the College of Engineering invites you to join us in honoring him. A memorial Mass will be held on Friday, November 1, at 4:00 p.m. in the Holy Cross Chapel of the Stinson-Remick Hall of Engineering. It will be followed by a reception in the Stinson-Remick atrium. The Mass will be celebrated by the Rev. Edward A. Malloy, C.S.C. , President Emeritus of the University of Notre Dame, and concelebrated by the Rev. Theodore M. Hesburgh , President Emeritus of the University of Notre Dame. All are invited to attend this special event in Jim's honor. So that we can properly plan for the reception, please RSVP to Michele Tharp at [email protected] by Friday, October 4, if you plan to attend. The College of Engineering is also launching a graduate fellowship in electrical engineering in Jim’s name. Tax deductible contributions can be made here and designated for the “James L. Massey Graduate Fellowship in Electrical Engineering.” Questions regarding the fund Department of Electrical Engineering can be directed to Nathan Utz, academic advancement director for , - the College of Engineering, [email protected] . We hope you will Department of Electrical Engineering consider making a generous contribution to this worthy cause. Finally, if you know of other individuals at Notre Dame, in the South Bend area, or in the wider community who knew Jim and The Frank M. -
Mathematics of Data Science
SIAM JOURNAL ON Mathematics of Data Science Volume 2 • 2020 Editor-in-Chief Tamara G. Kolda, Sandia National Laboratories Section Editors Mark Girolami, University of Cambridge, UK Alfred Hero, University of Michigan, USA Robert D. Nowak, University of Wisconsin, Madison, USA Joel A. Tropp, California Institute of Technology, USA Associate Editors Maria-Florina Balcan, Carnegie Mellon University, USA Vianney Perchet, ENSAE, CRITEO, France Rina Foygel Barber, University of Chicago, USA Jonas Peters, University of Copenhagen, Denmark Mikhail Belkin, University of California, San Diego, USA Natesh Pillai, Harvard University, USA Robert Calderbank, Duke University, USA Ali Pinar, Sandia National Laboratories, USA Coralia Cartis, University of Oxford, UK Mason Porter, University of Califrornia, Los Angeles, USA Venkat Chandrasekaran, California Institute of Technology, Maxim Raginsky, University of Illinois, USA Urbana-Champaign, USA Patrick L. Combettes, North Carolina State University, USA Bala Rajaratnam, University of California, Davis, USA Alexandre d’Aspremont, CRNS, Ecole Normale Superieure, Philippe Rigollet, MIT, USA France Justin Romberg, Georgia Tech, USA Ioana Dumitriu, University of California, San Diego, USA C. Seshadhri, University of California, Santa Cruz, USA Maryam Fazel, University of Washington, USA Amit Singer, Princeton University, USA David F. Gleich, Purdue University, USA Marc Teboulle, Tel Aviv University, Israel Wouter Koolen, CWI, the Netherlands Caroline Uhler, MIT, USA Gitta Kutyniok, University of Munich, Germany -
Claude Elwood Shannon (1916–2001) Solomon W
Claude Elwood Shannon (1916–2001) Solomon W. Golomb, Elwyn Berlekamp, Thomas M. Cover, Robert G. Gallager, James L. Massey, and Andrew J. Viterbi Solomon W. Golomb Done in complete isolation from the community of population geneticists, this work went unpublished While his incredibly inventive mind enriched until it appeared in 1993 in Shannon’s Collected many fields, Claude Shannon’s enduring fame will Papers [5], by which time its results were known surely rest on his 1948 work “A mathematical independently and genetics had become a very theory of communication” [7] and the ongoing rev- different subject. After his Ph.D. thesis Shannon olution in information technology it engendered. wrote nothing further about genetics, and he Shannon, born April 30, 1916, in Petoskey, Michi- expressed skepticism about attempts to expand gan, obtained bachelor’s degrees in both mathe- the domain of information theory beyond the matics and electrical engineering at the University communications area for which he created it. of Michigan in 1936. He then went to M.I.T., and Starting in 1938 Shannon worked at M.I.T. with after spending the summer of 1937 at Bell Tele- Vannevar Bush’s “differential analyzer”, the an- phone Laboratories, he wrote one of the greatest cestral analog computer. After another summer master’s theses ever, published in 1938 as “A sym- (1940) at Bell Labs, he spent the academic year bolic analysis of relay and switching circuits” [8], 1940–41 working under the famous mathemati- in which he showed that the symbolic logic of cian Hermann Weyl at the Institute for Advanced George Boole’s nineteenth century Laws of Thought Study in Princeton, where he also began thinking provided the perfect mathematical model for about recasting communications on a proper switching theory (and indeed for the subsequent mathematical foundation. -
Digital Communication Systems 2.2 Optimal Source Coding
Digital Communication Systems EES 452 Asst. Prof. Dr. Prapun Suksompong [email protected] 2. Source Coding 2.2 Optimal Source Coding: Huffman Coding: Origin, Recipe, MATLAB Implementation 1 Examples of Prefix Codes Nonsingular Fixed-Length Code Shannon–Fano code Huffman Code 2 Prof. Robert Fano (1917-2016) Shannon Award (1976 ) Shannon–Fano Code Proposed in Shannon’s “A Mathematical Theory of Communication” in 1948 The method was attributed to Fano, who later published it as a technical report. Fano, R.M. (1949). “The transmission of information”. Technical Report No. 65. Cambridge (Mass.), USA: Research Laboratory of Electronics at MIT. Should not be confused with Shannon coding, the coding method used to prove Shannon's noiseless coding theorem, or with Shannon–Fano–Elias coding (also known as Elias coding), the precursor to arithmetic coding. 3 Claude E. Shannon Award Claude E. Shannon (1972) Elwyn R. Berlekamp (1993) Sergio Verdu (2007) David S. Slepian (1974) Aaron D. Wyner (1994) Robert M. Gray (2008) Robert M. Fano (1976) G. David Forney, Jr. (1995) Jorma Rissanen (2009) Peter Elias (1977) Imre Csiszár (1996) Te Sun Han (2010) Mark S. Pinsker (1978) Jacob Ziv (1997) Shlomo Shamai (Shitz) (2011) Jacob Wolfowitz (1979) Neil J. A. Sloane (1998) Abbas El Gamal (2012) W. Wesley Peterson (1981) Tadao Kasami (1999) Katalin Marton (2013) Irving S. Reed (1982) Thomas Kailath (2000) János Körner (2014) Robert G. Gallager (1983) Jack KeilWolf (2001) Arthur Robert Calderbank (2015) Solomon W. Golomb (1985) Toby Berger (2002) Alexander S. Holevo (2016) William L. Root (1986) Lloyd R. Welch (2003) David Tse (2017) James L. -
Principles of Communications ECS 332
Principles of Communications ECS 332 Asst. Prof. Dr. Prapun Suksompong (ผศ.ดร.ประพันธ ์ สขสมปองุ ) [email protected] 1. Intro to Communication Systems Office Hours: Check Google Calendar on the course website. Dr.Prapun’s Office: 6th floor of Sirindhralai building, 1 BKD 2 Remark 1 If the downloaded file crashed your device/browser, try another one posted on the course website: 3 Remark 2 There is also three more sections from the Appendices of the lecture notes: 4 Shannon's insight 5 “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.” Shannon, Claude. A Mathematical Theory Of Communication. (1948) 6 Shannon: Father of the Info. Age Documentary Co-produced by the Jacobs School, UCSD- TV, and the California Institute for Telecommunic ations and Information Technology 7 [http://www.uctv.tv/shows/Claude-Shannon-Father-of-the-Information-Age-6090] [http://www.youtube.com/watch?v=z2Whj_nL-x8] C. E. Shannon (1916-2001) Hello. I'm Claude Shannon a mathematician here at the Bell Telephone laboratories He didn't create the compact disc, the fax machine, digital wireless telephones Or mp3 files, but in 1948 Claude Shannon paved the way for all of them with the Basic theory underlying digital communications and storage he called it 8 information theory. C. E. Shannon (1916-2001) 9 https://www.youtube.com/watch?v=47ag2sXRDeU C. E. Shannon (1916-2001) One of the most influential minds of the 20th century yet when he died on February 24, 2001, Shannon was virtually unknown to the public at large 10 C. -
MIMO Wireless Communications Ezio Biglieri, Robert Calderbank, Anthony Constantinides, Andrea Goldsmith, Arogyaswami Paulraj and H
Cambridge University Press 978-0-521-13709-6 - MIMO Wireless Communications Ezio Biglieri, Robert Calderbank, Anthony Constantinides, Andrea Goldsmith, Arogyaswami paulraj and H. Vincent Poor Frontmatter More information MIMO Wireless Communications Multiple-input multiple-output (MIMO) technology constitutes a breakthrough in the design of wireless communication systems, and is already at the core of several wireless standards. Exploiting multi-path scattering, MIMO techniques deliver significant performance enhancements in terms of data transmission rate and interference reduction. This book is a detailed introduction to the analysis and design of MIMO wireless systems. Beginning with an overview of MIMO technology, the authors then examine the fundamental capacity limits of MIMO systems. Transmitter design, including precoding and space–time coding, is then treated in depth, and the book closes with two chapters devoted to receiver design. Written by a team of leading experts, the book blends theoretical analysis with physical insights, and highlights a range of key design challenges. It can be used as a textbook for advanced courses on wireless communications, and will also appeal to researchers and practitioners working on MIMO wireless systems. Ezio Biglieri is a professor in the Department of Technology at the Universitat Pompeu Fabra, Barcelona. Robert Calderbank is a professor in the Departments of Electrical Engineering and Mathematics at Princeton University, New Jersey. Anthony Constantinides is a professor in the Department of Electrical and Electronic Engineering at Imperial College of Science, Technology and Medicine, London. Andrea Goldsmith is a professor in the Department of Electrical Engineering at Stanford University, California. Arogyaswami Paulraj is a professor in the Department of Electrical Engineering at Stanford University, California. -
IEEE Information Theory Society Newsletter
IEEE Information Theory Society Newsletter Vol. 53, No.4, December 2003 Editor: Lance C. Pérez ISSN 1059-2362 The Shannon Lecture Hidden Markov Models and the Baum-Welch Algorithm Lloyd R. Welch Content of This Talk what the ‘running variable’ is. The lectures of previous Shannon Lecturers fall into several Of particular use will be the concept of conditional probabil- categories such as introducing new areas of research, resusci- ity and recursive factorization. The recursive factorization tating areas of research, surveying areas identified with the idea says that the joint probability of a collection of events can lecturer, or reminiscing on the career of the lecturer. In this be expressed as a product of conditional probabilities, where talk I decided to restrict the subject to the Baum-Welch “algo- each is the probability of an event conditioned on all previous rithm” and some of the ideas that led to its development. events. For example, let A, B, and C be three events. Then I am sure that most of you are familiar with Markov chains Pr(A ∩ B ∩ C) = Pr(A)Pr(B | A)Pr(C | A ∩ B) and Markov processes. They are natural models for various communication channels in which channel conditions change Using the bracket notation, we can display the recursive fac- with time. In many cases it is not the state sequence of the torization of the joint probability distribution of a sequence of model which is observed but the effects of the process on a discrete random variables: signal. That is, the states are not observable but some func- tions, possibly random, of the states are observed. -
Nested Tailbiting Convolutional Codes for Secrecy, Privacy, and Storage
Nested Tailbiting Convolutional Codes for Secrecy, Privacy, and Storage Thomas Jerkovits Onur Günlü Vladimir Sidorenko [email protected] [email protected] Gerhard Kramer German Aerospace Center TU Berlin [email protected] Weçling, Germany Berlin, Germany [email protected] TU Munich Munich, Germany ABSTRACT them as physical “one-way functions” that are easy to compute and A key agreement problem is considered that has a biometric or difficult to invert [33]. physical identifier, a terminal for key enrollment, and a terminal There are several security, privacy, storage, and complexity con- for reconstruction. A nested convolutional code design is proposed straints that a PUF-based key agreement method should fulfill. First, that performs vector quantization during enrollment and error the method should not leak information about the secret key (neg- control during reconstruction. Physical identifiers with small bit ligible secrecy leakage). Second, the method should leak as little error probability illustrate the gains of the design. One variant of information about the identifier (minimum privacy leakage). The the nested convolutional codes improves on the best known key privacy leakage constraint can be considered as an upper bound vs. storage rate ratio but it has high complexity. A second variant on the secrecy leakage via the public information of the first en- with lower complexity performs similar to nested polar codes. The rollment of a PUF about the secret key generated by the second results suggest that the choice of code for key agreement with enrollment of the same PUF [12]. Third, one should limit the stor- identifiers depends primarily on the complexity constraint. -
MASTER of ADVANCED STUDY New Professional Degrees for Engineers University of California, San Diego of California, University
pulse cover12_Layout 1 6/22/11 3:46 PM Page 1 Entrepreneurism Center • Research Expo 2011 In Memory of Jack Wolf Jacobs School of Engineering News PulseSummer 2011 MASTER OF ADVANCED STUDY New Professional Degrees for Engineers University of California, San Diego of California, University > dean’s column < New Interdisciplinary Degree Programs for Engineering Professionals Jacobs School of Engineering The most exciting and innovative engineering often occurs on the interface between traditional disciplines. We are extending our interdisciplinary Leadership Dean: Frieder Seible collaborations — which have always been at the core of the Jacobs School culture Associate Dean: Jeanne Ferrante — to new graduate education programs for engineering professionals. Associate Dean: Charles Tu Associate Dean for Administration and Finance: Beginning this fall, the Jacobs School will offer four new interdisciplinary Steve Ross Master of Advanced Study (MAS) programs for working engineers: Wireless Executive Director of External Relations: Embedded Systems, Medical Device Engineering, Structural Health Monitoring, Denine Hagen and Simulation-Based Engineering. Academic Departments Bioengineering: Shankar Subramanian, Chair TThese master degree programs are engineering equivalents of MBA programs Computer Science and Engineering: at business management schools. Geared to early- to mid-career engineers Rajesh Gupta, Chair Electrical and Computer Engineering: with practical work experience, our new MAS programs align faculty research Yeshaiahu Fainman, Chair strengths with industry workforce needs. The curricula are always jointly offered Mechanical and Aerospace Engineering: by two academic departments, so that the training focuses in a practical way on Sutanu Sarkar, Chair NanoEngineering: industry-specific application areas that are not available through traditional master Kenneth Vecchio, Chair degree programs.