IEEE Information Theory Society Newsletter
Total Page:16
File Type:pdf, Size:1020Kb
IEEE Information Theory Society Newsletter Vol. 67, No. 2, June 2017 EDITOR: Michael Langberg, Ph.D. ISSN 1059-2362 Editorial committee: Alexander Barg, Giuseppe Caire, Meir Feder, Joerg Kliewer, Frank Kschischang, Prakash Narayan, Parham Noorzad, Anand Sarwate, Andy Singer, and Sergio Verdú. President’s Column Rüdiger Urbanke Some of you might have noticed a monthly itsoc.org/news-events/recent-news/call-for- email from ITSOC in your mail box with the short-video-proposal table of contents of the upcoming IT Transac- tions (if not, check you spam filter and make Talking about movies. Mark Levinson, the di- sure you paid your dues!). This was suggest- rector of the Shannon movie, has started edit- ed by Prakash Narayan, our Editor-in-Chief. ing the material from the February shoot and Thanks also to Elza Erkip, our First Vice Presi- is making plans for additional shooting of dent, for her support, and to Anand Sarwate, flashback scenes and interviews. He has also the Chair of the Online Committee, for posting begun speaking to a composer and graphics this information on our web page. We hope people. The role of Shannon has already been that you like the idea. The early feedback has cast—sorry, you can stop practicing juggling. been very positive. But we can do even better. But if you have ideas about how best to ex- Dave Forney has suggested that we bring the plain central concepts of information theory format into the 21st century and perhaps com- to a large audience, let us know. bine with regular communication to our mem- bers. After playing around with various HTML Unfortunately, I also have to bring you very versions and mailers all I can say: Who knew that spamming sad news. Mary Elizabeth (Betty) Moore Shannon, the re- could be so complicated! Please keep the comments coming. markable wife of Claude, has died. You can find a reprint of the obituary that has appeared in the Boston Globe and online On a related note, if you are looking for papers from our com- in the New York Times in this Newsletter. munity make sure that once in a while you check out Xplore. And if you are writing a paper ensure that you also include Besides the Shannon movie, a Shannon biography, authored the reference to the final version of the paper. This is a small by Jimmy Soni and Rob Goodman, has just been published. step for any author, but has potentially a large impact for our You can order it at https://www.amazon.com/Mind-Play- society since it will ensure that our click rates adequately re- Shannon-Invented-Information/dp/1476766681 flect our work and contributions. Read it, and if you enjoy it, recommend it to your friends, like As part of the Shannon Centennial Celebration, the IEEE In- it, and up-vote it. An enthusiastic support by our community formation Theory Society launched last year a pilot project to will help push the book onto bestseller lists, boosting our out- create short videos. The objective was to showcase intriguing reach efforts. results from Information Theory (in a broad sense) to a large non-expert audience. Two videos, one on Network Coding And if you want to do more, follow Christina Fragouli’s sug- and one on Space Time codes, have almost been completed. gestion. Start your Christmas shopping early this year and get a We have decided to extend this project and we are now look- few copies for your loved ones or anybody who is digitally chal- ing for volunteers to help define and create the next three vid- lenged but not entirely hopeless. Want more ideas? How about eos! If there is a concept or an idea, rooted in information the- donating a copy to your local library or placing a copy in the ory, that you think the whole world should know about, this waiting room of a train station or dentist office. You never know is your chance. A call for video proposal has been released where the next Claude will pass by, looking for inspiration. on the ITSOC website with more information. We look for- ward to your creative ideas and suggestions. http://www. (continued on page 10) 2 From the Editor Michael Langberg Dear colleagues, fellow colleagues for their outstanding research accom plish- ments and service recognized by our community. Spe- This issue opens with an intriguing survey cifically, congratulations to Ido Tal and Alexander Vardy article by Adam Smith, “Information, on their 2017 IEEE Communications Society and Informa- Privacy and Stability in Adaptive Data tion Theory Society award winning paper “List Decoding Analysis,” addressing the subtleties in of Polar Codes,” and to the members of our society that data analytics when collected data is re- have recently been named Distinguished Lecturers. We used. The survey connects the study of continue with a special “Behind the scenes Q&A” with adaptive data analytics with certain in- the authors of the upcoming Claude Shannon biography. formation measures and measures of dis- Thanks to authors Jimmy Soni and Rob Goodman for pre- tributional stability. The latter, termed paring the Q&A and for pursuing this impressive and im- “differential privacy,” has seen a sig- portant project. nificant amount of interest lately and has recently awarded Adam Smith (together We conclude with Tony Ephremides’s Historian’s column; our “Students’ Corner’’ with Cynthia Dwork, Frank McSherry, column, by Mine Alsan and Basak Güler, presenting the recent initiatives of the In- and Kobbi Nissim) with the prestigious formation Theory Student Subcommittee; and a note by Emanuele Viterbo and Elza Gödel Prize on their outstanding work Erkip, “Thinking about organizing an ITW or ISIT?,” outlining the ins and outs of introducing differential privacy. Many organizing a workshop or conference in our society. Thanks to the contributors for thanks to Adam Smith for his generous their efforts. willingness and significant efforts in pre- paring the survey. Continuing our remembrance and honoring of Sol Golomb, an extraordinary scholar and long time newsletter contributor, the fourth and final collection of Sol’s earlier The issue continues with a number of newsletter puzzle columns appears in this issue. This last collection includes solu- announcements, regular columns, and tions to the second collection of puzzles published in the December 2016 issue of the reports. We start by congratulating our newsletter. continued on page 15 IEEE Information Theory Table of Contents Society Newsletter President’s Column. 1 IEEE Information Theory Society Newsletter From the Editor . 2 (USPS 360-350) is published quarterly by the Information Theory Society of the Institute of Information, Privacy and Stability in Adaptive Data Analysis ...................3 Electrical and Electronics Engineers, Inc. IEEE Communications Society and Information Headquarters: 3 Park Avenue, 17th Floor, New York, NY 10016-5997. Theory Society Joint Paper Award for 2017 .................................. 11 Cost is $1.00 per member per year (included IEEE Information Theory Society’s New Distinguished Lecturers...............11 in Society fee) for each member of the Information Theory Society. Printed in the Behind the Scenes: A Q&A with the Authors of Claude U.S.A. Periodicals postage paid at New York, NY and at additional mailing offices. Shannon’s New Biography ................................................ 11 Postmaster: Send address changes to IEEE The Historian’s Column...................................................13 Information Theory Society Newsletter, IEEE, 445 Hoes Lane, Piscataway, NJ 08854. Students’ Corner: The Information Theory Student © 2016 IEEE. Information contained in this Subcommittee Goes Social! ................................................ 14 newsletter may be copied without permis- sion provided that the copies are not made or Thinking About Organizing an ITW or ISIT?.................................16 distributed for direct commercial advantage, and the title of the publication and its date Golomb’s Puzzle Column™ Collection, Part 4 ...............................17 appear. In Memoriam: Mary Elizebeth (Betty) Moore Shannon . 41 IEEE prohibits discrimination, harassment, and bullying. For more information, visit http://www.ieee.org/web/aboutus/ Nominations Sought for 2018 and 2019 Leaders ..............................41 whatis/policies/p9-26.html. Recent Publications.......................................................42 Promoting Sustainable Forestry Call for Papers . 47 SFI-01681 Conference Calendar . 52 IEEE Information Theory Society Newsletter June 2017 3 Information, Privacy and Stability in Adaptive Data Analysis Adam Smith* Abstract To formalize our situation somewhat, imagine there is a popula- tion that we wish to study, modeled by a probability distribution P. Traditional statistical theory assumes that the analysis to be per- An analyst selects a sequence of analyses M1, M2, ..., that she wishes formed on a given data set is selected independently of the data to perform (we specify the type of analysis to consider later). In an themselves. This assumption breaks downs when data are re-used ideal world (Figure 1, left), the analyst would run each analysis Mi (i) across analyses and the analysis to be performed at a given stage on a fresh sample X from the population. For simplicity, we only depends on the results of earlier stages. Such dependency can arise discuss i.i.d samples in this article; we may assume that each sample (i) when the same data are used by several scientific studies, or when a X has n points, drawn independently from P. In the real (adaptive) single analysis consists of multiple stages. setting, the same data set X gets used for each analysis (Figure 1, right). The challenge is that we ultimately want to learn about P, How can we draw statistically valid conclusions when data are re- not X, but adaptive queries can quickly overfit to X. used? This is the focus of a recent and active line of work. At a high level, these results show that limiting the information revealed by Our goal is to relate these two settings—to develop techniques that earlier stages of analysis controls the bias introduced in later stages allow us to emulate the ideal world in the real one, and understand by adaptivity.