Information Theory Workshop (ITW) September 3-7, 2012 Lausanne - Switzerland
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Digital Communication Systems 2.2 Optimal Source Coding
Digital Communication Systems EES 452 Asst. Prof. Dr. Prapun Suksompong [email protected] 2. Source Coding 2.2 Optimal Source Coding: Huffman Coding: Origin, Recipe, MATLAB Implementation 1 Examples of Prefix Codes Nonsingular Fixed-Length Code Shannon–Fano code Huffman Code 2 Prof. Robert Fano (1917-2016) Shannon Award (1976 ) Shannon–Fano Code Proposed in Shannon’s “A Mathematical Theory of Communication” in 1948 The method was attributed to Fano, who later published it as a technical report. Fano, R.M. (1949). “The transmission of information”. Technical Report No. 65. Cambridge (Mass.), USA: Research Laboratory of Electronics at MIT. Should not be confused with Shannon coding, the coding method used to prove Shannon's noiseless coding theorem, or with Shannon–Fano–Elias coding (also known as Elias coding), the precursor to arithmetic coding. 3 Claude E. Shannon Award Claude E. Shannon (1972) Elwyn R. Berlekamp (1993) Sergio Verdu (2007) David S. Slepian (1974) Aaron D. Wyner (1994) Robert M. Gray (2008) Robert M. Fano (1976) G. David Forney, Jr. (1995) Jorma Rissanen (2009) Peter Elias (1977) Imre Csiszár (1996) Te Sun Han (2010) Mark S. Pinsker (1978) Jacob Ziv (1997) Shlomo Shamai (Shitz) (2011) Jacob Wolfowitz (1979) Neil J. A. Sloane (1998) Abbas El Gamal (2012) W. Wesley Peterson (1981) Tadao Kasami (1999) Katalin Marton (2013) Irving S. Reed (1982) Thomas Kailath (2000) János Körner (2014) Robert G. Gallager (1983) Jack KeilWolf (2001) Arthur Robert Calderbank (2015) Solomon W. Golomb (1985) Toby Berger (2002) Alexander S. Holevo (2016) William L. Root (1986) Lloyd R. Welch (2003) David Tse (2017) James L. -
Principles of Communications ECS 332
Principles of Communications ECS 332 Asst. Prof. Dr. Prapun Suksompong (ผศ.ดร.ประพันธ ์ สขสมปองุ ) [email protected] 1. Intro to Communication Systems Office Hours: Check Google Calendar on the course website. Dr.Prapun’s Office: 6th floor of Sirindhralai building, 1 BKD 2 Remark 1 If the downloaded file crashed your device/browser, try another one posted on the course website: 3 Remark 2 There is also three more sections from the Appendices of the lecture notes: 4 Shannon's insight 5 “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.” Shannon, Claude. A Mathematical Theory Of Communication. (1948) 6 Shannon: Father of the Info. Age Documentary Co-produced by the Jacobs School, UCSD- TV, and the California Institute for Telecommunic ations and Information Technology 7 [http://www.uctv.tv/shows/Claude-Shannon-Father-of-the-Information-Age-6090] [http://www.youtube.com/watch?v=z2Whj_nL-x8] C. E. Shannon (1916-2001) Hello. I'm Claude Shannon a mathematician here at the Bell Telephone laboratories He didn't create the compact disc, the fax machine, digital wireless telephones Or mp3 files, but in 1948 Claude Shannon paved the way for all of them with the Basic theory underlying digital communications and storage he called it 8 information theory. C. E. Shannon (1916-2001) 9 https://www.youtube.com/watch?v=47ag2sXRDeU C. E. Shannon (1916-2001) One of the most influential minds of the 20th century yet when he died on February 24, 2001, Shannon was virtually unknown to the public at large 10 C. -
IEEE Information Theory Society Newsletter
IEEE Information Theory Society Newsletter Vol. 53, No.4, December 2003 Editor: Lance C. Pérez ISSN 1059-2362 The Shannon Lecture Hidden Markov Models and the Baum-Welch Algorithm Lloyd R. Welch Content of This Talk what the ‘running variable’ is. The lectures of previous Shannon Lecturers fall into several Of particular use will be the concept of conditional probabil- categories such as introducing new areas of research, resusci- ity and recursive factorization. The recursive factorization tating areas of research, surveying areas identified with the idea says that the joint probability of a collection of events can lecturer, or reminiscing on the career of the lecturer. In this be expressed as a product of conditional probabilities, where talk I decided to restrict the subject to the Baum-Welch “algo- each is the probability of an event conditioned on all previous rithm” and some of the ideas that led to its development. events. For example, let A, B, and C be three events. Then I am sure that most of you are familiar with Markov chains Pr(A ∩ B ∩ C) = Pr(A)Pr(B | A)Pr(C | A ∩ B) and Markov processes. They are natural models for various communication channels in which channel conditions change Using the bracket notation, we can display the recursive fac- with time. In many cases it is not the state sequence of the torization of the joint probability distribution of a sequence of model which is observed but the effects of the process on a discrete random variables: signal. That is, the states are not observable but some func- tions, possibly random, of the states are observed. -
Information Theory and Statistics: a Tutorial
Foundations and Trends™ in Communications and Information Theory Volume 1 Issue 4, 2004 Editorial Board Editor-in-Chief: Sergio Verdú Department of Electrical Engineering Princeton University Princeton, New Jersey 08544, USA [email protected] Editors Venkat Anantharam (Berkeley) Amos Lapidoth (ETH Zurich) Ezio Biglieri (Torino) Bob McEliece (Caltech) Giuseppe Caire (Eurecom) Neri Merhav (Technion) Roger Cheng (Hong Kong) David Neuhoff (Michigan) K.C. Chen (Taipei) Alon Orlitsky (San Diego) Daniel Costello (NotreDame) Vincent Poor (Princeton) Thomas Cover (Stanford) Kannan Ramchandran (Berkeley) Anthony Ephremides (Maryland) Bixio Rimoldi (EPFL) Andrea Goldsmith (Stanford) Shlomo Shamai (Technion) Dave Forney (MIT) Amin Shokrollahi (EPFL) Georgios Giannakis (Minnesota) Gadiel Seroussi (HP-Palo Alto) Joachim Hagenauer (Munich) Wojciech Szpankowski (Purdue) Te Sun Han (Tokyo) Vahid Tarokh (Harvard) Babak Hassibi (Caltech) David Tse (Berkeley) Michael Honig (Northwestern) Ruediger Urbanke (EPFL) Johannes Huber (Erlangen) Steve Wicker (GeorgiaTech) Hideki Imai (Tokyo) Raymond Yeung (Hong Kong) Rodney Kennedy (Canberra) Bin Yu (Berkeley) Sanjeev Kulkarni (Princeton) Editorial Scope Foundations and Trends™ in Communications and Information Theory will publish survey and tutorial articles in the following topics: • Coded modulation • Multiuser detection • Coding theory and practice • Multiuser information theory • Communication complexity • Optical communication channels • Communication system design • Pattern recognition and learning • Cryptology -
IEEE Information Theory Society Newsletter
IEEE Information Theory Society Newsletter Vol. 63, No. 3, September 2013 Editor: Tara Javidi ISSN 1059-2362 Editorial committee: Ioannis Kontoyiannis, Giuseppe Caire, Meir Feder, Tracey Ho, Joerg Kliewer, Anand Sarwate, Andy Singer, and Sergio Verdú Annual Awards Announced The main annual awards of the • 2013 IEEE Jack Keil Wolf ISIT IEEE Information Theory Society Student Paper Awards were were announced at the 2013 ISIT selected and announced at in Istanbul this summer. the banquet of the Istanbul • The 2014 Claude E. Shannon Symposium. The winners were Award goes to János Körner. the following: He will give the Shannon Lecture at the 2014 ISIT in 1) Mohammad H. Yassaee, for Hawaii. the paper “A Technique for Deriving One-Shot Achiev - • The 2013 Claude E. Shannon ability Results in Network Award was given to Katalin János Körner Daniel Costello Information Theory”, co- Marton in Istanbul. Katalin authored with Mohammad presented her Shannon R. Aref and Amin A. Gohari Lecture on the Wednesday of the Symposium. If you wish to see her slides again or were unable to attend, a copy of 2) Mansoor I. Yousefi, for the paper “Integrable the slides have been posted on our Society website. Communication Channels and the Nonlinear Fourier Transform”, co-authored with Frank. R. Kschischang • The 2013 Aaron D. Wyner Distinguished Service Award goes to Daniel J. Costello. • Several members of our community became IEEE Fellows or received IEEE Medals, please see our web- • The 2013 IT Society Paper Award was given to Shrinivas site for more information: www.itsoc.org/honors Kudekar, Tom Richardson, and Rüdiger Urbanke for their paper “Threshold Saturation via Spatial Coupling: The Claude E. -
Network Information Theory
Network Information Theory This comprehensive treatment of network information theory and its applications pro- vides the first unified coverage of both classical and recent results. With an approach that balances the introduction of new models and new coding techniques, readers are guided through Shannon’s point-to-point information theory, single-hop networks, multihop networks, and extensions to distributed computing, secrecy, wireless communication, and networking. Elementary mathematical tools and techniques are used throughout, requiring only basic knowledge of probability, whilst unified proofs of coding theorems are based on a few simple lemmas, making the text accessible to newcomers. Key topics covered include successive cancellation and superposition coding, MIMO wireless com- munication, network coding, and cooperative relaying. Also covered are feedback and interactive communication, capacity approximations and scaling laws, and asynchronous and random access channels. This book is ideal for use in the classroom, for self-study, and as a reference for researchers and engineers in industry and academia. Abbas El Gamal is the Hitachi America Chaired Professor in the School of Engineering and the Director of the Information Systems Laboratory in the Department of Electri- cal Engineering at Stanford University. In the field of network information theory, he is best known for his seminal contributions to the relay, broadcast, and interference chan- nels; multiple description coding; coding for noisy networks; and energy-efficient packet scheduling and throughput–delay tradeoffs in wireless networks. He is a Fellow of IEEE and the winner of the 2012 Claude E. Shannon Award, the highest honor in the field of information theory. Young-Han Kim is an Assistant Professor in the Department of Electrical and Com- puter Engineering at the University of California, San Diego. -
Quantum Leap FINAL Case Study
Case study A Quantum Leap at INI Case study: Mathematics for Quantum Information “The application of quantum technologies to In their original proposal, two years earlier, MQI encryption algorithms threatens to dramatically programme organisers Richard Jozsa (Cambridge), impact the US government’s ability to both protect Noah Linden (Bristol), Peter Shor (MIT) and its communications and eavesdrop on the Andreas Winter (Bristol/CQT, Singapore) had communications of foreign governments,” correctly anticipated the readiness of their according to an internal document leaked by discipline for such breakthroughs predicting whistle-blower Edward Snowden. That the UK Government takes quantum computing seriously “we see this as a moment of substantial scientific is evinced by its commitment to “provide £270 opportunity: we believe we are at the beginning million over 5 years to fund a programme to of a period in the subject where diverse areas of support translation of the UK’s world leading mathematics will play an increasing important quantum research into application and new role. A Newton Institute programme at this time industries – from quantum computation to secure offers the possibility of major influence in the communication”, as announced by Chancellor of speed and direction of the field”. the Exchequer George Osborne in his 2013 Autumn Statement. This statement by the UK In October 2013, Alexander Holevo, a pioneer in Chancellor came during the closing weeks of an the field of Quantum Information Science and INI programme on Mathematics for Quantum winner of the Humbolt Prize, gave the Rothschild Information (MQI) and within days of an Lecture for the MQI programme. -
SHANNON SYMPOSIUM and STATUE DEDICATION at UCSD
SHANNON SYMPOSIUM and STATUE DEDICATION at UCSD At 2 PM on October 16, 2001, a statue of Claude Elwood Shannon, the Father of Information Theory who died earlier this year, will be dedicated in the lobby of the Center for Magnetic Recording Research (CMRR) at the University of California-San Diego. The bronze plaque at the base of this statue will read: CLAUDE ELWOOD SHANNON 1916-2001 Father of Information Theory His formulation of the mathematical theory of communication provided the foundation for the development of data storage and transmission systems that launched the information age. Dedicated October 16, 2001 Eugene Daub, Sculptor There is no fee for attending the dedication but if you plan to attend, please fill out that portion of the attached registration form. In conjunction with and prior to this dedication, 15 world-renowned experts on information theory will give technical presentations at a Shannon Symposium to be held in the auditorium of CMRR on October 15th and the morning of October 16th. The program for this Symposium is as follows: Monday Oct. 15th Monday Oct. 15th Tuesday Oct. 16th 9 AM to 12 PM 2 PM to 5 PM 9 AM to 12 PM Toby Berger G. David Forney Jr. Solomon Golomb Paul Siegel Edward vanderMeulen Elwyn Berlekamp Jacob Ziv Robert Lucky Shu Lin David Neuhoff Ian Blake Neal Sloane Thomas Cover Andrew Viterbi Robert McEliece If you are interested in attending the Shannon Symposium please fill out the corresponding portion of the attached registration form and mail it in as early as possible since seating is very limited. -
Analysis of Quantum Error-Correcting Codes: Symplectic Lattice Codes and Toric Codes
Analysis of quantum error-correcting codes: symplectic lattice codes and toric codes Thesis by James William Harrington Advisor John Preskill In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California 2004 (Defended May 17, 2004) ii c 2004 James William Harrington All rights Reserved iii Acknowledgements I can do all things through Christ, who strengthens me. Phillipians 4:13 (NKJV) I wish to acknowledge first of all my parents, brothers, and grandmother for all of their love, prayers, and support. Thanks to my advisor, John Preskill, for his generous support of my graduate studies, for introducing me to the studies of quantum error correction, and for encouraging me to pursue challenging questions in this fascinating field. Over the years I have benefited greatly from stimulating discussions on the subject of quantum information with Anura Abeyesinge, Charlene Ahn, Dave Ba- con, Dave Beckman, Charlie Bennett, Sergey Bravyi, Carl Caves, Isaac Chenchiah, Keng-Hwee Chiam, Richard Cleve, John Cortese, Sumit Daftuar, Ivan Deutsch, Andrew Doherty, Jon Dowling, Bryan Eastin, Steven van Enk, Chris Fuchs, Sho- hini Ghose, Daniel Gottesman, Ted Harder, Patrick Hayden, Richard Hughes, Deborah Jackson, Alexei Kitaev, Greg Kuperberg, Andrew Landahl, Chris Lee, Debbie Leung, Carlos Mochon, Michael Nielsen, Smith Nielsen, Harold Ollivier, Tobias Osborne, Michael Postol, Philippe Pouliot, Marco Pravia, John Preskill, Eric Rains, Robert Raussendorf, Joe Renes, Deborah Santamore, Yaoyun Shi, Pe- ter Shor, Marcus Silva, Graeme Smith, Jennifer Sokol, Federico Spedalieri, Rene Stock, Francis Su, Jacob Taylor, Ben Toner, Guifre Vidal, and Mas Yamada. Thanks to Chip Kent for running some of my longer numerical simulations on a computer system in the High Performance Computing Environments Group (CCN-8) at Los Alamos National Laboratory. -
Admissibility Properties of Gilbert's Encoding for Unknown Source
216 IEEE TRANSACTIONS ONINFORMATION THEORY,JANLJARY 1912 Admissibility Properties of Gilbert’s Encoding for Unknown We now wish to show that U*(b) minimizes Source Probabilities E{U@,P) I x1,x2,. .,X”h (7) THOMAS M. COVER the expected average code length conditioned on the data, if in fact p is drawn according to a Dirichlet prior distribution with parameters ,&), defined by the density function Abstract-Huffman coding provides optimal source encoding for (&A,. events of specified source probabilities. Gilbert has proposed a scheme &dPl,PZ,. ,PN) = C&J,,. .,a,)p;1-‘p;i-‘. .p;N-l, for source encoding when these source probabilities are imperfectly known. This correspondence shows that Gilbert’s scheme is a Bayes pi 1 0, zpp1 = 1, I* r 1, (8) scheme against a certain natural prior distribution on the source where C(1i,. J,) is a suitable normalization constant. This follows probabilities. Consequently, since this prior distribution has nonzero from application of Bayes’ rule, from which it follows that the posterior mass everywhere, Gilbert’s scheme is admissible in the sense that no distribution of p given X is again a Dirichlet distribution, this time source encoding has lower average code length for every choice of source with parameters Izl + nl, given by probabilities. APP,,P2,. ‘,PN I x1,x2,. .9X”) I. INTRODUCTION = C(Il + n1, a2 + Q,‘. ., a&l + nN)p;‘+“‘-‘p:2+“2-1.. ., CODINGFOR KNOWN~ ++Q- 1 Consider the random variable X, where Pr {X = xi} = pi, i = PN 3 p*ro,zppi=1. (9) 1,2,. ,N,pi 2 0, Xpi = 1. -
Andrew J. and Erna Viterbi Family Archives, 1905-20070335
http://oac.cdlib.org/findaid/ark:/13030/kt7199r7h1 Online items available Finding Aid for the Andrew J. and Erna Viterbi Family Archives, 1905-20070335 A Guide to the Collection Finding aid prepared by Michael Hooks, Viterbi Family Archivist The Andrew and Erna Viterbi School of Engineering, University of Southern California (USC) First Edition USC Libraries Special Collections Doheny Memorial Library 206 3550 Trousdale Parkway Los Angeles, California, 90089-0189 213-740-5900 [email protected] 2008 University Archives of the University of Southern California Finding Aid for the Andrew J. and Erna 0335 1 Viterbi Family Archives, 1905-20070335 Title: Andrew J. and Erna Viterbi Family Archives Date (inclusive): 1905-2007 Collection number: 0335 creator: Viterbi, Erna Finci creator: Viterbi, Andrew J. Physical Description: 20.0 Linear feet47 document cases, 1 small box, 1 oversize box35000 digital objects Location: University Archives row A Contributing Institution: USC Libraries Special Collections Doheny Memorial Library 206 3550 Trousdale Parkway Los Angeles, California, 90089-0189 Language of Material: English Language of Material: The bulk of the materials are written in English, however other languages are represented as well. These additional languages include Chinese, French, German, Hebrew, Italian, and Japanese. Conditions Governing Access note There are materials within the archives that are marked confidential or proprietary, or that contain information that is obviously confidential. Examples of the latter include letters of references and recommendations for employment, promotions, and awards; nominations for awards and honors; resumes of colleagues of Dr. Viterbi; and grade reports of students in Dr. Viterbi's classes at the University of California, Los Angeles, and the University of California, San Diego. -
International Online Conference One-Parameter Semigroups of Operators
International Online Conference One-Parameter Semigroups of Operators BOOK OF ABSTRACTS stable version from 20:21 (UTC+0), Friday 9th April, 2021 Nizhny Novgorod 2021 International Online Conference “One-Parameter Semigroups of Operators 2021” Preface International online conference One-Parameter Semigroups of Operators (OPSO 2021), 5-9 April 2021, is organized by the International laboratory of dynamical systems and applications, and research group Evolution semigroups and applications, both located in Russia, Nizhny Novgorod city. The Laboratory was created in 2019 at the National research university Higher School of Economics (HSE). Website of the Laboratory: https://nnov.hse.ru/en/bipm/dsa/ HSE is a young university (established in 1992) which rapidly become one of the leading Russian universities according to international ratings. In 2021 HSE is a large university focused on only on economics. There are departments of Economics (including Finance, Statistics etc), Law, Mathematics, Computer Science, Media and Design, Physics, Chemistry, Biotechnology, Geography and Geoinformation Technologies, Foreign Languages and some other. Website of the HSE: https://www.hse.ru/en/ The OPSO 2021 online conference has connected 106 speakers and 26 participants without a talk from all over the world. The conference covered the following topics: 1. One-parameter groups and semigroups of linear operators; 2. Nonlinear flows and semiflows; 3. Interplay between linear infinite-dimensional systems and nonlinear finite-dimensional systems; 4. Quan- tum