Contents Telektronikk Feature: Information Theory and its Applications Volume 98 No. 1 – 2002 1 Guest Editorial; Geir E. Øien ISSN 0085-7130 A Historical Perspective on Information Theory Editor: Ola Espvik 3 Information Theory: The Foundation of Modern Communications; Tel: (+47) 913 14 507 Geir E. Øien email: [email protected] 20 On Shannon and “Shannon’s Formula”; Lars Lundheim Status section editor: Per Hjalmar Lehne 30 Statistical Communication Theory 1948 – 1949; Nic. Knudtzon Tel: (+47) 916 94 909 email: [email protected] Novel Developments on Channel Capacity and Spectral Efficiency Editorial assistant: Gunhild Luke 35 The True Channel Capacity of Pair Cables With Respect to Near End Tel: (+47) 415 14 125 Crosstalk; Nils Holte email: [email protected] 47 Bounds on the Average Spectral Efficiency of Adaptive Coded Modulation; Editorial office: Kjell J. Hole Telenor Communication AS Telenor R&D 53 Breaking the Barriers of Shannon’s Capacity; An Overview of MIMO NO-1331 Fornebu Wireless Systems; David Gesbert and Jabran Akhtar Norway Tel: (+47) 67 89 00 00 Turbo Coding and Iterative Decoding: Theory and Applications email: [email protected] 65 An Introduction to Turbo Codes and Iterative Decoding; Øyvind Ytrehus Editorial board: Berit Svendsen, CTO Telenor 78 Theory and Practice of Error Control Coding for Satellite and Ole P. Håkonsen, Professor Fixed Radio Systems; Pål Orten and Bjarne Risløw Oddvar Hesjedal, Director Bjørn Løken, Director Modulation, Coding and Beyond Graphic design: Design Consult AS, Oslo 92 A New Look at the Exact BER Evaluation of PAM, QUAM and PSK Constellations; Pavan K. Vitthaladevuni and Mohamed-Slim Alouini Layout and illustrations: Gunhild Luke and Britt Kjus, 106 Performance Analysis of Adaptive Coded Modulation with Antenna Diversity Telenor R&D and Feedback Delay; Kjell J. Hole, Henrik Holm and Geir E. Øien Prepress and printing: 114 Shannon Mappings for Robust Communication; Tor A. Ramstad Optimal as, Oslo Circulation: Historical Papers 3,750 129 Information Theory; Nic. Knudtzon 139 Statistical Communication Theory; Nic. Knudtzon 145 Statistically Optimal Networks; Nic. Knudtzon Special 153 Multiple Bottom Lines? Telenor’s Mobile Telephony Operations in Bangladesh; Arvind Singhal, Peer J. Svenkerud and Einar Flydal Status 163 Introduction; Per Hjalmar Lehne 164 UMTS Network Domain Security; Geir M. Køien Guest Editorial GEIR E. ØIEN On February 24, 2001, Claude Elwood Shannon In a world where predictions for the future per- died at the age of 84. As the father of, and most formance of telecommunication systems some- important contributor to, the field of information times seem to be made more out of marketing theory, he ranks as one of the most brilliant and concerns than out of a scientifically sound important of all 20th century scientists. In a trib- judgement, information theory still has a lot to ute speech made by senator John D. Rockefeller teach us – insights that are sometimes sobering, in the US Congress after Shannon’s death, his but may also be encouraging. The optimistic work was referred to as “The Magna Carta of the vision suggested by some, that “any telecommu- information age”. Had there been awarded a nication service can be made available any- Nobel prize within the information and commu- where, anytime, and to anyone” in the future, nication sciences, no one would have been a can fairly easily be shown to have no roots in Geir E Øien more worthy candidate than Shannon. reality. One example may illustrate this: The claims originally made regarding the available Shannon’s ideas, first presented to the world at rates and coverage for the upcoming Universal large in his seminal 1948 paper “A Mathemati- Mobile Telecommunications System (UMTS) cal theory of communication” in Bell System so far seem a bit over-optimistic … Technical Journal, have been crucial in enabling the information and communication technologi- However, in some cases information theory can cal advances which have created today’s infor- also be used to uncover a performance potential mation society. As with all true pioneers, Shan- beyond what was previously thought possible, non’s way of thinking about information and and aid in the design of systems realizing this communication represented a true paradigm potential. As an important example of the appli- shift. For example, prior to the arrival of his cability of Shannon’s results, his theory showed seminal papers of the late 40s and 50s, there us how to design more efficient communication simply had been no satisfactory way of mod- and storage systems by demonstrating the enor- elling and analyzing the process of information mous gains achievable by coding, and by provid- generation, transfer, and reception from a trans- ing the intuition for the correct design of coding mitter to a receiver over a noisy communication systems. The sophisticated coding schemes used channel – which actually is a generic description in systems as diverse as deep-space communica- of how all practical communication systems work. tion systems, and home compact disk audio sys- Front cover: tems, owe their success to the insights provided With Shannon’s introduction of a generic com- by Shannon’s theory. Information appears as a munication system model, his view of informa- change in the detectable tion as a probabilistic entity (sidestepping its Shannon published many more important and pattern actual semantic meaning), the insight that the influential works in a variety of disciplines, The artist Odd Andersen process of information transmission is funda- including Boolean algebra and cryptography. visualises a set of planes mentally stochastic in nature, and his invention His work has had an influence on such diverse as areas for information to of precise mathematical tools to give a complete fields as linguistics, phonetics, psychology, gam- appear. Whatever kind of predictable pattern that performance analysis of his model, the door was bling theory, stock trading, artificial intelligence, already may exist on those suddenly opened to a much more fundamental and digital circuit design. It also has strong links planes has no interest. Only understanding of the possibilities and limitations to disciplines such as thermodynamics and bio- when that pattern is changed of communication systems. In the words of chemistry. has information occurred. another notable information theorist (and former When part of the pattern of one plane is moved through colleague of Shannon), David Slepian, “Proba- Shannon was also known for his playfulness and a transmission channel to bly no single work in this century has more pro- eclectic interests, which led to famous stunts another plane and changes foundly altered man’s understanding of commu- such as juggling while riding a unicycle down its pattern, this is seen as nication than C.E. Shannon’s article, “A mathe- the halls of Bell Labs. He designed and built information received by the matical theory of communication”, first pub- chess-playing, maze-solving, juggling, and new plane. lished in 1948. The ideas in Shannon’s paper mind-reading machines. These activities bear The artist’s generic message: were soon picked up by communication engi- proof to Shannon’s claim that his motivation Information has been was always curiosity more than usefulness. In produced and understood neers and mathematicians around the world. when the pattern of one plane They were elaborated upon, extended, and com- an age where basic research motivated purely by has become changed and plemented with new related ideas. The subject scientific curiosity sometimes seem to be at the detected. thrived and grew to become a well-rounded and losing end as far as public interest and funding is Ola Espvik, Editor in Chief exciting chapter in the annals of science.” concerned, this is a statement worth remember- Telektronikk 1.2002 1 ing. The success of information theory clearly In fact, these are particularly exciting times for shows how one person’s curiosity can translate information theory, because we now finally have into very useful results. available powerful techniques for actually approaching the performance limits predicted In Norway, Shannon’s memory and the achieve- by Shannon. One important example is the class ments of information theory were recently hon- of error control codes called “turbo codes”. Why ored with a Claude E. Shannon In Memoriam do these codes work so well? Because their con- Seminar, arranged by Telenor R&D at Kjeller struction turns out to be based to a great extent on the August 9, 2001. The seminar drew over upon the “nonconstructive” proof techniques 100 participants from Norwegian industry and used by Shannon in his analysis! academia. They came to listen to technical talks by some of Norway’s foremost experts within Regarding my four subgoals; first, the issue the fields of information and communication should place information theory in a historical theory, as well as to some historically and philo- context, while at the same time introducing its sophically flavored talks. The seminar was high- basic principles and discussing their importance. lighted by a unique reminiscence by former The papers collected under the heading “A his- Telenor R&D director Nic. Knudtzon, the only toric perspective on information theory” serve Norwegian to have met Shannon at the pioneer- this purpose. ing time when information theory was actually born. This special issue of Telektronikk was Secondly, I wanted to show that information the- inspired to a great extent by the success of this ory is still very much an active research field of seminar, and you will find papers by many of the practical importance, and that the Norwegian same scientists here – including Dr. Knudtzon’s research community is currently making some wonderful reminiscence, as well as three 1950 important contributions to this research. Thirdly, papers of his which rank as the very first presen- some of the most important applications of tations of information theory made in Norway.