SCIENCE'S COMPASS respective probabilities, while the channel channels, can have useful redundancy (extra RETROSPECTIVE: COMMUNICATION contributed the set of possible disturbances symbols for error detection and correction) ("noise") to the message. added back. He also showed that reliable Claude E. Shannon circuits could be built with unreliable parts, Shannon redefined the entropy of again using redundancy, akin to achieving (1916-2001) thermodynamics as a measure of uncertainty reliable communication over unreliable Solomon W. Golomb on probability distributions. Although (i.e., noisy) channels. crediting the term "bit" (for "binary digit") His inventive mind enriched many fields, to J. W. Tukey, for his own purposes, The Shannon bit is now universally but Claude Shannon's enduring fame will Shannon defined bit as the amount of recognized as the basic unit of information. surely rest on his 1948 paper "A information gained (or entropy removed) It has been proposed to rename this unit the Mathematical Theory of Communication" upon learning the answer to a question Shannon. If a message consists of N and the ongoing revolution in information whose two possible answers were equally Shannons, then the theoretically best technology it engendered. likely a priori. (When one possible answer source encoding could express it in N binary is more likely than the other, the answer digits. Shannon was born on 30 April 1916 in conveys less than one bit of information.) Petoskey, Michigan. After obtaining He derived formulas for the information rate Shannon was a talented gadgeteer who bachelor's degrees in mathematics and of a source and for the capacity of a channel, built some of the earliest robotic automata, electrical engineering at the University of each measured in bits per second and proved game-playing devices, and puzzle- solving Michigan, he went to the Massachusetts that for any information rate R less than the machines. He could juggle while riding a Institute of Technology (MIT), where, after channel capacity C, it is possible (by unicycle and designed machines to juggle spending the summer of 1937 at Bell suitable encoding) to send information at and to ride unicycle-like vehicles. Not Laboratories, he wrote one of the greatest rate R, with an error rate less than any working in a Nobel Prize field but in the new master's theses ever (l). He showed that the preassigned E, over that channel. His science he had invented, he received symbolic logic of George Boole's 19th ingenious proof considers the set of all innumerable honors and awards, including century Laws of Thought provided the possible encodings of source messages into the U.S. National Medal of Science (1966), perfect mathematical model for switching streams of binary digits and shows that an Israel's Harvey Prize (1972), and Japan's theory (and indeed for the subsequent logic encoding chosen at random from this set Kyoto Prize (1985). The Information design of digital circuits and computers). will have the desired property with Theory Group (later Society) of the Institute This work was awarded the prestigious extremely high probability. of Electrical and Electronics Engineers Alfred Noble Prize of the combined established the Shannon Award (originally engineering societies of the United States in called Shannon Lecture) as its highest 1940. honor. In 1973, Shannon delivered the first Shannon Lecture in Ashkelon, Israel. I had From 1938, Shannon worked at MIT with spent most of the fall of 1959 at MIT, where Vannevar Bush's "differential analyzer," the I had gotten to know Shannon quite well, ancestral analog computer. After obtaining but it was an unexpected honor when he his Ph.D. at MIT, he spent the academic year attended my Shannon Lecture in 1985 in 1940-1941, working under mathematician Brighton, England—the only one he Hermann Weyl in Princeton, where he began attended after his own. to think about the mathematics underlying communications. In 1941, he returned to In 1956, Shannon left Bell Labs for MIT, Bell Labs for the next 15 years, initially where he was Donner Professor of Science working on projects related to the war from 1958 until his retirement in 1978. He effort, including cryptography (2). Perhaps died on 24 February 2001, in Medford. it was thinking about cryptography that led Massachusetts, aged 84. Shannon to his breakthrough in "A Mathematical Theory of Communication," In 1964, the title of a book I edited and published in two parts in the Bell System coauthored, Digital Communications, was Technical Journal (BSTJ) m 1948. still considered an oxymoron: To most communication engineers, signals were At the start of this epic paper, he quite obviously analog. But in the late acknowledged the work of Harry Nyquist and 1940s, the transistor was invented, also at RVL. Hartley at Bell Labs in the 1920s. But Generations of coding theorists have Bell Labs. Shannon's remarkable theorems Shannon, like Newton "standing on the struggled to find codes that perform as well told communication engineers what goals to shoulders of giants," was able to see much as Shannon's "random" ones. Richard W strive for, and integrated circuits provided farther than his predecessors. Early in the Hamming at Bell Labs and Marcel Golay at ever-improving hardware to realize these paper, he wrote that the "semantic aspects IBM Research Labs pioneered the theory of goals. Thus, the foundation for the digital of communication are irrelevant to the error-correcting codes in the late 1940s, communications revolution was laid. It is engineering problem. The significant largely independent of Shannon's work. no exaggeration that Claude Shannon was aspect is that the actual message is one However, some communications systems the Father of the Information Age and his selected from a set of possible messages." today achieve performance within 0.115% intellectual achievement one of the greatest [Shannon's emphasis] of the Shannon limit by using codes closer of the 20th century. Shannon recognized, however, that it to Shannon's original "random codes" than References and Notes was not only the size of the set of possible to the block codes of Hamming or Golay. 1. Published in 1938 as "A Symbolic Analysis of Relay and messages that was important, but also their Switching Circuits" in the Transactions of the American respective a priori probabilities. His great Shannon pioneered the study of source Institute of Electrical Engineers insight was to think in terms of statistical coding (data compression) to remove all 2. Shannon's classified report, "A Mathematical Theory of Cryptography" (1945), was declassified and published in 1949 ensembles: He saw the source as the set of useless redundancy from source messages, in the Bell System Technical JoumaL all messages that may be sent, with their which, if they are to be sent over noisy www.sciencemag.org SCIENCE VOL 29220 APRIL 2001 p455 The author is at the Department of Electrical Engineering-Systems, University of Southern Califomia, Los Angeles, CA 90089, USA. E- mail: [email protected]

www.sciencemag.org SCIENCE VOL 29220 APRIL 2001 p455