Department of Electrical Engineering and Computer Science
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Digital Communication Systems 2.2 Optimal Source Coding
Digital Communication Systems EES 452 Asst. Prof. Dr. Prapun Suksompong [email protected] 2. Source Coding 2.2 Optimal Source Coding: Huffman Coding: Origin, Recipe, MATLAB Implementation 1 Examples of Prefix Codes Nonsingular Fixed-Length Code Shannon–Fano code Huffman Code 2 Prof. Robert Fano (1917-2016) Shannon Award (1976 ) Shannon–Fano Code Proposed in Shannon’s “A Mathematical Theory of Communication” in 1948 The method was attributed to Fano, who later published it as a technical report. Fano, R.M. (1949). “The transmission of information”. Technical Report No. 65. Cambridge (Mass.), USA: Research Laboratory of Electronics at MIT. Should not be confused with Shannon coding, the coding method used to prove Shannon's noiseless coding theorem, or with Shannon–Fano–Elias coding (also known as Elias coding), the precursor to arithmetic coding. 3 Claude E. Shannon Award Claude E. Shannon (1972) Elwyn R. Berlekamp (1993) Sergio Verdu (2007) David S. Slepian (1974) Aaron D. Wyner (1994) Robert M. Gray (2008) Robert M. Fano (1976) G. David Forney, Jr. (1995) Jorma Rissanen (2009) Peter Elias (1977) Imre Csiszár (1996) Te Sun Han (2010) Mark S. Pinsker (1978) Jacob Ziv (1997) Shlomo Shamai (Shitz) (2011) Jacob Wolfowitz (1979) Neil J. A. Sloane (1998) Abbas El Gamal (2012) W. Wesley Peterson (1981) Tadao Kasami (1999) Katalin Marton (2013) Irving S. Reed (1982) Thomas Kailath (2000) János Körner (2014) Robert G. Gallager (1983) Jack KeilWolf (2001) Arthur Robert Calderbank (2015) Solomon W. Golomb (1985) Toby Berger (2002) Alexander S. Holevo (2016) William L. Root (1986) Lloyd R. Welch (2003) David Tse (2017) James L. -
4 Classical Error Correcting Codes
“Machines should work. People should think.” Richard Hamming. 4 Classical Error Correcting Codes Coding theory is an application of information theory critical for reliable communication and fault-tolerant information storage and processing; indeed, Shannon channel coding theorem tells us that we can transmit information on a noisy channel with an arbitrarily low probability of error. A code is designed based on a well-defined set of specifications and protects the information only for the type and number of errors prescribed in its design. Agoodcode: (i) Adds a minimum amount of redundancy to the original message. (ii) Efficient encoding and decoding schemes for the code exist; this means that information is easily mapped to and extracted from a codeword. Reliable communication and fault-tolerant computing are intimately related to each other; the concept of a communication channel, an abstraction for a physical system used to transmit information from one place to another and/or from one time to another is at the core of communication, as well as, information storage and processing. It should however be clear 355 Basic Concepts Linear Codes Polynomial Codes Basic Concepts Linear Codes Other Codes Figure 96: The organization of Chapter 4 at a glance. that the existence of error-correcting codes does not guarantee that logic operations can be implemented using noisy gates and circuits. The strategies to build reliable computing systems using unreliable components are based on John von Neumann’s studies of error detection and error correction techniques for information storage and processing. This chapter covers topics from the theory of classical error detection and error correction and introduces concepts useful for understanding quantum error correction, the subject of Chapter 5. -
IEEE Information Theory Society Newsletter
IEEE Information Theory Society Newsletter Vol. 63, No. 3, September 2013 Editor: Tara Javidi ISSN 1059-2362 Editorial committee: Ioannis Kontoyiannis, Giuseppe Caire, Meir Feder, Tracey Ho, Joerg Kliewer, Anand Sarwate, Andy Singer, and Sergio Verdú Annual Awards Announced The main annual awards of the • 2013 IEEE Jack Keil Wolf ISIT IEEE Information Theory Society Student Paper Awards were were announced at the 2013 ISIT selected and announced at in Istanbul this summer. the banquet of the Istanbul • The 2014 Claude E. Shannon Symposium. The winners were Award goes to János Körner. the following: He will give the Shannon Lecture at the 2014 ISIT in 1) Mohammad H. Yassaee, for Hawaii. the paper “A Technique for Deriving One-Shot Achiev - • The 2013 Claude E. Shannon ability Results in Network Award was given to Katalin János Körner Daniel Costello Information Theory”, co- Marton in Istanbul. Katalin authored with Mohammad presented her Shannon R. Aref and Amin A. Gohari Lecture on the Wednesday of the Symposium. If you wish to see her slides again or were unable to attend, a copy of 2) Mansoor I. Yousefi, for the paper “Integrable the slides have been posted on our Society website. Communication Channels and the Nonlinear Fourier Transform”, co-authored with Frank. R. Kschischang • The 2013 Aaron D. Wyner Distinguished Service Award goes to Daniel J. Costello. • Several members of our community became IEEE Fellows or received IEEE Medals, please see our web- • The 2013 IT Society Paper Award was given to Shrinivas site for more information: www.itsoc.org/honors Kudekar, Tom Richardson, and Rüdiger Urbanke for their paper “Threshold Saturation via Spatial Coupling: The Claude E. -
THE INTELLECTUAL ORIGINS of the Mcculloch
JHBS—WILEY RIGHT BATCH Top of ID Journal of the History of the Behavioral Sciences, Vol. 38(1), 3–25 Winter 2002 ᭧ 2002 John Wiley & Sons, Inc. (PHYSIO)LOGICAL CIRCUITS: THE INTELLECTUAL ORIGINS OF THE Base of 1st McCULLOCH–PITTS NEURAL NETWORKS line of ART TARA H. ABRAHAM This article examines the intellectual and institutional factors that contributed to the col- laboration of neuropsychiatrist Warren McCulloch and mathematician Walter Pitts on the logic of neural networks, which culminated in their 1943 publication, “A Logical Calculus of the Ideas Immanent in Nervous Activity.” Historians and scientists alike often refer to the McCulloch–Pitts paper as a landmark event in the history of cybernetics, and funda- mental to the development of cognitive science and artificial intelligence. This article seeks to bring some historical context to the McCulloch–Pitts collaboration itself, namely, their intellectual and scientific orientations and backgrounds, the key concepts that contributed to their paper, and the institutional context in which their collaboration was made. Al- though they were almost a generation apart and had dissimilar scientific backgrounds, McCulloch and Pitts had similar intellectual concerns, simultaneously motivated by issues in philosophy, neurology, and mathematics. This article demonstrates how these issues converged and found resonance in their model of neural networks. By examining the intellectual backgrounds of McCulloch and Pitts as individuals, it will be shown that besides being an important event in the history of cybernetics proper, the McCulloch– Pitts collaboration was an important result of early twentieth-century efforts to apply mathematics to neurological phenomena. ᭧ 2002 John Wiley & Sons, Inc. -
Your Brain Is Like a Computer: Function, Analogy, Simplification
M. Chirimuuta November 2018 draft. Please do not circulate or quote without permission. Comments welcome! Your Brain is Like a Computer: Function, Analogy, Simplification M. Chirimuuta ([email protected]) History & Philosophy of Science, University of Pittsburgh ABSTRACT The relationship between brain and computer is a perennial theme in theoretical neuroscience, but it has received relatively little attention in the philosophy of neuroscience. This paper argues that much of the popularity of the brain-computer comparison (e.g. circuit models of neurons and brain areas since McCulloch and Pitts [1943]) can be explained by their utility as ways of simplifying the brain. More specifically, by justifying a sharp distinction between aspects of neural anatomy and physiology that serve information-processing, and those that are ‘mere metabolic support,’ the computational framework provides a means of abstracting away from the complexities of cellular neurobiology, as those details come to be classified as irrelevant to the (computational) functions of the system. I argue that the relation between brain and computer should be understood as one of analogy, and consider the implications of this interpretation for notions of multiple realisation. I suggest some limitations of our understanding of the brain and cognition that may stem from the radical abstraction imposed by the computational framework. 0. PREAMBLE: LEIBNIZ THE INVENTOR Many histories of computation begin with the unrealised ambition of Gottfried Leibniz to devise a “universal characteristic”, a symbolic language in which factual propositions could be represented and further truths inferred by means of a mechanical calculating device (Davis 2000). Amongst the 20th century pioneers of computer science and artificial intelligence who took Leibniz for an inspirational figure1 were Warren McCulloch and Walter Pitts (Lettvin 2016: xix). -
Contrasting Theories of Life: Historical Context, Current Theories
BioSystems 188 (2020) 104063 Contents lists available at ScienceDirect BioSystems journal homepage: www.elsevier.com/locate/biosystems Review article Contrasting theories of life: Historical context, current theories. In search of an ideal theory Athel Cornish-Bowden <, María Luz Cárdenas Aix Marseille Univ, CNRS, BIP, IMM, Marseille, France ARTICLEINFO ABSTRACT Keywords: Most attempts to define life have concentrated on individual theories, mentioning others hardly at all, but here Autocatalytic sets we compare all of the major current theories. We begin by asking how we know that an entity is alive, and Autopoiesis continue by describing the contributions of La Mettrie, Burke, Leduc, Herrera, Bahadur, D'Arcy Thompson and, Chemoton especially, Schrödinger, whose book What is Life? is a vital starting point. We then briefly describe and discuss Closure (M, R) systems, the hypercycle, the chemoton, autopoiesis and autocatalytic sets. All of these incorporate the Hypercycle Metabolic circularity idea of circularity to some extent, but all of them fail to take account of mechanisms of metabolic regulation, (M, R) systems which we regard as crucial if an organism is to avoid collapsing into a mass of unregulated reactions. In Origin of life a final section we study the extent to which each of the current theories can aid in the search for a more Self-organization complete theory of life, and explain the characteristics of metabolic control analysis that make it essential for an adequate understanding of organisms. Contents 1. General introduction -
Urban Flooding Centre for Neuroscience Solid Waste
Connect ISSN 2454-6232 WITH THE INDIAN INSTITUTE OF SCIENCE Urban Flooding Causes and answers Centre for Neuroscience From brain to behaviour Solid Waste Management February 2016 Volume 3 • Issue 1 Towards zero waste CONTRIBUTORS Nithyanand Rao is a Consultant Editor at the Archives N Apurva Ratan Murty is a PhD student at the Centre for and Publications Cell Neuroscience Sudhi Oberoi is a Project Trainee at the Archives and Soumyajit Bhar is a PhD student at ATREE Publications Cell Akanksha Dixit is a PhD student in the Department of Science Media Center is a joint initiative of IISc and Biochemistry Gubbi Labs Sakshi Gera is a PhD student in the Department of Debaleena Basu is a PhD student in the Centre for Molecular Reproduction, Development and Genetics Neuroscience Disha Mohan is a PhD student in the Molecular Karthik Ramaswamy is the Editor, CONNECT, at the Biophysics Unit Archives and Publications Cell Sayantan Khan is an undergraduate student Manbeena Chawla is a Research Associate at the Centre Ankit Ruhi is a PhD student in the Department of for Infectious Disease Research Mathematics Manu Rajan is a Technical Officer at the Archives and Megha Prakash is a Consultant Editor at the Archives and Publications Cell Publications Cell Nisha Meenakshi is a PhD student in the Department of Raghunath Joshi is a PhD student in the Department of Electrical Engineering Materials Engineering Bharti Dharapuram is a PhD student in the Centre for Sridevi Venkatesan is an undergraduate student Ecological Sciences Navin S is a Master’s student -
Kathleen A. Powers Dissertation
UC Berkeley UC Berkeley Electronic Theses and Dissertations Title The Cybernetic Origins of Life Permalink https://escholarship.org/uc/item/9d8678w5 Author Powers, Kathleen Alicia Publication Date 2020 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California The Cybernetic Origins of Life By Kathleen A. Powers A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Rhetoric and the Designated Emphasis in Science and Technology Studies in the Graduate Division of the University of California, Berkeley Committee in charge: Chair Professor David Bates Professor James Porter Professor Emeritus Gaetan Micco Professor Sandra Eder Fall 2020 1 Abstract The Cybernetic Origins of Life by Kathleen A. Powers Doctor of Philosophy in Rhetoric and the Designated Emphasis in Science and Technology Studies University of California, Berkeley Professor David Bates, Chair This dissertation elucidates the cybernetic response to the life question of post-World War II biology through an analysis of the writings and experiments of Warren S. McCulloch. The work of McCulloch, who was both a clinician and neurophysiologist, gave rise to what this dissertation refers to as a biological, medical cybernetics, influenced by vitalist conceptions of the organism as well as technical conceptions of the organ, the brain. This dissertation argues that the question ‘what is biological life?’ served as an organizing principle for the electrical, digital model of the brain submitted in “Of Digital Computers Called Brains” (1949) and the formal, mathematical model of the brain required by the McCulloch-Pitts neuron in “A Logical Calculus of the Ideas Immanent in Nervous Activity” (1943). -
Digital Communication Systems ECS 452
Digital Communication Systems ECS 452 Asst. Prof. Dr. Prapun Suksompong (ผศ.ดร.ประพันธ ์ สขสมปองุ ) [email protected] 1. Intro to Digital Communication Systems Office Hours: BKD, 6th floor of Sirindhralai building Tuesday 14:20-15:20 Wednesday 14:20-15:20 1 Friday 9:15-10:15 “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.” Shannon, Claude. A Mathematical Theory Of Communication. (1948) 2 Shannon: Father of the Info. Age Documentary Co-produced by the Jacobs School, UCSD-TV, and the California Institute for Telecommunications and Information Technology Won a Gold award in the Biography category in the 2002 Aurora Awards. 3 [http://www.uctv.tv/shows/Claude-Shannon-Father-of-the-Information-Age-6090] [http://www.youtube.com/watch?v=z2Whj_nL-x8] C. E. Shannon (1916-2001) 1938 MIT master's thesis: A Symbolic Analysis of Relay and Switching Circuits Insight: The binary nature of Boolean logic was analogous to the ones and zeros used by digital circuits. The thesis became the foundation of practical digital circuit design. The first known use of the term bit to refer to a “binary digit.” Possibly the most important, and also the most famous, master’s thesis of the century. It was simple, elegant, and important. 4 C. E. Shannon: Master Thesis 5 Boole/Shannon Celebration Events in 2015 and 2016 centered around the work of George Boole, who was born 200 years ago, and Claude E. Shannon, born 100 years ago. Events were scheduled both at the University College Cork (UCC), Ireland and the Massachusetts Institute of Technology (MIT) 6 http://www.rle.mit.edu/booleshannon/ An Interesting Book The Logician and the Engineer: How George Boole and Claude Shannon Created the Information Age by Paul J. -
MIT 150 | the Birth of Time-Shared Computing—Robert Fano (1985)
MIT 150 | The Birth of Time-Shared Computing—Robert Fano (1985) FERNANDO The history of time sharing is the subject of the lecture. And I'm not going to try to address that. Instead, I want CORBATO: to address the context of our speaker who has played a key role. And I want to go back to his roots a bit, because I want to explain why he's in a position to give this distinguished lecture. I won't discuss his books. He has several. I won't discuss his long list of papers. I won't discuss is patents, the awards, or the many academies he belongs to-- has been elected to. He has a truly distinguish record. As the program notes briefly outline, Bob Fano was born in Torino, Italy. He came to this country when Europe came under the totalitarian grip. He came to the US and MIT in particular. He got his bachelor's in 1941. And he went on from there as a student engineer at General Motors, whose specialty was power engineering. And he tells me that he was in charge of maintaining the welding machines. And he lasted all of six months. And he decided that was not for him and came back to MIT to enter graduate school, clearly a great decision. As a graduate student, he had a nice career of being a TA and then an instructor when that was a meaningful rank in those days. It still is. He became a staff member of the radiation laboratory during wartime. -
Andrew J. and Erna Viterbi Family Archives, 1905-20070335
http://oac.cdlib.org/findaid/ark:/13030/kt7199r7h1 Online items available Finding Aid for the Andrew J. and Erna Viterbi Family Archives, 1905-20070335 A Guide to the Collection Finding aid prepared by Michael Hooks, Viterbi Family Archivist The Andrew and Erna Viterbi School of Engineering, University of Southern California (USC) First Edition USC Libraries Special Collections Doheny Memorial Library 206 3550 Trousdale Parkway Los Angeles, California, 90089-0189 213-740-5900 [email protected] 2008 University Archives of the University of Southern California Finding Aid for the Andrew J. and Erna 0335 1 Viterbi Family Archives, 1905-20070335 Title: Andrew J. and Erna Viterbi Family Archives Date (inclusive): 1905-2007 Collection number: 0335 creator: Viterbi, Erna Finci creator: Viterbi, Andrew J. Physical Description: 20.0 Linear feet47 document cases, 1 small box, 1 oversize box35000 digital objects Location: University Archives row A Contributing Institution: USC Libraries Special Collections Doheny Memorial Library 206 3550 Trousdale Parkway Los Angeles, California, 90089-0189 Language of Material: English Language of Material: The bulk of the materials are written in English, however other languages are represented as well. These additional languages include Chinese, French, German, Hebrew, Italian, and Japanese. Conditions Governing Access note There are materials within the archives that are marked confidential or proprietary, or that contain information that is obviously confidential. Examples of the latter include letters of references and recommendations for employment, promotions, and awards; nominations for awards and honors; resumes of colleagues of Dr. Viterbi; and grade reports of students in Dr. Viterbi's classes at the University of California, Los Angeles, and the University of California, San Diego. -
Robert Mario Fano: Un Italiano Tra Gli Architetti Della Società Dell’Informazione
Robert Mario Fano: un italiano tra gli architetti della Società dell’Informazione Benedetta Campanile ˗ Seminario di Storia della Scienza ˗ Università degli Studi di Bari Aldo Moro ˗ [email protected] Abstract: Italian Jewish immigrated in US during the World War II, Rob- erto Mario Fano (1917-2016) is considered at Massachusetts Institute of Technology (MIT) an architect of Information Society, because he was pio- neer of concept of “making the power of computers directly accessible to people”. Really Fano, in the early 1960s, earned the support of the Com- mand & Control Branch of ARPA of US Department of Defense for a big program in computer field, the Project MAC, that had a civil aim: permit people to communicate directly with a computer from remote locations. The implementation of the Time-Sharing System, realized by the theoretical physicist Fernando J. Corbató, was the focus of the project. Fano served as founding director of MIT’s Project MAC and helped to change the compu- tational landscape to support interactive process. The information pro- cessing in time-sharing became a way for many people to use a computer and to break away from the dreaded world of batch processing. The concept of “computer utility” became the creed of Fano and his colleagues. The im- pact of Project MAC on the education and research generated a new disci- pline, the Computer Science, and the large federal funds for academic re- search carried out a new knowledge that changed the society organization. Keywords: Robert Fano, Project MAC, Big Science The New Frontier of which I speak is not a set of promises ..