0471679100.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
Understanding Information Transmission John B. Anderson Rolf Johannesson A John Wiley & Sons, Inc., Publication Understanding Information Transmission IEEE Press 445 Hoes Lane Piscataway, NJ 08854 IEEE Press Editorial Board Stamatios V. Kartalopoulos, Editor in Chief M. Akay M.E. El-Hawary F. M. B. Periera J. B. Anderson R. Leonardi C. Singh R. J. Baker M. Montrose S. Tewksbury J. E. Brewer M. S. Newman G. Zobrist Kenneth Moore, Director of IEEE Book and Information Services (BIS) Catherine Faduska, Senior Acquisitions Editor Anthony VenGraitis, Project Editor Understanding Information Transmission John B. Anderson Rolf Johannesson A John Wiley & Sons, Inc., Publication Copyright # 2005 by the Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as per- mitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect tot the accuracy or comple- teness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the U.S. at 877-762-2974, outside the U.S. at 317-572-3993 or fax 3170572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print, however, may not be available in electronic format. Library of Congress Cataloging-in-Publication Data: Anderson, John B., 1945- Understanding information transmission/John B. Anderson and Rolf Johannesson. p. cm. Includes bibliographical references and index. ISBN 0-471-67910-0 (pbk.) 1. Signal theory (Telecommunication) I. Johannesson, Rolf, 1946- II. Title. TK5102.5.A535 2005 621.38202—dc22 2004057151 Printed in the United States of America 10987654321 Contents Preface vii 1. Introduction: First Ideas and Some History 1 1.1 What is communication? 2 1.2 Why digital communication? 6 1.3 Some history 8 1.4 A few remarks on intellectual history 27 1.5 Conclusions 28 References 29 2. Mathematical Methods of Information Transmission: Why Sinusoids? 30 2.1 Linear, time-invariant (LTI) systems 31 2.2 On the importance of being sinusoidal 43 2.3 The Fourier transform 48 2.4 What is bandwidth? 58 2.5 Discrete-time systems 66 2.6 Conclusions 69 References 70 Problems 70 3. Information Sources: What is Out There to be Sent? 77 3.1 What is text? 78 3.2 What is speech? 81 3.3 What is music? 88 3.4 What is an image? 94 3.5 What is video? 98 3.6 Conclusion 102 References 102 Problems 103 4. Transmission Methods: How is Information Sent? 105 4.1 Communication channels 105 4.2 Analog modulation 117 vi Contents 4.3 Digital modulation 127 4.4 FM stereo, television and a little about electronics 139 4.5 Conclusions 146 References 147 Problems 147 5. Information Theory and Coding: What did Shannon Promise? 150 5.1 Information theory—a primer 152 5.2 Methods of source coding 179 5.3 Methods of channel coding 189 5.4 Trellis coded modulation 199 5.5 Conclusions 205 References 206 Problems 207 6. Cryptology: FUBSWRORJB?? 211 6.1 Fundamentals of cryptosystems 211 6.2 Caesar and Vigene`re ciphers 215 6.3 The Vernam cipher and perfect secrecy 219 6.4 Stream ciphers 220 6.5 Block ciphers 223 6.6 Cryptomachines during World War II 224 6.7 Two-key cryptography 228 6.8 Conclusions 239 References 239 Problems 239 7. Communication Networks: Let’s Get Connected 241 7.1 An overview of information networks 241 7.2 Circuit switching: The telephone net 252 7.3 Mobile telephony 260 7.4 The Internet 265 References 275 Appendix A: Complex Numbers 276 Appendix B: Sinusoids and Circuit Theory 281 Appendix C: Probability Theory: A Primer 297 Index 306 About the Authors Preface Understanding Information Transmission is an introduction to the whole field of information engineering. Its seven chapters span the nature, storage, transmission, networking and protection of information. The book has two intentions: First, it is a second-year course book for new university programs in Information Technology (IT); secondly, it appears within the IEEE “Understanding Series” which was designed for those who wish to learn a new field on their own. In our case the field is the rapidly evolving field of IT. A special feature of the book is its treatment of the spectacular history of the subject, its people and inventions, and its social effects on all of us. As a text, the book has been used in the required introductory course on Infor- mation in the first year of the Information and Communication Technology engin- eering degree program at Lund University in Sweden. The present version has been used for three years in that course. One of us (JBA) also worked within the IT program at Rensselaer Polytechnic Institute. Baccalaureate IT programs like these are starting up all over the world. Because they are so new it is worthwhile to look at their curriculum requirements. These four-year IT programs combine courses from a variety of information disciplines. A typical curriculum might consist of courses in communication engin- eering, signal processing, software engineering, programing, computer science, mathematics (discrete mathematics, probability, linear algebra/complex variables), man–machine issues, psychology, and linguistics. To this may be added in some countries a proportion of liberal arts, economics, and management electives. Certain traditional engineering courses, for example, thermodynamics, mechanics, and materials, are often squeezed out. The prerequisites for our book are those for a course early in this structure. We assume a modest first-year university (U.S.) or gymnasium (Europe) preparation in mathematics (specifically, calculus, complex variables, and elementary probability) and physics (electromagnetic waves and DC electrical circuits). Review appendices are offered to help a reader who is missing parts of the prerequisites. Information engineering is a subject that is full of mathematics, and Chapter 2 is devoted to the core mathematical techniques of the discipline. The chapter develops Fourier transforms and bandwidth, linear systems and convolution, and some circuit ideas. viii Preface An IT degree program can be administered by an electrical engineering (EE), computer engineering, or computer science department, or possibly by a school of information technology. A program offered by one of the first three is necessarily a compromise compared to a traditional electrical engineering or computer science program. A degree recipient lacks circuit, fields and waves, and physics/ device courses compared to an electrical engineer; he or she lacks advanced pro- graming, computer science and architecture courses compared to a computer science major. The combining of parts of the EE and computer science curricula, as well as courses from the softer sciences, places a special strain on IT programs. These compromises are present in the pages of Understanding Information Trans- mission. It is necessary to skip over many details. Furthermore, an understanding of the information processing discipline must be achieved in one book rather than several, and in an IT program this must occur at an early point in the curriculum. The pedagogy behind this book and its choice of contents evolved over many years. A book like this can only arise from thousands of encounters with students, and we would like to acknowledge first the students and colleagues who made it possible, and especially the students in the first three years of the Infocom Program at Lund University. Next we wish to thank the Information Technology Department at Lund, which set aside significant resources for the preparation of the book. Particularly, we are grateful to Lena Ma˚nsson and Doris Holmqvist, whose enthusiasm and outstanding skill at Latex made it possible to complete this manuscript. Within IEEE and Wiley, thanks go to the production staff at Wiley and to our editors Tony VenGraitis and John Griffin, whose warm support we enjoyed from the first day. Thanks also are due to our colleague James Massey and to Oliver, the world’s leading canine information theorist, for debating many issues with us. Finally, we are indebted to Jennifer Bissett for transforming her views of information transmission into a striking cover. JOHN B. ANDERSON ROLF JOHANNESSON Lund, Sweden December 2004 Chapter 1 Introduction: First Ideas and Some History This book is about the transmission of information through space and time. We call the first communication. We call the second storage. How are these done, and what resources do they take? These days, most information transmission is digital, meaning that information is represented by a set of symbols, such as 1 and 0.