Strategic Computing History of Computing I. Bernard Cohen and William Aspray, editors William Aspray, John von Neumann and the Origins of Modern Computing Charles J. Bashe, Lyle R. Johnson, John H. Palmer, and Emerson W. Pugh, IBM’s Early Computers Paul E. Ceruzzi, A History of Modern Computing I. Bernard Cohen, Howard Aiken: Portrait of a Computer Pioneer I. Bernard Cohen and Gregory W. Welch, editors, Makin’ Numbers: Howard Aiken and the Computer John Hendry, Innovating for Failure: Government Policy and the Early British Com- puter Industry Michael Lindgren, Glory and Failure: The Difference Engines of Johann Mu¨ller, Charles Babbage, and Georg and Edvard Scheutz David E. Lundstrom, A Few Good Men from Univac R. Moreau, The Computer Comes of Age: The People, the Hardware, and the Software Emerson W. Pugh, Building IBM: Shaping an Industry and Its Technology Emerson W. Pugh, Memories That Shaped an Industry Emerson W. Pugh, Lyle R. Johnson, and John H. Palmer, IBM’s 360 and Early 370 Systems Kent C. Redmond and Thomas M. Smith, From Whirlwind to MITRE: The R&D Story of the SAGE Air Defense Computer Alex Roland with Philip Shiman, Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983–1993 Rau´l Rojas and Ulf Hashagen, editors, The First Computers—History and Architectures Dorothy Stein, Ada: A Life and a Legacy John Vardalas, The Computer Revolution in Canada: Building National Technologi- cal Competence, 1945–1980 Maurice V. Wilkes, Memoirs of a Computer Pioneer Strategic Computing DARPA and the Quest for Machine Intelligence, 1983–1993 Alex Roland with Philip Shiman The MIT Press Cambridge, Massachusetts London, England 2002 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form or by any electronic or mechanical means (including photocopying recording, or information storage and retrieval) without permission in writing from the publisher. This book was set in New Baskerville by Achorn Graphic Services, Inc., and printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Roland, Alex, 1944– Strategic computing : DARPA and the quest for machine intelligence, 1983–1993 / Alex Roland with Philip Shiman. p. cm. — (History of computing) Includes bibliographical references and index. ISBN 0-262-18226-2 (hc. : alk. paper) 1. High performance computing. 2. Artificial intelligence. I. Shiman, Philip. II. Title. III. Series. QA76.88 .R65 2002 004.3—dc21 2001056252 Contents List of Illustrations ix Preface xi Acknowledgments xv Chronology: Key DARPA Personnel during the Strategic Computing Program xix List of Acronyms xxi Introduction 1 Part I 1 Robert Kahn: Visionary 13 Switching and Connecting 14 The Information Processing Techniques Office (IPTO) 21 AI Autumn 26 An Agent of Restoration 32 2 Robert Cooper: Salesman 39 The Pull of Robert Cooper 39 The Defense Advanced Research Projects Agency (DARPA) 42 An Agent of Transition 46 Testing the Market 56 vi Contents The Pyramid as Conceptual Icon 63 Of Time Lines and Applications 71 3 Lynn Conway: Executor 83 Connecting with the Customer 83 The Divorce of the Dynamic Duo 95 Process and Personality 102 The Plight of the Program Manager 109 Part II 4 Invisible Infrastructure: MOSIS 117 The Transformation of Microelectronics 117 From VLSI to MOSIS 119 The Value Added 130 Public Good, Political Liability 136 Other Infrastructure 140 5 Over the Wall: The SC Architectures Program 149 The Connection Machine 149 The Wall 154 Stephen Squires, Human Agent 158 Defining the First Generation 165 Applying the Gray Code 174 6 Artificial Intelligence: The Search for the Generic Expert System 185 Artificial Intelligence (AI) 185 Expert Systems 190 Contents vii IntelliCorp and Teknowledge 196 Managing Innovation 201 The Failure to Connect 206 The Rest of AI 208 7 Putting SC to Work: The Autonomous Land Vehicle 215 Crazy Walking War Machines 215 Working with Industry 222 The Tyranny of the Demonstration 229 The New Generation System 237 The ALV and Its Shadow 243 Part III 8 ISTO: The Middle Years of Strategic Computing, 1985–1989 251 Reconciling Form and Function 251 Reconciling Vision and Budget 261 Strategic Computing at Age Four 264 Waiting for the Wave 272 Strategic Computing 2 279 9 The Disappearance of Strategic Computing 285 The FCCSET Initiative 286 Connecting the Supercomputers 293 The Gap between Conception and Legislation 296 Fields and Boehm in the Last Ditch 302 The Politics of Change 310 Disconnecting the Last Veteran 314 viii Contents 10 Conclusion 319 Why? 320 How? 321 What? 325 Notes 333 Note on Sources 397 Index 405 Illustrations 2.1 DARPA organizational chart, June 1980 47 2.2 Dertouzos seating chart 59 2.3 “Program Structure and Logic,” the SC pyramid before it be- 68 came a pyramid 2.4 Time line 70 2.5 SCI pyramid 77 2.6 Visualizing and interpreting the program’s compartments, 78 elements, and time line 3.1 DARPA organizational chart, November 1984 101 3.2 “Going Away” gift picture for Lynn Conway 105 3.3 The fully articulated SC pyramid 108 4.1 Junction transistor 121 4.2 NPN transistor 122 4.3 Schematic of a field effect (FET) or metal-oxide semiconduc- 122 tor (MOS) transistor 4.4 Field effect or metal-oxide semiconductor (MOS) chip 123 4.5 The MOSIS system 125 4.6 λ rules 128 4.7 A wafer with multiple chips 140 5.1 The Connection Machine 178 7.1 Pilot’s Associate 217 7.2 The Autonomous Land Vehicle 227 8.1 DARPA organizational chart, September 1987 257 9.1 FCCSET “Advanced Computer Research Program Structure 292 and Goals” 9.2 “The Scalable Parallel Computing Spiral” 299 9.3 DARPA-ISTO Technology Maturity Model 304 9.4 DARPA organizational chart, July 1991 312 9.5 Pyramid of High Performance Computing environments 317 This page intentionally left blank Preface This story has its own story. Philip Shiman and I were commissioned by the Defense Advanced Research Projects Agency (DARPA) to prepare this study. Most of the research and writing were done under subcontract between the Naval Warfare Systems Center (NWSC), an agent of DARPA, and Duke University, which employed Shiman and me. Execu- tion of this contract was fraught with difficulty, and the contract was terminated short of completion. The book that follows is best under- stood in the light of this relationship between the historians and the object of their study. On the recommendation of I. Bernard Cohen and Merritt Roe Smith, members of a DARPA panel overseeing this book project, Keith Un- capher contacted me in March of 1992. I then submitted a proposal to serve as principal investigator, with Research Associate Philip Shiman, to prepare a 400-page history of the Strategic Computing Program (SCP). The three-year project would be funded by the Information Sci- ence and Technology Office (ISTO) of DARPA. On 11 November 1992, Shiman, Uncapher, and I met with Steven Squires, director of ISTO. At that meeting Squires displayed a remarkable intelligence, a boyish enthusiasm for computers, a petulant imperiousness, a passion for Isaac Asimov and Star Trek, and a permanent membership card to Disney- world, which he wore on a chain around his neck. Prudence dictated that Shiman and I walk away from this undertaking, but we succumbed to a siren song of curiosity. Squires claimed that the world was in a Seldon crisis, a reference to the Asimov Foundation trilogy. This classic of science fiction, which may be seen as a rumination on the collapse of the Roman empire or a metaphor for the Cold War, envi- sioned two “foundations” chartered at opposite ends of a galactic em- pire, which visionary Hari Seldon knew was coming to an end. Seldon tells one of the foundations, analogous to the United States or the free xii Preface world, that it must survive a series of crises. Only then will it endure the millennium of chaos before the birth of galactic civilization. Squires clearly implied that he and his colleagues in computer research were solving critical problems, thereby ushering in a new era of civilization. Shiman and I both wanted to know more about Squires and the strange world he inhabited. We sparred with Squires over the terms of the contract and our method of operation. He wanted us to work primar- ily from interviews with him and his colleagues and to write up the story as they saw it. We insisted that we would need other sources of evidence to validate whatever interpretation we settled on. We left the inter- view, I now think, each believing that the other had agreed to do it his way. As we set about our work, it quickly became apparent that working with DARPA would prove more difficult than we had anticipated. It took eight months just to get the terms of the contract settled. The authoriza- tion chain led from Squires as director of ISTO, through a contracting officer at DARPA, the NWSC, a subcontracting agent (at first, Meridian Corporation, then Applied Ordnance Technology [AOT], and then Me- ridian again [now a subsidiary of Dyncorp]), through Duke University, and finally to the historians. The main issue in dispute was intellectual control over the findings of the study. Duke insisted that the historians function as researchers, delivering in the final report their “best effort” to understand and explicate SCP. DARPA, through its agents, insisted that the historians deliver an acceptable manuscript. When no resolu- tion was achieved after months of negotiation, AOT replaced Meridian as the contracting agent and a contract was quickly signed along the lines proposed by Duke. Eighteen months later, however, when the con- tract came to be renewed, DynCorp replaced AOT and the same dispute resumed.