Illustrative Risks to the Public 1.37 Achieving Better System Development and Operation . 57 in the Use of Computer Systems 1.38 The Proper Role of Technology? . 58 and Related Technology Peter G. Neumann, Computer Science Laboratory, 2 Reference Materials 58 SRI International, Menlo Park CA 94025-3493 2.1 Books ............................... 58 Risks cases as of August 29, 2004 2.2 InsideRisksColumns . .. .. .. 58 Copyright 2004, Peter G. Neumann, SRI International EL243, Menlo Park CA 2.3 Pun-intended definitions . 60 94025-3493 (e-mail [email protected]; http://www.CSL.sri.com/neumann; This list summarizes items that have appeared in the Internet Risks Forum Digest telephone 1-650-859-2375; fax 1-650-859-2844): Editor, ACM SIGSOFT Software (RISKS) – which I moderate (comp.risks newsgroup) – and/or published ACM Engineering Notes, 1976–93, Assoc.Ed., 1994–; Chairman, ACM Committee on SIGSOFT Software Engineering Notes (SEN). In this collection of mostly Computers and Public Policy (CCPP); Moderator of the Risks Forum (comp.risks); one-liner summaries, (R i j) denotes RISKS volume i issue j; (S vol no:page) cofounder with Lauren Weinstein of People For Internet Responsibility denotes an issue of SEN, where there has been one volume per year, with vol 28 (http://www.pfir.org). being the year 2003; page numbers are given regularly from 1993 on; (SAC vol no) indicates an item in the quarterly SIGSAC Security and Control Review, where vol 16 is 1998, which was the final volume. The RISKS-relevant SEN material prior to 1995 is summarized in my Computer-Related Risks book (see below). SEN Contents material is now being brought on-line by Will Tracz: http://www.acm.org/sigsoft 1 Collected Items Listed by Categories 2 Some incidents are well documented, while others need further study. A few are of questionable authenticity, and are noted as such (“bogus???”). Please send me 1.1 Recent yet-to-be-merged items . 2 corrections and new cases, along with suitable references. This document is 1.2 Items Related to 11 Sep 2001 and Its Aftermath . 4 updated at least quarterly and is browsable on-line (ftp://ftp.CSL.sri.com/neumann/illustrative.html courtesy of 1.3 Space................................ 4 Otfried Cheong’s Hyperlatex). [Hyperlatex is wonderful Free Software: 1.4 Defense .............................. 6 http://www.cs.uul.nl/˜otfried/Hyperlatex).] This document is also printable in a two-column 8-point format (illustrative.pdf and illustrative.ps). 1.5 Military Aviation . 7 1.6 Commercial Aviation . 7 SEN regular issues, by year, volume&number 1.7 Rail, Bus, and Other Public Transit . 10 ..1976,vol 1: #1 = May; #2 = Oct ================================== 1.8 Automobiles ............................ 11 ..year 1977 78 79 80 81 82 83 84 85 1.9 Motor-Vehicle and Related Database Problems . 12 volume 2 3 4 5 6 7 8 910 --------------------------------- 1.10 Electrical Power (nuclear and other) and Energy . 12 Jan #1 1 1 1 1 1 1 1 1 1.11 Medical, Health, and Safety Risks . 14 Apr #3 2 2 2 2 2 2 2 2 Jul #4 3 3 3 3 3 3 4 3 1.12 Other Environmental Risks . 15 Oct #5 4 4 4 5 4 5 5 5 1.13 Robots and Artificial Intelligence . 15 ================================== ..year 1986 87 88 89 90 91 92 93 94 1.14 Other Control-System Problems . 16 volume 11 12 13 14 15 16 17 18 19 1.15 Other Computer-Aided-Design Problems . 16 --------------------------------- 1.16 Accidental Financial Losses, Errors, Outages . ..... 16 Jan #1 1 1 1 1 1 1 1 1 Apr #2 2 2 2 2 2 2 2 2 1.17 Financial Frauds and Intentionally Caused Losses . ..... 18 Jul #3 3 3 5 3 3 3 3 3 1.18 Stock-Market Phenomena . 19 Oct #5 5 4 6 5 4 4 4 4 ================================== 1.19 Telephone Frauds . 20 ..1995,vol20: #1=Jan; 2=Apr; 3=Jul; 5=Dec 1.20 Other Telephone and Communication Problems . 20 ..1996,vol21: #1=Jan; 2=Mar; 4=Jul; 5=Sep ..1997,vol22: #1=Jan; 2=Mar; 4=Jul; 5=Sep 1.21 ElectionProblems . 22 ..1998,vol23: #1=Jan; 3=May; 4=Jul; 5=Sep 1.22 InsuranceFrauds . .. .. .. 25 ..1999,vol24: #1=Jan; 3=May; 4=Jul ..2000,vol25: #1=Jan; 2=Mar; 3=May; 4=Jul 1.23 SecurityProblems . 25 ..2001,vol26: #1=Jan; 2=Mar; 4=Jul, 6=Nov 1.24Cryptography............................ 39 ..2002,vol27: #1=Jan; 2=Mar; 3=May, 5=Sep 1.25 April Foolery and Spoofs . 40 ..2003,vol28: #2=Mar; 3=May; 4=Jul, 6=Nov ..2004,vol29: #2=Mar; 3=May; 4=Jul, 1.26 PrivacyProblems. 40 1.27 Spamming, Junkmail, and Related Annoyances: . 45 Read the Risks Forum as comp.risks if you can, or send e-mail to 1.28 Other Unintentional Denials of Service: . 46 [email protected] for a subscription, single text line "subscribe" (append desired address only if not your From: address), or "info" for info. Send 1.29 Law Enforcement Abuses, False Arrests, etc.. 48 contributions to [email protected]. Archives are available at http://www.risks.org, 1.30 Identity Theft, Internet Fraud, Mistakes, Related Problems . 49 which redirects to Lindsay Marshall’s Web site at Newcastle http://catless.ncl.ac.uk/Risks/, including a nice search facility. Specific issues can 1.31 Other Legal Implications . 50 be read directly as http://catless.ncl.ac.uk/Risks/I.J.html 1.32 Other Aggravation . 50 [where I=volume#, J=issue#]. SRI’s archive is at ftp://ftp.sri.com/risks or by "ftp ftp.sri.com", "login anonymous", "cd 1.33 Calendar/Date/Clock Problems including Y2K . 53 risks" (which gets the "dir" for the current volume, and "cd i" then gets you into the 1.34 TheGameofChess: . .. .. .. 56 subdirectory for noncurrent volume i). An Australian mirror is at http://the.wiretapped.net/security/textfiles/risks-digest/. “Inside Risks” distills some 1.35 Miscellaneous Hardware/Software Problems . 56 of the discussion into a monthly inside-back-cover column in the Communications 1.36 Other Computer System Development Difficulties . 57 of the ACM. The list of columns to date is given at the end of this document. 1 My book (Peter G. Neumann, Computer-Related Risks, Addison-Wesley (ISBN V(m/f?) Canadian grocery chain Sobeys’ software crash lasts 5 days (R 21 22) 0-201-55805-X) and ACM Press (ACM Order 704943), 1995) summarizes many of Vf University of Washington server crash leaves thousands of students unable to these cases and provides additional analysis. (A few errata for the first three register for classes (R 22 38) printings are on my Web page, noted above.) Most of the (S vol no) items listed i Risks of not-quite-identical keyboard layouts (R 21 26) ¾¼ below for ÒÓ are discussed in the book; more recent items generally include $h United Airlines Web site for one hour accidentally offered SFO-Paris round trips the relevant on-line (R i j) references. If you cannot find the book in a bookstore, it for cost of taxes and fees only (roughly $30 instead of $300) (R 21 24-25) is on amazon.com, or call A-W within the U.S. at 1-800-822-6339 – or if you are f Japanese modem misdialing seemingly at random in pulse-dial mode (R 21 25) outside of the U.S., 1-617-944-3770 and ask for International Orders. The book is eh EoExchange shuts down free ad-supported services without warning; customer now also available in Japanese (ISBN 4-89471-141-9). Instead of trying to produce data lost (R 21 32) a second edition in the face of a massive influx of new RISKS cases, the fourth and i PC virtual-parrot squawks confuse firemen (S 26 6:10, R 21 46) fifth printings of the book gives the URL for the Addison-Wesley Web site mf Power outage causes motorized-chair shopping carts to run amok; default state (http:www.awl.com/cseng/titles/ISBN-0-201-55805-X/), which includes the first brake off, in forward gear (R 21 50) chapter of the book and an extended preface. That Web site and my own contain m Brownout-lowered voltages take out computers in Livermore (R 21 51) further material that would otherwise have gone into the second edition. f Cable theft results in network congestion when Seti@Home screen savers are unable to access Seti servers (R 21 48,53) Henry Petroski (among others) has noted that we rarely learn from our successes, fe Carefully planned seamless British Telecom BT SurfTime upgrade seemed very and must learn more from our failures. The collection of cases cited here provides seamy, with premature cancellation of old service (R 21 44) rich opportunities for reflection that could help us to avoid similar problems in the hie Risks in MacOS 10.2.4 update and httpd.conf replacement (R 22 56) future. Unfortunately, it also demonstrates that the same types of mistakes tend to $fe UK magistrates courts staff upgrade failure requires two sets of systems instead of recur, over and over... one, and a huge windfall for the deficient contractor (R 21 59) SEN and RISKS also consider approaches for developing better computer systems, fe Adobe Acrobat 5.0 pdf upgrade not backward compatible (R 21 59) e.g., safer, more reliable, more secure, fewer cost and schedule overruns, etc. There e NASA data from 1970s lost due to “forgotten” file format (R 21 56) are many approaches to developing sound systems; none is guaranteed. Whereas f California DMV sorting machine sends licenses to wrong people; 8-year-old sorting the emphasis in the following list is on problems rather than on would-be solutions, machine blamed (R 21 39) the pervasive nature of the problems suggests that techniques for the effective * IBM auto dashboard system can shoot water at drivers not answering questions development and operation of computer-related systems are frequently ignored.