Biblio Tech Editor: Marc Donner, [email protected]

AI Bites Man?

ver the years, people have explored the broader Cyberpunk MARC and its roots DONNER implications of many seminal ideas in technology These works naturally evolved into a Department scarier version of the future. Cyber- Editor through the medium of speculative fiction. Some punk, one of the most fascinating threads in speculative fiction, is epito- of these works tremendously influenced the tech- mized in the work of William Gib- O son, who startled us many years ago nical community, as evidenced by the broad suffusion of terms into with a short story called “Johnny Mnemonic,” now included in the 1986 collection Burning Chrome (and its working vocabulary. When that is clearly derived from our own made into an unsuccessful 1995 Robert Morris disrupted the bur- but that makes a few technically plau- movie starring Keanu Reeves). Cy- geoning Internet in 1988, for ex- sible changes to our underlying as- berpunk stories generally feature a ample, the computer scientists try- sumptions. Vernor Vinge’s True dystopic world in the near or distant ing to understand and counteract Names represents such a world, in future in which technologies emerg- his attack quickly deemed the of- which the size and power of com- ing today have changed the ground fending software a “worm,” after a puter systems has grown to the point rules of life. term first introduced in John where artificial intelligence capable of Brunner’s seminal 1975 work, The passing the Turing test is beginning to 1949 Shockwave Rider. Brunner’s book emerge. Vinge’s most fascinating There isn’t a straight line from worlds launched several terms that became speculations involve the genesis and that resemble ours to cyberpunk. The standard labels for artifacts we see utility of these artificial intelligences, genre morphed over the years and today, including “virus.” and he explores the notion that AI decades through a variety of novels. In future installments of this de- might emerge accidentally, a theme Although cyberpunk is most strongly partment we’ll look at the important that appears elsewhere in books like identified with William Gibson, its writers, thinkers, works, and ideas in Robert A. Heinlein’s The Moon is a roots go much further back—all the speculative fiction that have got us Harsh Mistress and in Thomas J. way to George Orwell’s Nineteen thinking about the way technological Ryan’s The Adolescence of P-1. Eighty-Four. change could affect our lives. This is In True Names, Vinge suggests a In 1949, when Orwell published not to imply that science fiction writ- radical use for such AI capabilities, the book that is now a staple of US ers represent a particularly prescient namely the preservation of the self high-school curricula, television was bunch—I think the norm is ray guns beyond the body’s death. Forget still a novelty in most households, al- and spaceships—but when they’re cryogenically freezing your brain or though the technology itself had good, they’re very good. And what- body in hope that someone will been around for 20 years. With TV’s ever gets us thinking is good. “cure” old age, he says—instead, fig- successful integration into modern To get started, let’s take a look at ure out how to save the contents of life, Orwell’s vision of a totalitarian some of the key subgenres and eras in your memory and the framework of future in which governmental con- science fiction’s history (see the “In- your personality in a big enough trol is mediated through two-way fluential Books” sidebar, p. 65). computer. If this AI passes the Turing television feels somewhat dated. test, then certainly your friends and Anyway, Orwell’s mastery of the lan- Worlds relatives won’t be able to tell the dif- guage and deep insights into many like our own ference. But will you know you’re human issues, including the relation- Some of the best (and earliest) science there? Will this AI be self-aware? ship between memory and truth (as fiction work speculates on a world Will it have a soul? Winston Smith discovers when he

PUBLISHED BY THE IEEE COMPUTER SOCIETY 1540-7993/03/$17.00 © 2003 IEEE IEEE SECURITY & PRIVACY 63 Biblio Tech

those of the 1969 world that pub- lished the book. Editor’s Intro Brunner, writing six years later, explores more fundamental ques- n this department, we will explore a range of thinking about the way techno- tions of identity and human relation- Ilogical change might affect our lives. We won’t consider just any science fiction ships in a future world in which a vast story that seems attractive. We want a set of works and authors proved to be sig- global network of computers has nificant, even if they only speculate on a world that is clearly derived from our own. changed the dynamic. This world is We seek submissions from engineers, scientists, and writers on these topics. scary and alien, although not as scary Reviews of individual works or of groups of related works are welcome. We would and alien as the one that Gibson like interesting interviews with prominent creators of speculative fiction, whether would reveal just six years later. authors, screenwriters, or directors. Any work that relates or did relate current Brunner makes clear the scariness of events in technology to possible future impacts is fair game. an entirely digitally mediated iden- Send your submissions, ideas, and comments to Marc Donner, donner@ tity early in the book when Nicky tinho.net. Halflinger’s entire world—electric power, telephone, credit, bank ac- counts, the works—is turned off in starts a diary and discovers the sub- ine living and finding happiness. I revenge for a verbal insult. versive power of a historical record) cherish the humor and the optimism Like Star Wars two years later, the have prevented obscurity. about relationships between artificial technological marvels of The Shock- An open question is whether new and natural intelligences that led wave Rider are a bit creaky and imper- technology tips the balance toward Heinlein to name the leading human fect, rendering them adjuncts to a central control, as Orwell feared, or character Manuel just so Mike the plausible future world rather than toward liberty, as many have specu- computer could say things to him central artifacts worthy of attention lated when considering the role of like, “Man, my best friend.” themselves. This is characteristic of faxes, photocopiers, and even the In- Things were changing rapidly in this genre’s best writing—it validates ternet in the collapse of the former the technical world in 1969 as well. the importance of technology by pay- Soviet Union. Dating back to that year, all the docu- ing only peripheral attention to the ments that have described and de- technology itself. 1969 fined the Internet have been numbers In the technical world, Vint Cerf, Heinlein’s thinly veiled romance of in the RFC (Requests for Com- Yogen Dalal, and Carl Sunshine pub- the American Revolution, The Moon ments) series. Each document is lished RFC 675 “Specification of In- is a Harsh Mistress, begins with Manuel numbered sequentially, starting with ternet Transmission Control Pro- Garcia O’Kelly’s discovery that the RFC 1. RFC 4 (see www.faqs.org/ gram” in December 1974, making it Lunar Authority’s central computer rfcs/rfc4.html) is dated 24 March the earliest RFC with the word “In- (“Mike”) has become conscious and is 1969. It documents the Arpanet, ternet” in the title. In November developing a sense of humor. I still use which would later become known as 1975, Jon Postel published RFC 706 Heinlein’s observation that some jokes the Internet, as having four nodes. “On the Junk Mail Problem.” are “funny once” in teaching my own Two years later, Intel would introduce young son about humor. its 4004, the first commercial micro- 1977 As with True Names, Mike acci- processor. The 4004 had a 4-bit-wide In 1977, Macmillan published Thomas dentally reaches a level of complexity arithmetic logic unit (ALU) and was J. Ryan’s novel The Adolescence of P-1. It that mystically tips it over the edge developed for use in a calculator. was an age when vinyl records had to from being a machine to being a per- be turned over, when everyone son. Among Mike’s numerous 1975 smoked (although not tobacco), when achievements that anticipate contem- The Shockwave Rider is more about the 256 Mbytes of core was an amount be- porary technological progress is the potential role of computers, net- yond imagination, and when a charac- creation of a synthetic person, Adam works, and technology in society than ter in a book could refer to 20,000 ma- Selene, presented entirely through The Moon is a Harsh Mistress. In Hein- chines as “all the computers in the video and audio. lein’s work, the computer’s role is not country.” Unlike the cyberpunk main- much different than that of a person In Ryan’s book, as in Heinlein’s, stream, which Heinlein anticipated with magical powers. The computer’s computer intelligence emerges acci- by over a decade, Mistress shows a accomplishments are technically dentally, although in this case by the world vastly different from this one plausible, but the operational aspects networking of many computers but in which most of us could imag- of Heinlein’s society are much like rather than through the assembly of a

64 JANUARY/FEBRUARY 2003 http://computer.org/security/ Biblio Tech

large single machine. The precipitat- along with a new name—the Inter- story transposed to the key of sci- ing event is the creation of a learning net. Computer scientists around the ence fiction. program by a brilliant young pro- country were avidly reading RFC Despite the uninspired company, grammer, Gregory Burgess, whose 789, which documented a now-fa- True Names had an electrifying effect fascination with cracking systems mous meltdown of the Arpanet. on the computer-science community. leads him to construct several recog- Epidemiologists were talking about The title of the novel refers to a com- nizable AI artifacts. Of course, the an outbreak of a hitherto very rare mon theme of fairy tales and magical great pleasure of fiction is the ability cancer called Kaposi’s Sarcoma, an logic—knowing something’s “true to elide the difficult details of building outbreak that would be recognized name” gives you complete power over things such as P-1’s program genera- in the following year as a harbinger it. In the world that Vinge concocts, tor, which is the key to its ability to of a new and terrifying disease: knowing a computer wizard’s true evolve and grow in capabilities be- AIDS. IBM, acceding to an internal name permits you to find his or her yond those that Burgess originally de- revolution driven by its microcom- physical body. Even if entering the veloped for it. puter hobbyists, introduced a new Other Plane didn’t leave your body The Adolescence of P-1 is full of product code-named “Peanut,” the inert and defenseless, revealing the quaintly outdated references to data- IBM Personal Computer, which body’s location renders it vulnerable to processing artifacts that were current catapulted Intel and Microsoft to the attack from a variety of long-range in the mid 1970s, reflecting Ryan’s forefront. Pundits were moaning weapons. More than that, however, as day job as a computer professional on that US industrial prowess was a in The Shockwave Rider, exposure of the West coast. Those whose careers thing of the past and that in the fu- your true name makes your infrastruc- brought them into contact with IBM ture Americans were destined to play ture vulnerable to a range of denial-of- mainframes in their heyday will be third fiddle, economically, to the service attacks. This represents a rather amused by the author’s use of opera- Japanese and the Germans. simplistic view of security models, al- tional jargon to provide atmospherics Vinge’s True Names is a novelette, though one that the modern world in the book. a short novel, rather than something hasn’t left very far behind, seeing how Ryan also takes a much less that could be published economi- only a relatively few years ago a Social Polyanna-ish view of the relationships cally as a monograph. As a result, it Security Number was all you needed between humans and artificial intelli- was published in a cheesy Dell series to access most of someone’s assets. gences. Unlike Heinlein, who clearly called “Binary Star,” each number of William Gibson’s “Johnny Mne- expresses in Mistress that sentience which featured two short novels monic” appeared in Omni magazine implies a certain humanistic benevo- printed back to back, with the rear in May 1981. It introduced a world lence, Ryan explores the notion that cover of one being the upside-down destined to become famous with Gregory Burgess’s AI must have a front cover of the other. For you in- books like 1984’s Neuromancer and strong will to survive, which would curable trivia nuts, True Names ap- 1986’s Count Zero. lead it to be untrusting toward people. peared with a truly dreadful effort In 1981, only the paranoid were P-1 at one point commits murder, for called Nightflyers, a gothic horror saying what Johnny Mnemonic says, example, and unapologetically ex- plains its actions to Burgess. Ryan wrote only one book, so he must not have derived much encour- Influential books agement from the book’s reception, which is unfortunate. His writing is a ost of the books described in this installment have been recently reissued bit uneven, but it’s certainly enter- M(original editions are out of print, for the most part). The publication dates taining, and his sense of the important in this list reflect that discrepancy and are not the books’ original year of publi- issues has held up well. cation.

1981 J. Brunner, The Shockwave Rider, Ballantine Books, 1990. For some reason, 1981 saw the pub- W. Gibson, Burning Chrome, Ace Books, 1994. lication of two seminal stories in the R.A. Heinlein, The Moon Is a Harsh Mistress, St. Martin’s Press, 1997. cyberpunk oeuvre. In the technol- G. Orwell, Nineteen Eighty-Four, Knopf, 1992. ogy world, the Arpanet was prepar- T.J. Ryan, The Adolescence of P-1, MacMillan Publishing, 1977. ing to transition from the old NCP N. Stephenson, Snow Crash, Bantam Doubleday, 2000. technology, which it had outgrown, N. Stephenson, The Diamond Age, Bantam Doubleday, 2000. to the new IP and TCP protocols V. Vinge, True Names, Tor Books, 2001. that would bring it fame and fortune

JANUARY/FEBRUARY 2003 http://computer.org/security/ 65 Biblio Tech

precise, price—is so deeply built into our psyche that thinking about alter- A set of nongeographic structures might native models is very difficult. I re- member a short story, read years ago emerge, perhaps like Medieval guilds. (title and author lost to me), which explored the same issue much more superficially, although it came to some of the same conclusions. In this “We’re an information economy. proposition, but he does it particu- story, a pair of matter-duplicating ma- They teach you that in school. What larly well. Stephenson proposes that a chines is left mysteriously on a they don’t tell you is that it’s impossi- set of nongeographic structures might doorstep somewhere. Once they be- ble to move, to live, to operate at any emerge, perhaps like Medieval guilds, come widely available, all material level without leaving traces, seem- structures that organize people into scarcity is banished. What drives eco- ingly meaningless fragments of per- groups based on some other selection nomic activity? Why do people sonal information. Fragments that criteria, possibly entirely voluntary. work, strive, compete? can be retrieved, amplified, … .” Brunner comes close to the same no- In The Diamond Age, Stephenson Today, however, every consumer tion, although his organizing entities asserts that the drive to strive and with a credit card and an Internet are corporations and the geographic compete won’t go away just because connection understands this point in- government continues to have a mo- the material forces that created it dis- tuitively. Who says nothing changes? nopoly on force. For Stephenson, appear. He combines the notion of however, the US government is just very small machines and the recently 1992 one of the many competing groups demonstrated capability to manipu- The year after the Gulf War was a participating in the game. late individual atoms and creates a US presidential election year. He raises fundamental questions, world in which atomic raw materials UUNET and ANS, among others, though. How will people organize are piped to nanotechnical factories were duking it out over the Inter- themselves? Religion? Race? Occu- called matter compilers, which can net’s commercialization. Bloody pation? Philosophy? Ethnic origin? assemble virtually anything, given the civil war was beginning in the terri- These self-organized groups could design. Scalability arguments underlie tory previously known as Yu- manifest themselves as a collection of his claim that the fabricated objects goslavia. And Bantam published confederated enclaves providing will have a certain limited physical Neal Stephenson’s Snow Crash. economic, physical, and emotional aesthetic, something that Alvy Ray Stephenson, like Ryan and Vinge, security to their…members? Citi- Smith and others who have explored is a writer with real experience as a zens? Subjects? His insight is a pow- the use of fractals and other tech- computer professional. Unlike Hein- erful one. The craving for these niques for adding a realistic tinge of lein and writers like him, for whom forms of security is deeply rooted randomness to computer-generated technological artifacts always have an and part of what makes us human. images might dispute. aura of magical unreality, Stephen- What makes Orwell’s Nineteen son’s grasp of the underlying technol- Eighty-Four ring so false to us, and ogy is so deep and his writing skills so accentuates the horror of Orwell’s hope you had as much fun reading powerful that he is able to weave an vision, is the complete loss of any ac- Ithis brief history as I had in re- entirely credible world. knowledgement of those needs in searching and writing it. Preparing it In Snow Crash, the world starts out people. Stephenson corrects that gave me an opportunity to revisit as the ultimate virtual reality video omission, and the world of Snow some of my favorite books and try to game. What Stephenson then ex- Crash that results is not nearly as articulate my reasons for believing plores is the possibility that these syn- dystopic as Orwell’s or even Gib- them important. In future columns, thetic worlds will become real, at least son’s. we will examine some of these books in the sense that the things that hap- in greater detail, along with the work pen in them can be of material signif- 1995 of other writers and thinkers. icance in the meatspace world that With the publication of The Diamond our physical bodies inhabit. Age, subtitled “A Young Lady’s Illus- Marc Donner is an executive director in Stephenson explores a fascinating trated Primer,” Stephenson explores the Institutional Securities division of Mor- thesis—suppose the taxing ability of the implications of a world in which gan Stanley where he focuses on system geography-based governments is material scarcity is no longer an as- and data architecture around client rela- tionships. He is a member of the IEEE eroded in fundamental ways. He’s not sumption. The relationship between Computer Society and Usenix. Contact the first to have considered this scarcity and value—or, to be more him at [email protected].

66 JANUARY/FEBRUARY 2003 http://computer.org/security/