Everything is Intertwingled :: New Perspectives on Computer History and a Community of Users
Accepted for publication for the Interarts Journal, Edited by Ela Kagel and Jocelyn Robert by Lisa Moren, August 2011
Like a warbling fly through virtual reality, the nausea is in my head. The angular caps of the good dark, cool-colored water were now a soft, warm ripple that looked like something between the plasticity of a Pixar animation and the solarizing effects of an early 1960s Nam June Paik video, when digital effects were brand new and unnatural color contrasts were big. If you bent down to eye level with the water, the contrast allowed you to see the hollow oval shapes of the shimmering oil sheen.
I was a cynical artist surrounded by sci-fi enthusiasts, and like them, empowered by the accessibility of such high-end technology even I was if unsure of what it all meant. In 1991 at the Banff Centre for Art, I saw artists, humanist, feminists, people of diverse ethnicity and sexual orientations, as well as art engineers, who emerged from a 1980s art movement fraught with theories and politics, congregating at the residencies in Canada. They worked in a variety of materials and questioned the rationale of material vs. its simulation, interface and code, and had lofty goals of reversing the once believed inevitable one-point perspective destiny of technology into a pluralist, post-modern direction.
The engineering oriented artists at the Banff residency brought a wealth of information that could tell us where our cursors came from and about maps of the future, but more significantly they could tell us why. They knew about emerging forms of writing that branch out upon request through a new kind of literature called hypertext. They could trace it’s lineage to 1963, when Ted Nelson primarily argued that the experience of retrieving data from computer systems should be developed not only with the user in mind, but in collaboration with the user, or better, a community of users. This was radical in the early 1960s when several corporations, including IBM, were in a battle over whether to manufacture office equipment and big linear, sequential computers, or, figure out the bugs in the less dependable and more risky, smaller, non-linear, modular systems communication-based. IBMs President, Thomas Watson Jr., began a research program in 1963 gambling that the company’s main commodity was centered on the movement of information, not making big office machines. Days after the announcement of their new product, the System/360, IBMs stock nearly doubled to what it had been 18 months prior1 and they set the industry on a risky path of networked communication that depended on input from their community of users.2
The young Ted Nelson watched this path. Motivated by the inadequacies of his education, and its inability to meet his own learning style, Nelson was first inspired by the Memex. The memex was first described in “As We May Think” by Vannevar Bush, in 1945 (see fig 1). As the former Director of the research team for the Manhattan Project during WWII, Bush sought to transfer his atomic research away from the one-point perspective goal of destruction, into a collective memory system that could better explore loftier goals of scientific truth:
“Presumably man’s spirit should be elevated if he can better review his shady past and analyze more completely and objectively his present problems... The applications of science have built man a well-supplied house, and are teaching him to live healthily therein. They have enabled him to throw masses of people against (one) another with cruel weapons. They may yet allow him truly to encompass the great record and to grow in the wisdom of race experience. He may perish in conflict before he learns to wield that record for his true good.3 — Vannevar Bush, As We May Think, The Atlantic Monthly, 1945 Bush imagined a great record of knowledge holding all of human memory in order for society to become wise, or at least more broadly spend their time engaging in the sublime rather than the atom bomb. As a scientist, Bush understood that researchers worked best by finding patterns in seemingly disparate contexts. With his hypothetical Memex machine, he sought to create a data archive and retrieval interface that more directly compared associative and visual patterns, that mimicked how scientific brains naturally process information. Bush knew that computers of the time used a linear, sequential logic that had been imposed since the Renaissance and was no longer relevant. In fact, the random access of data, non-sequential processing and parallel performance of equations were at the core of computer processing. His Memex proposed to be a more flexible interface based directly on comparing visual information, rather than tediously
1 http://finance.yahoo.com/q/hp?s=IBM&a=09&b=1&c=1963&d=04&e=1&f=1964&g =d 2 http://www-03.ibm.com/ibm/history/history/decade_1960.html 3 http://www.w3.org/History/1945/vbush/vbush8.shtml sorting through intermediums such as library card catalogs. In 1945, “As We May Think” argues that the interface of the Memex can bring us closer to the way we think, and therefore it’s a path to wisdom over destruction.
Ted Nelson also looked at his 1960s contemporaries, sociologist and engineer Douglas Engelbart and Ivan Sutherland. Both researchers were also deeply influenced by Vannevar Bush’s philosophical text and were just as eager to engineer aspects of the Memex. Ivan Sutherland’s 1962 MIT Phd thesis project was the flexible interface Sketchpad that allowed a user to draw on a screen. Based on an x,y axis, a pen allowed the user to draw lines on a computer monitor between 2 points and interactively modify that line. By 1964, Sutherland briefly became Director of the US Military project ARPA4 and an inspiration to Doug Engelbart. In 1962 Engelbart published Augmenting Human Intellect: A Conceptual Framework where he builds on the foundation laid by Vannevar Bush. Engelbart saw the current state of technological advancement limited by an impossible Catch-22 loop. He believed that the limitations in our technologies control the knowledge that we are able to learn from them, and in turn, the knowledge-output from computers restricts our ability to process further knowledge or advance the medium. Similar to playing poker or the stock market solely on mathematics, Vannevar Bush expressed the unnecessary limitations of computers when he said “If scientific reasoning were limited to the logical processes of arithmetic, we should not get far in our understanding of the physical world.”5 He sought to break the cycle at Stanford Research Lab (SRI), and with ARPA support he led the Augmentation Research Center where in 1968 he presented his famous public demonstration (dubbed mother of all demo’s) of this concept by tying a brick to a pencil and writing with it. He said that in order to loose the brick we needed to make computer information more transparent, he believed interfaces needed to be a more direct augmentation to the way we think. Engelbart modernized the work laid out by Bush and saw a future where people drive computers as intuitively as they navigate cars, with unmediated eye-body coordination using foot pedals, stick shifts and steering wheels. His SRI research exploited natural body
4 Advanced Research Projects Agency (ARPA), the US Military precursor to ARPANET, Milnet and the Internet. 5 Multimedia: From Wagner to Virtual Reality, Edited by Randall Packer and Ken Jordan, W.W. Norton & Company Ltd, London, 2001. Essay: Vannevar Bush, “As We May Think” pp 145. movements through a knee interface or a head-mounted device that was attached to the chin or nose. He also mined and applied lost inventions, such as the teleautograph, a pre-cursor to the fax machine that could transmit image data via telephone wire in 1888. With the desire to make the connection between human gesture and thought as direct as possible, Engelbart and his team designed objects that pioneered the field of human-computer interface. After explaining what he meant by augmenting human intellect, he put down the brick and demonstrated his own patented computer mouse.6 (see fig 2) Because Englebart’s mouse was inspired by Sutherlands use of the x, y axis in Sketchpad, the interactive mouse and screen were all compatible.
From these experiences, Nelson laid out a breadth of interface designs that turned known hierarchical learning structures into a format that was closer to the way the way that Ted Nelson’s brain thought. For example, Nelson’s Thinkertoys is a concept that exemplifies that “the written word is nothing less than tracks left by the mind”7 and therefore fragmentary by nature. He sought simplistic, flexible interfaces that were transparent enough for the user to focus on ideas. This goes well with his stimulating interactive learning devices, such as a joystick, that are again, closer to the way that Nelson and many others think (see fig 4). The word toy is both a pun and a direct observation of how our child-like minds are enthusiastic about developing new information that favors hands-on experiences over books and lectures. Nelson wasn’t against reading, he was looking for a new delivery system of literature, science, history, art, and saw the Gutenberg book format as a highly malleable and temporary format. Like Bush’s archive of scientific memory, Nelson imagined a Babel-like archive containing all knowledge linked in a complex and interwoven labyrinth. He believed that knowledge was already “intertwingled” and that the labyrinth-like system capable of connecting such knowledge
6 Many computer industry people such as IBM executives witnessed this demonstration in which the mouse design was based in part on the teleautograph. Englebarts assistant Bill English, later worked at Xerox Parc where a track ball version of the mouse was noticed by Steve Jobs during a 3 day visit in 1980. In 1983 Apple Computers launched the Lisa Mouse. http://en.wikipedia.org/wiki/Computer_mouse 7 Computer Lib/Dream Machines by Theodore Nelson, Chicago, IL copyright 1974, 5th Printing, March 1977, pp 111. was what he coined hypertext in the mid-1960s. Nelsons own words offer a raw insight to how his own processing synapses work:
A grand hypertext, then, folks, would be a hypertext consisting of “everything” written about a subject, or vaguely relevant to it, tied together by editors (and NOT by “programmers,” dammit), in which you may read in all the directions you wish to pursue. There can be alternative pathways for people who think different ways. People who have to have one thing explained to them at a time— many have insisted to me that this is normal, although I contend that it is a pathological condition—may have that; others, learning like true human beings, may gather and sift impressions until the ideas become clear.8
Even for large corporations, the humanizing of technology in the early 1960s was a shift away from product driven machines and into communication sharing tools and user-controlled networks. It was also a philosophical separation away from the fixed meaning in traditional media-delivery systems that were authoritative and hierarchical, and away from single-objective systems such as missile calculations. Instead, the shift moved toward democratic ideas of a community of users, computer-human interfaces, transparent information, and modular, multi- perspective technologies based on collective knowledge. The user was becoming part of the meaning of the system that they participated in and the blurring of user and author is introduced into computer culture.
It is no accident that artists from this time period were also taking fixed meaning off the wall and into locations that were unpredictable and open to random actions by the environment and their audience. As early as 1958 Allan Kaprow produced modular performances he called happenings, and his installations allowed his art viewers — or participants — to interact with the work through physical engagement. Fluxus artist, Dick Higgins was writing modular scripts for theater when in 1965 he coined the term Intermedia in order to describe these emerging relationships between art and life, and how flexible concepts take precedence over both fixed meaning and materials used. (see fig. 7) Higgins’s charts and diagrams that describe his theories resemble those of Ted Nelson’s, as well as Higgins’s similarly rambling texts that go with them. Although the single-objective, self-destructing kinetic machine by Jean Tinguely was pronounced the avant garde’s technological answer to art when it was on exhibit at the Museum of Modern Art in
8 Ibid. Computer Lib/Dream Machines. pp.32 1960; it was the underground performance work from this time-period produced by Higgins, Kaprow and others, while less technological in form, was nevertheless more similar in spirit to the human-computer interaction work of Nelson, Englebart and Sutherland. Hierarchies between art, artist and audience were blurring, acknowledging that the audience is part of the system that produces meaning in performance art, concept art, Fluxus and happenings was pervasive in the early 1960s.
Hypertext is a small tip of the scope of Nelson’s media concepts. His hypergrams and textstretch diagrams, Xanadu ramblings in his flipped upside down book, “ComputerLib/Dream Machines”, (see fig. 5) laid the foundation for many contemporary digital fields such as dynamic interface popular on smart phones, readers and tablets. His hypermap foresees both mapquests ability to change lists into maps and vice versa, and the google earth perspective of the world today. He describes how to dissect a frog on the screen with a light pen, hyper-comics, and on and on (see fig. 4). Ted Nelson lectured on hypertext in the mid-60s and his work was not published until 1974, but it soon became a cult mantra for geeks who had the technical know- how, but needed a purpose. Those who carried ComputerLib/Dream Machines in their backpack include founders, CEOs and designers for major software companies founded in the 1970s and 80s, including Apple Computers. Even established companies like CERN allowed the youthful programmer Tim Berners-Lee to develop and market the first branching web system able to access information through linking nodes. In 1989 Berners-Lee chose to name it hypertext. Along with the development of web browsers, hypertext allowed the nearly two-decade-old Internet to be accessed in a way that was closer to how non-experts think.
In the 1970s and 1980s, the fields of art and computers had only marginal overlap9 and the majority of the artists emerging in the early 1990s were skeptical that the Bush, Sutherland, Engelbart and Nelson lineage had any key to make their version of post-modernism relevant. As a generation of competent users, by now they already had the democratic freedom to drive the technology themselves without a prestigious residency at Xerox Parc or Bell Labs in order to
9 After several failed major museum exhibitions in the 1970s, only a few marginal venues and prestigious institutions, including MIT, Ars Electronics, Xerox Parc were among the institutions engaged seriously in art and technology research and display throughout the 1980s. make calculators cry, subjective maps, or a traditional native longhouse in cyberspace. (see fig 11) Interface is the content10 was one of the mantras during the emergence of a generation of projects where LED forests were sensual library retrieval systems and tables and photographs spoke when touched. Even fluids were used as interfaces precisely because they were so repelled by electronics itself and became an apt metaphor for feminine leaky bodies that were also once excluded by the technological patriarchy (see fig 12). While software studies is significant to trace the objectives of code from generation to generation, these emerging artists believed that if we focus exclusively on software, with code taking a leadership role, we merely extend the avant garde trajectory into new media, and miss the point of how integrated software is with the rest of culture and it’s social, political and economic complexities. Furthermore, if we progress purely on our knowledge gained through understanding software and code, we will be in the same Catch-22 loop described by Englebart 30 years prior where the knowledge-output from software restricts our ability to further process knowledge, and therefore further creativity. The artists of the 1990s were empowered by the hope to take the historical advancement of technology off the Cartesian grid trajectory that led predominantly to destruction and phobias, and divert it into a future that was gray, organic, complex and inclusive of diversity. What their utopia sought was to make media an art form that brought us closer to the world of knowledge, truth, and even femininity with non-western perspectives in it’s technical make-up, but what they got was the Internet with all its beautiful flaws, abstract links between continents of thought, and eventually, more and more corporate control. Nevertheless, I put on my gloves and headgear and walked around the streets of a parallel universe…
Images and colors are more intense than normal and the nausea returns. In order to relax, I move into a virtual pub for something to eat, but the warp speed delivery and various shapes of saturated cheese on my pizza, make me flinch without warning and the nausea consumes me again. The burnt orange color of my beer is also creepy. When I walk outside, I zoom into a pattern of too perfectly dispersed clouds that reminding me of the oil spill patterns and I’m again overwhelmed with a wave of dizziness. The nausea moves with me, slowly dissipating until the next abrupt reminder comes of when Captain Patrick put his hand on my shoulder
10 The Construction of Experience: Interface as Content by David Rokeby Digital Illusion: Entertaining the Future with High Technology," Clark Dodsworth, Jr., Contributing Editor © 1998 by the ACM Press, a division of the Association for Computing Machinery, Inc. (ACM) published by Addison-Wesley Publishing Company http://homepage.mac.com/davidrokeby/experience.html and said, “Are you alright?”, with his handkerchief over his nose and mouth. “That can happen when you lean over the side of the boat and then stand up quickly,” he said. But we both knew it was more. “It’s probably the chemicals, but it could be that we need to constrain the motion in the headgear.”
Today, artists are looking back as much as they are looking forward. Like the emergence of new media artists of the 1990s, intermedia artists today also seek new paradigms. However, artists are no longer deluded into thinking that new technologies are a path to new ideas. The classical media art projects of Paul DeMarinis undermine the objectives typical in the trajectory of both the avant garde and computer history. Fusing unexpected methods reminiscent of both author Jorge Luis Borges and Douglas Engelbart, DeMarinis merges poetry and politics from one time period, with abandoned inventions from another. Two projects each using the phenomena of water or fire as the sole output of recognizable audio are found in DeMarinis’s work “Raindance” and “One Bird.” “Raindance” is based on lost scientific research that dates as early as 1837. In this installation, DeMarinis allows audio frequencies to be carried on a precise number of droplets falling onto umbrellas that are carried by outdoor museum participants. A user, or group of users, approaches only a shower apparatus and are offered the interface of an ordinary umbrella. The shape of the umbrella acts as a parabolic speaker allowing the vibration in the droplets to translate a predictable melody to whomever stands under the umbrella. In his project “One Bird”, the viewer encounters a brass bird-cage on a stand with a flame trapped inside of it. The flame is a Bunsen burner with a small metal rod intersecting the flame that is connected to a disc player featuring a 19th c. operatic voice. The flame itself is amplifying Adelina Patti’s voice as the sole output. No speaker exists. Paul’s own patented technology was first discovered by Lee De Forest who lived during an era where candlelight and electric light illuminated side-by-side, and thereby allowing him to make his discovery while observing the flickering flames on his gas-lit chandelier. De Forest never saw through his hypothesis, but instead used this knowledge to invent the first vacuum tube in 1906, which became the necessary foundation for the invention of radio, television, radar and computers.
While some artists seek paradigms from the past, other artists seek a community of users through relational art, or finding ways where the viewer is acknowledged as a more meaningful participant in the art encounter. Human-computer interface began as a means for users to more deeply encounter knowledge, but the field has grown far beyond the one-on-one encounter. Ted Nelson foresaw “that when the real media of the future arrive(s), the smallest child will know it right away (and perhaps first). That, indeed, should and will be the criterion. When you can’t tear a teen kid away from the computer screen, we’ll have gotten there” but 40 years later we’re all connected to a community of users that Mr. Nelson can’t even imagine. Nelson’s human-computer interface thinkertoy has arguably evolved into a real lego drone toy11 being manufactured and even the United Nations has become connected to the manufacturing of the Playstation game. A UN investigation learned that much of the wars in the Democratic Republic of Congo (DRC) were advanced by international companies, including Americans, who sided with, and backed, the rebels in order to invade the DRC and illegally mined coltan, a necessary mineral to make tantalum, vital for the manufacturing of computers and portable devices.
“Kids in Congo were being sent down mines to die so that kids in Europe and America could kill imaginary aliens in their living rooms,” said Oona King, Member of British Parliament (1997 to 2005).12 The thieves were allegedly motivated by the high American demand for Playstation 2 in 2001. 80% of the worlds coltan comes from DRC and 40% of it’s products were sold to the US in 2007. 4 million people have died in the DCR in 10 years, in addition to the children who are still mining today and have no idea what coltan is used for. (see fig. 8 and 9”)
Since the early 1990s, the democratic technologies that acknowledge that the user is part of the system that artists, designers and engineers created have become ubiquitous. In order to be relevant artists are involved in politics, media, materials and communities in order to generate a media art that embraces a broader view than what the Catch-22 knowledge derived from the study of software, code and hardware alone can ever offer. The 1960s cultural phenomenon of human-computer interface and community of users has expanded with the consequences of the
11 http://www.nytimes.com/2011/05/16/technology/16drones.html 12 Inside Africa's PlayStation War, by John Lasker, Tuesday, 08 July 2008 http://www.towardfreedom.com/home/content/view/1352/1 Internet and the North American Free Trade Agreement (NAFTA) since the 1990s. Media arts has moved from utopian ideas of one-on-one encounters about learning and better scientific research tools, to global modular socio-economic, political and ecological systems including American children playing with communities of unknowing participants. (See interviews of DCR children in coltan mines: http://pulitzercenter.org/video/congos-bloody-coltan)
This leads to questions such as how does our human-technological system interface with the world and how can humans better relate collectively to natural systems? These are the questions that Natalie Jeremijenko asks in her projects. She’s worked on developing “calm technologies” at Xerox Parc, from robotics to biological and environmental systems including Amphibious Architecture, Environmental Health Clinic and her work in progress Cross (X) Species Adventures in Biodiversity. XSpecies explores how human ecology is evolving due to environmental pillaging by natural and technological causes. For better or worse, the food chain is shifting and XSpecies uses gastronomical recipes on never before consumed species, and considers what humans have in common with those species. A community-of-participants often join Natalie and adventure into an unknown dinner at museums and galleries.
Patrick explained to me that the currents naturally form parallel circular patterns that bring these dispersed oil pieces together and keep them together. Unfortunately these are the same opposing currents that bring the smallest critters on the food chain together, from plankton to bait and briar shrimp. Therefore the food and oil will naturally congregate, and the basis of the ecosystem will immediately die. Patrick offered his passionate knowledge for details of both the ecosystem and the technologies of the oil and gas rigs alike. It was clear he grew up impressed with the harmonious balance in which the two worlds had co-existed. However, his pride diminished with each blob of oil, rather than fish, that we were catching.
Later we entered an oil slick. It was a solid area, several acres in diameter and different in texture and light than the rest of the Bay. The slick was strange, horrifying and almost mesmerizing in its subtle character. To be surrounded by it was a sad and calm experience, not the nauseas experience that was to come. Soon we moved from this soft, warm rippled water into smaller, white capped clean ripples — as distinct a transition as moving between the former East and West Berlin, including color, light, sound and the symbol of it all.
After the emergence of the Internet in the early 1990s, 3D, immersive virtual reality quickly became the silliest of cyber sci-fi, but it’s interesting that the search for how algorithms, pixels and grids will resolve our world, end up giving us a doll-like impression of how we may think that our universe operates. I can’t help but wonder in anticipation of 2040, the estimated year when all the astronauts and engineers involved with human moon landings will have died, if one of our most popular community of users, the American conspiracy theorists, will hold that no human has ever stepped foot on the moon? Will we be a community of users drunk in our games, immersive media and networked worlds of our own creation that art, life, fact and fiction blurring? In order to see and believe how inextricably connected our software, food chain, economy and world community really are, perhaps artists don’t need to vomit in the brackish waters devastated by the Deep Water Horizon rig disaster, like I did (or was it just a nauseating cyberspace fly through) to see that everything really is intertwingled.
“In an important sense there are no “subjects” at all; there is only all knowledge, since the cross-connections among the myriad topics of this world simply cannot be divided up neatly.”
Ted Nelson, Dream Machines, 197413 (page 45 DM) 5th Ed. 1977
13 Computer Lib/Dream Machines by Theodore Nelson, 1974, 5th Printing, Chicago, IL March 1977. Pp 45.