The Pre-Commercial Internet Under DARPA

Nurturing the Accumulation of Innovations: Lessons from the Internet by Shane Greenstein1 As the Internet diffused throughout the 1990s, it touched a wide breadth of economic activities. The diffusion transformed the use of information technology throughout the economy. It led to improvements in products, lower prices, the development of new capabilities, and the development of many innovations that enabled productivity improvements among business users. It diffused to the majority of homes and businesses, altering the way people shop, research, play, and relate socially. The Internet began as a government sponsored operation in the 1970s and 1980s and grew into a commercial industry in the 1990s. At first, the Internet lacked market-oriented focusing devices and/or economic inducement mechanisms typically associated with directing efforts toward the most valuable innovative outcomes.2 There were contracts for carrier services between government buyers and commercial suppliers, for example, but no general market orientation towards the pricing of the exchange of traffic between carriers. There also were a few providers of Internet equipment for government users, but no waves of inventive entrepreneurial entry. There were managers who understood the specific needs of their niche user communities, but no possibility for tailoring new products and services to every potential new set of users. How could an institutional setting that lacked market-orientation yield a set of innovations that supported the creation of massive market value only a few years later? This chapter helps explain the progression. The chapter divides the Internet’s development into a pre-commercial and commercial era. The pre-commercial period encompasses the 1970s and 1980s and some of the early 1990s, when the government controlled the research and development of the Internet and its components. The commercial period, which arose after the government opened control of the network to commercial interests, encompasses the mid-1990s and onward to present day. These two eras illustrate two distinct models for accumulating innovations over the long haul. The pre-commercial era illustrates the operation of several useful non-market institutional arrangements. It also illustrates a potential drawback to government sponsorship – in this instance, truncation of exploratory activity. The commercial era illustrates a rather different set of lessons. It highlights the extraordinary power of market-oriented and widely distributed investment and adoption, which illustrates the power of market experimentation to foster innovative activity. It also illustrates a few of the conditions necessary to unleash such growth, such as standards development and competition, and nurturing legal and regulatory policies. The Pre-commercial Internet under DARPA3 It may be tempting to compare the Internet to historically archetypical big inventions sponsored by government, such as the Manhattan and the Apollo projects. However, these archetypes for developing technical breakthroughs simply are not good models for understanding what happened during the creation of the Internet. The Internet was not a single urgent project in a single lab devoted to engineering a single object. In fact, the early development of the Internet is rather less exciting than commonly assumed. The Internet began as a government sponsored project with a restricted set of users and uses. It occurred at a time when theory pointed in a few promising directions, but nobody, not even the experts, knew where implementation would lead. The project involved a vastly dispersed set of technically adept 1 Elinor and Wendell Hobbs Professor, Kellogg School of Management, Northwestern University. I thank Guy Arie, Tim Bresnahan, David Clark, David Crocker, Rebecca Henderson, Franco Malerba, Richard Newell, Bonnie Nevel, John Quarterman, Craig Partridge, Richard Schmalensee, Alicia Shems, Scott Stern, and Stephen Wolff for extraordinarily useful conversations and comments. This essay is part of a larger project funded by the Kaufman Foundation, the Searle Center at Northwestern University, and the Dean’s Office at the Kellogg School of Management at Northwestern. I am grateful for the funding. I am responsible for all remaining errors. 2 For more on focusing devices in general, see e.g., Rosenberg (1977). 3 Much of the material in this first section summarizes Greenstein (2009a). participants with a shared interest in the project, but otherwise heterogeneous needs and outlooks. The Internet developed slowly, and through a rather mundane process, accumulating capabilities over time from a vast number of contributors. The Internet’s early development fit into an archetype called “collective invention.” Collective invention is “a process in which improvements or experimental findings about a production process or tool are regularly shared.” 4 There was no single user who demanded a technology such as the Internet, nor any single inventor for it. Rather, five partially overlapping groups played a role in shaping attributes that each valued, with an accumulation of innovative contributions over time. The first two groups were the primary decision makers at funding agencies: the Department of Defense and the National Science Foundation. The other three were programmers/inventors, administrators, and application users. Many were funded by the government agencies and given considerable discretion. Others became participants over time and added their own contributions within their own budgetary limitations. The U.S. military budget served as the first source of funds for the pre-commercial Internet, while the National Science Foundation largely served as the source of funds from 1986 to the end of government involvement in 1995. At a very basic level, the U.S. government paid for much of the research and development of the Internet during this period, and the government was the organization behind the early “demand” for the breakthrough technical achievements that became recognized as the Internet. This effort did not begin like a military procurement project, as if the Internet were a military rocket procured from several suppliers. In such a procurement process, a group of expert U.S. military personnel consider and deliberate at great length with great foresight about the needs of the government, and then they issue a set of specifications for a product or service with a pre-determined set of attributes. However, in the case of the Internet, the Department of Defense’s Department of Advanced Research Projects Agency (DARPA),5 funded the core research and development work that led to the Internet – indeed, this work was but one of many DARPA projects on the frontiers of computer science.6 The project first intended to build prototypes for a packet-switching data-communications network of networks, which pushed the boundaries of computing at the time. Both “packet-switching” and “a network of networks” were budding theoretic concepts. DARPA’s administrators wanted innovations in the form of ideas, new designs, and new software. The administrators also desired all of these innovations be portable to military operations in the long run, as required under the Mansfield Amendment (stipulating that Department of Defense funding be relevant to military’s mission). While the demand for these innovative solutions was quite general, the specifics were undetermined. The U.S. military faced issues with its own computing facilities and operations that justified the research and development expense, even though its own managers could not concretely describe the object that would result.7 The military sought a robust design for a communications network, and its potential value was self-evident. Keeping communications functioning in spite of a blown/cut line, for example, has military value in hostile battlefield conditions. Packet switching held the promise to achieve this attribute by allowing data to flow along multiple paths, unlike a circuit-switched telephone network in which calls follow a pre-set path programmed into central office telephone switches. In principle, an inexpensive packet switching network could also cover vast geographic distances, which could support the sharing of expensive computing resources over such distances. That too had self-evident military value. For example, military users in many locations – even potentially dangerous locations – could access databases housed in another (potentially safer) location. 4 See Allen (1983) for the introduction and illustration of the concept. The quote is from Meyer (2003), who provides further extensions to modern examples. 5 This organization was originally founded as the Advanced Research Projects Agency (ARPA), and became DARPA in 1972. For the sake of simplicity, I use only the name DARPA throughout this chapter. 6 Norberg et al. (1996), Roland and Shiman (2002). 7 Norberg et al. (1996) repeatedly stress that DARPA’s funding of packet switching research in the 1960s and 1970s met concerns about whether the funding was relevant to military mission, as required by the Mansfield Amendment, which was proposed several times, and eventually passed in 1973. The research anticipated enhancing the “command and control” capabilities of commanders increasingly reliant on their computing resources. 1 An additional technical and pragmatic aspiration also played a role. An ideal network of networks could facilitate the movement of data between computer systems – mostly mainframes in this

Load more