LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 26

HISTORY OF EDITED BY MISCHA SCHWARTZ

AN EARLY HISTORY OF THE

INTRODUCTION also offer a personal account of the stages of development in Internet histo- same events, as an autobiographical ele- ry. I present these threads and phases It is impossible to place the origins of ment in this story. In doing so, I aim to chronologically so we can revisit the the Internet in a single moment of time. further contextualize publications from history as it unfolded. One may find One could argue that its roots lie in the the period — my primary source materi- elaborations on this history in two earli- earliest communications technologies of als — with details from firsthand experi- er papers [4, 5] centuries and millennia past, or the ence. This perspective may add to our beginnings of mathematics and logic, or depth of historical understanding, in THE RESEARCH THREAD even with the emergence of language which the extent of personal detail does In January 19571 I began as a graduate itself. For each component of the mas- not imply a greater importance to the student in electrical at Mas- sive infrastructure we call the Internet, events presented. In focusing on the sachusetts Institute of Technology there are technical (and social) precur- work of individual researchers and (MIT). It was there that I worked with sors that run through our present and developers, I rely on the various publi- , who inspired me to our histories. We may seek to explain, cations that followed the work of these examine behavior as large numbers of or assume away, whatever range of individuals to link this story to the factu- elements (nodes, users, data) interact- component technologies we like. It is al historical record we will follow. There ed; this led me to introduce the concept equally possible to narrow Internet his- are, of course, many important personal of distributed systems control and to tory down to specific technologies with and institutional stories that have yet to include the study of “large” networks in which we are the most familiar. be told. The University of at my subsequent thesis proposal. In that There are also many individuals that (UCLA) is heavily men- MIT environment I was surrounded by may be said to have “predicted” the tioned in this column, as it was the site many and realized that it Internet. In 1908, foresaw of so much foundational work. I view would soon be necessary for them to [1] a technology that would allow “a this period as a synergistic surge of tech- communicate with each other. Howev- business man in to dictate nology, engineered by a magnificent er, the existing tech- instructions, and have them instantly group of researchers and developers nology of was woefully appear in type at his office in London amidst a defining period of challenge, inadequate for supporting communica- or elsewhere” and would allow global creativity, invention, and impact. tion among these data sources. This access to “any picture, character, draw- was a fascinating and important chal- ing, or print.” Thirty years later, H. G. lenge, and one that was relatively unex- Wells articulated [2] his idea of a BEFORE THE BEGINNING: plored. So I decided to devote my Ph.D. “World Brain” as “a depot where WO HREADS THAT EET research to solving this problem, and to knowledge and ideas are received, sort- T T M develop the and understanding ed, summarized, digested, clarified and The Internet did not suddenly appear of networks that could properly support compared.” These ideas were followed as the global infrastructure it is today, data communications. by a 1945 essay [3] by , and neither did it form automatically Circuit switching is problematic predicting a machine with collective out of earlier . Dur- because data communications is bursty, memory that he called the , with ing the late 1950s and early 1960s, two that is, it is typically dominated by short which “Wholly new forms of encyclope- independent threads were being woven. bursts of activity with long periods of dias will appear, ready-made with a One was the research thread that even- inactivity. I realized that any static mesh of associative trails running tually led to the net- assignment of network resources, as is through them, ready to be dropped into works of today’s Internet. This thread the case with circuit switching, would be the memex and there amplified.” followed three possible paths to the extremely wasteful of those resources, These predictions, however, do not technologies that eventually emerged; whereas dynamic assignment (I refer to help us understand why the specific the researchers involved were, in this as “dynamic resource sharing” or events, innovations, people, and circum- chronological order, myself, , “demand access”) would be highly effi- stances that formed our Internet and . Below we explore cient. This was an essential observation, emerged when they did. Doing so is not these three paths, which were indepen- and in 1959 it launched my research possible from the scale of centuries or dently pursued in the quest to provide thread as I sought to design a new kind single individuals. This column’s focus is data networking theory, architecture, of network. Its architecture would use on the defining inventions and decisions and . The second thread dynamic resource allocation to support that separate early technologies that was the creation and growth of the the bursty nature of data communica- were clearly not the Internet, from a Advanced Research Projects Agency tions, and eventually provide a structure wide range of recent inventions that may (ARPA), the institution that funded for today’s packet-switched networks. help characterize our Internet, but were and deployed these technologies — a also built within it. Thus, in this column that, as we will see, was by no we trace both the early history of the sci- means automatic. These two threads 1 Later that year on October 4, I experienced a ence and infrastructure that emerged as merged in the mid-1960s, creating the widely shared feeling of surprise and embarrass- the ARPANET, and the trajectory of historical “break” that led to the ment when the launched Sputnik, development it set for the even broader ARPANET. Once these threads the first artificial Earth satellite. In response, construct that we now call the Internet. merged, the implementation and President Eisenhower created ARPA on Febru- As one of many individuals who par- deployment phase began, bringing in ary 7, 1958 to regain and maintain U.S. techno- ticipated in the Internet’s early history, I other key contributors and successive logical leadership.

26 0163-6804/10/$25.00 © 2010 IEEE IEEE Communications Magazine • August 2010 LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 27

HISTORY OF COMMUNICATIONS

This concept of resource sharing was As we see below, I was able to apply 1962, which turned out to be the first emerging at that time in a totally differ- Jackson’s result to represent the data paper [11] to introduce the concept of ent context: that of timesharing of com- networks of interest by making serious breaking messages into smaller fixed- puter power. Timesharing was based on modifications to his model. length pieces (subsequently named the same fundamental recognition that So the stage was set: There was a “packets,” as explained below). In it I users generate bursty demands, and thus need to understand and design general- provided a mathematically exact analysis expensive resources were wast- purpose data networks of the mean response time, and showed ed when a computer was dedicated to a that could handle bursty data traffic, the advantages to be gained by utilizing single user. To overcome this inefficien- there was an emerging approach based packet switching for this new network.3 cy, timesharing allocated the computer on resource sharing in timeshared sys- Note that the fixed length packets I to multiple users simultaneously, recog- tems, there was an existing special-pur- introduced did not match the randomly nizing that while one user was idle, oth- pose network that suggested it could be chosen lengths of the model, but fortu- ers would likely be busy. This was an done, and there was a body of queueing nately, the key performance measure I exquisite use of resource sharing. These theory that looked promising. solved for, the overall mean system ideas had roots in systems like SAGE [6] As a result, I prepared and submitted response time, did not require that and in the MIT Compatible Time-Shar- my MIT Ph.D. thesis proposal [9] in assumption, so the mathematical model ing System (CTSS [7]), developed in May 1961, entitled “ Flow in properly reflected the behavior of fixed 1961 by Fernando Corbato (among the Large Communication Nets” in which I length packets as well. first timesharing systems to be imple- developed the first analysis of data net- I also developed optimal design pro- mented). The principles and advantages works. I chose a queueing theoretic cedures for determining the network of timesharing were key to my realiza- model based on Jackson’s model to char- capacity assignment, the topology, and tion that resource sharing of communi- acterize a data network as a network of the procedure. I introduced and cation links in networks could provide communication channels whose purpose evaluated distributed adaptive routing for efficient data communications, much was to move data messages from their control procedures, noting that net- like the resource sharing of processors in origin to their destination in a hop-by- work/routing control is best handled by timeshared systems was accomplishing. hop fashion. Each channel was modeled sharing control among all the nodes In addition, there was already an as a resource serving a queue of data rather than relegating control to one or example of a special-purpose data net- messages awaiting transmission; I dis- a small number of nodes. This dis- work that used resource sharing: the cussed how “The nets under considera- tributes the control load (thereby not store-and-forward telegraph network. tion consist of nodes, connected to each unduly loading any one ), intro- The challenge I faced was to create an other by links. The nodes receive, sort, duces the ability to change routes on appropriate model of general-purpose store, and transmit messages that enter the fly dynamically (based on current data communications networks, to solve and leave via the links….” My underly- load, connectivity, and destination for their behavior, and to develop an ing model assumed that the stream of address), enables the network to scale effective design methodology for such messages had randomly chosen lengths to a very large number of nodes, and networks. and, when applied to data networks, dramatically improves the robustness of To do this, I sought to develop a yielded a problem whose exact solution the network. model with dynamic resource sharing, turned out to be hopelessly intractable. I Whereas my focus was not principal- incorporating the fact that data traffic altered the model and also introduced a ly on the engineering details of packet was unpredictable as well as bursty. In critical mathematical assumption, the networks, I did address engineering order to clear up some misconceptions Independence Assumption,2 which details when I built a complete network regarding what I and other investigators tamed the problem and allowed for an simulation model and conducted exten- were doing in the field in the early days, elegant solution. With this solution, I sive simulation experiments confirming I will devote some space in the follow- was able to solve for the many perfor- the correctness of the theory. These ing paragraphs to discuss the relation- mance measures of these networks. For experiments included detailed message ship between dynamic resource sharing example, I showed that by scaling up the blocks (with headers, origin and desti- and packet switching, where the latter is network traffic and properly, nation addresses, priority indicators, but one of many ways to realize the for- one could reduce the system response routing labels, etc), dynamic adaptive mer. The basic structure I chose was time, increase the network efficiency, routing tables, priority queueing struc- that of a queue since it is a perfect and increase the network , all tures, traffic specifications, and more.4 resource sharing mechanism. A queue simultaneously [10]. Packetization was an integral part of is dynamic, adaptive, and efficient, and In the course of examining data net- a much broader body of knowledge that does not wait for a message that is not work performance, it became clear to had to be developed to prove the case there, but rather transmits a message me that it was important to explore the for data networks. Indeed, packetization already waiting in the queue. Moreover, manner in which mean response time alone was not the underlying technology the performance measures one consid- was affected when one introduced a pri- ers in are response ority queueing discipline on the traffic. I time, throughput, efficiency, buffering, chose to understand this influence in the 3 One of the important advantages of using pack- priorities, and so on, and these are just case of a single node first and then to ets turned out to be that short messages would the quantities of interest in data net- apply the results to the general network not get “trapped” behind long messages; I was works. In the late 1950s, the published case. This led to a publication in April able to show this gain in response time exactly. literature contained almost no work on networks of queues. However, a singu- 4 Access to my simulation notes can be found at lar exception to this was the work by 2 Chapter 3 of my dissertation [16] elucidates http://ucla.worldcat.org/title/leonard-klein- James Jackson, who published a classic this problem and the role of the Independence rockscorrespondence-course-and-research-notes- paper [8] on open networks of queues. Assumption. 1961-1972//263164964&referer=brief_results

IEEE Communications Magazine • August 2010 27 LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 28

HISTORY OF COMMUNICATIONS

that led to ARPANET design funda- work was done independently of the put good ideas forward, but it was also mentals. To be sure, packetization was work that I had done earlier at MIT necessary to prove that the concepts and remains a core element of today’s and, in many ways, the results we were quantitatively sound, and then to networking technology, but it is not iden- achieved in addressing the problem of implement and deploy an operational tical to network efficiency. Rather, the packet networks were complementary. network that would bring these ideas fundamental gain lies in dynamic The third early contributor to packet and designs to use. resource sharing. It is important to point switching was Donald Davies, of the out that there are many ways in which National Physical Laboratory (NPL) in THE ARPA THREAD dynamic resource sharing can be accom- the . He began think- Let us step back chronologically and plished, with packet switching being only ing about packet networks in 1965 and now pursue the second thread: the role one such method; other methods include coined the term “packet” that year. In a of ARPA in defining the need for a polling [12], [13], privately circulated paper [21] dated data network, putting the management asynchronous time-division multiple June 1966, he described his design for a structure in place to enable its develop- access (ATDMA) [14], carrier sense data network and used my earlier theo- ment, and providing the funding neces- multiple access with collision detection ry to calculate its performance. Davies sary for its implementation and (CSMA/CD) [15], and others. lectured to a public audience in March deployment. I completed and filed my Ph.D. dis- 1967, recommending the use of his tech- J. C. R. Licklider (“Lick”) entered sertation [16] in December 1962, having nology for the design of a public the story when he published his land- created a mathematical theory of pack- switched data network, and published mark 1960 paper [26] “Man-Computer et switching for dynamic resource shar- an October 1967 paper [22] with his Symbiosis.” He defined the title as “an ing, thus providing the fundamental NPL group in which details of the expected development in underpinnings for ARPANET technol- design were first described in an open interaction between men and electronic ogy. I showed that these networks were publication. This plan was for an NPL computers.” This work envisaged a sys- efficient, stable, scalable, robust, adap- Data Communications Network, but the tem “to enable men and computers to tive, and, most of all, feasible. Decades U.K. Department of Trade and Indus- cooperate in making decisions and con- of important research on these topics try only authorized the implementation trolling complex situations without have since taken place around the of one node. That node became - inflexible dependence on predeter- world. tional in 1970. Further details of a full mined programs”; he had seen such a By the time my dissertation was pub- network design were described by the flexible system in the aforementioned lished as the first book [17] on comput- NPL team in 1968 [23, 24] and 1969 SAGE system. Once again, we find a er networks in 1964, the idea of [25]; it is not clear where a multiple- forecast of what future telecommunica- packetization itself was appearing more node deployment by this team might tions might provide — and Lick was broadly. The next contributor to packet have led, but it obviously had potential. perhaps the first to write at a time when switching was Paul Baran of the RAND This reluctance to support an NPL viable ways to create that future were Corporation, who was busy working on packet-switched network was reminis- emerging. Although a visionary, Lick military command and control systems cent of the view taken by AT&T and was not a networking technologist, so during the early 1960s with the goal of DCA in not supporting an implementa- the challenge was to finally implement using redundancy and digital technolo- tion of the RAND work. such ideas. gy to design a robust multilateral mili- The work of Baran and Davies In May 1962 Lick and Welden Clark tary communications network. He focused on the engineering and archi- outlined their views on how networking recognized the vulnerability of the tele- tectural issues of the network design. computers could support social interac- phone network due to its centralized My work emphasized and provided the tion, and provide networked access to architecture. In September 1962 he mathematical underpinnings and sup- programs and data [27]. This extended published a paper [18] on how “hot porting simulation experiments of the his earlier ideas of what he now potato” adaptive alternate routing pro- network analysis and design, including referred to as a Galactic Network (in cedures and distributed principles could optimization as well as formulating the fact, he nicknamed his group of com- utilize a “standard message block,” also basic principles of packet networks that puter experts “The Intergalactic Net- to fall under the “packet” umbrella, include dynamic resource sharing; this work”). which will be addressed below. His pur- quantitatively showed that these net- Lick was appointed as the first direc- pose was to create a network capable of works were feasible. My trajectory was of ARPA’s newly formed Informa- functioning after a Soviet nuclear attack more fortunate as the ARPA thread tion Processing Techniques Office [19]. In August 1964 he produced a set rolled out and adopted my principles (IPTO) in October 1962. He quickly of 11 important reports [20] reinforcing for their design of the ARPANET, and funded new research into advanced his prior description with simulations provided me the opportunity to partici- computer and networking technologies and elaborating on many details of the pate in its implementation and deploy- as well as areas that involved man-com- design. He, too, discovered the impor- ment. Different trajectories were taken puter interaction and distributed sys- tance of going to digital networks and by Baran and then later by Davies, with tems. of the robustness provided by distribut- Baran’s unsuccessful attempts to get his By the end of 1962, Lick had articu- ed routing. He attempted to get AT&T ideas implemented and with Davies’ lated his grand vision for the Galactic to implement the design, but failed to frustration by the foot-dragging of the Network, of which I was unaware, and I convince them (presumably due to their U.K. government. It was not enough to had laid out the mathematical theory of analog mindset). In 1965 RAND packet networks, of which Lick was also approached the Air Force to implement unaware. These ideas would soon inter- it, but they deferred to the DCA5; at 5 Defense Communications Agency, which was sect and reinforce each other in a series this point, Baran decided not to pursue renamed in 1991 to be today’s (2010) Defense of key events between 1962 and 1969. I the implementation any further. Baran’s Information Systems Agency — DISA. joined the UCLA faculty in 1963. Lick

28 IEEE Communications Magazine • August 2010 LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 29

HISTORY OF COMMUNICATIONS

passed the directorship of IPTO to Ivan ceeded Sutherland as IPTO director in University of , where Sutherland, an MIT colleague of mine, in August 1966, brought in Roberts as the ARPANET planning was discussed in September 1964. In that role Sutherland IPTO chief scientist that December. detail. It was there that the basic specifi- wished to connect UCLA’s three IBM Bringing Roberts in to manage the net- cations for the underlying network were mainframes in a three-node on-campus working project turned out to be a criti- debated among us PIs. For example, , which would have cal hire as Roberts was to contribute at Wesley Clark put forward the concept been easy to accomplish with the means all levels to the coming success of data of using an unmanned at I had laid out in my Ph.D. dissertation. networking. each location to handle all of the switch- However, the UCLA network was never The research and ARPA threads ing and communications functions; it realized due to administrative discord. had now merged, and the project would was to be called an Interface Message Nevertheless, the seeds for an ARPA- soon become the ARPANET. Processor (IMP). This would offload the funded network had now been sown. These were critical steps in Internet networking functions from the , Early the next year (1965), Suther- history, for not even in the post-war greatly simplify the design by requiring land awarded Larry Roberts (another did technological progress only one interface to be written for each MIT colleague of mine who was quite flow directly from ideas. In contrast to host to the standard IMP, and at the familiar with my networking research) a refusals from the private sector to fund same time would decouple the network contract to create a dialup 1200 b/s data the beginnings of the project, ARPA design from any specific host hardware connection across the United States. made available the will and funding of and . Another specification had Later that year, Roberts accomplished the U.S. government.7 ARPA’s man- to do with the measure of reliability of this in collaboration with Thomas Mar- agement and support fostered the early the planned network; this we specified ill, demonstrating that such a connec- culture of shared, open research that by requiring that the topological design9 tion required a different, more was crucial to the success of the produce a “two-connected net,” thus sophisticated network than the tele- ARPANET program. guaranteeing that no single failure phone network offered [28]. would cause any non-failed portion of Meanwhile, at ARPA, Sutherland the network to lose connectivity. recruited Robert Taylor to become THE BEGINNING: THE Yet another requirement we intro- associate director of IPTO in 1965. AUNCH duced was for the network to provide While there, Taylor also recognized the ARPANET L an experience as if one were connected need for a network, this time specifical- The commitment to create the to a local timeshared computer even if ly to connect ARPA research investiga- ARPANET was now in play. Roberts that computer was sitting thousands of tors to the few large expensive research was empowered to develop the network miles across the network; for this we computers across the country. This concept based on Lick’s vision, my the- specified that short messages should would allow them to each other’s ory, and Taylor’s application. have response times no greater than hardware, software, and applications in There were basically two matters to 500 ms (the network design provided a cost-effective fashion.6 Taylor then be considered in this project. One was 200 ms at its inception). Moreover, dropped into the office of the ARPA the issue of creating the switches and since this was to start out as an experi- director, Charlie Herzfeld, to request links underlying the network infra- mental network, I insisted that appro- funding for this nascent networking structure, with the proper performance priate measurement tools be included project. Herzfeld was a man of action characteristics, including throughput, in the IMP software to allow for tracing who knew how to make a fast decision, response time, buffering, loss, efficiency, of packets as they passed across the and within 20 minutes he allocated $1 , topology, channel capacity, network, taking of snapshots of the million to Taylor as initial funding for routing procedure, queueing discipline, IMP and host status at any time, artifi- the project. Taylor, who had since suc- reliability, robustness, and cost. The cial traffic generation, gathering and other was to create the appropriate pro- forwarding of about the net- tocols to be used by the attached (host) work, and a mechanism for controlling 6 This sharing of resources was the primary computers8 so that they could properly these measurements. motivation for creating the ARPANET. Paul communicate with each other. Following this April meeting, Baran developed a network design (described Shortly after his arrival, Roberts Roberts put together his outstanding above) that would maintain communications called a meeting of the ARPA Principal plan for the ARPANET design and pre- — and specifically, Second Strike Capability — Investigators (PIs) in April 1967 at the sented it as a paper [29] at a conference in the event of a nuclear attack by the USSR. in Gatlinburg, Tennessee in October His and my work served different aims. When 1967. At this conference, Roger Scant- ARPA began work on the ARPANET, my work 7 In sharp contrast to ARPA’s enthusiasm for lebury of the NPL also presented their was used for the reasons described herein. His networking, in the early 1960s, when I intro- aforementioned jointly published paper application to military communications gave duced the ideas of packet-switched networks to [22] describing a local network they rise to the that the ARPANET was creat- what was then the world’s largest networking were developing. It was during a con- ed to protect the United States in case of a company, AT&T, I met with narrow-minded versation with Scantlebury at this meet- nuclear attack. This is not to take away from and failed thinking, and was summarily dis- ing that Roberts first learned of the Baran’s accomplishments; indeed, by the time missed by them. They commented that packet NPL work as well as some details of the the ARPANET began in 1969, he had moved switching would not work, and even if it did, work by Baran at RAND. The research on to different projects, including the Institute they wanted nothing to do with it. Baran had a by myself at MIT, by Baran at RAND, for the Future (he stepped back into the ARPA similar reaction from AT&T. foray in 1974–1975 to recommend that an early commercial version of the ARPANET be insti- 8 A major challenge for such a network was that 9 To assist with the topological design, Network tuted beside the original research-driven net- it would connect computers with incompatible Analysis Corporation (NAC), whose CEO was work). hardware and software. Howard Frank, was brought in as a contractor.

IEEE Communications Magazine • August 2010 29 LYT-HISTORY-August-Kleinrock 7/21/10 2:40 PM Page 30

HISTORY OF COMMUNICATIONS

ARPA (example)

U. Lincoln Lab

Harvard U. Proj. MAC SRI UCLA

UC Santa BBN Barbara U. UC Berkeley Pentagon B.T.L. Dartmouth SDC College

Carnegie Stanford U. RAND Washington Mellon U. Univ.

U. Michigan

Figure 1. 19-node ARPANET as shown in the original RFQ.

and by Davies, Scantlebury, et al. at for Quotation (RFQ) [32] for the net- The RFQ resulted in 12 proposals NPL had all proceeded independently, work IMPs was mailed to 140 potential being submitted in August 1968 (notably mostly without the researchers knowing bidders. The 19-node example to be missing were IBM and AT&T). As these about the others’ work. There was, delivered by the contractor is shown in proposals were being evaluated at though, some cross-fertilization: Davies Fig. 1. ARPA, Roberts awarded a research con- had used my analytical model for data The handling of data streams speci- tract to me at UCLA in October to cre- networks in his work; as a result of dis- fied that the would communicate ate the Network Measurement Center cussions at this conference, Roberts with other hosts by sending messages (NMC). The task of the NMC was to adopted Davies’ word “packet” for the (of maximum length 8192 ) to their measure the behavior of the ARPANET small fixed length pieces I had suggest- attached IMPs, that these messages by conducting experiments to determine ed we break messages into, and which would be broken into packets (of maxi- its faults, performance, and outer limits Baran referred to as “message blocks”; mum length 1024 bits each — thus, at (through the use of stress tests). I was its fixed length was chosen to be 1024 most 8 packets per message) by the fortunate to have a star team12 of gradu- bits for the ARPANET design (both IMP, and that IMPs would communi- ate student researchers, developers, and Baran and Davies had suggested this cate with each other using these pack- staff for this project; a number of these same length); as a result of the discus- ets. The movement of packets through appear in continued roles later in this sion with Scantlebury, Roberts decided the subnetwork of IMPs was to be con- story. A week before Christmas 1968, [30] to upgrade the backbone line speed trolled by a distributed dynamically Bolt, Beranek and Newman (BBN) won from 9.6 kb/s to 50 kb/s for the updated routing based on net- the competitive bid and was awarded the ARPANET design. work connectivity and loading as well as contract to develop the IMP-to-IMP Following these 1967 meetings, a packet destination and priority. Errors subnetwork. The BBN team,13 super- sequence of drafts for the IMP specifi- in packet transmission between IMPs vised by , produced some cation was prepared.10 This culminated were managed by error detection and remarkable accomplishments. This team in March 1968 when Roberts and Barry retransmission. Packets were to be had selected the DDP-516 Wessler produced the final version of reassembled into their original messages minicomputer with 12 kb of memory for the IMP specification, which they then at the destination IMP before delivery the program to be the machine on which discussed at an ARPA PI meeting later to the destination host. The basic struc- that month. On June 3, 1968, the ture of this IMP specification contained ARPANET Program Plan [31] was for- contributions from a number of individ- 11 Roberts also goes on to say that my disserta- mally submitted to ARPA by Roberts, uals, including my own research. tion was “critical to my standing up to them and and it was approved on June 21, 1968. Roberts had been well aware of my betting it would work.” The ARPANET procurement process work since my time at MIT, where we was now officially underway. were officemates, later stating,11 “In 12 Key members of my UCLA team included a By the end of July 1968, a Request order to plan to spend millions of dol- research team (Jerry Cole, Al Dobieski, Gary lars and stake my reputation, I needed Fultz, Mario Gerla, Carl Hsu, Jack Zeigler), a to understand that it would work. With- software team (, , Ger- 10 Among those involved in these first drafts out Kleinrock’s work of Networks and ard DeLoche, Charley Kline, Bill Naylor, Jon were Frank Westervelt, Elmer Shapiro, Glen Queueing Theory, I could never have Postel), a hardware engineer (Mike Wingfield), Culler, and myself. taken such a radical step.” [33] and others.

30 IEEE Communications Magazine • August 2010 LYT-HISTORY-August-Kleinrock 7/21/10 12:29 PM Page 31

HISTORY OF COMMUNICATIONS

deploy a working packet-switched net- SRI work. Recall that the other task was to create the appropriate protocols to be XDS used by the attached (host) computers 940 so that they could properly communi- cate with each other. IMP This second task was assigned to the four chosen ARPANET research sites to figure out on their own. Thus began another thread of innovative develop- I M DEC ment that characterized the ARPANET P PDP-10 culture. This thread actually begins in the summer of 1968 when Elmer UTAH Shapiro of SRI, in response to a request I by ARPA, called a meeting of program- IBM mers from among those first sites that 360/75 M P were to be connected into the ARPANET. Their main charge was to UCSB IMP study and resolve the issues of host-to- host communication. Present at this meeting was one programmer from each of the first four sites to receive XDS X-7 IMPs as follows: Steve Crocker (UCLA), (SRI), Ron UCLA Stoughton (UCSB), and Steve Carr (). This group, plus Figure 2. The initial four-node ARPANET (1969). the many others who joined later, were soon to be named the Network Work- ing Group (NWG) with Shapiro its first the IMP would be based; they were con- UCLA by early September 1969. Their chairman.14 UCLA’s served tracted to implement the IMP functions performance was outstanding. The first as the (RFC) by modifying the hardware and software IMP at UCLA was to be followed by the editor (a role he held until his untimely of the DDP-516, to connect these IMPs second IMP in October to SRI, the third death in 1998). They had no official to long-haul 50 kb/s lines leased by IMP in November to the University of charter against which to work, and so Roberts from AT&T under the DoD California at Santa Barbara (UCSB), were afforded the unique opportunity Telpak tariff, and to deploy the subnet- and the fourth IMP in December to the to invent and create as needed. There work. The BBN team developed an ele- University of Utah. The initial network was no sense of qualifying membership; gant host-IMP design that met the was to be that shown in Fig. 2. all one had to do was to contribute and ARPA specifications; this specification These four sites were selected due to participate. Their focus moved to the was written as BBN Report 1822 [34] by their ability to provide specialized net- creation of high level interactions and, Robert Kahn, who was in charge of the work services and/or support. Specifical- eventually, to the notion of a layered system design at BBN (Kahn appears ly, UCLA (connecting an SDS Sigma-7 set of protocols (transport services later in this story in some very significant Host computer) would provide the below a set of application-specific pro- roles, as we shall see below). One of the NMC (under my supervision), SRI (con- tocols). Basically, this was a highly BBN team, Dave Walden, points out necting an SDS 940 host computer) resourceful, self-formed, collegial, that he was most likely the first pro- would provide Doug Englebart’s Human loosely configured group of maverick grammer on the Internet by virtue of Intellect Augmentation System (with an graduate students who we (the ARPA having done code design for the IMP in early version of in his NLS PIs) had empowered to design and their 1968 response to the RFQ. Where- system) as well as serve as the Network implement the protocols and software as members of the BBN team were busy Information Center (under Elizabeth for the emerging network. They took on testing the IMP’s ability to provide IMP- [Jake] Feinler’s supervision), UCSB the challenge we ceded to them and to-IMP data exchanges, testing the (connecting an IBM 360/75 host com- behavior of a network of IMPs was diffi- puter) would provide interactive graph- cult to do in a laboratory environment; ics (under Glen Culler’s and Burton 14 The names of some of the other key individu- the true behavior was more properly Fried’s supervision), and the University als who participated early on in the NWG tested in the deployed network with real of Utah (connecting a DEC PDP-10 include , Vint Cerf, Danny Cohen, traffic and with many nodes, which is host computer) would provide advanced Bill Duvall, Michel Elie, Jack Feinler, Jon Pos- exactly what the NMC was designed to 3D graphics (under the supervision of tel, and Joyce Reynolds. do. Basically, BBN was given less than ). The fact that Heart nine months to deliver the first IMP to and his team at BBN succeeded in deliv- 15 It is remarkable how effective the RFCs, the ering this new technology with new NWG and the IETF have served the network applications and new users in an on- community. In spite of the fact that they are 13 Key members of Heart’s team included Ben time, on-budget fashion was incredible. loosely structured and involve large numbers of Barker, Bernie Cosell, Will Crowther, Robert But this contract to develop the outspoken professionals, they have been able to Kahn, , Truett Thach, Dave underlying network was only the first of move forward on a number of critical Internet Walden, and others. the two key tasks that were needed to issues.

IEEE Communications Magazine • August 2010 31 LYT-HISTORY-August-Kleinrock 7/21/10 12:29 PM Page 32

HISTORY OF COMMUNICATIONS

Figure 3. The entry in the original IMP log, which is the only record of the first message transmission on the Internet.

created an enduring NWG structure hosts. Their argument was that their node of the fledgling network, as bits that later led to today’s Internet Engi- hosts were already fully utilized serving moved between the IMP and the Sigma- neering Task Force (IETF).15 their local customer base. Eventually 7. This is often regarded as a very sig- Once the IMP-host specification was they relented and provided their expect- nificant moment in the Internet’s released by BBN in the spring of 1969, ed traffic loads. That traffic was history. the NWG began to focus on the lower- used in the July 1968 RFQ [32] and in a In early October the second IMP level issues such as message formats. paper I published [36], thereby sealing was delivered by BBN to SRI in Menlo They decided to exchange ideas through their commitment. Park, California. The first high-speed a very informal set of notes they On July 3, 1969, two months before link of what was to become the Internet referred to as “Requests for Comments” the IMP was due to arrive, UCLA put was connected between those two IMPs (RFC). The first RFC [35], entitled out a press release [37] announcing the at the “blazing” speed of 50 kb/s. Later “Host Protocol,” was written by Crocker imminent deployment of the in October, SRI connected their SDS in . Crocker became the sec- ARPANET. In that release I described 940 host computer to their IMP. ond Chairman of the NWG early on. what the network would look like, and The ARPANET’s first host-to-host We now had the two main what would be a typical application. I message was sent at 10:30 p.m. on Octo- ARPANET development efforts under- am quoted in the final paragraph as say- ber 29, 1969 when one of my program- way: ing, “As of now, computer networks are mers, Charley Kline, and I proceeded • A formal contract with BBN to cre- still in their infancy, but as they grow up to “login” to the SRI host from the ate the IMP-IMP subnetwork and become more sophisticated, we will UCLA host. The procedure was for us • An informal group of programmers probably see the spread of ‘computer to type in “log,” and the system at SRI (mostly graduate students) who utilities,’ which, like present electric and was set up to be clever enough to fill were charged with developing the utilities, will service individual out the rest of the command, adding Host-to-Host Protocol homes and offices across the country.” “in,” thus creating the word “login.” Things began to move rapidly at this It is gratifying to see that the “computer Charley at our end and Bill Duvall at point. The date of the first IMP deliv- utilities” comment anticipated the emer- the SRI end each had a telephone head- ery, scheduled to arrive to us at UCLA gence of web-based IP services, that the set so they could communicate by voice in early September 1969, was fast “electric and telephone utilities” com- as the message was being transmitted. approaching. Meanwhile, at the NMC, ment anticipated the ability to plug in At the UCLA end, we typed in the “l” we were busy collecting data so that we anywhere to an always on and “invisi- and asked SRI “did you get the l?”; could predict performance of the net- ble” network, and that the “individual “got the l” came the voice reply. We work based on my earlier theory. For homes and offices” comment anticipat- typed in the “o,” “did you get the o?,” this, it was necessary to estimate the ed ubiquitous access. However, I did not and received “got the o.” UCLA then traffic loads that the host sites would foresee the powerful social networking typed in the “g,” asked “did you get the present to the network. Roberts and I side of the Internet and its rapidly grow- g?,” at which point the system crashed! contacted a number of the early sites ing impact on our society. This was quite a beginning. So the very and asked them how much traffic they On Saturday, August 30, 1969, the first message on the Internet was the expected to generate and to which other first IMP arrived at UCLA. On Septem- prescient word “lo” (as in, “lo and sites. We also asked them how much ber 2, the day after Labor Day, it was behold!”). This, too, is regarded as a traffic they would allow into their sites; connected via a 15-foot cable to the very significant moment in the Inter- to my surprise, many refused to allow UCLA host computer, our SDS Sigma- net’s history. any traffic from the network to use their 7 machine. This established the first The only record of this event is an

32 IEEE Communications Magazine • August 2010 LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 33

HISTORY OF COMMUNICATIONS

entry in our IMP log recording it as (Two years later, in May 1972, another phone company continued to ignore it, shown in Figure 3. Here we see that on key session at the same conference was but the open network that was the October 29, 1969, at 10:30 pm, we at devoted to the presentation of five ARPANET thrived. UCLA “Talked to SRI Host to Host.” papers [41] that updated the Soon, additional networks were In November and December the ARPANET of the art; this, too, added to the ARPANET, the earliest of IMPs and hosts at UCSB and the Uni- was packaged into a second special which were those whose origins came versity of Utah were connected, respec- ARPA pamphlet.) In mid-1970 the first out of work on networking. tively, thus completing the initial cross-country link was added with a Connecting the ARPANET with these four-node network. Further IMP deliv- connection from UCLA to BBN, and different networks proved to be a feasi- eries were halted until we had an by July the network contained 10 IMPs. ble but not seamless opportunity to test this four-node net- The net grew to 15 IMPs by March issue, and it received a great deal of work, and test it we did. Among other 1971. In September 1971 BBN intro- attention. The of net- things, we were able to confirm with duced a terminal interface processor works was referred to as “internetwork- measurements some of our theoretical (TIP) that conveniently would allow a ing” during the 1970s, a neologism from models of network delay and through- terminal to connect directly to the which the expanded ARPANET was put as presented by Gerry Cole [38]. ARPANET without the need to con- eventually renamed as the Internet. The ARPANET had now been nect through an attached host. Later in Let us briefly trace the work on wire- launched. We now turn to the story of the year, BBN slipped in a “minor” fea- less networking that led to these addi- its rollout through its first decade. ture called electronic mail. Electronic tional networks, which themselves mail had existed since the mid-1960s for forced attention on improving interop- standalone timeshared computer sys- erability solutions. As pointed out THE FIRST DECADE: tems, but in late 1971 at BBN, Ray above, these networks were based on OUR ODES AND Tomlinson added a small patch to it wireless multi-access communications in F N that allowed the mail to pass between which a shared channel is accessed by THEN THE WORLD different computers attached to the many users. By late 1970, Norm Abram- ARPANET using an experimental file- son had developed AlohaNet [43] in By the time the first four nodes were sharing network program called , a 9600 b/s packet net deployed in , Roberts CPYNET. Once he saw that it worked, based on the novel “unslotted (pure) (who had succeeded Taylor in September he sent an message to his group ALOHA” multi-access technique of to become the IPTO director) once again at BBN announcing this new capability, random access. In this scheme (unsyn- met with the NWG and urged them to and so “The first use of network email chronized) terminals transmit their extend their reach beyond what they had announced its own existence.” [42]. fixed length packets at any time over a articulated in their first RFC [35], “Host This capability went out as a general shared channel at random times; if Protocol.” This led them to develop a TENEX release in early 1972. By July more than one transmission overlaps symmetric Host-to-Host Protocol, the 1972, Roberts added a management (i.e., collides), then destructive interfer- first implementation of which was called utility to network email that allowed ence prevents any of the involved pack- the (NCP) listing, selective reading, filing, forward- ets from succeeding. This tolerance of and was described by Crocker in RFC 36 ing, and replying to email messages. In collisions was a departure from the in March 1970 [39]. This less than a year email accounted for the more standard methods of wireline was to reside in the host machines them- majority of the network traffic. The net- communications to control multi-access selves and included a hierarchy of lay- work’s ability to extend communication systems that used demand access meth- ered protocols to implement more between people was becoming evident, ods (queueing, polling, etc., as men- complex protocols. As NCP began a nascent image of Lick’s vision. tioned earlier) and allowed only one deployment, the network users could Later that year, in October 1972, the transmission at a time (thus precluding begin to develop applications. The NCP first public demonstration of the such collisions). In 1973 Abramson cal- was the first protocol stack to run on the ARPANET technology took place at culated the capacity of the unslotted ARPANET, later to be succeeded by the International Conference on Com- ALOHA system [44], which had a maxi- TCP/IP. The trajectory of protocol stack puter Communications (ICCC) in mum efficiency of 18 percent, and in development touched on below is anoth- Washington, DC. Kahn, who by now 1972 Roberts calculated the capacity of er example of multiple possible paths had been hired into ARPA by Roberts, a synchronized version (i.e., slotted that led the way from the ARPANET as organized this large and very successful ALOHA) [45] whose capacity was dou- it evolved into the Internet. demonstration in which dozens of ter- bled to 37 percent. However, these After the short evaluation period minals in Washington accessed dozens analyses ignored an essential issue with following the initial four-node deploy- of host computers throughout the Unit- random access to shared channels: that ment, a continual succession of IMPs ed States in a continuously reliable they are fundamentally unstable, and and networks were then added to the fashion for the three-day duration of some form of dynamic control was ARPANET. In May 1970, at the AFIPS the conference. needed to stabilize them, for example, a Spring Joint Computer Conference, a The reaction of the computer manu- backoff algorithm to control the way in landmark session was devoted to the facturers to this ARPANET phe- which collided transmissions are presentation of five papers [40] regard- nomenon was to create proprietary retransmitted. This stability issue was ing the newly emerging ARPANET network architectures based on their first identified and addressed by Lam technology; these papers were packaged own brand of computers.16 The tele- and myself [46, 47]. into a special ARPA pamphlet that was It is interesting to note that the widely circulated in the community and ALOHA systems studies eventually led spread information of the then-current 16 Among the proprietary networks were IBM’s to an investigation of carrier sense mul- technology that had been deployed. SNA and DEC’s DECnet. tiple access (CSMA) as another wire-

IEEE Communications Magazine • August 2010 33 LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 34

HISTORY OF COMMUNICATIONS

less access method. CSMA itself led growth. NCP was now handling the net- of the Internet. In 1973–1974 DARPA to consider a variation work-to-network interconnection of commissioned three independent imple- called CSMA with collision detection AlohaNet and the U.K. university net- mentations of TCP: Cerf at Stanford (CSMA/CD), which was the basis for work, both of which were attached to University, Tomlinson at BBN, and the original development. the ARPANET. The problems resulting Kirstein at University College London. Based on these concepts, Metcalfe and from interconnected heterogeneous In addition, David Clark of MIT implemented CSMA/CD networks were becoming clear, and worked on a compact version of TCP on a network, which was included the network-to-network proto- for the personal worksta- up and running by November 1973. In col conversion needed between any tion in the mid-1970s and later for the sum, they created the Ethernet, which (and every) pair of networks that were IBM PC desktop computer; David is today perhaps the world’s most per- interconnected. It was clear that the Reed, also of MIT, was working on vasive networking technology [48]. Eth- combinatorial complexity of this pair- among high-perfor- is crucial to the story of NCP and wise protocol conversion would present mance computers on LANs for the Lab- TCP/IP, for researchers at Xerox PARC considerable problems as the number of oratory for Network built on this technology in efforts to attached networks scaled up. TCP/IP (whose work was merged with the gen- address the challenges of internetwork- was soon to emerge as the response eral TCP project in 1976). In August ing. Implemented in 1974 and published chosen to address these problems. 1976 these led to the in 1975, the PARC Universal Packet At DARPA17 in early 1973, Kahn first experimentation using TCP to con- (PUP) remained an internetwork archi- was the program manager responsible nect two different networks: the packet tecture as late as 1979 [49]. PUP was for, among other things, the ground using Stanford’s TCP one potential means through which to network and the satellite implementation, and the ARPANET improve on NCP, although as we see packet radio network. He recognized using BBN’s TCP implementation. Fol- below, that role was later taken on by the differences between the ARPANET lowing that, in 1977, Kahn implemented TCP/IP. This is one of many stories running NCP, and these two radio net- the satellite reservation protocol that call out for more research into the works. As a result, he set out to design Roberts had designed, creating a sec- histories and the individuals involved. a scalable end-to-end protocol that ond path from the ARPANET to the Let us now return to the story of the would allow dissimilar networks to com- United Kingdom, sharing the capacity above-mentioned wireless technologies municate more easily. In the summer of of a 64 kb/s IV satellite broad- to help explain the motivation that led 1973 Kahn discussed his approach for cast channel among a number of ground to TCP/IP (as different from that which dealing with this internetwork complex- stations in Europe and the East Coast motivated PUP). These technologies ity with Vint Cerf of Stanford who had of the United States. This Atlantic led to wireless networks that attached considerable knowledge of NCP, since Packet Satellite Net (later to be called to the ARPANET, thereby exposing the he had been a key member of the SATNET) was the ARPANET’s second nature of the problems of supporting UCLA software group involved in the international connection. This was the connectivity among heterogeneous net- NCP design. Together, they drafted a second two-network TCP demonstra- works. detailed design of a new protocol, the tion. Then a three-network demonstra- The first step was taken in Decem- Transmission Control Program (TCP). tion of TCP was conducted on ber 1972, when an IMP in California TCP was to take over the NCP’s func- November 22, 1977, when the packet used a satellite channel to connect to tions, but handle them in a more uni- radio network, SATNET, and AlohaNet through an ALOHA host in form manner: it would allow ARPANET were interconnected to Hawaii. Thus, the ARPANET, running applications to run over an internet- allow an Internet transmission to take the existing host-to-host Network Con- work while hiding the differences place between a mobile packet radio trol Protocol, NCP, was now connected between network protocols by using a van at SRI and a USC/ISI host comput- to a ground radio packet network, the uniform internetwork protocol. They er (both in California) via an interconti- AlohaNet. This was the first new net- distributed this design at a computer nental connection through University work to connect to the ARPANET. communications conference held at College London. This impressive feat AlohaNet had its own protocol and was Sussex University in September 1973. was the first three-network TCP-based working independent of ARPANET, (In October 1973 Roberts left IPTO to interconnection. yet a gateway provided internetwork become CEO of , the first This first version of TCP only sup- connectivity between the two. In 1972 commercial packet switching network ported virtual circuits at the transport Roberts extended the ARPANET to carrier.) By 1974, Cerf and Kahn level (which is fine for applications that over a that ARPA fleshed out their design and published a require reliable transmission). But it had already installed to receive seismic definitive paper [50] on TCP. Underly- failed to support, among other things, data and then extended it to London in ing TCP was the key idea of an open real-time traffic such as packet voice the United Kingdom. This was the that allowed pack- where many aspects of the session flow ARPANET’s first international connec- et networks of different types to inter- were more properly handled by the tion. In London Peter Kirstein then connect with each other and for application as opposed to the network. built a gateway to connect the computers to exchange information That is, real-time traffic called for sup- ARPANET to a network built with end-to-end across these interconnected port of an “unreliable” transport another protocol between the U.K. uni- networks. mechanism that would cope with missed versities. This was another case of dif- This contribution by Cerf and Kahn packets, packets with errors, out-of- ferent networks “internetworking,” and was a critical step in the development order packets, delayed packets, and so as this function became an increasingly on. The use of unreliable transport sup- important focal point of ARPANET port was already in use with NCP, prior development, the network came to be 17 ARPA was renamed DARPA in March 1972 to TCP; specifically, the early known as the Internet to reflect this when the word “Defense” was prepended. ARPANET IMP protocol allowed for

34 IEEE Communications Magazine • August 2010 LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 35

HISTORY OF COMMUNICATIONS

unreliable transport by use of what was ) and was referred of the first measurements we made was called type 3 packets (also known as to as TCP/IP. This version has come to to determine the throughput from “raw” messages), which were intro- be known as IPv4. Along with the split UCLA to UCSB in the initial four-node duced by Kahn in the BBN 1822 report. into TCP and IP, the capability to sup- network shown in Figure 2; note that However, BBN was concerned that the port unreliable transport (i.e., type 3 there are two paths between these two uncontrolled use of these packets would packet functionality) was included. The nodes. Whereas only one path was degrade the , so formal name for this unreliable trans- tagged as active in the routing tables at they regulated the use of type 3 packets port support was the User any one time, we found that both paths to be on a limited, scheduled basis. In Protocol (UDP) [53]. were carrying traffic at the same time 1973–1974 Danny Cohen of USC/ISI In 1980 the U.S. Department of since queued traffic continued to feed implemented a Defense (DoD) declared [54] the one of the paths when the other path (NVP) [51] under ARPA support and TCP/IP suite to be the standard for was tagged. Among the more spectacu- requested BBN to allow him to use type DoD. In January 1983, TCP/IP became lar phenomena we uncovered were a 3 packets; with Kahn’s influence, BBN the official standard [55] for the series of lockups, degradations, and allowed this. Cohen’s real-time network ARPANET; after a short grace period traps in the early ARPANET technolo- voice experiments required the ability of a few months, no network was gy, most of which were unintentional to cope with unreliable data transport. allowed to participate in the Internet if and produced unpredicted side effects. The early Version 1 design of TCP in it did not comply with IPv4. Of course, These measurements and experiments 1974 did not support it, nor did Version Internet protocols never stop develop- were invaluable in identifying and cor- 2 when it was implemented around ing, and the 1998 upgrade to Version 6 recting design issues for the early 1977. dramatically extends the ARPANET, and in developing a philos- It was around this time that pressure and introduces some significant security ophy about flow control that continues for supporting unreliable transport in enhancements. It is still in the process to inform us today. Moreover, it provid- TCP came from Cohen, now joined by of being deployed worldwide. ed us, as researchers, a wealth of infor- and Reed, and with involve- Meanwhile, as the 1970s rolled out, mation for improving our theoretical ment from Crocker and Bob Braden. in addition to the ARPANET and models and analysis for more general That is, they advocated modifying TCP TELENET, other packet networks were networks. In July 1975 responsibility for such that type 3 packet functionality being designed across the globe in this the ARPANET was given to DCA. This would be supported alongside reliable period. Peter Kirstein, in his earlier terminated the systematic measure- data transport. Cohen convinced Jon paper [56] in this IEEE Communica- ment, modeling, and stress testing that Postel of this, and Postel added a fur- tions Magazine History of Communica- the UCLA NMC had performed for ther concern, addressing layer viola- tions series, addresses much of the almost six years, and was never again tions, stating “We are screwing up in international work, especially the U.K. restored for the Internet.18 our design of internet protocols by vio- story (to which we refer the reader for It is outside the scope of this column lating the principle of layering. Specifi- more details). As a result of these to address Internet histories beyond cally we are trying to use TCP to do two national and international activities, an those of its early period as the things: serve as a host level end-to-end effort, spearheaded by Roberts, was put ARPANET. Likewise, I have not done protocol, and to serve as an Internet forth that resulted in the International justice to the untold stories that packaging and routing protocol. These Consultative Committee on Telephone abound, but I hope to have convinced two things should be provided in a lay- and Telegraph (CCITT) Recommenda- the reader that many people contribut- ered and modular way. I suggest that a tion X.25. This agreed-upon protocol ed to its success. This early history of new distinct internetwork protocol is was based on virtual circuits — which the Internet, the first decade of design needed, and that TCP be used strictly as was to be the CCITT’s own equivalent and deployment of the ARPANET, laid a host level end to-end-protocol.” [52] of TCP — and was adopted in 1976 foundations on which today’s networks Postel then went on to describe how to [57]. During this period, the Network depend and continue to develop. break TCP into “two components: the Measurement Center (NMC) at UCLA hop-by-hop relaying of a message, and was deeply involved in measuring, test- the end-to-end control of the conversa- ing, stressing, and studying the 18 The work of the NMC required a strong tion.” A robust internetworking solution ARPANET. Bill Naylor and I published degree of cooperation from BBN since it was was no easy task, and today’s TCP/IP a summary of the tools used by the they who controlled any changes to the network was built with much experimentation on NMC as well as details of a weeklong code and architecture. At the NMC, each time the ground laid by NCP. measurement and evaluation of the we discovered a lockup, hardware problem, or Thus, there was a clear call to results in 1974 [58]. In 1976 I published other measured network problem, we alerted TCP, splitting the function of network- the first book that described the BBN so that they would take corrective action. layer connectivity, which involved ARPANET technology, including its Over time we developed an efficient working addressing and forwarding, from its analytical modeling, design, architec- relationship with them, and errors were dealt transport-layer end-to-end connection ture, deployment, and detailed mea- with more expeditiously. It is worthwhile noting establishment, which also involved flow surements. A summary of the that the history of packet networks has met with control, , retransmis- ARPANET principles and lessons institutional impediments to its progress, as sion, and more. TCP Version 3 (1978) learned appeared in a 1978 paper [59] have so many other technical advances over the introduced the split into two compo- after almost a full decade of experience course of history. In this case I have called out nents, but it was only in TCP Version 4 with the use, experimentation, and mea- three with which I was personally involved: (1980, with an update in 1981) that we surement of packet networks; this paper AT&T's lack of interest in packet switching, the see a stable protocol running that sepa- was part of a special issue on packet researchers’ reluctance to connect to the early rated out the (IP) communications which contains a num- network, and the above-mentioned negotiation from TCP (which now stood for Trans- ber of key papers of that era [60]. One with BBN.

IEEE Communications Magazine • August 2010 35 LYT-HISTORY-August-Kleinrock 7/20/10 10:28 AM Page 36

HISTORY OF COMMUNICATIONS

ACKNOWLEDGMENT tion Network For Computers Giving Rapid [41] Special session on the ARPANET in Proc. Response At Remote Terminals,” ACM Symp. Spring Joint Comp. Conf., AFIPS Conf. Proc., Op. Sys. Principles, Gatlinburg, TN, Oct. 1967. vol. 40, May, 1972: I want to thank Bradley Fidler, Mischa [23] D. W. Davies, “The Principles of a Data S. M. Ornstein et al., “The Terminal IMP for the Schwartz, and the reviewers of this col- Communication Network for Computers and ARPA Computer Network,” pp. 243–54. umn for their helpful comments and Remote ,” Proc. IFIP Hardware H. Frank, R. Kahn, and L. Kleinrock, “Computer (Edinburgh1968) D1. Communication Network — Design Experi- suggestions. [24] R. A. Scantlebury, P. T., Wilkinson, and K. ence with Theory and Practice,” pp. 255–70. A. Bartlett, “The Design of a Message S. D. Crocker et al., “Function-Oriented Proto- REFERENCES (A PARTIAL LIST OF Switching Centre for A Digital Communica- cols for the ARPA Computer Network,” pp. tion Network,” Proc. IFIP Hardware (Edin- 271–79. SELECTED REFERENCES) burgh 1968) D26. R. H. Thomas, D. A. Henderson, “McROSS — A [25] R. A. Scantlebury, “A Model for the Local Multi- System,” pp. [1] W. W. Massie and C. R. Underhill, “The Area of a Network 281–94. Future of the Wireless Art,” Wireless Telegra- Objectives and Hardware Organization,” Proc. L. G. Roberts, “Extensions of Packet Communi- phy and Telephony, 1908, pp. 67–71. 1st ACM Symp., Problems in the Optimization cation Technology to a Hand Held Personal [2] H. G. Wells, World Brain, Methuen & Co. of Data Communications Systems, Pine Moun- Terminal,” pp. 295–98. Ltd., 1938. tain, GA, Oct. 13–16, 1969, pp. 183–204. [42] R. Tomlinson, “The First Network Email,” [3] J. Nyce and P. Kahn, From Memex to Hyper- [26] J. C. R. Licklider, “Man-Computer Symbio- http://openmap.bbn.com/~tomlinso/ray/firste text — Vannevar Bush and the Mind’s sis,” IRE Trans. Human Factors in Electronics, mailframe. Machine, Academic Press, Inc. 1991 vol. HFE-1, Mar. 1960, pp. 4–11. [43] N. Abramson, “The AlohaNet — Surfing for [4] L. Kleinrock, “ and Its [27] J. Licklider, and W. Clark, “On-Line Man- Wireless Data,” IEEE Commun. Mag., vol. 47, Flexible Future,” IEEE Wireless Commun., Computer Communication,” Spring Joint no. 12, Dec. 2009, pp. 21–25. Feb. 2008, pp. 8–18. Comp. Conf., National Press, Palo Alto, CA, [44] “The ALOHA System — Another Alternative [5] B. Leiner et al., “The Past and Future History May 1962, vol. 21, pp. 113–28. for Compute r Communication,” Proc. AFIPS of the Internet,” Commun. ACM, vol. 40, no. [28] T. Marill and Lawrence Roberts, “Toward a Fall Joint Comp. Conf., vol. 37, 1970, pp. 2, Feb. 1997, pp. 102–08. Cooperative Network of Time-Shared Com- 281–85. [6] K. C. Redmond and T. M. Smith, From Whirl- puters,” Fall AFIPS Conf., Oct 1966. [45] L. G. Roberts, “ALOHA Packet System with wind to MITRE: The R&D Story of The SAGE [29] L. Roberts, “Multiple Computer Networks and without Slots and Capture,” ARPA Net- Air Defense Computer, The MIT Press, 2000. and Intercomputer Communication,” ACM work Info. Center, Stanford Research Inst., [7] F. J. Corbató, M. M. Daggett, and R. C. Symp. Principles, Gatlin- Menlo Park, CA, ASS Note 8 (NIC 11290), Daley, “An Experimental Time-Sharing Sys- burg, TN, Oct. 1967. June 1972. tem,” AFIPS Conf. Proc., vol. 21, 1962 SJCC, [30] See the fourth entry under “1967” in [46] L. Kleinrock and S. Lam, “Packet Switching pp. 335–44. http://www.packet.cc/ in a Slotted Satellite Channel,” AFIPS Conf. [8] J. R. Jackson, “Networks of Waiting Lines,” [31] L. Roberts, “Resource Sharing Computer Net- Proc., vol. 42, Nat’l. Comp. Conf., New York, , vol. 5, no. 4, Aug. works,” June 3, 1968; http://ia310808.us. June 1973, AFIPS Press, pp. 703–10. 1957, pp. 518–21. archive.org//load_djvu_applet.php?file=0/items/ [47] S. Lam and L. Kleinrock, “Packet Switching [9] L. Kleinrock, “Information Flow in Large ResourceSharingComputerNet works/Resource- in a Multiaccess Broadcast Channel: Dynamic Communication Nets,” Ph.D. thesis proposal, SharingComputerNetworks.djvu Control Procedures,” IEEE Trans. Commun., MIT, May 1961. [32] “Specifications of Interface Message Proces- vol. COM-23,Sept. 1975, pp. 891–904. [10] L. Kleinrock, “Creating a Mathematical The- sors for the ARPA Computer Network,” [48] R. Metcalfe and D. Boggs “Ethernet: Dis- ory of Computer Networks,” INFORMS-Oper- Department of the Army DBD/bj/OX 5-0494, tributed Packet-Switching for Local Comput- ations Research, Jan.–Feb. 2002. July 29, 1968; http://www.cs.utexas.edu/ er Networks,” Commun. ACM, vol. 19, no. 5, [11] L. Kleinrock, “Information Flow in Large users/chris/DIGITAL_ARCHIVE/ARPANET/RFQ- July 1976, pp. 395–404. Communication Nets,” RLE Quarterly ARPA-IMP.pdf [49] D. R. Boggs et al., “Pup: An Internetwork Progress Report, MIT, Apr. 1962. [33] L. G. Roberts, http://www.ziplink.net/users/ Architecture,” IEEE Trans. Commun., COM- [12] J. F. Kurose, M, Schwartz, and Y. Yemini, lroberts/SIGCOMM99_files/v3_document.htm 28, no. 4, pp. 612–24, Apr., 1980, p. 26. “Controlling Window Protocols for Time- [34] “Interface Message Processor: Specifications [50] V. G. Cerf and R. E. Kahn, “ A Protocol for Constrained Communication in a Multiple for the Interconnection of a Host and an Packet Network Interconnection,” IEEE Trans. Access Environment,” Proc. 8th Symp. Data IMP,” BBN Report 1822; http://www.bit- Commun. Tech., vol. COM-22, V 5, May Commun., Oct. 3–6, 1983, pp. 75–84. savers.org/pdf/bbn/imp/BBN1822_Jan1976.pdf 1974, pp. 627–41. [13] G. R. Andrews, “The Design of a Message [35] S. Crocker, RFC 1, “Host Protocol”; [51] D. Cohen, S. Casner, and J. W. Forgie, “A Switching System: An Application and Evalu- http://www.faqs.org/rfcs/rfc1.html Network Voice Protocol NVP-II,” ISI/RR-81- ation of Modula,” IEEE Trans. Software Eng., [36] L. Kleinrock, “Models for Computer Net- 90, Apr. 1981. vol. SE-5, No. 2, Mar. 1979. works,” Conference Record, IEEE ICC, Boul- [52] J. Postel, “Comments on Internet Protocol [14] W. W. Chu, “Demultiplexing Considerations der, CO, June 1969, pp. 21-9–21-16. and TCP,” IEN 2, 1977. for Statistical Multiplexors,” IEEE Trans. [37] T. Tugend, “UCLA to be first Station in [53] J. Postel, “RFC 768 — User Datagram Proto- Commun., COM-20: 603–609, June 1972. Nationwide Computer Network,” UCLA Press col,” 1980; http://www.faqs.org/rfcs/rfc768.html [15] L. Kleinrock, Queueing Systems, Volume II: Release, 3 July 1969. http://www.lk.cs.ucla. [54] G. P. Dinneen, “Host-to-Host Data Commu- Computer Applications, Wiley Interscience, edu/LK/Bib/REPORT/press.html. nications Protocols,” Apr. 3, 1980; New York, 1976 [38] G. C. Cole, “Computer Network Measure- http://www.potaroo.net/ietf/ien/ien152.txt [16] L. Kleinrock, Message Delay in Communica- ments; Techniques and Experiments,” UCLA [55] J. Postel, “NCP/TCP Transition Plan,” RFC tion Nets with Storage, Ph.D. dissertation, School of Engineering and Applied Science, 801, Nov. 1981; http://www.rfceditor. MIT, Cambridge, MA, 1962. Engineering Report UCLA-ENG-7165 Oct. 1971. org/rfc/rfc801.txt [17] L. Kleinrock, Communication Nets; Stochas- [39] S. Crocker, RFC 36, “Protocol Notes”; [56] P. T. Kirstein, “The Early History of Packet tic Message Flow and Delay, McGraw-Hill, http://www.rfcarchive.org/getrfc.php?rfc=36 Switching in the UK,” IEEE Commun. Mag., New York, 1964. [40] Special session on the ARPANET in Proc. vol. 47, no. 2, Feb. 2009, pp. 18–26 [18] P. Baran, “On Distributed Communication Spring Joint Comp. Conf., AFIPS Conf. Proc., [57] T. Rybcznski, “Packet Switching Networks” Rand Paper P-2626, Sept. 1962. vol. 36, May, 1970, pp. 543–97: (1975–1985): A Canadian Perspective,” IEEE [19] D. Hochfelder and P. Baran, An Oral Histo- L. G. Roberts and B. D. Wessler, “Computer Net- Commun. Mag., vol. 47, no. 12, Dec. 2009, ry, IEEE History Center, 24 Oct. 1999, work Development to Achieve Resource pp. 26–31. http://www.ieeeghn.org/wiki/index.php/Oral- Sharing,” pp. 543–49. [58] L. Kleinrock and W. Naylor, “On Measured History:Paul_Baran#Institute_For_the_Future. F. Heart et al., “The Interface Message Processor Behavior of the ARPA Network,” AFIPS Conf. 2C_1967.2B for the ARPA Computer Network,” pp. Proc., vol. 43, National Comp. Conf., Chica- [20] P. Baran et al., “On Distributed Communi- 551–67. go, IL, May 1974, AFIPS Press, pp. 767–80. cations,” The RAND Corporation, Santa L. Kleinrock, “Analytic and Simulation Methods [59] L. Kleinrock, “Principles and Lessons in Monica, CA, Aug. 1964. in Computer Network Design,” pp. 569–79. Packet Communications,” Proc. IEEE, Special [21] D. W. Davies, “Proposal for a Digital Com- H. Frank, I. T. Frisch, and W. S. Chou, “Topologi- Issue on Packet Communications, vol. 66, munication Network,” unpublished memo, cal Considerations in the Design of the ARPA no. 11, Nov. 1978, pp. 1320–29. June 1966, http://www.archive.org/details/ Network,” pp. 581–87. [60] Proc. IEEE, Special Issue on Packet Commu- NationalPhysicalLaboratoryProposalForADigi- S. Carr, S. Crocker, and V. Cerf. “Host-Host nications, vol. 66, no. 11, Nov. 1978. talCommunicationNetwork in the ARPA Net- [22] D. W. Davies et al., “A Digital Communica- work,” pp. 589–98.

36 IEEE Communications Magazine • August 2010