• Vint Cerf and Bob Kahn are this year’s recipients of the ACM Turing Award (the highest award in • Computer Science) for their pioneering work in internetworking. To commemorate their award, and • • with the gracious consent of IEEE (which originally published the paper) and the authors, SIGCOMM • is reprinting their famous early paper on internetworking, “A Protocol for Packet Network • Interconnection,” from IEEE Transactions on Communications of May 1974. • • When the paper was written, Bob was early in his thirteen year-career at the US Defense Advanced • Research Projects agency and Vint was a junior professor at Stanford. Bob eventually left DARPA and • has continued in a distinguished career of nurturing research through the not-for-profit Corporation for • • National Research Initiatives (CNRI), which he founded in 1986 and has led ever since. Vint soon left • Stanford to spend several years at DARPA. After DARPA, Vint worked at MCI, leading the develop- • ment of MCI Mail, then worked with Bob at CNRI, then rejoined MCI in 1994, where he is senior vice • president for technology strategy. • • Throughout this time, both Vint and Bob have been actively involved in Internet activities. For many • years, Vint served on (and for some years, chaired) the Internet Activities Board. Vint is currently • • chairman of the board of the Internet Corporation for Assigned Names and Numbers (ICANN). Bob, • Vint and CNRI have provided essential support services for the Internet Engineering Task Force • (IETF) for many years. • • Closer to home, both Bob and Vint have served as chairs of SIGCOMM. And both have received the • ACM SIGCOMM Award: Bob in 1993 and Vint in 1996. • • I continue to find reading this paper intellectually rewarding, both for what it contains and what it does • not. What it contains, of course, is a lot of the ideas that eventually became TCP and IP. It is some- • what astonishing to see how much was known in 1974, only 5 years after ARPANET was turned on. • Equally interesting are things that were missing, such as congestion coordination with routers (e.g. the • • TCP congestion window). And there are also a few (not many) ideas that turned out to be bad (such • as shrinking the receiver window from the right). All in all a rewarding read and I hope you’ll agree, • worth reprinting 31 years later! • • • Craig Partridge • Chief Scientist, BBN Technologies • past-chair, ACM SIGCOMM • • • • • • • • • • • • • • • • • • • • • • • • • • • a c m s i g c o m m • • • •

ACM SIGCOMM Computer Communication Review 69 Volume 35, Number 2, April 2005 ACM SIGCOMM Computer Communication Review 70 Volume 35, Number 2, April 2005 IEEE TR.INSACTIOSS OX COMMUNIC~LTIOKS, VOL. COM-22, NO. 5, MAY 1974 637

A Protocol For Packet Network Intercommunication

VINTON G. CERF AND ROBERT E. ICAHN, MEMBER, IEEE

Absfract-A protocol that supports the sharing of resources that set of computer resources called HOSTS, a set of one or exist in different packet switching networksis presented. Theproto- more packetswitches, and a collcction of communication col provides for variation in individual network packet sizes, trans- media thatinterconnect the packct switches. Within mission failures, sequencing,flow control, end-to-end error checking, and the creation and destruction of logical process-to-process con- each HOST, wc assume thatthere exist processes which nections. Some implementation issues are considered, and problems must communicate with processes in their own or other such as internetwork routing,accounting, and timeouts areexposed. HOSTS. Any current definition of a process will be adequate for our purposes [13]. These processes are generally the INTRODUCTION ultimate source and destination of data in the network. Typically, within an individual network,there exists a 1 THE LAST few years considerable effort has been protocol for communicationbetween any source and I"expended on the design and implementation of packet destination process. Only the source anddestination switching net\vorla of resources that exist in different packct switching net- ratcs, buffering and signaling strategies,routing, propa- works. gation delays, etc. In addition, somc mechanism is gen- Aftera brief introduction to internetwork protocol erallyprcscnt for errorhandling anddetermination of issues, we describe the function of a GATEWAY as an intcr- status of the networkscomponents. face bctwccn nctn-orks and discuss its role in the protocol. Individual pacltct switching nctn;orlStanford University, Stanford, Calif. missing data. End-to-cnd restoration proccduros are It. E. Kahn is with theInformation Processing Technology Office, AdvancedResearch Projects Agency, Department of De- desirable to allow complete recovery from these con- fense, Arlington, Va. ditions.

ACM SIGCOMM Computer Communication Review 71 Volume 35, Number 2, April 2005 636 IEEE TRAKSACTIONS ON COMMUNICATIONS. MAY 1074

intact the internal operation of each individual network This is easily achieved if twonetworks interconnect a: if each were a HOST to the other network, but withoul PACKET-SWITCHING SUBNETWORK /n\ utilizingor indeed incorporating anyelaborate HOS~ protocol transformations. It is thus apparent that the interfacebetween network; PS I PS must play a central role in the development of any net (-) work interconnection strategy. We give a special name tc this interface that performs these functions and call it : GATEWAY.

THE GATEWAYNOTION

PACKET-SWiTCHINGNETWORK PSSWITCH = PACKET In Fig. 2 we illustrate three individual networkslabelec Fig. 1. Typical packet switchingnetwork. A, B, and C which are joined by GATEWAYS M and N GATEWAY A// interfaces network A with network B, anc 5) Status ,information,rout,ing, fault detection, and GATEWAY N interfacesnetwork B to network C. W isolation are typically different in each network. Thus, to assume that an individual network may have more t,ha~ obtain verification of certainconditions, such as an in- one GATEWAY (e.g., network B) and that there may b accessible ordead destination, various kinds of coordi- more than one GATEWAY path to use in going between I nation must be invoked between the communicating net- pa,ir of networks. The responsibility for properly routin1 works. data resides in the GATEWAY. It would be errtremely convenient if all the differences In practice, a GATEWAY between two networks may b betweennetworks could be economically resolved by composed of twohalves, each associated with it,s ow1 suitableinterfacing at .thenetwork boundaries. For network. It is possible to implement each half of a GATE many of the differences, this objective can be achieved. WAY so it need only embed internetwork packets in loca However, both economic and technical considerations lead packetformat or extractthem. We propose thatth us to prefer that the interface be as simple and reliable GATEWAYS handleinternetwork packet,s in a standarc as possible and deal primarily with passing data between format, but me are not proposing any particular trans networks that use differentpacket switching strategies. mission procedure between GATEWAY halves. The question now arises as to whether the interface Let us now trace the flow of data through the inter ought to account for differences in HOST or process level connectednetworks. We assume a packet of data fron protocols by transforming thesource conventions into the process X entersnetwork A destined for process Y il correspondingdestination conventions. We obviously network C. The address of Y is initially specified b: wantto allow convcrsion betweenpacket switching process X and the address of GATEWAY M is derked fron strategies at the interface, to permitinterconnection of the address of process Y. We nmakc no attempt to spccif: existing and planncd networks. However, the complcxity whether the choice of GATEWAY is made by process X and dissimilarity of the Hosl7 or process level protocols its HOST, or one of thc packet switches in network -4. Thl makes it desirable to avoid having to transform between packet traverses network A until it reaches GATEWAY iI4 them at the interface,even if thistransformation were At the GATEWAY, the packet is reformatted to meet thl always possiblc. Rather,,compatible HOST and process requirements of network B, account is taken of this uni levcl protocols must bc developed to achicvceffective of flow between A and B, and the GATEWAY delivers ths intcrnctxork resourcc sharing. The unacceptable al- packet to network B. Again the dcrivation of the nex ternative is for every HOST or process to implcmcnt every GATEWAY address is accomplished based on the address o protocol (a potentially unbounded number) that may be the destination Y.In thiscase, GATEWAY AT is the next one needed to cornmunicatc with other networks. We there- Thc packet traverscs network R until it finally rcache fore assume that a comnmn protocol is to beused between GATEWAY N whcrc it is formattcd tomcet the requirement HOST'S or processes in diffcrcntnetworks and that the of network C. Account is again taken of this unit of flo~ interface bctn-ccn networks should takc as small a role as betwccnnetworks B and C. Upon enteringnetwork C possiblc in this protocol. the packet is routedto the Hosr in which process I To allow nc:tworl

ACM SIGCOMM Computer Communication Review 72 Volume 35, Number 2, April 2005 packet size parameters of one network from the internal \ packet size parameters of all other networks. N 2) It would be very difficult to increase the maximum W permitted packet size in response to new technology (e.g., GATEWAY GATEWAY large memory systems, higher data rate communication Fig. 2. Three networks interconnected by two GATEWAYS. facilities, etc.) since this would require the agreement and then implen-rentation by all participating networks. 3) Associativeaddressing and pa.clcet encryption may require the size of a particular pa'ckct to cxpand during (may be null) b- Internetwork Header transit for incorporation of new information. LOCAL HEADER SOURCE DESTINATION SEQUENCE NO. BYTE COUNTIFLAG FIELD\ TEXT ICHECKSUM Provision for fragmentation (regardless of where it is Fig. 3. Internetworkpacket format (fields not shown to scale). performed) permits packet sixc variations to be handled on an individualnetwork basis without global admin- istrationand also permits HOSTS and processes tobe worlc header, is illustrated in Fig. 3. The source and desti- insulated from changes in the pa,ckct sizes permitted in nation entries uniforndy and uniquely identify the address any networks through which their data must pass. of every HOST in the composite network. Addressing is a If fragmentation must be done, it appears best to do it subject of considerablecomplexity which is discussed upon entering the nestnetu-orlc at theGAPEWAY since only in greater detail in the nextsection. Thenext two entries in t.his GATEWAY (and not the othernetLvorlcs) must be awarc the header provide a sequence number and a byte count of the int.ernal packet size parameters which made the that may be used to properly sequence the packets upon fragmentation necessary. delivery tothe dest'inationand may also enable the If a GATEWAY fragnwnts an incoming packet intot'T1-o or GATEWAYS to detect fault conditions affecting the packet. more paclcet,s, they must eventually bepassed along to the The flag field is used to convey specific control information destination HOST asfragnxnts or reassembledfor the and is discussed inthe sect.ion onretransmission and HOST. It is conceivable that one might desirethe GArrEwAY duplicatedetection later. The remainder of thepacket to perform the rea.ssenlbly to simplify the taskof the desti- consists of text for deliveryto the destination anda trailing nation HOST (or process) and/or to take advantage of a check sum used for end-to-end software verification. The larger packet size. We take the position tJhat GATEWAYS GATEWAY does not modify the text andmerely forwards the shouldnot perform thisfunction since GATEWAY re- check sum along without computing or recomputing it. assen-rbly can lead to serious buffering problems, potential Each nct\r-orlr may need to augment the packet format deadlocks, the necessity for all fragments of a packet to before it can passt'hrough the individualnetu-ork. We pass through the same GArrEwA>r,and increased dclay in havc indicated a local header in thefigure which is prefixed transmission. Furthermore, it is not sufficient for the to the beginning of the packet. This local header is intro- GATEWAYS to provide this functionsince the final GATEWAY duced nlcrely t'o illustrate the concept of embedding an may also haveto fragment a paclxt fortransmission. intcrnetworlc packet in the format of the individual net#- Thus the destination HOST must be prepared to do this work through which thepacket must pass. It will ob- task. viously vary in its exact form from network to network Let us now turn briefly to the somewhat unusual ac- and may evenbe unnecessary in some cases. Although not counting effect 11-hich arises when a packet may be frag- explicitly indicated in the figure, it is also possiblc that a mentedby one or more GATEWAYS.We assume, for local trailer may be appended to the end of the packet. simplicity, that each network initially charges a fixed rate Unless all transnlittedpackets are legislatively re- per paclrct transmitted, regardless of distancc, and if one stricted to be small enough to be accepted by cvcry in- network can handle a larger packet size tlml another, it dividual network, the GATEWAY may be forced to split a charges a proportionally larger price per paclcct. We also packet int,o two or more smaller packets. This action is assume that a subsequentincrease in any network's called fragmentation and mustbe done in sucha way that packet size docs not result in additionalcost per packet to the destination is able to piece togcthcr the fragmcntcd its users. The charge to a uscr thusremains basically packet. It isclear that the internct\vorl; header format constant through any net which must fragmcnt a packet. imposesminimuma packet size which allnetworks The unusual cffcct occurs when a paclcct isfragmented into mustcarry (obviouslyall networks will wantto carry smaller packets which must individually pass through a packets larger than this minimum). We believe the long subsequent nctxvork with a largerpacket size than the rangcgrowth and development of internctworl; com- original unfragmented packet. We expect that most net- munication would be seriouslyinhibited by specifying works \vi11 naturally selech packet sizesclose to one how much larger than the minimum a paclcct sizc can bc, anot'her, but in anycase, an increase in packet size in one for tjhc follo\\-ing reasons. net, even when it causes fragmentation, will not increase 1) If a maximum permitted packetsize is specified then the cost of transnlission and may actually decrease it. In it bccomos impossible to completely isolate the internal the event that any other packet charging policies (than

ACM SIGCOMM Computer Communication Review 73 Volume 35, Number 2, April 2005 G40 IEEE TRANSACTIONS ON COMMUNICATIONS, MAY 1974 the one me suggest) are adopted,differences in cost can be construct exactly, and in order, the stream of text bytes used as an economic lever toward optimization of indi- produced bythe source TCP. Atthe destination, this vidual network performance. stream must first be parsed into segments and these in turn must be used to reconstruct messages for delivery to PROCESS LEVEL COMMUNICATION the appropriate processes. There are fundamental problems associated with this We suppose that processes wish to communicate in full strategy due to thepossible arrival of packets out of order duplexwith their correspondent’s usingunbounded but atthe destination.The most critical problem appears finite length messages. A single character might constitute to be the amountof interference that processes sharing the the text of a message from a process to a terminal or vice sameTCP-TCP byte stream may cause among them- versa. An entire page of characters might constitute the selves. This is especially so at the receiving end. First, text of a message from a file to a process.- A data stream the TCP may be put tosome trouble to parse the stream (e.g., a continuously generated bit string) can be repre- back into segments and then distribute them to buffers sented as a sequence of finite length messages. where messages are reassembled. If it is not readily ap- Within a HOST we assume theexistence of .a transmission parent that all of a segmenthas arrived (remember, it controlprogram (TCP) which handles the transmission may come as severalpackets), the receiving TCP may and acceptance of messages on behalf of the processes it have to suspend parsing temporarily until more packets serves. The TCP is in turn served by one or more packet have arrived. Second, if a packet is missing, it may not be switches connected to the HOST in which the TCP resides. clear whether succeeding segments, evenif they are identi- Processes thatwant to communicatepresent messages fiable, can be passed on to thereceiving process, unless the to the TCPfor transmission, and TCP’Rdeliver incoming TCP has knowledge of some process level sequencing messages tothe appropriate destination processes. We scheme. Such knowledge would permit the TCP to decide allow the TCP to break up messages into segments be- whether a succeeding segment could be delivered to its cause the destination may restrict the amountof data that waiting process. Finding the beginning of a segment when mayarrive, because the local networkmay limit the there are gaps in the byte stream mayalso be hard. maximumtransmission size, or because theTCP may Case 2) : Alternatively, we might take the position that need to share its resources among many processes con- the destination TCP should be able to determine, upon currently.Furthermore, me constrain thelength of a its arrival and without additional information, for which segment to an integral number of 8-bit bytes. This uni- process or processes a received packet is intended, and if formity is mosthelpful in simplifying the software needed so, whether it should be delivered then. with HOST machines of differentnatural wordlengths. If the TCPis to determine for which process an arriving Provision at the process level can be made for padding a packet is intended, every packet must contain a proces6 message that is not an integral number of bytes and for header (distinct from the internetwork header) that com- idcntifyingwhich of the arrivingbytes of text contain pletely identifies thc destination process. For simplicity, information of interest to the receiving process. we assume that each packet contains text from a single Multiplexingand demultiplexing of segmentsamong process which is destined for a single process. Thus each processes are fundamental t.asks of the TCP. On trans- packet need contain only one process header. To decide mission, a TCP must multiplex together segments from whether the arriving datais deliverable to the destination different source processes andproduce internetwork process, the TCP must be a.ble to determine whether the packets for delivery to one of it.s serving packet switches. data is in the proper sequence (we can make provision On reception, a TCP will accept a sequence of packets for the destination process to instruct its TCP to ignore from its serving packet switch(es). From this sequence sequencing, but thisis considered a special case). Withthc of arrivingpackets (generally from different HOSTS), assumption that each arriving packet contains a process the TCP must beable to reconstruct anddeliver messages header, the necessary sequencing and destination procesf to the proper destinationprocesses. ident)ification is immediately available to the destinatior We assume that every segment is augmented with ad- TCP. ditionalinformation that allows transmittingand re- Both Cases 1) and 2) provide for the demultiplexing ceiving TCP’s to identify destination andsource processes, and delivery of segments todestination processes, but respectively. At this point, we must face a major issue. only Case 2) does so without the introductionof potential How should the source TCI’ format segments destined for interprocess interference. Furthermore, Case 1) introduceE the same destination TCP?We consider two cases. extramachinery to handle flow controlon a HOST-to- Case 1) : If we take t.he position that segment boundaries HOST basis! since there must also be some provision for are immaterial and that a byte stream can be formed of proccss level control, and this machineryis little used since segments destined for the same TCP, then we may gain the probability is small that within a given HOST, two improvcd transmission efficiency and resource sharing by processes dlbe coincidentally scheduled to send messages arbitrarily parceling the stream into packets, permitting to the same destination HOST. For this reason, we select manystgments to sharea single internetworkpacket the method of Case 2) as a part of the internetwork headcr. Howcver, this position results in the need to re- transmission QrOtOCOl.

ACM SIGCOMM Computer Communication Review 74 Volume 35, Number 2, April 2005 CERFAND KAHN: PACKET NETWORK INTISRCOMMUNICATION 641

ADDRESSFORMATS 8 16 The selection of address formats is a problem between NETWORK TCP IDENTIFIER networks because the local network addresses of TCP's Fig. 4. ',TCPaddress. may vary substantially in format and size. A uniform in- ternetwork TCP addressspace, understood by each Even though the transmitted port name field is large, GATEWAY and TCP, is essential to routing and delivery it is still a compact external name for the internal repre- of internetwork packets. sentation of the port. The use of short names for port Similartroubles are encountered when we dealwith identifiers is often desirable to reduce transmission over- process addressing and, more generally, port addressing. head and possibly reduce packet processing time at the We .introduce the notion of ports in order to permit a dehnationTCP. Assigning short names to each port, process to distinguish between multiple message streams. however,requires an initialnegotiation between source The port is simplya designator of one suchmessage stream and destination to agree on a suitable short name assign- associated with a process. The meansfor identifying a port ment, the subsequentmaintenance of conversion tables are generally different in different operating systems, and at both the source and the destination, and a final trans- therefore, to obtain uniform addressing, a standard port action to release the short name. For dynamic assignment address format is also required. A port address designates of port names, this negotiation is generally necessary in a full duplex message stream. any case. TCP ADDRESSING SEGMENT AND PACKET FORMATS TCP addressing is intimatelybound up in routing issues, since a HOST or GATEWAY must choose a suitable As shown in Fig. 5, messages are broken by the TCP destination HOST or GATEWAY for an outgoing int,ernetworl< into segments whose format is shown in more detail in packet. Let us postulate the following address format for Fig. 6. The field lengths illustrated are merely suggestive. the TCP address (Fig. 4). The choice for network identi- The first two fields (source port and destination port in fication (8 bits) allows up to 256 distinct networks. This the figure) have already been discussed in the preceding size seems sufficient for the forseeable future. Similarly, section on addressing. The uses of t.he third and fourth the TCP identifier field permits upto 65 536 distinct fields (window and acknowledgmentin the figure) will TCP's to be addressed, which seems more than sufficient be discussed later in the sectionon retransmission and for any given network. duplicate detection. As each packet passes through a GATEWAY, the GATEWAY We recall from Fig. 3 that aninternetwork header con- observes the destinationnetwork ID to determine how tains both a sequence number and a byte count, aswell as to route the packet. If the destinationnetwork is con- a flag field and a check sum. The USCS of these fields are nected to theGATEWAY, the lower 16 bits of the TCPaddress explained in the following section. are used to produce a local TCP address in the destination network. If the destination network is not connectedto the REASSEMBLY AND SEQUENCING GATEWAY, the upper S bits are used to select a subsequent The reconstruction of a message at the receiving TCP GATEWAY. We malx noeffort to specify how each in- clearly requires' that eachinternetwork packet carry a dividualnetwork shall associate the internetwork TCP sequence number which is unique to its particular desti- identifier with its local TCP address. We also do not rule nation port message stream. The sequence numbers must out the possibility that the local network understands the bemonotonic increasing (or decreasing) since thcyare internetwork addressing scheme and thus alleviates the used to reorder and reassemble arrivingpackets into a GATEWAY of the routing responsibility. mcssage. If the space of sequence numbers were infinite, we could simply assign the next one to each new packet. PORT ADDRESSING Clearly, this space cannot be infinite, andwe will consider A receiving TCP is faced with the task of demultiplex- what problems a finite sequence number space will cause ing the stream of internetworkpackets it receives and when we discuss retransmission and duplicate detection reconstructing the original messages for each destination in the next section. We propose the following scheme for process. Eachoperating system has its own internal performing the sequencing of packets and hence the re- means of identifying processes and ports. We assume that construction of messages by the destination TCP. 16 bits aresufficient to serve as intcrnctwork portidentifiers. A pair of ports will exchange one or more messages over Asending process nccd not know how the destination a period of time. We could view the sequence of messages port identification will be used. The destination TCP produced by one port as if it were embedded in an in- will be ablc to parse this number appropriately to find finitely longstream of bytes. Each byteof the message has the proper buffer into which it will place arriving packets. a unique sequence number which we takc to be its byte We permit a large port number field to support processcs location relativc to the beginning of the stream. When a which wantto distinguishbctween many different messages streams concurrently. In reality, we do not care Inthe case of encryptedpackets, a preliminary stage of re- how the 16 bits are sliced up by the TCP'sinvolved. assembly may be required prior to decryption.

ACM SIGCOMM Computer Communication Review 75 Volume 35, Number 2, April 2005 643 IEEE TRANSACTIONS ON COMMUNICATIOKS, MAY 197'

byte identification-sequence number

First Message LH = Local Header IH = InternetwolX Header PH = Process Header CK = Checksum (SEQ = k) Fig. 5. Creation of segments and packets from messages. Fig. 7. Assignment of sequencenumbers.

32 32 16 16 En

Source Port DertinatianIPortPort Source Wmdow ACK Text (Field sizes in bits1 16 bits ,+JPlOLIIl Hed..LJS R E E _.. YESM Fig. 6. Segment format (process header and text). NL Ill I LEnd of Message when set = 1 segmentis extracted from the message bythe source End of Segment when set = 1 TCPand formatted for internetworktransmission, the Release Use of ProcessIPort when set=l Synchronize to Packet Sequence Number when set = 1 relative location of the first byte of segment text is used as Fig. 8. Internetwork header flag field. the sequencenumber for the packet. Thebyte count field in the internetwork header accounts for all the text in-the segment (but docs not include the check-sum bytes - 1000 bytes . or t'he bytes in either internetxork or process header). 100101 102 . . . We emphasize that the sequence number associated with I TEXT OFMESSAGE A a given packet is unique only to the pair of ports that are communicating(see Fig. 7). Arrivingpackets are ex- SEQ CT ES EM 500 2 amined to determine for which port they are intended. SRC 500 1001 0 DST PH TEXT CK 1- 1- internetwork header segment 1 The sequence numbers on each arriving packet are then split by --+ source used to determine the relative location of the packet text TCP. -. in the messages under reconstruction. We note that this SEQ EMCT ES 500 2 allows the exact position of the data in the reconstructed SRC DST 600 1 500 1 PH TEXT CK message to bedetermined even n-hen pieces 'arestill

missing. 2 250~~~ ~ Every segment produced by a source TCP is packaged SRC DST 250 1000 0 / PH TEXT packet A1 in a single internetwork packet and a check sum is com- split SRC DST 350 1 2500 PH TEXT CK packet A12 puted over the text andprocess header associated with the by GATEWAY segment. SRC DST 600 250 0 0 PH TEXT packet AZ1 The splitting of messages into segments by the TCP and the potential splitting of segments into smaller pieces SRC DST 850 250 1 1 PH TEXT CK packet A22 by GATEWAYS creates the necessity for indic,ating to- the Fig. 9. Message splitting and packet splitting. destination TCP when the end of asegment (ES) has arrived and when the end of a message (EM) has arrived. The flag field of the internetwork header is used for this of internetwork packets. Packets A1 and A2 have the purpose (see Fig. S) . ES bitsset, and A2 hasits En1 bitset as well. Whe The ES flag is set by the source TCP each time it prc- packet A1 passes through the GATEWAY, it is split into tw pares a segment for transmission. If it should happen that pieces: packet A 11 for which neither EM nor ES bits a1 the message is completely contained in the segment, then xt, and packet A12 whose ES bit is set. Similarly, packt the EM flag would also be set. The EM flag is also set on A, is split such that thefirst piece, packet A21, has neithe the last segment of a message, if the message could not bit set, but packet A22 has both bits set. The scyuenc becontained in one segment, These two flags are used number field (SEQ) and the byte countfield (CT) of eac bythe destination TCP, respectively, to discover the packet is modified by the GATEWAY to properly identif presence of a check sum for a given segment and todiscover the t'ext bytes of each packet. The GATEWAY need on1 that a complete message has arrived. cxamine the internetmork header to do fragmentation. The ES and EM flags in the internetwork header are The destination TCP, upon reassembling segment 9 known to theGATEWAY and areof special importance when will detect the ES flag and will verify the check sum packets must be split apart for propagation through the knows is contained in packet iz12. Upon rcceipt of pack( next local network. We illustratetheir use with an ex- Az2,assuming all other packets have arrived, the dest ample in Fig. 9. nation TCP detects that it has reassembled a complel The original message -4 in Fig. 9 is shown split into two message and can now advise the destination process of il segments A and Az and formatted'by the TC1' into a pair rcceipt,:

ACM SIGCOMM Computer Communication Review 76 Volume 35, Number 2, April 2005 CRRF AND KAHX: PACKET NETWORK INTERCOMMUNICATION 643

RETRANSMISSIONAND DUPLICATE Left Window Edge I DETECTION 0 a a+w- 1 n-1 No transmissioncan be 100 percent reliable. We 1- 1- window propose a timeout and positive acknowledgment mecha- -4 nism which will allow TCP’s torecover from packet losses packet sequence number space from one HOST to another. A TCP transmits packets and I< -1 waits for replies (acknowledgements) that are carried in Fig. 10. The windowconcept. the reverse packet stream. If no acknowledgment for a particularpacket is received, the TCP will retransmit. It is our expectation that the HOST level retransmission Source Address mechanism,which is described inthe following para- Destination graphs, will notbe called uponvery often in practice. I Address I Evidence already exists2 that individual networks can be effectively constructed without this feature. However, the inclusion of a HOST retransmissioncapability makes it possible to recover from occasional network problems and 6 Next Read Position 7 End ReadPosition allows a wide range of HOST protocol strategies to be in- 8 corporated. We envision it will occasionally be invoked to 9 Timeout allow HOST accommodation to infrequent overdemandsfor 10 limited buffer resources, and otherwise not used much. Anyretransmission policy requiressome means by Fig. 11. Conceptual TCBformat. which the receiver can detect duplicate arrivals. Even if an infinite number of distinct packet sequence numbers expected. This effectively acknowledges bytes in between. were available, the receiver mould still have the problem The left window edge is advanced to the next sequence of knowing how long to rememberpreviously received number expected. packets in order to detect duplicates. Matters are compli- 2) Packets arriving with a sequence number to the left cated by the fact that only a finite number of distinct of the window edge (or, in fact, outsideof the window) are sequence numbers are in fact available, and if they are discarded, and the current leftwindow edge isreturned as reused, the receiver must be able to distinguish between acknowledgment. new transmissions and retransmissions. 3) Packets whosesequence numbers lie withinthe A window strategy, similar to that used by the French receiver’s window but do not coinicide with the receiver’s CYCLADES system (voie virtuelle transmission mode [SI) leftwindow edge areoptionally kept ordiscarded, but andthe ARPANET verydistant HOST connection [lS], are notacknowledged. This is the case when packets arrive is proposed here (see Fig. 10). out of order. Suppose that the sequence number field in the inter- We make some observations on this strategy. First, all network header permits sequence numbers to range from computations with sequence numbers and window edges 0 to n - 1. We assume that the sender will not transmit must be mademodulo n (e.g., byte0 follows byte n - 1). more than w bytes without receiving an acknowledgment. Second, w must be less than n/Y; otherwisea retrans- The w bytes serve as the window (see Fig. 11). Clearly, mission may appear to the receiver to be a new trans- w must be less than n. The rules for sender and receiver mission in the case thatthe receiver hasaccepted a are as follows. window’s worth of incoming packcts, but all acknowledg- Sender: Let L be the sequence number associated with ments havc been lost. Third, the receiver can either save the left window edge. or discard arriving packets whose!sequence numbers do 1) Thesender transmits bytes from segments whose not coincide with the receiver’s left window. Thus, in the text lies between L and up to L + w - 1. simplestimplementation, the receiver need notbuffer 2) On timeout(duration unspecified), the sender more than onepacket per message stream if space is retransmits unacknowledged bytes. critical. Fourth,multiple packets can be aclrnowledgcd 3) Onreceipt of acknowledgment consisting of the simultaneously.Fifth, the receiver is ableto deliver receiver’s currentleft window edge, the sender’s,left messages to processes in their proper order as a natural windowedge is advanced over the aclrnowledged bytes result of the reassemblymechanism. Sixth, when dupli- (advancing the rightwindow edge implicitly). catesarc detected, the acknowledgmentmethod used Receiver: naturally works to rcsynchronizcscndcr and receiver. 1) Arriving packets yhose sequence numbers coincide Furthermore, if the rcccivcr acceptspackets whose with the receiver’s current left window edge are acknowl- sequcncenumbcrs lie withinthe current window but edged by sending to the source the next sequence number

Actually n/2 is merely a convenient number to use; it is only The ARPANET is one such example. required that a retransmission not appear to be a new transmission.

ACM SIGCOMM Computer Communication Review 77 Volume 35, Number 2, April 2005 644 IEEE TRANSACTIOM ON COMMUNICATIONS, MAY 1974 which are not coincident with the left window edge, an TCP INPUT/OUTPUT HAND,LING acknowledgment consisting of thecurrent left window The TCPhas a component which handles input/output edge wouldact as a stimulus causeto retransmission of the (I/O) to and from the network4 When a packet has ar- unacknowledgedbytes. Finally, we mention an overlap rived, it validatesthe addresses and places the packet problemwhich results from retransmission, packet ona queue. A pool of buffers can be set up to handle splitting,and alternate routing of packetsthrough dif- arrivals, andif all available buffers areused up, succeeding ferent GATEWAYS. arrivals can be discarded since unacknowledged packet5 A600-byte packetmight pass through one GATEWAY will be retransmitted. andbe broken into two 300-byte packets. On retrans- On output, a smaller amount of buffering is needed, mission, the samepacket might bebroken into three since process buffers can hold the data to be transmitted 200-byte packets going throughadifferent GATEWAY. Perhaps double buffering mill be adequate. We make nc Since each byte has a sequence number, there is no con- attemptto specify how the bufferingshould bedone fusion at the receiving TCP. We leave for later the issue except to require that it be able to service the network of initiallysynchronizing thesender and receiver left with as little overhead as possible. Packet sized buffers window edges and the window size. one or more ring buffers, or any other combination art possible candidates. FLOWCONTROL When a packet arrivesat thedestination TCP, itis placec on a queue which the TCP services frequently. For ex Every segment that arrives at the destination TCP is ample, the TCPcould be interrupted when a queue place ultimately acknowledged byreturning the sequence ment occurs. The TCP then attempts toplace the packel number of the next segment which must be passed to the textinto the properplace in’the appropriate proces! process (it may not yet have arrived). receive buffer. If the packet terminates a segment, ther Earlier we described the use of asequence number it canbe checksummed and acknowledged.Placemeni spaceand window to aidin duplicate detection. Ac- may fail for several reasons. knowledgments are carried in the processheader (see I) Thedestination .process may not be. prepared tc Fig. 6)and- along withthem there is proviqion for a receive from the.etated source, or the destination port 11 “suggested window”.which the receiver can use to control may not exist. the flow of data from the sender. This is intended to be 2) There may be insufficient buffer space for the text the main component of the process flow control mecha- 3) The beginningsequence number of thetext ma3 nism. The receiver is frcc to vary the windo& size accord- not coincide with the nextsequence number to bedeliverec ing to any algorithm it desires so longas the window to the process (e.g., the packet has arrived out of order) size never exceeds half thc sequence number space.3 In the first case, the TCP should simply discard thf This flow controlmechanism is exceedinglypowerful packet (thus far, noprovision has been made for err01 and flexible and does notsuffer fromsynchronization acknowledgments). Inthe second andthird cases, thc troubles that may be encountered by incremental buffer packet sequence number can be inspected to determinc allocationschemes [9],[lO]. Hoivever, it relies heavily whether the,packet textlies within the legitimate ivindow on an effective retransmission strategy. The receiver can for reception.If it does, the TCP mayoptionally keep thc reduce the window even while packets are en route from packetqueued for later processing. If not,the TCI the senderwhose window is presentlylarger. The net can discard thepacket. In either case theTCP car effect of thisreduction will be thatthe receiver may optionally acknowledge with the current leftwindow edge discardincoming packets (theymay be outside thc It may happen that the process receive buffer is no’ window) and reiterate thc currentwindow size along with present in the active memoryof the HOST, but is stored or a current window edge as acknowledgment.’By the same secondary storage. If this is the case, the TCPcan promp token, the sender can, uponoccasion, choose to send more the scheduler to’bring in the appropriate buffer and thc than a window’s worth of data on the possibility that the packet can be queued for latcr processing. reccivcr will expand the window to accept it (of course, the If therc are no niore input buffers available to the TCI sender must notsend more,than half the sequence number for temporary queueing of ‘incoming packets, and ifthc space at any time).Normally, we would expect the sender TCI’ cannot quickly use the arriving data (c.g., a TCI toabide by thc window limitation.Expansion of the to TCP message) , then thc packet is discarded. Assuminf window by the rcccivcr mcrcly allows more data to beac- a sensibly functioning system, no other processes than thc cepted. Vor the receiving HOST with a small amount of one for which the packet was intended should be affectec buffer space, a strategy of discarding all packets whose by this discarding. If the delayed processing queue grow scqucncc numbers do not coincide with the currcnt left cdgc of the window is probably necessary, but itwill incur thc cxpcnsc of cxtra delay and overhead for retransmis- Thiscomponent canserve tohandle other protocols whoss associated control programs are designated by internetwork destina sion. tion address.

ACM SIGCOMM Computer Communication Review 78 Volume 35, Number 2, April 2005 CICRF AND KAHB: PACKET NETWORK INTERCOMMUKICATION 645 excessively long, any packets in it can be safely discarded The read and writepositions move circularly aroundthe since none of themhave yet been acknowledged. Con- transmit buffer, with the write position always to the left gestion at the TCPlevel is flexibly handled owing to the (module the buffer size) of the read position. robustretransmission andduplicate detection strategy. The next packet sequence number should be constrained to be less than or equal to the sum of the current ac- TCP/PROCESSCOMMUNICATION knowledgment and the window fields. In any event, the In order to send a message, a process sets up its text next sequence number should not exceed the sum of the in a buffer region in its own address space, inserts the current acknowledgment and half of the maximum possible requisite control information (described in the following sequence number(to avoid confusing the receiver’s list) in a transmit control block (TCB) and passes control duplicate detection algorithm). A possible buffer layout to the TCP. The exact form of a TCB is not specified is shown in.Fig. 12. here, but it might take the form of a passed pointer, a The RCBis substantially the same, except that theend pseudointerrupt, or variousother forms. To receive a read field is replaced by apartial segment check-sum message in its address space, a process sets up a receive register which permits the receiving TCP to compute and buffer,inserts the requisitecontrol information in a remember partial check sums in the event thata segment receive control block (RCB)and again passes control arrives in several packets. When the final packet of the to the TCP. segment arrives, the TCPcan verify the check sum and if In some simple systems, the buffer space may in fact successful, acknowledge the segment. be provided by the TCP. For simplicity. we assume that a ring buffer is used by each process, but other structures CONNECTIONSAND ASSOCIATIONS (e.g., buffer chaining) are not ruled out. Much of the thinkingabout process-to-process com- A possible format for the TCB is shown in Fig. 11. The municationin packet switched networks has been in- TCB containsinformation necessary to allow the TCP fluenced by the ubiquitous telephone system. The HOST to extract andsend the process data. Some of the informa- HOST protocol for the ARPANET deals explicitly with the tion might be implicitly known, but we are not concerned opening and closing of simplex connectionsbetween with that level of detail. The various fields in the TCB processes [9],[10]. Evidence has been presented that are described as follows. message-based “connection-free” protocols canbe con- 1) Source Address: This is the full net/HosT/TCP/port structed [12], and this leads us to carefully examine the address of the transmitter. notion of a connection. 2) Destination Address: This is the full net/HOST/ The term connection has -a wide variety of meanings. It TCP/port of the receiver. can refer to a physical or logical path between two en- 3) Next Packet Sequence Number: This is the sequence tities, it can refer to the flow ovcr the path, it can in, numbcrto be used for the nextpacket the TCP will ferentially refer to an action associated with the setting transmit, from this port. up of a path, or it can refer to an association between two 4) Current Buffer Size: This is the present size of the or moreentities, with or withoutregard to any path process transmit buffer. between them. In this paper, we do not explicitly reject 5) Next Write Position: This is the address of the next the term connection, since it is in such widespread use, position in the buffer at which the process can place new and does connotea meaningful relation, but consider data for transmission. it exclusively in the sense of an association between twoor 6) Next Read Position: This is the address at which the more entities without regardto a path. To be more precise TCP should begin reading to build the next segment for about our intent, we shall define the relationship between output. t\+o or more ports that are in communication, or are pre- 7) End Rewd Position: This is the address at which the pared to communicate tobe an association. Portsthat TCI’ should halt transmission. Initially 6) and 7) bound are associated with each other are called associates. the message which the process wishes to transmit. It is clear that for any communication to take place S) Number of Retransllzissions/ndnxill1u1tL Retransmis- between two processes, one must be able to address the sions: These fields enable the TCP to Beep track of the other. The two important cases here are that the deiti- numbcr of times it has retransmitted the data andcould be nation port may havea global and unchanging address or omitted if the TCP is not to give up. that it may globallybe uniquebut dynamically reassigned. 9) Timeout/Flwgs: Thetimeout fieldspecifies the While in either case the sender may have to learn the delay after which unacltnowledgcd data should be rctrans- destination address, given the destination name, only in mittcd. The flag ficld is uscd for semaphores and other the second instance is there a requirement for learningthe TCl’/proccss synchronization, status reporting, ctc. address from the destination (or its representative) each 10) Current 9cX:nozulerlg,trent/TYirLdo,w:The current time an association is desired. Only after the source has acltnowledgmcnt ficld identifies the first byte of data learned horn to address the destination can an association still unaclmo~vledgcd by thc destination TCP. be said to have occurred. But this is not yet sufficient. If

ACM SIGCOMM Computer Communication Review 79 Volume 35, Number 2, April 2005 646 COMMUNICATIONS, ON IEEE TRANSACTIONS MAY 1974 / . Current Message processes. This idea violates t.he premise that interprocess communica,tionshould not requirecentralized control. Sent. Acked Sent. Not Acked Not Sent Partial Next Message One would have to extend the central registry service to Current Ack Next Seq. No. t include all HOST’S in all the interconnected networks to iw,ndowNext’Read Endiead 1 Next Write apply this idea to our situation, and we therefore do not . Sire Transmit Buffet att’empt to adopt it. Fig. 12. Transmit buffer layout. Let us consider the situationfrom the standpointof the TCP. In order to send or receive data for a given port, the TCP needs to set up a TCB and RCB and initialize ordering of delivered messages is also desired, both the window size and left window edge for both. On thc TCP’smust maintain sufficient infornmtion to allow receive side, this task might even be delayed until the proper sequencing. When this information is also present firstpacket destined for a given port arrives. By con- at bothends, t,hen an association is said to have occurred. vention, the firstpacket should be marked so that tht Note that we have not said anything about a path, nor receiver will synchronize to thereceived sequence number anything which implies that either end be aware of the On the send side, the firstrequest to transmit coulc condition of theother. Only when bothpartners are cause a TCBto be setup with some initial sequenct prepared to communicate with’ each other has an associ- number(say, zero) andan assumed window size. Thc ation occurred, andit is possible that neitherpartner receiving ‘I’CP canreject the packet if it wishes anc may be able to verify that anassociation exists until some notify the sending TCP of the correct window size via thc data flows between them. acknowledgment mechanism, but only if either 1) we insist that thefirst packet bea complete segment CONNECTION-FREE PROTOCOLS WITH 2) an acknowledgment can be sent for the first packel ASSOCIATIONS (even if not a segment, as long as the acknowledg Inthe ARPANET, the interface message processors nlent specifies the next sequence number such tha (IMP’S) do not have to open and close connections from the source also understands thatno bytes have beer source to destination. The reasonfor this is that con- accepted). nections are, in effect, always open, since the addresk of It is apparent, therefore, that thesynchronizing of windov every source and destination is never5 reassigned. When size and left window edge can be accomplished withou the name and the place are static and unchanging, it is what would ordinarily’be called a connection setup. only necessary to label a packet with source and desti- The firstpacket referencing a newly created RCE nation to transmit it through the network.In our parlance, sent from ‘one associate to another can be marked with : every source and destination forms an association. bit which requests that the receiver synchronize his lef In thc casc of processes, however, we find that port window edge with the sequence number of the arrivint addresses are continually being used and reused. Some packet (see SYN bit in Fig. S) . The TCPcan examine thc ever-present processes could be assigned fixed addresses source and destination port addresses in the packet an( which do not change (e.g., the logger process). If we sup- in the RCB to decide whether to accept or ignore thc posed, however, that every TCP had an infinite supply of request. port addresses so that no old address would ever be reused, Provision should be made for a destination process tc then any dynamically created port would be assigned the specify that it is willing to LISTEN to a specific port o nextunused address. In such an environment,there “any” port. This last idea permits processes such as thl could never be any confusion by source and destination logger process to accept data arrivingfrom unspecifiec TCP as to theintended recipient or implied source of each sources. This is purely a HOST hatter, however. message, and all ports would bc associates. The initial packet may contain datawhich can be store( Unfortunately,,TCP’s (or moreproperly, operating or discarded by the destination, depending on the avail systems) tend not to have an infinite supply of internal ability of destination buffer spaceat thetime. In the othe port addresses.Thcse internal addresscs are reassigned direction, acknowledgment is returned for receipt of datr aft‘er the demise of each port. Walden [lZ] suggests that which also specifies the receiver’s window size. a set of uniqueuniform external port addresses could If the receiving TCP should want to reject the syn be supplied by a ccntral rcgistry. A newly created port chronization request, it merely transmits an acknowledg could apply to the central registry for an address which ment carrying a release (REL) bit (see Fig. 8) indicatini the central registry would guarantee to beunused by any that the destination port address is unknown or inacces HOST system in thc network. Each TCY could maintain sible. The sending HOST waits for the acknowledgmen tablcs matching external names with internal ones, and (after accepting or rejecting the synchronization request use the external ones for communicationwith other before sending the nestmessage or segment. This rejectiol is quite different from a negative data acknowledgment We do not have explicit negative acknowledgments. If nc 5 Unless the IMP is physicallymoved toanother site, or the HOST is connected to a different IMP. acknowledgment is returned, the sending HOST ma:

ACM SIGCOMM Computer Communication Review 80 Volume 35, Number 2, April 2005 CICRF AND KAHN:, PACKET 1\1

Conf..AFIPS Conf. Proc..,- vol. 36. Montvale.~I N. J.:~ AFIPS pairs at both ends which rendezvous with the exchanged Press: 1970, pp.~545-549. data and then disappear. If the overhead of creating and L. Pouzin,“Presentation and major designaspects of the CYCLADEScomputer network,” in Proc. 3rd Data Com- dcstroyingRCB’s and TCB’s is small, sucha protocol munications Symp., 1973. F. It. E. Dell, “Features of a proposed synchronous data net- work,” in Proc. 2nd Syrnp. Problems in the Optimization of Data S. Crocker of ARPA/IPT. Communications Systems, 1971, pp. 50-57.

ACM SIGCOMM Computer Communication Review 81 Volume 35, Number 2, April 2005 648 IEEE TRANSACTIONS ON COMMUNICATIONS, M9Y 1972

[41 R.. A. Scantleburyand P. T. Wilkinson, “The design of a Vinton G. Cerf wasborn in New Haver switching system to allow remote access to computer services Conn., in 1943. He did undergraduate wor by other computers and terminal devices,” in Proc. 2nd Symp. inmathematics at StanfordUniversit) Problc~nsin the Optimization of Data Communications Systems, 1971, pp. 160-167. Stanford, Calif., and received the Ph.D. de 11. L. A. Barber, “The European computer network project,” gree in computer science from the Universit, in ComputerCommunications: Impacts and Implications, ofCalifornia at Los Angeles, Los Angeler S. Winkler, Ed. Washington, D. C., 1972, pp. 192-200. Calif., in 1972. 13.. Despres, “A packet switching network wlth graceful satu- Hewas with IBM in LosAngeles fror ratedoperation,” in ComputerCommunications: Impacts and 1965 through 1967 andconsulted and/c Implications, S. Winkler, Ed. Washington, D. C., 1972. pp. worked part timeat UCLA from 1967 throug 345-3.51. 1972. Currently he is Assistant Professor c R. E. Kahn and W. I<. Crowther, “Flow control in a resource- Computer Science and Electrical Engineeringat Stanford Universit: sharingcomputer network,” IEEE Trans.Commun., vol. COM-20, pp. 539446, June 1972. and consultant to CabledataAssociates. Most of his current researc J. F. Chambon, M. Elie, J. Le Bihan, G. LeLann, and H. Zim- is supported by theDefense Advanced Research Projects Agency an merman,“Functional specification of transmissionstation in by the NationalScience Foundation on thetechnology and economic theCYCLADES network. ST-ST protocol” (in French), of computer networking. He is Chairman of IFIP TC6.1, an intel I.R.I.A. Tech. Rep. SCH502.3, May 1973. nationalnetwork workinggroup which is studyingthe probleI [9I 8. Carr, S. Crocker, and V. Cerf, “HOST-HOST Communica- of packet network interconnection. tion Protocol In the AItPA Network,” in Spring Joint Com- putcrConf., AFIPS Conf.Proc., vol. 36. Montvale,N. J.: AFIPS Press, 1970, pp. 589-597. * A. McKenzie, “HOST/HOST protocol for the AItPA network,” Robert E. Kahn (”65) was born inBrooklyr in CurrentNetwork Protocols, NetworkInformation Cen., N. Y., on December 23, 1938. He received th Menlo Park, Calif., NIC 8246, Jan. 1972. B.E.E. degree from the City College of Ne L. Pouzin, “Address format In Mitranet,” NIC 14497, INWG 20, Jan. 1973;< York,New York, in 1960, andthe M.P D.Walden, A system forinterprocess communication in a and Ph.D.degrees from PrincetonUniversit: resourcesharing computer network,” Commun.Ass. Comput. Princeton, N. J., in 1962 and 1964, r( Mach., vol. 1.5, pp. 221-230, Apr. 1972. spectively. B.Lampson, “A scheduling philosophy for multiprocessing From 1960 to 1962 he was a Member of th systems,” Commun. Ass. Comput. Mach., vol. 11, pp. 347-360, TechnicalStaff of BellTelephone Labor: May 1968. tories, Murray Hill, N. J., engaged in traff F. E. Heart, R. E.Kahn, S. Ornstein,W. Crowther, and andcommunication studies. From 1964 t D. Walden, “The interface messageprocessor for the ARPA 1966 he was a Ford Postdoctoral Fellow and an Assistant Professc computernetwork,” in Proc. SpringJoint Computer Conf., AFIPS Conf. Proc., vol. 36. Montvale, N. J.: AFIPS Press, of Electrical Engineering at the Massachusetts Institute of Ted 1970, pp. -551-567. nology,Cambridge, where he worked on communications and ir N. G. Anslow and J. Hanscoff, “Implementation of inter- formation theory. From 1966 to 1972 he was a Senior Scientist 2 nationaldata exchangenetworks,” in ComputerCommunica- BoltBeranek and Newman, Inc., Cambridge, Mass., where l- tions: Imnacts and Implications. S. Winkler, Ed. Washington, worked on computer communications networkdesign and techniquc 11. c., 1672, pp. 181-is4 for distributed computation. Since 1972 he has been with the Ac A. McKenzie, “HosT/HosT protocoldesign considerations,” vancedResearch Projects Agency, Department of Defensl INWG Note 16, NIC 13879, Jan. 1973. Arlington, Va. R. E. ehn, “Resource-sharingcomputer communication networks, Proc. IEEE, vol. 60, pp. 1397:1407, Nov. 1972. Dr. Kahnis a member of Tau Beta Pi, Sigma Xi, Eta Kappa NI Bolt,Beranek, and Newman, “Specificatlon for the intercon- theInstitute of MathematicalStatistics, and the Mathematic: nection of a. host and an IMP,” Bolt Beranek and Newman, Association of America. He wasselected to serve as a Nation: Inc., Cambrldge, Mass., BBN Rep. 1822 (revised), Apr. 1973. Lecturer for the Association for Computing Machinery in 1972.

ACM SIGCOMM Computer Communication Review 82 Volume 35, Number 2, April 2005