The DARPA Packet Radio Network Protocols

JOHN JUBIN AND JANET D. TORNOW, ASSOCIATE, IEEE

Invited Paper

In this paper we describe the current state of the DARPA packet scribe the algorithms used to route a packet through the radio network. Fully automated algorithms and protocols to orga- packet radio communications subnet.In Section V, we ex- nize, control, maintain, and move traffic through the packet radio network have been designed, implemented, and tested. By means amine the protocols for transmittingpackets. In Section VI, of protocols, networks of about 50 packet radios with some degree wedescribesomeofthehardwarecapabilitiesofthepacket of nodal mobility can be organized and maintained under a fully radio that strongly influence the design and characteristics distributed mode of control. We have described the algorithms and of the PRNET protocols. We conclude by looking brieflyat illustrated how the PRNET provides highly reliable network trans- some applications of packet radio networks and by sum- port and datagram service, by dynamically determining optimal routes, effectively controlling congestion, and fairly allocating the marizing the state of the current technology. channel in the face of changing link conditions, mobility, and varying traffic loads. II. DESCRIPTIONOF THE PACKET RADIO SYSTEM I. INTRODUCrlON A. Broadcast Radio In 1973, the Defense AdvancedResearch Projects Agency (DARPA) initiated research on the feasibilityof usingpacket- The PRNET provides, via a common radio channel, the switched, store-and-forward radio communicationsto pro- exchange of data between computers that are geographi- vide reliable computer communications [I].This devel- cally separated. As a communications medium, broadcast opment was motivated by the need to provide computer radio (as opposed to wires and antennadirected radio) pro- network access to mobilehosts and terminals, andto provides important advantages to theuser of the network. One vide computer communications in a mobile environment. of the benefits is mobility; a packet radio(PR) can operate Packet radio networking offers a highly efficient way of while in motion. Second, the network can be installed or using a multiple-access channel, particularly with bursty deployed quickly; there are no wires to set up. A thirdad- traffic [2]. The DARPA Packet Radio Network (PRNET) has vantage is the ease of reconfiguration and redeployment. evolved through the years to be a robust, reliable, opera- The PRNET protocols take advantage of broadcasting and tional experimental network[3]. The development process common-channel properties to allow the PRNET to be ex- has been of an incremental, evolutionary nature [4]; as al- panded or contracted automatically and dynamically. A gorithms were designed and implemented, new versions group of packet radios leavingthe original area simply de- ofthe PRNET with increasedcapabilities were demon- parts. Having done so, it can function as an autonomous strated. The PRNET has been in daily operation for exper- group and may later rejoin the original network or joinan- imental purposes for nearly tenyears. In this paper wede- other group. scribe the currentstate of the DARPA PRNET. The broadcasting and common channel propertiesof ra- We begin by providing asynopsis of the PRNET system dio have disadvantages too. These properties, for all prac- concepts, attributes, and physical components in Section tical purposes, prohibit the building ofa radio that is able 11. In Section Ill, we illustrate the mechanisms by which a to transmit and receive at the same time. Therefore, the packet radio automaticallykeeps track of a potentially con- PRNET protocols must attemptto schedule each transmis- tinuouslychanging network topology. InSection IV, we de- sion when the intendedPR is not itself transmitting. Also, transmissions often reach unintended PRs and interfere Manuscript received February l, 1986; revised July 30,1986.The with intended receptions. Therefore, the protocols must work of j. jubin was supported by the Defense Advanced Research attempt to schedule each transmission when the intended Projects Agency of the Department of Defense under Contract PR is not receiving another PR’s transmission. MDA903-85-C4205. Thework of J. D. Tornow was supported by the Defense Advanced Research Projects Agency of the Department of Defense under Contract MDA90MSC4254. B. Automated Network Management J.Jubin is with Collins Defense Communications, Rockwell In- ternational, Richardson, TX 75081, USA. The PRNET features fully automated network manage- j. D. Tornow is with SRI International, Menlo Park, CA 94025, ment. It is self-configuring upon network initialization,re- USA. configures upon gain or loss of packet radios, and has dy-

0018-9219/87/01ooM)21501.000 1987 IEEE

PROCEEDINGS OF THE IEEE, VOL. 75, NO. 1, JANUARY 1987 21 namic routing. The network operator simplyhas to ensure tion. The digital subsystem controls the routing and flow that the packet radios are deployed so that every packet of packets between PRs while the RF subsystem transmits radio is situated so as to have at leastone otherpacket radio and receives packets over the radio channel. The PR re- within line-of-sight, and then turn theradios on; the packet ceives packetsof data either fromits wire interface or from radio is intended to operate unattended. Once installed, the radio channel. Each PR is responsible for receiving a the system discovers the radio connectivity betweenpacket packet and relaying it on toa PR that is one hopcloser to radios and organizesrouting strategies dynamically on the the final destination. The packets can be routed eitherto basis of this connectivity. After initialization, most com- another PR over the radio channel oran to attached device munication networks maintaina static topology. A unique (i.e., host computer or terminal) viathe wireinterface. The feature of the PRNET is the ease with which network to- protocols that runin the PRs encompass the Physical, Data pology can be altered without affecting user‘sthe ability to Link, and Network layers, asdefined in ISO’s Open Systems communicate. Although RF connectivity is difficult to pre- Interconnection Reference Model 161. dictandmayabruptlychangeinunexpectedwaysasmobile It is the local broadcast nature of the packet radio that packet radios move about, the automated network man- givesthePRNETitsuniquenetworkingcharacteristic:aPR’s agement procedures usedin the PRNET are capableof sens- transmission is received by allPRs within line-of-sight; e.g., ingtheexistingconnectivityin realtimeandthenexploiting in Fig. 2, a connecting line indicates which PRs are within this connectivity in order to continuously transport data line-of-sight of each other. Since PRs L, N, and Q are all and control packets, all in a way that is totally transparent within line-of-sight of M, a transmission by M can be re- to the users. ceived by allof these PRs; they aresaid to be one hopaway from M. Note, however, that a transmission by, say, PR P C. Network Components can be received only by PRs L and N. In general, a PRNET consists of many PRs that are not all within line-of-sight of The PRNET system comprises: each other, and packets must traverse multiple hops to The PRNET subnet, which consists of the packet radios. reach their destination. The PRNET subnet providesthe means of interconnecting a community of users. The collection of devices (host computers and terminals), each attached to a packet radio via awire high-level data link control (HDLC) interface, that wish to exchange data in real time.

The primary component of the packet radio communication network system is the packet radio. The generation Fig. 2. Small packet radio network. of PR that is supporting the currentresearch is designed to supportthe development and evaluation of advanced packet networking concepts and techniques. This packet In order fora user to send data across a PRNET, a device radio equipmenthas been designatedthe LowcostPacket (such as a small host computer) must be connected to a Radio (LPR) [5], Fig. 1. The LPR consists of both digital and packet radio viaan HDLC wire interface. Since the DARPA RF subsystems. It is capable of omnidirectional, spread- PRNET is a part of theDARPA Experimental Internet System spectrum, half-duplex transmissionlreception at 40@ and [7l, the devices are responsible for running theDoD-stan- IWkbit/s data rates. It supports pseudo-noise code mod- dard internetwork-, transport-, and application-level pro- ulation of the information bits and forward error correc- tocols (IP, TCP, and TELNET). These protocols ensure that the end-to-end communication between hosts is reliable and robust, and allowhosts on thePRNET to communicate with computers on various other packet-switched satellite, terrestrial, radio, and local area networks that also partic- ipate in the DARPA Internet. A host computer may be di- rectly interfacedto a PR. If a user wishes to send data across the PRNET from a terminal or host that does not run the required protocols,a Network Interface Unit (NIU)[8], Fig. 3, may be used between the terminal or host and the PR.

Fig. 1. Low-cost Packet Radio. Fig. 3. NetworkInterface Unit.

22 PROCEEDINGS OF THE IEEE, VOL. 75, NO. 1, JANUARY 1987 The NIU performs thenecessary host-to-host and terminal- congested neighborhoods. However, performanceis likely specific protocols. The PRNET can also be accessed from to suffer when “actual” neighborhoodsare large. Enhance- other networks via an Internet gateway. In addition tosup- ments to the protocols which will prevent degradation of porting data, the PRNET protocols can also support a lim- performance in large neighborhoods (such as adaptively ited amount of digitizedspeech. Fig.4 illustrates thePRNET setting dynamic link-level transmission parameters)is an- with user devices as a part of the Internet. Note that, or- other subject of the DARPA PRNET program’s future re- ganizationally, the devices lie outside the PRNET subnet: search. the network appears as a black box providing packet communication service between pairs of user devices. E. Loading, Debugging, and Monitoring The protocol software and replacements (“patches”) to operating-system firmwarein thePR can be loadedinto the PR from a terminal via an RS-232 interface. A plethora of terminal commands-display memory, alter memory, select/display trace breakpoint, display state, execute jobs, execute procedure,etc.-are available for local debugging. Protocolsoftware and firmware patches canalso be loaded from an already loaded and running neighbor PR via the radio channel. New versions of softwarewill be automatically distributed thoughout thePRNET via neighbor loading. HOST PRs can alsobe loaded and debugged from a remote host computer. All the commands available to a local operator are available to a remote operator.The remote operatorcan load/debug not only PRa that is executing the regular protocol software, but also a”frozen” PR, that is, a PR that has stopped executing the regular programbecause of a self- detected fault. Anotherhost computer function called the Network Monitor is used toaid inobserving and analyzingthePRNET. @gj Each PR continuously gathers measurements on bidirec- Fig. 4. Packet radio network in the internet. tional linkquality, nodal capacity,and routecharacteristics. The Network Monitor will collect data from each of the packet radios and display them graphically to aid the net- D. Network Size work designers in characterizing and understanding the The current network protocols are designed to support network behavior. The Network Monitor is currently defined as an optional component of the networkbecause it 138 entities, which can be any combination of PRs and at- does not contribute inany way to the management of the tached devices, in asingle network. For example, a typical network may be composed of 50 packet radios, many of network. which have attached devices andsome of whichserve only as repeaters. Multiple packet radio networks operatingon 111. LINK CONNECTIVITYAND ROUTE CALCULATION different frequencies may be interconnected via Internet The PRNET features fully distributed network manage- gateways to support applications which require larger net- ment [IO]. Each packet radio gathers and maintains enough works. Research is currently being conducted to designal- information about network topology so that it can make gorithms which allow a single network to organize and independent decisions about how to route data through manage thousands of packet radios without Internetgate- the network to any destination, even before it is given a ways.An accompanying paperin this issue[9]describes this packet to deliver or forward. The network information is and other future research of the DARPA PRNET program stored in three tables: more completely. AsthenumberofPR~increases~thenumberofneighbors neighbortable (those packet radiosin direct line-of-sight, describedSec- in tiertable tion Ill) per PR is likely to increase. The current protocols device table. allow each packet radioto support up sixteen to neighbors. These tables are established by the packet radio upon in- If however, packet radiosare sited moredensely, creating itialization, andare updated automaticallybythe PRs as the neighborhoods of more thansixteen radios, the congested topology changes. In the following paragraphs, we de- PRs employ an algorithmto determine their “logical” scribe howthetablesareestablished, maintained,and used. neighborhood, the effect of whichis to limit tosixteen the number of “actual” neighbors that congested PRs track. A. Neighbor Table The primary decision criterion of this algorithm is to ex- clude the prospective neighborwith themost neighborsin After a PR (say PR L in Fig. 2) has been powered on, and common. The algorithm is careful to preventisolated has loaded its protocol softwareinto RAM, it begins the pro- cliques, that is, radios whichare not ableto communicate cess of establishing and maintaining local connectivity.As with the rest of the network. Therefore, the network dy- long as the PR is powered up, it will broadcast a Packet Ra- namically accommodatesnetwork topologies that result in dio Organization Packet (PROP) every 7.5 s, announcing its

IUBIN AND TORNOW DARPAPACKET RADIO NETWORKPROTOCOLS 23 existenceand information aboutthe networktopologyfrom spect to PR f,and that it is at tier 2 with respect to PR L via its perspective. However, when PR L is first powered up, it PR P. Thus tier information ripples outward from each PR does not know anything about the networkso its PROP sim- at an average rate of 3.75 s per hop, eventually reachingall ply announces its existence. The PROP is received by all the PRs in atiered ringfashion. In thisway, ultimatelyevery neighboring radios (those in directconnectivitywithL). The radio knows its distance in tiers fromany prospective des- set of radios that hear the PROP note the event in their tination PR and which PR is the nextPR enroute to thatdes- neighbortables. Similarly, PRLwill hearthe periodicPROPs tination PR.This information is maintained continuously by of its neighboring PRs and willstart to buildits own neigh- each PR in its tier table, as shown in Fig. 5. bor table. OncePR L begins participating in the network by passing data, the data packets are used in addition to the PROPs to maintain the neighbor table. In addition to maintaininga list of the PRs that are one hop away, the neighbor table tracks the bidirectional qual- ityofthelinkto/fromthoseneighbors.EachPRkeepsarun- ning count of the numberits of transmissions and includes this count in its PROP. This value is used by the receiving PRs to evaluate the quality of the link. The link quality is measured in terms of the ratio of number ofpackets cor- Fig. 5. Typical tier table for PR N. rectly received from the transmittingPR during aPROP interval to the number of packets that the transmittingPR actually transmitted during that same interval. For example, The goal of the tier tableis always to maintain the “best” if PR L transmits 80 packets in a 7.5-s period and PR M re- information about howto get to a destination packet radio. ceives 50 of them correctly, the L-to-Mlink qualityis 5 (just The ”best” route is currently defined as the shortest route high enough for a”good“ rating). Once the calculationhas with good connectivity oneach hop. Therefore, the tierta- been completed, the receivingPR zeros out the count of ble is updated onlywhen certain conditionsare met. Let us the number of packets received from the transmitting PR consider that N has received aPROP from M, and examine and stores the running countof transmissions by the trans- the conditions for whichN updates its tier table, where the mitting PR, so that if a PROP is missed the calculation will update for any given destination PR consists of storing: still be valid. Continuing theexample, once PR M has de- as the next PR enroute: M termined that the link qualityis $,it willset the number of * as the tier: M’s reported tier, with respect to the des- packets received fromL to zero, and store ’80’ as the last tination PR, incremented by one (referredto as ‘T’ be- count of packets transmitted by L. Then, say PR M receives low). another PROPfrom PRLwhich indicatesthatPRL’s running transmission count is now 180. PR M subtracts 80 from 180 A necessary condition is that M be a bidirectionally good to get the transmission count since the last computation. neighbor of N. In addition to the good neighborliness,any If PR M has received 90 packets since the last computation, oneofthefollowingconditionsmustbemetforNtoupdate the new L-to-M link quality is A. its tier table: Quality values are smoothed, and hysteresis is applied There is no stored tier data for that destination PR. when switching between good and bad ratings. Separate The prospective new tier (‘T’) is strictly less than the values aremaintained for the400- and 100-kbitk datarates. stored one. This neighbor table, then, represents a PR’s accumulated M is the same PR that is already stored as the next PR. knowledge about all thePRs that are one hopaway. If a PR moves, its new connectivity is dynamically determined by When the link quality to a neighboringPR (say the link the same mechanism. from PR N to PR M) becomes “bad,” all routes in N‘s tier table forwhich the neighborPR M is the nextPR in the route are also set bad. This means that M can no longer provide €3. Tier Table a reliableway for N to send packetsto a destinationPR, and Routing in thePRNET is done by havingeachPR maintain anewnextPRshould bechosen,andth,usanew,good route knowledge of thebest PR to forwardpackets to every pro- can be formed even if it is longer than the old, bad one. Just spective destination. This is similar to the early ARPANET as good tier dataare spreadvia PROPs, the news of badtier routing algorithm [Ill. Each PR that receives PR L’s PROP data is also spread via PROPs. PR N will include thenews notes that it is only one radio hopaway, so it is at “tier 1” in its PROP that it can no longer providea route to thedes- with respect to PR L, and includes this information about tination PRs that it reached via M. It is important tospread the network topology inits next PROP. Still other PRs hear the bad news in an orderly fashion. For example, suppose the PROPs broadcast by thePRs which are at tier 1with re- PR M is the next PR with respect to destination PR L in PR spect to L, note that they are one hop away from the PRs N’s table. Assumethat M no longer has connectivity with from which thePROP was received, and therefore at tier 2 L, so it marks its tier data bad and broadcasts this infor- with respect to PR L (which emitted the original PROP). mation in its next PROP. If N does not receive the PROP for These second-tier PRs, in turn, relay this information out some reason,then Nwill continue to announce itsin PROP another tier in the PROPs, and so on. For example, in Fig. that it has tier data to L, and M will assume that it can get 2, PRs P, MI and Q learn that theyare at tier 1 with respect to L via N. Mechanisms exist in theprotocols to break these to PR L after receiving PR L’s PROP. The next time thatPRs routeloops and to ensurethat obsolete data do not P, MI and Q broadcast their PROPs, they include the in- override the up-to-datedata. formation that theyare at tier 1 with respect to L. When PR Information gathered from the headers of data packets N receives PR f’s PROP, it knows that it is at tier 1with re- is also used to update the tier tables. Under heavy traffic

24 PROCEEDINGS OF THEIEEE, VOL. 75, NO. 1,JANUARY 1987 loads, this greatly improves the network’sresponsiveness to its destination because multiple-hop flooding woulduse to a rapidly changing topology, and contributes signifi- too much of the finitecapacity of the network’s common cantly to thePRNET’s automatic reconfigurationcapability. channel. Instead, generally speaking, a packet traverses a single path through the network, and is acknowledged at C. Device Table every packet radio along the path (Section IV-B). Multiple- path forwarding may occur only ifalternate routing has to The PRNET provides a dynamic addressing capability. The be employed (Section IV-D). Forwarding is accomplished DARPA packet radio network’s device-to-PR mapping is via information read from the device and tiertables (Sec- known as logical addressing. With logical addressing, the tions Ill-B and Ill-C)and from the packet headers (Section relationship of a deviceto a PR may be alteredas necessary. IV-A). Forward error correctionincreases the probabilityof So that all packet radios can become aware of all the at- error-freereception, and cyclic redundancy checksums tached devices connected to the subnet, each such device prevent, with high probability, the forwarding of packets periodically sends acontrol packet across thewire interface that are not error-free (Section IV-C). to its attached PR. The PR notes the existence of the attached device and includesit in its PROP to bepropagated to the otherradios in the networkat an averagerate of3.75 A. Packet Headers s per hop.Device-to-PRcorrespondence information isalso Every packet transmitted by every PR contains several learned from headers of data packets.Through these mech- headers, which add about10 percent to thepacket length. anisms, ultimately allpacket radiosknow ofall devices and Each header corresponds to a protocol layer. Strict layering to whichpacket radiosthe devices are attached. If a device produces a clean, structured design, but also causes the is moved from onePR to another, or ifit goes down, its new headers to be longer than they could be because of du- status is made known to the network in thesame manner. plication ofsome fields and fragmentation of others among This device-to-PRcorrespondence informationis stored by the headers. Thepacket headers that are of concern to this each packet radio in a device table. paper are the end-to-end (ETE) headerand the routing In Fig. 6, Device 1 launches a packet destinedfor distant header. Device 2. The packet is sent across the wire interfaceto PR ET€ Header: The ETE header is created by the source device. It contains thesource device ID, which is used to update the PRs’ stored device-PR correspondence data (Sec- tion Ill-C), and the destination device ID, which is used in forwarding (Section IV-B). Also in theETE header is a type- of-service flag that the source PR transfers to the routing headerto customize forwarding forlow delay/reliabilityap- plications such as speech (Section V-D).TheETE header stays Fig. 6. Packet radio network with attached devices. onthe packet from its creationby the source device throughout its forwarding through thePRNET including its delivery to thedestination device. L, which uses its device table to map Device-id 2 to its at- Routing Header: The routing header is created by the tached PR (N).The device-to-PR mapping is totally trans- source PR, encapsulating the ETE header. Its fields that are parent to theuser and to thedevice. PR L then uses its tier of concern to this paper are: tabletoinitiatetheroutingofthepacketthroughthepacket radio communications subnet towardsPR N. N‘s device table has Device 2 stored as an attached device, so once the Protocol packet has arrived at N, it is passed over the wire interface Routing Function Section Header Field Used In Described In to Device 2. If the Device2 is no longer attachedto N when the packet arrives, the packet will be discarded by N. It Source PR ID acknowledgment IV-B IV-D would bea violation of the layering architectureto recom- alternate routing Sequence acknowledgment IV-B pute theaddress of the destination end device while rout- number alternate routing IV-D ing at the subnet layer. However, the PRNET’s logical ad- Speech type-of- dressing facilitates the rapid dissemination of Device 2’s service flag transmission V- D new address. Consequently, when theend-to-end reliable Previous PR ID acknowledgment IV-B Previous PR‘s protocol, TCP, initiates a retransmissionof the packet (hav- transmit count pacing V-A ing failed to receive an acknowledgment), the network is Transmitting PR likely to beable to route thepacket to the newaddress of ID acknowledgment IV-B Device 2. Transmitting PR’s V-A In addition, thePRNET offers generic logicaladdressing. transmit count pacing Next PR ID forwarding IV-B Generic addressing allows a deviceto request service from acknowledgment IV-B any other device in ageneral category, e.g., any name server pacing V-A or any gateway. The packet is addressed to an ID that may Lateral alternate belong to more than onedevice. The network will deliver routing flag alternate routing IV-D Alternate routing the packet to theclosest device with therequested generic request flag alternate routing IV-D ID. Tier alternate routing IV-D Destination PR ID forwarding IV-B IV. FORWARDING PROTOCOLS alternate routing IV-D Despite the local broadcast nature of the packet radio, a user packet is not flooded throughout thePRNET to get The routing header stays on thepacket throughout its for-

JUBlN AND TORNOW DARPA PACKET RADIO NETWORK PROTOCOLS 25 warding through thePRNET subnet. The sourcePR ID, se- Other PRs within range also receive each transmitted quencenumber, and destination PR ID createdby the packet. For example, L's transmission is received not only source PR stay fixed throughout thepacket's journey to the by M but also by PandQ. Since neither P nor Q is the next destination PR. The rest of the fieldsare updated byevery PR in the routingheader, they bothdiscard the packet. The intermediate packet radio. The routingheader is stripped packet transmitted by M is also received by L and Q; it is off the packet when it is delivered by the destinationPR to discarded by Q and processed by L as a passive acknowl- the destination device. edgment as explained in the following section. Acknowledgment: Let's look at the same sequence of events from the perspective of just the packet transmis- B. Forwarding-Acknowledgment-Retransmission Protocol sions: Forwarding: Packets are forwarded over a single path Device 1 PR L PR M PR N Device 2 through the PRNET by each packet radio using the infor- wire mation in the packet's headers and in its own device and +----_- > tier tables. Each PR uses this information, first, to decide radio whether it should be the oneto transmit the packet on, sec- + ---_-_> ond, to update the routingheader before transmitting the radio packet on, and third, to update its own tables. <------+------> By example, let us enumerate the steps in fowarding a wire packetthroughthenetworkofFig.6,fromDevicel,through +------> PRs L, M, and N, to Device 2: radio < ---__-+ Device 1 Noticethat PR M's single radiotransmissionis received not Sets, in ETE header, destination device ID = 2. Wire-transmits the packet. only at the next PR,PR N, but also at the previous PR, PR PR L L.Thusthesingletransmissionnoton1yforwardsthepacket Receives packet. on to thenext PR, but also acknowledges to the previous Sets, in routing header: PR reception ofthe packet bythe transmittingPR. This kind of acknowledgment, which takes advantage of the broad- destination PR ID = N (from device table) cast nature ofpacket radio, is known as a passive acknowl- = 4 (signifies an attached previous PR ID edgment. Since the destination PR N wire-transmits the device) packet to forward it to thedestination Device2, PR N must transmitting PR ID = L also radio-transmit aseparateacknowledgmentto acknowl- next PR ID = M (fromtier table) edge reception of the packet to PR M. This kind of ac- tier = 2 tier(from table) knowledgment, which consists of only the routingheader Radio-transmits the packet. of the packet, is known as an active acknowledgment. PR M A packet that is received at a packet radio qualifiesas an Receives packet. acknowledgment if it meets two criteria: Determines, from next PR ID = M inrouting It is a duplicate of theoriginal, indicated by its source header, that it should process the packet. PR ID and sequence number matchingthose of the original Sets, in routing header: packet. previous PR ID = L It has made it farther downstream toward the desti- transmitting PR ID = M nation, indicated by one of several criteria, including: next PR ID = N (fromtier table) tier = 1 tier(from table) It, or perhapsan alternate-routed(Section IV-Dlcopy Radio-transmits the packet. from another PR, got to the intended next PR-in- PR N dicated by its transmitting PR ID being thesame as Receives packet. the original packet's next PR ID. Determines, from next PR ID = N inrouting It came from this PR-indicated by its previous PR header, that it should process the packet. ID being the same as this PR's ID. Determines, from destination PR ID = N in rout- It or a copy gotto a PR that at least thinks it is closer ing header, that packet is at the destinationPR. to the destination PR-indicated by its tier being Determines, from destination deviceID = 2 in ETE smaller than the originalpacket's. header and from device table, that packet Retransmission: If a PR, say L, that has forwarded a packet should bewire-transmitted. does not receive an acknowledgment within a certain in- Wire-transmits the packet. terval, it retransmits the packet. L transmits the packet as Sets, in routing header: many as six times waiting foran acknowledgment and then previous PR ID = M finally discardsthe packet (dependingon the transport-level transmitting PR ID = N protocol toend-to-end retransmit as is required to provide next PR ID =d the desired end-to-end reliability).The interval betweenre- tier =d transmissions is determined by the pacing protocol (Sec- Radio-transmits the routing header (done for tion V-A) and grows with each packet that is not acknowl- acknowledgment purposes, explained inthe edged bya given nextPR, say M. When the intervalbecomes next section). too large, L sets its connectivity rating with M as non- Device 2 existent,withoutwaitingforamuchlongertimetotimeout Receives packet. on theabsence of any packets from M.

26 PROCEEDINGS OF THE IEEE, VOL. 75, NO. 1, JANUARY 1987 C. Error Control also create duplicate packets in thenetwork, as shown in the example in the precedingsection, namely at PRs Pand Forward Error Correction: The LPR uses long convolu- Q. Such duplicate packets are not desirable because their tional codes and sequential decoding to perform forward forwarding wastes some of the finite capacity of the net- error correction(FEC). The current software employs a cod- work’s common channel. Therefore, receivedpackets that ing rateofone-half (with provision for futureenhancement otherwise would be forwarded but thatare recognized as to variable rate coding). Half-ratecoding doubles thetrans- duplicates are filtered out instead of being forwarded. mission time and sequential decoding increases the pro- Packets are uniquely identified for the purpose of du- cessing time significantly, but FEC often prevents retrans- plicate filtering in thesame way that they are identified to mitting(described in theprevious section), thus often qualify as acknowledgments (SectionIV-B), namely,via the preventing even more delay and processing in environ- packet header’ssource PR ID and sequence number. These ments with high bit error rates. two entities are together known as the uniquepacket iden- Cyclic Redundancy Checksum: The LPR also has hard- tifier (UPI). The UP1 of a newly received packet aboutto be ware to generatea 32-bit cyclicredundancy checksum queuedforforwardingiscomparedtotheUPlsof1)packets (CRC). This provides error detection with a very low un- that are currently queued for forwarding 2)and the last few detected errorrate. Received packetsthat fail theCRC test, packets that have been forwarded andacknowledged; any even if they have been sequentially decoded, are simply match causes the new, duplicate packet to be discarded. discarded andtherefore treatedas if they had notbeen re- An active acknowledgment is transmitted to prevent the ceived at all. The packet can be checksummed by sections transmitter of the duplicatepacket, that is the previousPR, with individual CRCs. The current software checksums a from retransmitting unnecessarily any more, in the follow- forwarded packet’s header and textseparately. This allows ing circumstances. An active acknowledgment is always a PR that receives the header correctly (afterFEC) to use it transmitted in the second case above, since the original as a passive acknowledgment and to update its neighbor packet has already been acknowledged and therefore will table and tier table, even if the text has uncorrectable er- no longer be (re)transmitted and thereby provide acknowl- rors. edgment to theprevious PR. For the same reason, if an acknowledgment to thequeued packetis received beforethe D. Alternate Routing packet has had a chance to be (re)transmitted in the first If a PR has not received an acknowledgment after three case above, anactive acknowledgmentis also transmitted. transmissions of a packet, it sets the alternate routing request flagin thepacket headerto request forwarding help V. TRANSMISSIONPROTOCOLS on thelast three retransmissions. In some respects, the last Section IV described how a PR decides that a received threeretransmissionsaretreated thesameasthefirstthree. packet should be forwarded and howit selects the next PR For example, they are all separated by the same retrans- to which thepacket should be forwarded. After those de- mission pacing interval.Also, if thedesired nextPR receives cisions are made,the packet is then put in the radio transmit one of thelast three transmissions, it forwards the packet queue accordingto a protocol which promotes”fairness” on as usual, even though thealternate routing request flag in the use of the PRNET resources (Section V-B). The time was set. However, a PR whose ID is not the next PR ID in at which it is selected to be transmitted is determined by the header also accepts such a packetfor forwarding ifits athree-component pacing protocol (Section V-A). The tier tableshows the tier to the distinationPR to be less than, transmission time may be further delayed slightly by the or equal to, the tier in theheader. carrier sense multiple access protocol (Section V-C). Trans- Returning to the example from Fig. 6 and Section IV-B, mission parameters are chosen accordingto themeasured let us say that PR P receives a packet transmitted by PR L link quality and to thetype-of-service desired by the user whose destination PR ID is N, whose tier is 2, and whose (Section V-D). next PR ID is M. Let us also say that the packet’s alternate routing request flag is set. In this case, P will accept the A. Pacing packet for forwarding, even thoughit is not M, because its stored tier to N is 1, which is less than the tier of 2 in the The pacing protocol [I21 provides flow and congestion packet header. PR Q will also accept the same packet for control while contributingto theefficient butfair use of the forwarding because its stored tierto N, 2, is the same as the radio channel. Thereare three interrelated componentsto packet header’s. To prevent a packet from beingalternate pacing: single-threading, measuring the forwarding delay routed at the same tier forever in a circle around thedes- through each neighboring PR, and applying a function of tination PR, the lateral alternate routingflag in theheader the measured delay to separate (re)transmissions to each is set when this “lateral” alternate routing is in progress, neighbor PR. prescribing that the next helper(assuming alternate rout- Single-Threading: Single-threading requires that apacket ing help is still needed) must be strictly closer to thedes- transmitted to a certain next PR be acknowledged (ordis- tination. carded after six transmissions) before another packet is transmitted to the same PR. Recall from Section IV-B that the acknowledgmentis generally provided passively by the E. Duplicate Filtering same transmission that forwardsthe packet, the effect being An unnecessary retransmission occurswhen the nextPR that the acknowledgmentis not transmitted until thePR is is slow in forwarding or when the transmittingPR does not ready to accept another packet from thesame previous PR. successfully receive the next PR’s acknowledgment. When- (The special kind ofacknowledgment, theactive acknowl- ever an unnecessaryretransmission occurs, the next PR may edgment, is not transmitted untilafter the packet has been receive a duplicate of a packet it has already queued for forwarded in all cases.) The net result is that congestion is forwarding or hasalreadyforwarded. Alternate routingmay deflected as much as possible away from a network bot-

JUBIN AND TORNOW: DARPA PACKET RADIO NETWORK PROTOCOLS 27 tleneck back to the source PR. In addition, there is a limit In recognition of this free-frame nature, the pacing pro- of two buffers allotted perPR for packets received overthe tocolapplies a multiplier of three to the measured/ wire interface for entry into the PRNET. This reflects the smoothed forwarding delay, “D,” to separate transmis- congestioncompletely out of the subnet. Thus single- sions of different packets and retransmissions of thesame threading providesa form of flow controlas well as conges- packet to that neighbor PR. Note in Fig. 7 that PR L waits tion control. to transmit packet 2 until three times the D measured Fonvarding Delay: To get an indication of how muchde- through PR M has elapsed since it (L) transmitted packet 1. lay should be applied between packet (re)transmissions, This same conservative algorithm is applied in thecase of each PR measures the forwardingdelay of each packet that Wkbit/s transmissions, even though the actual transmis- it forwards througheach neighbor PR. A PR, say L, measures sion time is a less significant part of the totalD in thecase forwarding delay by recording the timeat which its trans- of Wkbit/s transmissions. For the case of packets justen- mission completes and then subtractingthat recorded time tering the PRNET from an attached device, the separation from the time at which the reception of the acknowledg- between transmissions is a more complicated function of ment from thenext PR, say M, of the original packet com- D: it is slightly greater than the three timesD (the formula pletes. (The propagation timeis negligiblecompared to the for radio-received packets) for small values of D, and then transmission time.) If L had to retransmit the packet, it increases proportional to the square of D. The purpose is knows from which of its retransmissions to measure (ac- to increase the choking effect rapidlyas the perceived net- tually, estimate) the time of transmission,because the ac- work congestion increases. knowledgment’s header contains the previous PR’s (L’s) In summary, pacing adapts PRs’times of transmissionto transmit count, which is a record of the transmitting PR’s provide congestion control in theface of a bottleneckPR, (L’s)transmit count when the packetwas actually received but on the other hand, uses the channel efficiently when by M. If noacknowledgmentis received,theentiretimethat there is no bottleneck. a PR waited for the acknowledgment since the first transmission of the packetis used as the forwardingdelay mea- B. “Fairness” Queuing surement. As a general rule, packets are handled first-in first-out The forwarding delay encompasses all the processing, (FIFO). However, thereare two exceptionsto the FIFO rule. queuing, carrier-sensing/randomization (described in Sec- The first exception is a result of the pacing protocol de- tion V-C), and transmittingdelays which a neighborPR ex- scribed in the preceding section.Pacing can slow the rate periencesfromallitsownactivityandfromtheactivitygoing of forwarding to one neighborPR compared to the rate to on in its neighborhood before it completes the transmis- another neighbor PR. Therefore, a packet j headed in a sion of the acknowledgment. Forwarding delay measure- “fast” direction and therefore ready to go, in the pacing ments are exponentially smoothed and stored on a per- sense, is transmitted before one i headed in a “slow” di- neighbor PR basis. Smoothingthe measurements maintains rection and therefore notready to go, even if iarrived be- ashort-term historyofthedelaythrougheach neighborPR, forej. There is, however, a single radio transmit queue,so which is presumed to be valid for the next transmission/ac- that, when several packets are ready to go at the same time, knowledgment cycle. Furthermore, when coupledwith sin- they will be transmitted FIFO, that is, i before j. gle-threading, the forwardingdelay ofthe highest delayPR The second exception to the FIFO rule is that received along a forwarding pathis reflected backto thesource PR. packets are not always entered at the tail of the radio trans- facing Function: Consider the worst case scenario for mit queue. The purpose of this exceptionis to overcome radio channeluse: a sequence of maximum-length packets the inherent unfairness that may exist among various re- transmitted at the slower IWkbit/s data rate with half-rate ceive links. Consider in Fig. 8 three streams of packets FEC. For this scenario, the processing delay is small com- merging at one PR (X), all going to the same neighbor PR pared to the transmissiodreception delay. Furthermore (V):one stream is coming over a weak radio link from PR (returning to the example from Section IV-61, the trans- A, another is coming over a strong radio linkPR and the mitting PR, must allow time for the nextPR, M, not only B, L, last is coming over the practically error-free wire interface to receive L‘s transmission andto forward it on butalso to from device C. With straight FIFO queuing, the wire inter- receive the acknowledgment from its (M’s) next PR, N. In other words, the packetradio’s forwarding cyclehas three frames; no PR, generally speaking,can transmit more than one-third of the time. The three-frame nature of the forwarding cycle is illustrated in Fig. 7.

TIME RADIO -PRr - 0 WIRE ARWARDS DEVICE C FoRWARDS PACKET 1 N M RECEIVES PACKET 1 Fig. 8. LPR with variablestrength receive links.

face stream would monopolize PR X’s buffer and transmission resources, and the weak radiolink would notget its share. Furthermore, the pacing protocolwould provide positive feedback to furtherexacerbate the situation.

28 PROCEEDINGS OF THE IEEE, VOL. 75, NO. 1, JANUARY 1987 When faced with a monopolizing receive link that has rcliviiis a. more than one of its packets destined for thesame neigh- PRO -1 bor PR already in the radio transmit queue, the “fairness” algorithm takes a packet thatarrives over a differentreceive /ki nceFlion mmh- PRL &-?lrsnynigicn link, destined for thesame neighbor PR, and.places it in the ~ ------‘D - queue ahead of thesecond packetfrom the monopolizing link. Two examples of this “butting in line” in the interest l=l IT1 PR M of fairness are shown in the following scenario based on 1- [-...I Fig. 8. (The “butting” packets are packets 5 and 6.) 1 /-*kyimernl- Packet arrival event at PR X ‘=I PRX’s transmit queue (all packets destined for PR Y) < -headtail- > lina- *e: 0-L 1A -pnpagaimbineImmPROtoPRL Packet 1 arrives from PR A 04 -pnfsgaianbinefmrnPROloPRM Packet 2 arrives from PR B IA, 2B L-M I popapatien bine tmrn PR L lo PR M Packet 3 arrives from device C IA, 2B, 3C deleccion - bl-~DrCirrlodcm&bl lh Packet 4 arrives from device C IA, ZB, 3C, 4C Fig. 9. CSMA delay interval. Packet 5 arrives from PR B IA, 2B, 3C, 5B, 4C Packet 6 arrives from PR A IA, ZB, 3C, 6A, 5B,4C The CSMArandomization delay protocolto decrease the C. Carrier Sense Multiple Access probability of overlapping transmissions, then, is as follows: The pacing protocol described inSection V-A prescribes intertransmission delays that foster noninterfering trans- 1) If the channel is busy: mission patterns. However, irregularly spaced user traffic a) monitor the channel until it goes idle; is bound to cause more than one PR to schedule trans- b) select a random number anddelay for that num- missions which overlapin time. One problem withtwo PRs ber of the delay intervals defined above; transmitting at the same time is that, if their transmissions c) if the channelis (still oragain) busy, go backto step are intended foreach other, neither will be received, since a). the packet radiois half-duplex. Another problemwith over- Otherwise.. . lapping transmissions is that they could interfere, or col- 2) Transmit the packet. lide, with each other at the PR(s) that can receive both trans- If two or morePRs are unfortunate enough toselect the missions. same smallest number in step Ib), there will be simulta- The carrier sense multiple access (CSMA) protocol at- neoustransmissions.Themorelikelyoccurrenceisthaton1y tempts to prevent a PR from transmittingat the same time one PR selects the smallest number. That PR, say L, will a neighborPR is transmitting [13]. A PR can tell if a neighbor transmit, and the rest of the PRs will wait long enough to PR is transmitting by reading its hardware indication “bit- be ableto sense that the channelis busy again. Those PRs synchronization-in-lock.” This condition is also described that are in connectivity withL will be able to carrier-sense by thephrases “the channel is busy” and“a carrier is being and refrain from transmitting.Those PRs that are not incon- sensed.” Whenever a carrier is being sensed, a PR will re- nectivity with L, also known as “hidden PRs,” will not be frain from transmitting. able to sense L’s transmission and may themselves also Even though PRs can carrier-sense, there is still a latent transmit;however, the randomization delaying will de- problem if more than onePR, say PR L and PR M, have trans- crease the degree of overlap of the transmissions, thus missions being held off by carrier-sensing the transmissionhelping the “capture” effect (SectionVI-B) to allow the first of anotherPR, say PR Q. Let us saythat L decides to transmit transmission to succeed. The range from which the random as soon as it senses the reception from Q complete. The number is selected increases with each retransmission that problem is that there is a finite delay from the time when fails to elicit an acknowledgment, to further decrease the M senses the channel idle afterits reception from Q com- probability of collision and to furtherincrease the proba- pletes until the time when it firstsenses the channel busy bility of capture. from the transmission from L; if M believed the bit-syn- Notice that noCSMA randomization delayat all is applied chronization-in-lock indication during this interval, it wouldif the channelis initially sensed idle. This is exactly what is end up overlapping transmissionswith L. The components desired for the most frequently encountered case of the of this delay intervalare listed below and their relationship first transmission ofa packet bya given PR. If aCSMA delay is shown in Fig. 9: were applied in this case, it might workagainst the pacing protocol attempting to determine the appropriate time to the time from when the reception fromQ completes transmit and actually cause PRs’ transmissions to unne- at M until the same reception completes at due to L, cessarily overlap. However, there are two cases when an the difference in propagation time fromQ to M com- initial CSMA randomization delayis applied no matterwhat pared to that from Q to L; the initial state of the channel is: L‘s receive-to-transmit switchover time; the propagation time from when L starts the actual when a PR accepts for forwarding a packet that was transmitting until M starts receiving; requesting alternate routing-to avoid several PRs re- M‘s bit-synchronization-in-lock acquisition time, also sponding simultaneously to the request; known as bit-synchronization detection time. on retransmissions, that ison transmissionsof a packet

JUBlN AND TORNOW DARPAPACKET RADIO NETWORK PROTOCOLS 29 whosefirst transmission has notbeen acknowl- threecoding rates-7/8,3/4, and 1/2-with constraint lengths edged-to get PRs whose previous transmissions of a 91,63, and 36, respectively. They provide increasing de- packet have collided out of sync with each other, so grees of gain-up to 9 dB at 1/2 rate-at the expense of in- their retransmissions do not also collide. creased channel and decoding overhead. Applyingan error correcting code will increase the probability of packet’s For these two cases, the protocol definedabove is still ap- successful reception and the link throughput. On the other plied, but starts with step Ib) instead of la). hand, when the noise and interference levels are low, the high overhead involved in a low-rate codewill tend tode- TransmissionParameters D. crease the link throughputsince most ofthe packetswould For most users’ packets, standard defaultvalues areused be received correctlywithout FEC. Thus highly dynamical- for most of theparameters affectingtheir transmission: to- gorithms must be used to estimate the channel’s current tal number of retransmissions, number of retransmissions condition and to determine the correspondingbest selec- before alternate routing is requested, the maximum value tion of FEC. of forwarding delay. Data rate is the onlyparameter that is The LPR’s FEC hardware also provides to thesoftware the selected dynamically formost users’ packets. The400-kbitl option to select 2-bit soft decision decoding (the bits ac- sdata rate is favored to minimizetransmission time;the100- tuallybeing generated duringdemodulation and then kbitls rate is selected onlywhen the link quality to the next passed on to the sequentialdecoder). Soft decision in- PR is bad atthe400-kbitls ratebut goodat the 100-kbit/s,rate. creases error correcting power in a Gaussian noise envi- One special type-of-service is offered to users. It provides ronment at the expense of greater decoding overhead. The lower delay, and thus higherthroughput, butless reliability current software employs 1/2-rate coding with hard deci- and was designed primarily for packetized speech. A user sion. The effect of burst noise is mitigated by interleaving can invoke speech type-of-service on a packet-by-packet (that is, shufflingthe bit order) on thetransmit sideand then basis via the speech type-of-service flag in the ETE header. de-interleaving on the receive side; the net result is that A comparison between the values of transmission param- burst noise gets spread across 128 bits. eters for speech type-of-service versus regular service is Error Detection:After error correction on the receive shown below: side, the LPR provides error detectionvia a 32-bit cyclic redundancy checksum (CRC). On thetransmit side the CRC Type of Service is generated before FEC encoding. Regular Speech Regular D ata rateData (kbitsls) 400, unless 400 always B. Modulation 100 is the only good link quality SymbolModulation: As discussed briefly inSection V-D, there are two selectable data rates, 400 and 400 kbits/s for Total numbertransmissions of 6 3 symbol modulation by the body ofpacket. the The 400-kbitl Number of transmissions s rate is used whenever possibleto minimizechannelusage. before alternate routing is requested 3 1 T Maximum forwarding delay r TRANSMISSION RECEPTION T T (“D“) 858 ms 30 ms Function Parameters FuMion Parameters Measurements 3RC generation packet CRC checking sectioning (error detection) VI.HARDWARE CAPABILITIES t FEC I enccdirQ rate FEC Tarasoft FEC error count :convolutional (sequenlial) jecision This section discusses in more detail theLPR’s hardware 3ncading) dewdiw capabilites: the Data Link Layer functions introduced in (error mneclbn) Section IV and V, and the Physical Layer functions not yet t nterleavingI deinteriaaving introduced. This section also presents the parameters and measurements available to the software, including some 1 t areamble preamble bit sync lime of arrival that are not currently beingused, and discusses potential aeneraion contents, detection. whether to frame syw ways in which theycan be used. Amore detailed discussion time-la0 detection of the LPR hardware is in an accompanying paper in this i and issue [14]. 9YmbOl data rate symbel lardisolt signal + noise, An overview of the LPR’s hardware functions is given in demodulation decision mise. muitipath Fig.10.Thefiguretracesapacketthroughthestagesleading modulatir t chip mdutation key, time slot chip (By. time slot up to radio transmission and following reception. It also (PN encoding) duration or time demodulation juration, table 01 sbt itsen, (PN encoding) qualiliers shows the hardware parameters availablefor the software qualifier to set at each transmit and receivestage and the measure- I ments provided to the software at each receive stage. up-conversionI trequency down-conversion lnquency t I receive power A. Error Control adaptive power attenuation automatic gain control control (AGC) level Forward Error Correction: As discussed briefly inSection 1 t IV-C, the LPR uses long convolutionalcodes generated on transmission- reception the transmitside and sequentialdecoding on the receiving side to perform forward error correction (FEC). There are Fig. 10. Stages of packet transmission and reception.

30 PROCEEDINGS OF THE IEEE, VOL. 75, NO. 1, JANUARY 1987 However, the lOO-kbit/s rate providesan additional 6-dB re- time after a change. This feature can be used to provide ceive processing gain overthe 400-kbitls rate. Furthermore, frequency division multiple access (FDMA). the IOO-kbit/s rate provides more resistance to multipath Adaptive PowerControl: The hardware providesthe soft- due to longer symbol time. ware the capability to select, on a packet-by-packet basis, Chip Modulation: One of the LPR's most important fea- attenuation to the nominal 5-W transmitted signal of 0 to tures is the direct-sequence spread-spectrum waveform 24 dB in 8-dB steps. This feature can be used to decrease produced by chip modulationat a rate of 12.8 megachips or increase network connectivity or to match the signal per second usinga pseudo-noise(PN) sequence. Spreading power of transmissionsfrom various-distanced PRs at each provides a processing gain over noise and interference. Thereceiver, to mitigate the "near-far" problem. gain is approximately 13 dB at the 400-kbitls data rate and 19 dB at the IOO-kbit/s data rate. The particular PN code se- D. Receive Measurements quence generated is a functionof the 56-bit key, the 25-bit slot time, and 32-bita entity that wecall the "qualifier." For Several measurements on the receive side are provided a packet to be detected at all, the code sequence on the to thesoftware: receivepower (ACC), signal + noise, noise, receive side must match that on the transmitside, and thus multipath, and FECerrorcount. Thesecan be usedto better all three parameters of which the codesequence is a func- quantify the link qualities between neighboring PRs (Sec- tion must also match. The PN-code-generation algorithm tion Ill-A) that provide the basis for the PRNET routing al- ' is the National Bureau of Standard's Data EncryptionStan- gorithms (Section Ill-B). dard (DES) 1151. These receive measurements can also be used to better Even when the slot time and the qualifierare fixed, the decide whichparameterslvalues to employ on the transmit generated PN code changes with each bit of the packet side of the link.For example, when appreciable multipath body. This bit-by-bit code changingcan provide "capture," is present, the 100-kbitls data rate should be used. Also, a the successful reception of apacket transmitted from one low signal-to-noise ratio, as calculated from signal + noise PR even when overlapped by the transmission of a packet and noise measurements, can be improved by a combi- from another PR at least one preamble timelater [16]. Let- nation of higher power (that is lower attenuation), lower ting theslot clock runmakes the codegenerated afunction FEC coding rate, and lower data rate. of the slot time at the beginning of the transmission pro- The problem is that changes in parametershalues to im- ducing "code slotting" and a time-varying waveform. The prove the performance of one link tendto degrade the per- time slot duration can be as small as 5.12 ms. For a code- formance of other links: Higher transmit powerincreases slotted packet to be detected at all, the receive side must the range and the contribution ofinterference; lower FEC employ the same slot time as the transmit side did when coding rate and lower data rate each increases the length it transmitted the packet. Code slotting requires the net- of timeof interference. Such network self-interferencecan work tobe synchronizedso that transmitting and receiving be mitigated by employing CDMA andlor FDMA. Here the PRs' slots coincide to a large degree. Synchronization ac- main issue to be explored in future work is dynamic sub- curacyis providedbythe hardwarecapabi1ityto"time-tag," network organization in theface of changing connectivity. that is, to read the time toa resolution ofps 5 at a particular point, called "frame synch," in the transmission/reception VII. CONCLUSION of the preamble. On thetransmit side, the timeis put in the packet being transmitted. On the receive side, the time is The success of theDARPA packet radio program has led provided alongwith thepacket itself(containing thetrans- to the investigation ofpacket radio applicationsin both the mit time-tag) to the software. militaryand commercialarenas. Experimental packet radio PN qualifiers workas follows: The transmit side puts the networks have been established byboth the Armyand the index to a commonly known table of qualifiervalues into Strategic Air Command for purposes of determining the the preamble of thepacket andthen PN-encodes the body applicability of PR technology to support distributed, sur- ofthepacketusingthequalifiervalueitse1f;thereceiveside vivable command, control, and communications[IA. The thenfetchestheindexfromthepreambleandthenthevalue military has also initiated an investigation of the potential from itsown tabkanduses itto PN-decodethe body. Using for extending the capabilities of existing military point-to- various PN qualifierswould provide code division multiple point radios with packet appliques. A packet appliqueis a access (CDMA). The qualifier could be the receiving PR's set of processors (such as a personal computer) which is ID, in which case receiver-directed CDMA would be pro- interfaced to existing radio equipment. The applique con- vided. Or the qualifier could be a "group" ID; a network stitutes the digital portion of the packet radio node; thus could bedivided into groups to mitigate theadverseeffects the protocols that control the routingand flow of packets of overdenseness, or swaths of PRs could be grouped into among nodes are implemented in it. The enhancement of "superhighways." the radio equipment by the addition ofpacket the applique greatly increases the network's flexibility by supportingau- tomatic communication amongnodes thatare not in direct C. RF Stages connectivity with one another. RF Conversion: There are 20 RF frequencies between Another application of packet radio networks currently 1718.4and 1840.0 MHz selectable in theLPR. This capability beingexploredistherequirementforprovidingdistributed can be used to set upcolocated but noninterferingPRNETs. access to either distributed or centralized information. Li- The LPR is frequency-agile to the extent that the hardware brary automation is a good example of this need. As tech- provides to the software the capability to change the fre- nology advances, more and more interlibrary information quencyonapacket-by-packetbasiswithonlya3-mssettling is being exchangedelectronically. Until now,very little

JUBINAND TORNOW: DARPA PACKET RADIO NETWORK PROTOCOLS 31 progress has been made in obtaining fixed-cost telecom- architecture for Open Systems Interconnection,”IEEE Trans. munication services for libraries. Except for expensive and Commun., vol. COM-28, no. 4, pp. 425-432, Apr. 1980. B. Leiner, R. Cole, J. Postel, and D.Mills, “The DARPA Internet specialized facilities (such as private microwave) that are Protocol Suite,” /€E€ Trans. Commun., vol. COM-23, no. 3, pp. beyond the budgetary reach of virtually all libraries, one 29-34, Mar. 1985. can expectto pay common carriers every monthservices for D. Beyer,M. Lewis, andJ. Tornow, “Network interface unit,” used, andcommon-carrier rateshave beensteadily in- Tech. Rep. SRNTN 35, SRI International, Sept. 1985. N. Shacham andJ. Westcott, “Future directionsin packet ra- creasing. Packet radio, however, offers librariesthe poten- dio architectures and protocols,” this issue, pp. 83-99. tial for purchasingdatacommunication capabilities at a rea- J. Westcott and J. Jubin, “A distributed routing design for a sonable fixed cost. broadcast environment,” in MILCOM’82 Proc., 1982. Single-hop packetradio networks are also being used to J. M. McQuillan and D.C. Walden, “The ARPA Network de- improve the efficiency of commercial operations. For ex- sign decisions,” Comput. Networks, vol.l, pp. 243-289, Aug. 1977. ample, the useofa small hand-held radiowith a limitedkey- N. Cower and J. Jubin, ”Congestion control using pacingin board is being experimentedwith by restaurants. The wait- a packet radio network,” in Proc. /€E€ Millitary communica- ers, bartenders, and cooks are all equipped with a packet tion Conf. (M/LCOM’82), vol. 1, pp. 23.1-1-23.1-6, Oct. 1982. radio. The waiter entersan order andits destination (either [I 31 L.KleinrockandF.A.Tobagi,”Packetswitchinginradiochan- the bartender or thecook), then waits to receive a packet nels: Part I-Carrier sense multiple-access modes and their throughput-delay characteristics,” /€€E Trans. Commun., vol. indicating that the orderhas been completed. COM-23, no. 12, pp. 1400-1416, Dec. 1975. In summary, packet radio is an exciting technology that W. C. Fifer and F. J. Bruno, “The lowcost packet radio,” this is beginning to play an important role in the local distri- issue, pp. 33-42. bution of information. In this paperwe have presented the Fed. Inf. Process. Stand., Data Encryption Standard, publ. 46 ed., Nat. Bur. Stand., Washington, DC, 1977. current state of the DARPA packet radio network.The pri- D. H. Davis and S. A. Gronemeyer, “Performance of slotted marycomponent of thePRNET is the LPR. The LPR has many ALOHA random access with delay capture and randomized sophisticated features that can provide enhanced flexibility time of arrival,”/€E€ Trans. Commun., vol.COM-28, no. 5, pp. in designing a robust and reliable packet-switched com- 703-710, May 1980. munications network. Fullyautomated algorithmsand pro- M. S. Frankel, “Advanced technology testbeds for distributed, survivable command, control and communications,” tocols to organize, control,maintain, and move traffic in Proc. I€€€ MlLCOM Conf., vol. 3, Oct. 1982. through thePRNET have been designed, implemented, and tested. By means of these protocols, networks of about50 packet radios with some degree of nodal mobility can be John Jubinwas born in Philadelphia, PA, on March 25, 1947. He received the B.S.E. de- organized and maintained under a fully distributed mode gree in electrical engineering from Prince- of control. We have described the algorithms and illus- ton University, Princeton, NJ, in 1968, and trated how thePRNET system (i.e., the LPRs along with their theM.S. degreein operations research from attached devices) provides highly reliable network trans- Southern Methodist University,Dallas, TX, in 1977. port anddatagram service, by dynamically determiningop From 1969 to 1973 he was a communi- timal routes, effectively controlling congestion, and fairly cations-electronics engineer in the U.S. Air allocating the channel in theface of changing link condi- Force Communications Service.In 1973 he tions, mobility, and varying traffic loads. joined Texas Instruments, Dallas, TX, where he developed software for real-time seismic data processing sys- temsand for Global Positioning System user sets.In 1977 he joined ACKNOWLEDGMENT Rockwell International’s Collins Defense Communications orga- Many of ourcolleagues working on thepacket radio pro- nization, where hehas worked primarilyon DARPA’s Packet Radio and Survivable Radio Networks (SURAN) programs.is currentlyHe gram havecontributed to theideas presented in thispaper. principal investigator for Rockwell on SURAN.He has (co-)au- In particular, wewould like to acknowledge contributions thored four paperson packet radio network protocolsIEEE for con- from DARPA, CECOM, BBN Communications, Hazeltine ferences. He has designed packet-switched communications pro- Corporation, Rockwell Internationa1,andSRI International. tocols also for other systems, including the Ground Wave Emer- gency Network. REFERENCES Janet D. Tornow (Associate, IEEE) received [l] R. E. Kahn, “The organization of computer resources into a the B.S. degree in mathematics from Doug- packet radio network,” I€€€Trans. Commun., vol. COM-25, lassCollege, Rutgers University, New no. 1, pp. 169-178, Jan. 1977. Brunswick, NJ,in 1977, and the MS. degree [2] R. E. Kahn, S. Gronemeyer, J. Burchfiel, and R. C. Kunzelman, in operations research from Stanford Uni- “Advances in packet radio technology,” Proc. I€€€,vol. 66, versity, Stanford, CA, in 1979. no. 11, pp. 1468-1496, Nov. 1978. In 1978 she worked at Bell Laboratories. [3] J. Jubin,“Current packet radio network protocols,” in She joined SRI International in 1979, where lNFOCOM‘85 Proc., Mar. 1985. she is a Research Engineer. Her primary re- [4] K. Klemba et a/., “Packet radio executive summary,” Tech. sponsibility is principal investigator forSRI Rep. preparedfor Defense Advanced Research Projects on the DARPA Survivable Radio Networks Agency by SRI International, July 1983. program. Her researc I interests include the specification and de- [5] D. Behrman and W. C. Fifer, “A lowcost spread-spectrum velopment of packet,adio protocols, the integration of packet ra- packet radio,” in MILCOM’82 Proc., 1982. dio network system! and network characterization and perfor- [6] H. Zimmermann, “OS1 reference model-The IS0 model of mance evaluation.

32 PROCEEDINGS OF THE IEEE, VOL. 75, NO. 1, JANUARY 1987