IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-28, NO. 2, FEBRUARY 1980 27 1 preference for throughput over delay. This parameter can be stochastic fluctuationof the system load. Numerical results are presented considered as a further control tool to limit the throughput, and it is shown that the throughput-delay performance of a network can be thedelay, the average and/orthe maximum number of improved by proper selection of the design parmeters, such as the window tokens, and to limit the destination buffer utilization (Table size, the timeout period, etc. I?) ; The numerical examples indicate that the performance of INTRODUCTION a network when token limits are selected according to the A computer network may be thought of as a collection of suboptimalpolicy determined by our heuristic solution is resources to beused by a competing population of users. very close to the performance of an optimal policy (Fig. 3). Networkresources i~cludebuffers, transmission bandwidth, Thisfigure further indicates that the performance of a processortime, name space, table entries, logical channels, periodicdecision, is very close to the optimal as long as etc.; the user population includes any source of data which the decision period is chosen properly. This property is of requires transmission through the network. The collection of significant importance in practical application, as a periodic resourceshas a limited capacity which causes conflicts to decision reduces the overhead due to the signaling of control occuramong the users of thesystem. These conflicts may packets. resultin a degradation of systemperformance to the point that the system becomes clogged and the throughput goes to REFERENCES .~ zero [ 11.This behavior is typical of “contention”systems L. Kleinrock and p. Kermani,“Static flow control in store-and-forward in which the throughput will increase with the applied load computer networks,” IEEE Trans. Cornmun., this issue,pp. 271-279. P.Kermani, “Switching and flow control techniques in computer up to some optimum value, beyond which, due to unpredict- communicationnetworks,” Comput. Sci. Dep., Univ. Calif., Los ablebehavior by users and servers and additional user-user Angeles,UCLA-ENG-7802, Feb. 1978 ($so publishedas Ph.D. and user-server interactionand overhead, more load causes dissertation, Dec. 1977). areduction in throughput [2] .I Networkscannot afford to [31 A. Giessler, J. Haenle, A. Koenig, andE. Pade, “Packet networks with accept all thetraffic that isoffered without control; there deadlock-freebuffer allocation, an investigation by simulation,” Compur. Neworks, vol. 2, pp. 191-208, July 1978. mustbe rules which govern the acceptance of traffic from [41 L. Kleinrock, “Power and deterministicrules of thumb for probabilistic outsideand coordinate the flow inside the network. These problemsin computer communications,” in Proc. Inr. Cortf: on rules are commonly known as fZow.contro2 procedures. More Cotnmun., Boston, MA, June 1979, pp. 43.I. 143.I. 10. preciselyflow control is the set of mechanismswhereby a R. Howard, DynamicProgrumming and Markov Processes. Cambridge,MA: MIT, 1960. flow of data can be maintained within limits ‘compatible with B. L. Miller, “Dispatching from depot repair in a recoverableitem the amount of available resources [ 31 . ’ inventory system: On the optimality of a heuristic rule,” Munagemenr In order to keep the network traffic within desirable limits, Sci., vol. 21, pp. 316325, Nov. 1974. flow control procedures, among other things, must be equipped A. F. Veinott, Jr.,’ “The status of mathematicalinventory theory,” Munugernent Sci., vol. 2, pp. 745-777, July 1966. quippedwith throttling mechanisms. These mechanisms D. Blackwell, “Discrete dynamic programming,” Ann. Math. Sruris., include: credit (or tokens), which give permission for message VOI. 33, pp. 719-726, 1962. flow; a rate at which a given flow may proceed; a stop-andgo M. J. Sobel, “Optimal operation of queues,” in Marhemaricul Methods procedure which turns a flow on and off according to some in Queueing. Berlin: Springer-Verlag, 1974, pp. 231-261. criteria;the introduction of delay, so asto slowdown the flow,etc. [3]. A windowmechanism is an example of the credit scheme. Existing control methods in store-and-forward communica- tionnets can be classifiedas eitherlocal control or global control.Local control is appliedby a communication pro- Static Flow Control in Store-and-Forward ComputerNetworks cessor within the subnet OD the basis of its own as well as its immediateneighbors’ traffic data and resource utilization. Due tosome limitations [4], [ 51,local control is not,by LEONARD KLEINROCK, FELLOW, IEEE, AND PARVIZ KERMANI itself, sufficient to prevent congestion, and global control is necessary in order to further stop the input to the network Abstract-In this paper we develop an analytic model for end-to-end well before the network is loaded to saturation.’%his control communication protocols and studythe window mechanism for flow canbe accomplished by limiting the number of packets control in store-and-forward (inparticular message-switching) computer- simultaneouslycontained in the network. Examples of the based communication networks.We develop a static flow control model in existing methods ~f global flow control are: isarithmic flow which the parameters of the system are not dynamically adjusted to the control[4] studied for the NPL network;and end-toend flowcontrol ([6] andthe references therein] ) used inthe ARPANET,where, basically, the total number of credits Paper approved by the Editor for Computer Communication of the IEEE Communications Society for publication without oral presenta- between two users are limited. tion. Manuscript received June 13, 1978; revised June 27, 1979. This Most end-tolend flow control mechanisms use a variant of work was supported by the Advanced Research Projects Agency of the thecredit throttling technique and are usually described in Department of Defense under Contract MDA 903-77-CO272. terms of awindow mechanism [7], wherethe unacknowl- P. Kermani was with the Deqartment of Computer Science, Univer- edged messages (or packets) are limited to lie within a sliding sityof California, Los Angeles, CA 90024. Heis now with the IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598. window. L. Kleinrock is with the Department of Computer Science, Univer- End-to-endflow control is accomplished through inter- sity of California, Los Angeles, CA 90024. process communication protocols and any attempt to quanti- 0090-6778/80/0200-0271$00.75 @ 1980 IEEE tatiVelY studythe former should start with the development A ? ofan analytic model for the latter. Due to thecomplex DESTINATION CONTROLLER multivariate environment of these distributed dynamic control procedures,to date little work has been done in evaluating NETWORK the performance of flow' control procedures in terms of their 1 efficiencyand freedom from deadlock and degradation. In I I I I NETWORK NETWORK view of this, the recent work in the field has been extremely BOUNDARY BOUNDARY welcome [ 81 -[ 151. The purpose of this paper is to develop an analytic model Fig. 1. Structure of a network. forend-to-end communication protocols and to study the tokenmechanism forflow control store-and-forwardin computer based communication networks (we will be mainly concernedwith message- and packet-switching systems). We first develop a static flow control model in which the param- eters of the system are not dynamically adjusted to the sto- chastic fluctuations of the system load. We then present nu- merical results to show the effect of different parameters on the overall performance of the system. We use the static 'flow controlresults in a subsequent paper to develop a dynamic flow control system. Theanalysis carried out in thispaper is basedon many simplifyingassumptions. We considerthe calculations as an initial step towards a better understanding of more complex and more elaborate systems. It is believed that such a study is necessary even if it is based on idealized assumptions. RECEIVER (SUB) & 11. THE MODEL AND THE ASSUMPTIONS In this section we develop an analytic model for a network inwhich messages are flow cpntrolled with a token mecha- nism. We begin by describing the structure of the network and itscomponents. We thenelaborate on the assumptions in- volved in our analysis, and finally we present the analysis. A. Structure of' The Network regardingthe implications encountered in theexact analysis With a credit mechanism using tokens, the total number of the interested reader is referred to [ 131 ). messagesin the network between a source-destination pair A detailed specification of the structure of the subnetwork is restricted to a maximum value. We model the network as is not the object of our study; we view the subnetwork as a in Fig. 1, in whichthe traffic controller (TC) is responsible source of randomdelay which we choose torepresent by a for keeping the number of unacknowledgedmessages below properdistribution function (which will bediscussed in w, the limit on the number of tokens (we occasionally refer Section 11-B below). to w as the token limit). Fig.2(b) shows the structure of the -destination node. The structure of the TC is further elaborated in Fig. 2(a). Be'cause ofour requirement that the TC must receive an We maythink of a circulating pool of w "tokens,"each of ACK forthe last (re)transmitted message, tGe. destination which permits exactly one message to be accepted to the net- node may receive multiple copies