arXiv:math/0603325v1 [math.PR] 14 Mar 2006 06 o.1,N.1 244–294 1, No. DOI: 16, Probability Vol. Applied 2006, of Annals The etpce oteei n on rpTm RT ewe h t the between (RTT) Time Trip Round one is there so packet ment queue. bottleneck a through routed Reno TCP of hn,te h uptbffri h otrwl eabottleneck a be will router the in of so buffer interaction to the output file the a FTPs then a simultaneously to chine, station ethernet work switched every a If by router. connected are department versity
ENFEDCNEGNEO OE FMLIL TCP MULTIPLE OF MODEL A OF CONVERGENCE FIELD MEAN c n yorpi detail. 244–294 typographic 1, the and by No. published 16, Vol. article 2006, original Statistics the Mathematical of of reprint Institute electronic an is This nttt fMteaia Statistics Mathematical of Institute ONCIN HOG UFRIPEETN RED IMPLEMENTING BUFFER A THROUGH CONNECTIONS e od n phrases. and words Key 2 1 pnrciigaTPpce,tercpetsnsbc nackn an back sends recipient the packet, TCP a receiving Upon .Introduction. 1. M 00sbetclassifications. subject 2000 AMS 2005. January revised 2003; June Received eerhiiitddrn nitrsi tteUiest o University the at internship an A4551. during Grant initiated NSERC Research by part in Supported 10.1214/105051605000000700 al itiuino aktls n eue synchronizat reduced and loss thro packet through of higher achieved distribution variation loss, table delay packet reduced th reduced and delay with are dro reduced increases by objectives that overflows probability The buffer a length. the with before packets marking congestion buff or detect bottleneck a to through is multiplexed idea are sessions TCP tiple o esr ope ihteqee eew rv h conject the prove we as Here sizes [ queue. window in the [ of made with histogram in coupled the qu idea measure consider dom the key to is The by session 77–97] buffer. coupled TCP (2002) the systems each at of dynamical length sizes independent bott window like a The evolve through RED. multiplexed connectio implementing TCP regime buffer multiple avoidance for congestion model fluid the a proposed have 77–97] ihadtriitcqueue. densit deterministic size a window t the with converges comprising system limit this infinity, mean-field to ministic tends connections of ber acli coadadRyir[ Reynier and McDonald Baccelli, nvriyo taaand Ottawa of University E Rno al eeto)hsbe ugse hnmul- when suggested been has Detection) Early (Random RED yD .McDonald R. D. By efrac Evaluation Performance N C/Pcnetosi h ogsinaodnephase avoidance congestion the in connections TCP/IP mgn h cnrowhere scenario the Imagine C,RD enfil,dnmclsystems. dynamical mean-field, RED, TCP, 2006 , hsrpitdffr rmteoiia npagination in original the from differs reprint This . rmr 02,6K5 eodr 94C99. secondary 60K35; 60J25, Primary in h naso ple Probability Applied of Annals The 1 11 cl oml Sup´erieure Normale Ecole 1 ´ efrac Evaluation Performance n .Reynier J. and 20)7–7 ht stenum- the as that, 77–97] (2002) efrac Evaluation Performance N oksain nauni- a in stations work Ottawa. f 2 ion. deter- a o coupled y nequi- an 11 ughput, queue e edsatma- distant me r The er. leneck ran- a pping departmental (2002) sin ns eue ure estudy We . 11 , owledg- m a ime 2 D. R. MCDONALD AND J. REYNIER packet is sent and the acknowledgment is received. The acknowledgment contains the sequence number of the highest value in the byte stream suc- cessfully received up to this point in time. By counting the number of packets sent but not yet acknowledged, each source implements a window flow con- trol which limits the number of packets from this connection allowed into the network during one RTT. Duplicate acknowledgments are generated when packets arrive out of order or when a packet is lost. The link rate of the router is NL packets per second. We assume pack- ets have equal mean sizes of 1 data unit. We assume the packets from all connections join the queue at the bottleneck buffer and we denote by QN (t) the average queue per flow. We assume the scheduling to be FIFO. We imagine the source writes its current window size and the current RTT in each packet it sends, where by the current RTT we mean the RTT of the last acknowledged packet: N • Wn (t) is defined to be the window size written in a packet from connection n arriving at the router at time t. N • Rn (t) is defined to be the RTT written in a packet from connection n arriving at the router at time t (this RTT is the sum of the propagation delay plus the queueing delay in the router). • We shall assume that connections can be divided in d classes Kc, where c ∈ [1, 2,...,d] where the transmission time Tn of any connection n ∈ Kc equals the common transmission time Tc for class c (the notation will be clear from context). N At time t, source n has sent Wn (t) packets into the network over the N last Rn (t) seconds. The acknowledgments for these packets arrive at rate N N Wn (t)/Rn (t) on average. New packets are being sent at the rate acknowl- edgments come back to the source so we will define the instantaneous trans- N N N mission rate of source n at time t to be Xn (t)= Wn (t)/Rn (t). This def- inition models the transmissions over any long period of time T . During time T , the total number of packet-minutes of work done by the network for connection n is T T N N N Wn (t) dt = Xn (t)Rn (t) dt. Z0 Z0 Consequently, our definition of the transmission rate is consistent with a Little type formula which calculates the work done as the integral of the N N packet arrival rate Xn (t) times the work done for each packet, Rn (t). Under TCP Reno, established connections execute congestion avoidance where the window size of each connection increases by one packet each time a N packet makes a round trip, that is, each Rn as long as no losses or timeouts occur. During this phase, the rate the window of connection n increases N is approximately 1/Rn packets per second. The only thing restraining the MEAN FIELD FOR RED 3 growth of transmission rates is a loss or timeout. When a loss occurs the window is reduced by half. The source detects a loss when three duplicate acknowledgments arrive. The source cuts the window size in half and then starts a fast retransmit/fast recovery by immediately resending the lost packet. Fast retransmit/fast re- covery ends and congestion avoidance resumes when the acknowledgment of the retransmitted packet is received by the source. We assume the losses are only generated by the RED (Random Early Detection) active buffer man- agement scheme or by tail-drop. We neglect the possibility of transmission losses. We also neglect the possibility that some of the connections fall into time- out. This may occur if there is a loss when the window size is three or less. In this case there can’t be three duplicate acknowledgments. The source can’t recognize that a loss has occurred and essentially keeps on waiting for a long timeout period. Alternatively, if a retransmitted packet is lost, the source will fall into timeout. Two losses in the same RTT may not produce a timeout and may have a different effect on the window size depending on the version of TCP being used. The detection of the second loss may provoke a retransmission, but no window reduction with NewReno [11] or with SACK [13] or will only provoke a second window reduction after the acknowledgment of the retransmission of the first lost packet; that is, the discovery of the second loss occurs more than one round trip time after the second loss occurred. When the timeout period elapses, the source restarts quickly using slow-start and attempts to re-enter the congestion avoidance phase. Losses which occur simultaneously with packets arriving out of order are also a major cause of timeouts. In practice, a certain proportion of the connections will be in timeout at any given time. In effect, one has to rede- fine N if one wants to compare theoretical predictions with simulations (see [2]). We will assume the large buffer holds B packets and that, once this buffer space is exhausted, arriving packets are dropped. Such tail-drops come in addition to the RED mechanism. Here we take the drop probability of RED (of an incoming packet before being processed) to be a function of the queue size which is zero for a queue length below Qmin but rises linearly to pmax at Qmax and is equal to 1 above Qmax. Note that this is not exactly as originally specified in [12], where the drop probability was taken as a linear function of the exponential moving average of the queue size. Since we will let the number of sources tend to infinity, this averaging out of fluctuations is not necessary and, in fact, is deleterious since it adds further delays into the system. If all N connections are in congestion, avoidance, we can reformulate this drop probability in terms of QN , the queue size divided by N, as F (QN (t)), where F is a distribution function which is zero below qmin = Qmin/N but 4 D. R. MCDONALD AND J. REYNIER rises linearly to pmax at qmax = Qmax/N and jumps to 1 at qmax. Of course, the tail-drop scheme can be considered as the limiting case when qmin = 0, qmax = b (B = Nb) and pmax = 0. In [2] we used a fluid description of the queue and a continuous approx- imation of the loss rate of each connection to construct a model for the evolution of the fluid queue as a function of the histogram of window sizes. This model generalized the model in [15]. The goal of this paper is to prove N the conjecture in [2] that the histogram or empirical measure Mc (t, dw) of the window sizes in any class c (d = 1 in [2]) converges to a deterministic mean field limit with measure Mc(t, dw) at time t and, moreover, the relative fluid queue size QN (t) converges to a deterministic fluid queue Q(t). Construct a sequence f k of bounded, positive continuous convergence determining functions on [0, ∞) (see pages 111 and 112 in [10]). Define a metric for weak convergence for probability measures on [0, ∞) by defining the distance between probabilities µ and ν as ∞ 1 kµ − νk := 1 ∧ |hf k,µi − hf k,νi| , w 2k kX=1 ∞ where hf,µi = 0 f(w)µ(dw). Also, let k · ks denote the Skorohod distance between two elements of D[0, T ]. R Theorem 1. Under Assumptions 1 and 2 given in Section 2, the ran- dom measure of the window sizes of connections in each class c converge N in probability to a deterministic measure Mc(t, dw); that is, kMc (t, dw) − Mc(t, dw)kw → 0 in probability as N →∞. Mc(t, dw) is the marginal distri- bution of Mc(s − Rc(s), dv; s, dw), the deterministic limit of the joint distri- bution of the window sizes at time t and at time t − Rc(t). 1 R+ 1 R+ Let G = {g ∈ Cb ( ) : g(0) = 0}, where Cb ( ) is the space of bounded functions with bounded derivatives. For g ∈ G, c = 1,...,d,
hg, Mc(t)i − hg, Mc(0)i t 1 dg(w) (1.1) = , M (s, dw) R (s) dw c Z0 c + h(g(w/2) − g(w))v, Mc(s − Rc(s), dv; s, dw)i 1 × K(s − R (s)) ds R (s − R (s)) c c c t 1 dg(w) (1.2) = , M (s, dw) R (s) dw c Z0 c + h(g(w/2) − g(w)), e(s,s − Rc(s), w)Mc(s, dw)i 1 × K(s − Rc(s)) ds, Rc(s − Rc(s)) MEAN FIELD FOR RED 5
∞ where hg, Mc(t)i = w=0 g(w)Mc(t, dw), where Rc(t)= Tc + Q(t − Rc(t))/L, where K(t)= F (Q(t)) for Q(t) d dQ(t) c (1 − K(t)) = κ hw, Mc(t, dw)i − L dt c=1 Rc(t) (1.3) X d − c (1 − K(t)) − κ hw, Mc(t, dw)i − L χ{Q(t) = 0}. Rc(t) ! cX=1 When Q(t)= qmax, K(t) is determined by d −1 1 c (1 − K(t)) K(t)=max pmax, κ hw, Mc(t, dw)i , L c Rc(t) ! ! X=1 where hw, Mc(t, dw)i = w wMc(t, dw) is Lipschitz continuous in t for each c. R The numerical evaluation of the above equations is considered in Section 7. Note that there may be a discontinuity in K(t) when Q(t) hits qmax. The problem at qmax arises because F is not continuous and certainly not Lipschitz at this point. To justify this definition, we consider the Gentle RED variant [18], where b> 2qmax, and we extend the definition of F to rise linearly from pmax to 1 between qmax to 2qmax so F is Lipschitz. In Section 5 we modify Gentle RED so that the drop probability increases linearly from pmax at qmax to 1 at qmax + δ ≤ b. The weak limit of this modified Gentle RED as δ → 0 gives the discontinuous K(t) above. The reflection problem at queue size zero can be solved by the Skorohod construction. It is important to emphasize that we have not justified that the fluid models in [15] or [2] are the limit of some discrete packet level model. We are proving far less; that is, that the intuitively attractive fluid models in [15] or [2] do converge to a mean field limit. This convergence is implicitly assumed in the engineering literature and the resulting limit processes are used to analyze the stability of various Active Queue Management (AQM) control strategies (RED among others) [15]. We also make the simplifying modeling assumptions made in [2], although more precise alternatives are suggested d N (see the Doppler factor, [1 − dt Rn (t)] in (2.1)). These simplifications (or 6 D. R. MCDONALD AND J. REYNIER the assumption that mean windows one RTT apart are uncorrelated made in [15] but not by [2]) may be too gross and, to date, nobody has done the network measurements to check which assumptions make a significant difference. This is a major failing because the control theoretic analysis, as proposed, for instance, in [15] or [8], may be highly sensitive to these assumptions. The conclusion, as far as RED is concerned, is negative. If the RED param- eters are not chosen properly (as a function of RTT), then RED is unstable. The fixed point described in [2] may be unstable and a tiny oscillation is amplified until the queue size oscillates wildly. Even with a large buffer, the utilization may drop to less than one. Although one may criticize lacunae in the model, this conclusion is verified by simulation and for this reason, RED is rarely activated even though it is implemented on most routers. We are mainly interested in a mathematical proof of the convergence to the mean field so we will ignore timeouts and slow-start, as well as spe- cial details of congestion avoidance which would only serve to obscure the main ideas. We will, nevertheless, sketch how these extensions could be han- dled. Our method could be adapted to proving mean field limits for control schemes other than RED. [19] and [7] analyze time slotted rate-based and queue-based models with delay where the number of sources tend to infinity. For their queue-based model, they prove the convergence of the queue to a deterministic limit and propagation of chaos; that is, the transmission rates of each source converge to a system driven by the deterministic queue size. In their model the limiting, deterministic queue is not coupled with limiting distribution of the window sizes and there is no associated partial differential equation as in [2]. The only comparable analysis is the recent thesis [3]. The structure of the paper is as follows. In Section 2 we model the system of N windows coupled with the queue and then formulate this as a histogram of window sizes coupled with a queue. In Section 2.3 we summarize the mean field limit. The proof of the existence of this limit follows in Section 3. In Section 4 we establish the convergence to a unique limit. Finally, in Section 6 we establish Theorem 1. The TCP model we are studying can be viewed as N dynamical systems WN N N [the N window sizes := (W1 ,...,WN )] which evolve independently except through a shared resource (the queue QN ). The dynamics of the shared resource depend only on the distribution of the dynamical systems (in this case on the average window size). The standard approach is to prove existence and then uniqueness of the limit. That’s what we do here, but the mathematical innovation is to first create a modified system [here (WN ,QN ) is modified to (WN , QM )] where the dynamics of the shared resource (the modified queue QN ) depend on the expected value of the distribution (or, in this case, the expected value of the average window sizes of the modified MEAN FIELD FOR RED 7 system). Essentially, we just stick an expectation in front of the interaction N N term [here we replace the average window size W (t) by EW (t)]. Since the modified shared resource QN is deterministic, the modified dy- namical systems are independent. Moreover, it is easy to pick a convergent subsequence for the shared resource (here for QN → Q). It is then easy to N prove Wn converges to a limit Wn along the subsequence for each compo- nent n. This gives the existence of an infinite modified system [here (W, Q), where W = (W1, W2,...)]. Next, the key remark is that by the law of large numbers (and boundedness), 1 N W(t) := lim Wn(t)= EW(t); N→∞ N n X=1 that is, the infinite modified system is, in fact, a limit of the original system! Here this means we can rewrite (W, Q) as (W,Q) with W := (W1,W2,..., ) since the interaction term is W(t); that is, the interaction is through the window average and not the expected value of the window average. Next, we use a coupling argument to show each original dynamical system N (here each Wn ) converges almost surely in Skorohod norm to the infinite N limit (here Wn converges to Wn). This proves the propagation of chaos where each dynamical system (Wn) is independent and interacts with the other systems only through the deterministic shared resource Q. From this, we can show the mean field convergence (here the convergence of QN to Q and the histogram of the WN to the mean field limit). We have used Kurtz’s approach (e.g., [9]) of bringing back the particles; that is, not projecting onto the histogram of window sizes as is standard (e.g., [5]). We believe this is the most effective way to handle feedback delay because the histogram of window sizes is not a state. We think our approach has potential in other contexts. 2. The N-particle system and mean-field limit. 2.1. The N-particle Markov process. Our model takes into account the delay of one round trip time between the time the packet is killed and the time when the buffer receives the reduced rate. We assume window reductions at connection n occur because of a loss one round trip time in the past. To first order, the probability of a window reduction between time t and t + h is N N t+h−Rn (t+h) W (s) n KN (s) ds N N t−Rn (t) RT Tn (s) (2.1) Z N N d N Wn (t − Rn (t)) N N ∼ 1 − Rn (t) N N K (t − Rn (t))h, dt Rn (t − Rn (t)) 8 D. R. MCDONALD AND J. REYNIER N N since the probability a packet is dropped is proportional to Wn (t − Rn (t))/ N N N Rn (t − Rn (t)), the transmission rate one RTT in the past, times K (t − N Rn (t)), the drop probability one RTT in the past. The Doppler term [1 − d N dt Rn (t)] is a small correction that was overlooked in [2] and we will ignore it here. There are many ways of actually implementing packet drops once the drop probability p = KN (t) is determined at time t. We could drop packets deterministically one every 1/p packets, but this may introduce unwanted synchronization. In fact, [12] proposes two methods. In the first we sim- ply generate a Bernoulli random variable with probability p of dropping a packet. In the second the dropped packet is chosen uniformly among the next 1/p packets. In fact, it won’t matter which method is employed since, as N →∞, the contribution of each flow becomes negligible. Consequently, the packet arrivals of connection n are enormously spread out among the other packets. As far as connection n is concerned, packets are dropped ran- domly with probability p = KN (t) at time t. We therefore model the process of window reductions by a Poisson point process with stochastic intensity N N N Wn (t − Rn (t)) N N λn (t) := N N K (t − Rn (t)) Rn (t − Rn (t)) N [we can assume Wn (t)= wn for t< 0]. Of course, the second method proposed in [12] would induce a weak de- pendence between the Poisson processes for different connections. However, the interaction between flows is via the average window size and the minor weak dependence won’t prevent the average from converging to a determin- istic limit. We will assume the first method is used, but it would be possible to alter the argument in Sections 3 and 6 to account for weak dependence. We can construct the simple point process of window reductions: t ∞ N N N (t)= χ N (u)Υ (du, dv), n [0,λn (v)] n Z0 Z0 N where the Υn (u, v) are two-dimensional Poisson processes with intensities 1 on [0, T ] × [0, ∞). In addition, the sources evolve independently given the N N trajectory of K . We therefore assume the Υn (u, v) are independent. In order to derive strong convergence theorems as N →∞, we shall suppose, N without loss of generality, Υn (u, v)=Υn(u, v), where {Υn}n=1,...,∞ is a se- quence of i.i.d. two-dimensional Poisson processes with intensity 1 defined on a probability space (Ω, F, P ). In fact, Poisson processes in {(t, u) : 0 ≤ u ≤ λ(t), 0 ≤ t ≤ T } would do where λ is defined after Assumption 1 as an a priori bound on the transmission rate. Define Ft = σ{Υn(u, v); v ≤ t,n = 1, 2,... }. The above is a different version of the point process of window reductions than that in [2]. The laws are the same, so the resulting dynamical systems have the same distribution. Consequently, the convergence in probability proved in Theorem 1 is also valid for the version used in [2]. MEAN FIELD FOR RED 9 N N Differential equation for queue size. For Q(t) N N N dQ (t) Wn (t) N N = N (1 − K (t)) − NL dt n Rn (t) X=1 N N − Wn (t) N N + N (1 − K (t)) − NL χ{Q (t) = 0} n Rn (t) ! X=1 since the proportion KN (t) := F (QN (t)) of the total fluid, N N Wn (t) N , n Rn (t) X=1 is lost. The second term prevents the queue size from becoming negative. In effect, the queue can stick at 0 until a sufficient number of connections increase their window size. Dividing by N gives N N N dQ (t) 1 Wn (t) N = N (1 − K (t)) − L dt N n=1 Rn (t) (2.2) X N N − 1 Wn (t) N N + N (1 − K (t)) − L χ{Q (t) = 0}, N n Rn (t) ! X=1 with QN (0) = q(0). N If Q (t) reaches qmax and N N 1 Wn (t) (1 − pmax) N > L, N n Rn (t) X=1 N then the queue must jitter at qmax and the loss probability K (t) is deter- mined by N N N 1 Wn (t) (1 − K (t)) N = L. N n Rn (t) X=1 N In other words, if Q (t)= qmax, then N N −1 N 1 Wn (t) K (t)=max pmax, 1 − L N . ( N n Rn (t) ! ) X=1 To justify the above definition of KN (t), one would really need to show the N loss probability of a packet model jittering a qmax converges weakly to K (t). Instead, we show the loss probability as Gentle RED converges to KN (t) as 10 D. R. MCDONALD AND J. REYNIER Gentle RED converges to RED (see Section 5). We should also contrast this loss rate with that for small buffers (i.e., B is constant as N →∞) studied by [17]. For a small buffer, fluctuations will cause packet losses long before the total transmission rate reaches the link rate NL. Essentially one can model N N 1 N Wn (t) K as LB( N ), where LB can be calculated by finding the LN n=1 Rn (t) equilibrium distributionP of a suitable Markov chain as in [16]. However, since our buffer is scaled with N, fluctuations like this can be ignored. It is worth noting that our method would allow us to prove mean field convergence for the small buffer case. There would be no equation for the queue and the round trip times are constant. Differential equation for windows. There are three separate phases: con- gestion avoidance, timeout and slow start. We concentrate on describing the congestion avoidance phase. During congestion avoidance, while the queue is nonempty but less than the buffer size, the window size increases by one every time a complete window is acknowledged. In [2] and [15] this term was taken to be simply 1/Rn(t) (i.e., one packet increase per RTT) and we make the same approximation here. Note that this approximation ignores the fact that acknowledgments return to the sources at the link rate NL when the queue is nonempty. N If the source detects the loss of a packet at time t − Rn (t) because three duplicate acknowledgments arrive, the source cuts the current window size N − N − Wn (t ) by half to Wn (t )/2. The slow start threshold (ssthresh) is set N N − to Hn (t)= Wn (t )/2. The source then begins fast retransmit and fast recovery. The lost packet is retransmitted and, through window inflation packets, continue to be sent as if the window size is constant [or at least the average transmission rate is consistent with a constant window size N − Wn (t )/2]. We will ignore this effect and assume the window size increases at a rate of 1/Rn(t) even during fast recover; that is, we don’t include the term (1 − χSn(t)) in [2]. When the retransmitted packet is acknowledged, congestion avoidance resumes. Hence, the evolution of the window size in the congestion avoidance phase is described by the following stochastic dif- ferential equation: N − N 1 Wn (t ) N (2.3) dWn (t)= N dt − dNn (t), Rn (t) 2 N with Wn (0) = wn, n = 1,...,N, specified. Denote the vector of window sizes by WN (t). N − If we wished to model timeouts, we could define a function U(Wn (t )) equal to one if the connection falls into timeout during fast recovery and zero if not. Hence, the point process of falling into timeout is given by N − N U(Wn (t )) dNn (t). During the timeout phase, the window size is zero. MEAN FIELD FOR RED 11 The connection is described by ssthresh and the remaining time in timeout. After the timeout phase elapses, the source enters slow-start and doubles its window size starting from one every RTT until the window size reaches ssthresh, at which time congestion avoidance restarts. If another loss is de- tected before reaching the congestion avoidance phase, the connection will go into timeout. During the slow-start phase, the connection is described by the window size and ssthresh. At any time t, a certain proportion of the N connections will be in each phase. In the mean-field limit these proportions will converge to deterministic fractions. We will not show this here. In fact, we will simply ignore all the special details of fast recovery and timeouts. Assumptions on the initial state. Assumption 1. (i) Prior to time zero, the window size of connection n in the N connection system is a constant wn, where 0 ≤ wn ≤ Wmax. (ii) The transmission time Tn of connection n satisfies Tmin ≤ Tn ≤ Tmax for all n. (iii) QN (0) = q(0) a constant. Bound a(t) for the window size at time t. From Assumption 1(i), t t N a(t) := Wmax + ≥ wn + ≥ Wn (t) Tmin Tn at every time t. The stochastic intensity for the Poisson point process of N losses of connection n is λn (s) ≤ a(s)/Tmin =: λ(t) for all 0 ≤ t ≤ T . N N N Note that (1 − K (t)) ≥ 1 − pmax as long as Q (t) N Relation between RTT and queue size. Define φn (s) to be the future round trip time written into a packet leaving the source at time s. For N N N the above scenario, φn (s)= Tn + Q (s)/L. Also note that s + φn (s) is N monotonic because the derivative, if Q (t) N N N N N φn (s) is monotonic, Rn (t) is well defined and φn (t − Rn (t)) = Rn (t). Also, N N N N substituting Rn (t)= t − s into φn (s)= Tn + Q (s)/L, we get that Rn satisfies N N N (2.4) Rn (t)= Tn + Q (t − Rn (t))/L. Moreover, by taking the derivative of (2.4), we get 1 (2.5) (1 − R˙ N (t)) = n ˙ N N 1+ Q (t − Rn (t))/L N N N −1 1 Wn (t − Rn (t)) N N (2.6) = L N N (1 − K (t − Rn (t))) . N n Rn (t − Rn (t)) ! X=1 2.2. Reformulation in terms of a measure-valued process. Classes of connections. We will assume there are d classes of connec- tions Kc, c = 1,...,d, and all connections in class c have the same trans- N N mission time Tc. Hence, Rn = Rc for all n ∈ Kc. We will also assume the N proportion of the N connections in class c is κc . In addition to Assumptions 1, we assume the following: Assumption 2. Assumptions on connection classes: N (iv) The proportion of users in the class c : κc → κc for c = 1,...,d as N →∞. N (v) Let µc be the histogram of windows of connections from class c at N time 0. We suppose that, for all c, µc converges weakly to µc as N →∞, where the support of µc is concentrated on [0,Wmax]. Measure-valued process. In order to study the limiting behavior of the system as the number of connections N goes to infinity, we will first define c the empirical process (see [5, 6]) of those connections in class KN . For any Borel set A, define N N 1 N (2.7) Mc (t, A) := N χA(Wn (t))χ{n ∈ Kc} κc N n X=1 to be the associated probability-measure-valued process taking values in + + M1(R ), the set of probability measures on R = [0, ∞) furnished with the topology of weak convergence. MEAN FIELD FOR RED 13 How the future is determined. The sequence {Υn(u, v)}n∈N of indepen- dent Poisson processes with intensity 1 was defined on a probability space WN N N N N N {Ω, F, P }. (t) ≡ (W1 (t),...,WN (t)), Q (t) and M (t) ≡ (M1 (t),..., N Md (t)) can be constructed path by path as processes defined on {Ω, F, P } + ∞ + + d N taking values in (R ) , R and M1(R ) , where the coordinates of W (t) N above N are zero. It suffices to assume Wn (t)= wn for t ≤ 0 and build the solution of the system (2.3), (2.2) pathwise from jump point to jump point of Υn. Reformulating (2.3). Let hg,µi = g(w)µ(dw) Z and hId,µi = wµ(dw), Z so N N N 1 N W c (s) := hId, Mc (s)i = N Wn (s)χ{n ∈ Kc}. κc N nX=1 If g ∈ G, then N N hg, Mc (t)i − hg, Mc (0)i N t 1 dg N ds (2.8) = N χ{n ∈ Kc} (Wn (s)) N κc N n 0 dw Rc (s) X=1 Z N − N − N + (g(Wn (s )/2) − g(Wn (s ))) dNn (s) . In Section 6 we consider the limit of the above as N goes to infinity to obtain an equation for the evolution of the distribution of the windows. N N N Reformulating (2.2). For Q (t) t d N N N (1 − K (s)) (2.9) = κc hId, Mc (s)i N − L 0 " c Rc (s) Z X=1 d N − N N (1 − K (s)) N + κc hId, Mc (s)i N − L χ{Q (s) = 0} ds, c Rc (s) ! # X=1 where N N N (2.10) Rc (t)= Tc + Q (t − Rc (t))/L. 14 D. R. MCDONALD AND J. REYNIER N N If Q jitters at qmax, then the loss probability K (t) is given by d N −1 N N N (1 − K (t)) K (t)=max pmax, 1 − L κc hId, Mc (t)i N . ( Rc (t) ! ) Xc=1 2.3. Summary of the mean-field limit. We wish to show that (WN (t),QN (t)), the unique solution to the N-particle system, converges as N →∞. In Sec- tion 3 we first prove the existence of the following limit. Theorem 2. If Assumptions 1 and 2 hold, then there exists a unique strong solution (W, Q, (M1,...,Md)) to the following system. For Q(t) < qmax, K(t)= F (Q(t)) and Q(t) − Q(0) t d c (1 − K(s)) (2.11) = κ hId, Mc(s)i − L 0 " c Rc(s) Z X=1 d − c (1 − K(s)) + κ hId, Mc(s)i − L χ{Q(s) = 0} ds. Rc(s) ! # Xc=1 When Q(t)= qmax, then K(t) satisfies d −1 c (1 − K(t)) K(t)=max pmax, 1 − L κ hId, Mc(s)i . ( c Rc(t) ! ) X=1 Each window evolves according to − 1 Wn(t ) (2.12) dWn(t)= dt − dNn(t), Rn(t) 2 where Wn(0) = wn, n = 1,..., are specified, where t ∞ Nn(t)= χ[0,λn(t)](v) dΥn(u, v), Z0 Z0 where Wn(s − Rn(s)) λn(s)= K(s − Rn(s)) Rn(s − Rn(s)) and where Mc(t) is defined by 1 N hg, Mc(t)i = lim g(Wn(t))χ{n ∈ Kc} N→∞ N κc N n X=1 MEAN FIELD FOR RED 15 and, as a consequence, we can define 1 N W c(s)= hId, Mc(t)i = lim Wn(s)χ{n ∈ Kc}. N→∞ N Nκc n X=1 Consequently, from (2.11), t d W c(s) Q(t) − Q(0) = κc (1 − K(s)) − L 0 " c Rc(s) Z X=1 (2.13) − d W (s) + κc c (1 − K(s)) − L χ{Q(s) = 0} ds. c Rc(s) ! # X=1 The solutions Q(t) and R(t) = (R1(t),R2(t),... ) are deterministic, as are the Mc(t). Finally, the components of W are independent processes. Define N N d N SN 1 Wn (s) N W c (s) (2.14) N (s)= N = κc N , N Rn (s) Rc (s) nX=1 Xc=1 N where W c (s) is the average window size of connections in Kc among the N N S W first W1 ,...,WN . Define N (s) analogously from . From Theorem 2, we can define N 1 Wn(s) (2.15) S(s) = lim SN (s) = lim . N→∞ N→∞ N n Rn(s) X=1 In Section 4 we will prove Theorem 3 and show that there is only one strong solution to (2.13), (2.12) and that, in fact, the solution to (2.3) and (2.2) converges to this strong solution. N Theorem 3. If Assumptions 1 and 2 hold, then kMc (t) − Mc(t)kw, N N kQ (t) − Q(t)ks and kK (t) − K(t)ks converge to zero in probability, where + + Mc(t), q(t) and K(t) are deterministic functions of t ∈ R into M1(R ), R+ and R+, respectively, given in Theorem 2. N d N N Let P be the measure induced on D [0, T ] × C[0, T ] by ((W 1 ,..., W d ), QN ). Lemma 1. Under Assumptions 1 and 2, the measures P N are tight. Proof. We check the conditions for Theorem 12.3 in [4]. Condition (i) N N N is immediate since the sequences ((W 1 ,..., W d ),Q (t)) are bounded. 16 D. R. MCDONALD AND J. REYNIER In condition (ii) we are given positive constants ε and η. We can pick ∆ sufficiently small that the maximum growth of a window over a duration of length ∆ is less than ε/2; that is, pick ∆ < Tminε/2. Also pick ∆ suffi- ciently small that the probability of the event B that a Poisson process with intensity λ(T ) jumps twice within a duration ∆ is less than ηεa(T )/2. N Note that, by the construction of the window Wn , the event that the window is cut in half two times within a duration ∆ up to time T is contained in Bn, the event where Υn(t, λ(T )) has two jumps in an interval of duration ∆. Also, note the worst oscillation a window can make over a duration ∆ is ′ a(T ); that is, the biggest drop possible. Hence, the modulus wc(∆) of any N trajectory of W c , c = 1,...,d, as defined at (12.6) in [4] satisfies 1 N w′ (∆) ≤ (a(T )χ + ε/2). c N Bn nX=1 Hence, N N ′ 1 P (wc(∆) ≥ ε) ≤ P a(T ) χBn ≥ ε/2 N n ! X=1 2 ≤ P (B) ≤ η. εa(T ) Since we can make the above estimates simultaneously for each of the N N d classes, we see condition (ii) holds for the oscillations of (W 1 ,..., W d ). N N N The oscillations of Q (t) and of (R1 (t),...,Rd (t)) are uniformly bounded in C[0, T ] because each trajectory is uniformly Lipschitz. This shows the P N are tight. N Using Lemma 1, we can extract a subsequence Nk such that Q and N N ∞ ∞ ∞ (W 1 ,..., W d ) converge almost surely to Q and (W 1 ,..., W d ) in Skoro- N hod norm. The convergence of the components Wn follows. Unfortunately, ∞ ∞ ∞ we wouldn’t even know that the limits Q and (W 1 ,..., W d ) are deter- ministic. Using the above method and Jakubowski’s criterion (cf. [6], Theorem 3.6.4), N N we might be able to check that the measure valued processes M1 (t),...,Md (t) + in D([0, T ], M1(R )) are tight. We might even check that the bivariate mea- N N N N sure valued processes M1 (t − R (t), dv; t, dw),...,Md (t − R (t), dv; t, dw) + 2 in D([0, T ], M1(R ) ) are tight. We could then pick a convergent subse- quence and carry out the analysis in Section 6. We would obtain a limiting solution to Theorem 1. Unfortunately, we won’t know the solution is unique (and deterministic) because (1.1) and (1.3) don’t determine the solution since (M1(t),...,Md(t)) and Q(t) is not a state (because of the delay in the system). MEAN FIELD FOR RED 17 It may be possible to rectify this by defining the state at time t to be N N the entire trajectory of the measures M1 (t),...,Md (t) and Q(t) back at least one RTT. Indeed, the numerical procedure proposed in Section 7 shows how to maintain all this information to numerically solve (1.1) and (1.3). However, instead of trying to characterize tightness of such an ugly space, we will proceed in a more direct manner in the next section. 3. Existence of a limit. In this section we show the existence of the solution to (2.13), (2.12). 3.1. Modified system. We now introduce the modified system discussed in the introduction where QN is forced to be deterministic by modifying the equation for the evolution of the queue to (3.1). Then we extract a deterministic limit that turns to be a limit of our initial system. N N N Let K (t)= F (Q (t)) if Q(t) t d N N N (1 − K (s)) (3.1) = κc (EhId, Mc (s)i) N − L 0 " c Rc (s) Z X=1 d N − N N (1 − K (s)) + κc (EhId, Mc (s)i) N − L χ{Q(s) = 0} ds, c Rc (s) ! # X=1 N N N where W = (W1 ,..., WN ) satisfies the analogue of (2.3), where Wn(0) = wn, where N N N (3.2) Rc (t)= Tc + Q (t − Rc (t))/L, and where N N 1 N hId, Mc (s)i = N Wn (s)χ{n ∈ Kc}. κc N n X=1 N N If Q (t)= qmax, then the loss probability K (t) satisfies d N −1 N N EhId, Mc (t)i (3.3) K (t)=max pmax, 1 − L κc N . ( c Rc (t) ! ) X=1 N N We remark that when Q hits qmax, the loss probability K (t) jumps from N pmax to a value which keeps Q˙ (t) = 0 as long as d N N EhId, Mc (t)i κc N (1 − pmax) ≥ L. c Rc (t) X=1 N We will show below that EhId, Mc (t)i is Lipschitz continuous so, in fact, N N K (t)= pmax just before Q (t) leaves the boundary. 18 D. R. MCDONALD AND J. REYNIER c c c N c Solution for a given N. Let t (0) = 0, t (k+1)= t (k)+Tc +Q (t (k))/L c N c c such that t (k+1)−Rc (t (k+1)) = t (k). As long as we can define it, the se- quence (tc(k)) is increasing in k. Define Φc(t)=the first k such that tc(k) > t. We will construct our solution by recurrence from time ti to ti+1 by defin- c c ing ti+1 = minc t (Φ (ti)) starting from time t0 = 0. N N At time t0, we suppose W (t) and Q (t) are given (perhaps constant) N N for t ≤ 0= t0. We suppose (W (t), Q (t)) is defined for t ≤ ti, where ti is c a time such that ti = t (k) for some c and some k. This is certainly true at c c c time t0. Then Φ (ti) and t (Φ (ti)) are defined for all classes as is ti+1. Then if t ≤ ti+1, N N N Wn (t − Rn (t)) N N λn (t)= N N K (t − Rn (t)) Rn (t − Rn (t)) N can be defined, because, for each class, t − Rc (t) ≤ ti for s ≤ ti+1 by the N definition of ti+1 [recall (2.6)]. Hence, the point processes Nn (t) and the N trajectories W (t) are defined on [0,ti+1]. They are bounded and measur- able, thus, the expectations can be defined and, hence QN (t) can be defined. We have therefore checked the induction hypothesis up to time ti+1. c To conclude, we need to show that ti →∞. Notice that ti+d+1 ≥ min{t × c (Φ (ti) + 1) : c = 1,...,d} because otherwise the d + 1 values ti+j, j = 1,..., c c d + 1 must be chosen among the d values {t (Φ (ti))} and this is impossible. We conclude ti+d+1 ≥ ti + Tmin and, therefore, ti →∞ as i →∞. N N Lipschitz continuity of EhId, Mc (t)i and Q on [0, T ]. N N |EhId, Mc (t + h)i− EhId, Mc (t)i| N 1 N N ≤ N E|Wn (t + h) −Wn (t)|χ{n ∈ Kc} κc N n X=1 N h 1 N − N (3.4) ≤ N P (Wn (s )= Wn (s),t ≤ s ≤ t + h)χ{n ∈ Kc} Tmin κc N n X=1 N 1 N − N + a(T ) N P (Wn (s ) 6= Wn (s), κc N n X=1 (3.5) for some t ≤ s ≤ t + h)χ{n ∈ Kc}. The second term arises because even multiple jumps will create a difference less than the maximum window size. (3.4) is less than h/Tmin and tends to zero as h → 0. (3.5) is bounded by the probability a window makes a jump in an interval of length h. The N intensity function λ (t) is bounded by λ(t)= a(t)/Tmin, so the probability of a jump in an interval of length h is bounded by 1 − exp(a(T )h/Tmin). MEAN FIELD FOR RED 19 N N Hence, EhId, Mc (t)i is Lipschitz uniformly for N ∈ and t ∈ [0, T ]. From (3.1), it immediately follows that QN is also Lipschitz. N N Let mc (t)= EhId, Mc (t)i. N Lower bound on m˙ c (t). N N Lemma 2. The derivatives of mc (t) and Q (t) are bounded below uni- formly in N. Proof. Taking expectations, N EWn (t) − wn t 1 WN (s−) = E ds − n dN (s) RN (s) 2 n Z0 " n t 1 t WN (s−) WN (s − RN (s)) = ds − E n n n KN (s − RN (s)) ds. RN (s) 2 RN (s − RN (s)) n Z0 n Z0 n n Hence, N dEWn (t) dt 1 N − N N 1 N N = N − E[Wn (t )Wn (t − Rn (t))] N N K (t − Rn (t)) Rn (t) 2Rn (t − Rn (t)) 1 EWN (t−) a(t) ≥ − n . Tmax + qmax/L 2 Tmin N N Since EWn (t) ≤ a(t), it follows that dEWn (t)/dt is strictly bounded below by −C, where C is a positive constant that doesn’t depend on N or n. N Taking the average of the κcN windows with RTT, Tc shows the mc (t) are bounded below uniformly in N. N N 1 N EWn (t) Since R (t) ≤ Tmax +qmax/L, it follows that the derivative of N n N n=1 Rn (t) N is bounded below by the same constant. The fact that Q (t) is boundedP be- low uniformly in N follows from (3.1). Lemma 3. The sequence of functions KN (t) on [0, T ] is sequentially compact. N N N N Proof. As long as Q (t) 3.2. Existence of a limit for the modified system. In this section we shall be extracting subsequences of sequences, but we won’t reflect this in our notation until the end of this subsection. N N N Extraction of a limit for Q and EhId, Mc (t)i. Q (t) is deterministic. Moreover, the integrand in (3.1) is bounded by a constant B because the window sizes up to time t are bounded by a(T ) and RTT in greater than N Tmin > 0. Hence, Q is Lipschitz uniformly for N ∈ N and t ∈ [0, T ]. It follows that there is a subsequence and a Lipschitz function Q(t) such that QN (t) → Q(t) uniformly using the Ascoli–Arzel`atheorem plus the fact that a uniform limit of Lipschitz functions is Lipschitz. N We showed above that EhId, Mc (t)i is Lipschitz so again using the Ascoli–Arzel`atheorem we can take a further subsequence of N such that, N N for all c, mc (t)= EhId, Mc (t)i converges uniformly to a Lipschitz function mc(t). Note that taking the limit as N →∞ in Lemma 2 gives that the deriva- tives of mc(t) and Q(t) are bounded below. Convergence of the RTT. As a direct consequence of the convergence N N of Q , Rn (t) converges uniformly to Rn(t), where Rn(t)= Tn + Q(t − Rn(t))/L. MEAN FIELD FOR RED 21 Extraction of the limit for KN (t). By Lemma 3, we can extract a further subsequence so that KN converges in the Skorohod topology to a limit K N in D[0, T ]. If Q(t) d −1 mc(t) (3.6) max pmax, 1 − L κc . ( Rc(t) ! ) cX=1 First suppose t is not a point where K jumps. Then there exists a small in- terval I = (t − δ0,t], where Q(s)= qmax for s ∈ I. There are two possibilities: d d 1 − pmax 1 − pmax κcmc(t) >L or κcmc(t) = L. c Rc(t) c Rc(t) X=1 X=1 In the first case d κ m (t) 1−F (Q(t)) >L for t ∈ [t − δ, t] for some δ, c=1 c c Rc(t) N 0 <δ<δ0, sufficientlyP small. Since Q (t) converges to Q(t), we can pick N sufficiently large that QN (t) is in a narrow tube around Q(t). Moreover, d N N d c=1 κc mc (t) N c=1 κcmc(t) N (1 − F (Q (t))) → (1 − F (Q(t))). P Rc (t) P Rc(t) The right-hand side is strictly greater than L for t ∈ [ρ−δ, ρ] for δ sufficiently small and the same is true of the left-hand side for sufficiently large N. This shows that, for N sufficiently large, QN (t) will hit the boundary at a time inside [t − δ, t). Hence, for N sufficiently large, K(t) = lim KN (t) N→∞ d N −1 N mc (t) = lim 1 − L κc N→∞ N c Rc (t) ! ! X=1 d −1 mc(t) = 1 − L κc . Rc(t) ! ! Xc=1 In the second case, K(t)= pmax = F (qmax), so d −1 mc(t) K(t)=max pmax, 1 − L κc . ( c Rc(t) ! ) X=1 We have therefore established K(t)= F (Q(t)) if Q(t) d −1 mc(t) (3.7) K(t)=max pmax, 1 − L κc ( c Rc(t) ! ) X=1 22 D. R. MCDONALD AND J. REYNIER if Q(t)= qmax at all times t where K(t) doesn’t jump. Since the derivatives of mc(t) and Q(t) are bounded below, there is a time interval to the right of any jump time free of jumps. Since K(t) is right continuous, it follows that d −1 mc(t) K(t) = 1 − L κc c Rc(t) ! X=1 at jump times. N N Uniform convergence of Wn . We fix some coordinate n. Clearly, Wn (t) ≤ N a(t) for all 0 ≤ t ≤ T , so it follows that λn (t) is uniformly bounded by λ(t)= N a(t)/Tmin for all 0 ≤ t ≤ T and all N ≥ n. Consequently, Nn (t) ≤ N n(t), t ∞ 1 where N n(t)= 0 0 [0,λ(s))(u)Υn(du, ds). Hence, if for some trajectory ω N of Υn, N n(T ) ≤RmR, then Nn (T ) ≤ m for all n and N. For any trajectory ω of Υn, we can solve the system t 1 W (s−) (3.8) W (t) − w = ds − n dN (s) , n n R (s) 2 n Z0 n where Wn(t)= wn, for t ≤ 0, for n = 1,... and t ∞ 1 Nn(t)= [0,λn(s))(u)Υn(du, ds) Z0 Z0 and Wn(s − Rn(s)) λn(s)= K(s − Rn(s)). Rn(s − Rn(s)) Now consider the solution of (3.8) from jump point to jump point of Nn. Let Tm, m = 1, 2,..., be the jumps of Nn in [0, T ] corresponding to jump points (Xm,Ym), m = 1, 2,..., of Υn which satisfy Tm = Xm and Ym ≤ λn(Xm). The solution of (3.8) for t ≥ Tm is the deterministic addi- tive increase of the window size until time Tm+1 and this has zero chance of hitting (Xm+1,Ym+1) which is chosen according to a Poisson process. Hence, with probability one, the trace (t, λn(t)) for 0 ≤ t ≤ T will avoid the points of Υn(du, ds) for a given ω, so we can put a band of width ε around the trace N (t, λn(t)). Now as long as λn (t) lies in this band, then the point processes N N −1 Nn and Nn are identical. This will be the case if (Rn (t)) is sufficiently −1 N close to (Rn(t)) because the solutions Wn (t) and Wn(t) starting from the same point wn will rise together inside the band between jumps and at jumps will be cut in half together. We conclude that, with probability one, N Wn (t) converges uniformly to Wn(t) as N →∞. MEAN FIELD FOR RED 23 N N N Equation for the limit Wn. Since Rn (s), K (s) and Wn (s) converge uniformly to Rn(s), K(s − Rn(s)) and Wn(s), respectively, it follows that N λn (t) converges uniformly to Wn(s − Rn(s)) λn(t)= K(s − Rn(s)) Rn(s − Rn(s)) on [0, T ]. It therefore follows that both sides of t 1 WN (t) − w = ds n n RN (s) Z0 n t ∞ N − Wn (s ) − 1 N (u)Υn(du, ds) 2 [0,λn (s)) Z0 Zu=0 converge uniformly on D[0, T ] to (3.8). 3.3. Equations for the limit (W,Q). Determining Wc. Since Q(t) is deterministic, the equations in (3.8) are independent. This, in turn, means that 1 N EWc(s)= Wc(s) = lim Wn(s)χ{n ∈ Kc}. N→∞ N κc N n X=1 Note that this limit is not taken along a subsequence of N. This essentially follows from the law of large numbers. It suffices to consider the Wn(s)= f(Wn(0), Υn) defined by (3.8). f is defined on E = + ω [0, ∞) × DR+ [0, T ], where Wn(0) ∈ R = [0, ∞) and Υn ∈ DR+ [0, T ], the space of cadlag functions. E is a metric space with metric m = e⊕d, where e is the Euclidean metric on [0, ∞) and d is the Skorohod metric on DR+ [0, T ]. We have shown above that, for any initial point w0, the set of trajectories ω0 ω0 Υn (Υn is the point process Υn evaluated at the sample point ω0) such that ω0 the associated graph (t, λn(t)) does not hit any of the jump points of Υn has probability one. f is continuous on this set by the arguments used above. ω0 If the graph (t, λn(t)) avoids the points of Υn , then according to [4] or [10], ω1 ω0 for a point q = (w1, Υn ) ∈ E close to p = (w0, Υn ), we can find a strictly increasing mapping θ of [0, T ] onto itself with sup[0,T ] |θ(t) − t|≤ γ(θ), where ω1 ω0 γ(θ) → 0 as q → p such that Υn (θ(t)) is uniformly close to Υn (t). Notice that this means the jumps occur at the same times. ω1 The solution to (3.8) for Wn and λn for p and for (w1, Υn (θ(t))) are ω0 therefore uniformly close. Hence, at any fixed time s where Υn has no ω1 ω1 ω0 jumps, we will have Wn (s) and λn (s) uniformly close to Wn (θ(s)) and ω0 λn (θ(s)). Since θ(s) is arbitrarily close to s and since the chance of a jump arbitrarily near time s tends to zero, it follows that, for q sufficiently close 24 D. R. MCDONALD AND J. REYNIER ω1 ω0 to p, Wn (s) is arbitrarily close to Wn (s). This means f is continuous at ω0 ω0 (w0, Υn ) as long as Υn has no jumps at s and this has probability one. By hypothesis, 1 N lim δW (0)χ{n ∈ Kc}→ µc N→∞ N n κc N n X=1 and the Υn are i.i.d. independent Poisson processes, so the empirical measure of the pairs (Wn(0), Υn) converges; that is, N 1 N N δ(Wn(0),Υn)χ{n ∈ Kc }→ µc ⊗ ν, κc N n X=1 where ν is the distribution of Υn on DR+ [0, T ]. The result now follows since f is bounded and the set of discontinuities has probability zero relative to the limiting measure. N Determining Mc. In addition, the above means that Mc (t) converges weakly to Mc(t) almost surely P . For any continuous function with compact support, define the limiting measure Mc(t) by N 1 N hg, Mc(t)i = lim N g(Wn (t))χ{n ∈ Kc}. N→∞ κc N nX=1 The deterministic limit exists almost surely by the argument above. This also means that mc(t)= hId, Mc(t)i, so when Q(t)= qmax, K(t) satisfies d −1 N (1 − K(t)) K(t)=max pmax, 1 − L κc hId, Mc(s)i . ( c Rc(t) ! ) X=1 Moreover, N 1 N Wc(t) := lim Wn (t)χ{n ∈ Kc} N→∞ N κc N n X=1 = hId, Mc(t)i. Existence of a strong solution. Hence, along the subsequence N, we ob- tain an almost sure limit point (W, Q, (M1,..., Md)) which satisfies the modified system: (3.8) and Q(t) − Q(0) t d Wc(s) (3.9) = κc (1 − K(s)) − L 0 " c Rc(s) Z X=1 d − Wc(s) + κc (1 − F (Q(s))) − L χ{Q(s) = 0} ds, c Rc(s) ! # X=1 MEAN FIELD FOR RED 25 where 1 N Wc(t) := lim Wn(t)χ{n ∈ Kc} N→∞ N κc N n=1 (3.10) X = hId, Mc(t)i. We call the above a strong solution associated with the derived subse- quence. Moreover, (3.10) is true along all sequences. Hence, the solution to (3.9) and (3.8) is a strong solution of the system in Theorem 2. Extension to the timeout and slow-start phases. If we consider the ex- A tended system with timeouts and slow-start, we have to define Nc (t), the proportion of the connections from class c in congestion avoidance at time t. U S There will be similar proportions Nc (t) in timeout and Nc (t) in slow-start. The equation for the queue (neglecting boundary terms) becomes N d dQ (t) N A N N S N = [κc Nc (t)hId, Mc (t)i + κc Nc (t)hId,Hc (t)i] − L, dt c X=1 N where Mc (t) is the histogram of the window sizes of connections in conges- N tion avoidance and Hc (t) is the histogram of the window sizes of connections in slow start. We can force the queue to be deterministic by considering the modified system (again neglecting boundary terms): N d dQ (t) N A N N S N = [κc E(Nc (t))EhId, Mc (t)i + κc E(Nc (t))EhId, Hc (t)i] − L. dt c X=1 The window equations for the modified system are uncoupled as before. We can again pick subsequences so that QN (t) converges and then further sub- A U S sequences so that E(Nc (t)), E(Nc (t)) and E(Nc (t)) converge. As before, the limiting system is in fact a strong solution to the extended system. 4. Uniqueness of strong solutions. We have constructed a strong solution (W, R,Q,M) to (2.13) and (2.12). Our approach is to prove L1 convergence of (WN ,QN ) to the strong solution on [0, T ]: Proposition 1. Under Assumptions 1 and 2, DN (T ) → 0 as N →∞, where N N DN (t) := E sup|Q (τ) − Q(τ)| + EkW (t) − W(t)k, τ≤t where N WN W 1 N k (t) − (t)k = sup |Wn (τ) − Wn(τ)|. N 0≤τ≤t nX=1 26 D. R. MCDONALD AND J. REYNIER N N We conclude that Q and, hence, Rn converge in probability to Q and N Rn, respectively. Hence, for each window Wn solving (2.12), the intensity N λn (s) converges to Wn(s − Rn(s)) λn(s)= K(s − Rn(s)). Rn(s − Rn(s)) N This implies that each window Wn converges in probability to Wn in Sko- rohod norm. We also have a stronger result than the weak convergence of WN to W: N Corollary 1. If Assumptions 1 and 2 hold, then kMc (t)−Mc(t)kw → 0 in probability for any t ≤ T . Proof. For any bounded Lipschitz function g, N lim sup|Ehg, Mc (t)i− Ehg, Mc(t)i| N→∞ N 1 N = limsup E N g(Wn (t))χ{n ∈ Kc} N→∞ " κc N n=1 X N 1 − lim N g(Wn(t))χ{n ∈ Kc} N→∞ κc N # n=1 X N 1 N ≤ lim sup N E[g(Wn (t)) − g(Wn(t))]χ{n ∈ Kc} N→∞ κ N c n=1 X N 1 + limsup N g(Wn(t))χ{n ∈ Kc}− Ehg, Mc(t)i N→∞ κ N c n=1 X N 1 N ≤ lim sup N E|g(Wn (t)) − g(Wn(t))| N→∞ κc N nX=1 N 1 N ≤ Cg lim sup E|Wn (t) − Wn(t)|, N→∞ N n X=1 N where Cg is the Lipschitz constant divided by min{κc }. Hence, N lim sup|Ehg, Mc (t)i− Ehg, Mc(t)i| N→∞ N 1 N ≤ Cg lim sup E sup |Wn (τ) − Wn(τ)| N→∞ N n τ≤t X=1 N ≤ Cg lim supEkW (t) − W(t)k N→∞ MEAN FIELD FOR RED 27 → 0. Since we can construct a convergence determining sequence based on pos- itive bounded Lipschitz functions, the result follows immediately. N N N A similar argument shows that kMc (t − Rc (t), ·; t, ·) − Mc(t − Rc (t), ·; t, ·)kw → 0 in probability for any t ≤ T . To prove DN (t) → 0 as N →∞, we will establish a Gronwall inequal- t ity: DN (t) ≤ BN (t)+ C 0 DN (s) ds, where BN (t) → 0 as N →∞. It is eas- ier to establish the Gronwall inequality for Gentle RED where the drop R probability function Fδ(q) rises from pmax at qmax to 1 at qmax + δ and, hence, is Lipschitz. For Gentle RED, the results in Section 4.1 apply with N t S S ρ = T . Moreover, we will see below that B (t)= E 0 | N (s) − (s)| ds, where S and S were defined by (2.14) and (2.15). This means we can N R even get a rate of convergence using the Gronwall inequality since DN (t) ≤ N t N B (t)+ C 0 B (s) exp(C(t − s)) ds. Unfortunately for RED, when QN hits the boundary q , the dynamics R max change because F is not Lipschitz at qmax. Our solution to proving Propo- sition 1 is to prove convergence on [0, ρ], where ρ to the (deterministic) time when Q(t) first hits qmax. This is done in Section 4.1. In Section 4.2 we show the convergence of the transmission rates of the prelimit extends for a time Tmin beyond ρ (and, in fact, Tmin beyond any point in time). This holds because the transmission rates are determined one RTT in the past. This allows us to extend our proof to cross the boundary when Q hits qmax. Then, in Section 4.3, we prove convergence on the interval [0,σ], where ρ<σ and σ is the first time Q(t) leaves the boundary; that is, Q(t) Lemma 4. Q˙ (t) > −L + δ, where δ> 0 and Q˙ (t) ≤ a(t)/Tmin − L. Proof. Taking expectations of (2.12) and following the steps of Lemma 2, we get − 1 EWn(t ) a(t) EWn(t) − wn ≥ − . Tmax + qmax/L 2 Tmin Since the above bound is positive when EWn(t) is small, it follows that EWn(t) is uniformly bounded away from zero for all t and n. It therefore follows that W c(t) is uniformly bounded away from zero for all t, n and c. From (2.13), this means that Q˙ (t) > −L. The second inequality follows immediately from (2.13). 28 D. R. MCDONALD AND J. REYNIER Lemma 5. For 0 ≤ s ≤ t ≤ T , we have that N N N sup |Rn (s) − Rn(s)|, sup |Rn (s − Rn (s)) − Rn(s − Rn(s))| 0≤s≤t 0≤s≤t and N N sup |Q (s − Rn (s)) − Q(s − Rn(s))| 0≤s≤t N are bounded by C sup0≤s≤t−Tn |Q (s) − Q(s)|. N N Proof. Let φn(s)= Tn +Q(s)/L, gn(s)= s+φn(s), φn (s)= Tn +Q (s)/L N N N N and gn (s)= s + φn (s). For any t, let gn(u)= t and let gn (u )= t. Note N N N N that u ≤ t − Tn and u ≤ t − Tn. Hence, Rn (t) − Rn(t)= φn (u ) − φn(u)= N N N u − u. Suppose u ≤ u (or u ≤ u), the function gn increases (or de- N N N N N N creases) from gn(u)= t = gn (u ) to gn(u ) [from gn(u ) to gn (u )] by an N amount of at least δ(u − u)/L since, by Lemma 4, the derivative of φn(u) δ N N N is bounded below by δ/L. It follows that L |u − u| ≤ |gn (u ) − gn(u)|≤ N N sup0≤s≤t−Tn |gn (s) − gn(s)|≤ sup0≤s≤t−Tn |Q (s) − Q(s)|/L. Hence, N N N |Rn (t) − Rn(t)| = |u − u|≤ sup |Q (s) − Q(s)|/δ 0≤s≤t−Tn and this gives the first result. In the same manner we obtained (2.5), we have 1 (1 − R˙ n(t)) = 1+ Q˙ (t − Rn(t))/L and from Lemma 4, Q˙ (s − Rn(s))/L R˙ n(s)= 1+ Q˙ (s − Rn(s))/L ≤ |a(s)/Tmin − L|/(δ/L). Consequently, using the mean value theorem for s ≤ t, we have N N |Rn(s − Rn (s)) − Rn(s − Rn(s))|≤ C|Rn (s) − Rn(s)| ≤ C sup |QN (u) − Q(u)|. 0≤u≤t−Tn Finally, N N sup |Rn (s − Rn (s)) − Rn(s − Rn(s))| 0≤s≤t N N N ≤ sup |Rn (s − Rn (s)) − Rn(s − Rn (s))| 0≤s≤t MEAN FIELD FOR RED 29 N + sup |Rn(s − Rn (s)) − Rn(s − Rn(s))| 0≤s≤t N ≤ sup |Rn (s) − Rn(s)| 0≤s≤t N + sup |Rn(s − Rn (s)) − Rn(s − Rn(s))| 0≤s≤t ≤ C sup |QN (u) − Q(u)| 0≤u≤t−Tn by the first result and the inequality above. The third result follows because N N sup |Q (s − Rn (s)) − Q(s − Rn(s))| 0≤s≤t N N N ≤ sup |Q (s − Rn (s)) − Q(s − Rn (s))| 0≤s≤t N + sup |Q(s − Rn (s)) − Q(s − Rn(s))| 0≤s≤t N N ≤ sup |Q (s) − Q(s)| + C|Rn (s) − Rn(s)| 0≤s≤t−Tn since the derivative of Q is bounded and Rn ≥ Tn ≤ C sup |QN (u) − Q(u)| by the first result. 0≤u≤t−Tn 4.1. Convergence away from the boundary. We start with QN (0) = q(0) in the interior: • 0 N WN W 1 N k (t) − (t)k = sup |Wn (τ) − Wn(τ)|, N n 0≤τ≤t∧ρN X=1 where τ is a stopping time with respect to Ft. Define N N DN (t) := E sup |Q (τ) − Q(τ)| + EkW (t) − W(t)k. τ≤t∧ρN We will establish a Gronwall inequality: 30 D. R. MCDONALD AND J. REYNIER N t Proposition 2. DN (t) ≤ B (t) + C 0 DN (s) ds for t ≤ ρ and sup BN (t) → 0 as N →∞, where C is a canonical constant throughout t≤ρ R this calculation (which unfortunately depends on F ′). Moreover, ρN con- verges to ρ in probability. We just group all universal constants which do not depend on N into C. ′ We note that one of the factors in C is the Lipschitz constant, F (q),q Estimate for Q. E sup |QN (τ) − Q(τ)| 0≤τ≤t∧ρN τ SN N S + E sup | N (s)(1 − K (s)) − (s)(1 − K(s))| ds 0≤τ≤t∧ρN Z0 τ SN S N (4.1) ≤ E sup |( N (s) − N (s))(1 − K (s))| ds 0≤τ≤t∧ρN Z0 τ N + E sup |( SN (s) − S(s))(1 − K (s))| ds 0≤τ≤t∧ρN Z0 τ + E sup | S(s)(KN (s) − K(s))| ds 0≤τ≤t∧ρN Z0 and E sup |QN (τ) − Q(τ)| 0≤τ≤t∧ρN τ N N 1 Wn (s) Wn(s) ≤ E sup N − ds 0≤τ≤t∧ρN 0 N Rn (s) Rn(s) ! Z n=1 X τ + E sup | SN (s) − S(s)| ds 0≤τ≤t∧ρN Z0 τ + E sup S(s)|KN (s) − K(s)| ds 0≤τ≤t∧ρN Z0 t N N 1 Wn (τ) Wn(τ) ≤ E sup N − ds N N R (τ) R (τ) Z0 n=1 0≤τ≤s∧ρ n n X τ 1 + BN + E sup a(s)|KN (s) − K(s)| ds, N T 0≤τ≤t∧ρ Z0 min MEAN FIELD FOR RED 31 N t S S where B = E 0 | N (s) − (s)| ds. Next, R N Wn (τ) Wn(τ) E sup N − 0≤τ≤s∧ρN Rn (τ) Rn(τ) N 1 ≤ E sup |Wn (τ) − Wn(τ)| N N R (τ) 0≤τ≤s∧ρ n 1 1 + E sup |Wn(τ)| N − N R (τ) R (τ) 0≤τ≤s∧ρ n n 1 N (4.2) ≤ E sup |Wn (τ) − Wn(τ)| T N min 0≤τ≤s∧ρ a(s) N + 2 E sup |Rn (τ) − Rn(τ)| Tmin 0≤τ≤s∧ρN 1 N ≤ E sup |Wn (τ) − Wn(τ)| T N min 0≤τ≤s∧ρ 1 N + 2 a(s)CE sup |Q (τ) − Q(τ)| , Tmin 0≤τ≤s∧ρN using Lemma 5. Moreover, τ 1 E sup a(s)|KN (s) − K(s)| ds N T 0≤τ≤t∧ρ Z0 min τ ≤ CE sup |KN (s) − K(s)| ds 0≤τ≤t∧ρN Z0 (4.3) τ ≤ CE sup |F (QN (s)) − F (Q(s))| ds 0≤τ≤t∧ρN Z0 t ≤ C E sup |QN (τ) − Q(τ)| ds . Z0 0≤τ≤s∧ρN Hence, E sup |QN (τ) − Q(τ)| 0≤τ≤t∧ρN t N 1 1 N ≤ E sup |Wn (τ) − Wn(τ)| ds 0 Tmin N n 0≤τ≤s∧ρN Z X=1 32 D. R. MCDONALD AND J. REYNIER t 1 N N (4.4) + 2 a(s)CE sup |Q (τ) − Q(τ)| ds + B1 T N Z0 min 0≤τ≤s∧ρ t + C E sup |QN (τ) − Q(τ)| ds Z0 0≤τ≤s∧ρN t N ≤ B1 + C DN (s) ds. Z0 Estimate for W . From (2.12), N 1 N E sup |Wn (τ) − Wn(τ)| N n 0≤τ≤t∧ρN X=1 τ 1 N 1 1 (4.5) ≤ E sup E N − ds N N R (s) R (s) 0≤τ≤t∧ρ Z0 n=1 n n ! X N τ 1 1 N − N + E sup [Wn (s ) dNn (s) 2 N N n=1 0≤τ≤t∧ρ Z0 X (4.6) − − Wn(s ) dNn(s)] . Again, (4.5) is bounded by t a(s) N 2 E sup |Rn (τ) − Rn(τ)| ds T N Z0 min 0≤τ≤s∧ρ (4.7) t ≤ C E sup |QN (τ) − Q(τ)| ds. Z0 0≤τ≤s∧ρN N By the definition of Nn , τ W N (s−) n dN N (s) 2 n Z0 τ ∞ N − Wn (s ) = χ N (u)Υn(du, ds). 2 [0,λn (s)) Zs=0 Zu=0 Consequently, N τ 1 N − N − E sup [Wn (s ) dNn (s) − Wn(s ) dNn(s)] N N n=1 0≤τ≤t∧ρ Z0 X N τ ∞ 1 N − ≤ E sup |W (s )χ N (u) n [0,λn (s)) N n 0≤τ≤t∧ρN 0 u=0 X=1 Z Z MEAN FIELD FOR RED 33 − − Wn(s )χ[0,λn(s))(u)|Υn(du, ds) N t∧ρN ∞ 1 N − = E |W (s )χ N (u) n [0,λn (s)) N 0 u=0 nX=1 Z Z − − Wn(s )χ[0,λn(s))(u)|Υn(du, ds) N t∧ρN ∞ 1 N − = E |W (s )χ N (u) n [0,λn (s)) N n 0 u=0 X=1 Z Z − − Wn(s )χ[0,λn(s))(u)| du ds . Hence, N τ 1 N − N − E sup [Wn (s ) dNn (s) − Wn(s ) dNn(s)] N N n=1 0≤τ≤t∧ρ Z0 X N t ∧ρN 1 N − − N ≤ E |Wn (s ) − Wn(s )|λn (s) ∧ λn(s) ds N n 0 X=1 Z N t∧ρN 1 N − − N + E |Wn (s ) ∨ Wn(s )| · |λn (s) − λn(s)| ds N 0 nX=1 Z t N a(s) 1 N (4.8) ≤ E sup |Wn (τ) − Wn(τ)| ds 0 Tmin "N n 0≤τ≤s∧ρN # Z X=1 N t∧ρN 1 N (4.9) + E a(s)|λn (s) − λn(s)| ds , N n 0 X=1 Z N where λn (s) and λn(s) are less than (wn + s/Tmin)/Tmin = a(s)/Tmin. Also, N |λn (s) − λn(s)| N N N 1 N N ≤ |Wn (s − Rn (s)) − Wn(s − Rn (s))| N N K (s − Rn (s)) Rn (s − Rn (s)) N 1 N N + |Wn(s − Rn (s)) − Wn(s − Rn(s))| N N K (s − Rn (s)) Rn (s − Rn (s)) 1 1 N N + |Wn(s − Rn(s))| N N − |K (s − Rn (s))| Rn (s − Rn (s)) Rn(s − Rn(s)) 1 N N + Wn(s − Rn(s)) |K (s − Rn (s)) − K(s − Rn(s))|. Rn(s − Rn(s)) Hence, N N N N (4.10) |λn (s) − λn(s)| ≤ |Wn (s − Rn (s)) − Wn(s − Rn (s))|/Tn 34 D. R. MCDONALD AND J. REYNIER N (4.11) + |Wn(s − Rn (s)) − Wn(s − Rn(s))|/Tn N N 2 (4.12) + a(s)|Rn (s − Rn (s)) − Rn(s − Rn(s))|/Tn N N (4.13) + a(s)|K (s − Rn (s)) − K(s − Rn(s))|/Tn. t∧ρN N We must bound E( 0 |λn (s) − λn(s)| ds), so we must bound the ex- pectation of the integral of each of the above terms. The first (4.10) satisfies R t∧ρN N N N E |Wn (s − Rn (s)) − Wn(s − Rn (s))| ds/Tn 0 Z t 1 N ≤ E sup |Wn (τ) − Wn(τ)| ds . T N min Z0 0≤τ≤s∧ρ The second term (4.11) is bounded by t∧ρN N E |Wn(s − Rn (s)) − Wn(s − Rn(s))| ds Tn Z0 t∧ρN . 1 ≤ E du ds − N ∧ − − N ∨ − R (u) Z0 Z[s Rn (s) s Rn(s),s Rn (s) s Rn(s)] n 1 t∧ρN + E 2 − N ∧ − − N ∨ − Z0 Z[(s Rn (s)) (s Rn(s)),(s Rn (s)) (s Rn(s))] − Wn(u ) dNn(u) ds . Note that t∧ρN N N χ{(s − Rn (s)) ∧ (s − Rn(s)) ≤ u ≤ (s − Rn (s)) ∨ (s − Rn(s))} ds Z0 N N N N = (u + φn (u) ∨ φn(u)) ∧ (t ∧ ρ ) − (u + φn (u) ∧ φn(u)) ∧ (t ∧ ρ ), N N where φn(u)= Tn + Q(u)/L and φn (u)= Tn + Q (u)/L. Hence, t∧ρN N E |Wn(s − Rn (s)) − Wn(s − Rn(s))| ds Tn 0 Z . 1 t∧ρN ≤ E |R (s) − RN (s)| ds T n n min Z0 1 t∧ρN + E |φN (u) ∨ φ (u) − φN (u) ∧ φ (u)|W (u−)λ (u) du 2 n n n n n n Z0 1 t∧ρN ≤ E |R (s) − RN (s)| ds T n n min Z0 MEAN FIELD FOR RED 35 1 t∧ρN a2(u) + E |φN (u) − φ (u)| du 2 n n T Z0 min C t ≤ E sup |Q(s) − QN (s)| ds T N min Z0 0≤τ≤s∧ρ t 1 a2(s) + E sup |QN (τ) − Q(τ)| ds 2 T L N Z0 min 0≤τ≤s∧ρ t ≤ C E sup |QN (τ) − Q(τ)| ds, Z0 0≤τ≤s∧ρN where C is a constant. To bound the third term (4.12), for s ≤ t ∧ ρN , N N N |Rn (s − Rn (s)) − Rn(s − Rn(s))|≤ C sup|Q (τ) − Q(τ)| τ≤s by Lemma 5. Taking expectations gives t∧ρN a (s) E n |RN (s − RN (s)) − R (s − R (s))| ds T 2 n n n n Z0 n t ≤ C E sup |QN (τ) − Q(τ)| ds. Z0 0≤τ≤s∧ρN Similarly, to bound the fourth term (4.13), for s ≤ t ∧ ρN , a(s) N N |K (s − Rn (s)) − K(s − Rn(s))| Tn N N N ≤ C[|K (s − Rn (s)) − K(s − Rn (s))| (4.14) N + |K(s − Rn (s)) − K(s − Rn(s))|] N N N ≤ C[|Q (s − Rn (s)) − Q(s − Rn (s))| N + |Q(s − Rn (s)) − Q(s − Rn(s))|] N N ≤ C sup|Q (τ) − Q(τ)| + |Rn (s) − Rn(s)| τ≤s (4.15) ≤ C sup|QN (τ) − Q(τ)|, τ≤s since Q has a bounded derivative. Taking expectations shows the fourth term is bounded by t C E sup |QN (τ) − Q(τ)| ds. Z0 0≤τ≤s∧ρN 36 D. R. MCDONALD AND J. REYNIER Hence, we can bound (4.9) by t C E sup |QN (τ) − Q(τ)| Z0 " 0≤τ≤s∧ρN N 1 N + E sup E|Wn (τ) − Wn(τ)| ds N n 0≤τ≤s∧ρN # X=1 t ≤ C DN (s) ds. Z0 Putting together (4.8), (4.9) and (4.7), we get N 1 N E sup |Wn (τ) − Wn(τ)| N n 0≤τ≤t∧ρN X=1 t ≤ C E sup |QN (τ) − Q(τ)| Z0 " 0≤τ≤t∧ρN N 1 N + E sup |Wn (τ) − Wn(τ)| ds. N n 0≤τ≤t∧ρN # X=1 Hence, t N EkW (t) − W(t)k≤ C DN (s) ds. Z0 N Finally, add in (4.4) and we get our Gronwall inequality: DN (t) ≤ B (t)+ t C 0 DN (s) ds. R 4.2. When crossing or grazing the boundary. The construction of the Gronwall inequality in the previous subsection is fairly standard and is suf- ficient for proving mean field convergence when Gentle RED is used. This subsection, however, resolves the fundamental problem when Q(t) just grazes qmax when RED is used. For example, if we changed the dynamics on the N boundary to cause the queue to rapidly grow when Q hit qmax, then Q would not converge to Q along many sample paths. Those paths where QN just missed qmax would drop, while those that hit qmax would rise. Fortunately, our system allows us to resolve this problem. Using the delay, N we show below that sup0≤τ≤ρ+Tmin |SN (τ) − S(τ)|→ 0 as N →∞. In other words, we can extend the convergence of the transmission rate one RTT into the future beyond the first time to hit the boundary because the evolution of the windows for [ρ, ρ + Tmin] is already determined at time ρ. This forces QN to follow Q for one RTT after hitting the boundary. Hence, if S(t)(1 − N pmax) >L for t ∈ (ρ, ρ+δ), where δ> 0, then we are assured the prelimit Q MEAN FIELD FOR RED 37 hits the boundary close to where Q hits the boundary and that ρN converges to ρ. Moreover, we are assured that, for N large enough, there will be a last exit time ηN , where ρN ≤ ηN < ρ + δ when QN jitters off the boundary and that E|ηN − ρ|→ 0. We can therefore define σN unambiguously as the infimum of those times after ρ + δ that QN leaves the boundary. When Q grazes the boundary at time ρ, then S(t)(1 − pmax) 1 1 N τ (4.17) + E sup [W N (s−) dN N (s) − W (s−) dN (s)] . 2 N n n n n n=1 ρ≤τ≤t Zρ X Again, (4.16) is bounded by a(T ) T E sup |RN (τ) − R (τ)| min T 2 n n min 0≤τ≤ρ+Tmin ≤ CE sup |QN (τ) − Q(τ)| 0≤τ≤ρ by Lemma 5. We have already shown convergence up until ρ, so this tends to zero as N →∞. 38 D. R. MCDONALD AND J. REYNIER We estimate (4.17) as we did (4.6) to obtain the following terms corre- sponding to (4.8) and (4.9): 1 N τ E sup [W N (s−) dN N (s) − W (s−) dN (s)] N n n n n n=1 ρ≤τ≤t Zρ X t N a(s) 1 N (4.18) ≤ E sup |Wn (τ) − Wn(τ)| ds ρ Tmin " N n ρ≤τ≤s # Z X=1 N t 1 N (4.19) + E(a(s)|λn (s) − λn(s)| ds). N n ρ X=1 Z We can bound (4.19) using the decomposition given by (4.10), (4.11), (4.12) and (4.13). Since each of these terms involve times more than Tmin in the past, it is not hard to see that the integral of term (4.10) is bounded by 1 N N CE[ N n=1 sup0≤t≤ρ |Wn (t) − Wn(t)|] and that term (4.11) is bounded by N CE(supP0≤τ≤ρ |Q (τ) − Q(τ)|), as is the integral of (4.12) and (4.13). All of these bounds tend to zero as N →∞. N t We conclude that H(t) ≤ B (t)+ C 0 H(s) ds for 0 ≤ t ≤ ρ + Tmin, where BN (t) again denotes a term which goes to zero as N →∞. Using the Gron- R wall inequality, we conclude H(t) → 0 as N →∞. Hence we have convergence of the windows over [ρ, ρ + Tmin]. Refining the estimate (4.2), we get W N (τ) W (τ) E sup n − n RN (τ) R (τ) 0≤τ≤ρ+Tmin n n 1 ≤ E sup |W N (τ ) − W (τ)| T n n min 0≤τ≤ρ+Tmin 1 N + 2 a(s)CE sup |Q (τ) − Q(τ)| , Tmin 0≤τ≤ρ using Lemma 5. Both these estimates tend to zero as N →∞, so we have N shown sup0≤τ≤ρ+Tmin |SN (τ) − SN (τ)|→ 0 as N →∞. This completes the argument. Once we have convergence of the transmission rate until ρ+Tmin, the case where Q enters the boundary becomes obvious and, clearly, ρN converges to ρ. The case where Q grazes the boundary so ρ = σ is also resolved because, in probability, N SN (t)(1 − pmax) → S(t)(1 − pmax) 4.3. Mean-field convergence on the boundary. We now prove convergence on the interval t ∈ [0,σ]. Assume σ > ρ. In Section 4.2 we showed how to handle the case when the queue grazes the boundary. Define σN to be the N infimum over times greater than ρ + δ that Q is less than qmax, where δ<σ − ρ. For any 0 ≤ t ≤ σ, redefine N WN W 1 N k (t) − (t)k = sup |Wn (τ) − Wn(τ)|, N n 0≤τ≤t∧σN X=1 N where τ is a stopping time with respect to Ft and σ is the end of the first sojourn on the boundary by QN . Redefine N N DN (t) := E sup |Q (τ) − Q(τ)| + EkW (t) − W(t)k. τ≤t∧σN N t Again, we will establish a Gronwall inequality: DN (t) ≤ B1 (t)+C 0 DN (s) ds for t ∈ [0,σ] where sup BN (t) → 0 as N →∞. t≤σ 1 R The calculation is almost the same as in Section 4.1. We need only improve the bounds on the terms (4.3) and (4.9) via (4.14). For s ≤ t ∧ σN , where t ≤ σ, there are three possibilities; both QN (s) and Q(s) are away from the boundary or both are on the boundary or one is on the boundary and the N other isn’t. If Q (s) N N If Q (s)= Q(s)= qmax, K (s) and K(s) are given by N N SN (s)(1 − K (s)) = L and S(s)(1 − K(s)) = L, 40 D. R. MCDONALD AND J. REYNIER N with K (s), K(s) >pmax. Note this means N (4.20) SN (s) ≥ L/(1 − pmax) and S(s) ≥ L/(1 − pmax). N Hence, if Q (s)= Q(s)= qmax, N N |K (s) − K(s)| = |L/SN (s) − L/S(s)| 2 (1 − p ) N ≤ max |S (s) − S(s)| L N N ≤ C(|SN (s) − SN (s)| + |SN (s) − S(s)|) ≤ CDN (s)+ |SN (s) − S(s)|, using the same estimate as (4.1). Hence, to bound expression (4.3), for t ≤ σ, τ E sup |KN (s) − K(s)| ds 0≤τ≤t∧σN Z0 t t ≤ C DN (s) ds + |SN (s) − S(s)| ds Z0 Z0 τ N (4.21) + E sup [χ{Q (s) N However, if Q(s − Rn (s)) = Q(s − Rn(s)) = qmax, N |K(s − Rn (s)) − K(s − Rn(s))| N = |L/S(s − Rn (s)) − L/S(s − Rn(s))| (1 − p )2 ≤ max |S(s − RN (s)) − S(s − R (s))| L n n N ≤ C|Rn (s) − Rn(s)| ≤ C sup |QN (τ) − Q(τ)|. τ≤s∧σN We used the fact that S(s)= d κ mc(s) is Lipschitz, as was checked in c=1 c Rc(s) Section 3.2. P We can therefore bound the integral of the second expression in (4.22): t∧σN N E |K(s − Rn (s)) − K(s − Rn(s))| ds Z0 t ≤ C DN (s) ds Z0 τ N N +E sup [χ{Q (s − Rn (s)) N N Moreover, Q (s − Rn (s)) N N N N N N N N N (ρ + φn (ρ ), ρ + φn(ρ)) or when s ∈ (ρ + φn (ρ ), η + φn (η )). Hence, τ N N [χ{Q (s − Rn (s)) t∧ρN N E |K(s − Rn (s)) − K(s − Rn(s))| ds Z0 t N N ≤ C(|ρ − ρ| + |ρ − η |)+ C DN (s) ds. Z0 We can easily bound the integral of the first expression in (4.22) using the same estimates we made for (4.21): τ N N N E sup |K (s − Rn (s)) − K(s − Rn (s))| ds 0≤τ≤t∧ρN Z0 t N N N ≤ C DN (s) ds + B (t)+ C(E|ρ − ρ| + E|η − ρ|). Z0 N N t Adding these extra pieces together, we see D (t) ≤ B1 (t)+C 0 DN (s) ds, N N N N where B1 (t) = 2B (t)+ C(E|ρ − ρ| + E|η − ρ|). The first iteration es- N N NR tablished that ρ and η converge in probability to ρ, so B1 (t) → 0 as N N →∞. Consequently, supt≤σ D (t) → 0 and we now have convergence on [0,σ]. N Using the results in Section 4.2, we can show sup0≤τ≤σ+Tmin |SN (τ) − S(τ)|→ 0 as N →∞. Hence, if S(t)(1−pmax) 5. RED is a weak limit of Gentle RED. Define the drop probability function for Gentle RED by Fδ(q), which rises from pmax at qmax to 1 at qmax + δ. Section 4.1 gives mean field convergence for Gentle RED over any interval [0, T ] since Fδ is Lipschitz. Here we show that, as δ → 0, Gentle RED becomes RED and Fδ tends (weakly) to F , the drop probability function for RED. For any δ and any N, redefine the solution to the N-particle system in Section 2, (WN (t),QN (t)) by (Wδ,N ,Qδ,N ), so now (WN (t),QN (t)) only denotes the solution with loss function F . These processes are constructed iteratively on the almost surely finite number of segments defined by jumps of Υn(s, λ(T )); n = 1,...,N, where Υn,n = 1, 2,... are defined on the prob- Rδ,N δ,N δ,N ability space (Ω, F, P ). Let = (R1 ,...,RN ) be the corresponding round trip time delay of the connections. Let d Wδ,N δ,N N c (t) SN (t)= κc δ,N . c Rc (s) X=1 δ,N δ,N δ,N Let P be the measure induced on D[0, T ] × C[0, T ] by (SN ,Q ) (where coordinates greater than N are identically zero). In the same manner as Lemma 1, we can show the measures P δ,N are tight. Using this lemma we can now prove the following: Lemma 6. (Wδ,N ,Qδ,N , Rδ,N ) converge weakly to (WN (t),QN , RN ) as δ → 0. The lemma holds even for N = ∞. δ,N N δ,N N Proof. By hypothesis, Q (t)= Q (t)= q(0) and Wn (t)= Wn (t)= δ,N δ,N wn for t ≤ 0. We show the drop probability K (t)= Fδ(Q (t)) converges as δ → 0. δ ,N 0,N First, pick a subsequence δk such that P k converges weakly to P δ ,N Wδk,N k δk,N and, moreover, such that ( , SN ,Q ) converges almost surely to W0,N 0,N 0,N ( , SN ,Q ). δk,N δk,N δk,N δk,N If Q (t) δ ,N dV k (s) δ ,N = −c S k (s)V δk,N (s)+ Lc , ds δk N δk δk,N where cδk = (1 − pmax)/δk. Solve this equation from time ρ (t), where δk,N δk,N V (ρ (t)) = (1 − pmax) up to time t: t δk,N δk,N V (t) = (1 − pmax)exp − cδ S (u) du δ ,N k N Zρ k (t) t t δk,N + Lcδ exp − cδ S (s) ds du δ ,N k k N Zρ k (t) Zu v(ρδk ,N (t)) δk,N −v L = (1 − pmax) exp(−v(ρ (t))) + e δ ,N dv, 0 k Z SN (u(v)) t δk,N where v = u cδk SN (s) ds and u(v) is the inverse defined implicitly. δk,N δk,N δk,N Define DR (t)=0 if Q (t) ≤ qmax and for Q (t) >qmax, define L Dδk,N (t)= V δk,N (t) − δk,N SN (t) δk,N = (1 − pmax) exp(−v(ρ (t))) v(ρδk,N (t)) −v 1 1 + L e δ ,N − δ ,N dv 0 k k Z SN (u(v)) SN (t) δ ,N L − e−v(ρ k (t)) . δk,N SN (t) δ ,N 0,N k δk,N(t) 0,N Since (SN (t),Q ) converges almost surely to (SN (t),Q (t)) as 0,N δk,N 0,N δk → 0, it follows that ρ (t) converges to a limit ρ (t). Since SN (t) > 0 almost surely, v(ρδk,N (t)) →∞ as δ → 0 and for a fixed v, v δk = k (1−pmax) δ ,N t k δk,N u(v) SN (s) ds so u(v) → 0 as δk → 0. We conclude D (t) → 0 almost surely. R δ ,N k δk,N If we now take the limit as δk → 0 in (2.9) satisfied by (SN (t),Q (t)), 0,N 0,N(t) using loss function Fδk , we see (SN (t),Q ) satisfies (2.9) with loss 0,N 0,N 0,N function F , so K (t)= L/SN (t) if Q (t)= qmax. Moreover, the win- δk,N dow Wn satisfies (2.3), where the rate of window reductions δk,N δk,N δk,N Wn (t − Rn (t)) δk,N δk,N λ (t) := Fδ (Q (t − R (t))). n δk,N δk,N k n Rn (t − Rn (t)) MEAN FIELD FOR RED 45 δk,N 0,N Taking the limit as δk → 0, we see Wn converges to Wn , satisfying (2.3) where the rate of window reductions is 0,N 0,N 0,N Wn (t − Rn (t)) 0,N 0,N λn (t) := 0,N 0,N Fδk (Q (t − Rn (t))). Rn (t − Rn (t)) W0,N 0,N 0,N 0,N All this means = (W1 ,...,WN ) and Q are a solution to (2.3) WN N N N and (2.9), so, in fact, must be = (W1 ,...,WN ) and Q . Hence, the limit along subsequences is unique so we have proved weak convergence. 6. Mean-field stochastic differential equations. In this section we prove Theorem 1. We can reformulate (2.8) as in [2]. For g ∈ G, N N hg, Mc (t)i − hg, Mc (0)i N t 1 dg N 1 = N (Wn (s)) N ds κc N n 0 dw Rc (s) X=1 Z N − N − + (g(Wn (s )/2) − g(Wn (s ))) dNn(s) χ{n ∈ Kc} 1 N t dg 1 (6.1) = N χ{n ∈ Kc} (Wn(s)) N ds κc N 0 dw Rc (s) nX=1 Z N N + (g(Wn (s)/2) − g(Wn (s))) W N (s − RN (s)) × n c KN (s − RN (s)) ds RN (s − RN (s)) c c c N + Ec (t), N where Ec (t) is given by N t 1 N − − N N χ{n ∈ Kc} (g(Wn (s )/2) − g(Wn(s ))) dZn (s) κc N n 0 X=1 Z and t W N (s − RN (s)) ZN (t) − ZN (0) := dN N (s) − n n KN (s − RN (s)) ds . n n n RN (s − RN (s)) c Z0 c c Hence, N N hg, Mc (t)i − hg, Mc (0)i t 1 dg(w) = , M N (s) ds RN (s) dw c Z0 c 46 D. R. MCDONALD AND J. REYNIER N N (6.2) + h(g(w/2) − g(w))v, Mc (s − Rc (s), dv; s, dw)i 1 × KN (s − RN (s)) ds RN (s − RN (s)) c c c N + Ec (t). N We first show Ec is asymptotically small as N →∞. Recall N t N 1 N N Ec (t)= N Cc,n(s)Zc,n(ds), κc N n 0 X=1 Z where N N N Cc,n(s)= χ{n ∈ Kc}(g(Wn (s)/2) − g(Wn (s))) and N N Zc,n(t) − Zc,n(0) t W N (s − RN (s)) := dN N (s) − n c KN (s − RN (s)) ds χ{n ∈ K }. n RN (s − RN (s)) c c Z0 c c N If n ∈ Kc, Nn (s) is a point process adapted to Fn(t) with a stochas- N N N N N N tic intensity Wn (s − Rc (s))K (s − Rc (s))/Rc (s − Rc (s)). Consequently, N N Zn (t) is a martingale. Recall that Wn (s) is also adapted to Fn(t), so the right continuous version is Fn(t)-predictable. By Theorem T13 in [1], c 2 E(EN (t)) 1 = c 2 (κN N) N t N N N 2 Wn (s − Rc (s)) N N × E χ{n ∈ Kc} (Cn,c) (s) N N K (s − Rc (s)) ds " n 0 Rc (s − Rc (s)) # X=1 Z 1 ≤ c 2 (κN N) N t N N N N 2 Wn (s − Rc (s)) × E χ{n ∈ Kc} g(Wn (s)/2) − g(Wn (s)) N N ds n 0 Rc (s − Rc (s)) X=1 Z N t C1 N N ≤ c 2 χ{n ∈ Kc} E[Wn (s − Rc (s))] ds , (κN N) n 0 ! X=1 Z ′ where C1 is a constant depending on sup g and sup(g) . MEAN FIELD FOR RED 47 N We have the a priori bound Wn (t) ≤ a(t). Hence, N 2 tC1 E(Ec (t)) ≤ N 2 a(t). (κc ) N N 2 N So Ec (t) tends to 0 in L . Since, in addition, Ec (t) is a martingale, it follows that N N 2 2 P sup |Ec (t)| > λ ≤ E(Ec (T )) /λ , t∈[0,T ] N so the process Ec (t),t ∈ [0, T ], converges to zero in probability. The processes QN (t) and KN (t) converge in probability to the limit pro- N N cesses Q(t) and K(t), while (M1 (t),...,Md (t)) converges to (M1(t),...,Md(t)) in probability where the limit processes satisfy (2.13) and (2.12). Take the limit of (6.2) and we have our proof. 7. Numerical analysis. Assuming g(0) = 0 and that g(w) → 0 as w →∞, we can rewrite (1.2) as hg, Mc(t)i − hg, Mc(0)i t 1 = − hg(w), D M (s, dw)i R (s) w c Z0 c 1 + K(s − Rc(s)) Rc(s − Rc(s)) × hg(w), e(s,s − Rc(s), 2w) · Mc(s, 2dw) − e(s,s − Rc(s), w) · Mc(s, dw)i ds, where DwMc(s, dw), respectively DtMc(s, dw), is the Frechet derivative of the measure Mc(s, dw) with respect to w, respectively t. Consequently, 1 DtMc(t, dw)= − DwMc(t, dw) Rc(t) 1 + K(t − Rc(t)) Rc(t − Rc(t)) (7.1) × (e(t,t − Rc(t), 2w)Mc(t, d(2w)) − e(t,t − Rc(t), w)Mc(t, dw)). Neither Mc(t, dw) or Mc(s − Rc(s), dv; s, dw) is a state, but the above equation does provide enough information to evolve the system. Let µc(t) 48 D. R. MCDONALD AND J. REYNIER denote the process {Mc(s, dw); t − 1 ≤ s ≤ t} (all RTTs are less than 1). Using (1.1), we can evolve Mc(t, dw) from t to t + δt while Mc(t − s + δt, dw) is obtained by a time shift. Unfortunately, µc is not a practical state. Even if we discretize and only keep the trajectory of the process on a partition giving {Mc(si, dw); t − 1= s0 Fig. 1. Relative queue size with 200 sources. Fig. 2. Relative queue size with 400 sources. 50 D. R. MCDONALD AND J. REYNIER Fig. 3. Relative queue size with 800 sources. Fig. 4. Queue size of the mean field limit. MEAN FIELD FOR RED 51 (in packets per connection) decrease. We also see that the relative queue size of the Matlab numerical simulation is a bit high. This is because of timeouts as discussed in [2]. Acknowledgments. J. Reynier thanks Fran¸cois Baccelli for the supervi- sion of this work. R. McDonald thanks Tom Kurtz for his tutorial on how to bring back the particles into the proof of weak convergence to the mean-field model. He also thanks Ruth Williams for her suggestions [14]. Both of us thank Pierre Br´emaud for stopping us from going completely off the rails and finally we thank a careful referee for his many comments and suggestions. REFERENCES [1] Bremaud,´ P. (1981). Point Processes and Queues. Springer, Berlin. MR0636252 [2] Baccelli, F., McDonald, D. R. and Reynier, J. (2002). A mean-field model for multiple TCP connections through a buffer implementing RED. Performance Evaluation 11 77–97. [3] Bain, A. (2003). Fluid limits for congestion control in networks. Ph.D. thesis, Univ. Cambridge. [4] Billingsley, P. (1995). Convergence of Probability Measures, 3rd ed. Wiley, New York. MR1700749 [5] Dawson, D. A. (1983). Critical dynamics and fluctuations for a mean-field model of cooperative behavior. J. Statist. Phys. 31 29–85. MR0711469 [6] Dawson, D. A. (1993). Measure valued Markov processes. Ecole´ d’Et´ede´ Proba- bilit´es de Saint-Flour XXI. Lecture Notes in Math. 1541 1–260. Springer, Berlin. MR1242575 [7] Deb, S. and Srikant, R. (2004). Rate-based versus queue-based models of conges- tion control. In Proceedings of the Joint International Conference on Measure- ment and Modeling of Computer Systems 246–257. [8] Deng, X., Yi, S., Kesidis, G. and Das, C. (2003). A control theoretic approach in designing adaptive AQM schemes. Globecom 2003 2947–2951. [9] Donnelly, P. and Kurtz, T. (1999). Particle representations for measure-valued population models. Ann. Probab. 27 166–205. MR1681126 [10] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes; Characterization and Convergence. Wiley, New York. MR0838085 [11] Floyd, S. (1999). The NewReno modification to TCP’s fast recovery algorithm. RFC 2582. [12] Floyd, S. and Jacobson, V. (1993). Random early detection gateways for conges- tion avoidance. IEEE/ACM Trans. Networking 11 397–413. [13] Floyd, S., Mahdavi, J., Mathis, M. and Podolsky, M. (2000). An extension to the selective acknowledgement (SACK) Option for TCP. RFC 2883, Proposed Standard, July 2000. [14] Gromoll, H. C., Puha, A. L. and Williams, R. J. (2002). The fluid limit of a heav- ily loaded processor sharing queue. Ann. Appl. Probab. 12 797–859. MR1925442 [15] Hollot, C. V., Misra, V., Towsley, D. and Gong, W.-B. (2001). A control theoretic analysis of RED. In Proceedings of IEEE INFOCOM 2001 1510–1519. [16] Kuusela, P., Lassila, J., Virtamo, J. and Key, P. (2001). Modeling RED with idealized TCP sources. In Proceedings of IFIP ATM and IP 2001 155–166. 52 D. R. MCDONALD AND J. REYNIER [17] Raina, G. and Wischik, D. (2005). Buffer sizes for large multiplexers: TCP queueing theory and stability analysis. In Proceedings of NGI2005 79–82. [18] Rosolen, V., Bonaventure, O. and Leduc, G. (1999). A RED discard strategy for ATM networks and its performance evaluation with TCP/IP traffic. Computer Communication Review 29 23–43. [19] Shakkottai, S. and Srikant, R. (2003). How good are deterministic fluid models of internet congestion control? INFOCOM 2002 497–505. Department of Mathematics Department´ d’Informatique University of Ottawa Ecole´ Normale Superieure´ Canada France e-mail: [email protected] e-mail: [email protected]