Fair Scheduling in Input-Queued Switches under Inadmissible Traffic Neha Kumar, Rong Pan, Devavrat Shah Departments of EE & CS Stanford University {nehak, rong, devavrat}@stanford.edu
Abstract—In recent years, several high-throughput low-delay scheduling achieve 100% throughput. These results focus on admissible algorithms have been designed for input-queued (IQ) switches, assuming traffic conditions however, when in practice traffic is frequently admissible traffic. In this paper, we focus on queueing systems that violate admissibility criteria. inadmissible. Herein lies our motivation for this paper, where We show that in a single-server system with multiple queues, the Longest we study scheduling policies for inadmissible traffic conditions Queue First (LQF) policy disallows a fair allocation of service rates 1.We in IQ switches. also describe the duality shared by LQF’s rate allocation and a fair rate allocation. In general, we demonstrate that the rate allocation performed Under admissible traffic, stable scheduling algorithms grant by the Maximum Weight Matching (MWM) scheduling algorithm in over- every flow its desired service, and there does not arise a need loaded IQ switches is unfair. We attribute this to the lack of coordination for fairness in rate allocation 4. Under inadmissible traffic, not between admission control and scheduling, and propose fair scheduling al- all flows can receive desired service. We observe the rate allo- gorithms that minimize delay for non-overloaded queues. Keywords— Congestion Control, Quality of Service and Scheduling, cations performed by LQF and MWM in such a scenario, and Stochastic Processes and Queueing Theory, Switches and Switching, Re- prove that they lack fairness. This motivates our search for a source Allocation scheduling policy that performs a fair rate allocation, given in- admissibility. We now summarize our results. I. INTRODUCTION The input-queued (IQ) switch architecture is widely used in B. Outline and Results high-speed switching. This is due to its low memory bandwidth In section II, we formalize the notion of fairness that we will requirements compared to those of output-queued and shared- use. We then commence our study of IQ switch-scheduling un- memory architectures, making it the preferred choice. der inadmissible traffic conditions by observing the performance In an N × N IQ switch, we assume fixed-size cells (packets). of LQF in a system with multiple queues and an over-subscribed Each input has N FIFO virtual output queues (VOQs), one for server in section III. As an interesting side observation, we also each output 2. Packets queue up at the inputs, arriving at input i present the duality shared by the rate allocations determined by for output j at an average rate λij. In each time slot, at most one LQF and Max-Min fairness. In section IV, we provide Fair- packet can arrive at each input and at most one can be transferred LQF, an algorithm that incorporates fairness into LQF to per- to an output. Consider these conditions for Λ=[λij ]: form a provably fair rate allocation. N × N N N In section V, we extend our study to the switch and observe the rate allocation performed by MWM. We show that λij < 1, ∀i λij < 1, ∀j j=1 i=1 it lacks fairness, and propose the Fair-MWM algorithm in sec- tion VI, conjecturing that this too performs a fair rate allocation. The first condition is enforced by the line-rate constraints Our conclusions follow in section VII. at the inputs. Incoming traffic is called admissible when the second condition is satisfied and inadmissible otherwise. The II. FAIRNESS IN SCHEDULING switch scheduling problem reduces to a matching problem in a weighted bipartite graph with N inputs and N outputs 3. To define a fair rate allocation for flows, we use the estab- lished notion of Max-Min fairness [1]. A. Background and Motivation Definition 1 (Max-Min Fairness) Let there be n flows arriv- The primary performance metric of an IQ switch scheduling ing at a server of capacity C with rates λ1, ··· ,λn respectively. algorithm is the throughput it delivers. The Maximum Weight A rate allocation r =(r1, ··· ,rn) is called Max-Min fair iff Matching (MWM) algorithm has been shown to achieve 100% (i) n ri ≤ C, ri ≤ λi, and throughput when arriving traffic is admissible and obeys the (ii) any ri can be increased only by reducing rj s.t. rj ≤ ri. Strong Law of Large Numbers. Practical heuristics to approxi- Definition 2 (Fairness in a Switch) There exist N 2 flows in mate MWM [7] [4] too have been proposed. In a single-server an N × N switch. We call R =[rij ] a fair allocation for queueing system, Longest Queue First (LQF) is also known to Λ=[λij] iff it is Max-Min fair for every output. 1The formal definition of fairness is provided later. 2 The Virtual Output Queueing architecture improves performance by prevent- 4In this paper, each VOQ defines one flow. This can easily be generalized to ing Head-of-Line blocking [5]. accommodate the arrival of multiple flows at an input that are intended for the 3 In this paper, we take the weight of the edge (i, j) as the size of VOQij . same output.
IEEE Communications Society Globecom 2004 1713 0-7803-8794-5/04/$20.00 © 2004 IEEE III. LONGEST QUEUE FIRST AND FAIRNESS B. Rate Allocation under LQF We first consider a system of N queues and a server of unit When traffic is inadmissible, queues that do not receive de- capacity. The arrival rate for queue i, 1 ≤ i ≤ N, is denoted by sired service grow unboundedly. Since LQF is a non-idling pol- t λi, and the cumulative number of arrivals until time n is denoted icy, it services a queue in every time slot. For large enough , r t T t /t i by Ai(n), where Ai(0) = 0. We assume that the arrivals obey i( )= i( ) denotes the service rate allocated to queue the Strong Law of Large Numbers (SLLN) that states: by LQF. The fluid model equations (2) – (5) characterize LQF’s rate allocation for arrival rates λ1, ··· ,λn. Ai(n) For admissible traffic, techniques such as Lyapunov analysis lim = λi ∀iw.p.1 (1) n→∞ n may be used to show that LQF services every queue at its arrival Let Qi(n) denote the size of queue i at time n, Di(n) the rate. For inadmissible traffic, we state the following theorem to number of departures from queue i until time n (Di(0) = 0), describe LQF’s rate allocation: λ i ≤ and Ti(n) the number of time slots queue i is scheduled for ser- Theorem 1: Let i represent the arrival rate for queue , 1 i ≤ N λ ≥ λ ≥ ··· ≥ λ ≥ λ vice in [0,n]. LQF always serves the longest queue, breaking . Assume 1 2 N N+1 =0.Let r t i ties arbitrarily. For n ≥ 0, the equations below describe queue i( ) represent the rate allocated by LQF to queue in time slot t ∀t ≥ sizes, departures and the busyness of the server over time. . Then 0: (i) 0 ≤ rk(t) ≤ λk s.t. k rk(t)=1 Qi(n)=Qi(0) + Ai(n) − Di(n) (ii) Let 1 ≤ l ≤ N be the smallest index such that: D n D n − T n − T n − Q n − > l i( )= i( 1) + ( i( ) i( 1))( i( 1) 0) ∆ =( λi − 1)/l > λl+1 N i=1 T · T n n + i( ) is non-decreasing and i( )= Then ∀k, rk(t)=(λk − ∆) . That is, all queues served at i=1 a positive rate grow at the rate ∆, while the rest grow at their The dynamics of the system are completely described by respective arrival rates 6. N S(n)=(Qi(n),Di(n),Ti(n))i=1. WATER WATER λ r =λ 4 r = r = r 4 4 A. Fluid Model for a Queue 1 2 3 WATER λ λλ λ λλ We employ the fluid model technique to study LQF [3]. To 1 23 1 23 r do this, we define a sequence of systems indexed by = GRAVITY r r 1, 2, 3 ... with the associated description vectors S (t)= 3 WATER GRAVITY r r r N r r (Qi (n),Di (n),Ti (n))i=1. The relationship between S (t) and 2 r SOLID SOLID S(·) is described by: 1 Sr(t)=S(rt)/r
The fluid model solution of the system is characterized by (a) LQF (b) Max−Min Fair S¯(t) such that: Fig. 1. Comparison of LQF and Max-Min Fairness. S¯(t) = lim Sr(t) r→∞ Sr(t) is a non-deterministic quantity. We would like to show C. LQF vs. Max-Min Fairness that the limiting quantity S¯(t) is a deterministic solution to a set We momentarily digress to present an interesting duality that of differential equations. We proceed by showing the existence exists between the rates allocated by LQF and those allocated by of a limiting quantity and its characterization. Max-Min fairness. It is illustrated in Figure 1, where the shaded r r r r If for all i, |Di (t) − Di (t )|≤|t − t |, |Ti (t) − Ti (t )|≤ area represents a solid surface and the rest is empty space. There r |t − t |, and limr→∞ |Ai (t) − λit| =0then S¯(t) exists [2] and are N steps such that step i has depth λi and unit width, i.e. satisfies the fluid model equations 5 as follows: there are N columns with volumes λ1, ··· ,λN respectively and column i represents queue i with arrival rate λi. Q¯ t Q¯ λ t − D¯ t i( )= i(0) + i i( ) (2) We begin our experiment by pouring water (of unit volume) dD¯ i(t) dT¯i(t) through the hole. In Figure 1(a) gravity acts downwards, causing = if Q¯i(t) > 0 (3) dt dt the water to fill the columns deepest-first. In Figure 1(b) gravity N T¯ · T¯ t t acts upwards, so that the shallowest column fills up first. When i( ) is non-decreasing and i( )= (4) i i=1 the pouring is complete, the water level in column indicates the service rate allocated to queue i by LQF in 1(a) and Max-Min We can check that our system satisfies these conditions if we fairness in 1(b). assume that arriving traffic satisfies condition (1). We observe that downwards-acting gravity causes the water to By serving only the longest queue at all times, LQF further fill up the deepest column first, just as LQF services the longest imposes the following condition on the evolution of T¯i(·): queue at the expense of the less-demanding flows. Conversely, dT¯i(t) in Figure 1(b), the water first fills all columns equally. When it ∃j Q¯i(t) < Q¯j(t) =0. If s.t. then dt (5) reaches a depth of λN , the additional water fills the remaining
5For an example, see [3]. 6The proof can be found at http://simula.stanford.edu/∼neha/FMWM.pdf.
IEEE Communications Society Globecom 2004 1714 0-7803-8794-5/04/$20.00 © 2004 IEEE N − 1 columns equally and so on. After the water has been V. M AXIMUM WEIGHT MATCHING AND FAIRNESS poured, all partially filled columns have equal depth. Moreover, We now extend our study to the N × N switch and observe i the water level in column indicates the Max-Min fair rate allo- MWM’s rate allocation in an overloaded switch to show that it i cation to queue . lacks fairness. This experiment reveals the duality shared by the rates allo- An input-output matching is represented by a permutation cated by LQF and those allocated by Max-Min fairness. We matrix π =[πij ]i,j≤N where πij =1iff it matches input i to now propose an algorithm that incorporates Max-Min fairness output j. A scheduling algorithm S must determine a matching into LQF to provide a fair rate allocation. π(n) for each time slot n. The arrival rate for VOQij , 1 ≤ i, j ≤ N, is denoted by λij, IV. FAIR-LQF and the cumulative number of arrivals until time n is denoted by The Fair-LQF algorithm combines the advantages of Max- Aij (n), where Aij (0) = 0. We assume that the arrivals obey the Min fairness and LQF. Based on individual queue sizes and a Strong Law of Large Numbers (SLLN) that states: pre-specified threshold value, it maintains a congested list of 7 Aij (n) queues whose sizes exceed the threshold and an uncongested lim = λij ∀i, j w.p.1 (6) list of all other queues. In the limiting case of this threshold n→∞ n going to ∞, a congested queue is one that is served at less than Let Qij (n) denote the size of VOQij at time n, Dij (n) the its desired rate. number of departures from VOQij until time n (Dij(0) = 0), W n W n W n A. The Algorithm and ( )=[ ij ( )], where ij ( ) denotes the weight of the edge (i, j) at time n in the switch bipartite graph 11. Then the If nc represents the number of congested queues and nuc rep- weight of a matching π is defined thus: resents the number of non-empty, uncongested queues, for the n π next c time slots, Fair-LQF serves every congested queue ex- W (n)= πijWij (n)=πW(n) actly once. For the remainder nuc time slots, it mimics LQF. i,j The pseudocode for Fair-LQF is as follows: Definition 3 (Maximum Weight Matching) A maximum weight //Information Collection w if (queue_size(queue_ID) >= threshold) matching algorithm is one that determines a matching π (n) at add_to_congested_list(queue_ID); time n such that: // Scheduling // Step1: scheduling congested ports πw(n) = arg max{W π(n)} (7) m = number-of-congested-queues; π while(m!=0){ // Round-Robin We now consider the dynamics of the discrete-time switch. schedule-congested-queues(); m--; Let MWM be our scheduling algorithm and be a collection 12 } of all possible matchings . For any π ∈ ,letTπ(n) be the //Step2: scheduling uncongested ports cumulative amount of time that a permutation π is scheduled in m = number-of-nonempty-uncongested-queues; ,n T ∀π ∈ n ≥ while(m!=0){ the time interval [0 ].Again, π(0) = 0, .For 0, LQF-schedule-uncongested-queues(); the equations below describe queue sizes, departures, and the m-- ; busyness of the server over time: } Fair-LQF deviates from LQF as it proactively limits the Qij (n)=Qij (0) + Aij (n) − Dij (n) (8) throughput of a congested queue. Under admissible traffic, Fair- n LQF behaves identically to LQF, since the limiting size of the Dij (n)= πij 1Q (l)>0(Tπ(l) − Tπ(l − 1)) (9) ij queues is 0. Under inadmissible traffic, it does a provably Max- π∈ l=1 Min fair rate allocation. Tπ(·) is non-decreasing and Tπ(n)=n (10) Theorem 2: Let λ1 ≥ ··· ≥ λN be the arrival rates of N π∈ queues and the server capacity be 1.LetR1, ··· ,RN denote the Max-Min fair rate allocation, and r1, ··· ,rN denote the Fair- A. Fluid Model for a Switch LQF rate allocation. Then ∀k, rk = Rk, i.e. Fair-LQF is Max- Min fair 89. We use the fluid model technique again to study MWM. By appropriately scaling the discrete-time switch, we can obtain the B. Performance Evaluation fluid model for a switch [3]. Based on equations (8) – (10), We studied the performance of LQF and Fair-LQF via simu- it has been shown [3] that under condition (6), the continuous, lations and compared it to Max-Min fairness. The simulations deterministic fluid model of a switch is as follows: attest that the Fair-LQF rate allocation is Max-Min fair and pro- 10 vides lower delay for uncongested queues . Qij (t)=Qij (0) + λij t − Dij (t) ≥ 0,t≥ 0 (11) ˙ ˙ 7 Dij (t)= πij Tπ(t), if Qij (t) > 0 ,t≥ 0 (12) The threshold may be chosen freely and does not affect the long-term perfor- mance of the algorithm. π∈ 8The proof can be found at http://simula.stanford.edu/∼neha/FMWM.pdf. 9 11 This theorem holds regardless of the threshold value. In this paper, we let Wij (n) be Qij (n). 10The simulation results are at http://simula.stanford.edu/∼neha/FMWM.pdf. 12For an N × N switch, | | = N!.
IEEE Communications Society Globecom 2004 1715 0-7803-8794-5/04/$20.00 © 2004 IEEE Tπ(·) is non-decreasing, and Tπ(t)=t, t ≥ 0 (13) A. The Algorithm π∈ Fair-MWM combines the advantages of MWM and Max-Min VOQs For a function f, f˙(t) denotes its derivative at t, if one exists. fairness to provide fair treatment to all , ensuring a small π delay for uncongested queues. While all well-behaved 15 flows Assuming Tπ(t) is differentiable and ∃ π such that: W (t) < π receive desired service, the offending flows are allotted an even W (t), then MWM dictates that T˙π(t)=0. fraction of the leftover bandwidth. B. Rate Allocation under MWM When a queue exceeds a pre-specified threshold, say a per- 16 13 The service rate sij(t) for VOQij is defined as follows: centage of the buffer size , it is considered congested. Once a 17 congested queue VOQij is served, it gets blocked for nj time 1 slots, where nj is the number of non-empty queues containing sij(t)= πijTπ(t) t packets headed for output j. In the scenario that multiple out- π∈ puts have congested queues, nj may be different for every over- It has been shown [3] that under admissible traffic for MWM, all subscribed output j. Once the blocking is accounted for, the queue-sizes Qij(t)=0, ∀t ≥ 0. From equations (11) – (13) for matching is determined by MWM as before. The pseudocode admissible traffic, we can infer that sij (t) ≥ λij , ∀t ≥ 0.We forFair-MWMisasfollows: now characterize the rates s(t)=[sij (t)] under inadmissible //Information Collection traffic using the notion of a cycle. if (voq_size(voq_ID) >= threshold) add_to_congested_list(voq_ID); Definition 4 (Cycle) In an N × N bipartite graph, a cy- cle of length l is an ordered set of input-output pairs // Scheduling // Step1: MWM {(i1,o1), (i2,o2),··· , (il,ol)} comprised of edges connecting MWM_schedule_unblocked_voqs(); these pairs. Theorem 3: Let Λ=[λij ] s.t. λij > 0, ∀i, j.Lets =[sij ] //Step2: Blocking information represent the rates allocated by MWM. Create an N × N bipar- for (i=0; i
IEEE Communications Society Globecom 2004 1716 0-7803-8794-5/04/$20.00 © 2004 IEEE Compared to the rate of 0.55 in Rmmf (1), MWM allocates a This behavior of MWM is analogous to what we observed pre- service rate of 0.88 to the greedy queue VOQ11 and only 0.04 viously: greedy flows are served far more than their fair share. to the remaining VOQs (VOQ21,31,41) destined for output 1. Fair-MWM solves this problem by allocating Max-Min fair The Fair-MWM algorithm gives a rate allocation identical to rates: Rf−mwm = Rmmf . that given by Rmmf . We now test the robustness of Fair-MWM in a more com- plicated scenario, when there are multiple offending flows for 0.9 Fair-MWM multiple outputs. The arrival matrix is: MWM 0.8 0.60.00.20.1 0.7 0.60.20.00.1 Λ(4) = 0.6 0.00.60.00.1
11 0.5 0.20.60.00.1 S
0.4
0.3 The MWM and Max-Min fair rate allocations for Λ(4) are:
0.2 0.47 0.00.20.1 0.40.00.20.1 0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.47 0.06 0.00.1 0.40.20.00.1 λ Rmwm = Rmmf = 11 0.00.47 0.00.1 0.00.40.00.1 Fig. 2. MWM & Fair-MWM: Comparison of Rate Allocation 0.06 0.47 0.00.1 0.20.40.00.1
Again, our algorithm performs a rate allocation that is Max-Min Instead of fixing λ11 at 1.0, we now vary it from 0.1 to 1.0 R R and evaluate the performance of Fair-MWM. Figure 2 depicts fair, i.e. f−mwm = mmf . This concludes our study of fairness in switch-scheduling as that Fair-MWM successfully limits the service rate of VOQ11 we complete the analysis of Fair-MWM, having demonstrated to 0.55, once λ11 exceeds the fair share. Under MWM, how- the benefits it adds to the MWM scheduling algorithm. ever, the service rate for VOQ11 grows monotonically as λ11 increases. Since the service rates for other queues destined for VII. CONCLUSIONS the same output are not protected, they become much smaller than the demand rates. In practice, traffic is often inadmissible and queues are often When there exist multiple offending flows destined for the overloaded. The inadmissible traffic scenario, however, has thus same output, Fair-MWM first satisfies the service requirement far been neglected. In this paper, we showed that the LQF and for the good flows to that output and then splits the remaining MWM algorithms considered ideal for admissible traffic per- service capacity evenly among the offenders. For instance, view form poorly when traffic is inadmissible. Their allocation of ser- the following example: vice rates to input queues is biased in favor of offending (over- loaded) traffic flows, penalizing the well-behaved flows unjustly. 0.50.10.10.1 We addressed this problem by proposing algorithms based on 0.50.10.10.1 Λ(2) = 0.10.10.10.1 the notion of Max-Min fairness, also highlighting the duality 0.10.10.10.1 shared by Max-Min fairness and LQF. Finally, we observed that our algorithms, Fair-LQF and Fair-MWM, effectively perform a The Max-Min fair and MWM rate allocations for Λ(2) are: fair rate allocation with low delay, using the knowledge of queue 0.40.10.10.1 0.45 0.10.10.1 sizes alone. 0.40.10.10.1 0.45 0.10.10.1 The techniques we employed in this paper are of a general R = R = mmf 0.10.10.10.1 mwm 0.05 0.10.10.1 nature and can easily be adapted to study the performance of 0.10.10.10.1 0.05 0.10.10.1 other scheduling algorithms under inadmissible traffic.
The Fair-MWM and Max-Min fair rate allocations are identical. REFERENCES When there exist offending flows destined for multiple out- puts, Fair-MWM limits their service rates independently while [1] D. Bertsekas, R. Gallager, “Data Networks”, Prentice Hall, 1992, pp. 526. protecting other non-offending flows. For example, consider the [2] J.G. Dai, “Stability of fluid and stochastic networks”, Miscellanea Publi- cation, No. 9, Center for Mathematical Physics and Stochastics, Denmark, arrival matrix Λ(3): http://www.maphysto.dk, Jan. 1999. [3] J.G. Dai, B. Prabhakar, “The Throughput of Data Switches with and with- 0.80.00.00.0 out Speedup” in Proceedings of IEEE INFOCOM, Mar. 2000, pp. 556-564. 0.30.30.10.1 Λ(3) = [4] P. Giaccone, B. Prabhakar, D. Shah, “Switching under Energy Constraints” 0.00.80.00.0 in Asilomar Conference on Signals, Systems and Computers, Nov. 2002. 0.20.20.20.2 [5] N. McKeown, A. Mekkittikul, V. Anantharam, J. Walrand, “Achieving 100% throughput in an input-queued switch” in IEEE Transactions on Communications, vol. 47, n. 8, Aug. 1999, pp. 1260-1267. The MWM and Max-Min fair rate allocations for Λ(3) are: [6] D. Shah, D. Wischik, “Switch under Heavy Traffic”, under preparation, 2004. 0.70.00.00.0 0.50.00.00.0 [7] L. Tassiulas, “Linear Complexity Algorithms for Maximum Throughput 0.20.20.10.1 0.30.30.10.1 in Radio Networks and Input-Queued Switches” in Proceedings of IEEE R = R = mwm 0.00.70.00.0 mmf 0.00.50.00.0 INFOCOM, Apr. 1998, pp. 533-539. 0.10.10.20.2 0.20.30.20.2
IEEE Communications Society Globecom 2004 1717 0-7803-8794-5/04/$20.00 © 2004 IEEE