CS144: An Introduction to Computer Networks

Packet Switching

CS144, Stanford University Outline

1. End-to-end delay 2. Queueing delay 3. Simple deterministic queue model 4. Examples

CS144, Stanford University 2 Propagation Delay, tl: The time it takes a single bit to travel over a link at propagation speed c. l

l t = l c

Example: A bit takes 5ms to travel 1,000km in an optical fiber with propagation speed 2 x 108 m/s. CS144, Stanford University 3 Packetization Delay, tp: The time from when the first to the last bit of a packet is transmitted. p r bits/s p t = p r

Example 1: A 64byte packet takes 5.12ms to be transmitted onto a 100Mb/s link. Example 2: A 1kbit packet takes 1.024s to be transmitted onto a 1kb/s link.

CS144, Stanford University 4 Total time to send a packet across a link: The time from when the first bit is transmitted until the last bit arrives. l p r bits/s 푝 푙 푡 = 푡 + 푡 = + 푝 푙 푟 푐

Example: A 100bit packet takes 10 + 5 = 15ms to be sent at 10Mb/s over a 1km link.

CS144, Stanford University 5 End-to-end delay

A l1, r1 l2, r2

l3, r3 l4, r4 B

Example: How long will it take a packet of length p to travel from A to B, from when the 1st bit is sent, until the last bit arrives? Assume the switches store-and-forward packets along the path.

æ p l ö End-to-end delay, t = åç + i ÷ i è ri c ø CS144, Stanford University 6 A l1, r1 l2, r2 l3, r3 l4, r4 B

S1 S2 S3

p/r time A 1

p/r2 S1 æ ö l /c p li 1 p/r End-to-end delay, t = åç + ÷ S2 3 i è ri c ø l2/c p/r S3 4

l3/c

B time l4/c

CS144, Stanford University 7 Data H Other packets

A l1, r1 l2, r2 l3, r3 l4, r4 B

S1 S2 Q2(t) S3

p/r 1 æ ö A p li End-to-end delay, t = åç + +Qi (t)÷ p/r2 r c S1 i è i ø l /c Q2(t) 1 p/r S2 3 l2/c p/r S3 4

l3/c B time l4/c

CS144, Stanford University 8 Packet delay variation 100%

90% Stanford-Princeton Stanford-Tsinghua 4,000 km 10,000 km 80% Variation ~50ms Variation ~200ms 70% CDF (%) 60% 50%

40%

30%

20%

10%

0% 0 100 200 300 400 500 600 700 RTT (ms) CS144, Stanford University 9 Simple model of a router queue

CS144, Stanford University 11 Simple model of a router queue

퐴(푡): cumulative arrivals up until time 푡 퐷(푡): cumulative departures up until time 푡 푄(푡): number of packets in queue at time 푡

Router queue

Arriving packets 퐴(푡), 휆 퐷(푡) from different inputs R 푄(푡) Properties: 1. 퐴 푡 , 퐷 푡 non-decreasing 2. 퐴 푡 ≥ 퐷(푡)

CS144, Stanford University 12 Simple model of a queue

Cumulative number of bytes arrived up until time t. 퐴(푡) 퐴(푡) 퐷(푡)

푄(푡)

Cumulative number of bytes

Link R R rate time

퐷(푡) Properties: 1. 퐴 푡 , 퐷 푡 non-decreasing Cumulative number of bytes 2. 퐴 푡 ≥ 퐷(푡) departed up until time t.

CS144, Stanford University 13 Simple model of a queue 퐴(푡)

퐷(푡) 푑(푡)

Cumulative 푄(푡) number bytes of

time

Queue occupancy: 푄(푡) = 퐴(푡) − 퐷(푡).

Queueing delay, 푑(푡), is the time spent in the queue by a byte that arrived at time t, assuming the queue is served first-come-first-served (FCFS).

CS144, Stanford University 14 Example (store & forward)

Every second, a 500 bit packet arrives to a queue at rate 10,000b/s. The maximum

departure rate is 1,000b/s. What Cumulative number of bitsofnumber 퐴(푡) is the average occupancy of the 500 queue? 퐷(푡) 푄(푡)

0.05s 0.55s 1.0s 1.05s time

Solution: During each repeating 1s cycle, the queue fills at rate 10,000b/s for 0.05s, then empties at rate 1,000b/s for 0.5s. Over the first 0.55s, the average queue occupancy is therefore 250 bits. The queue is empty for 0.45s every cycle, and so average queue occupancy is (0.55 * 250) + (0.45 * 0) = 137.5 bits.

CS144, Stanford University 15 Example (“cut through”)

Every second, a 500 bit packet arrives to Cumulative

a queue at rate 10,000b/s. The bitsofnumber A(t) maximum departure rate is 1,000b/s. 500 What is the time average occupancy of D(t) the queue? Q(t)

0.05s 0.5s 1.0s time

Solution: During each repeating 1s cycle, the queue fills at rate 10,000b/s to 500-50=450 bits over the first 0.05s, then drains at rate 1,000b/s for 0.45s. Over the first 0.5s, the average queue occupancy is therefore 225 bits. The queue is empty for 0.5s every cycle, and so average queue occupancy: 푄ത 푡 = 0.5 × 225 + 0.5 × 0 = 112.5

CS144, Stanford University 16 What if some packets are more important than others?

CS144, Stanford University By default, switches and routers use FIFO (aka FCFS) queues

Buffer size, B Packets arriving Service Rate, R Departing to different ingress ports packets

CS144, Stanford University 18 By default, switches and routers use FIFO (aka FCFS) queues

Buffer size, B Packets arriving Service Rate, R Departing to different ingress ports packets

CS144, Stanford University 19 Some packets are more important For example: 1. The control traffic that keeps the network working (e.g. packets carrying routing table updates) 2. Traffic from a particular user (e.g. a customer paying more) 3. Traffic belonging to a particular application (e.g. videoconference) 4. Traffic to/from particular IP addresses (e.g. emergency services) 5. Traffic that is time sensitive (e.g. clock updates)

CS144, Stanford University 20 Flows When talking about priorities, it’s convenient to talk about a “flow” of packets that all share a common set of attributes. For example: 1. The flow of packets all belonging to the same TCP connection Identified by the tuple: TCP port numbers, IP addresses, TCP protocol 2. The flow of packets all destined to Stanford Identified by a destination IP address belonging to prefix 171.64/16 3. The flow of packets all coming from Google Identified by a source IP address belonging to the set of prefixes Google owns. 4. The flow of web packets using the http protocol Identified by packets with TCP port number = 80 5. The flow of packets belonging to gold-service customers Typically identified by marking the IP TOS (type of service) field

CS144, Stanford University 21 Outline

1. Strict Priorities 2. Weighted Priorities and Rate Guarantees

CS144, Stanford University 22 Strict Priorities

High priority flows

Low priority flows

CS144, Stanford University 23 Strict Priorities

High priority flows

Low priority flows

“Strict priorities” means a queue is only served when all the higher priority queues are empty

CS144, Stanford University 24 Strict Priorities: Things to bear in mind 1. Strict priorities can be used with any number of queues. 2. Strict priorities means a queue is only served when all the higher priority queues are empty. 3. Highest priority flows “see” a network with no lower priority traffic. 4. Higher priority flows can permanently block lower priority flows. Try to limit the amount of high priority traffic. 5. Not likely to work well if you can’t control the amount of high priority traffic. 6. Or if you really want weighted (instead of strict) priority.

CS144, Stanford University 25 How do I give weighted (instead of strict) priority?

CS144, Stanford University 26 CS144, Stanford University 27 Trying to treat flows equally

CS144, Stanford University 28 Trying to treat flows equally

While each flow gets to send at the same packet rate, the data rate is far from equal.

CS144, Stanford University 29 flows bit-by-bit

CS144, Stanford University 30 Scheduling flows bit-by-bit

Now each flow gets to send at the same data rate, but we no longer have “”.

CS144, Stanford University 31 Can we combine the best of both?

i.e. packet switching, but with bit-by-bit accounting?

CS144, Stanford University 32 Fair Queueing

3 2 1 5 2 4 3 12 1

6 5 4 3 2 1 5 2 4 3 1 2 1

Packets are sent in the order they would complete in the bit-by-bit scheme. Does this give fair (i.e. equal) share of the data rate?

CS144, Stanford University 33 Yes!

1. It can be proved that the departure time of a packet with Fair

Queueing is no more than Lmax/R seconds later than if it was scheduled bit-by-bit. Where Lmax is the maximum length packet and R is the data rate of the outgoing link. 2. In the limit, the two flows receive equal share of the data rate. 3. The result extends to any number of flows sharing a link.1

[1] “Analysis and Simulation of a Fair Queueing Algorithm” Demers, Keshav, Shenker. 1990. CS144, Stanford University 34 What if we want to give a different share of the link to each flow?

i.e., a weighted fair share.

CS144, Stanford University 35 Weighted Fair Queueing

3 2 1 3/4 3 2 2 1 1

6 5 4 3 2 1 3 2 2 1 1 1/4

As before, packets are sent in the order they would complete in the bit-by-bit scheme.

CS144, Stanford University 36 Weighted Fair Queueing (WFQ)

For any number of flows, 휙1 and any mix of packet sizes:

1. Determine the departure 휙 time for each packet using i R the weighted bit-by-bit scheme. ∑휙i = 1 2. Forward the packets in 휙N i order of increasing departure time.

Flow i is guaranteed to receive at least rate 휙 iR CS144, Stanford University 37 Weighted Fair Queueing (WFQ)

휙1

Packets arriving Classify packets 휙i at different Packet R into flows ingress ports scheduler

∑휙i = 1 휙N i

Flow i is guaranteed to receive at least rate 휙 iR CS144, Stanford University 38 Summary

1. FIFO queues are a free for all: No priorities, no guaranteed rates. 2. Strict priorities: High priority traffic “sees” a network with no low priority traffic. Useful if we have limited amounts of high priority traffic. 3. Weighted Fair Queueing (WFQ) lets us give each flow a guaranteed service rate, by scheduling them in order of their bit- by-bit finishing times.

CS144, Stanford University 39 Can we guarantee the delay of a packet across a network of packet switches?

CS144, Stanford University Delay guarantees: Intuition

A l1, r1 l2, r2 l3, r3 l4, r4 B

푄1 푡 푄2 푡 푄3 푡

푝 푙푖 End-to-end delay, 휏 = ෍ + + 푄푖(푡) 푖 푟푖 푐

The following values are fixed (or under our control): p, c, li and ri. If we know the upper bound of 푄1 푡 , 푄2 푡 , and 푄3 푡 , then we know the upper bound of the end-to-end delay.

CS144, Stanford University 41 Upper bound on Q 푡

b r

푏 푄 푡 ≤ 푟 Example: If a packet arrives to a FIFO queue of size 1 million bits, and the queue is served 106 at 1Gb/s, then the packet is guaranteed to depart within ൗ109 = 1ms. CS144, Stanford University 42 Delay guarantees: Intuition

푏1 푏2 푏3

A l1, r1 l2, r2 l3, r3 l4, r4 B

푄1 푡 푄2 푡 푄3 푡

4 3 푝 푙푖 End-to-end delay for a single packet, 휏 = ෍ + + ෍ 푄푖 푡 푟푖 푐 푖=1 푖=1 4 3 푝 푙 푏 ≤ ෍ + 푖 + ෍ 푖 푟푖 푐 푟푖 CS144, Stanford University 푖=1 푖=1 43 Why this is only an intuition…

1. Doesn’t tell us what happens when 푟2 < 푟1. Will packets be dropped?

2. Treats all packets sharing a queue as one big flow; it doesn’t give a different end-to-end delay to each flow.

Q: How can we give an upper bound on delay to each individual flow?

CS144, Stanford University 44 Weighted Fair Queueing (WFQ)

휙1

Packets arriving Classify packets 휙i to different Packet R into flows ingress ports scheduler

∑휙i = 1 휙N i

Flow i is guaranteed to receive at least rate 휙 iR CS144, Stanford University 45 Weighted Fair Queueing (WFQ)

휙1

bi Packets arriving Classify packets 휙i to different Packet R into flows ingress ports scheduler

∑휙i = 1 휙 i 푏 N delay ≤ 푖 휙푖푅

Flow i is guaranteed to receive at least rate 휙 iR CS144, Stanford University 46 Bounding end-to-end delay

CS144, Stanford University 47 Bounding end-to-end delay 1. Allocate a queue of size b for this flow 2. Assign a WFQ service rate of휙R

b b l 휙R 휙R l

l l b 휙R

푙 푝 푏 The end-to-end delay of a single packet of length p ≤ 4 + + 3 푐 푅 휙R CS144, Stanford University 48 What if two of the flow’s enter the network back-to-back? (A “burst”)

1. If the packets are far apart, then the queues drain the first packet before the second one arrives. All is good, and the delay equation holds.

2. If the packets are close together in a “burst”, then they can arrive faster than 휙R and the queue might overflow, dropping packets.

3. This might be OK in some cases. But if we want to bound the end-to- end delay of all packets, then we need to deal with bursts. How?

CS144, Stanford University 49 The leaky bucket regulator Limiting the “burstiness”

Tokens at constant rate, r size, s

Packets In Packets Out

Send packet if and only if we

CS144, Stanford University have tokens in the bucket The leaky bucket regulator Limiting the “burstiness”

Tokens at constant rate, r Token bucket size, s

Packets In Packets Out

Send packet if and only if we

CS144, Stanford University have tokens in the bucket The leaky bucket regulator Limiting the “burstiness”

Tokens at constant rate, r Token bucket size, s

Packets In Packets Out

Send packet if and only if we

CS144, Stanford University have tokens in the bucket The leaky bucket regulator

r 퐴(푡) r Cumulative Input to regulator s bits s

Packets In Packets Out 퐴(푡) 퐷(푡)

퐷(푡) Number of bits that can be sent in any period Output from regulator of length t is bounded by: 𝜎 + 𝜌푡 time It is also called a “(𝜎, 𝜌) regulator”

CS144, Stanford University The leaky bucket regulator Limiting the “burstiness”

r Cool theorem: If arrivals to the queue are 𝜎, 𝜌 -constrained, and if the queue is served at rate 휙푅 > 𝜌 and 푏 > 𝜎, then departures are also 𝜎, 𝜌 -constrained. Which means s arrivals to the next router are also 𝜎, 𝜌 -constrained.

b b 휙R 휙R

푏 ≤ If 휙푅 > 𝜌 and 푏 > 𝜎 then delay through the first router for all packets in the flow 휙R

CS144, Stanford University Putting it all together

r

s

b b l 휙R 휙R l l b l 휙R

푙 푝 푏 If 휙푅 > 𝜌 and 푏 > 𝜎 then the end-to-end delay of every packet of length p ≤ 4 + + 3 푐 푅 휙R

CS144, Stanford University In other words Leaky bucket (𝜎, 𝜌) regulator b1 b2 b3 휙1R 휙2R 휙3R

A l1, r1 l2, r2 l3, r3 l4, r4 B

푄1 푡 푄2 푡 푄3 푡

4 3 푝 푙푖 If we set 푏푖 > 𝜎, and 휙푖푅 > 𝜌 then 휏 = ෍ + + ෍ 푄푖 푡 푟푖 푐 푖=1 푖=1 4 푝 푙 3𝜎 ≤ ෍ + 푖 + 푟푖 푐 𝜌 CS144, Stanford University 푖=1 56 An Example Q: In the network below, we want to give an application flow a rate of 10Mb/s and an end to end delay of less than 4.7ms for 1,000 byte packets. What values of 𝜎 and 𝜌 should we use for the leaky bucket regulator? And what service rate and buffer size do we need in the routers? (Assume speed of propagation, 푐 = 2 × 108m/s).

Leaky bucket (𝜎, 𝜌) regulator

A 10km, 1Gb/s 100km, 100Mb/s 10km, 1Gb/s B

1 1 1 A: The fixed component of delay is (120푘푚Τ푐) + 8,000푏푖푡푠( + + ) = 0.7ms, leaving 109 100×106 109 4ms delay for the queues in the routers. Let’s apportion 2ms delay to each router, which means the queue in each router need be no larger than 2푚푠 × 10Mb/s = 20,000bits (or 2500bytes). Therefore, the leaky bucket regulator in Host A should have 𝜌 = 10푀푏/푠 and 𝜎 ≤ 20,000푏푖푡푠. WFQ should be set at each router so that 휙푖푅 ≥ 10푀푏/푠 and the flow’s queue should have a capacity of at least 2500bytes. CS144, Stanford University In practice

While almost all network equipment implements WFQ (even your WiFi router at home might!), public networks don’t provide a service to control end-to-end delay.

Why? - It requires coordination of all the routers from end to end. - In most networks, a combination of over-provisioning and priorities work well enough.

CS144, Stanford University 58 Summary

1. If we know the size of a queue and the rate at which it is served, then we can bound the delay through it.

2. WFQ allows us to pick the rate at which a queue is served.

3. With the two observations above, if no packets are dropped, we can control end-to-end delay.

4. To prevent drops, we can use a leaky bucket regulator to control the “burstiness” of flows entering the network.

CS144, Stanford University 59