Population dynamics in networks: from queues to PDEs and control, from P2P to deferrable power loads.
Fernando Paganini Universidad ORT Uruguay
Joint work with Andrés Ferragut Universidad ORT Uruguay
IMA, Sept 2015. Outline 1. Networking research: between queues and fluids.
2. M/G- Processor Sharing queues, point process model and its PDE fluid counterpart.
3. Dynamics of peer-to-peer networks.
4. Dispatching deferrable power loads. Mathematics in networking - a historical view • In the beginning we had queueing theory… – Erlang, circa 1910, dimensioning telephone switches. • Statistical multiplexing inspired packet networks… – Kleinrock, 60s – 70s, Internet pioneer and queueing theorist. • But queueing theory reached its limits in networking: – Limited results in network case, dubious assumptions. • Turn to macroscopic fluid flow models: – Bertsekas/Gallager in late ’70s, Kelly et al in ’90s. – Cover elastic traffic, apply to general networks. – Early ’00s, Network Utility Maximization for multiple layers. • Still, Internet jobs remain statistically multiplexed. – Queuing theory has a comeback. – But fluid models appear also at this scale. Classical M/M/1 queues
Poisson (λ ) arrivals exp(µ ) service times
A( t ) Equivalently: exponential job size, X( t ) constant service rate.
t exp(λ ) t λ Markov chain, state x-1 x is queue occupancy µ λ Stable (ergodic) if ρ = < 1. µ Invariant distribution is geometric: ρ ρ π (x )=x (1 − ), x ≥ 0. Networks of Queues and Data networks
• Queueing theory concepts influential in the early days of data networks: statistical multiplexing motivates “packet switching”. • But networks of M/M/1 queues are not easy to analyze. The steady-state distribution only known under narrow assumptions (Jackson networks). • Besides, packet networks do not match model assumptions: sending times not Poisson, packet length not exponential. Going from “molecular” to fluid models. Arrivals: rate λ pkt s/sec Real valued x( t ), following ODE λt x( t ) dx A( t ) =λ µ− , saturated to x ()0. t ≥ dt
Departures: µ pkt s/sec t Relationship with Markov model for M/M/1 queue: Xm ( mt ) scaling limit x( t )= lim , where X m (0) = m . m→∞ m • Beyond the M/M/1 assumptions, the fluid “tank” is a natural macroscopic first-principles model. Extends to varying rates. • Allows for a simpler transition to the network case. • Leads to way to study feedback control of rates. Fluid view of network congestion control.
ri ( t )
feedback
• Congestion signals from scarce resources drive input rates. Network Utility Maximization model (Kel ly et al). Decentralized control is primal or dual solution of convex program
max r ∑Uri( i ), subject to ∑ Rrclli i ≤ l ∀ . i i CONCAVE UTILITY LINK CAPACITY FUNCTION CONSTRAINTS The network at the scale of connections
• The previous fluid models treat connections as permanent. • But connections (flows) arrive, are served and leave, there is also statistical multiplexing at the scale of flows. • Back to queueing theory! Distinguishing features: 1. Service discipline : all flows present are served simultaneously, sharing the network capacity. 2. Job sizes : exponential distribution is too limiting. Heavy tails observed for Internet files: many short transfers, far fewer long connections. Outline 1. Networking research: between queues and fluids.
2. M/G- Processor Sharing queues, point process model and its PDE fluid counterpart.
3. Dynamics of peer-to-peer networks.
4. Dispatching deferrable power loads. The M/G/1-Processor Sharing Queue. i Poisson(λ) arrivals of file transfer jobs i Random file size, general distrib ution. c P x σG(σ ) = (Si ze > ). G(σ ) σ i Processor Sharing: service capacity c (bp s) shared between c x jobs present. Rate per job: r = . Special case of NUM . x i Roberts and Massoulié ’ 98 proposed this model for TCP traffic, studied it in the case of exponential file sizes. λ λ Markov chain, stable if ρ = < c. µ x-1 x µc i Can we handle a more realistic model of file sizes? M/G/1-PS Queue. i Good news (see Kelly '78): PS queue is i nsen sitiv e: in steady state, the population of jobs has the sam e distribution ge om (ρ) as in exponenti al cas e . i But # of jobs is not the entire state: residual work matters.
Compact representation of the state (Gromoll et al. '02, P.Robert '03): 0 σ1 σ2 σ3 σ x σ Point measure on R+ , each point is a job, position is residual work.
Arrivals: a new point mass appears, following distribution G(σ ). Service= motion to the left, speed c x . Depart when reaching σ = 0.
The model applies also to the M/ G / ∞ queue: here, each job present has a firm capaci ty for itself. ⇒ Points move at constant spee d . Fluid models for M/G-PS Queues • At the scale of jobs, stochastic models are very natural. • However, again they are hard to solve except for simple cases. • For a more general bandwidth sharing, (e.g. given by NUM), it is not easy to find steady-state distributions.
• Turn again to fluid models: • Job quantities become real numbers • Differential equations replace Markov processes. • How to account for residual work? Fluid distribution on R+ Service= "advection", rate r(σ ) f( t ,σ ) Departures σ Arriving "mass", follows file-size distr ibution PDE using cumulative distributions For the point mass process, X( t ) Φ(t ,σ ) the complementary cumulative distribution is a decreasing, piecewise constant stochastic process
Fluid version: deterministic σ1 σ 2 σ 3 σ n σ real-valued f unction F( t ,σ ) x( t ) representin g population o f jobs F( t ,σ ) with more residual wor k tha n σ. Dynamics: ARRIVALS PROPAGATION F( t+ dt ,σ ) = σλ σ G() dt+ F( t , + rdt ) rdt σ σ ∂Ft(,)σ ∂ Ft (,) r :rate per job. ⇒ = σλ σ G() + rFt (,,) ∂t ∂ σ [’P-Tang-Ferragut-Andrew , IEEE TAC '12 ]: first use of PDE to prove a stability conjecture for Internet bandwidth s hari ng with general f ile sizes. Outline 1. Networking research: between queues and fluids.
2. M/G- Processor Sharing queues, point process model and its PDE fluid counterpart.
3. Dynamics of peer-to-peer networks.
4. Dispatching deferrable power loads. BitTorrent-like peer-to-peer networks • Content = set of pieces. S S • Peers: – Seeders own entire content, willing to disseminate it. – Leechers downloading pieces, also contribute by
uploading piece s they have. µi • Information on who owns L L which pieces, passed around L L to achieve efficient e xchang e. L L L • “Tit-for-tat” rules give incentives do cooperate . p2p capacity scales with demand We a ssume up load ban dwidth
µi p er p eer is t he bottl eneck. Population dynamics of a swarm of peers PEERS (~hundreds)
leechers seeders Arrive Trans Depart
File PIECES sharing (~thousands)
i Coarse model [Yang-DeVe ciana '04, Qiu-Srikant '04]: x leech, y seed. i De tailed mod el [Kesidis et al .07, Massou lié-Vojnov ic '08, Zhu-Hajek '12 ]: pop ulation state per sub set of pi eces. Huge dimens ion, limited results. i Our intermediate model: track profile o f population as a function of residual work σ. Leecher dynamics as an M/G-PS queue i Leechers arrive as Poiss on(λ), seeking S S download of ra ndom size S, me an E[] S = 1. µ G σ(σ ) =P( S > ) µ σ L L Covers deterministic sizes: G σ(σ )=1[0,1) ( ) µ L L L L (peers want all the file, start with no content). L
Populations: xt() leechers. Take for simplicity yty () ≡ 0 seeder s .
Assumptions (empirically validated for peers with homo geneous µ ): i Efficient use of tota l u pload capacity Rup =µ( x + y 0 ). µ µ (x+ y ) y i Processor shari ng : r =0 =µ + 0 , rate per leecher . x x
Leecher dynamics is an M/G/" ∞ + 1"− Processor Sharing queue. From stochastic to fluid model i Exploiting insensitivity, we find the s teady-state distribution of peers λ ρ x+ y 0 present in the stochastic queu ρ e: ρ π ()x= C() ,0 x ≥. = . (x+ y 0 )! µ i Moreover, in steady state, residual jobs are independently drawn ∞ from ccdf G(σ )= G( s ) ds . [Pechinkin'83, Zachary '07]. For instance, ∫σ deterministic demands yield a u nifo rm download profile in e quilibrium.
More information from fluid model: x( t ) F( t ,σ ) σ ∂Ft(,)σ x+ y ∂ Ft (,) =λ σ µG() + 0 ∂t x ∂ σ r( F ,, t σ ) σ Note: in the case of exponentially distributed jobs, G(σ ) = e −σ the PDE admits a solution Ft(,)σ = xt ()e−σ , where xt () satisfi es
the ODE x (t )=λ µ − ( x + y 0 ). This is the model in [Qiu-Sr ikant '04] . Transient analysis with PDE model Consider a transient sc enario ∂ σFt(,)σ x+ y ∂ Ft (,) = µ 0 with no leecher arrivals. ∂t x ∂ σ Fixed seeders y( t ) ≡ y 0.
Initial population of lee chers x0, wi th partial c onten t F (0, σσ )= Φ ( ) . Theorem: transient time to empty Compare with detailed packet 11 Φ (σ ) simulations (in ns2) of BitTorrent system is T= d σ. ∫0 σµ Φ( ) + y0 1x 1 Bound: T ≤0 ≤ . µµ x0+ y 0 Coarser ODE model:
x (t )= −µ ( x + y0 ), x (0) = x 0 Predicts completion time 1 x T '= log 1 + 0 . Pessi mistic ! µ y0 Including arrivals: equilibrium and stability. σ∂Ft(,)σ x+ y ∂ Ft (,) xt()= Ft (,0); =λ µ + 0 ; ∂t x ∂ σ 0≡ F ( t ,1).
For concreteness, work with d etermi nistic j ob sizes: G σ()1σ = [0,1) ( ) .
Assume seeders alone cannot cope with demand: ρ > y0. PDE h as a unique equilibriu m: x* F σ* () ρ σ = ( −Y )(1 − ) . 0 x*
1 σ Corresponds to a uniform density in resi dual work. Consistent with queueing results.
Nonlinear PDE. Is the equilibrium global ly stable? Monotone dynamical systems [Hirsch-Smith ’06] i Banach space Y, ordering defined by close d convex cone Y +
yy≤−∈''⇔ yy Y+ . yy '' ⇔ yy −∈ IntY ()+ . i Semiflow Φt :XX → , on XY⊂ . Maps x (0) → xtt (),0 ≥ . i Φt monotone or order preserving if x(0) ≤ x'(0)⇒ xt ( )≤ xt '( ) . i Φt strongly monotone if x(0) < x '(0)⇒ xt ( ) xt '( ) fo r t > 0. i (Strong) order preservation rules out " recurrent" dynamics. i For instance, there can be no stable limit cycles. Proposition
Let Y be an ordered Banach space, and Φt a strongly monotone semiflow on X⊂ Y , with orbits of com pact closure.
If X is open, invariant under Φt and contains a single equilibrium poin t p, all trajectories in X converge to p . Is our transport dynamics monotone? Nat ural ordering in functions of σ : FF≥⇔ F() σ σσ ≥ F () ∀ . Corresponds to the cone of positive functions. σ∂Ft(,)σ ∂ Ft (,) = σλ + r( F , ) ∂t ∂ σ Order is preserved if r( F ) is decreasing: FF≥ ⇒ rF(,) σ σσ ≤ rF (,) ∀ .
µ(x+ y ) Works in particular for processor sharing: r( F ) = 0 . x But, precisely invoking the theorem is more technical. In particular, compactness. Turn to a fi nite-dimensional version. A spatially discretized dynamics
M Stat e spa ce Z ⊂ R xt()= zt0 () z( t ) 1 z( t ) Z 2 :={}zzz :0 ≥≥≥ 1 z M − 1 ≥ 0 . with component-wise orderin g. zj ( t ) z j=λ + Mr j( z )( zj+1 − z j ) ,j= 0, … , M − 1. zM−1( t ) gj ( z ) 0 0.2 0.4 0.6 0.8 1 Proposition: ∂r ∂g ()i If j ≤∀0 jkAz,, then () = is Metzler ( a ≥ 0 for any ij ≠ ). ∂z ∂ z ij k ∂r (ii ) If in additi on j < 0, then A ( z ) is irreduc ible . ∂z0 (iii ) If ∑ rzzj( )( j+1− z j ) > κµz 0 , traject ories ar e bounded . j Theorem : under ()-(i ii )-( iii ) , if the equilibrium is unique, µ ()x+ y it is globally attractive. Covers in particular the case r = 0 . x Extensions 1) Heter og eneo us upload rate class es µi , i= 1,..., n . σ∂Fti(,)σ ∂ Ft i (,) = σλ i + r i (,) F , ∂t ∂ σ i ri is not the same for all i . Depends on pi ece exchange rules. i Preceding theory extends to the µ 0 y case of "propo rtional reciprocity": ri( F ) =0 + µ i ∑ x j j 2) Dynamics in seeder s, generated by terminating leechers. i State is given by Ft(,)σ and yt () ; dynamics no lo nger monotone. i Local analysis around equilibrium. Can show stability using the small-ga in theorem, or a Lyapunov functional. i Can study the noise response studied through linearization, classical filtering. Matches well with variance in experiments. Outline 1. Networking research: between queues and fluids.
2. M/G- Processor Sharing queues, point process model and its PDE fluid counterpart.
3. Dynamics of peer-to-peer networks.
4. Dispatching deferrable power loads for frequency regulation in the power grid. Real time balancing in the power grid
secondary freq Reserves markets control Forward markets primary freq control sec min 5-60 min Hours, day ahead • Due to limited storage, electric demand and supply must tightly match. Various markets are set up with this objective. • At fast time-scale, balancing is a control engineering problem. • Fast-responding generators are set up to provide “secondary frequency regulation” by following a reference signal. • Can a smarter control of demand help with regulation? • Idea: exploit deferability of certain loads (e.g., electric vehicles), schedule to track a desired consumption profile. • Many references, in particular Poolla’s group in Berkeley. Deferrable loads as a controlled queue
Poisson(λ) load arrivals load served loads depart aggregator For the k − th arrival:
Qk : required energy. uk : service fraction for load k Q σ = k : service time at (fraction of time turned on, k p 0 nominal power . or fraction of nominal power) p :nominal power, assumed lk :laxit y (spare t ime). 0 common to all loads.
• Control decision: choice of uk for each of loads present . • Objective: control aggregate power cons umption to a desired reference. Initially, suppose constant reference. • Constraint: respect deadl ines as much as possible. Point process representation l • •
• • • • −(,1u − u )
• σ • i Arrivals: a new point mass appears, following the joint 2 distribution στ of (,)σ kkl∈R+. Let E[][] k = ; E l k = L . i Service time and laxity are consumed ac cording to fraction u. i Departures when reachi ng σ = 0. i Misses deadline if it crosses line l = 0. Equal sharing: uniform service fraction
Serve all loads present at u p 0 . l • σ τ • ⇒ Sojourn time: T=k , E[] T = • ku k u • • Poisson arrivals→ an M/G/∞ queue, • with stationary Poisson distribution n − λτρ ρ • σ P(N= n ) = e , w ith ρ = . • n! u λτ • Mean # of loads in system: E[]N =ρ = . u
• Consumed power is pNup=0 , mean p =E[][] p = λτ λ p0 = E Q k . Independent of u! Actually this is expecte d, matches mean demand .
• Var[N]= ρ ⇒ Var [ p]= up0 p.Closer to consta nt power as u ↓ . σ l • Probability of missing deadline: P > . Deteriorates as u ↓ . u1− u Least-Laxity-First Scheduling
Serve fraction u of loads l • • present, at nominal power. • Pick those with smallest laxity . • θ (t ) • Proposition: If µ σ k ∼ exp( ), • Fraction u population N as in previous case, • σ λτ Poisson . Same mean and varia nce. u Missed deadlines can be studied through the frontier process θ (t ). In the large scale limit, θ θ λ → ∞. (t ) → * . E[σ ] • If u >k =:η θ ,⇒ * > 0 ⇒ P []deadline miss→ 0 . E[σ k ]+ E [] k E[σ ] • If u Least-laxity first. Standard deviation u >η of power Fraction u <η of missed deadlines . η Alternatives for firm deadlines l • l • • −(u ,1 − u ) • k k • • • • • −(,1u − u ) • • σ σ "Exact scheduling": "Laxity expiring scheduling": Tailor service level uk so each Fixed service fraction u load leaves exactly at deadline. for loads with laxity, serve at full power when it expir es. Analysis of power variance: Analysis of power variance: change of variables turns it results available for into an M/G/∞ queue. exponential jobs/laxities. Fluid models for service deferral What if we want to track a non-constant power reference? Requires a controlled u( t ), queue operating ou tside equilibriu m. Turn to fluid population models. i The most complete representation would be a PDE in (σ ,l , t ). i Start with ODE in load populations, val id in exponential case. Single class, soft deadlines Firm deadlines, "laxity expiring" method, two classes of loads nt() ut () n ()t=λ − + v () t 1 1 τ n (t ) =λ −ntut()() − nt ()(1 − ut ()) + vt1 () noise τ L pt()= pntut0 ()() . 1 1 m (t ) = nt( )(1 − ut ( )) − mt ( ) + vt2 ( ) λτ L τ * pt()= p ntut ()() + mt () . Equilibrium: n = * . 0 u 2 * τλτλτ L(1− u ) Equil : n*= m * = . Lu τ *+−τ (1 u * ) Lu * +− (1 u * ) * * In both cases: p= λτ p0 , independent of u Control: tracking a regulation signal state Referen ce v δ p( t ) δr( t ) P e( t ) C − δu( t ) P: linearized plant around an operating point. C:Controller to track δ rt( ) of mean zero (since p * is fixed). H Design: feedforward + state feedback, 2 − optimal control. Results with real regulation signal from PJM operator. Conclusions • In various instances, network performance is dictated by the dynamics of populations (jobs, peers, energy loads,…). • Relevant stochastic queues: M/G – processor sharing, M/G/ ∞. Point process state. Stationary distribution can be sometimes be found through insensitivity. • Fluid differential equation models have a wider applicability. To capture general job sizes, a transport PDE is required. • Control theory tools apply to analysis (Lyapunov, small gain, monotone systems) or synthesis ( H2 regulator, etc.) • Future work : – Deferrable energy loads: other policies, decentralized implementation and incentives. – Processor-sharing in cloud computing systems . References • PDE model and Internet stability conjecture: – F. Paganini, A.Tang., A. Ferragut, L. Andrew, “Network Stability under Alpha Fair Bandwidth Allocation with General File Size Distribution”, IEEE Trans. on Automatic Control , Vol 57(3), pp. 579-591, 2012. • Peer-to-peer dynamics: – A. Ferragut, F. Paganini, “PDE models for population and download progress in P2P networks”, IEEE Trans. on Control of Nwk Sys, 2015 . – “A. Ferragut , F. Paganini, “ Queueing analysis of peer -to -peer swarms: stationary distributions and their scaling limits” Perf. Evaluation, 2015. – F. Paganini, A. Ferragut, “Monotonicity and global stability in download dynamics of content-sharing networks” Proc. CDC 2014, Los Angeles. • Deferrable power loads. – F. Bliman, A. Ferragut, F. Paganini , “Controlling aggregates of deferrable loads for power system regulation”, Proc. ACC 2015, Chicago. – A. Ferragut, F. Paganini , “Queueing analysis of service deferrals for load management in power systems”, Proc. Allerton Conference 2015. Thank you! Questions?