The Need for Realism when Simulating *

Kevin Mills and Chris Dabrowski National Institute of Standards & Technology

Communications & Networking Symposium April 3-6, 2016

*For more details see: NIST Technical Note 1905 http://dx.doi.org/10.6028/NIST.TN.1905 Total talk is ≈ 16 slides • Motivation – 2 slides • Research Questions and Approach – 2 slides • Models – 5 slides • Experiment Design – 1 slide • Results – 5 slides • Findings – 1 slide

April 3-6, 2016 Communications & Networking Symposium 2 Academics Model Spreading Network Congestion as a Percolation Process

All Find that Signals Appear Near a Critical Point in Abstract Network Models

April 3-6, 2016 Communications & Networking Symposium 3 Abstract Models Lack Key Traits of Real Networks

1. Human-engineered, tiered topologies, with propagation Routers & 2. buffer sizes finite Links 3. Router speeds varied to meet demands, limit losses 4. Injection from sources and receivers only at lowest tier Computers 5. Distribution of sources and receivers non-uniform 6. Connection of sources/receivers with few varied speeds 7. Duty cycle of sources exhibits cyclic behavior Users 8. Human sources exhibit limited patience 9. Sources transfer flows of various sizes

Protocols 10. Flows use the Transmission Control Protocol (TCP) to modulate injection rate based on measured congestion

DOES LACK OF REALISM MATTER WHEN SIMULATATING NETWORK CONGESTION?

April 3-6, 2016 Communications & Networking Symposium 4 Specific Research Questions

1. Does congestion spread in abstract models mirror spread in realistic models? 2. Are some elements of realism essential to capture when modeling network congestion? 3. Are some elements unnecessary? 4. What measures of congestion can be compared, and how, across diverse network models?

April 3-6, 2016 Communications & Networking Symposium 5 Research Approach

Abstract Realistic Model from Model from Literature Literature

Basic Model Factor into Behaviors Flexible Realism Elements Network that can be Simulator enabled or disabled (FxNS) Simulate Combinations of Realism Elements and Compare Patterns of Congestion

April 3-6, 2016 Communications & Networking Symposium 6 Models

• Abstract EGM Model→high abstraction • Realistic MesoNet Model→high realism • Flexible FxNS Model→combinations of realism from low to high

April 3-6, 2016 Communications & Networking Symposium 7 The Abstract (EGM) Model

P. Echenique, J. Gomez-Gardenes, and Y. Moreno, “Dynamics of Jamming Transitions in Complex Networks”, Europhysics Letters, 71, 325 (2005)

γ = -2.2

-γ Simulations based on 11,174-node scale-free graph, Pk ~ k & γ=2.2, taken from a 2001 snapshot of the Internet Autonomous System (AS) topology collected by the Oregon Router Server (image courtesy Sandy Ressler)

April 3-6, 2016 Communications & Networking Symposium 8 Details of the EGM Model Node Buffer Size: ∞ for EGM, all packets buffered, no packets dropped Injection Rate: p packets injected at random nodes (uniform) at each time step Destination Node: chose randomly (uniform) for each packet Forwarding Rate: 1 packet per node at each time step Routing Algorithm: If node is destination, remove packet; Otherwise select

next-hop as neighboring node i with minimum di System Response: proportion ρ of injected packets queued in the network

Computing di Measuring ρ h is a traffic awareness parameter, whose value 0 ... 1.

A = aggregate number of packets where i is the index of a node’s neighbor, t = time di is minimum #hops to destination via τ = measurement interval size neighbor i, and ci is the queue length of i. p = packet inject rate h = 1 is shortest path (in hops)

April 3-6, 2016 Communications & Networking Symposium 9 Comparative Simulation Results

FxNS Simulations with EGM Simulations All Realism Elements Disabled 1

0.8 h=0.85

0.6

ρ ρ 0.4 h=1

0.2

0 0 5 10 15 20 25 30 p p

April 3-6, 2016 Communications & Networking Symposium 10 The Realistic (MesoNet) Model K. Mills, E. Schwartz, and J. Yuan, "How to Model a TCP/IP Network using only 20 Parameters", WSC 2010, Dec. 5-8, Baltimore, MD.

I2c I1c I1d I2b I2d K1c K1d Category ID Name FxNS A2a A2b A2c I1b I2a

K1b K2a K2b A1a I1a I2 x1 topologySpecific NC A1b I2e K1a C1a H1c H1b I1 K1 K2c A2 18% H1a I2f x2 propagation delay DE A1c C1b H1d K2 Network A1 I2g K0a K2d H1e I x3 network14% speed VS C1c I0a A C1 H1f J1b K B2d H1 J1c x4 buffer provisioning PD B2c B2e J1a J1d C B2f H J1e J2a J1 L1a x5 number sources/sinks H2a L1b B2b B2g J2b L1c H2b J1f L0b L1d Sources & x6 source distribution SR C2 H2 J L0a B2 H2c J2c J2 Sinks x7 sink distribution C2a L1 B2a H2d L2a C2b J2d C2c H2e x8 source/sink speed VS H2f J2e J2f L L2 L2b M2f M2d M2e M2g x9 think time p B L2c M2c D2b D2c D2d D2a N1a L2d M2b M2 x10 patience n/a B1a B1 F1c F1d F1b N1b B0a D2e M1a M2a D2 N1c F1a M x11 web object file sizes FL B1b Users D2f M1b M1 N N1 N1d D2g F1 M1c x12 larger file sizes B1c B1d D F2a F2b N1e M1d N2 M1e M1f M1g N1f x13 localized congestion n/a F F2c D1a D1 F2 O N2a N2f O0a

x14 long-lived flows F0a F2d N2b N2e D1b N2c N2d D0a E O1 x15 control algorithm F2e O2 O2g D1c F2g F2f Congestion D1d O1a O1b O2f TCP E1a x16 initial cwnd E1 O1c Control O2e E2 O1c O2a x17 Initial sst G1a G O2b O2c O2d

E1b G1b P x18 measurement interval fixed E1c P1a P2g Simulation E2a E2c G1c G1 G2 G2f E2b P1 P2 x19 simulation duration fixed Control Figure 2. G2e P1b P2f

G1d

x20 startup pattern p G1e P1c P2a G1f G2d P2e G2a G2b G2c P1d P2b P2c P2d Comparisons of MesoNet Simulations vs. FxNS Simulations (all realism elements enabled) for eight MesoNet responses are available in NIST TN 1905 – Appendix A April 3-6, 2016 Communications & Networking Symposium 11 FxNS Combinations

7 Realism Elements 7 Dependencies among Realism Elements

34 Valid FxNS Combinations Seq Cmb TCP FL SR DE VS NC PD 1 c0 0 0 0 0 0 0 0 2 c1 0 0 0 0 0 0 1 3 c2 0 0 0 0 0 1 0 . . . 32 c123 1 1 1 1 0 1 1 33 c126 1 1 1 1 1 1 0 34 c127 1 1 1 1 1 1 1

April 3-6, 2016 Communications & Networking Symposium 12 Experiment Design

Enabled Disabled PD buffers = 250×router speed buffers = ∞ FIXED PARAMETERS 3-tier 218-node topology flat 218-node • 218-Router Topology (Fig. 2) as in Fig. 2 with routers topology as in NC labeled as core, PoP, D- Fig. 2 but with • Routing (SPF propagation delay) class, F-class or N-class routers unlabeled • Duration (200,000 ts per p) core 80 p/ts; PoP 10 p/ts; D-class 10 p/ts; F-class 2 all routers and VS p/ts; N-class 1 p/ts; fast sources/sinks 9 source/sink 2 p/ts; normal p/ts VARIABLE PARAMETERS source/sink 0.2 p/ts • Packet-Injection Rate p (up to 2500) core links have no propagation DE propagation delays delays • FxNS Combination 51,588 sources & 206,352 no sources or SR sinks deployed uniformly sinks deployed below access routers RESPONSES transfers are packet streams: sized randomly • Congestion Spread χ=|Gχ|/|GN| from Pareto distribution transfers are FL • (mean 350, shape 1.5) - individual packets Connectivity Breakdown α=|Gα|/|GN| streams set up with TCP • Proportion of Packets Delivered π connection procedures packet transmission • Scaled (0..1) Latency of Delivered Packets δ packet regulated by TCP transmissions not congestion-control TCP regulated by including slow-start (initial congestion- cwnd = 2 sst = 230/2) and Only concepts in common among all 34 control congestion avoidance combinations: graph and packet

April 3-6, 2016 Communications & Networking Symposium 13 Results1,2

[1] 136 xy-plots (34 FxNS combinations × 4 responses) are available at: http://tinyurl.com/poylful

[2] Related FxNS simulation data can be explored interactively using a multidimensional visualization created by Phillip Gough of CSIRO: http://tinyurl.com/payglq6

April 3-6, 2016 Communications & Networking Symposium 14 Results I – Abstract (C0) vs. Realistic (C127)

C0

C127 C127 C0

C127

C0

C0 C127

Plots for all responses and all 34 combinations available: http://tinyurl.com/poylful April 3-6, 2016 Communications & Networking Symposium 15 Results II – Congestion Spread χ All Combinations

April 3-6, 2016 Communications & Networking Symposium 16 Results III – Connectivity Breakdown α All Combinations

April 3-6, 2016 Communications & Networking Symposium 17 Results IV – Packet Delivery π All Combinations

April 3-6, 2016 Communications & Networking Symposium 18 Results V – Scaled Packet Latency δ All Combinations

April 3-6, 2016 Communications & Networking Symposium 19 Findings • Congestion spreads differently in abstract and realistic models • Hierarchical Router Speeds and TCP very important to model • Packet dropping important to model for accurate packet latencies • Propagation delay not important to model in a continental US network, but would be important to model in topologies where propagation delays exceed queuing delays • Congestion spread, connectivity breakdown and the effectiveness and efficiency of packet delivery can be measured using only two concepts: graphs and packets

April 3-6, 2016 Communications & Networking Symposium 20 For more of our research see: http://www.nist.gov/itl/antd/emergent_behavior.cfm

April 3-6, 2016 Communications & Networking Symposium 21