Provisioning IP Backbone Networks Based on Measurements Konstantina Papagiannaki A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy of the University of London. Department of Computer Science University College London 13th February 2003 2 ¢¡¤£¦¥¨§ © ¥ £ ¡ ¨¡ ¤¥ "!# %$'&"()¨¨"*+§¤§¤,¨-.* /¡ £¦¥¨£0¡¤1¤¥ 23¨4 4*65¨£¦7&"()¨¨8*6§¤§¤,¨-.* To the memory of my father Georgios D. Papagiannakis To my mother Kalliopi G. Papagiannaki Abstract The theme of this thesis is the enhancement of current IP backbone provisioning practices in the presence of additional network measurements. Current practices are heavily dependent on the intuition of the human operators. Traffic variability, scalability issues, lack of monitoring information, and complex interactions between inter- and intra-domain routing protocols result in network management techniques that usually rely on trial and error. In contrast with reductionist approaches, we demonstrate the benefits of using different types of monitoring information in the formalisation of different network provisioning tasks, and provide a methodological framework for their analysis. We use four main sources of network monitoring information: (i) GPS-synchronised packet traces listing every packet traversing a monitored unidirectional link, (ii) BGP routing table dumps, (iii) SNMP information collected since 1999, and (iv) topological information. Combining the above sources of information, and analysing them at the appropriate time scale, we demonstrate the benefits of additional measurements on three specific network provisioning tasks. First, we measure and analyse delay as experienced by packets while traversing a single router inside the network. We show that packets experience minimal queueing delay and that delay through the network is dominated by the propagation delay. Our results hold when network link utilisation stays moderate. However, links are likely to experience short-lived congestion episodes as a result of link or equipment failures. Our second network provisioning task regards the off-loading of congested links by the re-direction of high-volume flows. We propose a methodology for the identification of those flows traversing a link that contribute significant amounts of traffic consistently over time. Persistent link overload can only be resolved through additional provisioning. Our third task focuses on the prediction of where and when future provisioning will be required in the backbone. We obtain accurate predictions for at least six months in the future. Contents 1 Introduction 14 1.1 The Internet History . 14 1.2 From ARPANET to the INTERNET . 15 1.3 The Emergence of Internet Service Providers . 15 1.4 Today’s Internet . 16 1.5 Challenges . 17 1.6 Provisioning IP networks . 18 1.7 The need for additional network measurements . 18 1.8 Thesis Outline . 19 2 Background 22 2.1 The Internet Routing Hierarchy . 22 2.2 Backbone Network Architecture . 22 2.3 Intra-domain Routing . 24 2.4 Inter-domain Routing . 25 2.5 Network Management . 27 2.5.1 Simple Network Management Protocol (SNMP) . 27 2.5.2 Remote Monitoring MIB (RMON) . 28 2.6 Challenges . 28 2.6.1 Network Management . 28 2.6.2 Network Design . 29 2.6.3 Service Level Agreements . 30 2.6.4 New Services over Data Networks . 30 2.6.5 Traffic Variability . 30 2.6.6 Network Growth . 31 2.6.7 Accounting and Pricing . 31 2.7 Summary . 32 3 State of the Art 33 3.1 Network Measurements . 33 3.1.1 Active Network Measurements . 33 Contents 5 3.1.2 Passive Network Measurements . 34 3.2 The IP Monitoring (IPMON) infrastructure . 35 3.2.1 IPMON system architecture . 36 3.2.2 Data Rate Requirements . 37 3.2.3 Timestamp Requirements . 38 3.2.4 Physical Requirements . 39 3.2.5 Security Requirements . 39 3.2.6 Remote Administration Requirements . 40 3.3 Routing Protocol Listeners . 40 3.4 Network Provisioning Based on Measurements . 41 3.4.1 Network Traffic Demand Matrix . 41 3.4.2 Traffic Engineering . 42 3.4.3 Tuning Routing Protocol Parameters . 43 3.5 Summary . 44 4 Analysis of Measured Single-Hop Delay from an Operational Backbone Network 45 4.1 Introduction . 45 4.2 Measurement Environment . 47 4.2.1 Collected Data . 47 4.2.2 Router Architecture . 48 4.3 Delay Measurement . 51 4.3.1 Matching Packets . 51 4.3.2 Representativeness of the Data . 52 4.4 Delay Analysis . 53 4.4.1 General Observations . 56 4.4.2 Step-by-Step Analysis of the Single-Hop Delay . 57 4.4.3 Possible Causes for Very Large Delay . 62 4.4.4 Filtering Based on a Single Output Queue Model . 64 4.5 Analysis of Output Queueing Delay . 69 4.5.1 Tail Behaviour . 69 4.5.2 Impact of Link Utilisation on Output Queueing Delay . 71 4.6 Conclusions . 72 5 A Pragmatic Definition of Elephants in Internet Backbone Traffic 74 5.1 Introduction . 75 5.2 Measurement environment . 77 5.2.1 Collected measurements . 77 5.2.2 Traffic and Time Granularity . 78 5.3 Elephant Classification . 80 6 Contents 5.3.1 Single-Instant Classification . 81 5.3.2 Incorporating temporal behaviour in the classification . 82 5.3.3 Performance Metrics . 84 5.4 Results on the Single-Instant Classification Schemes . 85 5.4.1 Classification Results . 85 5.4.2 Temporal Behaviour of Classification . 87 5.4.3 Other Single-Instant Classification Techniques . 91 5.4.4 Other Timescales for Measurement and Classification . 92 5.5 Latent Heat Classification Results . 93 5.6 Profile of Elephant Flows . 94 5.7 Conclusions . 97 6 Long-Term Forecasting of Internet Backbone Traffic 99 6.1 Introduction . 100 6.2 Related Work . 101 6.3 Objectives . 102 6.4 Measurements of inter-PoP aggregate demand . 102 6.4.1 Data collected and analysis . 102 6.4.2 Initial observations . 103 6.5 Multi-timescale Analysis . 105 6.5.1 Wavelet MRA overview . 106 6.5.2 MRA application on inter-PoP aggregate demands . 107 6.5.3 Analysis of Variance . 110 6.5.4 Summary of findings from MRA and ANOVA . 111 6.5.5 Implications for modelling . 112 6.6 Time Series Analysis using the ARIMA model. 113 6.6.1 Overview of linear time series models . 113 6.6.2 Time series analysis of the long-term trend and deviation . 115 §©¨¢ ¡¤£¦¥ 6.6.3 Models for ¢¡¤£¦¥ , and . 115 6.6.4 Evaluation of forecasts . 117 6.7 Discussion and Future Work . 118 6.8.