IODA - Internet Outages: Detection & Analysis
Total Page:16
File Type:pdf, Size:1020Kb
IODA - Internet Outages: Detection & Analysis Alberto Dainotti [email protected] www.caida.org Center for Applied Internet Data Analysis University of California, San Diego CAIDA intro The Center for Applied Internet Data Analysis (CAIDA) is an independent analysis and research group based at the University of California's San Diego Supercomputer Center. CAIDA investigates both practical and theoretical aspects of the Internet. Center for Applied Internet Data Analysis http://www.caida.org/home/about/ University of California San Diego 2 www.caida.org CAIDA research highlights • topology analysis • modeling complex networks • Internet-scale router alias resolution • using hidden metric spaces • comparing IPv6 & IPv4 topology • Internet topology data sharing • geolocation analysis • comparing geolocation services • security & stability • IP reputation vs. governance • large-scale Internet outages • botnet activity • future Internet • BGP hijacks • IPv6 • Named Data Networking • Internet peering analysis • inferring AS relationships • visualization • AS ranking • interconnection economics • modeling peering strategies • transit pricing Center for Applied Internet Data Analysis http://www.caida.org/publications/ University of California San Diego 3 www.caida.org CHRONOLOGY CAIDA and Internet Outages •Jan/Feb 2011 - Internet Kill Switch in Egypt and Libya •Nov 2011 - We present a novel approach to study Internet Outages by combining different types of Internet measurements •Jan 2012 - We present a study on the impact of natural disasters on the network infrastructure •Sep 2012 - NSF funds CAIDA to further develop our methodology and build an experimental operational deployment to monitor the public IPv4 Internet (IODA) •2012 - 2015 — more science and a lot of engineering •Today a prototype that starts to be quite usable Center for Applied Internet Data Analysis University of California San Diego 4 www.caida.org BEFORE IODA post-event manual analysis •Country-level Internet Blackouts Egypt, Jan 2011 during the Arab Spring Government orders to shut down the Dainotti et al. “Analysis of Country-wide Internet Internet Outages Caused by Censorship” ACM SIGCOMM IMC 2011 •Natural disasters affecting the infrastructure climbs slowly, reaching pre-event levels only after a week, which correlates with the restoration of power in the Christchurch area [?]. Japan, Mar 2011 Dainotti et al. “Extracting Benefit from Earthquake 180 of Magnitude 9.0 EARTHQUAKE Harm: Using Malware Pollution to Analyze 160 the Impact of Political and Geophysical EPICENTER Events on the Internet” 140 ACM SIGCOMM CCR 2012 120 100 Center for Applied Internet Data Analysis University of California San Diego(a) Christchurch (b) Tohoku 80 5 www.caida.org 60 Figure 5: Networks selected within the estimated maximum radius of im- pact of the earthquake (20km for Christchurch and 304km for Tohoku). We Number of distinct IPs per hour 40 based our geolocation on the publicly available MaxMind GeoLite Country 20 database. 0 02-18 00:00 02-20 00:00 02-22 00:00 02-24 00:00 02-26 00:00 02-28 00:00 03-02 00:00 03-04 00:00 4 (x=137,y=3.59) 3.5 (x=6,y=2.0) 3 Figure 7: Rate of unique source IP addresses found in unsolicited traffic 2.5 reaching the UCSD network telescope from networks geolocated within a 2 ⇢max =20km range from the Christchurch earthquake epicenter. The 1.5 rate of distinct IPs per hour drops immediately after the earthquake. Peaks before the earthquake were above 140-160 IPs/hour on weekdays (weekend 1 is on 19-20 February), while the first peak after the earthquake is slightly 0.5 above 100 IPs/hour. Levels remain lower for several days, consistent with - Ratio of distinct IPs before/after earthquake the slow restoration of power in the area. θ 0 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500 Km Figure ?? plots the same graph for IBR traffic associated with the Christchurch Tohoku Tohoku earthquake, within a maximum distance ρmax =304km from the epicenter. The much steeper drop in the number of unique Figure 6: Measuring the impact of the earthquake on network connectivity as seen by the telescope: value of ✓ for all networks within a given range IPs per hour sending IBR traffic is consistent with the Tohoku earth- from the epicenter. The peak value ✓max reached by ✓ can be considered quake’s much larger magnitude than that of the Christchurch earth- the magnitude of the impact. quake. In the days after the event the IBR traffic starts to pick up again, but does not reach the levels from before the event during the analyzed time interval, also consistent with the dramatic and kilometers from its epicenter, consistent with the stronger magni- lasting impact of the Tohoku earthquake on Northern Japan. tude of Tohoku’s earthquake (see Table ??) and news reports re- garding its impact on buildings and power infrastructure. Table ?? 800 summarizes these indicators found for both earthquakes. 700 EARTHQUAKE Christchurch Tohoku 600 Magnitude (✓max) 2 at 6km 3.59 at 137km Radius (⇢max) 20km 304km 500 Table 3: Indicators of earthquakes’ impact on network connectivity as ob- 400 served by the UCSD network telescope. 300 Number of distinct IPs per hour 200 IBR traffic also reveals insight into the evolution of the earth- quake’s impact on network connectivity. Figure ?? plots the num- 100 ber of distinct source IPs per hour of packets reaching the telescope 03-04 00:00 03-06 00:00 03-08 00:00 03-10 00:00 03-12 00:00 03-14 00:00 03-16 00:00 03-18 00:00 03-20 00:00 03-22 00:00 from networks within the ρmax =20km radius from the epicenter of Christchurch’s earthquake. All times are in UTC. The time range starts approximately one week before the earthquake and ends two weeks after. We would not expect the IBR traffic to drop to zero, Figure 8: Rate of unique source IP addresses found in unsolicited traffic reaching the UCSD network telescope from networks geolocated within for two reasons. First, not all networks are necessarily disabled by ⇢max =304km of the Tohoku earthquake epicenter. The rate of distinct the earthquake. Second, the geolocation database services we use IPs per hour shows a considerable drop after the earthquake which does not are not 100% accurate. return to previous levels even after several days. For a few days before the event, peaks are always above 140 unique IP addresses per hour (IPs/hour) on weekdays, sometimes Figures ?? and ?? show that the rate of unique IP addresses per above 160 IPs/hour. In the 24 hours after the earthquake, the rate hour observed by the telescope matches the dynamics of the earth- drops, with a peak slightly above 100 IPs/hour. The IPs/hour rate quakes, reflecting their impact on network connectivity. In order to BEFORE IODA post-event manual analysis Egypt, Jan 2011 Government orders to shut down the Internet 4 months of work Center for Applied Internet Data Analysis University of California San Diego 6 www.caida.org IODA TODAY live Internet monitoring Last Christmas we made it possible for anybody to follow the North Korean disconnection almost live Center for Applied Internet Data Analysis University of California San Diego https://charthouse.caida.org/public/kp-outage 7 www.caida.org IODA TODAY let’s see how Internet providers are doing in the US EVERYTHING LOOKS FINE… Center for Applied Internet Data Analysis University of California San Diego 8 www.caida.org IODA TODAY Same type of graphs at the end of August 2014 TWO INDICATORS SHOW SOMETHING WENT WRONG WITH TIME WARNER Center for Applied Internet Data Analysis University of California San Diego 9 www.caida.org IODA the system at a glance BGP BGP TREAM PER-REGION PER-AS … VISIBILITY VISIBILITY BGPWATCHER FULLPFXS PEERTABLES BGPREADER BGPCORSARO BGPSTREAM LIBRARY BGPDUMP BGPARCHIVE BGPDOWNLOADER l CHARTH USE i Charthouse b Charthouse Frontend T Backend i Corsaro m IBR .cor .pcap tools cor2ascii cor-agg corsaro [your name here] e libcorsaro intervals I/O logging .log S ace r libt initialize process packet interval start interval end finalize e Whisper plugins Flow-Tuple DoS [yours!] .ft.cor .dos.cor [.you.cor] r i e Graphite s Active Probing data processing Detection & Inference pfx-to-AS libiPmeta Geo-location Center for Applied Internet Data Analysis University of California San Diego http://www.caida.org/funding/ioda/ 10 www.caida.org IODA Internet Outages: Detection & Analysis •multiple types of sources and methodologies ... - Routing Plane [BGP] BGP - Data Plane - Active probing [pinging + traceroutes] IBR ACTIVE - Passive [IBR] PROBING - easy to plug new sources •meta-data to extract liveness signals for various aggregations (countries, counties, cities, /24 address blocks, prefixes, ASNs) •combining signals to detect & monitor •trigger ad hoc active measurements when an event is detected •visual interface for analysis and dashboards Center for Applied Internet Data Analysis University of California San Diego http://www.caida.org/funding/ioda/ 11 www.caida.org BGP BGP Border Gateway Protocol •BGP - The protocol that establishes routes between ISP networks (autonomous systems) all over the world •Autonomous Systems (AS) - “a set of routers under a single technical administration, using an interior gateway protocol and common metrics to determine how to route packets within the AS, and using an inter-AS routing protocol to determine how to route packets to other ASs.” - RFC 4271 •Network Prefixes - Smaller networks are identified by a network address and a network mask - e.g. prefix 192.172.0.0/16 is assigned to AS99 and is reachable by AS67 through the AS path: AS67➔AS44➔AS15➔AS99 ➔ 192.172.0.0/16 - AS paths are computed by exchanging BGP update messages: “Hey, I’m AS44 and can reach 192.172.0.0/16 through AS15➔AS99” Center for Applied Internet Data Analysis University of California San Diego 12 www.caida.org BGP DATA COLLECTION Route Collectors and Route Monitors BGP •BGP measurement projects establish peering sessions with ASes to receive their routing tables (no exchange of other traffic) Figure 2: BGP Update Stream (sil: silence period; rec: session •RouteViews (Univ.