A second look at “Detecting third-party addresses in traceroute traces with the IP timestamp option”
Matthew Luckie, k claffy [email protected] ! CAIDA - University of California, San Diego
Passive and Active Measurement Conference (PAM) 2014
1 www.caida.org RELATED WORK ON TRACEROUTE SHORT-COMINGS • Significant volume of literature reporting the short-comings of traceroute
- Oliveira et al.: Observing the Evolution of Internet AS Topology. SIGCOMM 2007
- Willinger et al.: Mathematics and the Internet: a source of enormous confusion and great potential. AMS 2009
- Zhang et al.: Quantifying the pitfalls of traceroute in AS connectivity inference. PAM 2010
2 www.caida.or RELATED WORK ON TRACEROUTE SHORT-COMINGS • Significant volume of literature reporting the short-comings of traceroute
- Oliveira et al.: Observing the Evolution of Internet AS Topology. SIGCOMM 2007
- Willinger et al.: Mathematics and the Internet: a source of enormous confusion and great potential. AMS 2009
- Zhang et al.: Quantifying the pitfalls of traceroute in AS connectivity inference. PAM 2010
- P. Marchetta, W. de Donato, A. Pescape: Detecting third-party addresses in traceroute traces with IP timestamp option. PAM 2013
2 www.caida.or ASSUMED TRACEROUTE BEHAVIOR R4 AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
3 www.caida.or ASSUMED TRACEROUTE BEHAVIOR R4 AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1
3 www.caida.or ASSUMED TRACEROUTE BEHAVIOR R4 AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1 TTL=2 x3
3 www.caida.or ASSUMED TRACEROUTE BEHAVIOR R4 AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1 TTL=2 x3 TTL=3 y2
3 www.caida.or ASSUMED TRACEROUTE BEHAVIOR R4 AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1 TTL=2 x3 TTL=3 y2 AS path inference: X Y
3 www.caida.or HOW SHOULD A ROUTER BEHAVE? (RFC 1812)
4.3.2.4 ICMP Message Source Address
Except where this document specifies otherwise, the IP source address in an ICMP message originated by the router MUST be one of the IP addresses associated with the physical interface over which the ICMP message is transmitted. If the interface has no IP addresses associated with it, the router's router-id (see Section [5.2.5]) is used instead.
That is, the address of the out-bound interface from which the ICMP message is sent
4 www.caida.or OFF-PATH AND THIRD-PARTY ADDRESSES R4 AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
5 www.caida.or OFF-PATH AND THIRD-PARTY ADDRESSES off-path R4 interfaces AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
5 www.caida.or OFF-PATH AND THIRD-PARTY ADDRESSES third-party off-path address R4 interfaces AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
5 www.caida.or OFF-PATH AND THIRD-PARTY ADDRESSES third-party off-path address R4 interfaces AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1
5 www.caida.or OFF-PATH AND THIRD-PARTY ADDRESSES third-party off-path address R4 interfaces AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1 TTL=2 z1
5 www.caida.or OFF-PATH AND THIRD-PARTY ADDRESSES third-party off-path address R4 interfaces AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1 TTL=2 z1 TTL=3 y4
5 www.caida.or OFF-PATH AND THIRD-PARTY ADDRESSES third-party off-path address R4 interfaces AS Z z1 y4 src R1 R2 R3 dst S x1 AS X x2 x3 AS X y1 y2 AS Y y3 D
TTL=1 x1 TTL=2 z1 TTL=3 y4 AS path inference: X Z Y
5 www.caida.or FINDINGS OF MARCHETTA ET AL. (PAM2013)
• Most classifiable addresses in traceroute paths are off-path
• Consecutive off-path addresses are common
- More than half of off-path sequences were at least 3 hops
• Presence of off-path addresses in traceroute much more widespread than previously believed
6 www.caida.or WHAT MOTIVATED OUR WORK?
• If technique and results from Marchetta et al. PAM2013 are correct:
- traceroute is unfit for purpose, operationally and in research.
- their technique would help us make more solid inferences.
• But: no validation reported.
• Our goal was to assess the correctness of their technique and findings.
7 www.caida.or IP PRE-SPECIFIED TIMESTAMPS
Routers should embed IP Header a timestamp if the specified interface is visited Pre-specified IP #1 Space for TS #1 Pre-specified IP #N Space for TS #N IP Options Pre-specified IP #4 } Sherry et al. notation: Space for TS #4 a packet sent to Z that asks Transport Header the routers with addresses A, B, C, D to embed timestamps is written as Z|ABCD
8 www.caida.or CRITICAL ASSUMPTION (Marchetta et al. PAM 2013)
If a router does not embed a timestamp for a specified IP address when forwarding a packet to a destination X, but does embed timestamps when packets are sent to the router, then an address observed in traceroute towards X was not in the forwarding path.
9 www.caida.or TP-TRACEROUTE ALGORITHM (Marchetta et al. PAM 2013) e src dst R1 R2 a b c d f g
10 www.caida.or TP-TRACEROUTE ALGORITHM (Marchetta et al. PAM 2013) e src dst R1 R2 a b c d f g 1. UDP Timestamp g|gggg Port unreach w/ IP timestamp option quoted If no response, or no timestamp quote, then stop.
10 www.caida.or TP-TRACEROUTE ALGORITHM (Marchetta et al. PAM 2013) e src dst R1 R2 a b c d f g 1. UDP Timestamp g|gggg Port unreach w/ IP timestamp option quoted
b e 2. UDP Traceroute g g Infer the forward IP path towards g.
10 www.caida.or TP-TRACEROUTE ALGORITHM (Marchetta et al. PAM 2013) e src dst R1 R2 a b c d f g 1. UDP Timestamp g|gggg Port unreach w/ IP timestamp option quoted
b e 2. UDP Traceroute g g 3. ICMP Timestamp b|bbbb ICMP response w/1-3 timestamps from b b ICMP response w/1-3 timestamps from e 4. ICMP Timestamp e|eeee e If no response, or 0 timestamps, (does not support option) or 4 timestamps, (will embed timestamp regardless of visiting interface) then tptraceroute cannot infer if the interface was visited or not.
10 www.caida.or TP-TRACEROUTE ALGORITHM (Marchetta et al. PAM 2013) e src dst R1 R2 a b c d f g 1. UDP Timestamp g|gggg Port unreach w/ IP timestamp option quoted
b e 2. UDP Traceroute g g 3. ICMP Timestamp b|bbbb ICMP response w/1-3 timestamps from b b ICMP response w/1-3 timestamps from e 4. ICMP Timestamp e|eeee e Port unreach w/1-3 timestamps from b 5. UDP Timestamp g|bbbb g Because 1-3 timestamps were inserted by b, infer it is on-path
10 www.caida.or TP-TRACEROUTE ALGORITHM (Marchetta et al. PAM 2013) e src dst R1 R2 a b c d f g 1. UDP Timestamp g|gggg Port unreach w/ IP timestamp option quoted
b e 2. UDP Traceroute g g 3. ICMP Timestamp b|bbbb ICMP response w/1-3 timestamps from b b ICMP response w/1-3 timestamps from e 4. ICMP Timestamp e|eeee e Port unreach w/1-3 timestamps from b 5. UDP Timestamp g|bbbb g Port unreach w/0 timestamps from e 6. UDP Timestamp g|eeee g Because 0 timestamps were inserted by e, infer it is off-path
10 www.caida.or CROSS VALIDATION OF TP-TRACEROUTE
• Limit ourselves to interfaces we infer are the in-bound interface on a router:
- For those interfaces, what inference does tp-traceroute make?
11 www.caida.or CROSS VALIDATION OF TP-TRACEROUTE
• Limit ourselves to interfaces we infer are the in-bound interface on a router:
- For those interfaces, what inference does tp-traceroute make?
a b
11 www.caida.or CROSS VALIDATION OF TP-TRACEROUTE
• Limit ourselves to interfaces we infer are the in-bound interface on a router:
- For those interfaces, what inference does tp-traceroute make? point-to-point? (i.e. /30 or /31) a b
11 www.caida.or CROSS VALIDATION OF TP-TRACEROUTE
• Limit ourselves to interfaces we infer are the in-bound interface on a router:
- For those interfaces, what inference does tp-traceroute make? point-to-point? (i.e. /30 or /31) a a’ b
is a’ a /30 or /31 mate of b and on same router as a?
11 www.caida.or ARE a+a’ ON THE SAME ROUTER? • Also known as alias resolution
- extensive validation history: Rocketfuel SIGCOMM2002, Radargun IMC2008, MIDAR ToN2013
• Two techniques used in this work:
- repeated Ally-style tests
• using ICMP-echo, TCP-ack, and UDP probes
• monotonic IPID sequence from non-overlapping probes/replies to a and a’, repeated every 10 minutes for an hour to allow divergence
- one-off Mercator test (if necessary)
• responses to probes to a and a’ come from common source address
12 www.caida.or OUR METHOD
• Eight CAIDA Archipelago (Ark) vantage points (VPs)
• Each obtained 10K traceroutes to responding destinations chosen at random from ISI Census data
- for each hop, classify as on- or off-path using tp-traceroute (Marchetta et al.) technique
- for each link, infer if point-to-point (our cross-validation) using prefixscan in scamper
13 www.caida.or OUR DATA
• 197,335 IP-level links
• 81,315 inferred point-to-point IP links (interface B on-path)
- In our data, between 77.1% and 90.0% of interfaces in traceroute on point-to-point links were classified by tp-traceroute as off-path
Pre-specified IP Timestamps are an unreliable primitive to determine if an address is on- or off-path
14 www.caida.or WHAT FRACTION OF INTERFACES IN TRACEROUTE ARE IN-BOUND?
1 0.8 0.6 0.4 0.2 interfaces in paths
Fraction of inbound 0 ams3 nl gva ch nrt jp per au sin sg sql us syd au zrh2 ch Ark VP For 7 of 8 VPs, more than half of the interfaces observed in a traceroute were in-bound. Lower-bound: these are just the routers we could resolve for aliases.
15 www.caida.or LIMITATION (multiple point-to-point links can exist between routers, and the address observed in traceroute might be off-path)
a point-to-point R1 R2 links b
16 www.caida.or LIMITATION (multiple point-to-point links can exist between routers, and the address observed in traceroute might be off-path)
traceroute probe a point-to-point R1 R2 links b
16 www.caida.or LIMITATION (multiple point-to-point links can exist between routers, and the address observed in traceroute might be off-path)
traceroute probe a point-to-point R1 R2 links b ICMP response from off-path address b
16 www.caida.or SUMMARY
• Traceroute has an important role in overcoming the visibility issue of AS topology data because we have no other way of uncovering some peerings.
• Traceroute-derived AS paths are messy to work with due to artifacts in IP2AS mappings.
• Presence of off-path addresses not as wide-spread as suggested by Marchetta et al. in PAM2013.
• Deriving a technique that accurately infers AS links from traceroute paths remains an important and currently unsolved problem
17 www.caida.or CONFIDENCE IN ALIAS INFERENCES • Alias resolution has an extensive validation history - Rocketfuel SIGCOMM2002, Radargun IMC2008, MIDAR ToN2013 • IP-ID techniques are generally reliable when they declare two addresses as aliases 1 prefixscan N=86152 0.8 pairwise N=81315 50%: 42 0.6
CDF 0.4 50%: 75 0.2 0 1 10 100 1000 IP ID difference (log scale)
18 www.caida.or SAMPLING OF INTERFACES 0.7 0.6 0.5 1,104 (5.6%) 0.4 mixed N=1247 off path N=19727 CCDF 0.3 on path N=8956 0.2 0.1 0 1 10 100 1K Number of source destination pairs probed with UDP G|BBBB 5.6% of interfaces that we inferred as on-path were traversed in at least 8 source-destination peers with UDP G|BBBB packets. We are at least 99% confident the previous hop did not load balance UDP G|BBBB packets on a path avoiding B.
19 www.caida.or MIXED CLASSIFICATIONS 10K 1 2 3 4 10 path) >10 1K
100
10 Zero timestamps (off 139 1 437 102 1 10 100 1 3 timestamps (on path) Most interfaces with mixed behavior appeared as on-path for just one source-destination pair
20 www.caida.or