On Static Reachability Analysis of IP Networks

¡ ¡ ¡

Geoffrey G. Xie Jibin Zhan David A. Maltz Hui Zhang

¢ ¢ ¢ Albert Greenberg Gisli Hjalmtysson Jennifer Rexford

ABSTRACT Determining what kinds of packets can be exchanged The primary purpose of a network is to provide reach- between two hosts connected to a network is a difficult ability between applications running on end hosts. In and critical problem facing network designers and opera- this paper, we describe how to compute the reachability tors. To our knowledge, the problem is largely unexam- a network provides from a snapshot of the configuration ined in the networking research literature. Solving the state from each of the routers. Our primary contribution problem requires knowing far more than the network’s is the precise definition of the potential reachability of a topology or the routing protocols it uses. For example, network and a substantial simplification of the problem despite having a route to a remote end-point, a sender’s through a unified modeling of packet filters and routing packets may be discarded by a packet filter on one of the protocols. In the end, we reduce a complex, important links in the path. The network’s packet filters, routing practical problem to computing the transitive closure to policies, and packet transformations all must be taken into set union and intersection operations on reachability set account to even ask the simple and very important ques- representations. We then extend our algorithm to model tion of “can these two hosts communicate?” the influence of packet transformations (e.g., by NATs or This paper crystallizes the problem of calculating the ToS remapping) along the path. Our technique for static reachability provided by a network. By mapping packet analysis of network reachability is valuable for verifying filters, routing information, and packet transformations to the intent of the network designer, troubleshooting reacha- a single unified model of reachability we have determined bility problems, and performing “what-if” analysis of fail- how to transform this seemingly intractable problem into ure scenarios. a classical graph problem that can be solved with polyno- Index Terms—Routing, Static Configuration Analysis. mial time algorithms such as transitive closure. This is the primary contribution of this paper. I. INTRODUCTION A. Advantages of Automated Static Analysis While the ultimate goal of networking is to enable communication between hosts that are not directly connected, Currently, the common practice to determine if pack- a wide variety of mechanisms are being used to limit the ets can reach from one point in a network to another is to set of destinations the hosts can reach. For example, back- use tools such as ping and traceroute to send probe bone networks may provide Virtual Private Network ser- traffic that experimentally test whether reachability exists. vices to connect only remote offices belonging to the same In contrast, we have developed a static-analysis approach enterprise, and enterprise networks themselves are often that can be applied even if only a description of the net- segmented into departments or offices whose hosts must work is available. Static analysis has many advantages be isolated for business or security reasons. Also, due to over ping and traceroute, including: a configuration or design mistake, two hosts may not be § The ability to determine a description of the set of able to communicate under certain failure scenarios, even packets that could traverse the network from a given though the network remains connected; knowing when starting point to a given ending point, whereas exper- these vulnerabilities exist is crucial to building a more re- imental techniques can only check the reachability of liable network. the specific probe traffic they send. § The ability to calculate the set of routers and hosts Research sponsored by the NSF under ANI-0085920, ANI-0331653, and ANI-0114014. Views and conclusions contained in this document that a given packet could potentially reach, whereas are those of the authors. ping and traceroute can only check reachabil- * Naval Postgraduate School. [email protected]. This work was done while ity along the path currently selected by the routing G. G. Xie was a visiting scientist at Carnegie Mellon University.

£ protocols. ¥

Carnegie Mellon University. ¤ jibin,dmaltz,hzhang @cs.cmu.edu ¦

§ The ability to evaluate the reachability of a net- ¥ AT&T Labs–Research. ¤ albert,gisli,jrex @research.att.com. G. Hjalmtysson is also with Reykjav´ik University. work during its design phase—before the network 2

has been deployed or a problem has arisen. Network mechanisms is critical to accurately computing the reach- operators can perform our static analysis using only ability of a network. the configuration files used to program the network’s Third, we present an algorithm for the static analysis of routers, and these files are readily available to them. reachability for IP networks and explain how the network

§ The ability to verify whether the reachability a net- model can be populated by static analysis of the network’s work actually provides matches the designer’s in- router configuration files. tent. Static analysis can verify that Virtual Private Networks are, in fact, isolated from other traffic. It can also be used to conduct “what-if” analysis— C. Structure of the Paper predicting the effects of equipment failures and In Section II, we present a brief overview of the most planned maintenance on the communication between relevant aspects of how routers operate and are config- end hosts. While syntax verification of router configured. We then formally describe our framework for ana- uration has been evaluated [3], [8], there is little un- lyzing a network’s reachability in Section III, beginning derstanding of the power and limitation of semantic our analysis by focusing on packet filters. We present verification based on static analysis. our algorithm for calculating reachability in Section IV. Manually calculating the static reachability of a net- In Section V we show how to map routing information work is often impractical, as data show that campus, en- to packet filters and how this model of routing is popu- terprise, and backbone networks vary in size from 5 to 500 lated by analyzing the router configuration files. In Sec- routers, with the largest networks having on the order of tion VI we describe how packet transforming mechanisms 1,000 routers, and that real networks use a wide variety are handled. Section VII discusses the applications and of mechanisms to control the reachability they provide. limitations of our approach. After a brief overview of re- A survey of 31 production networks [10], including ex- lated work in Section VIII, the paper concludes in Sec- amples of both carrier backbone and enterprise networks, tion IX with a summary of our contributions. found that 10 out of the 27 enterprise networks had packet filters applied to their internal links. Several of the networks deliberately prevented some hosts from reaching II. BACKGROUND ON REACHABILITY others by preventing the distribution of routing informa- CONFIGURATION tion needed to direct packets between the hosts. Further In addition to forming the physical topology of routers complicating the question of a network’s reachability is and links, network operators must configure the protocols the use of mechanisms that actually transform packets as and mechanisms that collectively determine which hosts they travel across the network. For example, Network can communicate. Today’s routers offer a wealth of con- Address Translators (NATs) [15] that change a packet’s figuration options for enabling and tuning packet filters, source and destination address were found in the inte- routing protocols, and packet transformations. Our analy- rior of 10 of the 31 networks. Understanding the reach- sis techniques operate on a snapshot of the configuration ability “matrix” created by a network requires a frame- state for each of the routers in the network, as recorded in work for reasoning about the effects of all these different a configuration file. In well-managed networks, these files mechanisms—packet filters, routing policy, and packet are routinely captured and archived for backup purposes, transformations—at the same time. and are available to network operators. To make our discussion of the different reachability configuration options more concrete, we focus on the ex- B. Our Contributions ample enterprise network in Figure 1. The network has First, we formulate the problem of computing the reach- five routers R1 to R5 (depicted as solid rectangles) con- ability of a network and argue for the importance of craft- nected via physical links (depicted as solid lines) that ter- ing good solutions. We focus on the value of computing minate at interfaces (depicted as small circles). R1 and R3 reachability through static analysis. We rigorously define are remote sales offices connected directly to the central the reachability of a network, and we define expressions office where R2, R4, and R5 reside. R6 represents the ex- for upper and lower bounds on the reachability. ternal router in the service provider’s network where the Second, we describe a tractable framework for jointly enterprise connects to the Internet. reasoning about how packet filters, routing, and packet Each sales office has two subnets, A and B. Critical ac- transformations affect the reachability that a network pro- counting applications are run by hosts connected to subnet vides. Bringing together these three very different types of A, and general purpose computers are connected to subnet 3

B1 A5 net; instantiating this packet filter on the link from R1 to ACL 1 = interface A1 R1 R5 = subnet R5 is meant to prevent other subnets from accessing the ACL 1 = primary link R6 corporate servers on subnet A5. ACL2 drops all Sun ND = backup link ACL 4 R6 = external router protocol packets (protocol 77), which were implicated in A3 link to Internet R3 R2 R4 = packet filter an earlier attack on Cisco routers. Like ACL1, ACL3 per- ACL 2 ACL 3 mits TCP packets to the Microsoft SQL server (port 1433) B3 from hosts in A3, but denies them from any other subnet. Fig. 1. An example enterprise network with five routers ACL 3 also prevents multicast packets (in the IP address range 224.0.0.0/8) from leaving the office containing R3. ACL Definition ACL4 drops all Mobile IP packets (protocol 55), which ACL 1 permit tcp A1 A5 port eq 1433 deny tcp any any port eq 1433 were also implicated in an earlier attack on Cisco routers. ACL 2 deny 77 any any ACL 3 permit tcp A3 A5 port eq 1433 B. Routing Protocols deny tcp any any port eq 1433 deny ip any 224.0.0.0/8 Routing protocols influence reachability by controlling ACL 4 deny 55 any any the construction of the forwarding table on each router. Conceptually, a route is a network address (e.g., an IP ad- TABLE I dress and a mask length, such as 10.0.0.0/8) along with FOUR PACKET FILTERS INSTANTIATED IN FIGURE 1 additional attributes (e.g., numerical weights, AS paths, or next-hop IP address) that a router can use to determine B. Hosts on subnets A1 and A3 must be able to commu- which outgoing link to use to reach that subnet. A router nicate with corporate servers in subnet A5, but the net- can learn a route in several ways. First, a router knows work’s policy is to prevent any other hosts from commu- locally how to reach all directly-connected subnets—the nicating with the servers on A5 to reduce the chances of a incident links themselves. Second, the router may be con- server compromise. To make the network more resilient to figured with static routes that map a destination subnet link failures, the operators are planning to add two backup directly to one or more outgoing interfaces. Third, the links (shown with dashed lines). In Section 4 and beyond, router may learn the information dynamically through a we show that our reachability analysis technique can pre- routing protocol, such as OSPF [11], IS-IS [4], BGP [13], dict the effect of adding these links and prevent a design RIP [9], or EIGRP [17]. error that would violate the network’s goals. To control the sharing of routing information, each instance of a routing protocol runs as a separate routing process on the router. Just as with operating system process A. Packet Filters boundaries, by default no information is exchanged be- The simplest way to control reachability is to configure tween these entities, and they operate completely inde- an interface to filter unwanted packets in the data plane. pendently. Each routing process has a Routing Informa- Today’s routers allow operators to filter packets based on a tion Base (RIB) that stores the routes on which it operates, combination of fields in the packet header, such as source similar to the virtual memory space of a process. To sim- and destination IP addresses, type-of-service (ToS) bits, plify the discussion, we consider the directly-connected port numbers, and protocol. Each packet filter consists subnets and static routes as belonging to a single process of a sequence of clauses that that permit or deny certain that creates a local RIB. A router can run multiple routing packets based on their header fields. A filter can be in- processes simultaneously, including multiple instances of stantiated on a particular interface to filter incoming or the same routing protocol. For example, Figure 2 illus- outgoing packets. An interface may have different filters trates the routing processes (as represented by their RIBs) for incoming and outgoing packets, and different inter- for the network in Figure 1, after the two backup links faces may be assigned different filters. have been added. Router 2 runs two instances of OSPF Table I shows four access-control list (ACL) specifi- and one instance of BGP, and has a local RIB. cations, defined in the Cisco IOS language; packets not Routing processes do not exchange information unless matching any clause are permitted by default. Figure 1 specifically configured to do so. The dashed lines in Fig- shows where these ACLs are used to filter outgoing pack- ure 2 indicate adjacencies between routing processes on ets on four interfaces. ACL1 permits TCP packets des- different routers, or route redistribution between RIBs on tined to Microsoft SQL servers (port 1433) in subnet A5 the same router. For example, router 2 exchanges routing from hosts in A1, but denies them from any other sub- information via OSPF with router 1 and router 3; routes 4

i52 £¡£¡£

from the OSPF RIB are redistributed to BGP and adver- ¤¡¤¡¤

£¡£¡£ ¤¡¤¡¤

i11 ¦¡¦¡¦

£¡£¡£ ¤¡¤¡¤ tised via external BGP (EBGP) to router 6. The other in- ¦¡¦¡¦

BGP ¥¡¥¡¥OSPF Local

£¡£¡£ ¤¡¤¡¤

RIB ¦¡¦¡¦ i12 ¥¡¥¡¥RIB RIB i53 stance of OSPF on router 2 does the same for routers 4 Local OSPF i51 ¥¡¥¡¥ i14 RIB RIB Router 6 and 5. The routes from router 2’s local RIB are also redis- Router RIB tributed to the BGP RIB, and onward to router 6. Thus, Router RIB i13 Router 5 EBGP router 2 takes responsibility for ensuring that the subnets Router 1 i22 i23

RP1 RP2 i42

©¡©¡©

¡ ¡ §¡§¡§

in the enterprise network are reachable from the rest of the ¨¡¨¡¨ ¡ ¡

i31 ¢¡¢¡¢

©¡©¡©

¡ ¡

§¡§¡§

¨¡¨¡¨ ¡ ¡

OSPF ¢¡¢¡¢BGP Local OSPF i41 OSPF Local

©¡©¡©

¡ ¡

§¡§¡§ ¨¡¨¡¨

Internet via router 6. ¡ ¡ RIB ¢¡¢¡¢RIB RIB RIB RIB RIB OSPF i21 Local i43 Rather than exchanging routes to every subnet, the dis- RIB RIB i32 i24 i33 tribution of routes is governed by routing policies. A pol- Router RIB Router RIB Router RIB Router 4 Router 2 icy can be thought of as an annotation on the dashed line Router 3 denoting the exchange (e.g., routing policies 1 (RP1) and 2 (RP2) in Figure 2). For example, router 2 could be con- Fig. 2. Interactions of routing processes in the example network in Figure 1. Each routing process is depicted by the RIB that stores its figured to filter the route to subnet A5 (i.e., the sensitive routes. Dashed lines indicate the import, export, and redistribution corporate servers) when distributing routes via eBGP to of routes. Interfaces are marked with identifiers and the solid lines router 6. Modern routers have rich languages for specify- between interfaces are the physical links. ing routing policies, including the ability to select which routes should be imported or exported based on any of the stateful Network Address Translator (NAT) and firewall attributes associated with the route (e.g., the subnet or the devices may transform or rate-limit packets in complex AS path). Routing policies can also alter the attributes of ways, the functionality supported (and enabled) directly in the routes they accept (e.g., changing metrics or adding an the routers is often much simpler. In our analysis, we fo- AS number onto an AS path). cus on this simpler form of statically-configured transfor- Upon receiving multiple routes for the same subnet, the mations and how they influence the reachability between routing process must select a single best route. The se- end hosts. lection of the best route depends on the route attributes and logic defined for the particular protocol. For example, BGP has a complex multi-stage process for identi- III. PROBLEM FORMULATION fying the best route [16], whereas OSPF selects the path In this section, we formulate the reachability analysis with the smallest cost as the sum of the link weights [11]. problem. We first describe a graph model for computing If multiple RIBs on the same router have a best route for reachability which allows joint reasoning of the effects of the same subnet, the router must determine which routing packet filters and routing protocols. We then formally de- process should control the entry in the forwarding table. fine the reachability metrics targeted by our analysis. In For example, the router may impose a static ranking on particular, we introduce the concept of the instantaneous the routing processes (e.g., giving the local RIB priority reachability provided by a network, and explain why it is over BGP-learned routes). useful to develop bounds on the reachability provided by the network. We end this section with an example illus- C. Packet Transformations trating the potential value of being able to compute the The routers make packet filtering and forwarding deci- reachability bounds. sions based on fields in the header of each packet. How- ever, these header fields may change as a packet flows A. A Unifying Model through the network. For example, the network operator may configure router R2 in Figure 1 to reset the ToS bits The crux of determining the reachability of a network is of incoming packets from R6. If the enterprise network finding a way to unify two very different views of the net- assigns packets to different queues based on the ToS bits, work. The first is the graph of routers and links, illustrated setting the ToS bits to a default value would ensure that in Figure 1, where vertices are routers and edges are phys- traffic coming from the Internet does not enter the same ical links that may have packet filters applied on them. queue as high-priority internal traffic. Similarly, R2 could The second is the routing process graph, illustrated in Fig- be configured to map the source IP addresses of pack- ure 2, where vertices are routing processes and edges are ets leaving the network via R6, in order to use private adjacencies that implement routing policy. Unifying these IP addresses inside the enterprise and public addresses views requires combining the policies governing redistri- in communicating with the external Internet. Although bution of routes with the packet filters governing which

5 C

packets can traverse a link. This uniﬁed framework un- with a next hop of interface D , it means A might forward derlies our reachability analysis, and will be extended to packets to C out that interface. Therefore, we can treat this address packet transforms in Section VI. route as if it were a permit clause for C in the packet ﬁlter

We deﬁne the reachability analysis problem by extend- on interface D . Inversely, if router A holds no routes that C ing the graph of links and routers – annotating the edges could possibly send packets to destination out interface

of the graph more elaborately. Formally, we deﬁne the D , then we can add a clause to the packet ﬁlter on interface

¥ © graph ¢¡¤£¦¥¨§ © § where is the set of routers, is to drop all packets headed to destination . the set of directed edges deﬁning the connectivity between the routers, and is a labeling function that annotates the B. Formal Deﬁnitions of Reachability Metrics

edges in © . As the graph is directed, two routers directly We describe the reachability between two points in a connected by a physical link will have two edges between network in terms of the the subset of packets (from the

them, one in each direction. For each edge §© ,

universe of all IP packets) that the network will carry be-

! represents the policies governing the ﬂow of

1 tween those points. Thus, reachability from router D to packets from to .

router E is given by the subset of packets that the network

The challenge in building this graph model from the

E FG

will carry from D to and is denoted as . Note that H

static analysis of conﬁguration data is that cannot be F it is common for G to include packets that are neither a simple metric like an integer weight. It must embody the

sourced by a host connected to D nor destined to a host effects of the complex collection of packet ﬁlters and rout-

connected to E — this must be true if routers are to for-

ing protocols used by the network, but still be amenable

ward packets along multiple hops. to efﬁcient arithmetic-like manipulations. We deﬁne

Clearly, the action of the network’s routing protocols H

to be the set of packets that the network is able to carry

F D

will directly inﬂuence G , for if router has no routes

C C

from to . can also be represented by a packet ﬁlter

for destination , then packets to cannot be elements of

containing predicates that test properties of packet $ ,

F D G , since will be dropping those packets. More gen- returning true if the packet should be in the set . De- erally, the network is continually affected by events such

termining which packets can ﬂow from router to router

(' ) *

as link failures and changes in routing advertisements re- &

to router can then be written simply as or

# #

,+ - * ceived from peer networks. Through the action of routing . protocols and other mechanisms, each router will populate The advantage of representing the reachability problem its Forwarding Information Base (FIB) with information as a graph .¡/£¦¥¨§ © § is that it exposes the similarity determining the interface(s) out which each packet should between our problem and the class of well-known prob- be sent. We deﬁne the collective contents of the FIB on lems such as transitive closure and shortest-path compu- each router in the network to be the network’s forwarding

tation, allowing us to use their efficient solutions [1], [6] J state, denoted by I . We also define to represent the set when computing the reachability of a network. of all possible forwarding states that the network can pos- Packet filters defined by the network can be easily rep- sibly enter, as it responds to any imaginable set of external resented in the graph model . Network configuration 2

# advertisements, link failures, etc. files define a packet filter as a series of predicates over

# 1) Instantaneous reachability: The Reachability pro- packet elements. For example, may be “p.src addr

+ vided by the network will change as a function of the net- ¡21)3 4 128.2/16 p.dest port 0 ”, which accepts all packets

work’s forwarding state I , which may change from instant from the 128.2/16 subnet except for those going to port to instant as the network responds to events. Therefore, 135. By parsing the conﬁguration ﬁles we can extract the

# our ﬁrst step is to precisely deﬁne the reachability pro- predicate applied to the link from router to router , vided by a network at a single instant in time, assum-

and annotate the edge 5§6© with the set of pack-

# #

ing that the forwarding state in effect at that instant is £;$<=¡!1?> ets that accepts, i.e., ¡87$:9 . known. The intuition behind our framework for jointly mod- The inﬂuence of any given forwarding state I6KJ on

eling routing and packet ﬁltering is that routing can be the reachability in the network can be accounted for by

thought of as a kind of dynamically constructed packet ﬁl- incorporating additional packet ﬁlters into . In doing

ter. If routing process on router A holds a route for subnet so, the policy annotation at each edge in the reachability

@ L

We believe our framework can be trivially extended to handle mul- When conducting a particular analysis of a particular network, the

B M tiple physical links between A and , but for the remainder of this human conducting the analysis might want to restrict to the forward- paper we assume there is at most one physical link between each pair ing states reachable under a more restricted set of events, such as “no of routers. more than one link or router will fail at a time.”

£ I

analysis graph becomes a function of I , written as . tination, but Cisco has added several more decision steps

¡-

I § Assume £ to be a function that returns the set of in their implementation [16].

next hop routers to which router will forward packets The goal of most network designers is to ensure that

destined to IP subnet C while the network is in forward- the network’s behavior remains within some “acceptable

£ I

ing state I . can then be formally deﬁned as an operating region” under reasonable predictions of how

extension to the statically conﬁgured packet ﬁlters % . routers/links might fail or outside events might change.

This means that more useful than calculating the instanta-

C C

% '

I?=¡ 7$ 9 $ 7 9? £ I%§ > > £ .dst addr neous reachability of a network is the ability to calculate (1) bounds on the reachability provided by the network. That

is, given some set of reasonable events, predict the “op-

£ D § E D E Let ¢ be the set of all loop-free paths from to in erating region” of the network. We do this by deﬁning the network’s physical topology. Using all these concepts, two key bounds: the upper bound on reachability, which

we can now precisely deﬁne the instantaneous reachabil- is the largest set of packets the network will ever deliver E ity from D to provided by the network while at routing between two points, and the lower bound on reachability,

state I as: which is the largest set of packets the network will always

deliver between two points.

G £ I =¡ £ I?

F (2)

Reachability upper bound: Formally, we deﬁne the

¥¨¤ ¤¦¥¨§ © upper bound of the reachability over all routing states as follows: 2) Bounding the Instantaneous Reachability: In the-

ory, it should be possible to compute exactly what for- £

F ¡ F £ I? G

warding state, and thus what reachability, a network pro- G (3)

vides at any instant in time. After all, each router in ¥¨

the network is a computing device with its behavior pro- F Conceptually, G captures the notion that as external

grammed and controlled by conﬁguration commands. Un- events change, the path the network chooses for a packet E fortunately, computing the instantaneous reachability of moving from D to will change as a function of the route a network requires knowing the current topology (e.g., selection logic and the external routing advertisements.

which links and routers are up or down) and the exact in- Therefore, taking the union of the set of packets that can

E I

formation given to the network by neighboring domains traverse each path from D to under each state produces

in the outside world (e.g., the routing updates from BGP F

a superset of the instantaneous reachability — that is, G E peers). Dynamic information of this kind might not be is the set of packets that could potentially reach from D to

available (e.g., the network is not deployed yet), and its if the routing decisions were made appropriately. The set

use makes the instantaneous reachability results depend F

negation of G is particularly useful, as a packet appear- H

heavily on the exact inputs used. For example, if the ex-

F D

ing in this complement of G cannot ever reach from act set of routes offered by external peers to the network to E . Essentially, the packet is blocked along every pos- under analysis is known, then the reachability to those sible path. This allows us to verify whether the network destinations at that instant could be calculated. However, enforces security policies intended to isolate trafﬁc. the calculated reachability is applicable only in situations Reachability lower bound: Formally, we deﬁne the where the external peers offer exactly those routes, which lower bound for reachability as follows:

severely limits the usefulness of the reachability analysis.

F £ I? F ¡ G

Further, computing the instantaneous reachability of a G (4)

network requires knowing not only the conﬁguration state ¥¨

of each router, it requires the tedious and error-prone cod- Conceptually, G captures the notion that a packet per- E

ing of an exact bug-for-bug emulation of the decision mitted to reach between D and under all possible for-

J D logic used by the particular version of the software run- warding states I will always be able to get from to

ning on each router. (More than 200 different software E . For the lower bound to give useful information about

versions were used by the routers in the 31 production the network’s routing design, we ﬁrst need to restrict J to networks we recently examined in our study of IP routing those routing states induced from a set of network events

design [10].) While the routing protocols are deﬁned by targeted by the analysis. In particular, J should not in-

standards, each vendor has implemented them differently. clude any forwarding states corresponding to failure sce-

D E F

For example, the Border Gateway Protocol (BGP) [13] narios that would physically disconnect and (or G

deﬁnes a seven-step process for selecting a route to a des- will be trivially ).

H§¦

is useful to network designers because the net- It is straightforward to prove that this estimator G

H H H

work’s routing design guarantees that packets appearing

F F F

G G

G [19]. It can be shown that if we can as- E

in this set will be deliverable between D and as long as E sume that when only one path exists from D to the routing the network is not physically partitioned. Designers can design of network is such that the path will be used.3 then verify that trafﬁc requiring robustness appears in this In Section V, we describe an approach to approximat- set. ing the effect of the routing protocols on reachability that 3) Approximating the Reachability Bounds: As dis- yields tighter estimators. Our expectation is that further cussed earlier, it is difﬁcult and error-prone to precisely research will lead to better and better estimators. model the route selection logic and external routing advertisements under all events. Further, the size of J is C. Example Application of Reachability Analysis enormous, even for small networks. Combined together, To illustrate the value of the reachability bounds, let us

these two factors make it seem impossible to accurately

revisit the example network deﬁned in Section II where

£ I I

compute J or for every . Therefore, we cannot H

network operators were considering adding two backup

F use equation (3) to exactly compute G or equation (4)

links. At a ﬁrst glance, it may seem to be sufﬁcient to

to compute G . Instead, we must develop estimators for reconﬁgure routing parameters on the routers to use the

H H

F F

G and . backup links under failure scenarios. However, checking

H H H H

F F F F

G G G We denote estimators to G and as and , the reachability bounds reveals that such a design is incor- respectively. Ideally, these estimators should be looser rect. Speciﬁcally, the table below compares two particular

bounds, that is: reachability bounds before and after the backup links are

7 1 § § >

added, where ¨¢©¤ represents the set of packets

H¢¡ H£¡ ¡ H¤¡ H

F F F £ I F F

G G G G

Before After

¤ ¥ ¤ ¥

H H

F F

as this property maximizes the utility of G and in

¤ ¥ ¤ ¥'& ¤ ¥

verifying network properties. For example, a G that is H

permitted by ACL 1, 2 and 4, and so on. (The algorithms

F looser than G may incorrectly warn an operator that the

for computing these bounds are given in Section IV.) On

H¢¥ H

F F

G packets in G could violate the network’s trafﬁc one hand, the lower bound from router R1 to router R5

isolation policies, but in this situation a false-positive is 3 is further constrained by the addition of ¨£©£ . Recall

much better than a false-negative. that for TCP packets with port number 1433 (SQL traf-

H H

F F

Even simple estimators to G and still have value 1 ﬁc), ¨¢©¤ permits only those from hosts in A1 to hosts

in predicting network properties. For example, we can ob- 3 in A5 and ¨£©£ permits only those from A3 to A5. To-

tain simple estimators by ignoring the effect of the rout-

1 ¨£©£ 3

gether, ¨¢©¤ and will deny all TCP packets with

ing protocols entirely, so that models only the static port number 1433 from R1 to R5 as A1 and A3 use dis- packet filters defined on edge K§ . The upper bound tinct address ranges. This defeats the purpose of adding can then be calculated by finding the set of packets that at the new backup links as they will be totally ineffective for

least one path through the network will allow to pass from SQL trafﬁc from R1 to R5 under failure senarios. On the E D to , since there could be some routing state that chooses other hand, the upper bound from R3 to R5 is expanded,

this path for the packets. allowing a portion of multicast trafﬁc to spill out of R3 ,3

against the security policy established by ¨¢©¤ . Since

£ the backup links are not used under normal conditions,

F ¡

G (5)

ping and traceroute tools would not be of much help

¥¨¤ ¤¦¥¨§ © in detecting these problems without destructive tests (e.g., by shutting down a primary path).

Similarly, the lower bound can be calculated by ﬁnding E the set of packets that all paths from D to allow to pass,

IV. COMPUTING THE REACHABILITY BOUNDS E since, so long as D and are not partitioned, at least one of these paths will exist and could be chosen by the routing In this section, we present basic algorithms for com-

protocols. puting the simple reachability bound estimators deﬁned $

It is completely conceivable that a network could have a routing

design such that not all paths can be used, meaning that and can

¡ F

G (6)

be effectively partitioned even when there are still physical paths that ¥¨¤ ¤¦¥¨§ © connect them. 8 by equations (5) and (6). The same algorithms can also 3 {3,4,6,7} be used to calculate other (potentially tighter) reachability {3−5} F 3,2 {5−8} {5,8} {3−5} F 5,3 bound estimators as long as the approximation is based 1 2 {6−8} 5 F 2,1 on adding additional static restrictions to pre-conﬁgured {6,7} F 5,4 4 packet ﬁlters. Section V describes such an approxima- 4 F 4,2 tion method. We assume that there are no packet transformers in the Fig. 3. Example network for illustrating execution of algorithm.

network. We will relax this condition in Section VI.

Lower bound calculation. To compute G , we ﬁrst TABLE II

§ ©

prune all the edges that cannot be in any EXAMPLE EXECUTION OF BASIC ALGORITHM E

path from D to . This is accomplished by applying the

')(+* ')( ')( (.-

“Articulation Points and Biconnected Components” algo-

D E £ ©¢¡.¥

0 51 52 ¥1 ¨2 6 ¨3 54 ¥/

rithm for any pair of and , which is [1]. ©

After that, G is simply the intersection of for t he

¨ ¥/ 1 ¨2 51 52 ¥1 ¨2 6 ¨3 54 remaining edges. ©

Upper bound calculation. While the calculation of

G is not as straightforward, we observe that it closely

relates to the classical transitive closure algorithm. Con- plex structure than the one deﬁned in Section II.) For

% sider as describing the set of packets that ac- links with packet filters defined, the figure shows the set

cepts, with empty set and the set of all possible pack- of packets the ﬁlters will pass. For simplicity, packets are

¥87

7 4 >

ets denoted as £ . Our labeling function, , is a map represented by integers: refers to packets 5,6,7,

£ ¤ £¥£

from © to the power set of , , which is closed un- and 8. An uninstantiated set indicates a ﬁlter that

' E

der the operators ¦ and . It follows that properties and passes all packets. The destination router is set to 5.

3 algorithms in classical literature apply; in particular, so- The last column, when ¡ , gives the ﬁnal result of the lutions to compute transitive closure [1] and classical all- reachability from routers 1–4 to router 5. pairs shortest paths algorithms [6] Algorithm Complexity. The complexity of the illus-

Below is a dynamic programming formulation (as in 5

£¦¥ 9)

trative upper bound algorithm above is . However,

[1]) for calculating G , with the recurrence relation the reachability between all pairs of routers can be com-

F £ D § E ¨§8¡ © F £ D § F £ § E ¨§ F £ D § E §

£¦¥

, where puted also in via the same techniques used in the E represents the set of packets that can go from D to in up to Floyd-Warshall method [6] for computing all-pairs short- hops. The calculation starts from the destination router est paths.

E and extends the path by one hop with each iteration of Our reachability analysis framework is targeted at com- the outermost loop, and eventually taking all paths from D puting the reachability for a network operated and con- to E into consideration. trolled by a single organization, rather than the Internet

//Computing reachability upper bound matrix column as a whole. As discussed earlier, the sizes of such net-

D § E G D

1. Initialize F £ to for all ; works typically range from 5 to 1,000 routers. With

£¦¥

bounded like that, the time complexity is very rea-

2. for ( ¡!1 to ) do sonable. It should also be noted that the algorithm will

1 ¥

3. for ( D(¡ to ) do be run mainly as part of a design time tool installed on an

£ D § E ¡

4. F ; off-line system. In that case, timely execution is not a pri-

¡!1 ¥

5. for ( to ) do mary concern. For on-line troubleshooting, the size of

6©

6. if ( D § ) can be reduced by reﬁning the reachability analysis graph

F £ D § E ,¡5F £ D § E 7 F £ § E > ©

then G ; model to incorporate the routing realm abstraction so that

F £ D § E ,¡5F £ D § E 7. ; a node in the graph may represent a collection of routing

processes with the same external reachability [10].

F £ D § E

Table II shows the intermediate results of § ,

when running the dynamic program on the example net- It should be noted that we have made a simplifying assumption that

!;" work shown in Figure 3. (This network has a more com- step 6 has complexity : . In real networks, is often a nontrivial

predicate representation of a set of packets. Performing set operations

Also, the upper bound algorithm can compute the instantaneous over such representations may incur higher cost than : . We are

©&%

$ reachability if !#" is known for every edge. currently investigating this issue.

£ C

D £

V. CONVERTING ROUTING INFORMATION INTO conceptually a function ¢ that maps destination ad- C

PACKET FILTERS dress to a list of “routes.” A route ¢¥¤ is a tuple with the

In this section, we explain how the effects of routing on following ﬁelds deﬁned: ¦

reachability can be incorporated into our uniﬁed frame- ¢¥¤ .d = set of destination subnets this route applies to ¦

work by adding additional terms to the static packet ﬁlters ¢¥¤ .interfaces = the set of interfaces on the router that

defined in router configuration files. (We will use packets matching ¢§¤ .d might be routed out ¦ to denote the intersection of all packet filters configured ¢¥¤ .next hop ip = the set of routers (identified by their IP

address) whom packets matching ¢§¤ .d might be routed to- over edge K§ .) These terms restrict the set of pack-

wards

ets that can travel from to to those packets that the

7 >

¢¥¤ .type = interface,static original source of this route §2

network might route over the link . The fol- ¦ lowing subsections deﬁne the key elements in our model attributes.... = a list of key-value pairs

and then describe a four step algorithm for computing the The Router RIB (i.e., the routing table) of each router

additional terms that must be added to . The algo- maps the complete IP address space onto the set of in- rithm starts with the routes that are explicitly specified in terfaces according to a longest prefix match. If there is the configuration of the network. It then computes the no default route, all packets not matching a more specific

route are dropped. We formalize the action of the Router

C ¡

maximal set of routes that could possibly end up in each

£ I § RIB on router as , which returns the set of in- router, subject to the network’s routing policies. Finally, C terfaces that packets to destination should be sent out it uses these maximal sets of routes to compute the addi- 6 tional terms. when in forwarding state I . In this paper, we only con- We have formally established that our algorithm com- sider converged forwarding states — analysis of transient putes a tighter estimator for the reachability upper bound states is beyond the scope of this work. Previous works than the simple one deﬁned by equation (5). The details have shown how network conﬁgurations can be statically are omitted here for brevity. Instead, we refer interested checked to verify the forwarding state will converge. readers to [19], which also discusses a limitation of our al- For computing the upper and lower bounds on reach-

ability, which predict the network’s reachability over all

C ¡

gorithm in producing a tight estimator for the reachability

J £ I §

I , we do not need to compute but rather

C ¡

lower bound.

£ the function that speciﬁes all the interfaces router

might potentially use to forward packets to destination C .

C C

A. Deﬁnitions for Modeling Routes and RIBs ¡

£ ¡ © £ I §

That is, ¥ . Step 2 below shows how

J C

A destination subnet is traditionally deﬁned as an ad- ¡

£ we calculate by ﬂooding routes through the network dress and netmask (Section II). However, we need the and identifying on each router the interfaces that will be ability to reason about how routers will handle a set of C candidates for carrying trafﬁc to .

destinations. In particular, we will need a means to de- ¡ ¨

While functions like , , and others are router speciﬁc, C

scribe the set of all possible destinations. The conceptual ¡

for brevity we will omit the subscript (e.g., using

representation of this set as a list of all possible IPv4

£ instead of ) when it is clear from the context which destinations is unwieldy to work with in practice, so we router these functions are associated with. must ﬁnd a more concise notation. We adopt the representation deﬁned by Cisco, where

a set of destination subnets is represented by a list of B. Step 1: Initializing the RIBs

> 7 > 7 address/netmask-range . For example, 128.2/16-24 Initially all RIBs are cleared of all routes. Then the Lo- represents the set of all destinations whose ﬁrst bits are cal RIB on each router is populated with all the routes that

128.2 and whose netmasks are from 16 to 24 bits long; are explicitly created on the router by its conﬁguration.

¢ D ¢ >

7 0/0-32 represents the set of all possible IPv4 destina- For each router , each interface on will be assigned a

C >

tion subnets; and 7 0/1-32,128/1-32 represents the set of subnet by the conﬁguration ﬁle: this is represented by

C C

£ ¡ § 7)D>%§ 7©¢ >%§ > all possible destinations with the default route 7 0/0 re- setting LocalRIB type=interface .

moved. Our algorithms require that union and subtraction Routes manually conﬁgured to direct packets to des- C be well deﬁned on these sets of destinations, and this is tination out interface D are represented in the same

easily proven. Where an algorithm in this paper calls for a way. Static routes, which are manually conﬁgured routes C destination C , we can use either a single destination subnet that direct packets to destination out whichever in-

or a set of destination subnets interchangeably. terface is used to reach address , are represented as

©&% As described in Section II, each router contains one

" usually maps to a single interface, but may contain sev-

£ D RIB for each routing process that it runs. A RIB ¢ is eral interfaces if Equal Cost Multiple Path (ECMP) is in use. 10

i52

is the set of RIBs in the network and © RIB describes

¥¡¥¡¥

¦¡¦¡¦ ¡ ¡

i11 ¢¡¢¡¢ ¥¡¥¡¥

¦¡¦¡¦ d5 ¡ ¡

d1 BGP ¢¡¢¡¢OSPF Local the adjacencies between RIBs over which routes are im-

¥¡¥¡¥

¦¡¦¡¦ ¡ ¡ RIB ¢¡¢¡¢RIB RIB d5 d1 i12 i53 Local OSPF i51 d3 ported, exported, and redistributed. ¢ is the set of routing RIB RIB Router 6 i14 d1 d3

i13 Router RIB policies that govern how routes move between RIBs, i.e., d3 d1

Router 5

§ 2 © ¤ ¢ Router RIB d1 +d1,−d3 for , the policy determines

E−BGP RIB d3 d3

Router 1 d1 i22 i23

which routes can move from RIB to RIB . Unlike

d3 £¡£¡£

+d3,+d1 +d3,−d1 d3 i42 ¤¡¤¡¤

i31 d3,d1 ¡ ¡

£¡£¡£

¤¡¤¡¤ §¡§¡§ d1 ¨¡¨¡¨ packet ﬁlters in , routing policies in can transform OSPF ©¡©¡©

Local d1 d1 ¡ ¡

£¡£¡£

¤¡¤¡¤ §¡§¡§ ¨¡¨¡¨ OSPF Local ©¡©¡© OSPF

RIB RIB OSPF BGP i24 i41 Local

¡ ¡

£¡£¡£

¤¡¤¡¤ §¡§¡§ ¨¡¨¡¨ RIB the routes they are applied to by changing the route’s at-

RIB ©¡©¡© RIB i33 d3 d3 d3 d1 RIB RIBd3 d3 RIB i32 i21 d3 d3 d3 i43 tributes. Router RIB Router RIB Router RIB Router 3 Router 4 Note that graph may be partitioned and that adja- Router 2 RIB cencies among RIBs need not follow the physical links of

Fig. 4. RIB level view of the network illustrating the movement of the network. That is, the edge set of the RIB graph © RIB routes between RIBs. Routes, such as “d1”, “d3”, route ﬁlters, such as can be different from the edge set © of the physical graph “+d1,-d3” (“+” as permit, “-” as deny), and ﬂow of routes according to

in Section III. For example, there is a physical link be- redistribution policies are shown. Following the route ﬂow diagram, we can identify the origins of routes and through which interfaces the tween routers 1 and 5 in Figure 4, but no RIB adjacency routes are imported or exported. traverses it, or, in the case of networks using internal BGP

(IBGP), a single edge in RIB representing an IBGP adja- C

C cency may traverse multiple physical links.

¡ § 7 >%§ 7) >%§

LocalRIB £ type=static . The outgo-

D § ¢§¤

ing interface for a static route is determined in Step 3 us- We ﬁrst deﬁne a helper function push( ¢ ) that

¢ D ing a recursive lookup. takes route ¢§¤ found in and pushes it into all the ad- If we are computing the reachability upper bound, the jacent RIBs. Lines 2-5 prepare a candidate route for entry

RIBs of all routers external to the network are populated into the adjacent RIB. Line 6 applies the routing policy > with a single route with destination 7 0/0-32 — the set governing which routes can be pushed into the adjacent of all possible destinations. This is a conservative ap- RIB, potentially altering or dropping the route in the pro- proximation consistent with computing the upper bound cess. Lines 7-8 add the candidate route into the adjacent

on reachability, since whatever destinations the peer does RIB. >

advertise will be covered by 7 0/0-32 . ¢§¤

If we are computing the lower bound on reachability, push(RIB ,Route ) =

§ ©

the RIBs of all routers external to the network are left 1. Forall RIB

empty. This conservative approximation is consistent with 2. ¢ = router on which RIB resides

computing the lower bound on reachability, since in the 3. = router on which RIB resides

worst case the external routers will export no routes what- 4. ¢§¤ .interfaces = interfaces on where

§ >

soever to our routers, perhaps due to misconﬁguration, edge could arrive ¢

bugs, crashes, etc. 5. ¢§¤ .next hop ip =

¢§¤ ¡ ¤ £¢¥¤

If the routes the external peers are expected to export 6.

£¢§¤ =¡ £¢§¤ ¢¥¤

¨ £ ¨¡ ¨ £ ¢¥¤

be correspondingly tighter. £

D § ¢§¤ ¢?D Using push( ¢ ), we compute for each RIB on C. Step 2: Computing the Potential Set of Routes each router by iterative relaxation: applying push() to each route in each RIB until there are no changes in the In this step, we compute the set of routes that could contents of any RIB. potentially occupy each RIB by ﬂooding routes from the

local RIBs and external RIBs throughout the network. Much of the work in modeling routing lies with the pol-

£¢§¤ The ﬂooding process is governed by the routing policies icy ¤ in step 6. However, these expressions can be

between adjacent RIBs that determine which routes are directly extracted by parsing the description of the net-

passed, modified, or dropped. At the end of the step, we work (e.g., the router configuration files). ¤ must im- £

plement the export policy of RIB and the import pol- D

will have calculated for each RIB ¢ the maximal set of

£ £

icy of , but has tremendous ﬂexibility given its ability to

?D ¨£¢ D routes that ¢ could potentially hold and , the set

£ modify the routes it is applied to. Typical policies seen in D of destinations that ¢ covers.

As illustrated in Figure 4, the routing design of a net- real networks include:

and 128.2/16 and drops all other routes. Case 1 handles the base case of a simple route that for-

wards packets out a speciﬁc interface. Case 2 handles

¡ ¡

¥ 7£¥ 7 7 ¥

£¢§¤ ¡ ¢§¤ ¡ ¢§¤ 7 1 §)1 1£¢ 1£¢ >

¤ d d static routes, which require a recursive lookup to deter-

¡ 0 ¢§¤ if ¢§¤ d then return else mine which interfaces are used to reach the next-hop spec- iﬁed in the route. Case 3 handles BGP sessions, which are – A policy that governs the EBGP adjacency between carried in TCP session that can traverse multiple routers. AS1 and AS2, where all routes are passed, but AS1 must A recursive lookup for the address at the other end of the

prepend its AS number to the route’s AS path. session is required to determine which interfaces the TCP

session might arrive on, and thus what outgoing interfaces

£¢§¤=¡ ¢¥¤ £ ¨¥¤ 1 § ¢§¤ ¤ as path = concatenate as path might be used for routes learned from that session.7 Case 4 handles all other routing protocols, where the poten- – A policy that governs the EBGP adjacency between tial outgoing interfaces are those leading to the neighbor AS1 and AS2, where routes whose AS path matches a C routers from which the router imported route .

regular expression looking for AS3 are dropped.

£¢§¤ ¡ ¢§¤ ¡ ¢§¤

¤ as path as path /AS3/ E. Step 4: Computing Packet Filters that Represent the

¡ 0 ¢§¤

if ¢¥¤ as path then return else Effects of Routing

With in hand, we know the set of interfaces out

RIB which packets destined to might be sent. We ﬁrst com-

¡ ,

where is the length of the longest cycle in the

£ £ D

pute the inverse mapping of , , which returns the

9 ¢¥¤)9 graph and is the number of initial routes. The

RIB set of destination subnets that potentially map to interface

C C

, ¡ ,

factor results from the potential need to split a route

£ D ¡ 7 9 £ D D , i.e., . Using we map the

into multiple routes each time it is pushed, where is the routing table information to packet ﬁlters as follows: fraction of policies that require splitting routes. From our

Let ¢ denote the router under consideration. For all in-

experience so far, is small for real networks.

terfaces D on and for all routers that are directly con- D

nected to ¢ via interface , add the following clauses to

D. Step 3: Computing for each router .

Recall that is a router speciﬁc function that returns

7$ 9 £ D >

the set of interfaces out which the corresponding router ¡ p.dst addr (7)

C C

£ might forward a packet destined to . To calculate This ﬁlter will pass any packet going to a destination

for a router, we go through all the RIBs on that router

C that ¢ might possibly route out the link to , and drop all

looking for routes that cover , and then union together the

the packets that ¢ would never route via .

£ interfaces for those routes. is deﬁned recursively, and

the base cases are generally routes found in the Local RIB.

§ ¢?D

For readability, we introduce a helper function ifs( ¢§¤ ) VI. HANDLING PACKET TRANSFORMS

¢ D that computes the interfaces to which route ¢¥¤ in might In this section, we reﬁne the basic algorithm presented direct packets. in Section IV so that it will work with networks that in-

clude packet transforming ﬁlters.8

! £ C £ C

¡ We have found that this reﬁnement can be accom-

£¢§¤ § ¢ D § £ ¨¡ ¢§¤=¡ ¢ D £

where

G G plished without changing the fundamental structure of the basic algorithm. Speciﬁcally, we separate the packet

transforming parts from these ﬁlters and introduce virtual

& ¢§¤ interfaces if is a LocalRIB, and

# In IBGP it is possible to explicitly set a “third-party” next-hop, but

# #

# this is unusual. If we see the conﬁguration commands for this, we set

rt.type = interface;

# C

¡ to be all interfaces on the router for all potentially learned over

& £

if is a LocalRIB, and

$ rt.next hop ips this session.

! £

£¢§¤ § ¨¡

# rt.type = static; For networks containing packet transformers, the set of packets that

C ¡

# a destination can receive may be different than the set a source can send

£ &

if is a BGP-RIB;

# rt.next hop ips to that destination. For this paper, we calculate reachability as the set

# % # of packets the source can send, although our results can be extended to

also calculate a set describing what those packets might look like on

'¥¤)( * ¢ & D I+* ¢§¤ interfaces arrival at the destination. 12

3 {3,4,6,7} F t {3−5} F 3,2 u u,v v’ v’,v v {1,2}−>{5,6} {5−8} {3,4,5} F 5,3 1 1’ 2 {6−8} {5,8} 5 F 2,1 (Case 1) {6,7} F 5,4 4 F 4,2

Fig. 6. Packet transform example t F

u u,u’ u’ u,v v

F £ D § E ,¡5F £ D § E 7 F £ § E >

¡ Fig. 5. Explicitly modeling packet transformations using ¢¡ and a The intuition behind the new clause in 6’ is that if the

virtual node.

§ E

set of packets described by F £ can reach from to

E D ¤

, then only those packets arriving at that G transforms

§ E

components to represent them explicitly in the reachabil- into a packet in the set F £ will be able to reach from

E ¤

ity analysis graph . This is illustrated in Figure 5. There D to . To ﬁnd this set of packets that will transform into

F £ § E ¤ £ F £ § E are two cases: (1) the packet transformation is applied

, we calculate G .

after the packet ﬁltering (e.g., ToS remarking), and (2) the

£¦¥ 9 The complexity of the new algorithm is still . transformation ¤ is applied before the packet filtering (e.g., It should be noted that the algorithm requires two addi- NAT). In ether case, a virtual node-edge pair is introduced tional elements to be complete: (i) an efficient method to to model the separate processing stage. Each virtual edge compute the inverse function, and (ii) a condition to throw is labeled with a ¤ function representing a packet trans- out paths with loops because a looping path containing a form. packet transform edge may alter the outcome. Luckily, We have discovered that packet transforms may have the inverse function for commonly used transforms, such two undesirable properties that can complicate the reacha- as NAT and ToS remarking, are very simple — though in bility analysis. First, a transform might not be one-to-one. general the inverse of other transforms may be more com- For example, in the case of ToS remarking, multiple ToS plicated. For brevity, the details of (ii) are omitted. values may be mapped into one single ToS value. Also, Let’s revisit the example network in Figure 3 to illus- in the case of NAT, one external address pool is typically trate the steps of the refined algorithm. Suppose node 1

reused for many hosts as long as no two hosts use the same

1 § > © 7 4 § ¢ > now uses a leading packet transform: 7 , external address and port number at the same time. meaning that packets 1 and 2 each will be mapped into Second, a transform may not even be a deterministic either packet 5 or 6 before processed by node 1’s packet function. In some modes of NAT, a packet is not always ﬁlter. The new reachability analysis graph becomes Fig- transformed into the same packet; the source address the ure 6 and the execution steps of the reﬁned algorithm are

packet gets depends on the current availability of the ad-

F £ 1 § 4%

shown in Table III. At the last step ( ¡ ), is dress pool. To address these problems, we deﬁne a gen- changed due to the transform.

eralized inverse function of ¤ , over an arbitrary packet set

Our framework currently requires that packet trans-

¥¦¥

7$ 9¨§ ¤ £;$< > ¤ £ ¡

¤£ , as: © , which returns the forms be maps over sets of packets. They cannot test set of all possible packets that can be transformed using ¤ to a packet in . a property of a packet and behave one way if the prop- Using the inverse transform function, we have refined erty is true and another way if the property is false. In the basic algorithm to handle packet transforms. Specifi- particular, some networks include functionality called a cally, only steps 1 and 6 of the original algorithm need to “stateful firewall”. These are like a NAT, but only cre- be changed. ate the mapping when a packet traverses from the inside

of the ﬁrewall to the outside. Since our framework cur-

F £ D § E

1’.For all D , initialize as follows: rently has no notion of whether a packet has already been

KD § E

to G , if is ﬁlter sent through the stateful ﬁrewall from inside to outside, E

to set of all packets, if KD § is transformer we cannot directly model the reachability the stateful ﬁre-

ity assuming a packet has traversed the ﬁrewall, in which

6© KD §

6’. if ( D § and is transformer) case the ﬁrewall functions as a NAT described above, and

F £ D § E ,¡5F £ D § E ¤ £ F £ § E

then G ; again assuming no packet has traversed it, in which case

TABLE III ﬁlters used to achieve them. We also plan to explore the EXAMPLE EXECUTION OF REFINED ALGORITHM trade-offs between using routing policies and packet ﬁl-

ters in constraining reachability, and create guidelines for

')(+* ')( ')( ')( (

, 6

selecting one mechanism over the other. In particular, we

¥/ 0 ¥1 ¨2 51 52 51 52 6 ¨3 54

¥/ ¨ 51 ¥2 ¥1 ¨2 51 52 51 52 6 ¨3 54

Inﬂuence of dynamic routing information: Our up-

H H

F F

G per and lower bounds ( G and ) define an “envelope” VII. REACHABILITY ANALYSIS IN LARGER that constrains the influence of dynamic information, such CONTEXT as topology changes or routes learned from neighboring In this section, we describe how static reachability anal- domains, on network reachability. We plan to analyze ysis relates to our larger goal of understanding and im- existing networks in terms of the range between the up- proving the design of IP networks and routing policies. per and lower bounds. The gap between the upper and We also discuss the limitations of static analysis and how lower bounds may reflect the purpose of the network— to move beyond them. to provide broad reachability for many client domains to the entire Internet or to to provide narrow reachability for A. Understanding and Improving Routing Design client domains to specific network services. Alternatively, a wide range might imply the need for more protective Our work on static reachability analysis contributes to packet and route filtering, whereas a narrow range may our broader research agenda of improving routing design overly constrain the ability of the network to adapt to dy- and network robustness. Today, routing design is largely a namic changes. complex “art” mastered by an increasingly overwhelmed For each of these avenues for future work, our reacha- community of highly-skilled human operators. We aim to bility analysis offers a general and concise way to analyze

uncover the fundamental abstractions, such as the reach- H

H and compare routing designs at a level of abstraction well

F F

G ability bounds G and , that can be used to validate, above the low-level details of router configuration com- evaluate, and even generate the routing design for a net- mands and specific routing protocols. work. Our analysis framework opens several avenues for ongoing work: Verification of network design goals: A network has B. Moving Beyond Static Analysis some (explicit or implicit) design goals for providing Although static analysis provides significant insights, reachability between certain parties under certain condi- dynamic information determines where a network actu-

tions. For example, a network may need to ensure that ally operates in the space between the lower and upper £ two business competitors (customers ¨ and ) can never bounds on reachability. Our static analysis can be ex-

reach each other under any circumstances. This property tended by incorporating measurements of the dynamic

can be checked directly by ensuring that G is empty for state of the network and the routes learned from neigh-

¨ E all D in customer ’s network and in ’s network, and boring domains: vice versa. Alternatively, a network may need to ensure Dynamic network state: The configuration state de- that a customer can reach a data center; this can be as- fines the IP links and routing protocol adjacencies that sured by analyzing the lower bound on reachability. Re- could exist, without indicating whether they do exist at peating the reachability analysis on the subgraphs formed any given time. Various kinds of measurement data can after link and node deletions can test that network reacha- provide the missing information. The up/down status of bility persists under certain failure modes. links and sessions can be tracked via the Simple Network Design patterns and best common practices: The Management Protocol (SNMP) or vendor-specific “sys- same reachability goals can be satisfied by a wide variety log” data. In addition, a routing monitor [14] can contin- of different routing designs. Our concise representation uously track the topology (routers and links) and config- of reachability provides an appealing way to characterize urable parameters (e.g., OSPF link weights) within each and compare routing designs and identify common ways routing instance. of configuring a network to satisfy the goals. Using router Routing information from neighbors: Similarly, configuration data for several networks, we plan to iden- static analysis considers the route advertisements that tify common kinds of reachability goals and the combi- could come across links and sessions to neighboring do- nations of routing protocols, routing policies, and packet mains, rather than the ones that are available at any given 14 time. The set of routes announced by a neighboring do- ACKNOWLEDGMENT main could be gleaned through route monitoring or peri- We thank Nick Feamster and the anonymous reviewers odic dumps of the Routing Information Base (RIB) at the for their valuable comments and suggestions. edge routers. In addition to identifying which prefixes are REFERENCES advertised, the RIB data would identify the route attributes [1] A.V. Aho, J.E. Hopcroft, and J.D. Ullman. The Design and Anal- (such as AS path in BGP) that might affect how the receiv- ysis of Computer Algorithms. Addison-Wesley, 1974. ing router modifies or selects routes. In addition, the RIB [2] Randy Bush and Timothy G. Griffin. Integrity for virtual private would indicate whether the neighbor advertises subnets of routed networks. In Proc. IEEE INFOCOM, 2003. a given prefix that would have preference over the super- [3] Don Caldwell, Anna Gilbert, Joel Gottlieb, Albert Greenberg, Gisli Hjalmtysson, and Jennifer Rexford. The cutting EDGE of net in “longest prefix match” forwarding of IP packets. IP router configuration. In Proc. ACM SIGCOMM Workshop on Hot Topics in Networking, November 2003. VIII. RELATED WORK [4] R. Callon. RFC 1195 - Use of OSI IS-IS for routing in TCP/IP Many “ping” and “traceroute” tools have been de- and dual environments, 1990. veloped to help troubleshoot reachability problems in a [5] D.-F. Chang, R. Govindan, and J. Heidemann. The temproal and topological characteristics of BGP path changes. In Proc. Inter- live network. However, they are limited to the check- national Conference on Network Protocols, November 2003. ing the instantaneous reachability for the particular type [6] Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest. of probe packets they generate. There has been signifi- Introduction to Algorithms. MIT Press (McGraw-Hill), 1990. cant progress [18], [12], [5] in understanding the behav- [7] Nick Feamster and Hari Balakrishnan. Verifying the correctness of wide-area Internet routing. Technical Report MIT-LCS-TR- ior of operating networks by measuring the routing pro- 948, Massachusetts Institute of Technology, May 2004. tocols and establishing the root cause of changes. Our [8] Anja Feldmann and Jennifer Rexford. IP network configuration approach does not attempt to describe the detailed behav- for intradomain traffic engineering. IEEE Network Magazine, ior of the routing protocols, and it applies to packet filters pages 46–57, September/October 2001. and packet transformation as well as routing. [9] C. Hedrick. RFC 1058 - Routing Information Protocol, 1988. [10] D. Maltz, G. Xie, J. Zhan, H. Zhang, G. Hjalmtysson, and Bush and Griffin [2] formulate and derive sufficient A. Greenberg. Routing design in operational networks: A look conditions for the connectivity (reachability) constraints from the inside. In Proc. ACM SIGCOMM, August 2004. of Virtual Private Routed Networks (VPRNs). Our work [11] J. Moy. RFC 2178 - OSPF Version 2, 1997. is complementary, but broader in scope in that we frame [12] Packet Design, Inc. Route Explorer. http://www.packetdesign.com/products/products.htm. and tackle the general problem of reachability. [13] Y. Rekhter and T. Li. RFC 1771 - A Border Gateway Protocol 4 (BGP-4), 1995. IX. CONCLUSIONS [14] Aman Shaikh and Albert Greenberg. OSPF monitoring: Ar- This paper rigorously formulates the challenging prob- chitecture, design, and deployment experience. In Proc. lem of computing the reachability an IP network provides USENIX/ACM NSDI, March 2004. [15] P. Srisuresh and M. Holdrege. IP Network Address Translator and describes a framework that can be used to calculate it. (NAT) Terminology and Considerations. Internet Engineering The framework provides a unified way for jointly rea- Task Force, Aug 1999. RFC 2663. soning about the effects the three very different mecha- [16] Cisco Systems. BGP Best Path Selection Algorithm. nisms of packet filters, routing policy, and packet trans- http://www.cisco.com/warp/public/459/25.shtml. formations have on the network’s reachability. [17] Cisco Systems. Enhanced IGRP. http://www.cisco.com/- univercd/cc/td/doc/cisintwk/ito doc/en igrp.htm. Finally, we show how the framework can be applied to [18] Renata Teixeira and Jennifer Rexford. A measurement frame- a static description of the network’s definition, allowing work for pin-pointing routing changes. In ACM SIGCOMM it to be applied either during the network design process Workshop on Network Troubleshooting, September 2004. or to a deployed network. Our technique for static anal- [19] Geoffrey Xie, Jibin Zhan, David A. Maltz, Hui Zhang, Albert Greenberg, Gisli Hjalmtysson, and Jennifer Rexford. On static ysis of network reachability is valuable for verifying the reachability analysis of IP networks. Technical Report CMU- intent of the network designer, troubleshooting reachabil- CS-04-146, Carnegie Mellon University, June 2004. ity problems, and performing “what-if” analysis of failure scenarios. Now that we have this formal framework, our future work is focused on experimental evaluation of the algorithms on a set of networks. For example, our framework can be extended for computing finer-grain reachability bounds, such as ones that consider only a subset of packets (e.g., those carrying TCP port 1443 traffic) or a subset of paths (e.g., excluding certain links).