#CLUS Highly Available Wide Area Network Design

David Prall, Principal Systems Engineer. CCIE #6508 BRKRST-2042

#CLUS Agenda

• Introduction

• Cisco IOS and IP

• Convergence Techniques

• Design and Deployment

• Final Wrap Up

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Hidden Agenda • Introduction • Cisco IOS and IP Routing • Multiple Links/Multiple Paths • Load Sharing • Convergence Techniques • Interface Detection • Routing Protocols • Enhanced Object Tracking • First Hop Redundancy Protocols • Performance Routing • Cisco SD-WAN (Viptela) • Design and Deployment • MPLS Dual Carrier • MPLS + Internet

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 Your speaker

• David Prall • Principal Systems Engineer • World Wide Enterprise Networking • [email protected] • CCIE 6508 (R&S/SP/Security) • Started at Cisco July 10, 2000 • Washington, DC

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 Cisco Webex Teams

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Live Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKRST-2042 by the speaker until June 18, 2018.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 Goals

• Efficiently utilize available bandwidth

• Dynamically respond to all types of disruptions

• Leverage most effective design techniques that meet the design requirements

• Review today’s technology

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 Where Can Outages Occur?

Link or Device Failure

MPLS - SP A

C-A-R2 Link or Device Degraded

C-A-R1 C-A-R4

C-A-R3

HQ-W1 BR-W1

MPLS - SP B HQ-W2 BR-W2

C-B-R1 C-B-R4

• How does outage manifest? • How quickly can network detect? • How long is bidirectional reconvergence?

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Session Scope

• What methods are used for path selection and packet forwarding

• How does the network detect outages

• Focus on network survivability and effective utilization rather than sub-second convergence • Modern Design using SD-WAN

• Does not address “zero loss” considerations • Please review BRKRST-2365 Unified HA Network Design - The Evolution of the Next Generation Network • Other sessions delivered by Matt Birkner

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Defining Availability Availability Downtime / Year

• System Availability: a ratio of the 98.000000% 7.3 Days expected uptime to the 99.000000% 3.65 Days experienced downtime over a 99.500000% 1.825 Days period of time of the same duration 99.900000% 8.76 Hrs 99.990000% 52.56 Min Branch • Branch WAN High Availability: 99.999000% 5.256 Min WAN Between 99.99% and 99.999% 99.999900% 31.536 Sec HA Targets Ultra HA • Ultra High Availability: Between 99.999990% 3.1536 Sec Targets 99.9999% and 99.999999% 99.999999% .31536 Sec

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Building Highly Available WANs Redundancy and Path Diversity Matter

Downtime SINGLE per Year 99.95%* Downtime Downtime 99.90%* per Year per Year , MPLS 4 Hours 8 Hours Internet SINGLE PATH 4–9 Hours 22 Minutes 46 Minutes ISR ISR Branch WAN HA Solution 99.995% 99.995% 99.995% SINGLE ROUTER, 26+ Minutes DUAL PATHS MPLS MPLS MPLS Internet Internet Internet ISR ISR ISR

99.999% 99.999% 99.999%

DUAL ROUTERS, 5+ Minutes Internet DUAL PATHS MPLS MPLS MPLS Internet Internet

ISR ISR ISR ISR ISR ISR

* Typical MPLS and Business Grade Broadband Availability SLAs and Downtime per Year, calculated with Cisco AS DAAP tool. #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 Deployment Options

MPLS/MPLS MPLS/Internet MPLS/LTE Internet/Internet Internet/LTE MPLS MPLS Internet Internet LTE/LTE LTE LTE 100’s of Combinations

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 Agenda

• Introduction

• Cisco IOS and IP Routing • Multiple Links/Multiple Paths • Load Sharing

• Convergence Techniques • Design and Deployment

• Final Wrap Up

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Routing Table Basics Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override, p - overrides from PfR

p 10.0.0.0/8 is variably subnetted, 14 subnets, 5 masks B p 10.0.0.0/8 [20/0] via 172.16.0.6, 00:12:36 B p 10.3.0.0/16 [20/0] via 172.16.0.6, 00:12:36 B p 10.4.0.0/16 [200/0], 00:13:52, Null0 C p 10.4.0.41/32 is directly connected, Loopback0 D p 10.4.1.0/24 [90/307200] via 10.4.49.2, 00:14:32, Ethernet0/0 C p 10.4.49.0/30 is directly connected, Ethernet0/0 L p 10.4.49.1/32 is directly connected, Ethernet0/0 B p 10.9.0.0/16 [20/0] via 172.16.0.6, 00:12:36 100.0.0.0/8 is variably subnetted, 9 subnets, 2 masks B 100.64.0.0/24 [20/0] via 100.64.3.1, 00:13:43 C 100.64.3.0/24 is directly connected, Ethernet0/2 L 100.64.3.2/32 is directly connected, Ethernet0/2 172.16.0.0/16 is variably subnetted, 9 subnets, 2 masks B 172.16.0.0/31 [20/0] via 172.16.0.6, 00:12:36 C 172.16.0.6/31 is directly connected, Ethernet0/1 L 172.16.0.7/32 is directly connected, Ethernet0/1 #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 INFORMATIONAL Administrative Distance Default • The distance command is used to configure Route Source Distance a rating of the trustworthiness of a routing Connected 0 information source, such as an individual Interface router or a group of routers Static Route 1 EIGRP Summary 5 • Numerically, an administrative distance is a Route BGP External positive integer from 1 to 255. In general, 20 the higher the value, the lower the trust (eBGP) rating EIGRP Internal 90 OSPF 110 • An administrative distance of 255 means the IS-IS 115 routing information source cannot be trusted RIP 120 at all and should be ignored EIGRP External 170 BGP Internal 200 (iBGP) Unknown 255

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 Route Selection

• How is administrative OSPF EIGRP OSPF distance used to determine which route should be installed? 10.0.14.0/24 10.0.14.0/24 10.0.14.0/25 These Two Routes • Only identical routes Are Identical are compared EIGRP Internal = 90 • Identical prefixes with OSPF = 110 different prefix lengths EIGRP Internal Installed are not the same route router#show ip route 10.0.14.0 255.255.255.0 Routing entry for 10.0.14.0/24 Known via "eigrp 1", distance 90, metric 307200, type internal • The route from the Redistributing via eigrp 1 protocol with the lower Last update from 10.0.121.2 on Ethernet0/1, 00:01:32 ago Routing Descriptor Blocks: administrative distance * 10.0.121.2, from 10.0.121.2, 00:01:32 ago, via Ethernet0/1 is installed Route metric is 307200, traffic share count is 1 Total delay is 2000 microseconds, minimum bandwidth is 10000 Kbit Reliability 255/255, minimum MTU 1500 bytes Loading 1/255, Hops 1

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 Route Selection

• What about longest prefix comparison?

• Only identical routes are compared • Identical prefixes with different prefix lengths are not the same route

• The route with the longest prefix is router#show ip route 10.0.14.0 255.255.255.0 longer-prefixes 10.0.0.0/8 is variably subnetted, 9 subnets, 3 masks installed D 10.0.14.0/24 [90/307200] via 10.0.121.2, 00:01:35, Ethernet0/1 O 10.0.14.0/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2 O 10.0.14.128/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 Agenda

• Introduction

• Cisco IOS and IP Routing • Multiple Links/Multiple Paths • Load Sharing

• Convergence Techniques • Design and Deployment

• Final Wrap Up

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 Load Sharing

• Assume the same routing process attempts to install two routes for the same destination in the RIB

• The routing process may allow the second route to be installed based on its own rules

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 CEF Load Sharing

Per-Destination Per-Packet1 Default behaviour of IOS Universal Requires “ip load-sharing per- Algorithm “show cef state” packet” interface configuration1 Per-flow using destination hash Per-packet using round-robin method Packets for a given Packets for a given source/destination session will take source/destination session may take the same path different paths More effective as the number of Ensures traffic is more evenly destinations increase distributed over multiple paths Ensures that traffic for a given session Potential for packets to arrive out of arrives in order sequence

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 Load Sharing router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Known via "eigrp 100", distance 170, metric 3072256, type external Redistributing via eigrp 100 Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0 Route metric is 3072256, traffic share count is 1 .... 192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1 Route metric is 3072256, traffic share count is 1 ....

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 Load Sharing – with EIGRP Variance router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Known via "eigrp 100", distance 170, metric 3072256, type external Redistributing via eigrp 100 Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0 Route metric is 1536128, traffic share count is 2 .... 192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1 Route metric is 3072256, traffic share count is 1 ....

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 Load Sharing – with eBGP dmzlink-bw router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Known via "bgp 1", distance 20, metric 0 Tag 2, type external Last update from 10.0.122.2 00:00:16 ago Routing Descriptor Blocks: 10.0.122.2, from 10.0.122.2, 00:00:16 ago Route metric is 0, traffic share count is 1 .... * 10.0.121.2, from 10.0.121.2, 00:00:16 ago Route metric is 0, traffic share count is 2 .... router#show ip bgp 192.168.239.0 BGP routing table entry for 192.168.239.0/24, version 9 Paths: (2 available, best #2, table default) Multipath: eBGP .... 10.0.122.2 from 10.0.122.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, valid, external, multipath(oldest) DMZ-Link Bw 312 kbytes rx pathid: 0, tx pathid: 0 .... 10.0.121.2 from 10.0.121.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, valid, external, multipath, best DMZ-Link Bw 625 kbytes rx pathid: 0, tx pathid: 0x0

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 CEF Hashing and Exact Route

• • • “show ip cef exact-route [src-port] [dest-port]”

#show ip cef exact-route 1.1.1.1 2.2.2.2 1.1.1.1 -> 2.2.2.2 =>IP adj out of GigabitEthernet1, addr 10.255.0.1

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Performance Routing • Cisco SD-WAN (Viptela)

• Design and Deployment

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Interface Detection – Carrier-Delay

• Carrier-delay • If a link goes down and comes back up before the carrier delay timer expires, the down state is effectively filtered, and the rest of the software on the router is not aware that a link-down event occurred. • Imposes a default 2 second pause before processing interface events • Disabling carrier-delay speeds convergence upon interface events • Disabling carrier-delay can increase control-plane usage during repetitive interface events (flapping)

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Interface Detection - Dampening

• Dampening • Imposes a logarithmic delay based on interface events • Coupled with carrier-delay, dampening protects the control-plane from repetitive events by increasing the delay before processing up events should the interface flap. #conf t (config-if)#interface GigabitEthernet1 (config-if)#carrier-delay 0 (config-if)#dampening (config-if)#end #show dampening interface 1 interface is configured with dampening. No interface is being suppressed. Features that are using interface dampening: IP Routing

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Performance Routing • Cisco SD-WAN (Viptela)

• Design and Deployment

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Timers INFORMATIONAL

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Routing Protocol Neighbor Behavior INFORMATIONAL

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Routing Protocol Neighbor Behavior Adjust Hello Timers

R4#show ip bgp vpnv4 vrf cisco neighbor BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link BGP version 4, remote router ID 192.168.201.10 BGP state = Established, up for 1d10h Last read 00:00:19, hold time is 180, keepalive interval is 60 seconds BR-W1# router bgp 65110 R4#show ip bgp vpnv4 vrf cisco neighbor neighbor 192.168.101.9 timers 7 21 BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link BGP version 4, remote router ID 192.168.201.10 BGP state = Established, up for 00:01:23 Last read 00:00:03, hold time is 21, keepalive interval is 7 seconds

%Warning: A Hold Time of Less than 20 Seconds Increases the Chances of Peer Flapping #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 Bidirectional Forwarding Detection (BFD)

• Extremely lightweight hello protocol • IPv4, IPv6, MPLS, P2MP

• 10s of milliseconds (technically, microsecond resolution) forwarding plane failure detection mechanism.

• Single mechanism, common and standardized • Multiple modes: Async (echo/non-echo), Demand

• Independent of Routing Protocols

• Levels of security, to match conditions and needs

• Facilitates close alignment with hardware

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 Drivers for BFD • Link-layer detection misses some types of outages • e.g. Control Plane failure

• Control Plane failure detection is very conservative • 15-180 seconds in default configurations

• Link-layer failure detection is not consistent across media types • Less than 50ms on APS- protected SONET • A few seconds on Ethernet • Several seconds or more on WAN links

• Provides a measure of consistency across routing protocols

• Most current failure detection mechanisms are an order of magnitude too long for time-sensitive applications

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection interface GigabitEthernet4 ip address 10.3.255.9 255.255.255.252 bfd interval 50 min_rx 50 multiplier 3 router eigrp 1 network 10.3.0.0 0.0.255.255 bfd all-interfaces R1#show bfd neighbors details IPv4 Sessions NeighAddr LD/RD RH/RS State Int 10.3.255.10 4104/1 Up Up Gi4 Session state is UP and using echo function with 50 ms interval. interface GigabitEthernet2 Session Host: Software ip address 172.17.2.9 255.255.255.254 OurAddr: 10.3.255.9 bfd interval 333 min_rx 333 multiplier 3 Handle: 2 router bgp 65000 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3 neighbor 172.17.2.8 fall-over bfd Received MinRxInt: 1000000, Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(1371) Rx Count: 985, Rx Interval (ms) min/max/avg: 34/1978/1226 last: 290 ms ago Tx Count: 1372, Tx Interval (ms) min/max/avg: 71/1137/879 last: 721 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: EIGRP CEF Uptime: 00:20:06 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 My Discr.: 1 - Your Discr.: 4104 Min tx interval: 1000000 - Min rx interval: 1000000 Min Echo interval: 50000 #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection interface GigabitEthernet4 ip address 10.3.255.9 255.255.255.252 bfd interval 50 min_rx 50 multiplier 3 router eigrp 1 network 10.3.0.0 0.0.255.255 bfd all-interfaces IPv4 Sessions NeighAddr LD/RD RH/RS State Int 172.17.2.8 4102/1 Up Up Gi2 Session state is UP and using echo function with 333 ms interval. interface GigabitEthernet2 Session Host: Software ip address 172.17.2.9 255.255.255.254 OurAddr: 172.17.2.9 bfd interval 333 min_rx 333 multiplier 3 Handle: 1 router bgp 65000 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3 neighbor 172.17.2.8 fall-over bfd Received MinRxInt: 1000000, Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(6076) Rx Count: 4977, Rx Interval (ms) min/max/avg: 4/1970/1069 last: 491 ms ago Tx Count: 6077, Tx Interval (ms) min/max/avg: 754/1180/879 last: 655 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: BGP CEF Uptime: 01:29:04 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 My Discr.: 1 - Your Discr.: 4102 Min tx interval: 1000000 - Min rx interval: 1000000 Min Echo interval: 333000 #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 Routing Protocol Neighbor Behavior Detecting Unreachable Neighbor (Hello Timers vs. BFD)

R1#show clock *09:58:27.716 UTC Sat Jan 27 2018 R1# *Jan 27 09:58:40.612: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 10.3.255.10 (GigabitEthernet4) is down: holding time expired

R1#show clock *09:35:44.408 UTC Sat Jan 27 2018 R1# *Jan 27 09:35:45.571: %BFDFSM-6-BFD_SESS_DOWN: BFD-SYSLOG: BFD session ld:4101 handle:2,is going Down Reason: ECHO FAILURE *Jan 27 09:35:45.575: %BFD-6-BFD_SESS_DESTROYED: BFD-SYSLOG: bfd_session_destroyed, ld:4101 neigh proc:EIGRP, handle:2 act *Jan 27 09:35:45.580: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 10.3.255.10 (GigabitEthernet4) is down: BFD peer down notified

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Performance Routing • Cisco SD-WAN (Viptela)

• Design and Deployment

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 EOT, Static Routing, and DDR

• Enhanced Object Tracking (EOT)

• Static Routing Options • Floating Static Routes • Reliable Static Routing (RSR) using EOT

• Dial on Demand Routing (DDR) • EEM Script • DMVPN State Tracking

• More information: • https://www.cisco.com/c/en/us/support/docs/dial-access/dial-on- demand-routing-ddr/10213-backup-main.html

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 Enhanced Object Tracking (EOT) Local Significance

track object-number interface type number line-protocol track 1 interface serial 2/0 line-protocol track object-number interface type number ip routing track 2 interface ethernet 1/0 ip routing track object-number ip route IP-Addr/Prefix-len reachability track 3 ip route 10.16.0.0/16 reachability track object-number ip route IP-Addr/Prefix-len metric threshold track 4 ip route 10.16.0.0/16 metric threshold

Router#show track 100 Router#show track 103 Track 100 Track 103 Interface Serial2/0 line-protocol IP route 10.16.0.0 255.255.0.0 reachability Line protocol is Up Reachability is Up (EIGRP) 1 change, last change 00:00:05 1 change, last change 00:02:04 Tracked by: First-hop interface is FastEthernet0/0 GLBP FastEthernet0/1 1 Tracked by: GLBP FastEthernet0/1 1

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 Enhanced Object Tracking (EOT) External Significance

track object-number ip sla type number state track 5 ip sla 4 state track object-number ip sla type number reachability track 6 ip sla 4 reachability

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 Enhanced Object Tracking (EOT) Compound Operations

track object-number list boolean {and|or} and - both are up for object to be up or - one is up for object to be up track 5 list boolean or object 51 object 52 not ! Negates state of object track object-number list threshold {weight|percentage} track 6 list threshold weight object 61 weight 20 ! Twice as important object 62 ! Default weight 10 object 63 object 64 threshold weight up 30 down 25

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 Reliable Static Routing Tracking IP SLA track 4 list boolean or object 400 object 401 track 400 ip sla 400 reachability track 401 ip sla 401 reachability ip sla 400 icmp-echo 10.100.100.100 source-ip 10.1.2.120 timeout 100 frequency 10 ip sla schedule 400 life forever start-time now ip sla 401 icmp-echo 10.100.200.100 source-ip 10.1.2.120 timeout 100 frequency 10 ip sla schedule 401 life forever start-time now ! ip route 10.100.100.100 255.255.255.255 Ethernet 0/1 192.168.101.9 ip route 10.100.200.100 255.255.255.255 Ethernet 0/1 192.168.101.9 ip route 10.100.100.100 255.255.255.255 Null0 2 ip route 10.100.200.100 255.255.255.255 Null0 2 ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 ip route 10.100.0.0 255.255.0.0 192.168.201.9 200 BR-W1#show ip route track-table ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [up] BR-W1#show ip route 10.100.0.0 255.255.0.0 S 10.100.0.0/16 [1/0] via 192.168.101.9

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 Reliable Static Routing Tracking IP SLA

BR-W1# *Mar 12 03:57:28.367: %TRACKING-5-STATE: 400 ip sla 400 reachability Up->Down *Mar 12 03:57:37.374: %TRACKING-5-STATE: 401 ip sla 401 reachability Up->Down *Mar 12 03:57:38.137: %TRACKING-5-STATE: 4 list boolean or Up->Down

BR-W1#show ip route track-table ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [down] BR-W1#show ip route 10.100.0.0 255.255.0.0 longer-prefixes S 10.100.0.0/16 [200/0] via 192.168.201.9 S 10.100.100.100/32 [1/0] via 192.168.101.9, Ethernet0/1 S 10.100.200.100/32 [1/0] via 192.168.101.9, Ethernet0/1

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 EEM Script IPv6 Static Route Event Tracking ipv6 route 2001:DB8::12/128 2001:DB8:B::5

ip sla 610 icmp-echo 2001:DB8::12 source-interface GigabitEthernet0/1.99 threshold 1000 frequency 10 ip sla schedule 610 life forever start-time now

track 600 list threshold percentage object 610 2001:DB8:B::5 threshold percentage down 40 up 60 track 610 ip sla 610

event manager applet DISABLE-STATIC-IPv6 Don’t forget to reenable event track 600 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "no ipv6 route ::/0 2001:DB8:B::5" action 4 cli command "end" action 99 syslog msg “DEFAULT IPv6 ROUTE DISABLED" BR-RTR# 14:22:14: %TRACKING-5-STATE: 610 ip sla 610 state Up->Down 14:22:14: %TRACKING-5-STATE: 600 list threshold percentage Up->Down 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:DISABLE-STATIC-IPv6) 14:22:14: %HA_EM-6-LOG: DISABLE-STATIC-IPv6: DEFAULT IPv6 ROUTE DISABLED

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 Black Hole Route Detection ip sla 110 IPSLA with EEM icmp-echo 208.67.222.222 source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 ! fVRF configuration threshold 1000 frequency 15 ip sla schedule 110 life forever start-time now ip sla 111 icmp-echo 208.67.220.220 source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now

track 60 ip sla 110 reachability track 61 ip sla 111 reachability track 62 list boolean or IP SLA object 60 Probes object 61

(config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ?

event manager applet DISABLE-STATIC-GIG0-0 event track 62 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "no ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10" action 4 cli command "end" action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 DISABLED"

event manager applet ENABLE-STATIC-GIG0-0 event track 62 state up action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10" action 4 cli command "end" action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 ENABLED"

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Black Hole Route Detection IPSLA with Recursive Routing Interface GigabitEthernet0/0 vrf forwarding INET-PUBLIC1 ip address dhcp ip sla 110 icmp-echo 208.67.222.222 source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 ! fVRF configuration threshold 1000 frequency 15 ip sla schedule 110 life forever start-time now ip sla 111 IP SLA icmp-echo 208.67.220.220 source-interface GigabitEthernet0/0 Probes vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now

track 60 ip sla 110 reachability track 61 ip sla 111 reachability track 62 list boolean or object 60 object 61 (config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ? ip route 192.0.2.33 255.255.255.255 GigabitEthernet0/0 dhcp 10 ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 192.0.2.33 10 track 62

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46 EEM Script

3G Backup with Event Tracking ip sla 100 icmp-echo 192.168.4.22 source-interface GigabitEthernet0/1 threshold 1000 frequency 15 ip sla schedule 100 life forever start-time now

track 60 ip sla 100 reachability

Don’t forget to disable event manager applet ACTIVATE-3G event track 60 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "interface cellular0/0/0" action 4 cli command "no shutdown" action 5 cli command "end" action 99 syslog msg "Activating 3G interface" 14:22:14: %TRACKING-5-STATE: 60 ip sla 100 reachability Up->Down 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:ACTIVATE-3G) 14:22:14: %HA_EM-6-LOG: ACTIVATE-3G: Activating 3G interface 14:22:34: %LINK-3-UPDOWN: Interface Cellular0/0/0, changed state to up 14:22:34: %DIALER-6-BIND: Interface Ce0/0/0 bound to profile Di1 14:22:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0/0/0, changed state to up 14:22:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel11, changed state to up 14:22:40: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON 14:22:42: %DUAL-5-NBRCHANGE: EIGRP-IPv4 201: Neighbor 10.4.36.1 (Tunnel11) is up: new adjacency

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 DMVPN Interface State Control track 2 list boolean or 3G Backup with DMVPN object 101 not track 101 interface Tunnel100 line-protocol interface Tunnel200 if-state track 2 tunnel source Cellular0/0/0 end #show track 2 Track 2 List boolean or Boolean OR is Down 7 changes, last change 00:07:55 object 101 not Up Tracked by: IF-State Control 2 17:24:18.682: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel100, changed state to down 17:24:18.682: %TRACK-6-STATE: 101 interface Tu100 line-protocol Up -> Down 17:24:18.744: %TRACK-6-STATE: 2 list boolean or Down -> Up 17:24:28.683: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel200, changed state to up 17:24:29.276: %BGP-5-ADJCHANGE: neighbor 192.168.200.12 Up 17:24:37.505: %BGP-5-ADJCHANGE: neighbor 192.168.200.22 Up #show track 2 Track 2 List boolean or Boolean OR is Up 8 changes, last change 00:00:32 object 101 not Down Tracked by: IF-State Control 2 #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Performance Routing • Cisco SD-WAN (Viptela)

• Design and Deployment

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49 First Hop Redundancy Protocols (FHRP) Failure Protection for the First Hop IP Router

• Hot Standby Router Protocol (HSRP) • v2 IPv4 and IPv6 • Virtual Router Redundancy Protocol (VRRP) • RFC5798 (v3 IPv4 and IPv6), RFC3768 (v2 IPv4), RFC2338 (v1) • Gateway Load Balancing Protocol (GLBP) • IPv4 and IPv6

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50 Drivers for FHRPs

• Provide routing redundancy for access layer • How to handle failover when end-hosts have only a single IP default gateway and cached ARP entry

• Provide routing redundancy for devices that depend on static routing • Some firewalls do not support

• Independent of routing protocols • Works with any routing protocol and static routing

• Capable of providing sub-second failover

• Provides load sharing capabilities (GLBP) transparent to end host

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 Hot Standby Routing Protocol (HSRP)

interface FastEthernet0/0 ip address 10.1.2.2 255.255.255.0 interface FastEthernet0/0 standby version 2 ip address 10.1.2.3 255.255.255.0 standby 4 ip 10.1.2.1 standby version 2 standby 4 priority 110 standby 4 ip 10.1.2.1 standby 4 preempt standby 4 preempt standby 6 ipv6 autoconfig standby 6 ipv6 autoconfig standby 6 priority 110 standby 6 preempt standby 6 preempt ipv6 address 2001:DB8:5:1::2/64 ipv6 address 2001:DB8:5:1::1/64

BR-W1#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 110 P Active local 10.1.2.3 10.1.2.1 Fa0/0 6 110 P Active local FE80::A8BB:CCFF:FE00:3400 FE80::5:73FF:FEA0 :6 BR-W2#show standby brief Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 100 P Standby 10.1.2.2 local 10.1.2.1 Fa0/0 6 100 P Standby FE80::A8BB:CCFF:FE00:3300 local FE80::5:73FF:FEA0 :6

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 Hot Standby Routing Protocol (HSRP)

BR-W2#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 100 P Active local unknown 10.1.2.1 Fa0/0 6 100 P Active local unknown FE80::5:73FF:FEA0 :6

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53 Hot Standby Routing Protocol (HSRP)

#track 100 interface serial2/0 line-protocol ! interface FastEthernet0/0 standby version 2 standby 4 priority 110 standby 4 track 100 decrement 20 standby 6 priority 110 standby 6 track 100 decrement 20

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 Gateway Load Balancing Protocol (GLBP) BR-W1#show run int fa0/0 interface FastEthernet0/0 ip address 10.1.2.2 255.255.255.0 glbp 4 ip 10.1.2.1 glbp 4 preempt glbp 4 weighting 110 lower 100 glbp 6 ipv6 autoconfig glbp 6 preempt glbp 6 weighting 110 lower 100 ipv6 address 2001:DB8:5:1::1/64 BR-W1#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Active 10.1.2.1 local 10.1.2.3 Fa0/0 4 1 - Active 0007.b400.0401 local - Fa0/0 4 2 - Listen 0007.b400.0402 10.1.2.3 - Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600 local FE80::A8BB:CCF F:FE00:3400 Fa0/0 6 1 - Active 0007.b400.0601 local - Fa0/0 6 2 - Listen 0007.b400.0602 FE80::A8BB:CCFF:FE00:3400 - BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Standby 10.1.2.1 10.1.2.2 local Fa0/0 4 1 - Listen 0007.b400.0401 10.1.2.2 - Fa0/0 4 2 - Active 0007.b400.0402 local - Fa0/0 6 - 100 Standby FE80::7:B4FF:FE00:600 FE80::A8BB:CCFF:FE00:3300 local Fa0/0 6 1 - Listen 0007.b400.0601 FE80::A8BB:CCFF:FE00:3300 - Fa0/0 6#CLUS 2 - ActiveBRKRST-2042 0007.b400.0602© 2018 Cisco and/or its affiliates. local All rights reserved. Cisco - Public 55 Gateway Load Balancing Protocol (GLBP)

BR-W2# *May 26 19:09:14.260: %GLBP-6-STATECHANGE: FastEth0/0 Grp 4 state Standby -> Act ive *May 26 19:09:15.326: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Liste n -> Active *May 26 19:09:15.826: %GLBP-6-STATECHANGE: FastEth0/0 Grp 6 state Standby -> Act ive *May 26 19:09:16.856: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 6 Fwd 1 state Liste n -> Active

BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Active 10.1.2.1 local unknown Fa0/0 4 1 - Active 0007.b400.0401 local - Fa0/0 4 2 - Active 0007.b400.0402 local - Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600 local unknown Fa0/0 6 1 - Active 0007.b400.0601 local - Fa0/0 6 2 - Active 0007.b400.0602 local -

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56 GLBP with Enhanced Object Tracking

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 Enhanced Object Tracking (EOT) ip sla 100 Tracking IP SLA icmp-echo 10.100.100.100 source-ip 10.1.2.2 timeout 100 frequency 10 Lo0 10.100.100.100 Lo0 10.100.200.100 ip sla schedule 100 life forever start-time now ip sla 200 icmp-echo 10.100.200.100 source-ip 10.1.2.2 timeout 100 frequency 10 ip sla schedule 200 life forever start-time now ip route 10.100.100.100 255.255.255.255 FastEthernet0/1 192.168.101.9 ip route 10.100.200.100 255.255.255.255 FastEthernet0/1 192.168.101.9 ip route 10.100.100.100 255.255.255.255 Null0 2 ip route 10.100.200.100 255.255.255.255 Null0 2 BR-W1#show ip sla statistics IPSLA operation id: 100 Latest RTT: 1 milliseconds Latest operation start time: *04:42:11.444 UTC Tue Feb 17 2009 Latest operation return code: OK Number of successes: 46 Number of failures: 0 Operation time to live: Forever IPSLA operation id: 200 Latest RTT: 1 milliseconds Latest operation start time: *04:42:11.356 UTC Tue Feb 17 2009 Latest operation return code: OK Number of successes: 24 Number of failures: 0

Operation#CLUS time to BRKRSTlive:-2042 Forever© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58 Enhanced Object Tracking Tracking IP SLA BR-W1# track 100 ip sla 100 reachability BR-W1#show glbp track 200 ip sla 200 reachability FastEthernet0/0 – Group 4 track 1 list boolean or State is Active 1 state change, last state change 00:09:59 object 100 Virtual IP address is 10.1.2.1 object 200 Hello time 3 sec, hold time 10 sec interface FastEthernet0/0 Next hello sent in 2.336 secs ip address 10.1.2.2 255.255.255.0 Redirect time 600 sec, forwarder timeout 14400 sec glbp 4 ip 10.1.2.1 Preemption enabled, min delay 0 sec Active is local glbp 4 priority 110 Standby is 10.1.2.3, priority 105 (expires in 7.808 sec) glbp 4 preempt Priority 110 (configured) glbp 4 weighting 110 lower 100 Weighting 110 (configured 110), thresholds: lower 100, glbp 4 load-balancing weighted upper 110 Track object 1 state Up decrement 20 glbp 4 weighting track 1 decrement 20 Load balancing: weighted Group members: aabb.cc00.0110 (10.1.2.2) local aabb.cc00.0410 (10.1.2.3) There are 2 forwarders (1 active) Forwarder 1 State is Active Forwarder 2 State is Listen

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59 Enhanced Object Tracking Composite Failure BR-W1# *Feb 17 05:17:25: %TRACKING-5-STATE: 100 ip sla 100 state Up->Down *Feb 17 05:17:25: %TRACKING-5-STATE: 200 ip sla 200 state Up->Down *Feb 17 05:17:26: %TRACKING-5-STATE: 1 list boolean or Up->Down *Feb 17 05:17:38: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Active -> Listen BR-W2#show glbp FastEthernet0/0 – Group 4 State is Standby 1 state change, last state change 00:28:16 Virtual IP address is 10.1.2.1 Hello time 3 sec, hold time 10 sec Next hello sent in 1.856 secs Redirect time 600 sec, forwarder timeout 14400 sec Preemption enabled, min delay 0 sec Active is 10.1.2.2, priority 110 (expires in 10.400 sec) Standby is local Priority 105 (configured) Weighting 110 (configured 110), thresholds: lower 100, upper 110 Track object 1 state Up decrement 20 Load balancing: weighted Group members: aabb.cc00.0110 (10.1.2.2) aabb.cc00.0410 (10.1.2.3) local There are 2 forwarders (2 active) Forwarder 1 State is Active Forwarder 2 State is Active #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Performance Routing • Cisco SD-WAN (Viptela)

• Design and Deployment

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61 Enterprise WAN Challenge

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62 Performance Routing (PfR)

Si

• Uses Reachability, Delay, Loss, Jitter, and Load to determine the best path • PfR Components • BR—Border Router (Forwarding Path) • MC—Master Controller (Decision Maker)

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63 Traditional Topology

• Routing protocol selects path • Blackhole reconvergence can take minutes • Will not recover from brownouts

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64 PfR Enabled Topology L3—L7 Aware

. PfR uses routing protocol and policy to select path . Live Traffic Monitoring significantly improves reconvergence due to blackholes and brownouts

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Performance Routing • Cisco SD-WAN (Viptela)

• Design and Deployment

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 66 Overlay Management Protocol (OMP) vSmart • TCP based extensible control plane protocol • Runs between vEdge routers and vSmart controllers and between the vSmart controllers - Inside TLS/DTLS connections • Leverages address families to advertise reachability for TLOCs, unicast/multicast vSmart vSmart destinations (statically/dynamically learnt service side routes), service routes (L4-L7), BFD stats (TE and H-SDWAN) and Cloud onRamp for SaaS probe stats (gateway) - Uses attributes • Distributes IPSec encryption keys, and data and vEdge vEdge app-aware policies (embedded NETCONF) Note: vEdge routers need not connect to all vSmart Controllers

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67 Bidirectional Forwarding Detection (BFD)

vEdge • Path liveliness and quality measurement detection protocol - Up/Down, loss/latency/jitter, IPSec tunnel MTU • Runs between all vEdge and vEdge Cloud routers in the topology vEdge vEdge - Inside IPSec tunnels - Operates in echo mode - Automatically invoked at IPSec tunnel establishment - Cannot be disabled

• Uses hello (up/down) interval, poll (app-aware) vEdge vEdge interval and multiplier for detection - Fully customizable per-vEdge, per-color

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68 Application Visibility and Recognition Hidden

Deep Packet Inspection

Cloud Data Center App 1 App 2

App 3,000 Data Center vEdge Router MPLS 4G INET  App Firewall Small Office Home Office  Traffic prioritization Campus  Transport selection Branch

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69 Hidden vEdge Router QoS Data Policy vManage Classification of application traffic into QoS forwarding classes (queues)

vSmart

Ingress Interface Egress Interface QoS forwarding QoS classes Scheduler Out Application FC Q FC Q Traffic FC Q

Policing Map into Policing Shaping Bandwidth % (ACL Action) Egress Queue Buffer % Scheduling Priority ACL Match ACL Action/Data Drop Policy Control Plane (Map into FCs)

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70 Path Quality and Liveliness Detection Multiplier (n) • Each vEdge router sends BFD hello packets for path quality and liveliness detection - Packets echoed back by remote site

Hello Interval (ms) • Hello interval and multiplier determine how Liveliness many BFD packets need to be lost to Quality declare IPSec tunnel down App-Route Multiplier (n) • Number of hello intervals that fit inside poll interval determines the number of BFD Poll Interval Poll Interval Poll Interval (ms) packets considered for establishing poll interval average path quality • App-route multiplier determines number of poll intervals for establishing overall Hello Interval (ms) average path quality

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71 Critical Applications SLA

. vEdge Routers vManage App Aware Routing Policy continuously perform path App A path must have: Latency < 150ms liveliness and quality Loss < 2% measurements Jitter < 10ms

Internet Remote Site

MPLS Regional Path 2 Data Center

4G LTE

Path1: 10ms, 0% loss, 5ms jitter Path2: 200ms, 3% loss, 10ms jitter Path3: 140ms, 1% loss, 10ms jitter IPSec Tunnel

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 Hidden Site Redundancy - Routed . Redundant pair of vEdge routers operate in active/active mode SD-WAN Fabric . vEdge routers are one or more Layer 3 hops away from the hosts . Standard OSPF or BGP routing protocols are running between the redundant pair vEdge vEdge A vEdge B routers and the site router . Bi-directional redistribution between OMP and OSPF/BGP and vice versa on the vEdge routers Site - OSPF DN bit, BGP SoO community Router . Site router performs equal cost multipathing for remote destinations across SD-WA Fabric - Can manipulate OSPF/BGP to prefer one vEdge Host router over the other

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73 Hidden Site Redundancy - Bridged

. vEdge routers are Layer 2 adjacent to the hosts SD-WAN - Default gateway for the hosts Fabric . Virtual Router Redundancy Protocol (VRRP) runs between the two redundant vEdge routers - Active/active when using multi-group (per- VLAN) vEdge A vEdge B VRRP Active VRRP Standby . VRRP Active vEdge responds to ARP requests VRRP for the virtual IP with its physical interface MAC address - No virtual MAC . In case of failover, new VRRP Active vEdge router sends out gratuitous ARP to update ARP table on the hosts and mac address table on Host the intermediate L2 switches

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74 Transport Redundancy - Meshed . vEdge routers are directly connected to all the transports MPLS Internet - No need for L2 switches front-ending the vEdge routers . When transport goes down, vEdge routers detect the condition and bring down the tunnels built across the failed transport vEdge vEdge - BFD times out across tunnels . Both vEdge routers still draw the traffic for the prefixes available through the SD-WAN fabric . If one of the vEdge routers fails (dual failure), second vEdge router takes over forwarding the traffic in and out of site - Both transport are still available

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75 Transport Redundancy – TLOC Extension

. vEdge routers are connected only to their respective transports MPLS Internet . vEdge routers build IPSec tunnels across directly connected transports and across the transports connected to the neighboring vEdge router vEdge vEdge - Neighboring vEdge router acts as an underlay router for tunnels initiated from the other vEdge . If one of the vEdge routers fails (dual failure), second vEdge router takes over forwarding the traffic in and out of site - Only transport connected to the remaining vEdge router can be used

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 76 Path and Remote-End Redundancy

. vEdge routers leverage BFD for detecting Data tunnel liveliness Center • If intermediate network path through the SD-WAN fabric fails or if the remote-end vEdge router (e.g. data center) fails, BFD MPLS Internet hellos will time out and remote site vEdge router will bring down its relevant IPSec tunnels • Traffic will be rerouted after the failed condition had been detected - BFD hello timer and multiplier can be Remote tweaked for faster detection Site

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77 SD-WAN Demo Summary of Convergence Techniques

N/A1 N/A1

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79 Agenda

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment • MPLS Dual Carrier • MPLS + Internet

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80 Dual WAN (MPLS—Dual Carrier) PE-CE Protocol: BGP

• Default behavior: 1-way load sharing

• Load is shared from HQ to Branch

HQ-CORE1#show ip route D EX 10.1.2.0/24 [170/258816] via 10.1.1.110, 02:24:22, Vlan10 [170/258816] via 10.1.1.210, 02:24:22, Vlan10

• Only one link used Branch to HQ BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81 Dual WAN (MPLS—Dual Carrier) PE-CE Protocol: BGP Layer 3 Campus Locations • IGP (EIGRP examples) • Routes redistributed from BGP into IGP (match & tag) • BGP routes are treated as IGP external

• BGP • No iBGP required between HQ-W1 & HQ-W2 (CE routers) • Routes redistributed from IGP into BGP except those tagged as originally sourced from BGP

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82 Dual WAN (MPLS—Dual Carrier) Mutual Route Redistribution Detail

HQ-W1# router eigrp networkers address-family ipv4 unicast autonomous-system 65110 topology base redistribute bgp 65110 metric 45000 100 255 1 1500 address-family ipv6 unicast autonomous-system 65110 topology base redistribute bgp 65110 metric 45000 100 255 1 1500

HQ-W1# router bgp 65110 address-family ipv4 redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES address-family ipv6 redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES ! route-map BLOCK-TAGGED-ROUTES deny 10 match tag 65100 65200 route-map BLOCK-TAGGED-ROUTES permit 20 !

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 83 Dual WAN (MPLS—Dual Carrier) PE-CE Protocol: BGP Layer 2 Single Router Branch • Is it possible to load share from Branch to HQ?

• BGP Multipath • Allows installation of multiple BGP paths to same destination • Requirements (all must be equal) • Neighbor AS or AS-PATH • Weight BR-W1#show ip bgp • Local Preference Network Next Hop Metric LocPrf Weight Path • AS-PATH length • Origin * 10.100.0.0/16 192.168.201.9 0 65200 65200 ? • Med *> 192.168.101.9 0 65100 65100 ? BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 84 Dual WAN (MPLS—Dual Carrier) PE-CE Protocol: BGP Layer 2 Single Router Branch • Is it possible to load share from Branch to HQ? • maximum-paths 2

• Requires hidden command: • bgp bestpath as-path multipath- relax

router bgp 65110 bgp bestpath as-path multipath-relax address-family ipv4 maximum-paths 2 address-family ipv6 maximum-paths 2 BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.201.9, 00:03:44 [20/0] via 192.168.101.9, 00:03:44

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 Agenda

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment • MPLS Dual Carrier • MPLS + Internet

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 DUAL WAN (MPLS + Internet) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP • Headquarters WAN Edge • W1 learns Branch route via eBGP • W2 learns Branch route via EIGRP

• Headquarters Core • W1 redistributes eBGP into EIGRP, results in EIGRP external • W2 does not require redistribution, results in EIGRP internal • Core1, Core2 install Branch route via W2 HQ-W1#show ip route B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01 HQ-W2#show ip route D 10.1.2.0/24 [90/26882560] via 10.0.1.2, 00:00:04, Tunnel1 HQ-CORE1#show ip route D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:02:32, Vlan10

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 87 DUAL WAN (MPLS + Internet) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP • Single Router Branch WAN Edge • W1 learns HQ route via eBGP and EIGRP Internal • eBGP Administrative Distance preferred

BR-W1#show ip route B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58 B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 DUAL WAN (MPLS + Internet) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP • Dual Router Branch WAN Edge • W1 learns HQ route via eBGP • W2 learns HQ route via EIGRP • No redistribution configured • HSRP Primary is on W1 BR-W1#show ip route B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58 B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06 BR-W2#show ip route D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 00:10:56, Tunnel1 D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 00:10:57, Tunnel1 BR-W1#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/1 1 110 P Active local 10.1.2.220 10.1.2.1

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 89 DUAL WAN (MPLS + Internet) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP • How to force HQ to Branch traffic across MPLS (primary)? • Adjust administrative distance • For EIGRP routes learned via tunnel • Ensure administrative distance is HQ-W2# higher than that of EIGRP external (170) router eigrp 65110 network 10.0.1.0 0.0.0.7 distance 195 10.0.1.0 0.0.0.7 • Redistribute between two EIGRP Processes Forcing External as done between BGP and Campus EIGRP HQ-W2# Router eigrp 65100 network 10.0.1.0 0.0.0.7 router eigrp 65110 redistribute eigrp 65100 HQ-W1#show ip route B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01 HQ-W2#show ip route D EX 10.1.2.0/24 [170/261120] via 10.1.1.110, 00:07:25, GigE0/0 HQ-CORE1#show ip route D EX 10.1.2.0/24#CLUS [170/258816]BRKRST-2042 ©via 2018 Cisco10.1.1.110, and/or its affiliates. All rights 00:08:44, reserved. Cisco Public Vlan1090 DUAL WAN (MPLS + Internet) MPLS Failure • Failure within MPLS cloud • Dependent on provider • Worst Case • Link up neighbor down • Primary dependency BGP timers • End to end convergence time as long as BGP Holdtime • Configuration options • BFD for almost immediate notification • End-to-end Application Restoration as fast as PfR or SD-WAN detects HQ-W2#show ip route D 10.1.2.0/24 [195/26882560] via 10.0.1.2, 00:06:46, Tunnel1 HQ-CORE1#show ip route D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:09:18, Vlan10

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91 DUAL WAN (MPLS + Internet) MPLS Failure • Failure within MPLS cloud

• Suboptimal routing at Branch • HSRP primary remains unchanged at BR-W1 • Use EOT and move HSRP primary to BR-W2

BR-W1#show ip route D 10.100.100.0/24 [90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1 D 10.100.200.0/24 [90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1 BR-W2#show ip route D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 01:08:44, Tunnel1 D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 01:08:45, Tunnel1 #CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 92 Agenda

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment

• Final Wrap Up • Key Takeaways

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93 Key Takeaways

• Outages can manifest in many different ways. Network design should be based on application requirements to survive various outages.

• Cisco IOS has inherent load sharing capabilities. Analyze your network topology and use these to your advantage.

• End-to-end convergence time is a critical metric. Understand how localized topology changes affect end-to-end resiliency.

• Multiple links/paths not only increase network reliability but can improve application performance.

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 94 Key Takeaways

• IP SLA based monitoring can detect outage types that are virtually undetectable by traditional “hello based” techniques.

• Performance Routing permits path selection based on current real time characteristics.

• Performance Routing permits full utilization of available bandwidth

• Most effective network designs incorporate a combination of convergence techniques

• Cisco SD-WAN is utilizing these features, while simplifying deployment and management, and increasing application availability.

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 95 Complete your online session evaluation

Give us your feedback to be entered into a Daily Survey Drawing. Complete your session surveys through the Cisco Live mobile app or on www.CiscoLive.com/us.

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at www.CiscoLive.com/Online.

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 96 Cisco Webex Teams

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Live Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKRST-2042 by the speaker until June 18, 2018.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 97 Continue your Demos in Walk-in Meet the Related the Cisco self-paced engineer sessions education campus labs 1:1 meetings

#CLUS BRKRST-2042 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 98 Thank you

#CLUS #CLUS