TECCRS-2500.Pdf

TECCRS-2500

Enhancements and Trends in Enterprise WAN Design and Deployments

Dave Fusik Arvind Durai

David Prall Craig Hill Speakers

Dave David Arvind Craig Fusik Prall Durai Hill CCIE#4768 CCIE#6508 CCIE#7016 CCIE#1628 CCDE#2013::70

• We value your feedback - don't forget to complete your online session evaluations after each session & complete the Overall Conference Evaluation which will be available online • Visit the World of Solutions • Please switch off your mobile phones • Please remember to wear your badge at all times

• WAN Architectures and Design Principles

• Highly Available WAN Design

• QoS for the WAN & Automation use case

• IP Multicast for the WAN

• Advancements for L3 Segmentation in the WAN

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Events Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space cs.co/ciscolivebot#TECCRS-2500

Session 8:30 am – 10:30 am Break 10:30 am – 10:45 am Session 10:45 am – 12:45 pm Lunch 12:45 pm – 14:30 pm Session 14:30 pm – 16:30 pm Break 16:30 pm – 16:45 pm Session 16:45 pm – 18:45 pm

WAN Architectures and Design Principles

Dave Fusik, Customer Solutions Architect CCIE 4768 (R&S/Security), CCDE 2013::70 Agenda • WAN Technologies & Solutions • Wide Area Network Overview and Principles • WAN Transport and Overlay Technologies • Cisco vBranch with Enterprise NFV • SD-WAN • Demonstration • WAN Extension into the Cloud

• WAN Architecture Design Considerations and Best Practices

• Summary

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 Wide Area Network Overview and Principles The WAN Technology Continuum Early Networking Early-Mid 1990s Mid 1990s-Late 2000s Today Global Scale Flat/Bridged Multiprotocol Large Scale IP Ubiquity Experimental Networks Business Enabling Mission Critical Business Survival

Architectural Architectural Architectural Planning Lessons Lessons Lessons Protocols required for Route first, bridge only Redundancy Scale & Restoration if you must ? Build to Scale

DMVPN X.25 Frame-Relay IPv6 Internet 4G/LTE 1960 Protocol 1980 BGP GRE 2000 Future

• Build a network that can adapt to • Adapt to business changes a quickly changing business and rapidly and smoothly technical environment • Mergers & divestures • Realize rapid strategic advantage • Changes in the regulatory & from new technologies security requirements • IPv6: global reachability • Changes in public perception of services • Cloud: flexible diversified resources • Internet of Things • Fast-IT • What’s next?

IP/MPLS Core Tier 1 Tier

In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region

Internet Cloud

Public Voice/Video Mobility Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

Use hierarchy to manage network scalability and complexity while reducing routing algorithm overhead

Hierarchical design used Hierarchical design has to be… become any design that… • Three routed layers • Splits the network up into • Core, aggregation, access “places” or “regions” • Only one hierarchical • Separates these “regions” by hiding information structure end-to-end • Organizes these “regions” around a network core • “hub and spoke” at a macro level

Single Provider Design Dual Providers Design Overlay Network Design Enterprise homes all sites into Enterprise will single or dual home Overlay tunneling technologies a single MPLS VPN carrier to sites into one or both carriers to with encryption for provider provide L3 connectivity provide L3 MPLS VPN connectivity transport agnostic design

• Pro: Simpler design with • Pro: Protects against MPLS service • Pro: Can use commodity consistent features failure with Single Provider broadband services for lower cost higher bandwidth service • Con: Bound to single carrier • Pro: Potential business leverage for feature velocity for better competitive pricing • Pro: Flexible overlay network topology that couples from the • Con: Does not protect • Con: Increased design complexity physical connectivity against MPLS cloud failure due to service implementation with Single Provider differences (e.g. QoS, BGP AS • Con: Increased design Topology) complexity

• Con: Feature differences between • Con: Additional technology providers could force customer to needed for SLA over commodity use least common denominator transport services features.

. Dual Homed Non Transit Only advertise local prefixes (^$) Typically with Dual CE routers BGP design: eBGP to carrier iBGP between CEs Redistribute cloud learned routes into the site IGP . Single Homed Non Transit Advertise local prefixes and optionally use default route.

. To guarantee single homed site reachability to a dual homed site experiencing a failure, transit sites had to be elected. . Transit sites would act as a BGP bridge transiting routes between the two provider clouds. . To minimize latency costs of transits, transits need to be selected with geographic diversity (e.g. from the East, West and Central US.)

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 Single vs. Dual Carriers Single Provider Dual Providers Pro: Common QoS support Pro: More fault domains model Pro: More product offerings to Pro: Only one carrier to “tune” business Pro: Reduced head end Pro: Ability to leverage circuits vendors for better pricing Pro: Nice to have a second Pro: Overall simpler design vendor option Con: Carrier failure could be Con: Increased Bandwidth catastrophic “Paying for bandwidth twice” Con: No leverage to negotiate Con: Increased overall design lower costs complexity Con: May be reduced to “common denominator” between carriers Simplicity vs. Resiliency TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 WAN Transport and Overlay Technologies MPLS L3VPN Topology Definition

. MPLS WAN is provided by a service provider . As seen by the enterprise network, every site is one IP “hop” away . Equivalent to a full mesh, or to a “hubless” hub-and-spoke

VRF VRF VRF VRF VRF VRF

. Virtualization at Layer 3 forwarding ! PE Router – Multiple VRFs ip vrf blue . Associates to Layer 3 interfaces on router/switch rd 65100:10 . Each VRF has its own route-target import 65100:10 route-target export 65100:10 Forwarding table (CEF) ip vrf yellow rd 65100:20 Routing process (RIP, OSPF, BGP) route-target import 65100:20 route-target export 65100:20 . VRF-Lite ! interface GigabitEthernet0/1.10 Hop-by-hop ip vrf forwarding blue interface GigabitEthernet0/1.20 . MPLS VPN ip vrf forwarding yellow Multi-hop

E-Line (Point-to-Point) E-LAN (Point-to-Multipoint) . Replaces legacy TDM circuits and . Offers point to multipoint Frame-Relay/ATM virtual circuits connectivity (VCs) . Transparent to VLANs and Layer 2 . Point-to-point Ethernet VCs (EVCs) control protocols offer predictable performance for applications . 4 or 6 classes of QoS support . One or more EVCs allowed per single . Supports service multiplexing (Ex. physical interface (UNI) Internet access and corporate VPN via one UNI) . Supports “hub & spoke” topology

• MPLS Layer 3 Service • MetroE Layer 2 Service • Routing protocol dependent on • Flexibility of routing protocol and the carrier network topology independent of the carrier • Layer 3 capability depends on carrier offering • Customer manages layer 3 QoS • QoS (4 classes/6 classes) • Capable of transport IP and non- • IPv6 capability IP traffic. • Transport IP protocol only • Routing protocol determines • Highly scalable and ideal for large scalability in point-to-multipoint network topology

Layer 2 Overlays Layer 3 Overlays . Layer 2 Tunneling Protocol—Version 3 . IPSec—Encapsulating Security Payload (L2TPv3) (ESP) – Layer 2 payloads (Ethernet, Serial,…) – Strong encryption – Pseudowire capable – IP Unicast only . Other L2 overlay technologies – . Generic Routing Encapsulation (GRE) OTV, VxLAN – IP Unicast, Multicast, Broadcast – Multiprotocol support . Other L3 overlay technologies – MPLSomGRE, LISP, OTP TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Tunnelling GRE and IPSec Transport and Tunnel Modes IP HDR IP Payload

GRE packet with new IP header: Protocol 47 (forwarded using new IP dst) IP HDR GRE IP HDR IP Payload

20 bytes 4 bytes

IPSec Transport mode 2 bytes ESP ESP IP HDR ESP HDR IP Payload Trailer Auth 20 bytes 30 bytes Encrypted AuthenticatedAuthenticated IPSec Tunnel mode 2 bytes ESP ESP IP HDR ESP HDR IP HDR IP Payload Trailer Auth 20 bytes 54 bytes Encrypted Authenticated TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Cisco Site to Site VPN Technologies Comparison Features DMVPN FlexVPN GET VPN . Public or Private Transport . Public or Private Transport . Private IP Transport . Overlay Routing Infrastructure Network . Overlay Routing . Flat/Non-Overlay IP Routing . IPv4/IPv6 dual Stack . Large Scale Hub and Spoke . Converged Site to Site and . Any-to-Any; Network Style with dynamic Any-to-Any Remote Access (Site-to-Site)

. Dynamic Routing or IKEv2 . Active/Active based on . Transport Routing Route Distribution Failover Redundancy Dynamic Routing . COOP Based on GDOI . Server Clustering

. Unlimited . Unlimited . 8000 GM total Scalability . 3000+ Client/Srv . 3000+ Client/Srv . 4000 GM/KS

. Multicast replication in IP WAN . Multicast replication at hub . Multicast replication at hub IP Multicast network

. Per SA QoS, Hub to Spoke . Per Tunnel QoS, Hub to Spoke . Transport QoS QoS . Per SA QoS, Spoke to Spoke . Centralized Policy . Locally Managed . Central or Local Management Policy Control Management . Tunneled VPN . Tunneled VPN . Tunnel-less VPN Technology . Multi-Point GRE Tunnel . Point to Point Tunnels . Group Protection . IKEv1 & IKEv2 . IKEv2 Only . IKEv1 & IKEv2

. Branch sites establish an IPsec tunnel to, and SECURE ON-DEMAND TUNNELS register with, the hub site

. IP routing exchanges prefix information for each site ASR 1000 Hub . BGP or EIGRP are typically used for scalability IPsec Branch n . The WAN interface IP is the tunnel source address, VPN ISR so the provider network does see the customer IP

prefixes ISR ISR Branch 1 . Data traffic flows over the DMVPN tunnels Branch 2 . When traffic flows between spoke sites, the hub assists the spokes to establish a site-to-site tunnel Traditional Static Tunnels DMVPN On-Demand Tunnels . Per-tunnel QOS is applied to prevent hub site Static Known IP Addresses oversubscription to spoke sites Dynamic Unknown IP Addresses

Typical Cisco FlexVPN Deployment . Created to simplify the deployment of VPNs . Provides a unified ecosystem to cover all types of VPN: Remote Access, Teleworker, Site-to-site, Mobility, Managed security services, and others . A single FlexVPN deployment can accept multiple types of connections at the same time . Provides compatibility with any IKEv2-based third-party VPN vendors, including native VPN clients from Apple iOS and Android devices . Deployed over public or private transport . VPN dynamic policies (i.e. split-tunnel policy, encryption . Standards-based encryption technology policy, VRF selection, DNS server for remote access) . Highly secure parameters by default can be fully integrated with the AAA/RADIUS and . Superior hierarchical QoS per SA applied on a per peer basis . Hub Multicast, or transport, replication

. Uses Group Domain of Interest (GDOI – RFC 6407) to Tunnel-Less VPN over Private WAN distribute common IPsec keys to a group of VPN gateway devices . Key Servers (KSs) create and maintain the GETVPN control plane, centrally defining encryption policies that are pushed to IKE authenticated Group Members (GMs) WAN at the time of registration . GMs handle the encryption/decryption (i.e. the data plane) based on the downloaded, or local, policy Multicast . GETVPN preserves the original unicast or multicast . Scalable architecture for any-to- source and destination packet addresses which any connectivity and encryption provides the ability to route encrypted packets using . No overlays—native routing the underlying network routing infrastructure . Any-to-any instant connectivity . Cooperative KSs provides highly available control plane . Enhanced QoS . Efficient Multicast replication TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Link Speeds Out-Pacing IP Encryption

• Bandwidth application requirements out- pacing IP encryption capabilities

• Bi-directional and packet sizes further impact encryption performance

• IPSec engines dictate aggregate performance of the platform (much link lower throughput)

BW • Cost per bit for IPSec much more Link speed = Encryption speed expensive

time • Encryption must align with link speed (100G+) to support next-generation applications Link Speed IPSec Encryption Speed TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 MACSec – Line Rate L2 Encryption Solution Authenticated Encrypted 6 Bytes 6 Bytes 8-16 Bytes 4 Bytes 8 Bytes 2 Bytes 8-16 B 4 Bytes DMA SMAC 802.1AE Header 802.1Q CMD ETYPE PAYLOAD ICV CRC C 0x88e5 Packet SCI MACsec EtherType TCI/AN SL Number (optional) MACsec Tag Format ✓ Frames are encrypted and protected with Advanced Encryption Standard Galios/Counter Mode (AES-GCM-128)

✓ Line Rate Encrypted Ethernet performance of the port (PHY). Speeds 1/10G, 40G, 100G

✓ MACsec Ethertype is 0x88e5

✓ No impact to IP MTU/Fragmentation

✓ Reduced interoperability issues with other L3 Features

. Data Center Interconnect . Typically seen with utility . Clear-Tag Feature: Option . Point-to-point link dark company for 802.1Q tag in clear for fiber or DWDM  Metro-Ethernet connection . Point-to-point link dark fiber or DWDM  deployment . Provide line rate connection . Option to modify EAPoL encryption with high . Connecting utility stations destination MAC and speed links between DCs EtherType to avoid MKA for replication traffic together and provide link encryption packet being consumed

macsec

key 01

key-string 1234567890123456789012345678901234567890123456789012345678901234 cryptographic-algorithm aes-256-cmac

lifetime 00:00:00 january 01 2018 infinite

! macsec-policy ACME_100G

security-policy must-secure

window-size 128

cipher-suite GCM-AES-XPN-256

key-server-priority 0

! interface HundredGigE0/3/0/0

macsec psk-keychain DC1-to-DC2 policy ACME_100G

Service Provider Owned Routers/Bridges Data Data Center Public Carrier Center Ethernet Service Remote Central Campus/DC Campus/DC • Leverage Metro-Ethernet transport • Optimize MACsec + WAN features to accommodate running MACsec MKA Session over public Ethernet transport MACsec Secured Path / MKA Session • Target “line-rate” encryption for high-speed applications MACsec Capable Router

• Inter DC, MPLS WAN links, massive data projects MACsec Capable PHY

SP Owned Ethernet • Support 1/10/40/100G link speed Transport Device

Router order

WAN

Router order Line install Router delivery Router install Online router

Service 1

Appliance WAN

Service order Appliance delivery Appliance install Online appliance

Service 2 Appliance

Appliance WAN

Cisco DNA Center (DNAC) Cisco Network Service Orchestrator (NSO) / Virtual Managed Services (VMS)

Virtual WAN Virtual Wireless Virtual Router Virtual Firewall Optimization LAN Controller 3rd Party VNFs (ISRv/vEdge) (ASAv) (vWAAS) (vWLC)

Network Functions Virtualization Infrastructure Software (NFVIS)

ISR 4000 + Enterprise Network UCS C-Series UCS E-Series Compute System

Traditional Enterprise NFV

Physical Router Physical Router Virtual Router Ne w ! Virtual Router Virtual Services Virtual Services Virtual Services

4000 Series ISR + Enterprise Network Cisco® 4000 Series ISR UCS® E-Series Compute System (ENCS) UCS C-Series, COTS

Centralized services Upgradable hardware Elastic routing and services Elastic routing and services Fixed integrated services Deterministic routing Performance Router / Server Hybrid Conservative performance Early adopter

Cisco ONE™ Access to Ongoing License Investment Innovation Portability Protection

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 Branch/Campus Platform Built for Enterprise NFV Colocation Center ENCS 5000 Series for the Branch Public Cloud Best of Routing Complete Open for Third Party & Compute Virtualized Services Services and Apps

Enterprise Network Compute System

ENCS 5100 Series

8 Integrated LAN Ports ENCS 5400 Series with Optional POE USB 3.0 Storage 2 Onboard Gigabit Network Interface Hardware Acceleration for Ethernet ports Module for LTE & 2 HDD or SSD VM Traffic with SFP legacy WAN RAID 0 & 1

ISRv/vEdge ASAv/FTD* vWAAS vWLC Full DC-Class Application High Performance Built for small and Featured Optimization and medium branches Rich Features Functionality Akamai Connect

Windows Server Linux 3rd Party Active Directory, File Network Services Custom Applications Share, Server Management & DNS/DHCP Applications Monitoring

Branch router

IPS/IDS appliance

WAAS appliance Patch panel N F V I S

Firewall appliance N F V I S

A single x86 compute platform housing multiple VNFs

Application Traffic Per-Segment Secure Cloud Cloud Transport SLA Engineering Topologies Perimeter Path Accel Hub

Analytics Application Policies

Routing Security Segmentation QoS Multicast Svc Insertion Survivability Monitoring

Delivery Platform Operations

Broadband MPLS Cellular

Transport Independent Fabric

Orchestration Plane vOrchestrator

vBond MANAGEMENT

Management Plane API (Multi-tenant or Dedicated) ORCHESTRATION

vManage ANALYTICS Control Plane CONTROL (Containers or VMs)

INTERNET MPLS 4G vSmart

Data Plane (Physical or Virtual)

Data Center Campus Branch Home Office vEdge

APIs . WAN edge router . Leverages traditional routing 3rd Party vAnalytics protocols like OSPF, BGP and Automation VRRP vBond . Provides secure data plane with remote vEdge routers vSmart Controllers . Establishes secure control plane with vSmart controllers (OMP) MPLS 4G . Apply application aware routing INET policies vEdge Routers . Support Zero Touch Deployment . Physical or Virtual form factor Cloud Data Center Campus Branch SOHO (VNF) support

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Cisco SD-WAN Solution Elements Control Plane Control Plane vManage Cisco vSmart APIs 3rd Party . Centralized brain of the operation vAnalytics Automation . Facilitates fabric discovery . Dissimilates control plane vBond information between vEdges vSmart Controllers . Distributes data plane and app- aware routing policies to the vEdge routers MPLS 4G . Implements control plane policies INET vEdge Routers such as o Traffic Engineering o Service Chaining, o Multi-topology (hub/spoke, Cloud Data Center Campus Branch SOHO partial, or full mesh)

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 46 Cisco SD-WAN Solution Elements Management Plane Management Plane vManage Cisco vManage APIs . Single pane of glass for 3rd Party vAnalytics Automation Day0, Day1 and Day2 operations vBond . Multitenant with web scale . Centralized provisioning vSmart Controllers . Policies and Templates . Troubleshooting and MPLS 4G Monitoring INET vEdge Routers . Software upgrades . GUI with RBAC . Programmatic interfaces Cloud Data Center Campus Branch SOHO (REST, NETCONF) . Highly resilient

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 Cisco SD-WAN Solution Elements Orchestration Plane Orchestration Plane vManage Cisco vBond APIs 3rd Party vAnalytics . Orchestrates connectivity Automation . First point of authentication vBond (white-list model) . Distributes list of vSmarts/ vSmart Controllers vManage to all vEdge routers . Requires public IP Address MPLS 4G [could be behind 1:1 NAT] INET . Facilitates NAT traversal vEdge Routers . Highly resilient

Cloud Data Center Campus Branch SOHO

SDWAN and Services Core SD-WAN

ISR 1000 ISR 4000 ASR 1000 vEdge 100 vEdge 1000 vEdge 2000

• 100 Mbps • Up to 1 Gbps • 10 Gbps • 200 Mbps • Up to 2 Gbps • 2.5-200Gbps • 4G LTE & Wireless • Fixed • Modular • Next-gen • Modular • High-performance connectivity • Integrated service service w/hardware • Performance assist flexibility containers • Compute with UCS E • Hardware & software redundancy

Virtualization Public Cloud ENCS 5100 ENCS 5400

• Up to 250Mbps • 250Mbps – 2GB

• Overlay Management Protocol – Control plane protocol distributing reachability, security and policies throughout the fabric

• Transport Locator (TLOC) – Transport attachment point and next hop route attribute

• Color – Control plane tag used for IPSec tunnel establishment logic

• Site ID – Unique per-site numeric identifier used in policy application

• System IP – Unique per-device (WAN Edge and controllers) IPv4 notation identifier. Also used as Router ID for BGP and OSPF.

• Organization Name – Overlay identifier common to all elements of the fabric

• VPN – Device-level and network-level segmentation.

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 50 Overlay Management Protocol (OMP) Unified Control Plane . TCP based TLS/DTLS control plane protocol vSmart . Runs between vEdge routers and vSmart controllers and between the vSmart controllers . Advertises all control plane info (i.e. TLOCs, unicast/multicast destinations, L4-L7 service routes, BFD stats, etc.) . Enables edge-to-edge IPSec without reliance on

vSmart vSmart IKE or tunneling of traditional routing protocols . Lowers control plane complexity allowing high scalability

VS vEdge vEdge

Note: vEdge routers need not connect to all vSmart Controllers

vManage

NETCONF/YANG

Centralized Control Policy (Fabric Routing) Local Control Policy (OSPF/BGP) Centralized Data Policy Centralized Localized (Fabric Data Plane) Policies Policies Local Data Policy (QoS/Mirror/ACL) Centralized App-Aware Policy (Application SLA)

OMP

Centralized Data Policy Centralized App-Aware Policy vSmart (Fabric Data Plane) (Application SLA) WAN Edge

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 Fabric Operation Walk-ThroughOMP Update: vSmart OMP . Reachability – IP Subnets, TLOCs . Security – Encryption Keys DTLS/TLS Tunnel . Policy – Data/App-route Policies IPSec Tunnel OMP OMP BFD Update Update Policies OMP OMP Update Update

vEdge Transport 1 vEdge

TLOCs TLOCs

VPN1 VPN2 Transport 2 VPN1 VPN2 BGP, OSPF, BGP, OSPF, Connected, Connected, Static A B C D Static Subnets Subnets

VPN 1 Interface VPN1 SD-WAN VPN1 Interface IPSec VPN 2 VLAN VPN2 VPN2 VLAN Tunnel VPN 3 Ingress Egress WAN Edge WAN Edge

IP UDP ESP VPN Data 20 8 36 4 …

• Segment connectivity across fabric w/o • Labels are used to identify VPN for reliance on underlay transport destination route lookup • WAN Edge routers maintain per-VPN • Interfaces and sub-interfaces (802.1Q routing table tags) are mapped into VPNs

• Each VPN can have it’s own topology - Full-mesh, hub-and-spoke, partial- VPN1 VPN2 mesh, point-to-point, etc… • VPN topology can be influenced by leveraging control policies - Filtering TLOCs or modifying next-hop TLOC attribute for OMP routes Partial Mesh Point-to-Point • Applications can benefit from shortest path, e.g. voice takes full-mesh topology

VPN3 VPN4 • Security compliance can benefit from controlled connectivity topology, e.g. PCI data takes hub-and-spoke topology

Single-touch centralized vSmart security policy Controllers - Access Control List App - Application Firewalling Policies

ACL/ TransportsTransports ACL/ App App Transports

User Site Data Center Server vEdge vEdge

. Strong security posture - Regionalized stateful network Regional DC/Colo services vEdge . Multiple network services - Service chaining

Network Service Nodes Data traffic Control Plane TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 56 Use Cases & Deployments Supporting a diverse set of topologies and architectures at scale

Technology Use Cases – M&A, Line-of-business Fully Managed WAN With Centralized Control separation, Partner network Segmentation & Multi-Topology Data Center Virtual Fabric Data Center Enterprise NOC & NAC & MDM Access DC User Control

VPN1

t i MPLS S CoLo VPN2

Video WAN CoLo & DMZ VPN2 Video

Internet

VPN1 WAN Opt &

t caching i User Traffic Branch Public Cloud S User & Network Video Traffic routing & switching Services Viptela vEdge

• Independent and isolated virtual topologies operating at the same time Enterprise Unified Wireless Communications

14 Viptela Confidential 12 Viptela Confidential

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 vBranch and SDWAN Demonstration SD-WAN SD-WAN SD-WAN Site2 Site3 Site4 G0/2 G0/2 G1/0 11.2.0.21 11.3.0.31 11.4.0.41 G0/1 G0/0 encs5412-1 rtr2-1 rtr3-1 G0/1 G0/0 G0/0 DHCP: 100.66.101.50 pnpserver.cselab.com 192.168.99.10 Demo Topology DNS Server Transport1 Transport2

G0/0 G0/1 G0/0 G0/1 Data Center 11.1.0.11 11.1.0.12

V900 V99 .2 .1 rtr1-1 rtr1-2 G0/2 G0/2 172.16.1.0/24 192.168.99.0/24

SD-WAN vEdge authenticates Claim and Provision .20 Site1 using OTP and comes .112 .110 .111 .10 up in vManage .112 View point Windows vManage vBond vSmart Cisco DNAC jump host vSmart2 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public WAN Extension into the Cloud Cloud Connectivity Challenges

• Complexity & Dependency – Need a simple and scalable way to securely extend the private network across Multicloud environments Public Cloud

• Inconsistent security policies between private & public- Need to apply consistent security policies Users On-Prem Datacenters • Performance and ambiguity for best path to reach the cloud – Need to enhance application Applications experience Remote Branches

Software Performance Elasticity Same IOS XE software as the Available licenses range from ASR1000 and ISR4000 10 Mbps to 10 Gbps CSR 1000V App App CPU footprint ranges from Infrastructure Agnostic 1vCPU to 8vCPU Runs on x86 platforms OS OS Supported Hypervisors: Programmability Virtual Switch VMware ESXi, RHEL Linux KVM, NetConf/Yang, RESTConf, Guest Suse Linux KVM, Citrix Xen, Hypervisor Shell and SSH/Telnet Microsoft Hyper-V, Cisco NFVIS and CSP5000 Server License Options Supported Cloud Platforms: Term based 1 year, 3 year Amazon Web Services, Microsoft or 5 year Azure, Google Cloud Platform

Enterprise-class networking with rapid deployment and flexibility

Securely extend the private network VPC to the cloud from the Branch and DC VPC with CSR 1000v VPC Extend routing to multi-VPC environment with CSR 1000v VPC CSR1000v in Transit VPC VPC

CSR1000v Maintain application experience CSR1000v with QoS and AVC

ISR 4000 ASR 1000

Branch Enterprise DC

Application VPC Gateway Transit VPC Auto-scale • CSR deployed in application VPC • CSR deployed in dedicated Transit Hub, • Add another pair of CSRs to scale out • Provide IPsec gateway for entire VPC not in application VPC • Remote end (VGW) has multiple tunnels • Need high availability • High speed traffic routing for spoke and do L3 ECMP (Equal Cost Multiple VPC Path) • High availability is built-in natively • Monitors CSR real-time throughput and spin up new CSRs on demand

Corporate DC

AWS Managed VPN

Cisco Internet ISR/ASR

VGW

VLAN A VLAN B CSR 1000V VLAN C Corporate DC AWS Direct Customer Connect POP Cage

Cisco Private VIF Colocation Facility ISR/ASR CSR 1000V

SaaS IaaS Extranet SaaS IaaS Extranet SaaS IaaS Extranet

Data Center Data Center Data Center Data Center Data Center Data Center Network Hub or SAE

MPLS MPLS Internet

MPLS Internet

Cloud

Customers Network Hub or SAE

Colocation Centers

Employees

Security Agility & Performance Cost Savings Partners DMZ Central policy Rapid provisioning, Lower OpEx and Private Applications enforcement change control, scaling CapEx through NFV. Data Center via NFV fabric - Speed of Reduce circuit costs software with the and number of performance of hardware circuits.

Campus/ Data Data Center Center/ Campus WAAS Service

WAN Key Services/ Server Distribution s

VPN Termination

WAN Edge MPLS A MPLS B

Internet

Campus/ Core Layer Data Center

WAN Distribution Layer EIGRP AS 100

Summaries+ Default

DMVPN Hub Routers EIGRP AS = 100 EIGRP AS = 100 EIGRP AS = 100 iBGP Internet Edge

BGP AS = 65511 EIGRP AS = 200 MPLS CE BGP AS = 65511 Layer 2 WAN Routers CE Router

eBGP Layer 2 DMVPN 1 DMVPN 2 Internet MPLS A MPLS B WAN

WAN WAN WAN

. No Static Routes . No First Hop Redundancy Protocols TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 72 Optimize Convergence and Redundancy Multi-chassis EtherChannel

VSS/3850 Stacks

Si Layer 3 P-to-P Link Channel Member Removed IGP recalc

. Link redundancy achieved through . Provide Link Redundancy and reduce redundant L3 paths peering complexity . Flow based load-balancing through . Tune L3/L4 load-balancing CEF forwarding across hash to achieve maximum utilization . Routing protocol reconvergence . No L3 reconvergence required when when uplink failed member link failed . Convergence time may depends on . No individual flow can go faster than routing protocol used and the size of the speed of an individual member of routing entries the link

. ECMP convergence is dependent on the number of Si Layer 3 routes P-to-P Link

. MEC convergence is consistent, independent of the number of routes

2.5

2 ECMP MEC Max 1.5 VSS/3850 Stacks

0.5 sec sec of lost voice

0 1000 3000 6000 9000 12000 NumberNumber of Routes of Routes - Sup720C

. In principle, redundancy is easy . Any system with more parallel paths through the system will fail less often . The problem is a network isn’t really a single system but a group 2.5 of interacting systems . Increasing parallel paths increases routing complexity, therefore increasing convergence times Seconds

0 Routes 10000

. It is important to force summarization Campus/ Summary at the distribution towards WAN Edge Data Center 10.5.0.0/16 and towards campus & data center . Summarization provides topology change isolation. Summaries + . Summarization reduce routing table Default size. 10.4.0.0/16 0.0.0.0/0.0.0.0 interface Port-channel1 description Interface to MPLS-A-CE no switchport ip address 10.4.128.1 255.255.255.252 ip pim sparse-mode ip summary-address eigrp 100 10.5.0.0 255.255.0.0

MPLS A MPLS B

. Mutual route redistribution between protocols can cause routing loops without preventative measures IGP Domain . Use route-map to set tags and then redistribute (EIGRP/OSPF) based on the tags . Routes are implicitly tagged when distributed from Campus eBGP to EIGRP/OSPF with carrier AS . Use route-map to block re-learning of WAN routes via the distribution layer (already known via iBGP)

router eigrp 100 distribute-list route-map BLOCK-TAGGED-ROUTES in default-metric [BW] 100 255 1 1500 MPLS WAN redistribute bgp 65500

route-map BLOCK-TAGGED-ROUTES deny 10 BGP Domain match tag 65401 65402

route-map BLOCK-TAGGED-ROUTES permit 20

. Run iBGP between the CE routers to exchange prefixes associated with each Campus carrier . CE routers will use only BGP path selection information to select both the primary and secondary preferences for any destinations 10.5.128.0/21 announced by the IGP and BGP iBGP . Use IGP (OSPF/EIGRP) for prefix re- advertisement will result in equal-cost paths at remote-site MPLS A MPLS B bn-br200-3945-1# sh ip bgp 10.5.128.0/21 BGP routing table entry for 10.5.128.0/21, version 71 Paths: (2 available, best #2, table default, RIB-failure(17)) Not advertised to any peer 65401 65402, (aggregated by 65511 10.5.128.254) 10.4.142.26 from 10.4.142.26 (192.168.100.3) Origin IGP, localpref 100, valid, external, atomic- A B aggregate 65402, (aggregated by 65511 10.5.128.254) iBGP 10.4.143.26 (metric 51456) from 10.5.0.10 (10.5.0.253) 10.5.128.0/21 Origin IGP, metric 0, localpref 100, valid, internal, atomic-aggregate, best © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public Best Practice - Implement AS-Path Filter Prevent Branch Site Becoming Transit Network

. Dual carrier sites can unintentionally become Campus transit network during network failure event and causing network congestion due to transit traffic . Design the network so that transit path between two carriers only occurs at sites with enough bandwidth . Implement AS-Path filter to allow only locally originated routes to be advertised on the outbound updates for branches that should not MPLS A MPLS B be transit router bgp 65511 neighbor 10.4.142.26 route-map NO-TRANSIT-AS out ! ip as-path access-list 10 permit ^$ A B ! route-map NO-TRANSIT-AS permit 10 iBGP match as-path 10

• EIGRP • OSPF • Internal EIGRP – Admin Dist. 90 • Admin Dist. 110 • External EIGRP – Admin Dist. 170 • Route Preference • Metric Calculation • Intra-Area • metric = bandwidth + delay • Inter-Area • Bandwidth (in kb/s) • External E1 (Internal + External Cost) • Delay (in microseconds) • External E2 (External Cost) • Cost Calculation • Cost= Reference BW / Interface BW • Default Reference BW = 100Mbps

Campus . eBGP routes are redistributed into EIGRP 100 as EIGRP external routes with default Admin Distance 170 AS100

10.4.128.2 . Running same EIGRP AS for both campus and DMVPN network would result in Internet path

eBGP preferred over MPLS path

MPLS A Internet

EIGRP AS100

Campus D EX 10.5.48.0/21 [170/28416] via 10.4.128.2 . eBGP routes are redistributed into EIGRP 100 as external routes with default Admin Distance 170 EIGRP AS100 . Running same EIGRP AS for both campus and DMVPN 10.4.128.2 network would result in Internet path preferred over

MPLS path eBGP . Multiple EIGRP AS processes can be used to provide control of the routing . EIGRP 100 is used in campus location EIGRP 200 over DMVPN tunnels Internet MPLS A . Routes from EIGRP 200 redistributed into EIGRP 100 appear as external route (distance = 170)

EIGRP . Routes from both WAN sources are equal-cost paths. AS200 To prefer MPLS path over DMVPN use eigrp delay to modify path preference

MPLS CE router#

. eBGP as the PE-CE Routing Protocol Campus

. MPLS VPN as preferred path learned via EIGRP AS100 eBGP . Secondary path via backdoor IGP link R1 R2 (EIGRP or OSPF) over tunneled connection eBGP (DMVPN over Internet)

. Default configuration the failover to MPLS A Internet backup path works as expected

10.4.160.0/24

. eBGP as the PE-CE Routing Protocol Campus

. MPLS VPN as preferred path learned via EIGRP AS100 eBGP

. Secondary path via backdoor IGP link R1 R2 eBGP (EIGRP or OSPF) over tunneled connection IGP Backup Link (DMVPN over Internet)

. Default configuration the failover to MPLS A Internet backup path works as expected

10.4.160.0/24

. eBGP as the PE-CE Routing Protocol Campus

. MPLS VPN as preferred path learned via EIGRP AS100 eBGP

. Secondary path via backdoor IGP link R1 R2 eBGP (EIGRP or OSPF) over tunneled connection IGP Backup Link (DMVPN over Internet)

. Default configuration the failover to MPLS A Internet backup path works as expected RT: del 10.4.160.0 via 10.4.142.2, bgp metric [20/0] RT: delete route to 10.4.160.0/24 RT(multicast): delete subnet route to 10.4.160.0/24 %BGP-5-ADJCHANGE: neighbor 10.4.142.2 Down %BGP_SESSION-5-ADJCHANGE: neighbor 10.4.142.2 IPv4 Unicast topology base removed from session BGP Notification sent

RT: updating eigrp 10.4.160.0/24 (0x0): via 10.4.128.9 Po1 10.4.160.0/24 RT: add 10.4.160.0/24 via 10.4.128.9, eigrp metric [170/3584]

Campus . After link restore, MPLS CE router receives EIGRP BGP advertisement for remote-site route. AS100 . Does BGP route get (re)installed in the

route table? R1 R2

eBGP IGP IGP Backup Link

D EX 10.4.160.0/24 [170/3584]....

MPLS A Internet

R1# show ip route B 10.4.144.0/24 [20/0] via 10.4.142.2, 01:30:06 B 10.4.145.0/24 [20/0] via 10.4.142.2, 01:30:06 D EX 10.4.160.0/24 [170/3584] via 10.4.128.9, 00:30:06

B 10.4.160.0/24 [20/0].... 10.4.160.0/24

• BGP Prefers Path with: • Lowest Multi-Exit Discriminator (MED) • Highest Weight • Prefer Externals (eBGP over iBGP • Highest Local Preference paths) • Locally originated (via network or • Lowest IGP metric to BGP next aggregate BGP) hop (exit point) • Shortest AS_PATH • Lowest Router ID for exit point • Lowest Origin type IGP>EGP>INCOMPLETE (redistributed into BGP)

• Routes redistributed into BGP are considered locally originated and get a default weight of 32768

• The eBGP learned prefix has default weight of 0

• Path with highest weight is selected

ASR1004-1#show ip bgp 10.4.160.0 255.255.255.0 BGP routing table entry for 10.4.160.0/24, version 22 Paths: (3 available, best #3, table default) Advertised to update-groups: 4 5 65401 65401 10.4.142.2 from 10.4.142.2 (192.168.100.3) Origin IGP, localpref 200, valid, external Local 10.4.128.1 from 0.0.0.0 (10.4.142.1) Origin incomplete, metric 26883072, localpref 100, weight 32768, valid, sourced, best

• To resolve this issue set the weights on route learned via eBGP peer higher than 32768 neighbor 10.4.142.2 weight 35000

ASR1004-1#show ip bgp 10.4.160.0 255.255.255.0 BGP routing table entry for 10.4.160.0/24, version 22 Paths: (1 available, best #1, table default) Not advertised to any peer 65401 65401 10.4.142.2 from 10.4.142.2 (192.168.100.3) Origin IGP, metric 0, localpref 100, weight 35000, valid, external, best

ASR1004-1#show ip route .... B 10.4.160.0/24 [20/0] via 10.4.142.2, 05:00:06

IP/MPLS Core Tier 1 Tier

In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region

Internet Cloud

Public Voice/Video Mobility Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

• Understand how WAN characteristics can affect your applications • Bandwidth, latency, loss

• A modular hierarchical network infrastructure is the foundation for a solid WAN architecture

• Encryption is a foundation component of all WAN designs and can be deployed transparently

• Understand how to build wide area network leveraging Internet transport with SD-WAN

• Design a network with consistent behavior that provides predictable performance

• More is not always better - Keep it simple!

Abstract: Virtual Routing in the Cloud are key enablers of today’s revolutionary shift to elastic cloud applications and low-cost virtualized networking. The book covers every essential building block, present key use cases and configuration examples, illuminate design and deployment scenarios, and show how the CSR 1000V platform and APIs can enable state-of-the-art software-defined networks (SDN). Drawing on extensive early adopter experience, they illuminate crucial OS and hypervisor details, help you overcome migration challenges, and offer practical guidance for monitoring and operations. http://bit.ly/2l8UAod

Highly Available Wide Area Network Design

David Prall, Principal Systems Engineer CCIE #6508 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment

• Final Wrap Up

• Dynamically respond to all types of disruptions

• Leverage most effective design techniques that meet the design requirements

• Review today’s technology

Link or Device Failure

MPLS - SP A

C-A-R2 Link or Device Degraded

C-A-R1 C-A-R4

C-A-R3

HQ-W1 BR-W1

MPLS - SP B HQ-W2 BR-W2

C-B-R1 C-B-R4

• How does outage manifest? • How quickly can network detect? • How long is bidirectional reconvergence? TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 97 Session Scope • What methods are used for path selection and packet forwarding

• How does the network detect outages

• Focus on network survivability and effective utilization rather than sub-second convergence

• Modern Design using SD-WAN

• Does not address “zero loss” considerations • Please review BRKRST-2365 Unified HA Network Design - The Evolution of the Next Generation Network • Other sessions delivered by Matt Birkner

• System Availability: a ratio of the 98.000000% 7.3 Days expected uptime to the experienced 99.000000% 3.65 Days downtime over a period of time of 99.500000% 1.825 Days the same duration 99.900000% 8.76 Hrs 99.990000% 52.56 Min • Branch WAN High Availability: Branch WAN Between 99.99% and 99.999% 99.999000% 5.256 Min HA Targets 99.999900% 31.536 Sec • Ultra High Availability: Between 99.999990% 3.1536 Sec Ultra HA 99.9999% and 99.999999% 99.999999% .31536 Sec Targets

Downtime SINGLE per Year 99.95%* Downtime Downtime 99.90%* per Year per Year ROUTER, MPLS 4 Hours 8 Hours Internet SINGLE PATH 4–9 Hours 22 Minutes 46 Minutes ISR ISR Branch WAN HA Solution 99.995% 99.995% 99.995% SINGLE ROUTER, 26+ Minutes DUAL PATHS MPLS MPLS MPLS Internet Internet Internet

ISR ISR ISR

99.999% 99.999% 99.999%

DUAL ROUTERS, 5+ Minutes Internet Internet DUAL PATHS MPLS MPLS MPLS Internet

ISR ISR ISR ISR ISR ISR

* Typical MPLS and Business Grade Broadband Availability SLAs and Downtime per Year, calculated with Cisco AS DAAP tool. TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 100 Deployment Options

MPLS/MPLS MPLS/Internet MPLS/LTE Internet/Internet Internet/LTE MPLS MPLS Internet Internet LTE/LTE LTE LTE 100’s of Combinations

• Cisco IOS and IP Routing • Multiple Links/Multiple Paths • Load Sharing

• Convergence Techniques

• Design and Deployment

• Final Wrap Up

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 102 Routing Table Basics Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override, p - overrides from PfR

p 10.0.0.0/8 is variably subnetted, 14 subnets, 5 masks B p 10.0.0.0/8 [20/0] via 172.16.0.6, 00:12:36 B p 10.3.0.0/16 [20/0] via 172.16.0.6, 00:12:36 B p 10.4.0.0/16 [200/0], 00:13:52, Null0 C p 10.4.0.41/32 is directly connected, Loopback0 D p 10.4.1.0/24 [90/307200] via 10.4.49.2, 00:14:32, Ethernet0/0 C p 10.4.49.0/30 is directly connected, Ethernet0/0 L p 10.4.49.1/32 is directly connected, Ethernet0/0 B p 10.9.0.0/16 [20/0] via 172.16.0.6, 00:12:36 100.0.0.0/8 is variably subnetted, 9 subnets, 2 masks B 100.64.0.0/24 [20/0] via 100.64.3.1, 00:13:43 C 100.64.3.0/24 is directly connected, Ethernet0/2 L 100.64.3.2/32 is directly connected, Ethernet0/2 172.16.0.0/16 is variably subnetted, 9 subnets, 2 masks B 172.16.0.0/31 [20/0] via 172.16.0.6, 00:12:36 C 172.16.0.6/31 is directly connected, Ethernet0/1 L 172.16.0.7/32 is directly connected, Ethernet0/1 TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 103 INFORMATIONAL Administrative Distance Default Route Source Distance • The distance command is used to configure Connected 0 a rating of the trustworthiness of a routing Interface information source, such as an individual Static Route 1 router or a group of routers EIGRP Summary 5 Route • Numerically, an administrative distance is a BGP External 20 positive integer from 1 to 255. In general, the (eBGP) higher the value, the lower the trust rating EIGRP Internal 90 OSPF 110 • An administrative distance of 255 means the IS-IS 115 routing information source cannot be trusted RIP 120 at all and should be ignored EIGRP External 170 BGP Internal 200 (iBGP) Unknown 255

• How is administrative distance used to 10.0.14.0/24 10.0.14.0/24 10.0.14.0/25 10.0.14.128/25 determine which route These Two Routes should be installed? Are Identical EIGRP Internal = 90 • Only identical routes OSPF = 110 are compared EIGRP Internal Installed • Identical prefixes with router#show ip route 10.0.14.0 255.255.255.0 Routing entry for 10.0.14.0/24 different prefix lengths Known via "eigrp 1", distance 90, metric 307200, type internal are not the same route Redistributing via eigrp 1 Last update from 10.0.121.2 on Ethernet0/1, 00:01:32 ago • Routing Descriptor Blocks: The route from the * 10.0.121.2, from 10.0.121.2, 00:01:32 ago, via Ethernet0/1 protocol with the lower Route metric is 307200, traffic share count is 1 Total delay is 2000 microseconds, minimum bandwidth is 10000 Kbit administrative distance Reliability 255/255, minimum MTU 1500 bytes is installed Loading 1/255, Hops 1

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 105 Route Selection OSPF EIGRP OSPF • What about longest prefix comparison? 10.0.14.0/24 10.0.14.0/24 10.0.14.0/25 • Only identical routes are 10.0.14.128/25 These Two Routes compared Are Identical • Identical prefixes with different prefix lengths are not the same route OSPF Installed • The route with the longest Longer Prefixes prefix is router#show ip route 10.0.14.0 255.255.255.0 longer-prefixes installed 10.0.0.0/8 is variably subnetted, 9 subnets, 3 masks D 10.0.14.0/24 [90/307200] via 10.0.121.2, 00:01:35, Ethernet0/1 O 10.0.14.0/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2 O 10.0.14.128/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2 More Specific OSPF Override EIGRP TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 106 Agenda • Introduction

• Cisco IOS and IP Routing • Multiple Links/Multiple Paths • Load Sharing

• Convergence Techniques

• Design and Deployment

• Final Wrap Up

• Assume the same routing process attempts to install two routes for the same destination in the RIB

• The routing process may allow the second route to be installed based on its own rules IGP OSPF IS-IS EIGRP Route Cost Must be equal to Must be equal to Must be less than the installed route installed route variance times the lowest cost installed route

Maximum Paths Must be fewer than maximum-paths configured under the routing process (default = 4, maximum = 32)

Note: BGP default value for maximum-paths = 1

Per-Destination Per-Packet1 Default behaviour of IOS Universal Requires “ip load-sharing per-packet” Algorithm “show cef state” interface configuration1 Per-flow using destination hash Per-packet using round-robin method Packets for a given source/destination Packets for a given source/destination session will take the same path session may take different paths More effective as the number of Ensures traffic is more evenly destinations increase distributed over multiple paths Ensures that traffic for a given session Potential for packets to arrive out of arrives in order sequence

1Not available in IOS-XE based images TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 109 Load Sharing router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Known via "eigrp 100", distance 170, metric 3072256, type external Redistributing via eigrp 100 Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0 Route metric is 3072256, traffic share count is 1 .... 192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1 Route metric is 3072256, traffic share count is 1 ....

The Traffic Share Count Is Critical to Understanding the Actual Load Sharing of Packets Using These Two Routes 3072256/3072256 = 1

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 110 Load Sharing – with EIGRP Variance router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Known via "eigrp 100", distance 170, metric 3072256, type external Redistributing via eigrp 100 Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0 Route metric is 1536128, traffic share count is 2 .... 192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1 Route metric is 3072256, traffic share count is 1 .... If the Lower Metric Is Less than the Second Metric, the Traffic Share Count Will Be Something Other than 1 (EIGRP with Variance Configured) 3072256/3072256 = 1 3072256/1536128 = 2 2x Faster Link Gets 2 Flows vs. 1 Flow TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 111 Load Sharing – with eBGP dmzlink-bw router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Only routes learned Known via "bgp 1", distance 20, metric 0 via eBGP Neighbors Tag 2, type external Last update from 10.0.122.2 00:00:16 ago Routing Descriptor Blocks: 10.0.122.2, from 10.0.122.2, 00:00:16 ago Route metric is 0, traffic share count is 1 .... * 10.0.121.2, from 10.0.121.2, 00:00:16 ago Route metric is 0, traffic share count is 2 .... router#show ip bgp 192.168.239.0 BGP routing table entry for 192.168.239.0/24, version 9 Paths: (2 available, best #2, table default) Multipath: eBGP .... 2x Faster Link Gets 2 Flows vs. 1 Flow 10.0.122.2 from 10.0.122.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, valid, external, multipath(oldest) DMZ-Link Bw 312 kbytes rx pathid: 0, tx pathid: 0 .... 10.0.121.2 from 10.0.121.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, valid, external, multipath, best DMZ-Link Bw 625 kbytes rx pathid: 0, tx pathid: 0x0

• Now that we have load balancing • Which exact path are the flows using • “show ip cef exact-route [src-port] [dest-port]”

#show ip cef exact-route 1.1.1.1 2.2.2.2 1.1.1.1 -> 2.2.2.2 =>IP adj out of GigabitEthernet1, addr 10.255.0.1

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

• Carrier-delay • If a link goes down and comes back up before the carrier delay timer expires, the down state is effectively filtered, and the rest of the software on the router is not aware that a link-down event occurred. • Imposes a default 2 second pause before processing interface events • Disabling carrier-delay speeds convergence upon interface events • Disabling carrier-delay can increase control-plane usage during repetitive interface events (flapping)

• Dampening • Imposes a logarithmic delay based on interface events • Coupled with carrier-delay, dampening protects the control-plane from repetitive events by increasing the delay before processing up events should the interface flap. #conf t (config-if)#interface GigabitEthernet1 (config-if)#carrier-delay 0 (config-if)#dampening (config-if)#end #show dampening interface 1 interface is configured with dampening. No interface is being suppressed. Features that are using interface dampening: IP Routing

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

Keepalive (B) Holdtime (B,E,I) Hello (E,I,O) Invalid (R) Dead (O) Flush (R) Update (R) Holddown (R)

BGP 60 180

EIGRP 5 (60) 15 (180) (< T1) IS-IS 10 (3.333) 30 (10) (DIS) OSPF 10 (30) 40 (120) (NBMA)

RIP/RIPv2 30 180 180 240

Note: Cisco Default Values

R1 R4

Recovery Times by Protocol Link Down Link Up Link Up Link Up Line Protocol Down Loss 100% Neighbor Down Loss ~5%

BGP ~ 1 s 180 180 Never

EIGRP ~ 1s 15 (180) 15 (180) Never (< T1) IS-IS ~ 1s 30 (10) 30 (10) Never (DIS) OSPF ~ 1s 40 (120) 40 (120) Never (NBMA) RIP/RIPv2 ~ 1s 240 240 Never Note: Using Cisco Default Values

R1 R4 BR-W1

R3 R4#show ip bgp vpnv4 vrf cisco neighbor BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link BGP version 4, remote router ID 192.168.201.10 BGP state = Established, up for 1d10h Last read 00:00:19, hold time is 180, keepalive interval is 60 seconds BR-W1# router bgp 65110 R4#show ip bgp vpnv4 vrf cisco neighbor neighbor 192.168.101.9 timers 7 21 BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link BGP version 4, remote router ID 192.168.201.10 BGP state = Established, up for 00:01:23 Last read 00:00:03, hold time is 21, keepalive interval is 7 seconds When Configuring the Holdtime Argument for a Value of Less than Twenty Seconds, the Following Warning Is Displayed: %Warning: A Hold Time of Less than 20 Seconds Increases the Chances of Peer Flapping TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 120 Bidirectional Forwarding Detection (BFD)

• Extremely lightweight hello protocol • IPv4, IPv6, MPLS, P2MP

• 10s of milliseconds (technically, microsecond resolution) forwarding plane failure detection mechanism.

• Single mechanism, common and standardized • Multiple modes: Async (echo/non-echo), Demand

• Independent of Routing Protocols

• Levels of security, to match conditions and needs

• Facilitates close alignment with hardware

• Link-layer detection misses some types of outages • e.g. Control Plane failure

• Control Plane failure detection is very conservative • 15-180 seconds in default configurations

• Link-layer failure detection is not consistent across media types • Less than 50ms on APS- protected SONET • A few seconds on Ethernet • Several seconds or more on WAN links

• Provides a measure of consistency across routing protocols

• Most current failure detection mechanisms are an order of magnitude too long for time-sensitive applications

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 122 Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection interface GigabitEthernet4 ip address 10.3.255.9 255.255.255.252 bfd interval 50 min_rx 50 multiplier 3 router eigrp 1 network 10.3.0.0 0.0.255.255 bfd all-interfaces (Gi2) R1#show bfd neighbors details IPv4 Sessions NeighAddr LD/RD RH/RS State Int R1 10.3.255.10 4104/1 Up Up Gi4 (Gi4) Session state is UP and using echo function with 50 ms interval. interface GigabitEthernet2 Session Host: Software ip address 172.17.2.9 255.255.255.254 OurAddr: 10.3.255.9 bfd interval 333 min_rx 333 multiplier 3 Handle: 2 router bgp 65000 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3 neighbor 172.17.2.8 fall-over bfd Received MinRxInt: 1000000, Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(1371) R2 Rx Count: 985, Rx Interval (ms) min/max/avg: 34/1978/1226 last: 290 ms ago Tx Count: 1372, Tx Interval (ms) min/max/avg: 71/1137/879 last: 721 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: EIGRP CEF Uptime: 00:20:06 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 Configured in milliseconds (ms) My Discr.: 1 - Your Discr.: 4104 Min tx interval: 1000000 - Min rx interval: 1000000 Displayed in microseconds (µs) Min Echo interval: 50000 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection interface GigabitEthernet4 ip address 10.3.255.9 255.255.255.252 bfd interval 50 min_rx 50 multiplier 3 router eigrp 1 network 10.3.0.0 0.0.255.255 bfd all-interfaces (Gi2) IPv4 Sessions NeighAddr LD/RD RH/RS State Int R1 172.17.2.8 4102/1 Up Up Gi2 (Gi4) Session state is UP and using echo function with 333 ms interval. interface GigabitEthernet2 Session Host: Software ip address 172.17.2.9 255.255.255.254 OurAddr: 172.17.2.9 bfd interval 333 min_rx 333 multiplier 3 Handle: 1 router bgp 65000 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3 neighbor 172.17.2.8 fall-over bfd Received MinRxInt: 1000000, Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(6076) R2 Rx Count: 4977, Rx Interval (ms) min/max/avg: 4/1970/1069 last: 491 ms ago Tx Count: 6077, Tx Interval (ms) min/max/avg: 754/1180/879 last: 655 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: BGP CEF Uptime: 01:29:04 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 Configured in milliseconds (ms) My Discr.: 1 - Your Discr.: 4102 Min tx interval: 1000000 - Min rx interval: 1000000 Displayed in microseconds (µs) Min Echo interval: 333000 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public Routing Protocol Neighbor Behavior Detecting Unreachable Neighbor (Hello Timers vs. BFD)

100% Packet Loss R1 R2 (Link Up) EIGRP Default: Elapsed Time Between 10 – 15 Sec R1#show clock *09:58:27.716 UTC Sat Jan 27 2018 R1# 12.896 *Jan 27 09:58:40.612: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 10.3.255.10 (GigabitEthernet4) is down: holding time expired seconds

BFD: Elapsed Time Between 100 - 150 ms with 50ms interval R1#show clock *09:35:44.408 UTC Sat Jan 27 2018 R1# *Jan 27 09:35:45.571: %BFDFSM-6-BFD_SESS_DOWN: BFD-SYSLOG: BFD session ld:4101 handle:2,is going Down Reason: ECHO FAILURE *Jan 27 09:35:45.575: %BFD-6-BFD_SESS_DESTROYED: BFD-SYSLOG: 1.172 bfd_session_destroyed, ld:4101 neigh proc:EIGRP, handle:2 act 1 *Jan 27 09:35:45.580: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor seconds 10.3.255.10 (GigabitEthernet4) is down: BFD peer down notified

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

• Enhanced Object Tracking (EOT)

• Static Routing Options • Floating Static Routes • Reliable Static Routing (RSR) using EOT

• Dial on Demand Routing (DDR) • EEM Script • DMVPN State Tracking

• More information: • http://cs.co/ddrbackup

• Expands to https://www.cisco.com/c/en/us/support/docs/dial-access/dial-on-demand-routing-ddr/10213-backup-main.html

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 127 Enhanced Object Tracking (EOT) Local Significance Track Options Syntax Line-Protocol State track object-number interface type number line-protocol of Interface track 1 interface serial 2/0 line-protocol IP-Routing State of track object-number interface type number ip routing Interface track 2 interface ethernet 1/0 ip routing IP-Route track object-number ip route IP-Addr/Prefix-len reachability Reachability track 3 ip route 10.16.0.0/16 reachability Threshold* of IP- track object-number ip route IP-Addr/Prefix-len metric threshold Route Metrics track 4 ip route 10.16.0.0/16 metric threshold

Router#show track 100 Router#show track 103 Track 100 Track 103 IPv6 Interface Serial2/0 line-protocol IP route 10.16.0.0 255.255.0.0 reachability Line protocol is Up Reachability is Up (EIGRP) Support 1 change, last change 00:00:05 1 change, last change 00:02:04 15.3(3)S Tracked by: First-hop interface is FastEthernet0/0 15.4(1)T GLBP FastEthernet0/1 1 Tracked by: GLBP FastEthernet0/1 1 * EIGRP, OSPF, BGP, Static Thresholds Are Scaled to Range of (0 – 255) TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 128 Enhanced Object Tracking (EOT) External Significance

Track Options Syntax track object-number ip sla type number state IP SLAs Operation track 5 ip sla 4 state Reachability of an IP track object-number ip sla type number reachability SLAs Host track 6 ip sla 4 reachability

Types of IP SLA Probes: dhcp http path-jitter dns icmp-echo1 tcp-connect1 ethernet icmp-jitter udp-echo1 frame-relay mpls udp-jitter1 ftp path-echo voip

1Available for IPv6

Track Options Syntax

track object-number list boolean {and|or} and - both are up for object to be up or - one is up for object to be up list boolean track 5 list boolean or object 51 object 52 not ! Negates state of object track object-number list threshold {weight|percentage} track 6 list threshold weight object 61 weight 20 ! Twice as important list threshold object 62 ! Default weight 10 object 63 object 64 threshold weight up 30 down 25

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 130 Static Host Route Guarantees probe destination only reachable via desired Reliable Static Routing path track 4 list boolean or Tracking IP SLA object 400 object 401 Permanent to guarantee probes only utilize track 400 ip sla 400 reachability desired path. Stay down when down. track 401 ip sla 401 reachability ip sla 400 icmp-echo 10.100.100.100 source-ip 10.1.2.120 IP SLA IP SLA timeout 100 frequency 10 ip sla schedule 400 life forever start-time now ip sla 401 icmp-echo 10.100.200.100 source-ip 10.1.2.120

(.9) (.9) timeout 100 frequency 10 ip sla schedule 401 life forever start-time now ! ip route 10.100.100.100 255.255.255.255 Ethernet 0/1 192.168.101.9 permanent ip route 10.100.200.100 255.255.255.255 Ethernet 0/1 192.168.101.9 permanent ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 192.168.101.8/29 192.168.201.8/29 BR-W1 ip route 10.100.0.0 255.255.0.0 192.168.201.9 200

BR-W1#show ip route track-table ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [up] BR-W1#show ip route 10.100.0.0 255.255.0.0 S 10.100.0.0/16 [1/0] via 192.168.101.9

BR-W1# *Mar 12 03:57:28.367: %TRACKING-5-STATE: 400 ip sla 400 reachability Up->Down Unable to Reach *Mar 12 03:57:37.374: %TRACKING-5-STATE: 401 ip sla 401 reachability Up->Down IP SLA IP SLA *Mar 12 03:57:38.137: %TRACKING-5-STATE: 4 list boolean or Up->Down IP SLA Responders

(.9) (.9)

192.168.101.8/29 192.168.201.8/29

BR-W1#show ip route track-table ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [down] Floating Static BR-W1#show ip route 10.100.0.0 255.255.0.0 longer-prefixes BR-W1 S 10.100.0.0/16 [200/0] via 192.168.201.9 Installed S 10.100.100.100/32 [1/0] via 192.168.101.9, Ethernet0/1 S 10.100.200.100/32 [1/0] via 192.168.101.9, Ethernet0/1

IPv6 Reliable Static Routing added in 15.4(1)T

ip sla 610 Unable to Reach icmp-echo 2001:DB8::12 source-interface GigabitEthernet0/1.99 threshold 1000 IP SLA IP SLA frequency 10 Responder ip sla schedule 610 life forever start-time now

WAN RTR WAN RTR track 600 list threshold percentage object 610 2001:DB8:B::5 threshold percentage down 40 up 60 track 610 ip sla 610

event manager applet DISABLE-STATIC-IPv6 Don’t forget to reenable event track 600 state down action 1 cli command "enable" BR RTR action 2 cli command "configure terminal" action 3 cli command "no ipv6 route ::/0 2001:DB8:B::5" action 4 cli command "end" action 99 syslog msg “DEFAULT IPv6 ROUTE DISABLED" BR-RTR# 14:22:14: %TRACKING-5-STATE: 610 ip sla 610 state Up->Down 14:22:14: %TRACKING-5-STATE: 600 list threshold percentage Up->Down 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:DISABLE-STATIC-IPv6) 14:22:14: %HA_EM-6-LOG: DISABLE-STATIC-IPv6: DEFAULT IPv6 ROUTE DISABLED

15.4(1)T added Reliable Static Routing © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public Black Hole Route Detection ip sla 110 icmp-echo 208.67.222.222 source-interface GigabitEthernet0/0 IPSLA with EEM vrf INET-PUBLIC1 ! fVRF configuration threshold 1000 frequency 15 Lost connection to ISP but DHCP route ip sla schedule 110 life forever start-time now ip sla 111 stays in the routing table icmp-echo 208.67.220.220 source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now

track 60 ip sla 110 reachability track 61 ip sla 111 reachability track 62 list boolean or IP SLA object 60 Probes object 61

(config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ?

event manager applet DISABLE-STATIC-GIG0-0 event track 62 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "no ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10" action 4 cli command "end" Note: This method is compatible with action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 DISABLED" dual Internet DHCP design. event manager applet ENABLE-STATIC-GIG0-0 event track 62 state up action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10" action 4 cli command "end" action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 ENABLED"

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 134 Black Hole Route Detection IPSLA with Recursive Routing Interface GigabitEthernet0/0 vrf forwarding INET-PUBLIC1 Lost connection to ISP but DHCP route ip address dhcp stays in the routing table ip sla 110 icmp-echo 208.67.222.222 source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 ! fVRF configuration threshold 1000 frequency 15 ip sla schedule 110 life forever start-time now ip sla 111 IP SLA icmp-echo 208.67.220.220 source-interface GigabitEthernet0/0 Probes vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now

track 60 ip sla 110 reachability track 61 ip sla 111 reachability track 62 list boolean or Note: This method is compatible with object 60 dual Internet DHCP design. object 61 (config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ? ip route 192.0.2.33 255.255.255.255 GigabitEthernet0/0 dhcp 10 ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 192.0.2.33 10 track 62

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 135 EEM Script LTE Backup with Event Tracking ip sla 100 icmp-echo 192.168.4.22 source-interface GigabitEthernet0/1 threshold 1000 frequency 15 ip sla schedule 100 life forever start-time now

track 60 ip sla 100 reachability

event manager applet ACTIVATE-LTE Don’t forget to disable VPN RTR WAN RTR event track 60 state down action 1 cli command "enable" action 2 cli command "configure terminal" 192.168.4.22 NAS action 3 cli command "interface cellular0/0/0" action 4 cli command "no shutdown" action 5 cli command "end" (Ce0/0/0) action 99 syslog msg "Activating LTE interface" 14:22:14: %TRACKING-5-STATE: 60 ip sla 100 reachability Up->Down LTE-RTR 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:ACTIVATE-LTE) 14:22:14: %HA_EM-6-LOG: ACTIVATE-LTE: Activating LTE interface 14:22:34: %LINK-3-UPDOWN: Interface Cellular0/0/0, changed state to up 14:22:34: %DIALER-6-BIND: Interface Ce0/0/0 bound to profile Di1 14:22:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0/0/0, changed state to up 14:22:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel11, changed state to up 14:22:40: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON 14:22:42: %DUAL-5-NBRCHANGE: EIGRP-IPv4 201: Neighbor 10.4.36.1 (Tunnel11) is up: new adjacency http://www.cisco.com/go/cvd/wan VPN Remote Site over 3G/4G/LTE Technology Design Guide

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 136 DMVPN Interface State Control track 2 list boolean or LTE Backup with DMVPN object 101 not track 101 interface Tunnel100 line-protocol interface Tunnel200 if-state track 2 tunnel source Cellular0/0/0 end #show track 2 Track 2 List boolean or VPN RTR Boolean OR is Down WAN RTR 7 changes, last change 00:07:55 object 101 not Up

192.168.4.22 Tracked by: NAS IF-State Control 2 17:24:18.682: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel100, changed state to down 17:24:18.682: %TRACK-6-STATE: 101 interface Tu100 line-protocol Up -> Down (Ce0/0/0) 17:24:18.744: %TRACK-6-STATE: 2 list boolean or Down -> Up 17:24:28.683: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel200, changed state to up LTE-RTR 17:24:29.276: %BGP-5-ADJCHANGE: neighbor 192.168.200.12 Up 17:24:37.505: %BGP-5-ADJCHANGE: neighbor 192.168.200.22 Up #show track 2 Track 2 List boolean or Boolean OR is Up 8 changes, last change 00:00:32 object 101 not Down Tracked by: IF-State Control 2

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 138 First Hop Redundancy Protocols (FHRP) Failure Protection for the First Hop IP Router • Hot Standby Router Protocol (HSRP) • v2 IPv4 and IPv6 BR-W1 BR-W2 • Virtual Router Redundancy Protocol (VRRP) • RFC5798 (v3 IPv4 and IPv6), RFC3768 (v2 IPv4), RFC2338 (v1)

• Gateway Load Balancing Protocol (GLBP) • IPv4 and IPv6

• Provide routing redundancy for access layer • How to handle failover when end-hosts have only a single IP default gateway and cached ARP entry

• Provide routing redundancy for devices that depend on static routing • Some firewalls do not support dynamic routing

• Independent of routing protocols • Works with any routing protocol and static routing

• Capable of providing sub-second failover

• Provides load sharing capabilities (GLBP) transparent to end host

interface FastEthernet0/0 ip address 10.1.2.2 255.255.255.0 interface FastEthernet0/0 standby version 2 ip address 10.1.2.3 255.255.255.0 standby 4 ip 10.1.2.1 standby version 2 standby 4 priority 110 Active Standby standby 4 ip 10.1.2.1 Router Router standby 4 preempt standby 4 preempt BR-W1 BR-W2 HSRP standby 6 ipv6 autoconfig standby 6 ipv6 autoconfig (.2) (.3) standby 6 priority 110 VIP (.1) standby 6 preempt standby 6 preempt ipv6 address 2001:DB8:5:1::2/64 ipv6 address 2001:DB8:5:1::1/64 Default Gateway: (.1) DG MAC: MAC VIP

BR-W1#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 110 P Active local 10.1.2.3 10.1.2.1 Fa0/0 6 110 P Active local FE80::A8BB:CCFF:FE00:3400 FE80::5:73FF:FEA0 :6 BR-W2#show standby brief Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 100 P Standby 10.1.2.2 local 10.1.2.1 Fa0/0 6 100 P Standby FE80::A8BB:CCFF:FE00:3300 HSRP—Global IPv6 Addresses Available local FE80::5:73FF:FEA0 for Static Deployments :6

Active Local Router BR-W2 BR-W1 HSRP Failures (.2) (.3) (.1) VIP

Default Gateway: (.1) DG MAC: MAC VIP

BR-W2#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 100 P Active local unknown 10.1.2.1 Fa0/0 6 100 P Active local unknown FE80::5:73FF:FEA0 :6

Complex Failure

Upstream/Remote Requires “Enhanced Object Failures Tracking (EOT)”

Active Standby Active Router Router Router BR-W1 BR-W2 BR-W1 BR-W2 HSRP HSRP (.2) (.3) (.2) (.3) (.1) VIP (.1) VIP

#track 100 interface serial2/0 line-protocol ! interface FastEthernet0/0 standby version 2 standby 4 priority 110 standby 4 track 100 decrement 20 standby 6 priority 110 standby 6 track 100 decrement 20

R1#show bfd neighbors details

Active Local Router Registered protocols: HSRP BR-W2 BR-W1 HSRP Failures (.2) (.3) (.1) VIP

Default Gateway: (.1) DG MAC: MAC VIP

standby bfd all-interfaces ! default ! interface FastEthernet0/0 standby bfd ! Required only when all-interfaces disabled

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 144 Gateway Load Balancing Protocol (GLBP) BR-W1#show run int fa0/0 interface FastEthernet0/0 ip address 10.1.2.2 255.255.255.0 AVG = Active Virtual Gateway glbp 4 ip 10.1.2.1 glbp 4 preempt SVG = Standby Virtual Gateway glbp 4 weighting 110 lower 100 glbp 6 ipv6 autoconfig AVF = Active Virtual Forwarder glbp 6 preempt glbp 6 weighting 110 lower 100 ipv6 address 2001:DB8:5:1::1/64 BR-W1#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Active 10.1.2.1 local 10.1.2.3 Fa0/0 4 1 - Active 0007.b400.0401 local - AVG SVG Fa0/0 4 2 - Listen 0007.b400.0402 10.1.2.3 - Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600 AVF A AVF B BR-W1 BR-W2 local FE80::A8BB:CCF GLBP F:FE00:3400 (.2) (.3) Fa0/0 6 1 - Active 0007.b400.0601 local - VIP (.1) (.1) VIP Fa0/0 6 2 - Listen 0007.b400.0602 FE80::A8BB:CCFF:FE00:3400 - BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Standby 10.1.2.1 10.1.2.2 local Fa0/0 4 1 - Listen 0007.b400.0401 10.1.2.2 - Fa0/0 4 2 - Active 0007.b400.0402 local - Default Gateway: Default Gateway: (.1) Fa0/0 6 - 100 Standby FE80::7:B4FF:FE00:600 (.1) DG MAC: AVF B FE80::A8BB:CCFF:FE00:3300 DG MAC: AVF A local Fa0/0 6 1 - Listen 0007.b400.0601 FE80::A8BB:CCFF:FE00:3300 - Fa0/0 6 2 - Active 0007.b400.0602© 2019 Cisco and/or its affiliates. local All rights reserved. Cisco - Public TECCRS145 -2500 Gateway Load Balancing Protocol (GLBP)

AVG = Active Virtual Gateway SVG = Standby Virtual Gateway AVF = Active Virtual Forwarder

BR-W2# *May 26 19:09:14.260: %GLBP-6-STATECHANGE: FastEth0/0 Grp 4 state Standby -> Act ive *May 26 19:09:15.326: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Liste n -> Active *May 26 19:09:15.826: %GLBP-6-STATECHANGE: FastEth0/0 Grp 6 state Standby -> Act ive AVG *May 26 19:09:16.856: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 6 Fwd 1 state Liste AVF A n -> Active BR-W1 BR-W2 GLBP AVF B Local (.2) (.3) (.1) VIP Failures

BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Active 10.1.2.1 local unknown Fa0/0 4 1 - Active 0007.b400.0401 local - Fa0/0 4 2 - Active 0007.b400.0402 local - Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600 Default Gateway: (.1) Default Gateway: (.1) local unknown DG MAC: AVF A DG MAC: AVF B Fa0/0 6 1 - Active 0007.b400.0601 local - Fa0/0 6 2 - Active 0007.b400.0602 local -

AVG = Active Virtual Gateway SVG = Standby Virtual Gateway AVF = Active Virtual Forwarder Complex Failure Requires Upstream/Remote “Enhanced Object Failures Tracking (EOT)” Requires “Enhanced Object Tracking” AVF A AVG AVF A BR-W1 AVG BR-W2 BR-W1 BR-W2 GLBP GLBP AVF B (.2) AVF B (.3) (.2) (.3) (.1) (.1) VIP VIP

Branch

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 147 Enhanced Object Tracking (EOT) Tracking IP SLA ip sla 100 Lo0 Lo0 icmp-echo 10.100.100.100 source-ip 10.1.2.2 10.100.100.100 10.100.200.100 timeout 100 frequency 10 IP SLA IP SLA ip sla schedule 100 life forever start-time now ip sla 200 icmp-echo 10.100.200.100 source-ip 10.1.2.2 timeout 100 frequency 10 ip sla schedule 200 life forever start-time now ip route 10.100.100.100 255.255.255.255 FastEthernet0/1 192.168.101.9 permanent ip route 10.100.200.100 255.255.255.255 FastEthernet0/1 192.168.101.9 permanent BR-W1#show ip sla statistics IPSLA operation id: 100 Latest RTT: 1 milliseconds AVF A AVF B Latest operation start time: *04:42:11.444 UTC Tue Feb 17 2009 Latest operation return code: OK BR-W1 BR-W2 GLBP Number of successes: 46 (.2) (.3) Number of failures: 0 VIP (.1) (.1) VIP Operation time to live: Forever IPSLA operation id: 200 Latest RTT: 1 milliseconds Latest operation start time: *04:42:11.356 UTC Tue Feb 17 2009 Latest operation return code: OK Number of successes: 24 Number of failures: 0

BR-W1# track 100 ip sla 100 reachability BR-W1#show glbp track 200 ip sla 200 reachability FastEthernet0/0 – Group 4 track 1 list boolean or State is Active 1 state change, last state change 00:09:59 object 100 IP SLA IP SLA Virtual IP address is 10.1.2.1 object 200 Hello time 3 sec, hold time 10 sec interface FastEthernet0/0 Next hello sent in 2.336 secs ip address 10.1.2.2 255.255.255.0 Redirect time 600 sec, forwarder timeout 14400 sec glbp 4 ip 10.1.2.1 Preemption enabled, min delay 0 sec Active is local glbp 4 priority 110 Standby is 10.1.2.3, priority 105 (expires in 7.808 sec) glbp 4 preempt Priority 110 (configured) glbp 4 weighting 110 lower 100 Weighting 110 (configured 110), thresholds: lower 100, glbp 4 load-balancing weighted upper 110 Track object 1 state Up decrement 20 glbp 4 weighting track 1 decrement 20 Load balancing: weighted Group members: AVF A AVF B aabb.cc00.0110 (10.1.2.2) local aabb.cc00.0410 (10.1.2.3) BR-W1 BR-W2 GLBP There are 2 forwarders (1 active) (.2) (.3) Forwarder 1 VIP (.1) (.1) VIP State is Active Forwarder 2 State is Listen

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 149 Enhanced Object Tracking Composite Failure BR-W1# *Feb 17 05:17:25: %TRACKING-5-STATE: 100 ip sla 100 state Up->Down *Feb 17 05:17:25: %TRACKING-5-STATE: 200 ip sla 200 state Up->Down *Feb 17 05:17:26: %TRACKING-5-STATE: 1 list boolean or Up->Down *Feb 17 05:17:38: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Active -> Listen BR-W2#show glbp IP SLA IP SLA FastEthernet0/0 – Group 4 State is Standby 1 state change, last state change 00:28:16 Virtual IP address is 10.1.2.1 BR-W1 Remains Hello time 3 sec, hold time 10 sec Next hello sent in 1.856 secs Redirect time 600 sec, forwarder timeout 14400 sec Active Virtual Unable to Reach Preemption enabled, min delay 0 sec Either Gateway (AVG) Active is 10.1.2.2, priority 110 (expires in 10.400 sec) Standby is local IP SLA Priority 105 (configured) Responder Weighting 110 (configured 110), thresholds: lower 100, upper 110 AVF A Track object 1 state Up decrement 20 AVG AVF B BR-W2 Becomes Load balancing: weighted BR-W1 BR-W2 Group members: GLBP aabb.cc00.0110 (10.1.2.2) (.2) (.3) Active Virtual aabb.cc00.0410 (10.1.2.3) local (.1) VIP There are 2 forwarders (2 active) Forwarder (AVF) Forwarder 1 State is Active for both A and B Forwarder 2 State is Active TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 150 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 151 Overlay Management Protocol (OMP) vSmart • TCP based extensible control plane protocol • Runs between WAN Edge routers and vSmart controllers and between the vSmart controllers - Inside TLS/DTLS connections • Leverages address families to advertise reachability for TLOCs, unicast/multicast vSmart vSmart destinations (statically/dynamically learnt service side routes), service routes (L4-L7), BFD stats (TE and H-SDWAN) and Cloud onRamp for SaaS probe stats (gateway) - Uses attributes • Distributes IPSec encryption keys, and data and WAN Edge WAN Edge app-aware policies (embedded NETCONF) Note: WAN Edge routers need not connect to all vSmart Controllers

WAN Edge • Path liveliness and quality measurement detection protocol - Up/Down, loss/latency/jitter, IPSec tunnel MTU • Runs between all WAN Edge routers in the

WAN Edge WAN Edge topology - Inside IPSec tunnels - Operates in echo mode - Automatically invoked at IPSec tunnel establishment - Cannot be disabled

• Uses hello (up/down) interval, poll (app-aware) WAN Edge WAN Edge interval and multiplier for detection - Fully customizable per-WAN Edge, per-color

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 153 Path Quality and Liveliness Detection Multiplier (n) • Each WAN Edge router sends BFD hello packets for path quality and liveliness detection - Packets echoed back by remote site

Hello Interval (ms) • Hello interval and multiplier determine how Liveliness many BFD packets need to be lost to Quality declare IPSec tunnel down App-Route Multiplier (n) • Number of hello intervals that fit inside poll interval determines the number of BFD Poll Interval Poll Interval Poll Interval (ms) packets considered for establishing poll interval average path quality • App-route multiplier determines number of poll intervals for establishing overall Hello Interval (ms) average path quality

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 156 Critical Applications SLA . WAN Edge Routers vManage App Aware Routing Policy continuously perform path App A path must have: Latency < 150ms liveliness and quality Loss < 2% measurements Jitter < 10ms

Internet Remote Site

MPLS Regional Path 2 Data Center

LTE

Path1: 10ms, 0% loss, 5ms jitter Path2: 200ms, 3% loss, 10ms jitter Path3: 140ms, 1% loss, 10ms jitter IPSec Tunnel

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 157 Transport Redundancy - Meshed . WAN Edge routers are directly connected to all the transports MPLS Internet - No need for L2 switches front-ending the WAN Edge routers . When transport goes down, WAN Edge routers detect the condition and bring down the tunnels built across the failed transport WAN Edge WAN Edge - BFD times out across tunnels . Both WAN Edge routers still draw the traffic for the prefixes available through the SD-WAN fabric . If one of the WAN Edge routers fails (dual failure), second WAN Edge router takes over forwarding the traffic in and out of site - Both transport are still available

. WAN Edge routers are connected only to their respective transports MPLS Internet . WAN Edge routers build IPSec tunnels across directly connected transports and across the transports connected to the neighboring WAN Edge router WAN Edge WAN Edge - Neighboring WAN Edge router acts as an underlay router for tunnels initiated from the other WAN Edge . If one of the WAN Edge routers fails (dual failure), second WAN Edge router takes over forwarding the traffic in and out of site - Only transport connected to the remaining WAN Edge router can be used

. WAN Edge routers leverage BFD for Data detecting tunnel liveliness Center • If intermediate network path through the SD-WAN fabric fails or if the remote-end WAN Edge router (e.g. data center) fails, MPLS Internet BFD hellos will time out and remote site WAN Edge router will bring down its relevant IPSec tunnels • Traffic will be rerouted after the failed condition had been detected - BFD hello timer and multiplier can be Remote tweaked for faster detection Site

Excellent Option R2

R1 R4 SubOptimal Option

R3 Bad Option Effectiveness of Various Techniques for Different Outage Types Link Down Link Up Link Up Upstream Upstream Neighbor Down Loss ~5% Blackhole Brownout Routing Protocols

BFD N/A1 N/A1

EOT2 RSR3 using EOT (w/IP SLA) SD-WAN

1BFD Multihop support for Static and BGP routes 2Enhanced Object Tracking 3Reliable Static Routing TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 164 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment • MPLS Dual Carrier • MPLS + Internet

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 165 Dual WAN (MPLS—Dual Carrier) PE-CE Protocol: BGP 10.100.0.0/16 10.1.2.0/24 • Default behavior: 1-way 10.1.1.0/24 load sharing A-R1 MPLS - SP A A-R4

HQ- HQ-W1 CORE1 192.168.101.8/29 • Load is shared from HQ to BR-W1 192.168.201.8/29 Branch HQ-W2

HQ- B-R1 MPLS - SP B B-R4 CORE2

EIGRP eBGP eBGP HQ-CORE1#show ip route D EX 10.1.2.0/24 [170/258816] via 10.1.1.110, 02:24:22, Vlan10 [170/258816] via 10.1.1.210, 02:24:22, Vlan10

• Only one link used Branch to HQ BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00

• IGP (EIGRP examples) 10.100.0.0/16 10.1.2.0/24 • Routes redistributed from BGP into 10.1.1.0/24

IGP (match & tag) A-R1 MPLS - SP A A-R4

• BGP routes are treated as IGP external HQ- HQ-W1 CORE1 192.168.101.8/29 BR-W1 • BGP 192.168.201.8/29 HQ-W2 • No iBGP required between HQ-W1 & HQ- B-R1 MPLS - SP B B-R4 HQ-W2 (CE routers) CORE2 • Routes redistributed from IGP into BGP except those tagged as originally EIGRP eBGP eBGP sourced from BGP

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 167 Dual WAN (MPLS—Dual Carrier) Mutual Route Redistribution Detail Routes into EIGRP HQ-W1# router eigrp networkers address-family ipv4 unicast autonomous-system 65110 topology base redistribute bgp 65110 metric 45000 100 255 1 1500 10.1.1.0/2 4 address-family ipv6 unicast autonomous-system 65110 topology base redistribute bgp 65110 metric 45000 100 255 1 1500 BR AS 65100 HQ-W1 Routes into BGP HQ-CORE1 eBGP HQ-W1#

10.1.2.0/2410.1.1.0/2 router bgp 65110

iBGP address-family ipv4 EIGRP eBGP redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES HQ-W2 AS 65200 address-family ipv6 BR redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES HQ-CORE2 ! route-map BLOCK-TAGGED-ROUTES deny 10 10.100.0.0/16 BGP Redistribution to IGP automatically tags match tag 65100 65200 routes with neighbor route-map BLOCK-TAGGED-ROUTES permit 20 AS Number !

• Is it possible to load share 10.1.1.0/24 from Branch to HQ? A-R1 MPLS - SP A A-R4

• HQ- HQ-W1 BGP Multipath CORE1 192.168.101.8/29

BR-W1 • Allows installation of multiple 192.168.201.8/29 BGP paths to same destination HQ-W2

HQ- B-R1 MPLS - SP B B-R4 • Requirements (all must be equal) CORE2 • Neighbor AS or AS-PATH EIGRP eBGP eBGP • Weight BR-W1#show ip bgp • Local Preference • AS-PATH length Network Next Hop Metric LocPrf Weight Path • Origin * 10.100.0.0/16 192.168.201.9 0 65200 65200 ? • Med *> 192.168.101.9 0 65100 65100 ? BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00

• Is it possible to load share from 10.1.1.0/24

Branch to HQ? A-R1 MPLS - SP A A-R4

HQ- HQ-W1 • maximum-paths 2 CORE1 192.168.101.8/29

BR-W1 192.168.201.8/29 • Requires hidden command: HQ-W2

HQ- B-R1 MPLS - SP B B-R4 • bgp bestpath as-path multipath- CORE2 relax

EIGRP eBGP eBGP router bgp 65110 bgp bestpath as-path multipath-relax address-family ipv4 maximum-paths 2 address-family ipv6 maximum-paths 2 BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.201.9, 00:03:44 [20/0] via 192.168.101.9, 00:03:44

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment • MPLS Dual Carrier • MPLS + Internet

• Headquarters WAN Edge EIGRP BGP BGP • W1 learns Branch route via eBGP 10.100.0.0/16 10.1.2.0/24

• W2 learns Branch route via EIGRP 10.1.1.0/24

MPLS - SP A • Headquarters Core A-R1 A-R4 HQ-CORE1 HQ-W1 BR-W1

192.168.101.8/29 EIGRP • W1 redistributes eBGP into EIGRP, HSRP

results in EIGRP external Internet

VPN Tunnel • W2 does not require redistribution, HQ-CORE2 HQ-W2 BR-W2 results in EIGRP internal 10.0.1.0/29

EIGRP • Core1, Core2 install Branch route via W2 HQ-W1#show ip route B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01 HQ-W2#show ip route HQ to Branch Traffic Flows D 10.1.2.0/24 [90/26882560] via 10.0.1.2, 00:00:04, Tunnel1 Across Tunnel HQ-CORE1#show ip route D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:02:32, Vlan10

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 172 DUAL WAN (MPLS + Internet) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP • Single Router Branch WAN Edge • W1 learns HQ route via eBGP and EIGRP Internal • eBGP Administrative Distance preferred EIGRP BGP BGP

10.100.0.0/16 10.1.2.0/24

10.1.1.0/24

A-R1 MPLS - SP A A-R4

HQ-W1 HQ-CORE1 192.168.101.8/2 9 BR-W1 Branch to HQ Traffic Internet HQ-W2 VPN Tunnel HQ-CORE2 Flows Across MPLS 10.0.1.0/29 EIGRP BR-W1#show ip route B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58 B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06

EIGRP BGP BGP • Dual Router Branch WAN Edge 10.100.0.0/16 10.1.2.0/24 10.1.1.0/24

• W1 learns HQ route via eBGP A-R1 MPLS - SP A A-R4

HQ-W1

HQ-CORE1 BR-W1 HSRP 192.168.101.8/2 EIGRP • W2 learns HQ route via EIGRP 9

Internet

VPN Tunnel • No redistribution configured HQ-W2 BR-W2 HQ-CORE2 10.0.1.0/29

• HSRP Primary is on W1 EIGRP BR-W1#show ip route B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58 B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06 BR-W2#show ip route Branch to HQ Traffic D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 00:10:56, Tunnel1 D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 00:10:57, Tunnel1 Flows Across MPLS BR-W1#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/1 1 110 P Active local 10.1.2.220 10.1.2.1

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 174 DUAL WAN (MPLS + Internet) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP • How to force HQ to Branch traffic across MPLS (primary)? • Adjust administrative distance EIGRP BGP BGP • For EIGRP routes learned via tunnel 10.100.0.0/16 10.1.2.0/24 • Ensure administrative distance is 10.1.1.0/24

higher than that of EIGRP external (170) A-R1 MPLS - SP A A-R4 HQ-W2# router eigrp 65110 HQ-CORE1 HQ-W1 BR-W1

192.168.101.8/29 EIGRP network 10.0.1.0 0.0.0.7 Only change is on hub HSRP

distance 195 10.0.1.0 0.0.0.7 Internet

• Redistribute between two EIGRP Processes VPN Tunnel HQ-CORE2 HQ-W2 BR-W2

Forcing External as done between BGP and 10.0.1.0/29

Campus EIGRP EIGRP HQ-W2# Router eigrp 65100 network 10.0.1.0 0.0.0.7 Requires additional changes router eigrp 65110 or Proper Pre-Planning redistribute eigrp 65100 HQ-W1#show ip route Now: B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01 HQ to Branch Traffic Flows HQ-W2#show ip route Across MPLS D EX 10.1.2.0/24 [170/261120] via 10.1.1.110, 00:07:25, GigE0/0 HQ-CORE1#show ip route D EX 10.1.2.0/24 [170/258816]TECCRS-2500 via© 2019 Cisco10.1.1.110, and/or its affiliates. All rights 00:08:44, reserved. Cisco Public Vlan10175 DUAL WAN (MPLS + Internet) MPLS Failure

• Failure within MPLS cloud EIGRP BGP BGP

• Dependent on provider 10.100.0.0/16 10.1.2.0/24 • Worst Case 10.1.1.0/24 A-R1 MPLS - SP A A-R4 • Link up neighbor down HQ-CORE1 HQ-W1 BR-W1

192.168.101.8/29 EIGRP • Primary dependency BGP timers HSRP

• End to end convergence time as Internet long as BGP Holdtime VPN Tunnel HQ-CORE2 HQ-W2 BR-W2 • Configuration options 10.0.1.0/29 • BFD for almost immediate notification EIGRP • End-to-end Application Restoration as HQ Route Tables fast as SD-WAN detects After Failure: HQ-W2#show ip route HQ to Branch Traffic D 10.1.2.0/24 [195/26882560] via 10.0.1.2, 00:06:46, Tunnel1 HQ-CORE1#show ip route Flows Across Tunnel D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:09:18, Vlan10

MPLS Failure EIGRP BGP BGP

10.100.0.0/16 10.1.2.0/24 • Failure within MPLS cloud 10.1.1.0/24

• Suboptimal routing at Branch A-R1 MPLS - SP A A-R4

HQ-CORE1 HQ-W1 BR-W1 • 192.168.101.8/29

HSRP primary remains EIGRP HSRP

unchanged at BR-W1 Internet

VPN Tunnel • Use EOT and move HSRP HQ-CORE2 HQ-W2 BR-W2 primary to BR-W2 10.0.1.0/29

EIGRP Branch Route Tables BR-W1#show ip route D 10.100.100.0/24 After Failure: [90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1 Branch to HQ D 10.100.200.0/24 Traffic Flows [90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1 Across Tunnel BR-W2#show ip route D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 01:08:44, Tunnel1 D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 01:08:45, Tunnel1 TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 177 Agenda • Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment

• Final Wrap Up • Key Takeaways

• Outages can manifest in many different ways. Network design should be based on application requirements to survive various outages.

• Cisco IOS has inherent load sharing capabilities. Analyze your network topology and use these to your advantage.

• End-to-end convergence time is a critical metric. Understand how localized topology changes affect end-to-end resiliency.

• Multiple links/paths not only increase network reliability but can improve application performance.

• IP SLA based monitoring can detect outage types that are virtually undetectable by traditional “hello based” techniques.

• BFD is a lightweight tool for speeding convergence of all protocols.

• Cisco SD-WAN permits full utilization of available bandwidth and path selection based on current real time characteristics.

• Most effective network designs incorporate a combination of convergence techniques

• Cisco SD-WAN utilizes these features, while simplifying deployment and management, and increasing application availability.

WAN QOS TECCRS-2500

Arvind Durai, CCIE #7016 R/S and Security Director, Solutions Integration Architect, Cisco Advanced Services Goals

• To get a high-level overview of components in QOS technology for branches and WAN connections

• You should be able determine the correct options to design QOS for a branch

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 183 Agenda • Understanding QOS • Basic Elements of QOS • Design Elements for WAN QOS • Class Models for Enterprise QOS • Branch QOS Across Managed MPLS Service Provider • QOS and Encryption

• QOS Model for WAN Connectivity • Branch Router QOS • WAN Connection Examples

• Automation of WAN - use cases

• Key Takeways

Classification Queuing and Post-Queuing and Marking Dropping Operations

. Classification and Marking: The first element to a QoS policy is to classify/identify the traffic that is to be treated differently. Following classification, marking tools can set an attribute of a frame or packet to a specific value. . Policing: Determine whether packets are conforming to administratively-defined traffic rates and take action accordingly. Such action could include marking, remarking or dropping a packet. . Scheduling (including Queuing and Dropping): Scheduling tools determine how a frame/packet exits a device. Queuing algorithms are activated only when a device is experiencing congestion and are deactivated when the congestion clears. . Link Specific Mechanisms (Shaping, Fragmentation, Compression, Tx Ring) Offers network administrators tools to optimize link utilization

Voice SD Video Conf Telepresence Data

. Smooth . Bursty . Bursty . Smooth/bursty . Benign . Greedy . Drop sensitive . Benign/greedy . Drop sensitive . Drop sensitive . Delay sensitive . Drop insensitive . Delay sensitive . Delay sensitive . Jitter sensitive . Delay insensitive . UDP priority . UDP priority . UDP priority . TCP retransmits

Bandwidth per Call SD/VC has the Same HD/VC has Tighter Traffic patterns for Depends on Codec, Requirements as Requirements than Data Vary Among Sampling-Rate, VoIP, but Has VoIP in terms of jitter, Applications and Layer 2 Media Radically Different and BW varies based Traffic Patterns on the resolutions • Latency ≤ 150 ms (BW Varies Greatly) . Data Classes: • Jitter ≤ 30 ms . Latency ≤ 150 ms . Latency ≤ 200 ms . Mission-Critical Apps • Loss ≤ 1% . Jitter ≤ 30 ms . Jitter ≤ 20 ms . Transactional/Interactive Apps • Bandwidth (30- . Loss ≤ 0.05% . Loss ≤ 0.10% 128Kbps) . Bandwidth (1Mbps) . Bandwidth (5.5-16Mbps) . Bulk Data Apps One-Way Requirements One-Way Requirements One-Way Requirements . Best Effort Apps (Default) TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 186 Enabling QoS Elements that Affect End-to-End Delay

Cisco CallManager Cluster PSTN SRST Router IP WAN

Campus Branch Office Propagation Queuing Serialization CODEC and Network Jitter Buffer Variable Variable 6.3 µs/Km + G.729A: 25 ms (Can Be Reduced (Can Be Reduced Network Delay 20–50 ms Using LLQ) Using LFI) (Variable) End-to-End Delay (Should Be < 150 ms)

4/5 Class Model 8 Class Model 11 Class Model

Voice Voice Realtime Interactive-Video Video Streaming Video Call Signaling Call Signaling Call Signaling IP Routing Network Control Network Management Critical Data Mission-Critical Data Critical Data Transactional Data Bulk Data Bulk Data Best Effort Best Effort Best Effort

Voice Best Effort Call-Signaling 5% (62%) 33% Voice 33% Scavenger 1%

Best Effort Call-Signaling 25% Critical Data 36% 5%

Three-Class (VoIP and Data Only) Five-Class WAN Edge Model WAN Edge Model Best Effort Voice 18% 25%

Interactive-Video 15% Scavenger 1% Eleven-Class WAN Edge Model Bulk Data 4%

CBWFQ: Class-based weighted fair queuing (CBWFQ) extends the LLQ: The Low Latency Queuing feature brings strict standard WFQ functionality to provide support for user-defined traffic priority queuing to Class-Based Weighted Fair Queuing classes. For CBWFQ, you define traffic classes based on match criteria (CBWFQ). including protocols, access control lists (ACLs), and input interfaces. class-map class-map match …… match …… class-map class-map match …….. match …….. policy-map policy-map class name1 class name1 priority … bandwidth…. class name 2 class name 2 bandwidth … bandwidth … class class-default class class-default fair-queue fair-queue int gig 0/0 int gig 0/0 service-policy service-policy

Parent/ Child MQC with Shaper A hierarchical policy is a quality of service (QoS) model that class-map enables you to specify QoS behavior at multiple levels of match …… hierarchy. Depending on the type of hierarchical policy you class-map configure, you can use hierarchical policies to: match …… policy-map • Specify multiple policy maps to shape multiple class queues together priority ….. • Apply specific policy map actions on the class aggregate traffic bandwidth ……. class class-default • Apply class-specific policy map actions fair-queue policy-map • Restrict the maximum bandwidth of a virtual circuit (VC) while class class-default allowing policing and marking of traffic classes within the VC shape …… All hierarchical policy types consist of a top-level parent policy and service-policy one or more child policies. The service-policy command is used to int gig 0/0 apply a policy to another policy, and a policy to an interface, service-policy subinterface, virtual circuit (VC), or virtual LAN (VLAN).

Low Latency Queuing Link Fragmentation Police and Interleave VoIP IP/VC PQ TX Interleave Signaling Ring Packets Packets Critical Out In Fragment Bulk CBWFQ Mgmt FQ Default

Layer 3 Queuing Subsystem Layer 2 Queuing Subsystem

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public CBWFQ Operation IOS Interface Buffers policy-map CBWFQ class NETWORK-CONTROL bandwidth percent 5 Network Control CBWFQ class CALL-SIGNALING bandwidth percent 5 Call Signaling CBWFQ class OAM bandwidth percent 5 OAM CBWFQ class MM-CONFERENCING Packets FQ bandwidth percent 10 Multimedia Conferencing CBWFQ fair-queue In Tx-Ring FQ CBWFQ … Multimedia Streaming CBWFQ Scheduler FQ Packets Transactional Data CBWFQ Out FQ Bulk Data CBWFQ FQ Best Effort / Default CBWFQ FQ Pre-Sorters Scavenger CBWFQ TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 193 LLQ Operation

IOS Interface Buffers 1 Mbps VoIP policy-map LLQ Policer LLQ class VOIP priority 1000 Packets … In Packets Out CBWFQ Scheduler Tx-Ring

FQ Pre-Sorters CBWFQ

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 194 Multi-LLQ Operation IOS Interface Buffers policy-map MULTI-LLQ 1 Mbps class VOIP VoIP Policer priority 1000 class BROADCAST-VIDEO 4 Mbps Bscst-Video priority 4000 Policer LLQ class REALTIME- 5 Mbps INTERACTIVE RT-Interactive Policer priority 5000 …

Packets Packets In Out CBWFQ Scheduler Tx-Ring

CBWFQ

policy-map WAN class VOICE priority percent 10 class INTERACTIVE-VIDEO priority percent 23 class CRITICAL-DATA bandwidth percent 15 L3 Queue random-detect dscp-based class DATA bandwidth percent 19 Low Latency Queueing random-detect dscp-based class SCAVENGER bandwidth percent 5 Police VOICE class NETWORK-CRITICAL bandwidth percent 3 service-policy MARK-BGP PQ class class-default bandwidth percent 25 Police INTERACTIVE-VIDEO To Layer 2 random-detect Queueing CRITICAL-DATA Packets DATA Subsystem In SCAVENGER CBWFQ NETWORK-CRITICAL FQ class-default Weighted Random Early Detection (WRED) Random Early Detection (RED) Layer 3 Queueing Subsystem TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 196 What Are the QoS Implications of MPLS VPNs?

Bottom Line: Enterprises must Co-manage QoS with Their MPLS VPN Service Providers; Their Policies must Be Both Consistent and Complementary

Maximum One-Way Service-Levels Latency ≤ 150 ms/Jitter ≤ 30 ms/Loss ≤ 1%

Enterprise Enterprise Campus Remote-Branch Service Provider

PE PE CE CE

Maximum One-Way SP Service-Levels Latency ≤ 60 ms Jitter ≤ 20 ms Loss ≤ 0.5%

• Enterprise customers may need to re-mark traffic prior to forwarding to the MPLS provider. This ensures markings conform to the admission criteria defined by the provider, allowing traffic to be serviced by the appropriate queue within the provider network. • The same concept applies to traffic ingression the enterprise network from the provider cloud. • Certain applications may need to be re-marked to ensure the enterprise QoS strategy is properly applied.

Enterprise Network Provider Network

Enterprise Class Structure: Provider Class Structure: • Class 1 [DSCP A] • Class 1 [DSCP A] Provider Trust • Class 2 [DSCP C] • Class 2 [DSCP B] Enterprise Trust Boundary . Boundary • Class 3 [DSCP D] . PE Ingress Policing and Remarking • Class 4 [DSCP E] . PE-to-CE Queuing/Shaping/LFI • Class n [DSCP F] Maximum One-Way Service-Levels Latency ≤ 150 ms/Jitter ≤ 30 ms/Loss ≤ 1% TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 199 Enterprise-to-Service Provider Mapping Five-Class Provider-Edge Model Remarking Diagram

Enterprise DSCP PE Classes Application Routing CS6 Voice EF EF SP-Real Time 35% Interactive Video AF41  CS5 CS5 Streaming Video CS4  AF21 CS6 SP-Critical Mission-Critical Data AF31 AF31 20% Call Signaling CS3  CS5 CS3 Transactional Data AF21  CS3 AF21 SP-Video CS2 15% Network Management CS2 AF11/CS1 SP-Bulk 5% Bulk Data AF11 Scavenger CS1  0 SP-Best Effort 25% Best Effort 0

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 200 MPLS Short Pipe Mode DiffServ Tunneling Short Pipe Mode Operation Shaded Area Represents Provider DiffServ Domain Assume a Policer Remarks Unshaded Areas Out-of-Contract Traffic’s Represent Customer Top-Most Label to DiffServ Domain MPLS EXP 0 Here PE Edge (to CE) MPLS VPN Policies Are Based on CE Router PE Router Customer Markings P Routers

PE Router CE Router

IPP3/DSCP AF31 MPLS EXP 4 MPLS EXP 0 MPLS EXP 0 IPP3/DSCP AF31 Packet Initially MPLS EXP 4 MPLS EXP 4 IPP3/DSCP AF31 Original Customer- Marked IP ToS Marked to IPP3/ IPP3/DSCP AF31 IPP3/DSCP AF31 DSCP AF31 Top-Most Label Is Values Are MPLS EXP Values Topmost Label Is Popped (PHP), but Preserved Are Set Independently Marked Down by Egress Policy Is Based from IPP/DSCP Values a Policer on EXP 0 of Topmost Label

• The IPv6 implementation of DiffServ is identical to IPv4. The same classifiers can be used to differentiate both IPv6 and IPv4 packets, as follows:

• Source IP address, destination IP address, IP Protocol field, source port number, and destination port number

• IP precedence or DSCP values To match packets on both IPv4 and IPv6 • TCP/IP header parameters, such as packet length protocols: class-map match-all ipv6+ipv4forprec5 • Source and destination MAC addresses match precedence 5

To match packets for IPv6 protocols only: • The match precedence and match dscp commands class-map match-all ipv6onlyprec5 filter IPv4 and IPv6 traffic. match protocol ipv6 match precedence 5

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 202 Next Generation NBAR (NBAR2) SCE Classification +1000 Signatures Allows finer grained Innovations classification of traffic based IOS NBAR Advanced Classification Techniques Native IPv6 Classification on additional application level +150 Signatures Open API characteristics –http url, host, mime, User Agent and other fields •e.g. “match protocol http url *cisco.com*” NBAR2 matches http traffic to and from cisco.com –rtp payload-type • New DPI engine provides Advanced Application Classification •e.g. “match protocol and Field Extraction Capabilities from Service classification rtp video” matches rtp engine video traffic –citrix ica-tag, app • Protocol Pack allows adding more applications without •e.g. “match protocol upgrading or reloading IOS citrix ica-tag 0” • NBAR2 Protocol List - matches citrix traffic http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6558/ps661 with ica-tag 0 6/product_bulletin_c25-627831.html

Discover: 1000+ applications categorized to simplify management

HTTP Performance Collection: Enhanced application performance reports, url hit counts, top applications …

HTTP HTTP Control: Apply QoS, Acceleration and Path Control according to company performance expectations

Application Visibility and Natively Integrated into Simple to Enable Control Cisco Routers

class-map match-all business-critical match protocol citrix Application BW Priority match access-group 101 Committed BW (50% of the line) Business Critical Committed 30% High class-map match-any browsing Browsing 30% (=21% of the line) Normal match protocol attribute category browsing Excess BW Internal 60% (Out of (50% of the line) class-map match-any internal-browsing Browsing Browsing) match protocol http url “*myserver.com*” Remaining 70% (=42% of the line) Normal policy-map internal-browsing-policy class internal-browsing bandwidth remaining percent 60 policy-map my-network-policy class business-critical priority police percent 30 Remaining: class browsing Business-Critical: 70% of Excess BW bandwidth remaining percent 30 High Priority (=42% of line) service-policy internal-browsing-policy 30% committed Browsing: Internal-Browsing: 30% of Excess BW interface Serial0/0/0 60% of Browsing service-policy output my-network-policy (=21% of the line)

• QoS Policies based on CLI Templates

• Deploy/Un-deploy QoS Policies for devices/device and

site groups

• Queue drops and class hierarchy export through FNF

• New QoS Option Table Export

• CBQoS MIB based Statistics

Applied Policy Map In the Flow Record, collect policy qos class hierarchy policy-map P1 class C1 collect policy qos queue drops shaping average 16000000 service-policy child policy-map child class C11 Flow Hierarchy Queue id bandwidth remaining percent 10 class C12 Flow 1 P1, C1, C11 1 bandwidth remaining percent 70 class class-default Flow 2 P1, C1, C11 1 bandwidth remaining percent 20 Flow 3 P1, C1, C12 2 class-map match-all C1 match any class-map match-all C11 match ip dscp ef Queue id Queue packet drops class-map match-all C12 match ip dscp cs2 1 100 2 20

• For each flow, the class hierarchy and queue drops can now be exported through FNF • Class-ID to Name mapping provided through separate Option Templates

DSCP IP header IP Payload

Set DSCP Original IP Packet DSCP IP header IP Payload Classified IP packet DSCP copy DSCP New IP header built by tunnel entry point IP new hdr DSCP byte is copied

DSCP IP new hdr ESP header IP IP Payload IPSec packet QoS classification/marking must occur before encryption

QOS pre-classify is needed only if QOS classification requires Layer 3 and 4 information is needed for packet classification. This is need because original header fields for port numbers are encrypted. Packet Clone Particle1 class-map match-all TRANSACTIONAL-DATA Particle2 description Order Entry Application TN3270 Input Interface ParticleN match access-group 123 Clone Clone Particle %$#*&1 %$#*& 2 Crypto Engine %$#*& N

Output Interface QoS Classification access-list 123 permit tcp any host 10.45.15.1 eq telnet Encrypted Packet

• IPSEC Tunnel Mode ( no GRE): Pre-encryption • Unicast traffic can be transported queuing • Multicast or Routing Protocols cannot be transported • Cisco prefragmentation feature is supported • IPSEC Transport Mode (with IP GRE tunnel): • Unicast and Multicast traffic can be transported • Cisco pre-fragmentation feature is NOT supported • IPSEC Tunnel Mode (with IP GRE tunnel): • Unicast and Multicast traffic can be transported • Cisco pre-fragmentation feature is supported • Increases the packet size compare to other two IPSEC modes • Default mode for DMVPN

Note: GETVPN preserves the IP source and destination addresses during the IPSec encryption and encapsulation process. Therefore GET VPN integrates with features of QoS. Criteria's of IPSEC needs to be understood for GETVPN.

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 211 QOS and Anti-Replay Implication Outside . What is Anti Replay? Window 64 Packet Sliding Window • Anti Replay protects the message integrity of the IPSEC packet. The packet can be received in out of order and there is a 64 1 2 4 64 65 66 67 packet sliding window (this window size can be adjusted in IOS) 3 to tally the receipt of the peer packet. Anti-Replay Drop • The out of sequence packet should fall within the scope of this window. If they arrive out of the window they will be dropped. In IPSEC with QOS, the lower priority packets when delayed will be dropped. . Workaround for Anti replay issues in IPSEC with QOS enabled: IPSEC (with IP-GRE) with anti replay has issues when QOS is enabled. However, data packet ( lower in priority) will be dropped in comparison to voice packet. One of the design principle normally used is the queue limit should be reduced in descending order of application priority. Changing the size of the anti replay window, masks this issue.

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 212 Agenda • Understanding QOS • Basic Elements of QOS • Design Elements for WAN QOS • Class Models for enterprise QOS • Branch QOS Across Managed MPLS Service Provider • QOS and Encryption • QOS Model for WAN Connectivity • Branch Router QOS • WAN Connection Examples • Automation of WAN– use cases

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 213 WAN QoS – Implementing Per Site Traffic Shaping CE 10 Mbps 500 Mbps in to WAN can easily overrun the 10.5.144.0/21 CE lower speed committed rates at remote sites CE 10 Mbps 10.5.152.0/21 500 Mbps CE CE CE 802.1q 50 Mbps trunk CE CE CE Shape 50 Mbps (500 Mbps) CE CE 20 Mbps 10.5.168.0/21 Per-Site Shaping to Avoid Overruns CE CE

policy-map POLICY-MAP-Br210 policy-map POLICY-MAP-Br212 class VOICE class VOICE priority percent 10 priority percent 10 class INTERACTIVE-VIDEO Per Destination class INTERACTIVE-VIDEO priority percent 23 priority percent 23 class CRITICAL-DATA Service Policies class CRITICAL-DATA bandwidth percent 15 bandwidth percent 15 random-detect dscp-based random-detect dscp-based class DATA class DATA bandwidth percent 19 bandwidth percent 19 random-detect dscp-based random-detect dscp-based class SCAVENGER class SCAVENGER bandwidth percent 5 bandwidth percent 5 class NETWORK-CRITICAL class NETWORK-CRITICAL bandwidth percent 3 bandwidth percent 3 service-policy MARK-BGP service-policy MARK-BGP class class-default class class-default bandwidth percent 25 bandwidth percent 25 random-detect random-detect

ip access-list extended Br210-10.5.144.0 permit ip any 10.5.144.0 0.0.7.255 ! Per Destination ip access-list extended Br212-10.5.168.0 class-map match-all CLASS-MAP-Br210 permit ip any 10.5.168.0 0.0.7.255 match access-group name Br210-10.5.144.0 Class Maps ! class-map match-all CLASS-MAP-Br212 match access-group name Br212-10.5.168.0

policy-map POLICY-MAP-L2-WAN-BACKBONE-WITH-PER-SITE-SHAPERS class NETWORK-CRITICAL bandwidth percent 3 class CLASS-MAP-Br210 shape average 10000000 Shape to 10 Mbps to BR210 service-policy POLICY-MAP-Br210 class CLASS-MAP-Br212 shape average 20000000 Shape to 20 Mbps to BR212 service-policy POLICY-MAP-Br212

policy-map POLICY-MAP-L2-WAN-BACKBONE-WITH-PER-SITE-SHAPERS class NETWORK-CRITICAL bandwidth percent 3 class CLASS-MAP-Br210 Shape to 10 Mbps to BR210 shape average 10000000 service-policy POLICY-MAP-Br210 class CLASS-MAP-Br212 Shape to 20 Mbps to BR212 shape average 20000000 service-policy POLICY-MAP-Br212

policy-map WAN-INTERFACE-G0/0/4 class class-default Shape to 500 Mbps aggregate shape average 500000000 service-policy POLICY-MAP-L2-WAN-BACKBONE-WITH-PER-SITE-SHAPERS

child shapers 10 Mbps parent shaper 10 Mbps 50 Mbps Shape 50 Mbps (500 Mbps) 20 Mbps 20 Mbps

• NHRP group per policy ! DMVPN Hub Configuration Policy-map SHAPING-1.5MBPS Class class-default shape average 1500000 service-policy site Policy-map SHAPING-1.0MBPS Class class-default shape average 1000000 interfaceservice -Tunnel1policy site bandwidth 45000 ip address 10.0.0.1 255.255.255.0 ip nhrp map multicast dynamic ip nhrp map group group1 service-policy output SHAPING-1.5MBPS ip nhrp map group group2 service-policy output SHAPING-1.0MBPS ! Spoke Configuration interface Tunnel1 bandwidth 1500 ip address 10.0.0.2 255.255.255.0 ip nhrp group group1

Classification - Flow, match on 5-tuple (ACL, Data Policy) - Application, match on DPI (Data Policy) vEdge Q0 Per-Egress Interface Queuing Q1 - Q0 is LLQ Q2 - vEdge control traffic (DTLS/TLS, BFD, routing protocols) goes into Q0. Not subjected to LLQ policer

Q7 Egress InterfaceEgress Ingress InterfaceIngress Scheduling for Q1-Q7 is WRR* Drop is RED** or taildrop - RED drop profiles are linear, i.e. X% queue depth results in X% Classification Queuing drop probability

* Weighted Round-Robin ** Random Early Discard

• Shaper behavior • Single Rate Policer behavior - Forward shaper rate conforming traffic - Forward traffic conforming to policer rate o There are tokens in the bucket o There are tokens in the bucket - Queue shaper rate exceeding traffic - Drop traffic exceeding policer rate o There are no tokens in the bucket o There are no tokens in the bucket o Weighted Round-Robin - Configurable burst rate • Egress-only Shaping o Token bucket depth - Interface based • Ingress and Egress Policing - Interface/VLAN based - Access list classification - Flow policing, match on 5-tuple - Data Policy classification (ingress only) o Flow policing, match on 5-tuple o Application policing, match on DPI TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 219 Viptela QoS features -Marking/Re-marking

• Classification - Flow, match on 5-tuple (ACL, Data Policy) vEdge - Application, match on DPI (Data Policy) • Ingress interface marks/remarks inner

DSCP bits Egress InterfaceEgress Ingress InterfaceIngress - Copied to encapsulation DSCP bits

• Egress marks/remarks outer encapsulation Classification Marking, DSCP bits Re-marking - Inner DSCP bits not modified - Transport network QoS

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 220 Example - Viptela QoS Policy policy The configuration snippet is for policer bursty-traffic interface ge1/0, in VPN 1. The rate 1000000 burst 20000 policer monitors incoming traffic exceed remark on the interface. When traffic access-list policer-bursty-traffic sequence 10 exceeds 20 MB (configured in match the policer burst command). source-ip 56.0.1.0/24 action accept policer bursty-traffic default-action accept vpn 1 interface ge1/0 ip address 56.0.1.14/24 no shutdown access-list policer-bursty-traffic in

Design Operational Processes Processes • Ensuring network SLAs through QOS Application requires a holistic approach combining SLA Requirements design and operational processes

1. Network Verify the tier 1 application requirement SLA Specification 2. Review current Network SLA on the WAN links (create measurement techniques in this stage / input from High Level QoS Design capacity planning)

3. High level / Low level QoS Design Low Level QoS Design 4. and config Network SLA monitoring and capacity Network and Capacity of WAN links should always be SLA Monitoring Planning monitored to managed network services on QoS.

• Understanding QOS • Basic Elements of QOS • Design Elements for WAN QOS • Class Models for enterprise QOS • Branch QOS Across Managed MPLS Service Provider • QOS and Encryption • QOS Model for WAN Connectivity • Branch Router QOS • WAN Connection Examples • Automation of WAN - usecases

• Today’s networks are managed using CLI • CLI is used to perform basic device configuration and to deploy and change service configurations on the network (e.g. new VPN, new network segment, new subscriber)

• These configurations can be characterized as follows: • Day 0 – Basic device turn-up • Day 1 – Deploy device in network ready to receive service configurations • Day 2 – Apply and change service configurations • Transitioning to model-based management requires translation of the operations currently performed using CLI to equivalent operations using NETCONF or REST

Ansible Puppet OpenFlow Neutron ML2 Python SDK OpenStack API Agile DevOps Salt Netconf YANG

Chef Container

NX-API REST JSON Controller LXC

XML NFV

Template 4k-1 : { $vlanid : 10, $vlan_name : “vlan_red”, #Setup VLAN $intf_name : “Eth1/1”, cisco_vlan {"${vlanid}": $intf_ip : “1.1.1.1”, vlan_name => $vlanname, $intf_mask : “255.255.255.0” ensure => present } }, #Create VLAN Interface (step2) cisco_interface { $intfName : 4k-2 : { description => $vlanname, $vlanid : 20, shutdown => false, ipv4_address => $intf_ip, $vlan_name : “vlan_blue”, ipv4_netmask_length => $intf_ip_mask, $intf_name : “Eth2/1”, } $intf_ip : “2.1.1.1”, $intf_mask : “255.255.255.0” },

Puppet Master/Chef Server Nexus sends data and request cfg every 30 mins

Manifests/C SSL ookbooks Server sends config to switch Container

Agent • Puppet and Chef use a pull model (agent/client pulls from server) Router • Agent/Client lives in container (optionally directly in bash)

• Cisco modules in Puppet Forge or Chef Supermarket

Ansible Server

Server sends config when playbook is run

Playbooks CLI (SSH) No agent

Router

Unlike server configuration Ansible does not execute Python on-box • Ansible uses an agentless push model

• Uses YAML and Jinja2 templates

• Can configure using CLI (SSH) or NX-API

• Use nxos-ansible modules, or new core Ansible 2.1 modules

Day 0: Device provisioning - Multiple solutions Get a device into an operational state - Topology, traffic view tied to NMS - Automated alerts - Integration takes time Day 1:Provision services and might increase the TCO

Day to day operations, provisioning

Platform solution takes care of Day 0, Day 2 and ability to provide Day 1 Day2: Operate use cases Configuration 3rd Party Tools API REST On-box Python / TCL Management Tools EEM (Splunk, Nagios, etc.) (Puppet/Chef/Ansible)

Turn-Key Customizable

Turnkey solution stack for end- to-end enterprise Service-orchestration focused orchestration Modular solution architecture On-prem or cloud-based Build/design/run & Analytics Flexible demarcation between SP and Enterprise Virtual and Physical Multi-vendor / Multi-tenancy Support for SDA and IWAN Open API for Extensibility Customized SP service catalogues Multi-vendor & Multi-Tenant

Network and security LEARNING Automation services automation aligned with the IT Process

Proactive and predictive insights Analytics to assure service experience

INTENT CONTEXT Cisco API standardization and DNA-C as monetization for app dev and a Platform programmability

Cross Automation and Analytics Integration with offers from Edge Domain to Cloud including Security

SECURITY Cisco Cloud and hybrid deployment DNA-C of Cisco DNA-C to address Cloud different markets

25 devices Application Experience & Traffic Prioritization One Click QoS Policy Enforcement (Easy QoS) use-case 2

Identity Easy QoS Security Services

MS CUCM Cognitive Controller E

• Enterprise applications are automatically Surveillanc FTP classified and given right class of service e based on cisco validated design (CVD)

application mappings Platinum Gold • QoS policies are applied at a system level

with a single click of a button, improving Silver application performance and saving

valuable time/resources BestEffort

Cisco Validated Design {CVD}

Set to CVD

Applications Engineers • Logically centralized network services • Model based architecture REST, NETCONF, Java, Python, Erlang, CLI, Web UI • Data models written in YANG (RFC Service 6020) Model Service Manager • Structured representations of: • Service instances Device Manager Device • Network configuration and state Model • No hard-coded assumptions about: Network Element Drivers (NEDs) • Network services • Network architecture

NETCONF, REST, SNMP, CLI, etc • Network devices • Mapping service operations to network • VNFM configuration changes • Controller Apps • Transactional integrity • EMS and NMS • Multiprotocol and multivendor support Physical Networks Virtual Networks Network Apps

Yang Model Representation Internal NSO Representation

container service { service key "name";

container interface { interface type string; type container type { type int64; Router Interface number Configuration container number { Store leaf ip { type inet:ip-address;

}

leaf speed { type int64; ip speed } } {10.1.1.21} {100} } }

Service Models written independent of devices!

API calls to NSO to Map Service to Device Models

API with Input Parameters

{configure interface} {interface} {GigabitEthernet} {1} Call Map Commit Write {172.16.11.1} {100}0

Orchestrator • Hypervisor - to host VNFs • OVS (Open vSwitch)- provide communication REST or NETCONF between VNF and help in service chaining • LCM (life cycle management)- Monitors and V V V Manages VNFs N N N • PnP (Plug and Play) -Branch office zero touch F F F OAM&P deployment Software Cisco NFVIS Hypervisor | OVS |LCM | PNP Hardware UCS-C, UCS-E, x86

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 240 Cloud Services Platform 2100 Easy to use Automated Clustering High Performance • Turnkey and simple • Deploy services as fast • Shared pool of resources • PCIe Passthrough • Built for network, security, and as applications • Auto-deploy redundant HA • SR-IOV load balancing teams • Use DevOps to automate pair • Lifecycle management ACI services • Scale-out architecture • Provision a new service within • RESTful API minutes using GUI or CLI • NetConf/Yang REST GUI CLI API NSO NetConf

XRv Third KVM based ASAv 9000 party services

CSP 2100 SW, ConfD, Linux KVM, OVS, PCIe Passthrough, SR-IOV

Cisco UCS 1RU/2RU Modular Platforms, 1 & 10G SFP+ NICs NFS

WAN IP Multicast

Arvind Durai, CCIE 7016 Director, Solution Integration, Cisco Advanced Services - [email protected] Uses of Multicast Technology in Enterprise Networks Security Stock Exchange School Security

Bank security IP Cameras Retail behavior Video Conferencing E-Learning

Call Manager Call Manager

Live Web Cast of Minimally-Invasive Hip Replacement IP/TV Broadcast Server Program Manager

Web Server, Media Publishing Retail Catalog distribution Software Software Distribution

Store 1 Patch Store 2 Update Branch Office Store 3 Patch Update Branch Corp HQ Office Store 4 Patch Update Branch Retail Headquarters © 2019 Cisco and/or its affiliates. All rights reserved.Office Cisco Public Goals

• Understand the components of multicast technology for Branches and WAN connections

• Choose valid design options to design multicast in a branch

• Introduction • Multicast Design Components • Case Study • Key Takeaways Understanding Design Requirements for Multicast in a Branch

• Financial application The application type • Corporate Video • Imaging • Customized Multicast Application

• RP placement The Enterprise Multicast Domain • PIM domain • Unicast domain • QOS and Security Architecture

• Single link and redundant link access The Access methodology • Verify if encryption is deployed • Check if the branches have access through Service provider managed MPLS cloud.

• Extension of Enterprise domain or Local domain or Extension of enterprise domain & local domain • Size of each branch • Platform deployed

• Addressing Modes • PIM Sparse mode • Layer 2 Multicast • Rendezvous points • Multicast Domain security consideration • Multicast with Encryption • Multicast in Provider managed deployments • IPv6 Multicast Overview Administratively-Scoped Address Range

239.0.0.0 • Address Range: 239.0.0.0/8 RFC 2365 • Private multicast address space Org.-Local • Similar to RFC1918 private unicast Expansion address space • RFC 2365 Administratively Scoped Zones 239.192.0.0 • Organization-Local Scope (239.192/14) RFC 2365 – Largest scope within the Enterprise network (i.e. Enterprise- Org-Local wide) Scope – Expands downward in address range • Local Scope (239.255/16) 239.196.0.0 – Smallest possible scope within the Enterprise network 239.253.0.0 RFC 2365 – Expands downward in address range Local Scope Expansion – Other scopes may be equal but not smaller 239.255.0.0 RFC 2365 Local 239.255.255.255 Scope

(Not to scale.) TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 250 Multicast Domains Administratively-Scoped  Separate PIM domain • Separate RP Address Range • Separate Multicast group Example • Verify containment is Company ABC required for local scoping 239.0.0.0/8

LA NYC Campus Campus/ /Branch Branch

RFC 2365 Local Scopes 239.255.0.0/16 RFC 2365 Org-Local Scope Enterprise or Organization scope covers the entire enterprise network. 239.192.0.0/14 Local or Regional scope covers a subset of the enterprise scope

• Dense-mode (DM) • Uses “push” model • Traffic flooded throughout network • Pruned back where it is unwanted • Flood and prune behavior (typically every three minutes)

• Sparse-mode (SM) • Uses “pull” model • Traffic sent only to where it is requested • Explicit join behavior

S,G : Shortest path tree RP 2 *,G : Shared tree 4 5 6 7 Enterprise Source 9 3 FHR 10 8 LHR The last hop router check the routing to the RP 1 Receiver 1. Rec joins IGMP request to router. PIM *,g join towards the RP 8. If the check verifies an alternate path that is more optimal based 2. Rec state known at RP on unicast tree. 3. Source sends flow to the router 9. If the check verifies an alternate path that is more optimal based on unicast RIB. The new flow is built and upstream router gets a 4. First hop router sends a unicast register packet ( encap multicast prune. pack) to the RP 10. Flow is built to the receiver 5. Since the receiver state is maintained the RP will send a registry stop message to the FHR & (S,G) join 11. If all the receivers switch, then Prune is sent to the RP 6. S,G flow is built 7. S,G flow is built LHR and the receiver receives the flow TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 254 Multicast Forwarding

• Multicast routing is backwards from Unicast Routing • Unicast routing is concerned about where the packet goes • Multicast Routing is concerned about where the packet came from

• Multicast Routing uses “Reverse Path Forwarding”

• Membership Reports • Membership Queries • IGMP report sent by one host suppresses sending by • Queries sent to 224.0.0.1 with ttl = 1 others • One router on LAN is elected to send queries • Unsolicited reports sent by host, when it first joins the • Query interval 60–120 seconds group • Group-specific Query • Leave Group Message • Router sends Group-specific queries to make sure there • Host sends leave message if it leaves the group and is are no members present before stopping to forward the last member (reduces leave latency in comparison data for the group for that subnet to v1)

• Auto RP • BSR • Static RPs • Anycast RPs Static RPs (redundancy – No)

• Hard-coded RP address • When used, must be configured on every router • All routers must have the same RP address • RP fail-over not possible

• Exception: If Anycast RPs are used

• Command • ip pim rp-address

[group-list ] [override] • Optional group list specifies group range • Default: Range = 224.0.0.0/4 • Override keyword “overrides” Auto-RP information • Default: Auto-RP learned info takes precedence

• All routers automatically learn RP address • Candidate RPs • No configuration necessary except on: • Configured via global config command Candidate RPs • ip pim send-rp-announce scope Mapping agents [group-list acl] interval bidir

• Makes use of multicast to distribute information • Sent every 60 seconds can be tweaked • Holdtime = 3 x • Two IANA assigned Groups used Cisco-Announce - 224.0.1.39 (Candidate RP) Cisco-Discovery - 224.0.1.40 (Mapping Agent) • Mapping agents

• Permits backup RP’s to be configured • Receive RP-Announcements • Elects highest C-RP IP address as RP for group range • Can be used with admin scoping

• With ‘ip pim auto-rp listener’ feature enabled, • RP-Discovery messages contain: auto-rp can function in pim sparse mode interface • Elected RP’s from MA’s Group-to-RP Mapping Cache • This is considered a best practice • Configured via global config command • ip pim send-rp-discovery [] scope interval

• A single Bootstrap Router (BSR) is elected Candidate RPs • Multiple Candidate BSR’s (C-BSR) can be configured (for back up) • Unicast PIMv2 C-RP messages to BSR  Learns IP address of BSR from BSR messages • C-RP’s send C-RP announcements to the BSR  Sent every rp-announce-interval (default: 60 sec) • C-RP announcements are sent via unicast • Configured via global config command • BSR stores ALL C-RP announcements in the “RP-set” ip pim rp-candidate [group-list • BSR periodically sends BSR messages to all routers acl] • BSR Messages contain entire RP-set and IP address of BSR • Receive C-RP messages • All routers have the RP set and uses the same hash algorithm to  Accepts and stores ALL C-RP messages select RP  Stored in Group-to-RP Mapping Cache w/holdtimes • Messages sent using 224.0.0.13 throughout the network away from the BSR • BSR messages contain: • C-BSR with highest priority elected BSR  Contents of BSR’s Group-to-RP Mapping Cache  IP Address of active BSR  C-BSR IP address used as tie-breaker • Configured via global config command  (Highest IP address wins) ip pim bsr-candidate  The active BSR may be preempted [priority ]  New router w/higher BSR priority forces new election

• RFC 3446 “Anycast RP Mechanism . . .”

• Basic Concepts • Within a domain, deploy more than one RP for the same group range Give each RP the same IP address assignment Sources and receivers use closest RP • Use MSDP (Multicast Source Discovery Protocol) to communicate existence of Sources between RPs

Src Src

RPX 1 MSDP RP2 A SA SA B 10.1.1.1 10.1.1.1

Rec Rec Rec Rec

Src Src

RP1X RP2 A B 10.1.1.1 10.1.1.1

Rec Rec Rec Rec

RP1 MSDP RP2 A B ip pim rp-address 10.0.0.1 ip pim rp-address 10.0.0.1

C D

Interface loopback 0 Interface loopback 0 ip address 10.0.0.1 255.255.255.255 ip address 10.0.0.1 255.255.255.255

Interface loopback 1 Interface loopback 1 ip address 10.0.0.2 255.255.255.255 • “ip pim sparse-mode” should be ip address 10.0.0.3 255.255.255.255 ! enabled in all routers ! ip msdp peer 10.0.0.3 connect-source loopback 1 ip msdp peer 10.0.0.2 connect-source loopback 1 ip msdp originator-id loopback 1 ip msdp originator-id loopback 1

MSDP RP1 RP2 A B ip multicast-routing ip multicast-routing

C D

Interface loopback 0 Interface loopback 0 ip address 10.0.0.1 255.255.255.255 ip address 10.0.0.1 255.255.255.255 Interface loopback 1 Interface loopback 1 ip address 10.0.0.2 255.255.255.255 • “ip pim sparse-mode” should be ip address 10.0.0.3 255.255.255.255 ! enabled in all routers ! ip pim send-rp-announce loopback 0 scope 32 • “ip pim auto-rp listener” command ip pim send-rp-announce loopback 0 scope 32 ip pim send-rp-discovery loopback 1 scope 32 ip pim send-rp-discovery loopback 1 scope 32 ! ! ip msdp peer 10.0.0.3 connect-source ip msdp peer 10.0.0.2 connect-source loopback 1 loopback 1 ip msdp originator-id loopback 1 ip msdp originator-id loopback 1 TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 265 Multicast best practice features Control Plane Security — Constraining Auto-RP Messages

Need to Block All Need to Block All Auto RP Message from Auto-RP Message Entering or Leaving PIMv2 from Entering or Network Sparse Mode Leaving Network Border Network Border Router RP Discover RP Discover Router S0 A B S0 Neighboring MA Neighboring PIMv2 Domain PIMv2 Domain interface S0 Interface S0 . ip multicast boundary 1 ip multicast boundary 1 access-list 1 deny 224.0.1.39 access-list 1 deny 224.0.1.39 access-list 1 deny 224.0.1.40 access-list 1 deny 224.0.1.40 Prevents all Auto-RP messages from leaking between domains

(S, G) Register (unicast) RP Source

ip pim register-rate-limit 10

• Limits number of register messages per (S, G) • Limits load on DR and RP at the expense of dropping register messages that exceed limit • Receivers may experience data loss within first second in which register messages are sent from bursty sources First released in 11.3T with VRF Keyword added in 12.0(23)S & 12.2(13)T

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 268 Admission Control— Global / VRF Route Limits rtr-a> show ip mroute count ip multicast route-limit 1500 1460 IP Multicast Statistics 1460 routes using 471528 bytes of memory 404 groups, 2.61 average sources per group rtr-a

E0 Shared Tree 10.1.2.1 10.1.2.2 E0 2 PIM Join E1 rtr-b Rcvr A 10.1.4.2

%MROUTE-4-ROUTELIMITWARNING : multicast route-limit warning 1461 threshold 1460 . %MROUTE-4-ROUTELIMIT : 1501 routes exceeded multicast route-limit of 1500

The ASA firewall in single context / routed mode supports PIM. It can also have RP configuration. The RP configuration is really not recommended from the design point of view since troubleshooting and operational complexity increases. WAN router Pros . Firewall completely integrates in the multicast architecture

. Multicast Packet goes through the security policy in Firewall in a layer 3 Mode the Firewall (single context) with PIM enabled . NAT rule can apply to the multicast source servers Cons . The Firewall can only be configured in a Single Context Mode Branch inside router

In single and multiple context mode, the Firewall can pass multicast traffic through ACLs in transparent

mode WAN router WAN router

10.1.2.0/24, VLAN 20 10.1.1.0/24, VLAN 10 10.1.1.0/24, VLAN 10

Pros Routing adj Routing adj . Multicast packet will go through the and PIM and PIM neigbhors Firewall in a layer neigbhors Firewall in a security policy of the Firewall 2 Mode layer 2 Multiple context Mode . Single and multiple context Mode 10.1.1.0/24, VLAN 11 is supported 10.1.1.0/24, VLAN 11 10.1.2.0/24, VLAN 21 Cons

. Security policy/ configuration has to be dept B dept A changed to layer 2 mode Branch inside router

Multicast can be made to flow through GRE in the Firewall zone. Proper RPF checking mechanism should be used.

Pros WAN router WAN router . Can scale in single context and multiple GRE tunnel

context mode GRE tunnel GRE tunnel Routing adj Routing adj Firewall in a Firewall in a layer Cons and PIM and PIM layer 3 Mode neigbhors 3 Mode ( ACL is neigbhors ( ACL is configured to pass configured to . Design evaluation should be done on RPF the GRE traffic) pass the GRE traffic) check – the network can be made congruent or non congruent . Security policies can only be applied for the

GRE packet dept B dept A Branch inside router

• IPSEC This is a network layer 3 based solution. Encryption is achieved in layer 3 environments using IPSEC. IPSEC does not support multicast. For support of multicast, IPSEC over GRE solution is used.

Note: DMVPN replication takes place at the hub router. • GETVPN

GETVPN enables the router to apply encryption to non tunneled (that is, "native") IP multicast and unicast packets and eliminates the requirement to configure tunnels to protect multicast and unicast traffic.

. IGMP/PIM joins are signaled in control plane updates

. Each site vEdge Router chooses its desired Replicator

. Preserve standard multicast routing behavior over unicast core

Multicast control plane flow

• Source register itself to an RP • Receiver sends the (*,G) join • First Join gets forwarded to the vSmart as an OMP packet and then forwarded to the replicator • Replicator forwards (*,G) to the RP • RP forwards it to the source • Stream is forwarded to the receiver through the replicator. Stream never goes to vSmart • Once receiver has the source information, it will the join using (S,G) • First (S,G) join gets forwarded as an OMP control packet to the vSmart and then to replicator • Replicator then forward the (S,G) to the source • vEdge ignores subsequent joins and depends on the prune message to stop the stream from the replicator

. PIM-SM with Auto-RP . For cases with many receivers . Replicators can be at the source or dispersed at different geo locations

. PIM-SSM . For cases with many sources aggregating at a headend/DC site . Replicators should be defined at the receiver side . SSM mapping defined on a non-viptela device

Multicast fast convergence needs to have unicast fast convergence configured:

Case 1: With Unicast fast convergence only (traffic of 20 M unicast and 2 multicast streams) during link failure . unicast convergence is 0.324 sec . multicast convergence is 2.783 sec. Oops Multicast fast convergence configuration

Case 2: With Unicast AND Multicast Fast Convergence (traffic of 20 M unicast and 2 multicast streams) during link failure . the unicast convergence is 0.324 sec . multicast convergence is 0.512 sec. (pim query-interval & multicast rpf backoff feature)

Customer’s Point of View • Multicast Domain inside the Provider Network Blue connects each MVPN. CE Blue Multicast Domain Red CE PE Provider PE Net PE CE Blue Red

CE PE Red Multicast Domain PE PE

CE CE Red Blue

PIM on the edge

Unicast routing in overlay across MPLS mcast data Mcast signalling in overlay Mcast through core – GRE encap I have no receivers: I have receiver: I ignore I join

CE Leaf PE CE Receiver Leaf PE I have receiver: traffic rate exceeds thresholdCE I join Leaf PE

CE Receiver Leaf PE CE Leaf PE Source

Data MDT PIM message Join TLV carries C-(S,G) & P-group

PIM on the edge PIM join Unicast routing in overlay across MPLS Mcast signalling in overlay For Data MDT Group Mcast through core – GRE encap Configured on PE per VRF Range of groups

PIM join CE mcast data Leaf PE PIM Data-MDT Join TLV CE Receiver Leaf PE C-(S,G) CE P-Group Leaf PE PIM join

CE Receiver Leaf PE CE Leaf PE Source

For high rate sources, data-MDT created Removes traffic from default-MDT to offload PE’s that did not join stream

• MVPN needs to be enabled for multicast to traverse through enterprise managed MPLS layer 3 VPN cloud.

• For provider managed MPLS cloud, the enterprise routers do not need MVPN configured. The questions that requires to be asked from the provider to understand the multicast transport are :  PIM protocol support  RP propagation method support  Total number of state allowed per VRF

• Overview

PIM BGP

PIM PIM PIM PIM

Source PE PE Source PE PE MPLS cloud Receiver MPLS cloud Receiver S1,S2 S1,S2

PIM in Overlay BGP in Overlay

mLDP → PIM PIM → mLDP static map static map translation translation

PIM PIM PIM PIM

Source PE PE Source PE PE MPLS cloud Receiver MPLS cloud Receiver S1,S2 S1,S2

Static Inband

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 284 Overview of New MPLS Multicast transport options mLDP BGP MVPN Static • Multicast flow information encoded Used for advertisement of AD routes & • Uses RSVP-TE, LSPs are build from in the mLDP FEC (In-band C-mcast routes (*,G) and (S,G) the head-end to the tail-end. • Supports only P2MP LSPs signaling) • Two new extended communities for • Supports traffic engineering • LSPs are build from the leaf to the tunnel and label attribute (RFC 4271) – Bandwidth reservation root •The NLRI field in the contains the – Explicit routing • Supports P2MP and MP2MP LSPs MCAST-VPN NLRI – Fast ReRoute • “Control plane is P2MP or • P2P technology at control plane MP2MP (RFC 6826) • Data plane is P2MP Deployment Consideration: Deployment Consideration: • Deployment Consideration • Easy for SSM •Inherits P2P scaling limitations • Scalable due to receiver driven •Allows explicit or bandwidth • Complex to contraint routing tree building understand/troubleshoot for ASM • Does not support traffic •Supports Fast Reroute (FRR) engineering • Supports Fast Reroute (FRR) Newer technologies: via RSVP TE unicast backup BIER – Bit Indexed Explicit Replication (Stateless Multicast)-BRKIPM-2239 path Multicast transport without explicit tree-building protocols results in a considerable simplification.

• Multicast addresses are distinguishable from unicast addresses because they always begin with 0xFF

• Multicast addresses are all assigned out of the FF00::/8 block. Multicast addresses also have a scope associated with them. • Link Local Multicast Address- Link local multicast addresses are only intended for systems on a link and are not to be forwarded by network equipment off of that link. • Organization Multicast Address- Organizational multicast addresses are intended for use within an organization. • Global Multicast Address- Global multicast addresses are usable across the Internet • The benefits of IPv6 multicast address compare to IPv4 multicast address • Larger Addressing Space - implies the availability of plenty of addresses for multicast groups. • Addressing Scope - offers a cleaner way to contain the multicast traffic within the intended domain.

• MLD is used by IPv6 routers to discover multicast listeners (nodes that want to receive multicast packets destined for specific multicast addresses) on directly attached links.

• There are two versions of the protocol, MLDv1 and MLDv2. Similar to IGMP, MLD is built on top of ICMP. MLDv1 and MLDv2 map identically to the last two versions of IGMP, i.e. IGMPv2 and IGMPv3. • A querier is a network device, such as a router, that sends query messages to discover which network devices are members of a given multicast group. • A host is a receiver, including routers, which send report messages to inform the querier of a host membership. • MLDv2 enhances MLDv1 by enabling a node to express or combined reports interest in a particular source for a multicast group, and concatenates reporting. This capability optimizes the multicast operation through a more discrete control of group membership. This also provides support for SSM. • When “ipv6 multicast-routing “ is enabled, MLDv2 is enabled by default. Note: MLDv2 is backward compatible with MLDv1.

Static RP BSR RP ✔ ✔ PIM Anycast Embedded RP

• S11 sends multicast packet to the first hop- designated router. The Designated router will send a PIM register message to the RP1.

• RP1 is configured with RP2, 3,4 IP address as Anycast peer. Step 1 • Since the Register message did not come from one of the RPs in the anycast-RP set, the RP1 will then send a copy of the Register message to all RPs. Step 2 • In this case, this register message will use RP1s own Step 2 IP address as the source address for the PIM Register message. Step 2 • Now, in case of RP2 receives the Register message from RP1 and check the state table, since R1 are connected, the RP2 sends a Register-Stop back to RP1.

• This is state maintenance mechanism between the RPs RP1 joins the multicast PIM state for S1 by triggering a (S1,G) Join message toward S1 and (S1,G) state is created. After this RP2 also joins back to the source tree by creating S1,G join towards S1.

•IPv6 PIM provides embedded RP support. Embedded RP support allows the router to learn RP information using the multicast group destination address instead of the statically configured RP.

• Introduction • Key Elements—Multicast Branch Design • Multicast Design Components • Case Study • Key Takeaways Case Study 1: Multicast Local Scope only Branch Router Configuration

• Local Scope ONLY within the site Carrier Carrier Carrier Carrier • WAN links does not require to be PIM Uplink interface will Uplink interface enabled not require PIM will not require PIM

Site A Site 1

• PIM mode enabled on the • PIM mode enabled on the LAN interface LAN interface • Single router – no RP • Dual router – RO redundancy – use static redundancy – BSR, RP Anycast with Auto-RP or Auto-RP

Carrier Carrier Carrier Carrier Uplink interface will Uplink interface not require PIM will not require PIM

R2-Site 1

Site 1

R1-Site 1 R2- Site 1 ip multicast-routing ip multicast-routing ip multicast-routing ! ! ! interface Loopback0 interface Loopback0 interface Loopback0 ip address 3.3.3.3 255.255.255.255 ip address 4.4.4.4 255.255.255.255 ip pim sparse-mode ip pim sparse-mode ip address 1.1.1.1 255.255.255.255 ! ! interface Loopback10 interface Loopback10 ! ip address 10.10.10.10 255.255.255.255 ip address 10.10.10.10 255.255.255.255 ip pim rp-address 1.1.1.1 1 ip pim sparse-mode ip pim sparse-mode ! ! ! interface Ethernet0/0 interface Ethernet0/0 ip address 10.10.1.1 255.255.255.0 ip address 10.10.1.2 255.255.255.0 access-list 1 permit 239.192.1.0 0.0.0.255 ip pim sparse-mode ip pim sparse-mode ! ! ip pim rp-address 10.10.10.10 1 ip pim rp-address 10.10.10.10 1 ip msdp peer 4.4.4.4 ip msdp peer 3.3.3.3 ip msdp default-peer 4.4.4.4 ip msdp default-peer 3.3.3.3 ip msdp originator-id Loopback0 ip msdp originator-id Loopback0 ! ! access-list 1 permit 239.192.1.0 0.0.0.255 © 2019access Cisco-list and/or 1 permit its affiliates. 239.192.1.0 All rights 0.0.0.255 reserved. Cisco Public Case Study 2: Multicast Global and Local Scope only Branch Router Configuration

• Local Scope and Global scope within the Carrier Carrier Carrier Carrier site PIM enabled at the uplinks PIM enabled at the uplinks • WAN links require to be PIM enabled to participate in the global multicast domain • Local and Global Multicast Address • Local RP and Global RP

Site A Site 1

• PIM mode enabled on the • PIM mode enabled on the WAN & LAN interface LAN interface • Single router – no RP • Boundary to prevent the redundancy – use static local mcast address/RP to RP with Multicast be learnt in the global boundary command for domain local scope • Dual router – RP • Global Scope use Auto- redundancy – BSR, RP/Anycast ( to scope Anycast with Auto-RP or ranges) Auto-RP

Carrier Carrier . Branch router can have dual or single homed connection. The only thing PIM enabled at the uplinks that requires to be enabled for Multicast is RP information to route traffic between layer 3 segments in the branch. . If they are down stream routers “ip pim sparse-mode” should be enabled on the interface level . Single Router at the branch would have only one RP hence Static RP should be suffice. The local scope multicast address range defined in multicast addressing scheme for the local scope is 239.192.1.0 ip multicast-routing ! . The boundary command is used on an interface to block the local scope interface Ethernet0/0 multicast group from entering the WAN. The configuration command for ip address 10.1.1.1 255.255.255.0 ip pim sparse-mode static RP for local scope are: ip multicast boundary 2 ! ’ interface Ethernet0/0 The show ip mroute command shows two RP s – local scope ip address 10.1.2.1 255.255.255.0 (2.2.2.2) and global scope (1.1.1.1) learned via auto-rp. ip pim sparse-mode ip multicast boundary 2 R2#sh ip pim rp mapping ! PIM Group-to-RP Mappings interface Loopback0 ip address 2.2.2.2 255.255.255.255 Group(s) 239.1.2.0/24 ! RP 1.1.1.1 (?), v2v1 ip pim rp-address 2.2.2.2 1 Info source: 1.1.1.1 (?), elected via Auto-RP ip pim autorp listener ! Uptime: 00:07:43, expires: 00:00:07 access-list 1 permit 239.192.1.0 0.0.0.255 Acl: 1, Static access-list 2 deny 239.192.1.0 0.0.0.255 RP: 2.2.2.2 (?)

R1-Site 1 enable multicast-routing R2- Site 1 ip multicast-routing ip multicast-routing ! ! interface Loopback0 Configure the loopback used for MSDP interface Loopback0 ip address 3.3.3.3 255.255.255.255 relationship ip address 4.4.4.4 255.255.255.255 ip pim sparse-mode ip pim sparse-mode ! ! interface Loopback10 interface Loopback10 ip address 10.10.10.10 255.255.255.255 Configure the anycast RP address for the ip address 10.10.10.10 255.255.255.255 ip pim sparse-mode local branch ip pim sparse-mode ! ! interface Ethernet0/0 interface Ethernet0/0 ip address 10.10.1.1 255.255.255.0 ip address 10.10.1.2 255.255.255.0 ip pim sparse-mode ip pim sparse-mode ! Configure the boundary command to ! interface Ethernet1/0 localize the 239.192.0.0 group interface Ethernet1/0 ip address 10.10.21.1 255.255.255.0 ip address 10.10.22.1 255.255.255.0 ip multicast boundary 2 ip multicast boundary 2 ip pim sparse-mode ip pim sparse-mode ! ! ip pim rp-address 10.10.10.10 1 Ancyast / Auto-RP configuration ip pim rp-address 10.10.10.10 1 ip pim autorp listener auto-rp listener command will facilitate the ip pim autorp listener ip msdp pezer 4.4.4.4 ip msdp peer 3.3.3.3 ip msdp default-peer 4.4.4.4 branch participating in the global multicast ip msdp default-peer 3.3.3.3 ip msdp originator-id Loopback0 domain ip msdp originator-id Loopback0 ! ! access-list 1 permit 239.192.1.0 0.0.0.255 access-list 1 permit 239.192.1.0 0.0.0.255 access-list 2 deny 239.192.1.0 0.0.0.255 access-list 2 deny 239.192.1.0 0.0.0.255

• Introduction • Key Elements—Multicast Branch Design • Multicast Design Components • Key Takeaways

•Based on the application type, The Application Type •verify the number of (S,G) and (*,G) entries •PIM mode selection •Based on the application type, map multicast in the QOS architecture

•Scoping The Enterprise Multicast • RP placement Domain • PIM domain selection • PIM mode selection • QOS and Security consideration

The Access Methodology •Single link and redundant link access •Encryption requirement •Check if the branches have access through •Service provider managed MPLS cloud •Self managed VPN Multicast Requirement for •Scoping : the Branches • Extension of Enterprise domain • Local domain only • Extension of enterprise domain + local domain • Multicast protection

• Topics not covered in this presentation are: – Troubleshooting multicast – Multicast interaction with different applications – SSM and Bidir are not covered in the presentation

Abstract: Abstract: Understand the fundamental requirements • IP Multicast, Volume I thoroughly for inter-domain multicast covers basic IP multicast Design control planes for identifying source principles and routing and receiver, as well as the downstream techniques for building and control plane operating enterprise and service Support multicast transport where cloud provider networks to support service providers don’t support native applications ranging from videoconferencing to data multicast replication. Use multicast VPNs to logically separate traffic on the same physical infrastructure • Reflecting extensive experience Explore the unique nuances of multicast in working with Cisco customers, the data center the authors offer pragmatic Implement Virtual Port Channel (vPC), discussions of common features, Virtual Extensible LAN (VXLAN), and Cisco’s design approaches, deployment Application Centric Infrastructure (ACI) models, and field practices. Design multicast solutions for specific You’ll find everything from industries or applications specific commands to start-to- Walk through examples of best-practice finish methodologies: all you multicast deployments need to deliver and optimize any Master an advanced methodology for IP multicast solution. © troubleshooting2019 Cisco and/or its affiliates. large All rights IPreserved. multicast Cisco Public networks Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• Related sessions

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• Related sessions

Layer-3 Segmentation and Evolving Trends in the WAN L3 Segmentation and New Technology Trends for Next Generation WAN Designs

Craig Hill, Distinguished Systems Engineer [email protected] CCIE #1628 @netwrkr95 Agenda

• Introduction - Network Segmentation Drivers and Concepts

• Defining Key Deployment Options and Transport for L3 Segmentation WAN Designs

• Evolution and Trends for Core Backbone Designs

• Technology Deep-Dive on Options for L3 VPN Segmentation over IP

• “Innovations and Trends” Evolving in L3 Segmentation

• End to End WAN Design and Components (Summary of Session)

• Introduction - Network Segmentation Drivers and Concepts

• Defining Key Deployment Options and Transport for L3 Segmentation WAN Designs

• Evolution and Trends for Core Backbone Designs

• Technology Deep-Dive on Options for L3 VPN Segmentation over IP

• “Innovations and Trends” Evolving in L3 Segmentation

• End to End WAN Design and Components (Summary of Session)

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 308 Evolution of “Network” Segmentation …Means Many Things to Many People  • It has evolved a long way from technologies like TDM (1960’s) • From TDM, ATM/FR Virtual Circuits in the WAN, to… • VLANs in the Campus, to… Logical/Virtual Routers on routing devices, to…

• Virtual Machines on server clusters in the Data Center VPP/ Secure Domain OVS Routers Virtual Circuits MPLS Virtual VRF Lite CSR GRE HSRP MPLS VPN Port 1000v VPLS Channel VLANs AToM TDM SDN L2TPv3 Virtual Device NfV Context

Time © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 2018+ What Is Enterprise L3 “Network” Segmentation? • Giving One physical network the ability to support multiple L3 virtual networks

• End-user perspective does not change

• Maintains Hierarchy, Virtualises devices, data paths, and services Internal Separation (sales, eng) Merged Company Guest Access Network

Virtual Network Virtual Network Virtual Network

Actual Physical Infrastructure © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public Why L3 Network Segmentation? Key Drivers and Benefits • Cost Reduction - allowing a single physical network the ability to offer multiple virtual networks to tenants

• Simpler OAM—reducing the amount of physical network devices needing to be managed and monitored

• Security—maintaining segmentation of the network for different departments over a single device/Campus/WAN

• Agility – accelerates adding network segments (virtual) over same physical networks

• High Availability—leverage segmentation through clustering devices that appear as one (vastly increased uptime)

• Data Centre Applications • Offer per/multi-tenant segmentation from the DC into the WAN/campus/Branch and cloud • end-to-end continuity of Segmentation from server-to-campus-to-WAN

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 311 Why L3 Network Segmentation? L3 Network Segmentation Use Cases – Current and Evolving • Multi-Tenant Dwelling requiring Separation • Airports – (United, Delta, etc…), Government Facilities – (agencies sharing single building/campus), Intra Organisation segmentation – (sales, engineering, HR, LoB) • Company mergers – allowing slow migration for transition, overlapping addressing • IoT Device Isolation – segment (IP cameras, badge readers) from the user data

• Security for Isolation • Mandates to logically separate varying levels of security enclaves • Quarantine Zone – Honey Pot, Steered Traffic as result of DDoS, Anamoly Enforcement

• Regulation requirements • Health Care – HIPPA | Financial and Transactional – Sarbanes-Oxley, PCI Compliance

• Public Cloud and WAN Orchestration • L3 segmentation for “per tenant” environment, extend from Branch, to cloud, etc

WAN Segmentation Segmentation on/of Interconnect Device Device Pooling

WAN Si

“Virtualising” the Extending and Maintaining the “Virtualising” Routing and “Virtualised” Devices/Pools over Multiple Devices Forwarding of the Any Media to Function as a Device Single Device

Device Device Partitioning Pooling

VLANs VRFs Virtual Sw System (VSS) VNI (VXLAN) Virtual Port Channel (vPC) VDC (NX-OS) HSRP/GLBP (Virtual Device Context) Stackwise Cloud Services Router (CSR) Router Virtual Network ASR 9000v/nV Clustering ISRv Functions (VNF) w/ NFV Inter-Chassis Control IOS-XRv 64-bit Orchestration Protocol (ICCP)

Device WAN Segmentation Device Partitioning Interconnect Pooling

WAN Si

VLAN Virtual Sw System (VSS) L2 VPNs L3 VPNs VRF Virtual Port Channel (vPC) VXLAN VxLAN VRF-Lite, VRF-Lite over GRE HSRP/GLBP VDC (NX-OS) PW/VPLS MPLS L3 VPNs Stackwise (Virtual Device Context) L2 PW over IP MPLS L3 VPN over IP Inter-Chassis Control Protocol Cloud Services Router (CSR) (ICCP) IOS-XRv 64-bit OTV LISP Multi-tenant VXLAN to MPLS Integration TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 315 Agenda

• Introduction - Network Segmentation Drivers and Concepts

• Defining Key Deployment Options and Transport for L3 Segmentation WAN Designs

• Evolution and Trends for Core Backbone Designs

• Technology Deep-Dive on Options for L3 VPN Segmentation over the WAN

• “Innovations and Trends” Evolving in L3 Segmentation

• End to End WAN Design and Components (Summary of Session)

West Theater 1 Global

IP/MPLS Core Tier

Internet Cloud Public Voice/Video Mobility In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region Cloud On-Prem DC Edge -or- (CoLo)

CoLo Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

• Self Deployed MPLS Backbone Supporting MPLS BGP IP VPN

Services WAN

• Self deployed MPLS BGP IP VPNs

“over the top” of an SP Offered VPN LAN LAN transport

• Customer manages and owns: BGP MPLS IP VPN Backbone • PE, P, CE, all links for interconnect Branch CE • IP routing, provisioning, transport Site • L2/L3 VPN, TE, QoS SLA E2Eservice CE P Campus portfolio P DC Branch PE PE Site P • Customer controls service “turn up” CE Customer Owned IP/MPLS pace, Allows full control Backbone

• Requires more expertise on the

operations team Customer Owned Domain • Examples: MPLS core/edge, Segment Routing (MPLS/SRv6)

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 322 SP Offered IP VPN Transport Service (MPLS) Private IP VPN Service for Enterprise Remote Site Connectivity • CE Routers owned by customer SP Managed “IP VPN” Transport • PE Routers owned by SP

• Customer “peers” to “PE” via IP CE Site 1 SP Managed Domain • No labels are exchanged with SP PE L3 VPN CE Service Site 3 • SP advertises customer routes PE Provider PE network wide reachability Site 2 CE IP Routing Peer • Requires no management of the (BGP, Static)

core infrastructure by customer Customer Owned Customer Owned Domain Domain • Limits customers on control of services * No Labels Are Exchanged with the SP

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 323 “Our rapid change of network requirements can no longer wait 30-60 days for our service provider to modify our segmentation [VRF] requests. We need this change management to be in minutes and hours.”

Fortune 50 CIO

CE Site 1 SP Managed Domain

L3 VPN CE Service Site 3 PE Provider PE Site 2

CE IP Routing Peer (BGP, Static, IGP)

Customer Customer Managed Domain Managed Domain VRF’s

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 326 Private MPLS VPN ”over the top” of SP Offered IP VPN Transport owns CE SP Offered “IP VPN” Transport • CE customer owned, PE provider owned Extend L3 VRF Segmentation “Over the Top”

• Customer enables “PE ” CE functionality (BGP/OMP) on the CE Site 1 SP Managed Domain

• L3 VPN CE Customer Routing done “Over the Service Site 3 PE Provider PE Top” of the SP Site 2 • MPLS over GRE allows use of CE IP Routing Peer IPSec encryption (BGP, Static, IGP) Customer Customer • Scalable Solution Examples: Managed Domain Managed Domain VRF’s • Cisco SD-WAN (Viptela), MPLS VPN over mGRE, MPLS VPN over DMVPN * VPN Labels Exchanged between “CE” Devices, NOT SP

• Introduction - Network Segmentation Drivers and Concepts

• Defining Key Deployment Options and Transport for L3 Segmentation WAN Designs

• Evolution and Trends for Self Deployed MPLS Backbone Designs

• Technology Deep-Dive on Options for L3 VPN Segmentation over the WAN

• “Innovations and Trends” Evolving in L3 Segmentation

• End to End WAN Design and Components (Summary of Session)

1. Self Deployed MPLS Backbone - Supporting MPLS BGP IP VPN Services WAN

2. Self deployed MPLS BGP IP VPNs “over the top” of an SP Offered VPN LAN LAN transport

West Theater 1 Global

IP/MPLS Core Tier

Internet Cloud Public Voice/Video Mobility In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region Cloud On-Prem DC Edge -or- (CoLo)

CoLo Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

East Theater West Theater Global

IP/MPLS Core Tier 1 Tier

Internet Cloud Public Voice/Video Mobility In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region Cloud On-Prem DC Edge -or- (CoLo)

CoLo Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

3rd Wave --Evolved Programmable Network (EPN) Era, Digital 2nd Wave –MPLS Transformation Commoditisation of IP services plus high traffic growth limits profitability, 1st Wave– TDM forces architectural shift

Applications and Services TDM rigidity limits new services, Open Services Resources SDN Control forces architectural shift APIs Evolved Services Platform Open APIs

EvolvedEvolved Programmable Programmable Network Network Infrastructure TDM Era Network Function Virtualisation, Software Defined Networking, and Service Orchestration enable - Open and Dynamic - Optimal resource utilisation IP unleashes new wave of innovation and service revenues - Accelerated innovation - New services & revenues - Reduced costs - Reduced complexity

~5-10 Year Transition ~2-10 Years?

Identity: VLAN, IP address, ACL Intent-based policy, policy follow identity Command Line Interface API / Model-Driven (REST, YANG), SDN Controllers as development platforms Physical Devices Software Network Function Virtualization (NFV), and External Cabling Service Chaining, orchestration Waterfall Development (all or nothing) “DevOps” Mindset (continuous, version control) Single Vendor Mindset Open Source, open API/Models, SR/MPLS Periodic Centralized Polling Real time Telemetry, Pub/Sub, ML/AI into NetOps

Single Monolithic Images / Single Price Modular SW Packages / Pay per Use/Feature Limited Performance IP Encryption Line Rate Encryption - 10/40/100G (WAN MACsec)

✔ Simplified Core Backbone

✔ Support Massive Scale

✔ Incorporating automation, model-driven API’s, orchestration, and NFV

✔ Leverage Co-Location Facilities, Create “Cloud Edge” close proximity to apps

✔ Extend QoS and best path selection, beyond link cost (latency, jitter, loss, app)

✔ Leverage real-time model-driven telemetry collection for ML/AI benefits (security, optimized network operations (AIOps) )

✔ Support line-rate encryption (100/400G) transparent to network protocols

✔ Support new transitions – 400G, Massive Devices (IoT), 5G core requirements, Application routing

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 339 BRKRST-2124 – Introduction to Segment Routing Segment Routing 101 Tuesday, Jan 29, 11:00 AM - 12:30 PM | Hall 8.0, Session Room A101

• Simple to deploy and operate • Leverage existing MPLS forwarding, HW, and services • straight-forward ISIS/OSPF extension to distribute labels • LDP/RSVP not required • exponentially less state in the routing elements for TE • agnostic control-plane also applicable to IPv6

• Provide for optimum scalability, resiliency and virtualization • Tighter integration with application • simpler network, highly programmable

• Standards based driven The state is no longer in the network but in the packet

Path expressed in Data Plane Data the packet

MPLS IPv6 Dynamic path (segment labels) (+SR header)

Control Plane

Routing protocols with extensions SDN controller (IS-IS,OSPF, BGP)

Explicit path

Paths options

Dynamic Explicit (SPF computation) (expressed in the packet)

BRKRST-1124 – Introduction to Segment Routing

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 342 Top Use Cases Today for SR • Simplicity and complexity reduction in the core • Less protocols, reduced state, huge scale, highly programmable

• Protection with integrated TI-LFA FRR

• SR Traffic Engineering made simpler • BW optimization and capacity reaction (WAE + collection) • Disjointed paths (colored topology, SR Flex Algo) • SR-PCE (centralized SR-PCE, end-to-end awareness, multi-domain) • Low-latency services using Performance Monitoring (PM)

• SR On-Demand Next-Hops (BGP focused, SLA-aware per VPN)

• SR IGP Flexible Algorithms (next slide) • Topology defined by operator, per service Network Architecture, Topology and Capacity Trends

• Reduce/remove protocol state where you can! • simplifies troubleshooting, design, testing cycles

• Simplified Topology - Lean / Hollow Core • Core as a “Fabric” -- Any to Any becomes more feasible

• Service Segmentation • Simplified Traffic Steering for low latency or high bandwidth • Multi Planar Cores Multi-Planar Core Example • Software Router PE’s participating in IGP/BGP (ie. vRR, vPE) Many Real Deployments, 99.999+ Uptime

• High availability (5 9s+) • Fast converging (targeting now < .5 sec) • Low latency (<50ms) and low jitter for real time communication services • Unicast and multicast traffic (Layer 2 or Layer 3) • Ultra-High Scalability (thousands to 100,000+ nodes, global scale) • Converged applications on a shared network • Traffic Engineering as needed • Fault-domain isolation and service segmentation • Greater Efficiency (higher average utilization) • Secure and Programmable Infrastructure • Maintenance with little to no customer impact Next Generation SP Network Architectures: A Practical Path to Network Transformation - BRKSPG-2535 Tuesday, Jan 29, 5:00 PM - 6:30 PM | Hall 8.0, Session Room D130 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 347 Agenda

• Introduction - Network Segmentation Drivers and Concepts

• Defining Key Deployment Options and Transport for L3 Segmentation WAN Designs

• Evolution and Trends for Self Deployed MPLS Backbone Designs

• Technology Deep-Dive on Options for L3 VPN Segmentation over IP

• “Innovations and Trends” Evolving in L3 Segmentation

• End to End WAN Design and Components (Summary of Session)

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 348 Private IP VPN “Over the Top” Solution Options Global WAN Design – MPLS L3 VPN over IP East Theater West Theater Global

IP/MPLS Core Tier 1 Tier

In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region Cloud On-Prem DC Edge -or- (CoLo)

CoLo Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

• Customer may not control the WAN transport Between MPLS networks

• Cannot depend on “end to end” label forwarding for transport

• Customer requires encryption for their PE to PE MPLS traffic • No native MPLS encryption exists today, must leverage IP

• MPLS over IP allows MPLS VPN solutions to leverage cost effective IP transport

In Summary, the Implementation Strategy Described Enables the Deployment of BGP/MPLS IP VPN Technology in Networks Whose Edge Devices are MPLS and VPN Aware, But Whose Interior Devices Are Not (Source: RFC 4797)

• Segmentation component • Virtual Route Forwarding Instance (VRF)

• Control Plane component • MP-BGP (RFC 4364), Overlay Management Protocol (OMP)

• Data Plane component • MPLS over GRE/IP-UDP (RFC 4023)

• Service Support of Each Solution: QoS, IPv6, Encryption, Multicast, etc…

• State, country or Global based MPLS VPN where transport option is IP only

• Requires some ”business requirement” for segmentation (refer to L3 segmentation use cases) • Security, multi-tenant dwellings, Lines of Business segmentation,

• Campus and/or DC networks require ”policy” extension over the WAN

• Security focused customer leverages proprietary encryption devices

• Requires IP encryption, QoS/H-QoS, Multicast (MVPN), 6vPE, Inter-AS (some cases)

• Automation, management, provisioning options will be dictated by many factors (customer skill-set, interest in SDN controller, desire for “open” automation tools and/or multi-vendor support tools (Cisco NSO, “open source” tools)

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 361 WAN Segmentation Models CP: MP-BGP DP: MPLS over IP (GRE/UDP) 1. Self Deployed MPLS Backbone: Supporting MPLS BGP IP VPN Services WAN

2. Self deployed L3 VPNs: “Over the top” of SP Offered transport LAN LAN A. MPLS VPN over mGRE / DMVPN B. Cisco SD-WAN (Viptela)

GitHub Repo Location https://github.com/netwrkr95/mpls- mgre-configs Private MPLS VPN ”over the top” of SP Offered IP VPN Transport owns CE

• Offers MPLS-VPN over IP SP Managed “IP VPN” Service

• Inherit spoke-to-spoke communications MP-BGP VPNv4

• Uses standard RFC 4364 MP-BGP CE Site 1 control plane

• Uses standard MPLS over GRE data L3 VPN CE plane Service Site 3 PE Provider Site 2 PE • Offers dynamic Tunnel Endpoint next- CE hop via BGP

• Requires only a single IP address for VRF’s Customer transport over SP network Managed Domain

• Reduces configuration: Requires No mGRE Interface LDP, No GRE configuration setup GRE any-to-any

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 364 MPLS VPN over Multipoint GRE (mGRE) Control/Data Plane Example over Service Provider Model MP-BGP Signalling Internet mGRE Encapsulation Data Centre/HQ Branch Site RR Shared VRF SP IP VPN Campus DC c-PE Service C-PE mGRE mGRE VRF-Lite or MPLS c-PE = Customer PE VPN in Campus/DC Routing to SP BGP/Static BGP/Static

Enterprise Enterprise Routing Routing RR = iBGP Route • Routing and data forwarding done “Over the Top” of SP IP VPN Service Reflector • iBGP: (1) Advertise VPNv4 routes, (2) exchange VPN labels • eBGP: (1) exchange tunnel end point routes with SP (or directly connected) • Requires advertising a SINGLE IP prefix to SP (e.g. IP tunnel “end points”)

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 365 MPLS VPN over mGRE Model mGRE Interface is Dynamic and De-coupled from Physical Interfaces • System dynamically configures mGRE tunnel (via tunnel profile)

• mGRE tunnel is decoupled from physical interface

• User traffic is in VRF/VPNv4 of mGRE payload (hidden from provider)

• Only a single IP address (source GRE/BGP-source) advertised to provider

Source IP Address of • VRF, RD, RT mGRE tunnel advertised to provider network

WAN to To user Campus/DC Gold Provider networks with VRF mGRE SP WAN segmentation (802.1Q, Interface Transport port, etc…) Global PHY Blue • VRF, RD, RT Interface

Logical mGRE interface de-coupled from a physical interface

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 366 MPLS VPN over Multipoint GRE (mGRE) Feature Components PE2 2 PE3 4 iBGP View for PE 4 1 172.16.255.2 172.16.255.3 Tunnel Endpoint DB PE4 172.16.255.1 PE1 IP Transport 172.16.255.1 172.16.255.2 172.16.255.4 172.16.255.3 3 172.16.255.5 PE6 PE5 172.16.255.6 172.16.255.5 172.16.255.6 1. mGRE is a multipoint bi-directional GRE tunnel . Control Plane leverages RFC 4364 using MP-BGP Multipoint GRE 2 Interface Signalling VPNv4 routes, VPN labels, and building IP next hop (locally) 3. VPNv4 label (VRF) and VPN payload is carried in mGRE tunnel encapsulation . New encapsulation profile (see next slide) in CLI offers dynamic endpoint discovery: 4 (1) Sets IP encapsulation for next-hop (2) Installs signaled BGP peer and end-point into “tunnel endpoint database”

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 367 MPLS VPN over Multipoint GRE (mGRE) VPNv4 Configuration Example mGRE PE1 IPv4 PE4 CE2 CE1 Transport eBGP Lo0: 10.0.0.1 Lo0: 10.0.0.4 eBGP Example for PE4 interface Loopback0 ip address 10.0.0.4 255.255.255.255 ! l3vpn encapsulation ip Cisco Sets mGRE Encapsulation transport ipv4 source Loopback0 “Profile” for BGP Next-Hop ! router bgp 100 . . . address-family vpnv4 neighbor 10.0.0.1 activate Apply Route-Map to Received neighbor 10.0.0.1 send-community extended neighbor 10.0.0.1 route-map next-hop-TED in Advertisement from Remote iBGP exit-address-family . . . Neighbour ! route-map next-hop-TED permit 10 Use IP Encap (GRE) for Next-Hop and set ip next-hop encapsulate l3vpn Cisco Install Prefix in VPN Table as Connected IP Tunnel Interface TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 368 MPLS VPN over Multipoint GRE (mGRE) IPv6 (VPNv6) Configuration Example mGRE 2001:db8::2 /64 PE4 PE1 CE2 IPv4 Cloud E 1/0 CE1 eBGP Lo0: 10.0.0.1 Lo0: 10.0.0.4 eBGP Example for PE4 interface Ethernet 1/0 NOTE: Relevant MPLS VPN over mGRE vrf forwarding green Commands That Are Same for IPv4, Are Not ip address 209.165.200.253 255.255.255.224 Shown in This IPv6 Example ipv6 address 2001:db8:: /64 eui-64 ! router bgp 100 . . . IPv6 Address Applied to CE2 address-family vpnv6 Facing Interface neighbor 10.0.0.1 activate neighbor 10.0.0.1 send-community both neighbor 10.0.0.1 route-map next-hop-TED in Apply Route-Map to Received exit-address-family Advertisement from Remote iBGP . . . ! Neighbour (Same as vpnv4) route-map next-hop-TED permit 10 set ip next-hop encapsulate l3vpn Cisco set ipv6 next-hop encapsulate l3vpn Cisco Use IP Encap (GRE) for Next-Hop and Install IPv6 Prefix in VPNv6 Table as Connected Tunnel Interface

https://github.com/netwrkr95/mpls-mgre-configs Summary MPLS VPN over Multipoint GRE (mGRE) • Simple: • Only requires advertising a single IP prefix to SP for mGRE operation • Dynamic Tunnel endpoint discovery is done via iBGP/route-map (no static GRE tunnel) • Solution requires NO manual configuration of GRE tunnels. LDP NOT required! • E-BGP can/is still be used for route exchange (mGRE end-point) with the SP • Standards Based - Leverages standard MP-BGP control plane (RFC 4364) • Flexible - Supports MVPN and IPv6 per MPLS VPN model (MDT and 6vPE respectfully) • Multi-platform support: - ASR 1000 series, ISR/G2, ISR 4xxx, SUP-2T, Cloud Services Router (CSR) • Supports Inter-AS VPN, Multicast VPN (MVPN), standard QoS/H-QoS • Supports IPSec for PE-PE encryption (GET VPN or manual SA) • Scales to 2000 PE’s with ASR 1000 series

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns431/ns658/white_paper_c11-726689.pdf

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 372 Cisco Software Defined WAN (SD-WAN) for L3 Segmentation Cisco SD-WAN (Viptela) Architecture L3 Segmentation Using SD-WAN ANALYTICS Orchestration Plane MANAGEMENT

vBond API rd ORCHESTRATION 3 Party Applications Management Plane

vManage CONTROL

INTERNET MPLS 4G Control Plane

vSmart

Data Plane Data Centre Campus Branch Home Office

vEdge

IF IF MPLS • VPN 0: Transport (locked) • VPN 512: Mgmt (locked) Service Transport • VPN n: open user VPN (VPN n) (VPN0) IF IF INET

Management • VPNs enabler is VRF’s, each VRF having its (VPN512) own forwarding table IF • vEdge router allocates label to each of it’s service VPNs and advertises it as route attribute in OMP updates - VPN Labels used to identify customer VPN in the incoming packets

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 Secure Segmentation End-to-End Segmentation VPN 1 Interface VPN1 SD-WAN VPN1 Interface IPSec VPN 2 VLAN VPN2 VPN2 VLAN Tunnel VPN 3 Ingress Egress vEdge vEdge

IP UDP ESP VPN Data 20 8 36 4 …

• Segment connectivity across fabric w/o • Labels are used to identify VPN for reliance on underlay transport destination route lookup • vEdge routers maintain per-VPN routing • Interfaces and sub-interfaces (802.1Q table tags) are mapped into VPNs

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 377 Cisco SD-WAN (Viptela) L3 Segmentation (cont…) Per L3 VPN Topology and Mapping . Isolated virtual private networks across any transport . VPN isolation is carried over all transports - https://tools.ietf.org/html/rfc4023 . VPN mapping is based on physical vEdge Router interface, 802.1Q VLAN tag or a mix of both

IF Site 1

IF VPN TransportsTransports A IF VPN 802.1q B IF VPN C 802.1q Data Centre

Site 2 IPSec

IP UDP ESP VPN Data 20 8 36 4 … Label TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 Per L3 VPN Topology (Examples) Full-Mesh Hub-and-Spoke • Each VPN can have it’s own topology - Full-mesh, hub-and-spoke, partial- mesh, point-to-point, etc… VPN1 VPN2 • VPN topology is influenced by leveraging control policies - Filtering TLOCs or modifying next-hop TLOC attribute for routes Partial Mesh Point-to-Point • Customer mission, business, and applications can drive a certain topology: • Applications in single cloud or on- VPN3 VPN4 prem can benefit from hub-spoke • voice takes full-mesh topology • Security compliance - PCI data takes hub-and-spoke topology

VPN TLOC from the remote address From the controller

• Customer targeting WAN transport cost-reduction, leveraging Internet as a transport

• Customer applications distributed over different locations (public/private cloud) over various transport (Internet/4G, private MPLS)

• Desires central management, orchestration, ZTP, and embraces policy/templates

• Embraces DIA traffic steering for Internet-based SaaS (Office 365) Common Requirements

• Embraces the SD-WAN architecture, complying to SD-WAN RFP’s

• Traffic forwarding policies desired based on transport SLA with app QoE

• Desires cloud-offered controller/management suite

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 383 Summary and Positioning… What? When? Where? Summary of IP VPN WAN Techniques Strengths / Weaknesses to help evaluate decision criteria Excellent Option

SubOptimal Option MPLS VPN o mGRE Cisco SD-WAN (Viptela) R3 Bad Option Routers only (no controller req) Controller Based (central) routing calculations Native VPN Multicast (MVPN)

Application Awareness

Transport Agnostic (Internet)

Large Scale VRF (>64)

“SD-WAN” Requirement (RFP)

Per VPN Topology (p2p, mesh)

• Large-scale WAN designs need hierarchical and modular designs for scale

• There is a contrast of requirements for: • Core backbones vs. SD-WAN backhaul (details in next slide)

• Each design targets meeting specific business requirements: • Core: low-volume sites, high BW, features/functions richness • SD-WAN: high-volume sites, lower BW, use Internet for transport

• Large-scale Enterprise WAN must be modular, scalable, and inter operate with various places in the network, including in the public or private cloud

Leverage Provider Transport + Private MPLS Backbone . Leverage benefits from both solutions

L3 VPN L3 VPN 1. SD-WAN to leverage cost- Access CoLo (Equinix) Access effective remote access transport Network Network • SR Private 2. Private MPLS backbone for • Multi-planar MPLS Regional PoP control, immediate service • WAN MACsec Backbone (Co-Lo / on-prem) enablement, TE, QoS

L3 VPN . Reduces the failure domains (vs. Access L3 VPN a single large domain) Network Access Network . Inter-AS allows offering contiguous segmentation end to • Cisco SD-WAN (Viptela) end • MPLS VPN mGRE CoLo Facility • DMVPN © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 389 2-Tier Enterprise Wide MPLS Backbone Control Plane / Data Plane Usage per Domain

CONTROL-PLANE Tenant 1 1 OMP /MP-BGP OSPF / BGP OSPF / MP-BGP Tenant 2

ASR9000 Remote Office ASBR-1 ASBR-2

Remote Office SD-WAN CoLo or SR/MPLS On-Prem Transport Leverage Inter-AS Back-to- Backbone back option for extending SD- WAN Segmentation into the Remote Private MPLS VPN Network ASR9000 Office ASR9000 DATA-PLANE 2 MPLS over GRE IP / IP+GRE+IPSec MPLS

Tenant 1 vManage Tenant 2

vSmart ASR9000 Tenant-1 Controllers Back-to-back VRFs ASBR-2 ASBR-1

Internet vEdge vEdge Remote BGP, OSPF, static MPLS Backbone Office MPLS 10.1.1.0 /24 NH=ASBR-1 Cisco SD-WAN VirtualFabric vEdge ASR9000

ASR9000 2 Remote Branch Sites Private MPLS Backbone

Summary Enterprise WAN L3 Segmentation Solutions Let’s Recap

• Fully understand the application and network service requirements needed • Service turn-up times, transport available, operational expertise

• MPLS backbone target: larger-scale transport, TE, L2 VPN, tight control

• MPLS VPN over mGRE: simple MPLS VPN over IP, customer not ready for full-blown SD-WAN yet

• Cisco SD-WAN: business requirements align, applications scattered across multiple locations (on-prem, public cloud, SaaS), leverage Internet as transport, cloud managed offering is of interest

• Assure the solution chosen suits the operational skill set of the IT org

• Keep is simple whenever possible

• Introduction - Network Segmentation Drivers and Concepts

• Defining Key Deployment Options and Transport for L3 Segmentation WAN Designs

• Evolution and Trends for Self Deployed MPLS Backbone Designs

• Technology Deep-Dive on Options for L3 VPN Segmentation over IP

• “Innovations and Trends” Evolving in L3 Segmentation

• End to End WAN Design and Components (Summary of Session)

• Cloud Ready Network Design and Virtual DMZ

• Solutions for Extending L3 VPNs to AWS using Cloud Services Router (CSR)

• Leveraging High Speed Encryption in the WAN

• Leveraging automation in SD-WAN for Centralized Application Enforcement

• Majority of applications reside in on-prem DC

• WAN designs typically align Consumers (hub site resides at on- prem DC)

• Security DMZ typically co- located with apps and Providers internet connectivity in the private on-prem DC Private AWS Direct Connect Public DC Azure ExpressRoute IaaS/SaaS

Internet

• Leverage CoLo to improve proximity to cloud services; maintain application SLAs

• Single point to visualise and audit user to application relationships

• Cost Savings • Automation, PAG licensing, circuit CoLo and bandwidth savings Private Neutral Public • Placing DMZ in CoLo offers DC Facility IaaS/SaaS consistent policy enforcement, security & telemetry across multicloud Internet

vManage MANAGEMENT SaaS ORCHESTRATION

vSmart Zones Controllers Employees Co Lo Facility Internet Mobile Cisco SD- Workers MPLS WAN 4G/LTE IaaS Virtual Fabric vEdge Partners . Move enterprise WAN Hub Site (vEdge or MPLS PE) to a Carrier Neutral facility (e.g. Equinix)

Customers . Drives down circuit costs, reduces response time for apps hosted in public cloud (SaaS/IaaS)

Example shows SD-WAN. Private . Offers direct cross-connect to multitude of cloud MPLS backbone can also and internet services (AWS, Azure, Internet) leverage this with PE/P in CoLo . Option to host private ”DMZ” for security and visibility to those© 2019 Cisco services and/or its affiliates. to/from All rights reserved. enterprise Cisco Public Next Generation WAN Backbone Optimized Access, Scale, Security, Control

On-prem On-prem Data Data Center Center

•IP/MPLS Enterprise Site Enterprise Site MPLS-SR •Segment Routing Core •IPv6 SaaS PE2 SaaS

PE3 IaaS IaaS • Next Gen backbone moves WAN edge closer to applications, regardless of location CoLo Facility • Leverages MPLS-SR (lean, scalable), CoLo facilities, WAN MACsec MACsec Secured Path (high speed link security), SR-TE (path and latency control) MACsec Capable Router

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 400 Extending Enterprise Layer 3 Segmentation to Public Cloud Cisco Cloud Services Router (CSR) 1000V Cisco IOS XE Software in a Virtual Network Function Form-Factor

Software CSR 1000V • Familiar IOS XE software Infrastructure Agnostic App App • Runs on x86 platforms • Supported Hypervisors: VMware ESXi, Linux KVM, Citrix Xen, RP OS OS Microsoft Hyper-V, Cisco NFVIS (ISRv) and CSP2100 DP • Supported Cloud Platforms: Amazon AWS, Microsoft Azure Performance Elasticity Virtual Switch • Available licenses range from 10 Mbps to 10 Gbps Hypervisor • CPU footprint ranges from 1vCPU to 8vCPU License Options Server • Term based 1 year, 3 year or 5 year • PAK and Smart License enabled Enterprise-class Networking with Rapid Programmability Deployment and Flexibility • NetConf/Yang, RESTConf, GuestShell and SSH/Telnet

virtual private cloud

AWS cloud corporate office/branch

• Connect one or many physical locations in the enterprise network into an Amazon VPC. IPSec, DMVPN, FlexVPN, etc…

• Up to 1,000 concurrent VPN tunnels per CSR, and no per-tunnel charges from Amazon.

• Familiar configuration, familiar troubleshooting (i.e. not a black box feel in cloud)

• Introduce extending MPLS VPN segmentation (VRF) into AWS

VPC

Internet Enterprise Network New York

CSR – MPLS VPN over IP WAN Enterprise Network San Jose • Connect one or many physical locations into an Amazon VPC. IPSec, DMVPN, FlexVPN, EZVPN, etc…

• Up to 1,000 concurrent VPN tunnels per CSR. • Familiar configuration, familiar troubleshooting, not a black box.

VPC

Internet Enterprise Network New York

CSR – MPLS VPN over IP WAN Cisco SD-WAN Enterprise Network Virtual Fabric San Jose

• Desire to extend L3 VPN segments (per EC2) into a “single” VPC

• Extend MPLS VPN over WAN/SD-WAN to public cloud (AWS shown) Tenant/Mission 1 Tenant/Mission 2 • Leverage any transport (Internet, private MPLS VPN) and multiple connection models (AWS DX, Azure ExpressRoute) TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 407 Across regions, accounts/subscriptions Transit VPC VPC VPC VPC Shared A C Services …... • High Scale and Performance Spoke VPC • High Availability: Redundant VPN Tunnels with dynamic routing in a multi-AZ deployment

• Enterprise class routing features in CSR1 CSR2 the Transit VPC AZ1 AZ2 Transit VPC • Spoke VPC’s can leverage VGW or VPC CSRs Direct Connect • Scale-out options allow more Or Internet forwarding when needed on demand ASR Other • See BRKARC-2749 for more Provider information Networks Private DC © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Extending L3 VPN into Public Cloud Service

VPC

Internet Enterprise Network New York

CSR – MPLS VPN over IP WAN Cisco SD-WAN Enterprise Network Virtual Fabric San Jose

• Cisco SD-WAN, MPLS VPN over mGRE, and DMVPN can be leveraged Tenant/Mission 1 • Many automation options exist for VRF and/or CSR spin-up Tenant/Mission 2

BRKARC-2023 - Building Hybrid Clouds in Amazon Web Services with the CSR 1000v

BRKRST-2309 – Introduction to WAN MACsec Wednesday – 6/13, 1:30 PM Link Speeds Out-Pacing IP Encryption

• Bandwidth application requirements out-pacing IP encryption capabilities

• Bi-directional and packet sizes further impact encryption performance

• IPSec engines dictate aggregate performance of the platform (much lower throughput)

• Cost per bit for IPSec much more expensive link • Encryption must align with link speed (100G+) to support next-generation BW applications Link speed = Encryption Engine

time

• Hop-by-Hop Encryption model -Packets are decrypted on ingress port -Packets are in the clear in the device

Decrypt at Encrypt at -Packets are encrypted on egress port Ingress Egress

01101001010001001 01101001010001001 • Supports 1/10G, 40G, 100G encryption speeds 128bit AES GCM Encryption 01101001000110001001001000 everything in clear • Data plane (IEEE 802.1AE) and control plane (IEEE through the router 802.1x-Rev) MACsec PHY • Transparent to IPv4/v6, MPLS, multicast, routing • Encryption aligns with Link PHY speed (Ethernet)

128/256 bit AES GCM Encryption 128/256 bit AES GCM Encryption

01001010001001001000101001001110101 011010010001100010010010001010010011101010 01101001010001001

Encrypted Encrypted Segment Segment TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 426 BRKRST-2309 – Introduction to WAN MACsec What is “WAN MACsec? Secure Ethernet Link(s) over Public Ethernet Transport MKA Session

Service Provider • Leverage “public” standard-based OwnedEthernet Routers/Bridges transport

Data• Optimize MACsec + WAN features to accommodate running over public Ethernet Data Centre Public Carrier Centre transport Ethernet Service • TargetRemote “line-rate” encryption for high-speed applications Central Campus/DC Campus/DC• Inter DC, MPLS WAN links, massive data projects

• Targets 100G, but support 1/10/40G as well MACsec MKA Session

MACsec Secured Path / MKA Session MACsec Capable Router

MACsec Capable PHY

SP Owned Ethernet Transport Device

• AES-256 (AES/GCM) support – 1/10/40 and 100G rates • Target Next Generation Encryption (NGE) profile that currently leverages public NSA Suite B • Standards Based MKA key framework • (defined in 802.1X-2010) within Cisco security development (Cisco “NGE”) • Ability to support 802.1Q tags in clear • Offset 802.1Q tags in clear before encryption (2 tags is optional) • Vital Network Features to Interoperate over Public Carrier Ethernet Providers • 802.1Q tag in the clear • Ability to change MKA EAPoL Destination Address type • Ability to change MKA Ether-type value • Ability to configure Anti-replay window sizes • System Interoperability • Create a common MACsec integration among all MACsec platforms in Cisco and Open Standards

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 428 MACsec vs. “WAN” MACsec Support Capability MACsec WAN MACsec Data Plane Encryption AES-128 (AES-GCM) AES-128/AES-256 (AES-GCM)

1/10/40/100G AES-256/GCM No (AES-128 only) Yes

Control Plane Keying SAP (Cisco) MKA (IEEE)

802.1Q Tag in the Clear No Yes

Point to MultiPoint Topology No Yes

MKA EAPoL Tuning No Yes

MKA Ether Type Tuning No Yes

Anti Replay Window Support Limited Yes

Multi Vendor Support No Yes

• 10GE  100GE High speed Site to Site E-LINE / E-LAN - Point to Multipoint Branch n • Campus, WAN, DCDC, Metro E Branch 2 • Data Centre Interconnect

• High Speed replication and storage transfers Carrier Ethernet • IP/MPLS core/edge links (PE–P, P–P, PE–PE) Service • MPLS labels, VPN, Segment Routing is transparent to MACsec encryption Branch 1 Central • No GRE, simple. Encryption = Link BW Site • High Speed hub-and-spoke • Leverage low-cost/high-speed Metro E transport E-LINE - Point to Point • Simple configuration, no GRE tunnels Carrier Ethernet • Hybrid Encryption Design Options Service

• Ability to leverage BOTH MACsec and IPSec at various Central Central network points Site / DC 1 Site / DC 2

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 430 Hierarchical “Hybrid” MACsec + IPSec Design CSR MACsec IPsec High Throughput Encryption + Lower Scale Sites Lower Throughput Encryption + High Scale Sites

Co Lo Facility Regional IPsec Sites Hub 1 Branch Branch Internet Carrier Ethernet Service Branch Enterprise IPsec Network Branch

Internet Branch Regional Branch MPLS WAN Hub 2 (WAN MACsec) MACsec MACsec Metro E IPsec Branch Regional Hub 3 + DC • “Hybrid” design option for mix of scale, performance, leveraging Ethernet services • MACsec: Backbone/Core – Targets Higher BW, Lower Number of Sites • IPSec: Branch/back-haul – Targets Lower BW, high number of sites, cloud (CSR)

Platform Series MACsec Delivery MACsec Speed (AES- 256) ISR 4xxx Series • 1p / 2p Ethernet NIM • 1 GE ASR 1000 Series • Fixed and Modular solutions • 1GE, 10GE, * 100GE ASR 9000 Series • Modular Line Cards • 10GE, 40GE, 100GE Nexus 7700 Series ** • Modular M3 Series Card • 1/10GE, 40GE, 100GE Nexus 9000 Series • Fixed and Modular solutions • 10GE, 40GE, 100GE Optical NCS Series • Client ports • 10GE, 40GE, 100GE Catalyst Switching • Multiple Platforms C3850, Cat9K, Cat4K • 1GE, 10GE, 40GE Catalyst Switching ** • Cat 6K • 1GE, 10GE ** Currently does NOT support MKA key negotiation (SAP only) * Target Roadmap Capability TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 433 Global WAN Design – Target – WAN MACsec

East Theater West Theater Global

IP/MPLS Core Tier 1 Tier

In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region Cloud On-Prem DC Edge -or- (CoLo)

CoLo Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

BRKRST-2309 – Introduction to WAN MACsec

http://www.cisco.com/c/dam/en/us/td/docs/solutions/Enterprise/Security/MACsec/WP-High-Speed-WAN-Encrypt-MACsec.pdf

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 435 Leveraging Automation for Simplifying Network Operations SDN Solutions – Offer IT Organizations Choices Prescriptive Turn-key SDN “Open” Programmable Solutions “Open” Programmable Solutions Solutions with Cisco HW for Multi-Vendor

Prescriptive Solution “Do it Yourself” Solution

• Targets less experience in-house • Wants open-source options, with • Customer desires mixed-vendor • Requires much less operational Cisco hardware SDN and network environment expertise. • May require Cisco HW in some • Leverage open standard • Leverage open standard areas solutions (models, protocols) solutions (models, protocols)

• Require in-depth operational • Also requires in-house programmability skills in-house programming skills, and open standard data/control network

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 437 SDN Approach? Offer IT Organizations Choices Prescriptive Turn-key SDN “Open” Programmable Solutions “Open” Programmable Solutions with Cisco HW Solutions for Multi-Vendor

Prescriptive Solution “Do it Yourself” Solution

NSO vManage Cisco NSO YANG WAE vSmart Models YDK Cisco DNA XTC ACI Center • Examples: • Examples: • Examples:

• ACI (DC) • Network Service Orch • Same as Column #2 • SD-Access (campus/Branch) • YANG Models (native, open) + • SD-WAN (Cisco SD-WAN • Python (protocol libraries) (Viptela)/Meraki) • REST, RESTCONF • IP/MPLS / Segment Routing • SP WAN (NSO, WAE, XTC) • Other Tools: Ansible, Puppet, • E-VPN (BGP) / VXLAN • NFV – SP / Enterprise Chef, etc… • OpenFlow

Leveraging Simplified Centralized Policy Enforcement for Security Vulnerabilities Cisco SD-WAN – Central and Local Control

Policy Options: vSmart OMP Update: . Centralized Control . Reachability – IP Subnets, TLOCs . App-Aware routing . Security – Encryption Keys . Centralized Data . Policy – Data/App-route Policies Policies OMP OMP Update Update

vedge10 Transport 1 vedge20

TLOCs TLOCs

VPN1 VPN2 Transport 2 VPN1 VPN2 BGP, OSPF, BGP, OSPF, Connected, Connected, Static A B OMP C D Static DTLS/TLS Tunnel Subnets Subnets IPSec Tunnel

• With Centralized policies on vSmart controllers:

• Centralized Control policies affect routing policy to influence routing decisions on the vEdge routers. This type of policy allows you to set preferences for the routes or paths on the vSmart controller and is reflected in forwarding tables on the vEdge routers.

• Application-Aware routing policies select the best path for a given application based on SLA requirements. These requirements include latency, packet loss, and jitter. Application-aware routing policies are configured on vSmart controllers and are enforced by vEdge routers.

• Centralized Data policies - used for traffic classification, DSCP marking, path selection, service insertion, policing, etc. Data policies are configured on vSmart controllers and enforced by vEdge routers.

• REST API’s on vManage GET • Postman for REST API Testing PUT vManage POST MANAGEMENT

Demo vSmart 1. Use vManage

2. REST API (using Postman)

Policies Policies

vEdge 10 vEdge 20

Challenge

• The ability to rapidly block/rate-limit different traffic types in the WAN

• per box CLI does not scale and increases the “time to react”

• Suspected Vulnerabilities could be anomalies (infected) detected via third- party tools/applications

• Controlling Non-business applications also of interest (NCAA basketball during March Madness )

Solution Options

• Leverage the centralized control point (vSmart) to push policy enforcement

• Centralized pushing of “match” and “action” policy that blocks or strictly polices a specific application and/or DSCP marking

• Offer the ability for operators to leverage the GUI

• Additionally, offer API’s that allow same capability, allowing 3rd party applications, or open source tools (Ansible, Python, etc.) to trigger the enforcement

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 444 Option Demo - Dynamic Application Control policy 2 data-policy _VPN-99-List_RTBH-Spor_508995825 1 vpn-list VPN-99-List sequence 1 match app-list Suspect_Video_Apps GET ! PUT vManage POST MANAGEMENT action drop count Blocked-Video_347240515 log ! vSmart ! default-action drop ! …

Policies Policies lists app-list Suspect_Video_Apps app espn-browsing app cbs_video Modify an active policy ! vEdge 10 vEdge 20 site-list ALL-VPN-99-Router-List

https://github.com/netwrkr95 Ansible Playooks – Automate MPLS VPN VRF Deployments

• Ansible VRF Creation and Deployment Playbook (https://github.com/netwrkr95/ansible-mpls-vpn

Ansible Playbooks – MACsec Keychain Examples

• Ansible WAN MACsec Playbook and Configs (https://git.io/vQUR3 )

YANG Models – MACsec Keychain Examples (Using YDK)

• MACsec Key Chain Configuration applications (https://git.io/vH7uD )

• What is YDK? (https://developer.cisco.com/site/ydk/ )

Ansible Module Using YANG Models with YDK

• Ansible + YDK app (https://git.io/vH7XZ )

https://developer.cisco.com

• Introduction - Network Segmentation Drivers and Concepts

• Defining Key Deployment Options and Transport for L3 Segmentation WAN Designs

• Evolution and Trends for Self Deployed MPLS Backbone Designs

• Technology Deep-Dive on Options for L3 VPN Segmentation over the WAN

• “Innovations and Trends” Evolving in L3 Segmentation

• End to End WAN Design and Components (Summary of Session)

• Understanding topology, traffic patterns, VRFs, scale… is vital to choosing the “optimal” Segmentation WAN solution (self-deployed, over-the-top) • Existing L3 VPN solutions exist, but SD-WAN offers intelligent path control and future intelligence needed as apps are located in diversified locations • Consider the transition options of WAN design usage of CoLo (Equinix) as a key cloud ready WAN architecture component • L3 Segmentation solutions must be able to extend to public cloud, and leveraging API’s, as more enterprise apps move to public cloud • Embrace areas where automation and programmability can be leveraged to simplify, and innovate, in operations and deployment • Leverage WAN MACsec for high-speed encryption where applicable • Leverage the technology, but ALWAYS “Keep it Simple” when possible 

TECCRS-2500 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 450 Advanced WAN Design… Putting It All Together Enterprise-Wide WAN Design Putting it all Together East Theater West Theater Global

IP/MPLS Core Tier 1 Tier

In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region Cloud On-Prem DC Edge -or- (CoLo)

CoLo Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

. Leverage hierarchical design IP/MPLS Core Tier 1 Tier . WAN characteristics can affect your application behaviour (BW, latency, jitter)

. A QoS-enabled, highly-available network In-Theater

infrastructure is the foundation of the WAN IP/MPLS Core Tier Tier 2 architecture West Region East Region . Encryption is a key component of all WAN design topologies CoLo . Dual carrier designs provide resiliency at

multiple design tiers 3 Tier

. Leveraging CoLo and networking for Metro Metro Private Service Public Service public cloud now fundamental to WAN IP IP designs Service Service Public Cloud

East Theater West . Network design should target how Theater Global the applications survive a variation IP/MPLS Core of outages. 1 Tier . Leverage load sharing capabilities for more resiliency and application In-Theater

performance IP/MPLS Core Tier 2 Tier . End-to-end convergence time is the West Region East Region goal, and can be affected by localised topology changes CoLo . Consider IP SLA based monitoring and PfRv3 for real-time path 3 Tier

selection Metro Metro Private Service Public Service . Most effective network designs IP IP incorporate a combination of Service Service Public convergence techniques Cloud

East Theater West Theater . QoS should be considered at Global IP/MPLS Core

every tier of the WAN topology 1 Tier

. Hierarchical QoS will apply where any “sub-rate” services In-Theater

are offered IP/MPLS Core Tier 2 Tier West Region East Region . Good Multicast designs require a

solid foundational IPv4/v6 CoLo unicast network

. Multicast VPN applies to any 3 Tier

“self deployed” MPLS VPN Metro Metro services looking to leverage Private Service Public Service IP IP multicast, and still leverage Service Service

segmentation Public Cloud

East Theater West . Self Deployed MPLS VPN, Next Theater Global

Gen principles (SR, MP design) IP/MPLS Core Tier 1 Tier

. SD-WAN, MPLS VPN o mGRE

. MPLS VPN “Over the Top” In-Theater IP/MPLS Core

Solutions (Cisco SD-WAN (Viptela), 2 Tier MPLS VPN o mGRE, DMVPN) West Region East Region Internet Cloud . Cloud Ready Design, extend Public Voice/Video

segmentation to public cloud Mobility Tier 3 Tier . Look at MACsec where applicable Metro Metro Private Service Public Service . Begin to incorporate automation IP IP tools into network operations to Service Service Public simplify and error-proof Cloud

Related Demos in Walk-in Meet the sessions the Cisco self-paced engineer Showcase labs 1:1 meetings

• Please complete your Online Session Survey after each session

• Complete 4 Session Surveys & the Overall Conference Survey (available from Thursday) to receive your Cisco Live T- shirt

• All surveys can be completed via the Cisco Events Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com