Advanced Enterprise WAN Design and Deployment

Dave Fusik, David Prall, Arvind Durai, Craig Hill

TECCRS-2500 Cisco Webex Teams

Questions? Use Cisco Webex Teams to chat with the speaker after the session How 1 Find this session in the Cisco Events Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Dave David Arvind Craig Fusik Prall Durai Hill CCIE#4768 CCIE#6508 CCIE#7016 CCIE#1628 CCDE#2013::70

• 8:30 WAN Architecture and Design Principles

• 10:30 Break

• 10:45 Highly Available Wide Area Network Design

• 12:45 Lunch

• 14:30 WAN Services

• 16:30 Break

• 16:45 L3 Segmentation and Cloud Ready Solutions for the WAN

Dave Fusik

TECCRS-2500 Agenda

• Introduction

• What is Wide Area Network (WAN) Architecture and Design?

• What to consider when designing a WAN

• Impacts of Evolving technology on WAN design

• WAN Designs moving Forward

• Conclusions

• Allow the business to adapt to changes rapidly and smoothly • Shifting Markets and business models • Mergers and divestures • Regulatory and Security requirements Photo by Mikito Tateisi on Unsplash • Public perception of services

• Realize rapid strategic advantage from new • Cloud: flexible, diversified resources technologies • Software Defined Networking • Build a network that can adapt to a quickly • IPv6: global reachability evolving technology landscape • Internet of Things • 5G wireless • What’s next?

Early Networking Early-Mid 1990s Mid 1990s-Late 2000s Today Global Scale Flat/Bridged Multiprotocol Large Scale IP Ubiquity Experimental Networks Business Enabling Mission Critical Cloud Connected

Architectural Architectural Architectural Planning Lessons Lessons Lessons Protocols required for Route first, Bridge only if Redundancy Scale & Restoration must ? Build to Scale

DMVPN Frame-Relay IPv6 NFV Internet X.25 4G/LTE Protocol BGP 1960 1980 GRE 2000 Future

Metro- ARPAnet 1970 RIP (BSD) 1990 2010 Ethernet TCP/IP OSPF, Tag SDWAN ISDN, Switching GETVPN ATM TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 What is WAN Architecture and Design? WAN Architecture and Design

• Network Architecture • The way network devices and services are structured or organized to serve and protect the connectivity needs of client devices • Depending on the place in the network, the requirements and the threats vary, so different frameworks are built • In the WAN, this means connecting users to applications, between LAN locations, sometimes over long distances

• Network Design • The process of translating business needs, budget, and operational constraints into a technological approach that addresses the architectural requirements • Includes documentation, such as implementation guides and topology diagrams • WAN designs need to minimize cost and enhance user experience when serving distributed applications to distributed users

• Architecture looks toward strategy, structure and purpose • Design drives toward practice and implementation • Architecture goes nowhere without design • Design may be too singularly focused without architecture

Simplicity can often be synonymous with elegance but must be paired with functional

Modularity implies the use of building blocks that can be reused and fitted together to drive consistency

Hierarchy creates vertical flow to horizontal expansion with natural points of aggregation

These are the tools to achieve Structure

East Theater West Theater Global

IP/MPLS Core Tier1

In-Theater

IP/MPLS Core Tier2 West Region East Region

Internet Cloud

Public Voice/Video Mobility Tier3

Metro Metro Service Private Service Public IP IP Service Service

• Hierarchy—each layer has specific role • Modular topology—building blocks Core • Easy to grow, understand, and troubleshoot • Creates small fault domains— clear demarcations and isolation Aggregation • Promotes load balancing and redundancy • Promotes deterministic traffic Access patterns • Incorporates balance of both Layer 2 and Layer 3 technology, leveraging the strength of both • Utilizes Layer 3 routing for load balancing, fast convergence, scalability, and control

• No Core

• Fully-meshed distribution layers

• Physical cabling requirement Second Building Block–4 New Links • Routing complexity

4th Building Block 3rd Building Block 12 New Links 8 New Links 24 Links Total 12 Links Total 8 IGP Neighbors 5 IGP Neighbors

• Business Environment • Workforce Productivity • Market transitions • User experience • Competitive pressures • Access to resources • Project goals • Employee satisfaction • Mergers and acquisitions

• Costs • Compliance and Policy • OPEX and CAPEX • Government and Industry Regulations • Lifecycle and ROI • Security mandates • IT Capabilities • Reputation and perception • Opportunity costs

• Application requirements • Performance and Resiliency • Bandwidth, Latency, Jitter • Quality-of-Experience • Connectivity and Protocols • High Availability • L2 or L3, IPv4 or IPv6, Multicast, • Convergence and Recovery • Device quantities and capabilities • Policy and Compliance • Security • Existing Network • Segmentation Infrastructure • Encryption • Greenfield or Brownfield • Available documentation • Current designs and technologies

• Company Locations • Operational requirements • 10’s, 100’s, or 1000’s of sites • Access to resources • Where in the world • Transport options • Site diversity • Available power • retail store, campus, large • Size and quantity of equipment manufacturing plant, etc.

• Risks associated with the • Topology Implications Business and Technical • Single or dual connected requirements • Geographical dispersity • Local, Regional, Global • Network role • Data Center, Colo Facility, Branch, Remote access, Public/Guest access

• Assess system criticality • How to measure availability • Eliminate single points of failure • Failure detection and recovery • Environmental conditions

• In principle, redundancy is easy

• Any system with more parallel paths through the system will fail less often

• The problem is a network isn’t really a single system but a group of 2.5 interacting systems

• Increasing parallel paths increases routing complexity, therefore

increasing convergence times Seconds

0 Routes 10000

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 Current and Evolving Technologies that impact WAN design WAN Locations and Devices • Organization sites • Headquarters Campus • Branch Office • Retail store • Factory, etc.

• Remote Access • Mobile workers • Home office

• Cloud • Private Data Center • Physical devices • Virtualized Network • Public IaaS • Router/CPE Functions • SaaS • Firewall • Virtual router • Colocation Facility • Multi-purpose compute • Virtual Firewall • Client devices • etc…

ISR 900 ISR 1000 ISR 4000 ASR 1000

• WAN and voice module • Fixed and fanless • Integrated wired and flexibility • Hardware and software wireless access redundancy • IOS Classic based • Compute with UCS E • PoE/PoE+ • High-performance service with • Integrated Security stack hardware assist • WAN Optimization • Fixed Chassis

vEdge 100 vEdge 1000 & 2000 vEdge 5000 SD

• 4G LTE & Wireless • Modular - • Fixed/Pluggable Module WAN • RPS Virtual and Cloud

• Service chaining virtual CSR 1000V • Cisco DNA virtualization functions ISRv Cisco ENCS • Extend enterprise routing, • Options for WAN connectivity vEdge Cloud security & management to • Open for 3rd party services & cloud apps

Software Performance Elasticity Same IOS XE software as the Available licenses range from ASR1000 and ISR4000 10 Mbps to 10 Gbps CSR 1000V App App CPU footprint ranges from Infrastructure Agnostic 1vCPU to 8vCPU Runs on x86 platforms OS OS Supported Hypervisors: Programmability Virtual Switch VMware ESXi, RHEL Linux KVM, NetConf/Yang, RESTConf, Guest Suse Linux KVM, Citrix Xen, Hypervisor Shell and SSH/Telnet Microsoft Hyper-V, Cisco NFVIS and CSP5000 Server License Options Supported Cloud Platforms: Term based 1 year, 3 year Amazon Web Services, Microsoft or 5 year Azure, Google Cloud Platform

Enterprise-class networking with rapid deployment and flexibility

Software Performance Same software as the physical Available licenses range from vEdge router platforms 10 Mbps to 100 Mbps

Infrastructure Agnostic CPU footprint minimum 2vCPUs Runs on x86 platforms Supported Hypervisors: Positioning VMware ESXi, RHEL Linux KVM, Extends SD-WAN Overlay into Suse Linux KVM, Citrix Xen, Cloud Environments Microsoft Hyper-V, Cisco NFVIS and CSP5000 License Options Supported Cloud Platforms: Term based 1 year, 3 year Amazon Web Services, Microsoft or 5 year Azure, Google Cloud Platform

Enterprise-class networking with rapid deployment and flexibility

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 Branch/Campus Platform Built for Enterprise NFV Colocation Center ENCS 5000 Series for the Branch Public Cloud Best of Routing Complete Open for Third Party & Compute Virtualized Services Services and Apps

Enterprise Network Compute System

ENCS 5100 Series

8 Integrated LAN Ports ENCS 5400 Series with Optional POE USB 3.0 Storage 2 Onboard Gigabit Network Interface Hardware Acceleration for Ethernet ports Module for LTE & legacy 2 HDD or SSD VM Traffic with SFP WAN RAID 0 & 1

Enterprise Network Compute System

Before After

Branch router

IPS/IDS appliance

WAAS appliance Patch panel N F V I S Firewall appliance N F V I S

A single x86 compute platform housing multiple VNFs

Integrated Services Router - Virtual Cloud Services Router

Packaged for NFVIS Cloud and VDC Deployments Branch-Specific Features Aggregation Use-Cases Branch-Specific Pricing Flexible Pricing & Packaging Look-and-feel of an ISR 4000 Virtual ASR 1000 Series Not available separately Available on multiple platforms

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 WAN Connection and Transport Technologies • Dark Fiber • Highest flexibility, control, and security but only point-to-point connectivity • Most costly unless owned by the organization

• MPLS • Broadband • Widely available service with flexible bandwidth • Lower cost, high bandwidth Internet connectivity options • Organization manages a secure overlay VPN • Provider manages complex WAN routing with QoS between sites but has no control over latency or QoS SLAs • Available as wired (DSL, Cable) or wireless • Offers simplicity with global scale if the organization (3G/4G/5G or satellite) can afford it • Legacy T1 • Metro Ethernet • Last resort option but available anywhere • Layer 2 Ethernet connectivity service between up to hundreds of locations within a specific geographic • Cost comparable to Metro Ethernet but only 1.5Mbps region bandwidth • Organization manages its own routing and QoS • Point-to-point layer 2 connectivity and requires non- policies but may offer higher bandwidth at less cost Ethernet type port on router than MPLS

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 MPLS VPN Models CE = Customer Edge router Technology Options PE = Provider Edge router MPLS Layer-2 VPNs MPLS Layer-3 VPNs

•CE connected to PE via IP-based Point-to-Point Layer-2 VPNs Multi-Point Layer-2 VPNs connection (over any layer-2 type) –Static routing •CE connected to PE via •CE connected to PE via L2 connection (Eth, FR, Ethernet connection –PE-CE routing protocol; ATM, etc.) •CE-CE L2 (Eth) mp eBGP, OSPF, IS-IS •CE-CE L2 p2p connectivity •CE has peering relationship with PE connectivity •CE-CE routing •PEs participate in customer routing •CE-CE routing •No SP involvement •PEs maintain customer-specific routing •No SP involvement tables

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 Broadband Internet • Widely available in wired or wireless • Wired is generally an Ethernet handoff • High bandwidth to the Internet so creates security vulnerability that must be managed • Provides access to Public Cloud services such as IaaS and SaaS • Does not support QoS or Multicast • IPSec secure connections for private enterprise communication but this restricts some services • Overlay IP Encapsulation with IPSec creates a secure VPN tunnel between Enterprise locations • No service guarantee for critical applications but offers a low cost backup or bandwidth augmentation option

Layer 2 Overlays Layer 3 Overlays ▪ Virtual Extensible LAN (VXLAN) ▪ IPSec—Encapsulating Security Payload (ESP) – MAC-in-UDP encapsulation – Strong encryption – 24-bit segment ID for up to 16M – IP Unicast only logical networks ▪ Generic Routing Encapsulation (GRE) ▪ Other L2 overlay technologies – IP Unicast, Multicast, Broadcast – MPLS-over-GRE/mGRE, L2TPv3, – Multiprotocol support OTV ▪ Other L3 overlay technologies – MPLS-over-GRE/mGRE, LISP

IP HDR IP Payload

GRE packet with new IP header: Protocol 47 (forwarded using new IP dst) IP HDR GRE IP HDR IP Payload

20 bytes 4 bytes

IPSec Transport mode 2 bytes ESP ESP IP HDR ESP HDR IP Payload Trailer Auth 20 bytes 30 bytes Encrypted AuthenticatedAuthenticated IPSec Tunnel mode 2 bytes ESP ESP IP HDR ESP HDR IP HDR IP Payload Trailer Auth 20 bytes 54 bytes Encrypted Authenticated

• Single Carrier Designs • Enterprise homes all sites to a single MPLS VPN carrier for L3 connectivity • Simple design with consistent features • Bound to single carrier for feature velocity • Vulnerable to MPLS cloud failure scenario

• Dual Carrier Designs • Enterprise single/dual homes sites into one/both MPLS VPN carriers • Protection against full MPLS cloud failure • Leverage for competitive services pricing • Complexity from service differences between carriers (QoS, BGP AS, etc.) • Must settle for least common denominator features

• Hybrid and Overlay Designs • Tunneling/encryption enables transport agnostic design + On-demand or permanent backup links + Commodity broadband services offer lower cost, higher bandwidth + Flexible overlay topology independent of physical underlay connectivity

− Two “layers” to support Internet Internet Secure Overlay Secure Overlay − SLA over commodity transport services − Must consider potential for fragmentation

Internet Internet Internet Internet Internet Secure Overlay Secure Overlay Secure Overlay Secure Overlay Secure Overlay 1 2 1 2

Features DMVPN FlexVPN GET VPN ▪ Public or Private Transport Infrastructure ▪ Public or Private Transport ▪ Private IP Transport ▪ Overlay Routing ▪ Overlay Routing ▪ Flat/Non-Overlay IP Routing Network ▪ IPv4/IPv6 dual Stack ▪ Large Scale Hub and Spoke ▪ Converged Site to Site and ▪ Any-to-Any; Network Style with dynamic Any-to-Any Remote Access (Site-to-Site)

▪ Dynamic Routing or IKEv2 Failover ▪ Active/Active based on ▪ Transport Routing Route Distribution Dynamic Routing ▪ COOP Based on GDOI Redundancy ▪ Server Clustering

▪ Unlimited ▪ Unlimited ▪ 8000 GM total Scalability ▪ 3000+ Client/Server ▪ 3000+ Client/Server ▪ 4000 GM/KS

▪ Multicast replication in IP ▪ Multicast replication at hub ▪ Multicast replication at hub IP Multicast WAN network

▪ Per SA QoS, Hub to Spoke ▪ Per Tunnel QoS, Hub to Spoke ▪ Transport QoS QoS ▪ Per SA QoS, Spoke to Spoke ▪ Centralized Policy ▪ Central or Local ▪ Locally Managed Policy Control Management Management ▪ Tunneled VPN ▪ Tunneled VPN ▪ Tunnel-less VPN Technology ▪ Multi-Point GRE Tunnel ▪ Point to Point Tunnels ▪ Group Protection ▪ IKEv1 & IKEv2 ▪ IKEv2 Only ▪ IKEv1 & IKEv2

• Bandwidth application requirements out- pacing IP encryption capabilities

• Bi-directional and packet sizes further impact encryption performance

• IPSec engines dictate aggregate link performance of the platform (much lower BW throughput) Link speed = Encryption speed • Cost per bit for IPSec much more expensive time • Encryption must align with link speed Link Speed (100G+) to support next-generation IPSec Encryption Speed applications

• Hop-by-Hop Encryption model -Packets are decrypted on ingress port -Packets are in the clear in the device Decrypt at Encrypt at -Packets are encrypted on egress port Ingress Egress 01101001010001001 01101001010001001 • Supports 1/10G, 40G, 100G encryption speeds 128bit AES GCM Encryption 01101001000110001001001000 everything in clear • Data plane (IEEE 802.1AE) and control plane (IEEE through the router 802.1x-Rev) MACsec PHY • Transparent to IPv4/v6, MPLS, multicast, routing • Encryption aligns with Link PHY speed (Ethernet)

128/256 bit AES GCM Encryption 128/256 bit AES GCM Encryption

01001010001001001000101001001110101 011010010001100010010010001010010011101010 01101001010001001

Encrypted Segment Encrypted Segment

• Giving One physical network the ability to support multiple L3 virtual networks

• End-user perspective does not change

• Maintains Hierarchy, Virtualizes devices, data paths, and services

Internal Separation (sales, eng) Merged Company Guest Access Network

Virtual Network Virtual Network Virtual Network

Actual Physical Infrastructure

• Logical routing context within the same PE device

• Unique to a VPN

• Allows for customer overlapping IP addresses

• Deployment use cases • Business VPN services • Network segmentation • Data Center access

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 Why L3 Network Segmentation? Key Drivers and Benefits • Cost Reduction • Allowing a single physical network the ability to offer multiple virtual networks to tenants

• Simpler OAM • Reducing the physical network devices that • High Availability need to be managed and monitored • Leverage segmentation through clustering • Security devices that appear as one (vastly increased uptime) • Maintaining segmentation of the network for different departments over a single • Data Center Applications device/Campus/WAN • Offer per/multi-tenant segmentation from the • Agility DC into the WAN/campus/Branch and cloud • End-to-end Segmentation from-server-to- • Accelerates adding network segments (virtual) over same physical networks campus-to-WAN

Voice SD Video Conf Telepresence Data

▪ Smooth ▪ Bursty ▪ Bursty ▪ Smooth/bursty ▪ Benign ▪ Greedy ▪ Drop sensitive ▪ Benign/greedy ▪ Drop sensitive ▪ Drop sensitive ▪ Delay sensitive ▪ Drop insensitive ▪ Delay sensitive ▪ Delay sensitive ▪ Jitter sensitive ▪ Delay insensitive ▪ UDP priority ▪ UDP priority ▪ UDP priority ▪ TCP retransmits

Bandwidth per call SD/VC has the same HD/VC has tighter req’s Traffic patterns for Data depends on codec, requirements as VoIP, than VoIP for jitter and vary across applications Sampling-Rate, and but traffic patterns and BW varies based on Layer 2 Media BW varies greatly the resolutions Data Classes: • Latency ≤ 150 ms • Latency ≤ 150 ms • Latency ≤ 200 ms • Mission-Critical Apps • Jitter ≤ 30 ms • Jitter ≤ 30 ms • Jitter ≤ 20 ms • Transactional/Interactive Apps • Loss ≤ 1% • Loss ≤ 0.05% • Loss ≤ 0.10% • Bulk Data Apps • Bandwidth (30-128Kbps) • Bandwidth (1Mbps) • Bandwidth (5.5-16Mbps) • Best Effort Apps (Default) • One-Way Requirements • One-Way Requirements • One-Way Requirements

Relevant Business as usual Not Important

• Needed to support the • May or may not support • Consumer oriented core business objective business objectives directly traffic type

• Applications should be • The traffic can be grouped • Treated less than understood, marked and to qos class queues with best class effort treated in accordance to proper marking or just tied best practice to single qos class or default queues

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 46 QoS Tools and Techniques Classifying and Marking Scheduling • Network Based Application Recognition • Re-order and selectively drop during (NBAR2) congestion • Application Visibility and Control (AVC) • Class Based Weighted Fair Queuing (CBWFQ) • Layer 2 or 3 marking of CoS/EXP or DSCP/IP • Low Latency Queuing (LLQ) and Multi-LLQ precedence Link-specific tools • Traffic Shaping and Hierarchical QoS (HQoS) • Compression • Fragmentation and Interleaving

Policing and Markdown • Define traffic metering contracts • Markdown out-of-contract flows • Conform, Exceed, Violate actions

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 IP Multicast in the Enterprise WAN • IPs: 224.0.0.0 – 239.255.255.255 • L2 WAN transport allows Enterprise to fully manage the Multicast domain • Group destination IP, never a source • Can operate in Overlay but may • Single source transmission efficiently require head-end replication limiting delivered to a group of receivers overall efficiency • Protocol-Independent Multicast (PIM) relies on unicast routing to Unicast build a loop-free, hop-by-hop, path Source Receiver • PIM must be enabled along the entire end-to-end path Receiver • Not supported over the Internet Multicast • Service Providers offer MPLS VPN Source Receiver with Multicast capabilities Receiver

• Perimeter security required at all Security Tools Enterprise Internet connections points ✓ Firewalls • Private connections (eg. MPLS) provide ✓ Intrusion Prevention a relative level of security ✓ Visibility • Backhauling Internet traffic to data ✓ URL Filtering centers with appropriate perimeter ✓ Advanced Malware security creates latency, congestion, Protection and cost ✓ DNS Security • Deploying perimeter security at every ✓ Transport Security location for DIA even more costly and ✓ DDoS Protection difficult to manage ✓ etc… • The goal is a single security policy enforce across the entire WAN

• Complexity & Dependency - Need a simple and scalable way to securely extend the private network across Multicloud environments

• Inconsistent security policies between private & public - Need to apply consistent security policies

• Degraded application performance and ambiguity for best path to reach the cloud – Need to enhance application experience

• CSR deployed in • CSR deployed in • Add pair of CSRs to scale out application VPC dedicated Transit Hub • Remote end (VGW) has multiple • Provide IPsec gateway • High speed traffic tunnels and do L3 ECMP (Equal for entire VPC routing for spoke VPC Cost Multiple Path) • Need high availability • High availability is built- • Monitors CSR real-time throughput in natively and spin up new CSRs on demand

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 Connecting to Public Cloud Internet IPSec tunnel DX / ER to Public Direct Connect to Public connection from DC Cloud through SP Cloud through co-locations

Branch Branch

Branch Branch SP Internet MPLS SP Internet Data Center Carrier PE Colocation Facility Internet IPSec DX / ER DX / ER

VPC/VNet VPC/VNet VPC/VNet VPC/VNet

IPsec Tunnel MPLS carriers (L3 VPN Internet only for DX/ER from the co- from customer carrier) offers DX/ER as connectivity location to the cloud DC to the cloud SP Managed Service

Design Challenges with Growing Needs and New Innovation

Internet Internet Secure Overlay Secure Overlay

3G/4G/5G Secure Overlay

Internet Internet Internet Internet Internet 3G/4G/5G 3G/4G/5G Secure Overlay Secure Overlay Secure Overlay Secure Overlay Secure Overlay Secure Overlay Secure Overlay 1 2 1 2

Complexity Grows with Scale and Changing Business Requirements

• Some traffic, not all, destined to SaaS, IaaS (e.g. O365, Salesforce.com, or Azure) is critical

• Includes regular browsing traffic from each location

• MPLS can be an expensive conduit to a centralized Internet breakout point

• Enterprise pays for private bandwidth and then again for Internet bandwidth

• This change in traffic impacts capacity planning, application performance, and ultimately user satisfaction

• Major challenge to use traditional WAN features to deliver a cohesive solution and to troubleshoot

3rd Wave – EPN Evolved Programmable Network Era, 2nd Wave – MPLS Digital Transformation Commoditization of IP services plus high traffic growth limits profitability, 1st Wave – TDM forces architectural shift

Applications and Services TDM rigidity limits new services, Open APIs Services Resources SDN Control

forces architectural shift Evolved Services Platform Open APIs

EvolvedEvolved Programmable Programmable Network Network Infrastructure TDM Era Network Function Virtualization, Software Defined Networking, and Service Orchestration enable - Open and Dynamic - Optimal resource utilization IP unleashes new wave of innovation and service - Accelerated innovation revenues - New services & revenues - Reduced costs - Reduced complexity

~5-10 Year Transition ~2-10 Years?

Unlock the Power that Leverage the Enable Network Wide Exists Power of Existing Fidelity to an Expressed in the Network through Distributed Systems Intent (Policy) Abstraction, Automation, and Policy Enforcement

Cloud Service Management Automation Open and Assurance

Automation Analytics

Security and Programmable Principles Virtualization Compliance

Programmable Physical and Virtual infrastructure API Driven Insights and Experiences

Security

CiscoCloud Service vManage Management Automation Open and Assurance

Automation Analytics

Security and Programmable Principles SD-VirtualizationWAN Compliance

Programmable Physical and Virtual infrastructure API Driven Insights and ASR1k/ISR4k/vEdge Experiences

Embedded PolicySecurity Enforcement

Hardware Centric Software Driven

Manual Automated

Closed Programmable

Reactive Predictive

Network Intent Business Intent

CLOUD & ON-PREM AUTOMATION & SCALE SECURITY & COMPLIANCE ASSURANCE & ANALYTICS Hosted, delivered, managed Speed, flexible, zero-touch, Segmentation, Users, applications, devices policy driven threat mitigation

Analytics Application Traffic Per-Segment Secure Cloud Path Cloud Accel Transport SLA Engineering Topologies Perimeter (IaaS) (SaaS) Hub APPLICATION POLICIES

Monitoring Routing Security Segmentation QoS Multicast Svc Insertion Survivability SERVICES DELIVERY PLATFORM

Operations Broadband MPLS Cellular

ZERO TOUCH ZERO TRUST TRANSPORT INDEPENDENT FABRIC

Secure Application • Security Elastic ApplicationsQoE Connectivity Services • Connectivity

• Application Services Agile ConnectivityCloud Operations First Operations • Operations

Embedded Security Secure Bring-up

Security ApplicationsApplication Scalable Data-Plane Centralized Device Services Encryption Auth-DB

ConnectivityConnectivity Operations

Authenticated/Encrypted Control Plane Automatic Key Rollover

Provider/Transport Hybrid WAN Agnostic

LTE LTE

INTERNET INTERNET

MPLS MPLS Security ApplicationsApplication Services

Dynamic Per-VPN Segmentation/VPNs ConnectivityConnectivity Operations Topologies

Deep Packet Inspection Central Orchestration App Fingerprinting

DPI Engine

Transport SLA Monitoring Application Layer Security ApplicationsApplication Analytics LTE Services

INTERNET

MPLS

ConnectivityConnectivity Operations Cloud Services Application-Aware Integration Routing

SEN Overlay

Applications Template-based Security Application Zero Touch Provisioning Configurations Services

Programmatic APIs ConnectivityConnectivity Operations Open Object Model NetConf Ad-Hoc Adds/Moves/Changes

vManage Orchestration Plane vBond vSmart

MANAGEMENT vBond

Management Plane API vEdge (Multi-tenant or Dedicated)

ORCHESTRATION ANALYTICS

Control Plane (Containers or VMs)

CONTROL

Secure DTLS Control Channel Secure IPSEC Data Channel INET MPLS 4G

Data Plane (Physical or Virtual)

Data Center Campus Branch Home Office

Private Cloud Site Enterprise Controllers Virtual Private Cloud SaaS

App Servers

Servers SDWAN VPC VPC Headend VPC VPC Distro Switch

V V CE Routers

MPLS1 INET

V = Virtual Router

Single Dual Router Legacy Router Branch Branch Branch

Deploy Deploy Deploy

vManage vManage vManage Recommended

DTLS DTLS DTLS Or TLS Or TLS Or TLS Connections Connections Connections

vSmart vBond vSmart vBond vSmart vBond Cisco MSP Private Cloud Cloud Cloud

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 71 Cisco SDWAN Migration Strategy Gateway/DC Site Deployment ▪ Identify Gateway/DC Sites providing connectivity between BGP/OSPF SD-WAN and legacy sites

▪ Legacy sites talk to each other DC/Gateway Site directly

▪ SD-WAN sites talk to each other directly Internet SD-WAN MPLS Secure Fabric ▪ Legacy router/connectivity is dropped in the DC/Gateway sites once migration is complete

Legacy/MPLS Sites SD-WAN Sites

Cloud

Customers Cloud CloudonRamp onRamp Cloud orFor SAE Colo onRamp Colocation For Centers Colo

Employees

Security Agility & Performance Cost Savings Partners DMZ Central policy Rapid provisioning, Lower OpEx and Private Applications enforcement change control, scaling via CapEx through NFV. Data Center NFV fabric - Speed of Reduce circuit costs software with the and number of performance of hardware circuits.

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

Campus / Branch Campus / Branch Secure Mobile Secure Mobile

Early Networking Early-Mid 1990s Mid 1990s-Late 2000s Today Global Scale Flat/Bridged Multiprotocol Large Scale IP Ubiquity Experimental Networks Business Enabling Mission Critical Cloud Connected

Architectural Architectural Architectural Planning Lessons Lessons Lessons Protocols required for Route first, Bridge only if Redundancy Scale & Restoration must ? Build to Scale

DMVPN Frame-Relay IPv6 NFV Internet X.25 4G/LTE Protocol BGP 1960 1980 GRE 2000 Future

Metro- ARPAnet 1970 RIP (BSD) 1990 2010 Ethernet TCP/IP OSPF, Tag SDWAN ISDN, Switching GETVPN

Early Networking Early-Mid 1990s Mid 1990s-Late 2000s Today Global Scale Flat/Bridged Multiprotocol Large Scale IP Ubiquity Experimental Networks Business Enabling Mission Critical Cloud Connected

Architectural Architectural Architectural Architectural Lessons Lessons Lessons Lessons Protocols required for Route first, Bridge only if Redundancy Optimize for application Scale & Restoration must experience Build to Scale SDN delivers agility Central policy enforcement DMVPN Frame-Relay IPv6 NFV Internet X.25 4G/LTE Protocol BGP 1960 1980 GRE 2000 Future

Metro- ARPAnet 1970 RIP (BSD) 1990 2010 Ethernet TCP/IP OSPF, Tag SDWAN ISDN, Switching GETVPN

Backhauled Access Distributed Access Optimized Access

SaaS IaaS Extranet SaaS IaaS Extranet SaaS IaaS Extranet

Data Center Data Center Data Center Data Center Data Center Data Center Cloud onRamp for Colo

MPLS MPLS Internet

MPLS Internet

East Theater West Theater Global

IP/MPLS Core Tier1

In-Theater

IP/MPLS Core Tier2 West Region East Region

Internet Cloud

Public Voice/Video Mobility Tier3

Metro Metro Service Private Service Public IP IP Service Service

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

Campus / Branch Campus / Branch Secure Mobile Secure Mobile

• The goal is for a simple, modular, hierarchical, structured design

• Business, technical, and physical requirements and constraints must all be considered

• Desired WAN availability and services have design implications

• Evolving technology is driving new WAN designs

• Leveraging Internet, Cloud, and CoLo now fundamental

David Prall, Systems Architect CCIE #6508

TECCRS-2500 Agenda

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment

• Efficiently utilize available bandwidth

• Dynamically respond to all types of disruptions

• Leverage most effective design techniques that meet the design requirements

• Review today’s technology

Link or Device Failure

MPLS - SP A

C-A-R2 Link or Device Degraded

C-A-R1 C-A-R4

C-A-R3

HQ-W1 BR-W1

MPLS - SP B HQ-W2 BR-W2

C-B-R1 C-B-R4

• How does outage manifest? • How quickly can network detect? • How long is bidirectional reconvergence? TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 85 Session Scope

• What methods are used for path selection and packet forwarding

• How does the network detect outages

• Focus on network survivability and effective utilization rather than sub-second convergence

• Modern Design using SD-WAN

• Does not address “zero loss” considerations • Please review BRKRST-2365 Unified HA Network Design - The Evolution of the Next Generation Network • Other sessions delivered by Matt Birkner

Availability Downtime / Year • System Availability: a ratio of the expected uptime to the experienced 98.000000% 7.3 Days downtime over a period of the same 99.000000% 3.65 Days duration 99.500000% 1.825 Days 99.900000% 8.76 Hrs • Branch WAN High Availability: Between 99.99%(4) and 99.999%(5) 99.990000% 52.56 Min Branch WAN 99.999000% 5.256 Min HA Targets • Ultra High Availability: Between 99.999900% 31.536 Sec 99.9999%(6) and 99.999999%(8) 99.999990% 3.1536 Sec Ultra HA 99.999999% .31536 Sec Targets

Downtime SINGLE per Year 99.95%* Downtime Downtime 99.90%* per Year per Year ROUTER, MPLS 4 Hours 8 Hours Internet SINGLE PATH 4–9 Hours 22 Minutes 46 Minutes ISR ISR Branch WAN HA Solution 99.995% 99.995% 99.995% SINGLE ROUTER, 26+ Minutes DUAL PATHS MPLS MPLS MPLS Internet Internet Internet

ISR ISR ISR

99.999% 99.999% 99.999%

DUAL ROUTERS, 5+ Minutes Internet Internet DUAL PATHS MPLS MPLS MPLS Internet

ISR ISR ISR ISR ISR ISR

* Typical MPLS and Business Grade Broadband Availability SLAs and Downtime per Year, calculated with Cisco AS DAAP tool. TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 88 Redundancy and Path Diversity Matter

MPLS/MPLS Internet/Internet 100’s of Combinations MPLS/Internet Internet/LTE Dual Path’s Minimum MPLS/LTE LTE/LTE

Branch WAN HA Solution 99.995% 99.995% 99.995% SINGLE ROUTER, 26+ Minutes DUAL PATHS MPLS MPLS MPLS Internet Internet Internet

ISR ISR ISR

99.999% 99.999% 99.999%

DUAL ROUTERS, 5+ Minutes Internet Internet DUAL PATHS MPLS MPLS MPLS Internet

ISR ISR ISR ISR ISR ISR

• Introduction

• Cisco IOS and IP Routing • Multiple Links/Multiple Paths • Load Sharing

• Convergence Techniques

• Design and Deployment

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 90 Routing Table Basics Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override, p - overrides from PfR p 10.0.0.0/8 is variably subnetted, 14 subnets, 5 masks B p 10.0.0.0/8 [20/0] via 172.16.0.6, 00:12:36 B p 10.3.0.0/16 [20/0] via 172.16.0.6, 00:12:36 B p 10.4.0.0/16 [200/0], 00:13:52, Null0 C p 10.4.0.41/32 is directly connected, Loopback0 D p 10.4.1.0/24 [90/307200] via 10.4.49.2, 00:14:32, Ethernet0/0 C p 10.4.49.0/30 is directly connected, Ethernet0/0 L p 10.4.49.1/32 is directly connected, Ethernet0/0 B p 10.9.0.0/16 [20/0] via 172.16.0.6, 00:12:36 100.0.0.0/8 is variably subnetted, 9 subnets, 2 masks B 100.64.0.0/24 [20/0] via 100.64.3.1, 00:13:43 C 100.64.3.0/24 is directly connected, Ethernet0/2 L 100.64.3.2/32 is directly connected, Ethernet0/2 172.16.0.0/16 is variably subnetted, 9 subnets, 2 masks B 172.16.0.0/31 [20/0] via 172.16.0.6, 00:12:36 C 172.16.0.6/31 is directly connected, Ethernet0/1 L 172.16.0.7/32 is directly connected, Ethernet0/1

Default Route Source Distance • The distance command is used to configure Connected 0 a rating of the trustworthiness of a routing Interface information source, such as an individual Static Route 1 EIGRP Summary router or a group of routers 5 Route BGP External • Numerically, an administrative distance is a 20 positive integer from 1 to 255. In general, the (eBGP) EIGRP Internal 90 higher the value, the lower the trust rating OSPF 110 • An administrative distance of 255 means the IS-IS 115 routing information source cannot be trusted RIP 120 at all and should be ignored EIGRP External 170 BGP Internal 200 (iBGP) Unknown 255

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 92 Route Selection • How is administrative OSPF EIGRP OSPF distance used to determine which route should be installed? 10.0.14.0/24 10.0.14.0/24 10.0.14.0/25 10.0.14.128/25 • Only identical routes These Two Routes are compared Are Identical EIGRP Internal = 90 • Identical prefixes with OSPF = 110 different prefix lengths EIGRP Internal Installed are not the same route router#show ip route 10.0.14.0 255.255.255.0 Routing entry for 10.0.14.0/24 • The route from the Known via "eigrp 1", distance 90, metric 307200, type internal Redistributing via eigrp 1 protocol with the Last update from 10.0.121.2 on Ethernet0/1, 00:01:32 ago lower administrative Routing Descriptor Blocks: * 10.0.121.2, from 10.0.121.2, 00:01:32 ago, via Ethernet0/1 distance Route metric is 307200, traffic share count is 1 is installed Total delay is 2000 microseconds, minimum bandwidth is 10000 Kbit Reliability 255/255, minimum MTU 1500 bytes Loading 1/255, Hops 1

• Only identical routes are compared 10.0.14.0/24 10.0.14.0/24 10.0.14.0/25 • Identical prefixes with 10.0.14.128/25 different prefix lengths These Two Routes Are Identical are not the same route

• The route with the longest prefix is OSPF Installed installed Longer Prefixes router#show ip route 10.0.14.0 255.255.255.0 longer-prefixes 10.0.0.0/8 is variably subnetted, 9 subnets, 3 masks D 10.0.14.0/24 [90/307200] via 10.0.121.2, 00:01:35, Ethernet0/1 O 10.0.14.0/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2 O 10.0.14.128/25 [110/20] via 10.0.122.2, 00:00:50, Ethernet0/2 More Specific OSPF Override EIGRP TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 94 Agenda

• Introduction

• Cisco IOS and IP Routing • Multiple Links/Multiple Paths • Load Sharing

• Convergence Techniques

• Design and Deployment

• Assume the same routing process attempts to install two routes for the same destination in the RIB

• The routing process may allow the second route to be installed based on its own rules IGP OSPF IS-IS EIGRP Route Cost Must be equal to Must be equal to Must be less than the installed route installed route variance times the lowest cost installed route

Maximum Paths Must be fewer than maximum-paths configured under the routing process (default = 4, maximum = 32)

Note: BGP default value for maximum-paths = 1

Per-Destination Per-Packet1 Default behaviour of IOS Universal Requires “ip load-sharing per-packet” Algorithm “show cef state” interface configuration1 Per-flow using destination hash Per-packet using round-robin method Packets for a given source/destination Packets for a given source/destination session will take the same path session may take different paths More effective as the number of Ensures traffic is more evenly destinations increase distributed over multiple paths Ensures that traffic for a given session Potential for packets to arrive out of arrives in order sequence

1Per-Packet Not available in IOS-XE based images TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 97 Load Sharing – Equal Cost Multi-Path (ECMP) router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Known via "eigrp 100", distance 170, metric 3072256, type external Redistributing via eigrp 100 Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0 Route metric is 3072256, traffic share count is 1 .... 192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1 Route metric is 3072256, traffic share count is 1 ....

The Traffic Share Count Is Critical to Understanding the Actual Load Sharing of Packets Using These Two Routes 3072256/3072256 = 1

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 98 Load Sharing – with EIGRP Variance router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Known via "eigrp 100", distance 170, metric 3072256, type external Redistributing via eigrp 100 Last update from 192.168.245.11 on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * 192.168.246.10, from 192.168.246.10, 00:18:17 ago, via Serial2/0 Route metric is 1536128, traffic share count is 2 .... 192.168.245.11, from 192.168.245.11, 00:18:17 ago, via Serial2/1 Route metric is 3072256, traffic share count is 1 .... If the Lower Metric Is Less than the Second Metric, the Traffic Share Count Will Be Something Other than 1 (EIGRP with Variance Configured) 3072256/3072256 = 1 3072256/1536128 = 2 2x Faster Link Gets 2 Flows vs. 1 Flow © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public Load Sharing – with eBGP dmzlink-bw router#show ip route 192.168.239.0 Routing entry for 192.168.239.0/24 Only routes learned Known via "bgp 1", distance 20, metric 0 via eBGP Neighbors Tag 2, type external Last update from 10.0.122.2 00:00:16 ago Routing Descriptor Blocks: 10.0.122.2, from 10.0.122.2, 00:00:16 ago Route metric is 0, traffic share count is 1 .... * 10.0.121.2, from 10.0.121.2, 00:00:16 ago Route metric is 0, traffic share count is 2 .... router#show ip bgp 192.168.239.0 BGP routing table entry for 192.168.239.0/24, version 9 Paths: (2 available, best #2, table default) Multipath: eBGP .... 2x Faster Link Gets 2 Flows vs. 1 Flow 10.0.122.2 from 10.0.122.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, valid, external, multipath(oldest) DMZ-Link Bw 312 kbytes rx pathid: 0, tx pathid: 0 .... 10.0.121.2 from 10.0.121.2 (10.0.0.2) Origin IGP, metric 0, localpref 100, valid, external, multipath, best DMZ-Link Bw 625 kbytes rx pathid: 0, tx pathid: 0x0

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 100 CEF Hashing and Exact Route • Now that we have load sharing • What load-sharing algorithm • “show cef state” #show cef state CEF Status: RP instance common CEF enabled IPv4 CEF Status: CEF enabled/running dCEF enabled/running CEF switching enabled/running universal per-destination load sharing algorithm, id AE3030B1 IPv6 CEF Status: • Which exact path are the flows using • “show ip cef exact-route [src-port] [dest-port]” #show ip cef exact-route 1.1.1.1 2.2.2.2 1.1.1.1 -> 2.2.2.2 =>IP adj out of GigabitEthernet1, addr 10.255.0.1

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

• If a link goes down and comes back up before the carrier delay timer expires, the down state is effectively filtered, and the rest of the software on the router is not aware that a link-down event occurred.

• Imposes a default 2 second pause before processing interface events

• Disabling carrier-delay speeds convergence upon interface events

• Disabling carrier-delay can increase control-plane usage during repetitive interface events (flapping)

• Imposes a logarithmic delay based on interface events

• Coupled with carrier-delay, dampening protects the control-plane from repetitive events by increasing the delay before processing up events should the interface flap.

#conf t (config-if)#interface GigabitEthernet1 (config-if)#carrier-delay 0 (config-if)#dampening (config-if)#end #show dampening interface 1 interface is configured with dampening. No interface is being suppressed. Features that are using interface dampening: IP Routing

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

Keepalive (B) Holdtime (B,E,I) Hello (E,I,O) Invalid (R) Dead (O) Flush (R) Update (R) Holddown (R)

BGP 60 180

EIGRP 5 (60) 15 (180) (< T1) IS-IS 10 (3.333) 30 (10) (DIS) OSPF 10 (30) 40 (120) (NBMA)

RIP/RIPv2 30 180 180 240

Note: Cisco Default Values

R1 R4

Recovery Times by Protocol Link Down Link Up Link Up Link Up Line Protocol Down Loss 100% Neighbor Down Loss ~5%

BGP ~ 1 s 180 180 Never

EIGRP ~ 1s 15 (180) 15 (180) Never (< T1) IS-IS ~ 1s 30 (10) 30 (10) Never (DIS) OSPF ~ 1s 40 (120) 40 (120) Never (NBMA) RIP/RIPv2 ~ 1s 240 240 Never

Note: Using Cisco Default Values

R1 R4 BR-W1

R3 R4#show ip bgp vpnv4 vrf cisco neighbor BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link BGP version 4, remote router ID 192.168.201.10 BGP state = Established, up for 1d10h Last read 00:00:19, hold time is 180, keepalive interval is 60 seconds BR-W1# router bgp 65110 R4#show ip bgp vpnv4 vrf cisco neighbor neighbor 192.168.101.9 timers 7 21 BGP neighbor is 192.168.101.10, vrf cisco, remote AS 65110, external link BGP version 4, remote router ID 192.168.201.10 BGP state = Established, up for 00:01:23 Last read 00:00:03, hold time is 21, keepalive interval is 7 seconds When Configuring the Holdtime Argument for a Value of Less than Twenty Seconds, the Following Warning Is Displayed: %Warning: A Hold Time of Less than 20 Seconds Increases the Chances of Peer Flapping © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 108 Bidirectional Forwarding Detection (BFD)

• Extremely lightweight hello protocol • IPv4, IPv6, MPLS, P2MP

• 10s of milliseconds (technically, microsecond resolution) forwarding plane failure detection mechanism.

• Single mechanism, common and standardized • Multiple modes: Async (echo/non-echo), Demand

• Independent of Routing Protocols

• Levels of security, to match conditions and needs

• Facilitates close alignment with hardware

• Link-layer detection misses some types of outages • e.g. Control Plane failure

• Control Plane failure detection is very conservative • 15-180 seconds in default configurations

• Link-layer failure detection is not consistent across media types • Less than 50ms on APS- protected SONET • A few seconds on Ethernet • Several seconds or more on WAN links

• Provides a measure of consistency across routing protocols

• Most current failure detection mechanisms are an order of magnitude too long for time-sensitive applications

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 110 Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection interface GigabitEthernet4 ip address 10.3.255.9 255.255.255.252 bfd interval 50 min_rx 50 multiplier 3 router eigrp 1 network 10.3.0.0 0.0.255.255 bfd all-interfaces (Gi2) R1#show bfd neighbors details IPv4 Sessions NeighAddr LD/RD RH/RS State Int R1 10.3.255.10 4104/1 Up Up Gi4 (Gi4) Session state is UP and using echo function with 50 ms interval. Session Host: Software OurAddr: 10.3.255.9 Handle: 2 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3 Received MinRxInt: 1000000, Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(1371) R2 Rx Count: 985, Rx Interval (ms) min/max/avg: 34/1978/1226 last: 290 ms ago Tx Count: 1372, Tx Interval (ms) min/max/avg: 71/1137/879 last: 721 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: EIGRP CEF Uptime: 00:20:06 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 Configured in milliseconds (ms) My Discr.: 1 - Your Discr.: 4104 Min tx interval: 1000000 - Min rx interval: 1000000 Displayed in microseconds (µs) Min Echo interval: 50000 TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 111 Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection

(Gi2) IPv4 Sessions NeighAddr LD/RD RH/RS State Int R1 172.17.2.8 4102/1 Up Up Gi2 (Gi4) Session state is UP and using echo function with 333 ms interval. interface GigabitEthernet2 Session Host: Software ip address 172.17.2.9 255.255.255.254 OurAddr: 172.17.2.9 bfd interval 333 min_rx 333 multiplier 3 Handle: 1 router bgp 65000 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3 neighbor 172.17.2.8 fall-over bfd Received MinRxInt: 1000000, Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(6076) R2 Rx Count: 4977, Rx Interval (ms) min/max/avg: 4/1970/1069 last: 491 ms ago Tx Count: 6077, Tx Interval (ms) min/max/avg: 754/1180/879 last: 655 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: BGP CEF Uptime: 01:29:04 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 Configured in milliseconds (ms) My Discr.: 1 - Your Discr.: 4102 Min tx interval: 1000000 - Min rx interval: 1000000 Displayed in microseconds (µs) Min Echo interval: 333000 TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 112 Routing Protocol Neighbor Behavior Detecting Unreachable Neighbor (Hello Timers vs. BFD)

100% Packet Loss R1 R2 (Link Up) EIGRP Default: Elapsed Time Between 10 – 15 Sec R1#show clock *09:58:27.716 UTC Sat Jan 27 2018 R1# 12.896 *Jan 27 09:58:40.612: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor 10.3.255.10 (GigabitEthernet4) is down: holding time expired seconds

BFD: Elapsed Time Between 100 - 150 ms with 50ms interval R1#show clock *09:35:44.408 UTC Sat Jan 27 2018 R1# *Jan 27 09:35:45.571: %BFDFSM-6-BFD_SESS_DOWN: BFD-SYSLOG: BFD session ld:4101 handle:2,is going Down Reason: ECHO FAILURE *Jan 27 09:35:45.575: %BFD-6-BFD_SESS_DESTROYED: BFD-SYSLOG: 1.172 bfd_session_destroyed, ld:4101 neigh proc:EIGRP, handle:2 act 1 *Jan 27 09:35:45.580: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor seconds 10.3.255.10 (GigabitEthernet4) is down: BFD peer down notified

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

• Enhanced Object Tracking (EOT)

• Static Routing Options • Floating Static Routes • Reliable Static Routing (RSR) using EOT

• Dial on Demand Routing (DDR) • EEM Script • DMVPN State Tracking

• More information: • http://cs.co/ddrbackup

• Expands to https://www.cisco.com/c/en/us/support/docs/dial-access/dial-on-demand-routing-ddr/10213-backup-main.html

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 115 Enhanced Object Tracking (EOT) Local Significance Track Options Syntax Line-Protocol State track object-number interface type number line-protocol of Interface track 1 interface serial 2/0 line-protocol IP-Routing State of track object-number interface type number ip routing Interface track 2 interface ethernet 1/0 ip routing IP-Route track object-number ip route IP-Addr/Prefix-len reachability Reachability track 3 ip route 10.16.0.0/16 reachability Threshold* of IP- track object-number ip route IP-Addr/Prefix-len metric threshold Route Metrics track 4 ip route 10.16.0.0/16 metric threshold Router#show track 103 Router#show track 100 Track 103 Track 100 IP route 10.16.0.0 255.255.0.0 Interface Serial2/0 line-protocol reachability IPv6 Line protocol is Up Reachability is Up (EIGRP) Support 1 change, last change 00:00:05 1 change, last change 00:02:04 Tracked by: 15.3(3)S First-hop interface is FastEthernet0/0 15.4(1)T GLBP FastEthernet0/1 1 Tracked by: GLBP FastEthernet0/1 1 * EIGRP, OSPF, BGP, Static Thresholds Are Scaled to Range of (0 – 255) TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 116 Enhanced Object Tracking (EOT) External Significance

Track Options Syntax track object-number ip sla type number state IP SLAs Operation track 5 ip sla 4 state Reachability of an IP track object-number ip sla type number reachability SLAs Host track 6 ip sla 4 reachability

Types of IP SLA Probes: dhcp http path-jitter dns icmp-echo1 tcp-connect1 ethernet icmp-jitter udp-echo1 frame-relay mpls udp-jitter1 ftp path-echo voip

1Available for IPv6

Track Options Syntax

track object-number list boolean {and|or} and - both are up for object to be up or - one is up for object to be up list boolean track 5 list boolean or object 51 object 52 not ! Negates state of object track object-number list threshold {weight|percentage} track 6 list threshold weight object 61 weight 20 ! Twice as important list threshold object 62 ! Default weight 10 object 63 object 64 threshold weight up 30 down 25

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 118 Static Host Route Guarantees probe destination only reachable via desired Reliable Static Routing path Tracking IP SLA track 4 list boolean or object 400 object 401 Permanent to guarantee probes only utilize track 400 ip sla 400 reachability desired path. Stay down when down. track 401 ip sla 401 reachability ip sla 400 icmp-echo 10.100.100.100 source-ip 10.1.2.120 IP SLA IP SLA timeout 100 frequency 10 ip sla schedule 400 life forever start-time now ip sla 401 icmp-echo 10.100.200.100 source-ip 10.1.2.120

(.9) (.9) timeout 100 frequency 10 ip sla schedule 401 life forever start-time now ! ip route 10.100.100.100 255.255.255.255 Ethernet 0/1 192.168.101.9 permanent ip route 10.100.200.100 255.255.255.255 Ethernet 0/1 192.168.101.9 permanent ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 192.168.101.8/29 192.168.201.8/29 BR-W1 ip route 10.100.0.0 255.255.0.0 192.168.201.9 200

BR-W1#show ip route track-table ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [up] BR-W1#show ip route 10.100.0.0 255.255.0.0 S 10.100.0.0/16 [1/0] via 192.168.101.9

BR-W1# *Mar 12 03:57:28.367: %TRACKING-5-STATE: 400 ip sla 400 reachability Up->Down Unable to Reach *Mar 12 03:57:37.374: %TRACKING-5-STATE: ip sla 401 reachability 401 Up->Down IP SLA IP SLA IP SLA *Mar 12 03:57:38.137: %TRACKING-5-STATE: 4 list boolean or Up->Down Responders

(.9) (.9)

192.168.101.8/29 192.168.201.8/29

BR-W1#show ip route track-table ip route 10.100.0.0 255.255.0.0 192.168.101.9 track 4 state is [down] Floating Static BR-W1#show ip route 10.100.0.0 255.255.0.0 longer-prefixes BR-W1 S 10.100.0.0/16 [200/0] via 192.168.201.9 Installed S 10.100.100.100/32 [1/0] via 192.168.101.9, Ethernet0/1 S 10.100.200.100/32 [1/0] via 192.168.101.9, Ethernet0/1

IPv6 Reliable Static Routing added in 15.4(1)T

ip sla 610 Unable to Reach icmp-echo 2001:DB8::12 source-interface GigabitEthernet0/1.99 threshold 1000 IP SLA IP SLA frequency 10 Responder ip sla schedule 610 life forever start-time now

WAN RTR WAN RTR track 600 list threshold percentage object 610 2001:DB8:B::5 threshold percentage down 40 up 60 track 610 ip sla 610

event manager applet DISABLE-STATIC-IPv6 Don’t forget to reenable event track 600 state down action 1 cli command "enable" BR RTR action 2 cli command "configure terminal" action 3 cli command "no ipv6 route ::/0 2001:DB8:B::5" action 4 cli command "end" action 99 syslog msg “DEFAULT IPv6 ROUTE DISABLED" BR-RTR# 14:22:14: %TRACKING-5-STATE: 610 ip sla 610 state Up->Down 14:22:14: %TRACKING-5-STATE: 600 list threshold percentage Up->Down 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:DISABLE-STATIC-IPv6) 14:22:14: %HA_EM-6-LOG: DISABLE-STATIC-IPv6: DEFAULT IPv6 ROUTE DISABLED

15.4(1)T added Reliable Static Routing TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 121 Black Hole Route Detection ip sla 110 IPSLA with EEM icmp-echo 208.67.222.222 source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 ! fVRF configuration threshold 1000 frequency 15 Lost connection to ISP but DHCP ip sla schedule 110 life forever start-time now ip sla 111 route stays in the routing table icmp-echo 208.67.220.220 source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now

track 60 ip sla 110 reachability track 61 ip sla 111 reachability IP SLA track 62 list boolean or Probes object 60 object 61 (config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ?

event manager applet DISABLE-STATIC-GIG0-0 event track 62 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "no ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10" action 4 cli command "end" Note: This method is compatible with action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 DISABLED" dual Internet DHCP design. event manager applet ENABLE-STATIC-GIG0-0 event track 62 state up action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10" action 4 cli command "end" action 99 syslog msg “DEFAULT IP ROUTE via GIG0/0 ENABLED"

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 122 Black Hole Route Detection IPSLA with Recursive Routing Interface GigabitEthernet0/0 Lost connection to ISP but vrf forwarding INET-PUBLIC1 ip address dhcp DHCP route stays in the ip sla 110 icmp-echo 208.67.222.222 source-interface GigabitEthernet0/0 routing table vrf INET-PUBLIC1 ! fVRF configuration threshold 1000 frequency 15 ip sla schedule 110 life forever start-time now ip sla 111 IP SLA icmp-echo 208.67.220.220 source-interface GigabitEthernet0/0 Probes vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now

track 60 ip sla 110 reachability track 61 ip sla 111 reachability track 62 list boolean or Note: This method is compatible with object 60 dual Internet DHCP design. object 61 (config)#ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 10 ? ip route 192.0.2.33 255.255.255.255 GigabitEthernet0/0 dhcp 10 ip route 0.0.0.0 0.0.0.0 GigabitEthernet0/0 192.0.2.33 10 track 62

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 123 EEM Script LTE Backup with Event Tracking ip sla 100 icmp-echo 192.168.4.22 source-interface GigabitEthernet0/1 threshold 1000 frequency 15 ip sla schedule 100 life forever start-time now

track 60 ip sla 100 reachability

event manager applet ACTIVATE-LTE Don’t forget to disable VPN RTR WAN RTR event track 60 state down action 1 cli command "enable" action 2 cli command "configure terminal" 192.168.4.22 NAS action 3 cli command "interface cellular0/0/0" action 4 cli command "no shutdown" action 5 cli command "end" (Ce0/0/0) action 99 syslog msg "Activating LTE interface" 14:22:14: %TRACKING-5-STATE: 60 ip sla 100 reachability Up->Down LTE-RTR 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(EEM:ACTIVATE-LTE) 14:22:14: %HA_EM-6-LOG: ACTIVATE-LTE: Activating LTE interface 14:22:34: %LINK-3-UPDOWN: Interface Cellular0/0/0, changed state to up 14:22:34: %DIALER-6-BIND: Interface Ce0/0/0 bound to profile Di1 14:22:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0/0/0, changed state to up 14:22:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel11, changed state to up 14:22:40: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON 14:22:42: %DUAL-5-NBRCHANGE: EIGRP-IPv4 201: Neighbor 10.4.36.1 (Tunnel11) is up: new adjacency http://www.cisco.com/go/cvd/wan VPN Remote Site over 3G/4G/LTE Technology Design Guide

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 124 DMVPN Interface State Control track 2 list boolean or LTE Backup with DMVPN object 101 not track 101 interface Tunnel100 line-protocol interface Tunnel200 if-state track 2 tunnel source Cellular0/0/0 end #show track 2 Track 2 List boolean or VPN RTR Boolean OR is Down WAN RTR 7 changes, last change 00:07:55 object 101 not Up

192.168.4.22 Tracked by: NAS IF-State Control 2 17:24:18.682: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel100, changed state to down 17:24:18.682: %TRACK-6-STATE: 101 interface Tu100 line-protocol Up -> Down (Ce0/0/0) 17:24:18.744: %TRACK-6-STATE: 2 list boolean or Down -> Up 17:24:28.683: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel200, changed state to up LTE-RTR 17:24:29.276: %BGP-5-ADJCHANGE: neighbor 192.168.200.12 Up 17:24:37.505: %BGP-5-ADJCHANGE: neighbor 192.168.200.22 Up #show track 2 Track 2 List boolean or Boolean OR is Up 8 changes, last change 00:00:32 object 101 not Down Tracked by: IF-State Control 2

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

• Hot Standby Router Protocol (HSRP) • v2 IPv4 and IPv6 BR-W1 BR-W2 • Virtual Router Redundancy Protocol (VRRP) • RFC5798 (v3 IPv4 and IPv6), RFC3768 (v2 IPv4), RFC2338 (v1)

• Gateway Load Balancing Protocol (GLBP) • IPv4 and IPv6

• Provide routing redundancy for access layer • How to handle failover when end-hosts have only a single IP default gateway and cached ARP entry

• Provide routing redundancy for devices that depend on static routing • Some firewalls do not support dynamic routing

• Independent of routing protocols • Works with any routing protocol and static routing

• Capable of providing sub-second failover

• Provides load sharing capabilities (GLBP) transparent to end host

interface FastEthernet0/0 ip address 10.1.2.2 255.255.255.0 interface FastEthernet0/0 standby version 2 ip address 10.1.2.3 255.255.255.0 standby 4 ip 10.1.2.1 standby version 2 standby 4 priority 110 Active Standby standby 4 ip 10.1.2.1 Router Router standby 4 preempt standby 4 preempt BR-W1 BR-W2 HSRP standby 6 ipv6 autoconfig standby 6 ipv6 autoconfig (.2) (.3) standby 6 priority 110 VIP (.1) standby 6 preempt standby 6 preempt ipv6 address 2001:DB8:5:1::2/64 ipv6 address 2001:DB8:5:1::1/64 Default Gateway: (.1) DG MAC: MAC VIP

BR-W1#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 110 P Active local 10.1.2.3 10.1.2.1 Fa0/0 6 110 P Active local FE80::A8BB:CCFF:FE00:3400 FE80::5:73FF:FEA0 :6 BR-W2#show standby brief Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 100 P Standby 10.1.2.2 local 10.1.2.1 HSRP—Global IPv6 Addresses Fa0/0 6 100 P Standby FE80::A8BB:CCFF:FE00:3300 Available for Static Deployments local FE80::5:73FF:FEA0 :6

Active Local Router BR-W2 BR-W1 HSRP Failures (.2) (.3) (.1) VIP

Default Gateway: (.1) DG MAC: MAC VIP

BR-W2#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/0 4 100 P Active local unknown 10.1.2.1 Fa0/0 6 100 P Active local unknown FE80::5:73FF:FEA0 :6

Complex Failure

Upstream/Remote Requires “Enhanced Object Failures Tracking (EOT)”

Active Standby Active Router Router Router BR-W1 BR-W2 BR-W1 BR-W2 HSRP HSRP (.2) (.3) (.2) (.3) (.1) VIP (.1) VIP

#track 100 interface serial2/0 line-protocol ! interface FastEthernet0/0 standby version 2 standby 4 priority 110 standby 4 track 100 decrement 20 standby 6 priority 110 standby 6 track 100 decrement 20

R1#show bfd neighbors details

Active Local Router Registered protocols: HSRP BR-W2 BR-W1 HSRP Failures (.2) (.3) (.1) VIP

Default Gateway: (.1) DG MAC: MAC VIP

standby bfd all-interfaces ! default ! interface FastEthernet0/0 standby bfd ! Required only when all- interfaces disabled

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 132 Gateway Load Balancing Protocol (GLBP) BR-W1#show run int fa0/0 interface FastEthernet0/0 ip address 10.1.2.2 255.255.255.0 AVG = Active Virtual Gateway glbp 4 ip 10.1.2.1 glbp 4 preempt SVG = Standby Virtual Gateway glbp 4 weighting 110 lower 100 glbp 6 ipv6 autoconfig AVF = Active Virtual Forwarder glbp 6 preempt glbp 6 weighting 110 lower 100 ipv6 address 2001:DB8:5:1::1/64 BR-W1#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Active 10.1.2.1 local 10.1.2.3 Fa0/0 4 1 - Active 0007.b400.0401 local - AVG SVG Fa0/0 4 2 - Listen 0007.b400.0402 10.1.2.3 - Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600 AVF A AVF B BR-W1 BR-W2 local FE80::A8BB:CCF GLBP F:FE00:3400 (.2) (.3) Fa0/0 6 1 - Active 0007.b400.0601 local - VIP (.1) (.1) VIP Fa0/0 6 2 - Listen 0007.b400.0602 FE80::A8BB:CCFF:FE00:3400 - BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Standby 10.1.2.1 10.1.2.2 local Fa0/0 4 1 - Listen 0007.b400.0401 10.1.2.2 - Fa0/0 4 2 - Active 0007.b400.0402 local - Default Gateway: (.1) Default Gateway: Fa0/0 6 - 100 Standby FE80::7:B4FF:FE00:600 DG MAC: AVF A (.1) FE80::A8BB:CCFF:FE00:3300 DG MAC: AVF B local Fa0/0 6 1 - Listen 0007.b400.0601 FE80::A8BB:CCFF:FE00:3300 - Fa0/0 6 2 - Active 0007.b400.0602© 2020 Cisco and/or its affiliates. local All rights reserved. Cisco - Public Gateway Load Balancing Protocol (GLBP)

AVG = Active Virtual Gateway SVG = Standby Virtual Gateway AVF = Active Virtual Forwarder

BR-W2# *May 26 19:09:14.260: %GLBP-6-STATECHANGE: FastEth0/0 Grp 4 state Standby -> Act ive *May 26 19:09:15.326: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Liste n -> Active *May 26 19:09:15.826: %GLBP-6-STATECHANGE: FastEth0/0 Grp 6 state Standby -> Act ive AVG *May 26 19:09:16.856: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 6 Fwd 1 state Liste AVF A n -> Active BR-W1 BR-W2 GLBP AVF B Local (.2) (.3) (.1) VIP Failures

BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/0 4 - 100 Active 10.1.2.1 local unknown Fa0/0 4 1 - Active 0007.b400.0401 local - Fa0/0 4 2 - Active 0007.b400.0402 local - Fa0/0 6 - 100 Active FE80::7:B4FF:FE00:600 Default Gateway: (.1) Default Gateway: (.1) local unknown DG MAC: AVF A DG MAC: AVF B Fa0/0 6 1 - Active 0007.b400.0601 local - Fa0/0 6 2 - Active 0007.b400.0602 local -

AVG = Active Virtual Gateway SVG = Standby Virtual Gateway AVF = Active Virtual Forwarder Complex Failure Requires Upstream/Remote “Enhanced Object Failures Tracking (EOT)” Requires “Enhanced Object Tracking” AVF A AVG AVF A BR-W1 AVG BR-W2 BR-W1 BR-W2 GLBP GLBP AVF B (.2) AVF B (.3) (.2) (.3) (.1) (.1) VIP VIP

Branch

Tracking IP SLA ip sla 100 icmp-echo 10.100.100.100 source-ip 10.1.2.2 Lo0 Lo0 timeout 100 10.100.100.100 10.100.200.100 frequency 10 ip sla schedule 100 life forever start-time now IP SLA IP SLA ip sla 200 icmp-echo 10.100.200.100 source-ip 10.1.2.2 timeout 100 frequency 10 ip sla schedule 200 life forever start-time now ip route 10.100.100.100 255.255.255.255 FastEthernet0/1 192.168.101.9 permanent ip route 10.100.200.100 255.255.255.255 FastEthernet0/1 192.168.101.9 permanent BR-W1#show ip sla statistics IPSLA operation id: 100 Latest RTT: 1 milliseconds Latest operation start time: *04:42:11.444 UTC Tue Feb 17 2009 AVF A AVF B Latest operation return code: OK Number of successes: 46 BR-W1 BR-W2 GLBP Number of failures: 0 (.2) (.3) Operation time to live: Forever VIP (.1) (.1) VIP IPSLA operation id: 200 Latest RTT: 1 milliseconds Latest operation start time: *04:42:11.356 UTC Tue Feb 17 2009 Latest operation return code: OK Number of successes: 24 Number of failures: 0 Operation time to live: Forever

BR-W1# track 100 ip sla 100 reachability BR-W1#show glbp track 200 ip sla 200 reachability FastEthernet0/0 – Group 4 track 1 list boolean or State is Active 1 state change, last state change 00:09:59 object 100 IP SLA IP SLA Virtual IP address is 10.1.2.1 object 200 Hello time 3 sec, hold time 10 sec interface FastEthernet0/0 Next hello sent in 2.336 secs ip address 10.1.2.2 255.255.255.0 Redirect time 600 sec, forwarder timeout 14400 sec glbp 4 ip 10.1.2.1 Preemption enabled, min delay 0 sec Active is local glbp 4 priority 110 Standby is 10.1.2.3, priority 105 (expires in 7.808 sec) glbp 4 preempt Priority 110 (configured) glbp 4 weighting 110 lower 100 Weighting 110 (configured 110), thresholds: lower 100, glbp 4 load-balancing weighted upper 110 Track object 1 state Up decrement 20 glbp 4 weighting track 1 decrement 20 Load balancing: weighted Group members: AVF A AVF B aabb.cc00.0110 (10.1.2.2) local aabb.cc00.0410 (10.1.2.3) BR-W1 BR-W2 GLBP There are 2 forwarders (1 active) (.2) (.3) Forwarder 1 VIP (.1) (.1) VIP State is Active Forwarder 2 State is Listen

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 137 Enhanced Object Tracking Composite Failure BR-W1# *Feb 17 05:17:25: %TRACKING-5-STATE: 100 ip sla 100 state Up->Down *Feb 17 05:17:25: %TRACKING-5-STATE: 200 ip sla 200 state Up->Down *Feb 17 05:17:26: %TRACKING-5-STATE: 1 list boolean or Up->Down *Feb 17 05:17:38: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Active -> Listen BR-W2#show glbp IP SLA IP SLA FastEthernet0/0 – Group 4 State is Standby 1 state change, last state change 00:28:16 Virtual IP address is 10.1.2.1 BR-W1 Remains Hello time 3 sec, hold time 10 sec Next hello sent in 1.856 secs Redirect time 600 sec, forwarder timeout 14400 sec Active Virtual Unable to Reach Preemption enabled, min delay 0 sec Either Gateway (AVG) Active is 10.1.2.2, priority 110 (expires in 10.400 sec) Standby is local IP SLA Priority 105 (configured) Responder Weighting 110 (configured 110), thresholds: lower 100, upper 110 AVF A Track object 1 state Up decrement 20 AVG AVF B BR-W2 Becomes Load balancing: weighted BR-W1 BR-W2 Group members: GLBP aabb.cc00.0110 (10.1.2.2) (.2) (.3) Active Virtual aabb.cc00.0410 (10.1.2.3) local (.1) VIP There are 2 forwarders (2 active) Forwarder (AVF) Forwarder 1 State is Active for both A and B Forwarder 2 State is Active TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 138 Agenda

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques • Interface Detection • Routing Protocols • Static Routing and EOT • First Hop Redundancy Protocols • Cisco SD-WAN (Viptela)

• Design and Deployment

vSmart • TCP based extensible control plane protocol • Runs between WAN Edge routers and vSmart controllers and between the vSmart controllers - Inside TLS/DTLS connections • Leverages address families to advertise reachability for vSmart vSmart • Unicast/Multicast destinations (statically/dynamically learnt service side routes) • TLOCs • Network Service routes (L4-L7) • BFD stats (TE and H-SDWAN) • Cloud onRamp for SaaS probe stats (gateway) • Distributes IPSec encryption keys, and data and WAN Edge WAN Edge app-aware policies (embedded NETCONF) Note: WAN Edge routers need not connect to all vSmart Controllers

WAN Edge • Path liveliness and quality measurement detection protocol - Up/Down, loss/latency/jitter, IPSec tunnel MTU • Runs between all WAN Edge routers in the

WAN Edge WAN Edge topology - Inside IPSec tunnels - Operates in echo mode - Automatically invoked at IPSec tunnel establishment - Cannot be disabled

• Uses hello (up/down) interval, poll (app-aware) WAN Edge WAN Edge interval and multiplier for detection - Fully customizable per-WAN Edge, per-color

Multiplier (n) • Each WAN Edge router sends BFD hello packets for path quality and liveliness detection - Packets echoed back by remote site

Hello Interval (ms) • Hello interval and multiplier determine how Liveliness many BFD packets need to be lost to Quality declare IPSec tunnel down App-Route Multiplier (n) • Number of hello intervals that fit inside poll interval determines the number of BFD Poll Interval Poll Interval Poll Interval (ms) packets considered for establishing poll interval average path quality • App-route multiplier determines number of poll intervals for establishing overall Hello Interval (ms) average path quality

▪ WAN Edge Routers vManage App Aware Routing Policy continuously perform path App A path must have: Latency < 150ms liveliness and quality Loss < 2% measurements Jitter < 10ms

Internet Remote Site

MPLS Regional Path 2 Data Center

LTE

Path1: 10ms, 0% loss, 5ms jitter Path2: 200ms, 3% loss, 10ms jitter Path3: 140ms, 1% loss, 10ms jitter IPSec Tunnel

▪ WAN Edge routers are directly connected to all the transports MPLS Internet - No need for L2 switches front-ending the WAN Edge routers ▪ When transport goes down, WAN Edge routers detect the condition and bring down the tunnels built across the failed transport WAN Edge WAN Edge - BFD times out across tunnels ▪ Both WAN Edge routers still draw the traffic for the prefixes available through the SD-WAN fabric ▪ If one of the WAN Edge routers fails (dual failure), second WAN Edge router takes over forwarding the traffic in and out of site - Both transport are still available

▪ WAN Edge routers are connected only to their respective transports MPLS Internet ▪ WAN Edge routers build IPSec tunnels across directly connected transports and across the transports connected to the neighboring WAN Edge router WAN Edge WAN Edge - Neighboring WAN Edge router acts as an underlay router for tunnels initiated from the other WAN Edge ▪ If one of the WAN Edge routers fails (dual failure), second WAN Edge router takes over forwarding the traffic in and out of site - Only transport connected to the remaining WAN Edge router can be used

▪ WAN Edge routers leverage BFD for Data detecting tunnel liveliness Center • If intermediate network path through the SD-WAN fabric fails or if the remote-end WAN Edge router (e.g. data center) fails, MPLS Internet BFD hellos will time out and remote site WAN Edge router will bring down its relevant IPSec tunnels • Traffic will be rerouted after the failed condition had been detected - BFD hello timer and multiplier can be Remote tweaked for faster detection Site

Excellent Option R2

R1 R4 SubOptimal Option

R3 Bad Option Effectiveness of Various Techniques for Different Outage Types Link Down Link Up Link Up Upstream Upstream Neighbor Down Loss ~5% Blackhole Brownout Routing Protocols BFD N/A1 N/A1

EOT2 RSR3 using EOT (w/IP SLA) SD-WAN

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment • MPLS Dual Carrier • MPLS + Internet

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 153 Dual WAN (MPLS—Dual Carrier) PE-CE Protocol: BGP 10.100.0.0/16 10.1.2.0/24 • Default behavior: 1-way 10.1.1.0/24 load sharing A-R1 MPLS - SP A A-R4

HQ- HQ-W1 CORE1 192.168.101.8/29 • Load is shared from HQ to BR-W1 192.168.201.8/29 Branch HQ-W2

HQ- B-R1 MPLS - SP B B-R4 CORE2

EIGRP eBGP eBGP HQ-CORE1#show ip route D EX 10.1.2.0/24 [170/258816] via 10.1.1.110, 02:24:22, Vlan10 [170/258816] via 10.1.1.210, 02:24:22, Vlan10

• Only one link used Branch to HQ BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00

• IGP (EIGRP examples) 10.100.0.0/16 10.1.2.0/24 • Routes redistributed from BGP into 10.1.1.0/24

IGP (match & tag) A-R1 MPLS - SP A A-R4

• BGP routes are treated as IGP external HQ- HQ-W1 CORE1 192.168.101.8/29 BR-W1 • BGP 192.168.201.8/29 HQ-W2 • No iBGP required between HQ-W1 & HQ- B-R1 MPLS - SP B B-R4 HQ-W2 (CE routers) CORE2 • Routes redistributed from IGP into BGP except those tagged as originally EIGRP eBGP eBGP sourced from BGP

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 155 Dual WAN (MPLS—Dual Carrier) Mutual Route Redistribution Detail Routes into EIGRP HQ-W1# router eigrp networkers address-family ipv4 unicast autonomous-system 65110 topology base redistribute bgp 65110 metric 45000 100 255 1 1500 10.1.1.0/2 4 address-family ipv6 unicast autonomous-system 65110 topology base redistribute bgp 65110 metric 45000 100 255 1 1500 BR AS 65100 HQ-W1 Routes into BGP HQ-CORE1 eBGP HQ-W1#

10.1.2.0/2410.1.1.0/2 router bgp 65110

iBGP address-family ipv4 EIGRP eBGP redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES AS 65200 HQ-W2 address-family ipv6 BR redistribute eigrp 65110 route-map BLOCK-TAGGED-ROUTES HQ-CORE2 ! route-map BLOCK-TAGGED-ROUTES deny 10 10.100.0.0/16 BGP Redistribution to IGP automatically match tag 65100 65200 tags routes with route-map BLOCK-TAGGED-ROUTES permit 20 neighbor AS Number !

10.100.0.0/16 10.1.2.0/24

• Is it possible to load share 10.1.1.0/24 from Branch to HQ? A-R1 MPLS - SP A A-R4

• HQ- HQ-W1 BGP Multipath CORE1 192.168.101.8/29

BR-W1 • Allows installation of multiple 192.168.201.8/29 BGP paths to same destination HQ-W2

HQ- B-R1 MPLS - SP B B-R4 • Requirements (all must be equal) CORE2 • Neighbor AS or AS-PATH EIGRP eBGP eBGP • Weight BR-W1#show ip bgp • Local Preference • AS-PATH length Network Next Hop Metric LocPrf Weight Path • Origin * 10.100.0.0/16 192.168.201.9 0 65200 65200 ? • Med *> 192.168.101.9 0 65100 65100 ? BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.101.9, 00:34:00

10.100.0.0/16 10.1.2.0/24

• Is it possible to load share from 10.1.1.0/24

Branch to HQ? A-R1 MPLS - SP A A-R4

HQ- HQ-W1 • maximum-paths 2 CORE1 192.168.101.8/29

BR-W1 192.168.201.8/29 • Requires hidden command: HQ-W2

HQ- B-R1 MPLS - SP B B-R4 • bgp bestpath as-path multipath- CORE2 relax

EIGRP eBGP eBGP router bgp 65110 bgp bestpath as-path multipath-relax address-family ipv4 maximum-paths 2 address-family ipv6 maximum-paths 2 BR-W1#show ip route B 10.100.0.0/16 [20/0] via 192.168.201.9, 00:03:44 [20/0] via 192.168.101.9, 00:03:44

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment • MPLS Dual Carrier • MPLS + Internet

• Headquarters WAN Edge EIGRP BGP BGP • W1 learns Branch route via eBGP 10.100.0.0/16 10.1.2.0/24

• W2 learns Branch route via EIGRP 10.1.1.0/24

MPLS - SP A • Headquarters Core A-R1 A-R4 HQ-CORE1 HQ-W1 BR-W1

192.168.101.8/29 EIGRP • W1 redistributes eBGP into EIGRP, HSRP

results in EIGRP external Internet

VPN Tunnel • W2 does not require redistribution, HQ-CORE2 HQ-W2 BR-W2 results in EIGRP internal 10.0.1.0/29

EIGRP • Core1, Core2 install Branch route via W2 HQ-W1#show ip route B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01 HQ-W2#show ip route HQ to Branch Traffic Flows D 10.1.2.0/24 [90/26882560] via 10.0.1.2, 00:00:04, Tunnel1 Across Tunnel HQ-CORE1#show ip route D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:02:32, Vlan10

• Single Router Branch WAN Edge • W1 learns HQ route via eBGP and EIGRP Internal

• eBGP Administrative Distance preferred EIGRP BGP BGP

10.100.0.0/16 10.1.2.0/24

10.1.1.0/24

A-R1 MPLS - SP A A-R4

HQ-W1 HQ-CORE1 192.168.101.8/2 9 BR-W1 Branch to HQ Traffic Internet HQ-W2 VPN Tunnel HQ-CORE2 Flows Across MPLS 10.0.1.0/29 EIGRP BR-W1#show ip route B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58 B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06

EIGRP BGP BGP

10.100.0.0/16 10.1.2.0/24 • Dual Router Branch WAN Edge 10.1.1.0/24 A-R1 MPLS - SP A A-R4

• W1 learns HQ route via eBGP HQ-W1

HQ-CORE1 BR-W1 EIGRP 192.168.101.8/2 HSRP • W2 learns HQ route via EIGRP 9 Internet

VPN Tunnel • No redistribution configured HQ-W2 BR-W2 HQ-CORE2 • HSRP Primary is on W1 10.0.1.0/29 EIGRP

BR-W1#show ip route B 10.100.100.0/24 [20/0] via 192.168.101.9, 04:48:58 B 10.100.200.0/24 [20/0] via 192.168.101.9, 03:44:06 BR-W2#show ip route Branch to HQ Traffic D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 00:10:56, Tunnel1 D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 00:10:57, Tunnel1 Flows Across MPLS BR-W1#show standby brief P indicates configured to preempt. | Interface Grp Pri P State Active Standby Virtual IP Fa0/1 1 110 P Active local 10.1.2.220 10.1.2.1

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 162 DUAL WAN (MPLS + Internet) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP • How to force HQ to Branch traffic across MPLS (primary)? • Adjust administrative distance EIGRP BGP BGP • For EIGRP routes learned via tunnel 10.100.0.0/16 10.1.2.0/24 • Ensure administrative distance is 10.1.1.0/24

higher than that of EIGRP external (170) A-R1 MPLS - SP A A-R4 HQ-W2# router eigrp 65110 HQ-CORE1 HQ-W1 BR-W1

192.168.101.8/29 EIGRP network 10.0.1.0 0.0.0.7 Only change is on hub HSRP

distance 195 10.0.1.0 0.0.0.7 Internet

• Redistribute between two EIGRP Processes VPN Tunnel HQ-CORE2 HQ-W2 BR-W2

Forcing External as done between BGP and 10.0.1.0/29

Campus EIGRP EIGRP HQ-W2# Router eigrp 65100 network 10.0.1.0 0.0.0.7 Requires additional changes router eigrp 65110 or Proper Pre-Planning redistribute eigrp 65100 HQ-W1#show ip route Now: B 10.1.2.0/24 [20/0] via 192.168.101.2, 05:24:01 HQ to Branch Traffic Flows HQ-W2#show ip route Across MPLS D EX 10.1.2.0/24 [170/261120] via 10.1.1.110, 00:07:25, GigE0/0 HQ-CORE1#show ip route D EX 10.1.2.0/24 [170/258816] via© 2020 Cisco10.1.1.110, and/or its affiliates. All rights 00:08:44, reserved. Cisco Public Vlan10 DUAL WAN (MPLS + Internet) MPLS Failure

• Failure within MPLS cloud EIGRP BGP BGP

• Dependent on provider 10.100.0.0/16 10.1.2.0/24 • Worst Case 10.1.1.0/24 A-R1 MPLS - SP A A-R4 • Link up neighbor down HQ-CORE1 HQ-W1 BR-W1

192.168.101.8/29 EIGRP • Primary dependency BGP timers HSRP

• End to end convergence time as Internet long as BGP Holdtime VPN Tunnel HQ-CORE2 HQ-W2 BR-W2 • Configuration options 10.0.1.0/29 • BFD for sub-second notification EIGRP • End-to-end Application Restoration as HQ Route Tables fast as SD-WAN detects After Failure: HQ-W2#show ip route HQ to Branch Traffic D 10.1.2.0/24 [195/26882560] via 10.0.1.2, 00:06:46, Tunnel1 HQ-CORE1#show ip route Flows Across Tunnel D 10.1.2.0/24 [90/26882816] via 10.1.1.210, 00:09:18, Vlan10

10.100.0.0/16 10.1.2.0/24

10.1.1.0/24

• Failure within MPLS cloud MPLS - SP A A-R1 A-R4

HQ-CORE1 HQ-W1 BR-W1

• Suboptimal routing at Branch 192.168.101.8/29

EIGRP HSRP

• HSRP primary remains Internet

VPN Tunnel unchanged at BR-W1 HQ-CORE2 HQ-W2 BR-W2

10.0.1.0/29 • Use EOT and move HSRP EIGRP primary to BR-W2 Branch Route Tables BR-W1#show ip route D 10.100.100.0/24 After Failure: [90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1 Branch to HQ D 10.100.200.0/24 Traffic Flows [90/26885376] via 10.1.2.220, 00:22:42, FastEthernet0/1 Across Tunnel BR-W2#show ip route D 10.100.100.0/24 [90/26882816] via 10.0.1.1, 01:08:44, Tunnel1 D 10.100.200.0/24 [90/26882816] via 10.0.1.1, 01:08:45, Tunnel1

• Introduction

• Cisco IOS and IP Routing

• Convergence Techniques

• Design and Deployment

• Key Takeaways

• Outages can manifest in many ways. Network design should be based on application requirements to survive various outages.

• Cisco IOS has inherent load sharing capabilities. Analyze your network topology and use these to your advantage.

• End-to-end convergence time is a critical metric. Understand how localized topology changes affect end-to-end resiliency.

• Multiple links/paths increase network reliability and can be utilized to improve application performance.

• IP SLA based monitoring can detect outage types that are undetectable by “hello based” techniques.

• BFD is the lightweight tool for speeding convergence of all protocols.

• Cisco SD-WAN permits full utilization of available bandwidth and path selection based on current real time characteristics.

• Most effective network designs incorporate a combination of convergence techniques

• Cisco SD-WAN utilizes these features, while simplifying deployment and management, and increasing application availability.

Arvind Durai, Director Solutions Integrator Customer Experience (Cx)

TECCRS-2500 Goals

• WAN Services – QoS, Multicast, Security, Operational management

• To get a high-level overview of design components for each service type

• You should be able determine the correct options tied to end to end reference WAN reference architecture

1 7 QoS elements Architectural framework

• QoS design model • Understanding Service provider interaction on QoS • QoS on SDWAN • QoS on non SDWAN deployment

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 172 Quality of Service Operations How Does It Work and Essential Elements Classification Queuing and Post-Queuing and Marking Dropping Operations

▪ Classification and Marking: The first element to a QoS policy is to classify/identify the traffic that is to be treated differently. Following classification, marking tools can set an attribute of a frame or packet to a specific value. ▪ Policing: Determine whether packets are conforming to administratively-defined traffic rates and take action accordingly. Such action could include marking, remarking or dropping a packet. ▪ Scheduling (including Queuing and Dropping): Scheduling tools determine how a frame/packet exits a device. Queuing algorithms are activated only when a device is experiencing congestion and are deactivated when the congestion clears. ▪ Link Specific Mechanisms (Shaping, Fragmentation, Compression, Tx Ring) Offers network administrators tools to optimize link utilization

Voice SD Video Conf Telepresence Data

Bandwidth per Call SD/VC has the Same HD/VC has Tighter Traffic patterns for Depends on Codec, Requirements as Requirements than Data Vary Among Sampling-Rate, VoIP, but Has VoIP in terms of jitter, Applications and Layer 2 Media Radically Different and BW varies based Traffic Patterns on the resolutions (BW Varies Greatly) ▪ Data Classes: • Latency ≤ 150 ms ▪ Latency ≤ 150 ms ▪ Latency ≤ 200 ms ▪ Mission-Critical Apps • Jitter ≤ 30 ms ▪ Jitter ≤ 30 ms ▪ Jitter ≤ 20 ms ▪ Transactional/Interactive • Loss ≤ 1% ▪ Loss ≤ 0.05% ▪ Loss ≤ 0.10% Apps • Bandwidth (30-128Kbps) ▪ Bandwidth (1Mbps) ▪ Bandwidth (5.5-16Mbps) ▪ Bulk Data Apps One-Way Requirements One-Way Requirements One-Way Requirements ▪ Best Effort Apps (Default)

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 175 NBAR (NBAR2) Network Based Application Recognition SCE Classification +1000 Signatures Allows finer grained Innovations classification of traffic based IOS NBAR Advanced Classification Techniques Native IPv6 Classification on additional application level +150 Signatures Open API characteristics –http url, host, mime, User Agent and other fields •e.g. “match protocol http url *cisco.com*” NBAR2 matches http traffic to and from cisco.com –rtp payload-type • New DPI engine provides Advanced Application Classification •e.g. “match protocol and Field Extraction Capabilities from Service classification rtp video” matches rtp engine video traffic –citrix ica-tag, app • Protocol Pack allows adding more applications without •e.g. “match protocol upgrading or reloading IOS citrix ica-tag 0” • NBAR2 Protocol List - matches citrix traffic http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6558/ps661 with ica-tag 0 6/product_bulletin_c25-627831.html

Discover: 1000+ applications categorized to simplify management

HTTP Performance Collection: Enhanced application performance reports, url hit counts, top applications …

HTTP HTTP Control: Apply QoS, Acceleration and Path Control according to company performance expectations

Application Visibility and Natively Integrated into Simple to Enable Control Cisco Routers

Relevant Business as usual Not Important

• Needed to support • May or may not • Consumer oriented the core business support business traffic type objective objectives directly

➢ Applications should ➢ The traffic can be ➢ Treated less than best be understood, grouped to qos class class effort marked and treated in queues with proper accordance to best marking or just tied to practice single qos class or default queues

Voice Best Effort Call-Signaling 5% (62%) 33% Voice 33% Scavenger 1%

Best Effort Call-Signaling 25% Critical Data 36% 5%

Three-Class (VoIP and Data Only) Five-Class WAN Edge Model WAN Edge Model Best Effort Voice 18% 25%

Interactive-Video 15% Scavenger 1% Eleven-Class WAN Edge Model Bulk Data 4%

Low Latency Queuing Link Fragmentation Police and Interleave VoIP IP/VC PQ TX Interleave Signaling Ring Packets Packets Critical Out In Fragment Bulk CBWFQ Mgmt FQ Default

Layer 3 Queuing Subsystem Layer 2 Queuing Subsystem

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 180 CBWFQ Operation IOS Interface Buffers policy-map CBWFQ class NETWORK-CONTROL bandwidth percent 5 Network Control CBWFQ class CALL-SIGNALING bandwidth percent 5 Call Signaling CBWFQ class OAM bandwidth percent 5 OAM CBWFQ class MM-CONFERENCING Packets FQ bandwidth percent 10 Multimedia Conferencing CBWFQ fair-queue In Tx-Ring FQ CBWFQ … Multimedia Streaming CBWFQ Scheduler FQ Packets Transactional Data CBWFQ Out FQ Bulk Data CBWFQ FQ Best Effort / Default CBWFQ FQ Pre-Sorters Scavenger CBWFQ TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 181 LLQ Operation

IOS Interface Buffers 1 Mbps VoIP policy-map LLQ Policer LLQ class VOIP priority 1000 Packets … In Packets Out CBWFQ Scheduler Tx-Ring

FQ Pre-Sorters CBWFQ

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 182 Multi-LLQ Operation IOS Interface Buffers policy-map MULTI-LLQ 1 Mbps class VOIP VoIP Policer priority 1000 class BROADCAST-VIDEO 4 Mbps Bscst-Video priority 4000 Policer LLQ class REALTIME- 5 Mbps INTERACTIVE RT-Interactive Policer priority 5000 …

Packets Packets In Out CBWFQ Scheduler Tx-Ring

CBWFQ

policy-map WAN class VOICE priority percent 10 class INTERACTIVE-VIDEO priority percent 23 class CRITICAL-DATA bandwidth percent 15 L3 Queue random-detect dscp-based class DATA bandwidth percent 19 Low Latency Queueing random-detect dscp-based class SCAVENGER bandwidth percent 5 Police VOICE class NETWORK-CRITICAL bandwidth percent 3 service-policy MARK-BGP PQ class class-default bandwidth percent 25 Police INTERACTIVE-VIDEO To Layer 2 random-detect Queueing CRITICAL-DATA Packets DATA Subsystem In SCAVENGER CBWFQ NETWORK-CRITICAL FQ class-default Weighted Random Early Detection (WRED) Random Early Detection (RED) Layer 3 Queueing Subsystem

• The IPv6 implementation of DiffServ is identical to IPv4. The same classifiers can be used to differentiate both IPv6 and IPv4 packets, as follows:

• Source IP address, destination IP address, IP Protocol field, source port number, and destination port number

• IP precedence or DSCP values To match packets on both IPv4 and IPv6 • TCP/IP header parameters, such as packet length protocols: class-map match-all ipv6+ipv4forprec5 • Source and destination MAC addresses match precedence 5

To match packets for IPv6 protocols only: class-map match-all ipv6onlyprec5 • The match precedence and match dscp commands match protocol ipv6 filter IPv4 and IPv6 traffic. match precedence 5

Bottom Line: Enterprises must Co-manage QoS with Their MPLS VPN Service Providers; Their Policies must Be Both Consistent and Complementary

Maximum One-Way Service-Levels Latency ≤ 150 ms/Jitter ≤ 30 ms/Loss ≤ 1%

Enterprise Enterprise Campus Remote-Branch Service Provider

PE PE CE CE

Maximum One-Way SP Service-Levels Latency ≤ 60 ms Jitter ≤ 20 ms Loss ≤ 0.5%

• Enterprise customers may need to re-mark traffic prior to forwarding to the MPLS provider. This ensures markings conform to the admission criteria defined by the provider, allowing traffic to be serviced by the appropriate queue within the provider network. • The same concept applies to traffic ingression the enterprise network from the provider cloud. • Certain applications may need to be re-marked to ensure the enterprise QoS strategy is properly applied.

Enterprise Network Provider Network

Enterprise Class Structure: Provider Class Structure: • Class 1 [DSCP A] • Class 1 [DSCP A] Provider Trust • Class 2 [DSCP C] • Class 2 [DSCP B] Enterprise Trust Boundary . Boundary • Class 3 [DSCP D] . PE Ingress Policing and Remarking • Class 4 [DSCP E] . PE-to-CE Queuing/Shaping/LFI • Class n [DSCP F] Maximum One-Way Service-Levels Latency ≤ 150 ms/Jitter ≤ 30 ms/Loss ≤ 1% © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 188 Enterprise-to-Service Provider Mapping Five-Class Provider-Edge Model Remarking Diagram Enterprise DSCP PE Classes Application Routing CS6 Voice EF EF SP-Real Time 35% Interactive Video AF41 ➔ CS5 CS5 Streaming Video CS4 ➔ AF21 CS6 SP-Critical Mission-Critical Data AF31 AF31 20% Call Signaling CS3 ➔ CS5 CS3 Transactional Data AF21 ➔ CS3 AF21 SP-Video CS2 15% Network Management CS2 AF11/CS1 SP-Bulk 5% Bulk Data AF11 Scavenger CS1 ➔ 0 SP-Best Effort 25% Best Effort 0

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 189 MPLS Short Pipe Mode DiffServ Tunneling Short Pipe Mode Operation Shaded Area Represents Provider DiffServ Domain Assume a Policer Remarks Unshaded Areas Out-of-Contract Traffic’s Represent Customer Top-Most Label to DiffServ Domain MPLS EXP 0 Here PE Edge (to CE) MPLS VPN Policies Are Based on CE Router PE Router Customer Markings P Routers

PE Router CE Router

IPP3/DSCP AF31 MPLS EXP 4 MPLS EXP 0 MPLS EXP 0 IPP3/DSCP AF31 Packet Initially MPLS EXP 4 MPLS EXP 4 IPP3/DSCP AF31 Original Customer- Marked IP ToS Marked to IPP3/ IPP3/DSCP AF31 IPP3/DSCP AF31 DSCP AF31 Top-Most Label Is Values Are MPLS EXP Values Topmost Label Is Popped (PHP), but Preserved Are Set Independently Marked Down by Egress Policy Is Based from IPP/DSCP Values a Policer on EXP 0 of Topmost Label

• Classification - Flow, match on 5-tuple (ACL, Data Policy) - Application, match on DPI (Data Policy) vEdge Q0 • Per-Egress Interface Queuing Q1 Q2 - Q0 is LLQ - vEdge control traffic (DTLS/TLS, BFD, routing protocols) goes

Q7 into Q0. Not subjected to LLQ policer

Egress Egress Interface Ingress Ingress Interface

• Scheduling for Q1-Q7 is WRR* Classification Queuing • Drop is RED** or taildrop - RED drop profiles are linear, i.e. X% queue depth results in X% drop probability * Weighted Round-Robin ** Random Early Discard

• Shaper behavior • Single Rate Policer behavior - Forward shaper rate conforming traffic - Forward traffic conforming to policer rate o There are tokens in the bucket o There are tokens in the bucket - Queue shaper rate exceeding traffic - Drop traffic exceeding policer rate o There are no tokens in the bucket o There are no tokens in the bucket o Weighted Round-Robin - Configurable burst rate • Egress-only Shaping o Token bucket depth - Interface based • Ingress and Egress Policing - Interface/VLAN based - Access list classification - Flow policing, match on 5-tuple - Data Policy classification (ingress only) o Flow policing, match on 5-tuple o Application policing, match on DPI

• Classification - Flow, match on 5-tuple (ACL, Data Policy) vEdge - Application, match on DPI (Data Policy) • Ingress interface marks/remarks inner

DSCP bits Egress Egress Interface Ingress Ingress Interface - Copied to encapsulation DSCP bits • Egress marks/remarks outer encapsulation Classification Marking, DSCP bits Re-marking - Inner DSCP bits not modified - Transport network QoS

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 193 Example – SDWAN QoS Policy Localized CLI policy policy The configuration snippet is for policer bursty-traffic interface ge1/0, in VPN 1. The rate 1000000 burst 20000 policer monitors incoming traffic exceed remark on the interface. When traffic access-list policer-bursty-traffic sequence 10 exceeds 20 MB (configured in match the policer burst command). source-ip 56.0.1.0/24 action accept policer bursty-traffic GUI default-action accept vpn 1 interface ge1/0 ip address 56.0.1.14/24 no shutdown access-list policer-bursty-traffic in

CBWFQ: Class-based weighted fair queuing (CBWFQ) extends the LLQ: The Low Latency Queuing feature brings strict standard WFQ functionality to provide support for user-defined traffic priority queuing to Class-Based Weighted Fair Queuing classes. For CBWFQ, you define traffic classes based on match criteria (CBWFQ). including protocols, access control lists (ACLs), and input interfaces. class-map class-map match …… match …… class-map class-map match …….. match …….. policy-map policy-map class name1 class name1 bandwidth…. priority … class name 2 class name 2 bandwidth … bandwidth … class class-default class class-default fair-queue fair-queue int gig 0/0 int gig 0/0 service-policy service-policy

Parent/ Child MQC with Shaper A hierarchical policy is a quality of service (QoS) model that class-map enables you to specify QoS behavior at multiple levels of match …… hierarchy. Depending on the type of hierarchical policy you class-map configure, you can use hierarchical policies to: match …… policy-map • Specify multiple policy maps to shape multiple class queues together priority ….. • Apply specific policy map actions on the class aggregate traffic bandwidth ……. class class-default • Apply class-specific policy map actions fair-queue policy-map • Restrict the maximum bandwidth of a virtual circuit (VC) while class class-default allowing policing and marking of traffic classes within the VC shape …… All hierarchical policy types consist of a top-level parent policy and service-policy one or more child policies. The service-policy command is used to int gig 0/0 apply a policy to another policy, and a policy to an interface, service-policy subinterface, virtual circuit (VC), or virtual LAN (VLAN).

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 196 WAN QoS – Implementing Per Site Traffic Shaping CE 10 Mbps 500 Mbps in to WAN can easily overrun the lower 10.5.144.0/21 CE speed committed rates at remote sites CE 10 Mbps 10.5.152.0/21 500 Mbps CE CE CE 802.1q 50 Mbps trunk CE CE CE Shape 50 Mbps (500 Mbps) CE CE 20 Mbps 10.5.168.0/21 Per-Site Shaping to Avoid Overruns CE CE

policy-map POLICY-MAP-Br210 policy-map POLICY-MAP-Br212 class VOICE class VOICE priority percent 10 priority percent 10 class INTERACTIVE-VIDEO Per Destination class INTERACTIVE-VIDEO priority percent 23 priority percent 23 class CRITICAL-DATA Service Policies class CRITICAL-DATA bandwidth percent 15 bandwidth percent 15 random-detect dscp-based random-detect dscp-based class DATA class DATA bandwidth percent 19 bandwidth percent 19 random-detect dscp-based random-detect dscp-based class SCAVENGER class SCAVENGER bandwidth percent 5 bandwidth percent 5 class NETWORK-CRITICAL class NETWORK-CRITICAL bandwidth percent 3 bandwidth percent 3 service-policy MARK-BGP service-policy MARK-BGP class class-default class class-default bandwidth percent 25 bandwidth percent 25 random-detect random-detect

ip access-list extended Br210-10.5.144.0 permit ip any 10.5.144.0 0.0.7.255 ! Per Destination ip access-list extended Br212-10.5.168.0 class-map match-all CLASS-MAP-Br210 permit ip any 10.5.168.0 0.0.7.255 match access-group name Br210-10.5.144.0 Class Maps ! class-map match-all CLASS-MAP-Br212 match access-group name Br212-10.5.168.0

policy-map POLICY-MAP-L2-WAN-BACKBONE-WITH-PER-SITE-SHAPERS class NETWORK-CRITICAL bandwidth percent 3 class CLASS-MAP-Br210 shape average 10000000 Shape to 10 Mbps to BR210 service-policy POLICY-MAP-Br210 class CLASS-MAP-Br212 shape average 20000000 Shape to 20 Mbps to BR212 service-policy POLICY-MAP-Br212

policy-map POLICY-MAP-L2-WAN-BACKBONE-WITH-PER-SITE-SHAPERS class NETWORK-CRITICAL bandwidth percent 3 class CLASS-MAP-Br210 Shape to 10 Mbps to BR210 shape average 10000000 service-policy POLICY-MAP-Br210 class CLASS-MAP-Br212 Shape to 20 Mbps to BR212 shape average 20000000 service-policy POLICY-MAP-Br212

policy-map WAN-INTERFACE-G0/0/4 class class-default Shape to 500 Mbps aggregate shape average 500000000 service-policy POLICY-MAP-L2-WAN-BACKBONE-WITH-PER-SITE-SHAPERS

child shapers 10 Mbps parent shaper 10 Mbps 50 Mbps Shape 50 Mbps (500 Mbps) 20 Mbps 20 Mbps

• Basic Multicast Concepts • Multicast design per region • Multicast design in SDWAN • Multicast design in IPv6 • Multicast design across inter-regional

Live Web Cast of Minimally-Invasive Hip Replacement

Software Distribution Corporate Communication

Patch Update Branch Office

Patch Update Branch IP/TV Broadcast Server Program Manager Corp HQ Office Patch Web Server, Update Media Publishing Branch Software © 2020 Cisco and/or its affiliates. All rights reserved. OfficeCisco Public PIM Sparse mode (refresher on RFC4601)

RP 2 S,G : Shortest path tree *,G : Shared tree 4 5 6 7 Enterprise Source 9 3 FHR 10 8 LHR The last hop router check the routing to the RP 1 Receiver 1. Rec joins IGMP request to router. PIM *,g join towards the RP

2. Rec state known at RP 8. If the check verifies an alternate path that is more optimal based on unicast tree. 3. Source sends flow to the router 9. If the check verifies an alternate path that is more optimal 4. First hop router sends a unicast register packet ( encap based on unicast RIB. The new flow is built and upstream router multicast pack) to the RP gets a prune. 5. Since the receiver state is maintained the RP will send a 10. Flow is built to the receiver registry stop message to the FHR & (S,G) join 6. S,G flow is built 11. If all the receivers switch, then Prune is sent to the RP 7. S,G flow is built LHR and the receiver receives the flow

(S1,G) IGMP (S1,G) PIM join (S1,G) PIM join (S1,G) PIM join Report

(S,G) (S,G) (S,G) (S,G)

FHR PIM PIM LHR-DR Receiver Source1 multicast traffic

Source2 Source3 Source4 FHR: First Hop Router LHR: Last Hop Router • SSM: Source Specific Multicast

• Only (S,G) state

• Used in One to Many applications

• Receiver needs IGMP v3 (SSM mapping can be used)

• IGMP Version 3 supports source filtering, which is required for SSM. IGMP For SSM to run with IGMPv3, SSM must be supported in the router, the host where the application is running, and the application itself.

• Multicast routing is backwards from Unicast Routing • Unicast routing is concerned about where the packet goes • Multicast Routing is concerned about where the packet came from

• Multicast Routing uses “Reverse Path Forwarding”

• Membership Reports • Membership Queries • IGMP report sent by one host suppresses sending by • Queries sent to 224.0.0.1 with ttl = 1 others • One router on LAN is elected to send queries • Unsolicited reports sent by host, when it first joins the • Query interval 60–120 seconds group • Group-specific Query • Leave Group Message • Router sends Group-specific queries to make sure there • Host sends leave message if it leaves the group and is are no members present before stopping to forward the last member (reduces leave latency in comparison to v1) data for the group for that subnet

• Enables hosts to listen only to a specified subset of the hosts (unicast address) sending to the multicast group • Adds Include/Exclude Source Lists • Apps must be rewritten to use IGMPv3 Include/Exclude features • Reports are sent on 224.0.0.22 & all IGMPv3 enabled routers listen to this address

IGMP reports IGMP Queries

239.0.0.0 • Address Range: 239.0.0.0/8 RFC 2365 • Private multicast address space Org.-Local • Similar to RFC1918 private unicast Expansion address space • RFC 2365 Administratively Scoped Zones 239.192.0.0 • Organization-Local Scope (239.192/14) RFC 2365 – Largest scope within the Enterprise network (i.e. Enterprise- Org-Local wide) Scope – Expands downward in address range • Local Scope (239.255/16) 239.196.0.0 – Smallest possible scope within the Enterprise network 239.253.0.0 RFC 2365 – Expands downward in address range Local Scope Expansion – Other scopes may be equal but not smaller 239.255.0.0 RFC 2365 Local 239.255.255.255 Scope (Not to scale.) TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 208 Multicast Domains ✓ Separate PIM domain Administratively-Scoped • Separate RP Address Range • Separate Multicast group • Verify containment is Example Company ABC required for local scoping 239.0.0.0/8

LA NYC Campus Campus/ /Branch Branch

RFC 2365 Local Scopes 239.255.0.0/16 RFC 2365 Org-Local Scope Enterprise or Organization scope covers the entire enterprise network. 239.192.0.0/14 Local or Regional scope covers a subset of the enterprise scope

Type Redundancy Propagation Key Features

Static No Every downstream • ‘Override’ needed if enterprise has auto-rp and static needs a rp configures configuration Auto-RP Active/Standby No Downstream • Works with scoping router configuration • Cisco-Announce - 224.0.1.39 (Candidate RP) is needed Cisco-Discovery - 224.0.1.40 (Mapping Agent) • Need to configure ‘ip pim auto-rp listener’ • Highest RP address is elected as RP

BSR Active/Standby No Downstream • Does not work with scoping router configuration • Single bootstrap router with multiple candidate BSR is needed • C-BSR IP address used as tie-breaker ✓ (Highest IP address wins) • The active BSR may be preempted ✓ New router w/higher BSR priority forces new election • Contents of BSR’s Group-to-RP Mapping Cache

RP1 MSDP RP2 A B ip pim rp-address 10.0.0.1 ip pim rp-address 10.0.0.1

C D

Interface loopback 0 Interface loopback 0 ip address 10.0.0.1 255.255.255.255 ip address 10.0.0.1 255.255.255.255

Interface loopback 1 Interface loopback 1 ip address 10.0.0.2 255.255.255.255 • “ip pim sparse-mode” should be ip address 10.0.0.3 255.255.255.255 ! enabled in all routers ! ip msdp peer 10.0.0.3 connect-source loopback 1 ip msdp peer 10.0.0.2 connect-source loopback 1 ip msdp originator-id loopback 1 ip msdp originator-id loopback 1

MSDP RP1 RP2 A B ip multicast-routing ip multicast-routing

C D

Interface loopback 0 Interface loopback 0 ip address 10.0.0.1 255.255.255.255 ip address 10.0.0.1 255.255.255.255 Interface loopback 1 Interface loopback 1 “ ” ip address 10.0.0.2 255.255.255.255 • ip pim sparse-mode should be ip address 10.0.0.3 255.255.255.255 ! enabled in all routers ! ip pim send-rp-announce loopback 0 scope 32 • “ip pim auto-rp listener” command ip pim send-rp-announce loopback 0 scope 32 ip pim send-rp-discovery loopback 1 scope 32 ip pim send-rp-discovery loopback 1 scope 32 ! ! ip msdp peer 10.0.0.3 connect-source loopback 1 ip msdp originator-id loopback 1 ip msdp peer 10.0.0.2 connect-source loopback 1 ip msdp originator-id loopback 1

▪ IGMP/PIM joins are signaled in control plane updates

▪ Each site vEdge Router chooses its desired Replicator

▪ Preserve standard multicast routing behavior over unicast core

Multicast control plane flow

• Source register itself to an RP • Receiver sends the (*,G) join • First Join gets forwarded to the vSmart as an OMP packet and then forwarded to the replicator • Replicator forwards (*,G) to the RP • RP forwards it to the source • Stream is forwarded to the receiver through the replicator. Stream never goes to vSmart • Once receiver has the source information, it will the join using (S,G) • First (S,G) join gets forwarded as an OMP control packet to the vSmart and then to replicator • Replicator then forward the (S,G) to the source • vEdge ignores subsequent joins and depends on the prune message to stop the stream from the replicator

▪ PIM-SM with Auto-RP ▪ For cases with many receivers ▪ Replicators can be at the source or dispersed at different geo locations

▪ PIM-SSM ▪ For cases with many sources aggregating at a headend/DC site ▪ Replicators should be defined at the receiver side ▪ SSM mapping defined on a non-viptela device

Customer’s Point of View • Multicast Domain inside the Provider Network Blue connects each MVPN. CE Blue Multicast Domain Red CE PE Provider PE Net PE CE Blue Red

CE PE Red Multicast Domain PE PE

CE CE Red Blue

PIM on the edge

Unicast routing in overlay across MPLS mcast data Mcast signalling in overlay Mcast through core – GRE encap I have no receivers: I have receiver: I ignore I join

CE Leaf PE CE Receiver Leaf PE traffic rate I have receiver: exceeds CE I join thresholdLeaf PE

CE Receiver Leaf PE CE Leaf PE Source

Data MDT PIM message Join TLV carries C-(S,G) & P-group

PIM on the edge PIM join Unicast routing in overlay across MPLS Mcast signalling in overlay For Data MDT Group Mcast through core – GRE encap Configured on PE per VRF Range of groups

PIM join CE mcast data Leaf PE PIM Data-MDT Join TLV CE Receiver Leaf PE C-(S,G) CE P-Group Leaf PE PIM join

CE Receiver Leaf PE CE Leaf PE Source

For high rate sources, data-MDT created Removes traffic from default-MDT to offload PE’s that did not join stream

• MVPN needs to be enabled for multicast to traverse through enterprise managed MPLS layer 3 VPN cloud.

• For provider managed MPLS cloud, the enterprise routers do not need MVPN configured. The questions that requires to be asked from the provider to understand the multicast transport are : ✓ PIM protocol support ✓ RP propagation method support ✓ Total number of state allowed per VRF

• Overview PIM BGP PIM PIM PIM PIM

Source PE PE Source PE PE MPLS cloud Receiver MPLS cloud Receiver S1,S2 S1,S2

PIM in Overlay BGP in Overlay

mLDP → PIM PIM → mLDP static map static map translation translation

PIM PIM PIM PIM

Source PE PE Source PE PE MPLS cloud Receiver MPLS cloud Receiver S1,S2 S1,S2

Static Inband

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 219 Overview of New MPLS Multicast transport options mLDP BGP MVPN Static • Multicast flow information encoded Used for advertisement of AD routes & • Uses RSVP-TE, LSPs are build from in the mLDP FEC (In-band C-mcast routes (*,G) and (S,G) the head-end to the tail-end. • Supports only P2MP LSPs signaling) • Two new extended communities for • Supports traffic engineering • LSPs are build from the leaf to the tunnel and label attribute (RFC 4271) – Bandwidth reservation root •The NLRI field in the contains the – Explicit routing • Supports P2MP and MP2MP LSPs MCAST-VPN NLRI – Fast ReRoute • “Control plane is P2MP or • P2P technology at control plane MP2MP (RFC 6826) • Data plane is P2MP Deployment Consideration: Deployment Consideration: • Deployment Consideration • Easy for SSM •Inherits P2P scaling limitations •Allows explicit or bandwidth • Scalable due to receiver driven • Complex to tree building contraint routing understand/troubleshoot for •Supports Fast Reroute (FRR) • Supports Fast Reroute (FRR) ASM via RSVP TE unicast backup path or Loop free alternate Newer technologies: (LFA) path BIER – Bit Indexed Explicit Replication (Stateless Multicast)-BRKIPM-2239 Multicast transport without explicit tree-building protocols results in a considerable simplification.

• P2MP tree - Receiver driven – Root learned from routing

• MP2MP tree – Configuration driven – Root configured

• Protection by MPLS TE or Loop-Free Alternate (LFA)

• No PHP – top label identifies the tree

• Replication of mcast on the core routers

• FEC elements holds: Type of tree + Root + Opaque value: (S,G), MDT

number, LSP ID, …

Opaque Type Root Label Label FEC FEC Mapping TLV Element TLV Message

Multicast fast convergence needs to have unicast fast convergence configured: Case 1: With Unicast fast convergence only (traffic of 20 M unicast and 2 multicast streams) during link failure ▪ unicast convergence is 0.324 sec Oops ▪ multicast convergence is 2.783 sec.

Multicast fast convergence configuration

Case 2: With Unicast AND Multicast Fast Convergence (traffic of 20 M unicast and 2 multicast streams) during link failure ▪ the unicast convergence is 0.324 sec ▪ multicast convergence is 0.512 sec. (pim query-interval & multicast rpf backoff feature)

• Multicast addresses are distinguishable from unicast addresses because they always begin with 0xFF • Multicast addresses are all assigned out of the FF00::/8 block. Multicast addresses also have a scope associated with them. • Link Local Multicast Address- Link local multicast addresses are only intended for systems on a link and are not to be forwarded by network equipment off of that link. • Organization Multicast Address- Organizational multicast addresses are intended for use within an organization. • Global Multicast Address- Global multicast addresses are usable across the Internet • The benefits of IPv6 multicast address compare to IPv4 multicast address • Larger Addressing Space - implies the availability of plenty of addresses for multicast groups. • Addressing Scope - offers a cleaner way to contain the multicast traffic within the intended domain.

• MLD is used by IPv6 routers to discover multicast listeners (nodes that want to receive multicast packets destined for specific multicast addresses) on directly attached links.

• MLDv2 enhances MLDv1 by enabling a node to express or combined reports interest in a particular source for a multicast group, and concatenates reporting. This capability optimizes the multicast operation through a more discrete control of group membership. This also provides support for SSM.

• When “ipv6 multicast-routing “ is enabled, MLDv2 is enabled by default. Note: MLDv2 is backward compatible with MLDv1.

Static RP BSR RP ✔ ✔ PIM Anycast Embedded RP

• S11 sends multicast packet to the first hop- designated router. The Designated router will send a PIM register message to the RP1.

• RP1 is configured with RP2, 3,4 IP address as Anycast peer. • Since the Register message did not come from one Step 1 of the RPs in the anycast-RP set, the RP1 will then send a copy of the Register message to all RPs.

• In this case, this register message will use RP1s own Step 2 IP address as the source address for the PIM Step 2 Register message.

• Now, in case of RP2 receives the Register message from RP1 and check the state table, since R1 are Step 2 connected, the RP2 sends a Register-Stop back to RP1.

• This is state maintenance mechanism between the RPs RP1 joins the multicast PIM state for S1 by triggering a (S1,G) Join message toward S1 and (S1,G) state is created. After this RP2 also joins back to the source tree by creating S1,G join towards S1.

Embedded RP

•IPv6 PIM provides embedded RP support. Embedded RP support allows the router to learn RP information using the multicast group destination address instead of the statically configured RP.

•Based on the application type, The Application Type •verify the number of (S,G) and (*,G) entries •PIM mode selection •Based on the application type, map multicast in the QOS architecture

•Scoping The Enterprise Multicast • RP placement Domain • PIM domain selection • PIM mode selection • QOS and Security consideration

The Access Methodology •Single link and redundant link access •Encryption requirement •Check if the branches have access through •Service provider managed MPLS cloud •Self managed VPN Multicast Requirement for •Scoping : the Branches • Extension of Enterprise domain • Local domain only • Extension of enterprise domain + local domain • Multicast protection

Abstract: Abstract: Understand the fundamental requirements • IP Multicast, Volume I thoroughly for inter-domain multicast covers basic IP multicast Design control planes for identifying source principles and routing and receiver, as well as the downstream techniques for building and control plane operating enterprise and service Support multicast transport where cloud provider networks to support service providers don’t support native applications ranging from videoconferencing to data multicast replication. Use multicast VPNs to logically separate traffic on the same physical infrastructure • Reflecting extensive experience Explore the unique nuances of multicast in working with Cisco customers, the data center the authors offer pragmatic Implement Virtual Port Channel (vPC), discussions of common features, Virtual Extensible LAN (VXLAN), and Cisco’s design approaches, deployment Application Centric Infrastructure (ACI) models, and field practices. Design multicast solutions for specific You’ll find everything from industries or applications specific commands to start-to- Walk through examples of best-practice finish methodologies: all you multicast deployments need to deliver and optimize any Master an advanced methodology for IP multicast solution. © troubleshooting2020 Cisco and/or its affiliates. large All rights IPreserved. multicast Cisco Public networks WAN Security Security elements Architectural framework

• Cloud delivered Enterprise security • Efficiency based on Direct Inter Access • Branch Security using SDWAN

vBond Data Plane Security

Data Plane Control Plane ➢ Privacy & Encryption vSmart Controllers ➢ Key Exchange ➢ Data Plane Integrity ➢ Secure Segmentation MPLS 4G ➢ Network Address Translation INET ➢ Anti-Replay Protection

vEdge Routers Infrastructure Security

➢ Security Zoning Cloud Data Center Campus Branch CoLo ➢ DDoS Protection for Controllers ➢ DDoS Protection for WAN Edges ➢ Federated Security

Fabric Security • Centralized data policy is defined on vManage and distributed by vSmart vManage controllers • Centralized data policy match on application traffic of interest vSmart - DPI or 6 tuple matching • Centralized data policy takes drop Centralized Localized Data Policy Data Policy action to block unwanted traffic - Can log

SDWAN Edge SDWAN Edge • Localized data policy works similarly to centralized data policy, but it is distributed directly from vManage Trust Zone Un-trust Zone Fabric Security

• Local DIA or Regional Internet Exit - Per-VPN behavior • All traffic or policy based - 6-tuple or DPI matching INET NAT • Secure Access - Port-Address Restricted NAT Regional - Local Firewall Data Center - Regional Firewall NAT INET • For optimal quality of experience toward SaaS SD-WAN applications use Cloud onramp INET Fabric • Federated security enforcement with scaled MPLS attributes can be done using Umbrella (Local stack Data Center security and Cloud proxy to simplify distributed Remote Site security)

Umbrella

IoT All office locations Mobile Any device on your ON-NETWORK network OFF-NETWORK Branch Roaming laptops

Roaming Every port and protocol

ALL PORTS AND PROTOCOLS

PREDICTIVE UPDATES DNS and IP layer Umbrella / Talos and partner feeds

▪ Domain request UMBRELLA Custom domain lists STATISTICAL & ▪ IP response (DNS-layer) MACHINE LEARNING MODELS or connection (IP-layer) Custom IP lists (future)

ALLOW, BLOCK, PROXY INTERNET-WIDE TELEMETRY

WBRS / Talos + partner feeds HTTP/S layer Custom URL lists ▪ URL request AV ▪ File hash AMP

ALLOW OR BLOCK

2M+ live events per second 11B+ historical events

Guilt by inference ▪ Co-occurrence model ▪ Sender rank model Patterns of guilt ▪ Secure rank model ▪ Spike rank model Guilt by association ▪ Natural Language Processing rank model ▪ Predictive IP Space Modeling ▪ Live DGA prediction ▪ Passive DNS and WHOIS Correlation

time - time +

a.com b.com c.com x.com d.com e.com f.com

Possible malicious domain Possible malicious domain Known malicious domain Co-occurrence of domains means that a statistically significant number of identities have requested both domains consecutively in a short timeframe

DGA MALWARE EXPLOIT KIT PHISHING

Massive amount y.com of DNS request y.com is blocked before volume data is it can launch full attack gathered and

analyzed DNS REQUESTS DAYS

DNS request volume matches known exploit kit pattern and predicts future attack

Pinpoint suspicious domains and 209.67.132.476 observe their IP’s fingerprint

Identify other IPs – hosted on the 209.67.132.477 same server – that share the DOMAIN same fingerprint 209.67.132.478

209.67.132.479 Block those suspicious IPs and any related domains

500+

partnerships with top ISP and CDNS

All data centers announce same IP address

Requests transparently sent to fastest available

100% If down for any reason, uptime since 2006 automatically re-routes DDoS protection and to next fastest available global fail-over

vBond

Authenticated Sources

vSmart vManage

Implicitly SD-WAN IPSec Trusted Sources Control Plane Policing: SDWAN Edge ▪ 300pps per flow ▪ 5,000pps

Explicitly Defined Sources Cloud Security Implicit Deny except: 1. Return packets matching flow entry (DIA enabled) Unknown 2. DHCP, DNS, ICMP Sources * Can manually enable :SSH, NETCONF, NTP, OSPF, BGP, STUN Other

➢ Application Visibility and Granular control Inspect policy allows only return traffic to Outside Zone ➢ 1400+ layer 7 applications classified be allowed and drops using NBAR2* any new connections

➢ Drop traffic by application category or specific application Edge Device ➢ Segmentation

➢ PCI compliance Inside Guest Users Zone Zone Devices

Service-VPN 1 Service-VPN 2

• Snort is the most widely deployed IPS engine in the world

• Backed by global Threat Intelligence (TALOS) signatures updated automatically

• Signature whitelist support IPS • Real-time traffic analysis On-site Services

• Enforce Acceptable Use Controls

• Create custom Black and White Lists

• Block based on Web Reputation score

• 82+ Web Categories and dynamic updates

• Customizable End-user notifications

• Leading Security Efficacy for malware, phishing, and unacceptable requests by blocking based on DNS requests

• Supports DNScrypt (with Local Domain-bypass)

• Local Domain-bypass

• TLS decryption

Integration with AMP

• File reputation Internet Check Signature

• File retrospection Integration with ThreatGrid

• File Analysis Check file

Malware Sandbox Backed with valuable Threat Intelligence

ThreatGrid

G0/0 – LAN facing IP Dest DNS Lookup NBAR 2 Security VRF 4 CEF 5 G0/1 – WAN facing 1 3 Ingress G0/0

• LAN to WAN DNS FW IPS URL-F AMP NBAR NAT Egress G0/1 Security

DNS VRF 2 NAT 3 CEF 4 Layer 1 Ingress G0/1

• Automation architecture • Day 0/1 automation available options and differentiation • Custom vs prescriptive automation • Multi Domain Orchestration • Visibility and Telemetry

Ansible Puppet OpenFlow Neutron ML2 Python SDK OpenStack API Agile DevOps Salt Netconf YANG

Chef Container

NX-API REST JSON Controller LXC

XML NFV

Ansible Server Server sends config when playbook is run No Playbook CLI (SSH) agent s Router

Unlike server configuration Ansible does not execute Python on-box

• Ansible uses an agentless push model

• Uses YAML and Jinja2 templates

• Can configure using CLI (SSH) or NX-API

• Use nxos-ansible modules, or new core Ansible 2.1 modules

Day 0: Device provisioning - Multiple solutions Get a device into an operational state - Topology, traffic view tied to NMS - Automated alerts - Integration takes time Day 1:Provision services and might increase the TCO

Day to day operations, provisioning

Platform solution takes care of Day 0, Day 2 and ability to provide Day 1 Day2: Operate use cases Configuration 3rd Party Tools API REST On-box Python / TCL Management Tools EEM (Splunk, Nagios, etc.) (Puppet/Chef/Ansible)

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 260 IT Operations Framework -Enterprise Customer Landscape Service Life cycle Industry Drivers : • Improving agility and responsiveness to business demand Management • Monitoring, optimizing or securing the network more effectively • Lowering maintenance and service costs • Simplifying the network

Manage Outcome: • Maintain IT user experience visibility • Reduced Downtime • Cost effective operations

WAN DC Campus Cloud

Network Compute Network Compute Network Network Storage Storage

Security

Architecture Objectives / KPI Framework

Brown Field Green Field

Current use case or Domain based new immediate mitigation capabilities and cross domain pollination

New services onboarding, ACLs, RMA, OS Change Management 26% upgrades; typically brownfield

Traps and Syslogs collection, correlation, Incident Management 23% analytics, prioritization of troubleshooting actions

Performance data polled or pushed from device; Network Monitoring 22% analytics, trending and planning

Day 0/1 onboarding of devices and services; Device Installation 15% manual or remote; typically greenfield unless RMA N=100

Turn-Key Customizable

Turnkey solution stack for end- to-end enterprise Service-orchestration focused orchestration Modular solution architecture On-prem or cloud-based Build/design/run & Analytics Flexible demarcation between SP and Enterprise Virtual and Physical Multi-vendor / Multi-tenancy Support for SDA and IWAN Open API for Extensibility Customized SP service catalogues Multi-vendor & Multi-Tenant

DNA Center / vManage Network Services Orchestrator (NSO)

Network and security LEARNING Automation services automation aligned with the IT Process

Proactive and predictive insights Analytics to assure service experience

INTENT CONTEXT DNA-C API standardization and as a monetization for app dev and Platform programmability

Cross Automation and Analytics Integration with offers from Edge Domain to Cloud including Security

SECURITY

25 devices

Identity Easy QoS Security Services

MS CUCM Cognitive Controller E • Enterprise applications are automatically Surveillanc FTP classified and given right class of service e based on cisco validated design (CVD)

application mappings Platinum Gold • QoS policies are applied at a system level

with a single click of a button, improving Silver application performance and saving

valuable time/resources BestEffort

Cisco Validated Design {CVD}

Set to CVD

Applications Engineers • Logically centralized network services • Model based architecture REST, NETCONF, Java, Python, Erlang, CLI, Web UI • Data models written in YANG (RFC Service 6020) Model Service Manager • Structured representations of: • Service instances Device Manager Device • Network configuration and state Model • No hard-coded assumptions about: Network Element Drivers (NEDs) • Network services • Network architecture

NETCONF, REST, SNMP, CLI, etc • Network devices • Mapping service operations to network • VNFM configuration changes • Controller Apps • Transactional integrity • EMS and NMS • Multiprotocol and multivendor support Physical Networks Virtual Networks Network Apps

Yang Model Representation Internal NSO Representation

container service { service key "name";

container interface { interface type string; type container type { type int64; Router Interface number Configuration container number { Store leaf ip { type inet:ip-address;

}

leaf speed { type int64; ip speed } } {10.1.1.21} {100} } }

Service Models written independent of devices!

API calls to NSO to Map Service to Device Models

API with Input Parameters

{configure interface} {interface} {GigabitEthernet} {1} Call Map Commit Write {172.16.11.1} {100}0

Easy to use Automated Clustering High Performance • Turnkey and simple • Deploy services as fast • Shared pool of resources • PCIe Passthrough • Built for network, security, and as applications • Auto-deploy redundant HA • SR-IOV load balancing teams • Use DevOps to automate pair • Lifecycle management ACI services • Scale-out architecture • Provision a new service within • RESTful API minutes using GUI or CLI • NetConf/Yang REST GUI CLI API NSO NetConf

XRv Third KVM based ASAv 9000 party services

CSP SW, ConfD, Linux KVM, OVS, PCIe Passthrough, SR-IOV

Cisco UCS 1RU/2RU Modular Platforms, 1 & 10G SFP+ NICs NFS

• Operation visibility is tied to aligning data from various sources within an architectural domain and across multiple architectural domains • Telemetry data is used to predict patterns based on analysis from many sources with smaller sampling size

•The telemetry framework covers first data sources in the southbound side listed as • Telemetry, syslog, traps, SNMP and CLI • Other controllers that provide health index • The visual view of the collective raw data (without analytics just based on thresholds) can be added a layer to combine the data sources (e.g. Kafka data bus) • The data thresholds can be stored in a simple database (represented as a data lake) • This framework can be tied to an AI engine to provide custom outcomes • The AI engine is aligned to Topology, Inventory, Performance, and Configuration (Anomaly detection is tied to all the four pillars) • The maturity of this model is based on noise reduction aligned with business logic for criticality or existing process • Maturity in telemetry process helps development of new capabilities such as: data plane steering and optimized operational workflow management

ITSM

Workflow Telemetry BUS Engine API (Control/ Data Plane) /Pipeline NSO NSO - In sync with CMDB - WAN (multi vendor) orchestrator - Lifecycle management for API API tradition DC and DCNM APIC WAN - NSO aligned with tufin for firewall policy WAN - Traditional ACI MPLS DC

• Understand the application usage before adding services like QoS or Multicast to the WAN

• QoS should be always included in the initial WAN design deployment

• Leverage federated security cloud proxy and localized stack at the branch in a phased approach for consumption

• Don’t look at point solution for automation, rather look and the architecture and then fit the solution.

• Keep it Simple!

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 278 L3 Segmentation and Cloud Ready Solutions for the WAN Evolving Trends and Cloud Connector Solutions for Enterprise WAN Design Craig Hill, Distinguished Systems Engineer CCIE #1628 - Emeritus

@netwrkr95 Goals of This Session…

Understanding…

• The current drivers and importance of Layer 3 segmentation in next-generation WAN design

• The lead solutions for offering Layer 3 segmentation in government, enterprise, managed SP, and other entities

• The importance of aligning WAN designs with optimal connections to public cloud access

• High speed encryption options beyond IPSec

• Examples and importance of automation in next-gen operations

• Introduction - Network Segmentation Drivers and Concepts

• Evaluating WAN Solution Option Criteria for L3 Segmentation

• Evolving Trends for Self Deployed Backbone Designs

• Evolving Trends and Solution Options for L3 Segmentation over IP and to the Public Cloud

• Architecture and Technology Innovations and Trends for the WAN

• End to End WAN Design and Components (Summary of Session)

• Introduction - Network Segmentation Drivers and Concepts

• Evaluating WAN Solution Option Criteria for L3 Segmentation

• Evolving Trends for Self Deployed Backbone Designs

• Evolving Trends and Solution Options for L3 Segmentation over IP and to the Public Cloud

• Architecture and Technology Innovations and Trends for the WAN

• End to End WAN Design and Components (Summary of Session)

• It has evolved a long way from technologies like TDM (1960’s)

• From TDM, ATM/FR Virtual Circuits in the WAN, to…

• VLANs in the Campus, to… Logical/Virtual Routers on routing devices, to…

• Virtual Machines on server clusters in the Data Centre Secure Domain VPP/ VNF Routers OVS Virtual Circuits MPLS Virtual VRF Lite CSR GRE HSRP MPLS VPN Port 1000v VPLS Channel VLANs AToM TDM SDx L2TPv3 Virtual Device NfV Context

• Giving One physical network the ability to support multiple L3 virtual networks

• End-user perspective does not change

• Maintains Hierarchy, Virtualises devices, data paths, and services

Internal Separation (sales, eng) Merged Company Guest Access Network

Virtual Network Virtual Network Virtual Network

• Cost Reduction - allowing a single physical network the ability to offer multiple virtual networks to tenants

• Simpler OAM—reducing the amount of physical network devices needing to be managed and monitored

• Security—maintaining segmentation of the network for different departments over a single device/Campus/WAN

• Agility – accelerates adding network segments (virtual) over same physical networks

• High Availability—leverage segmentation through clustering devices that appear as one (vastly increased uptime)

• Data Centre Applications • Offer per/multi-tenant segmentation from the DC into the WAN/campus/Branch and cloud • end-to-end continuity of Segmentation from server-to-campus-to-WAN

• Multi-Tenant Dwelling requiring Separation • Airports – (United, Delta, etc…), Government Facilities – (agencies sharing single building/campus), Intra Organisation segmentation – (sales, engineering, HR, LoB) • Company mergers – allowing slow migration for transition, overlapping addressing • IoT Device Isolation – segment (IP cameras, badge readers) from the user data

• Security for Isolation • Key Fundamental element for Zero Trust Security framework • Quarantine Zone – Honey Pot, Steered Traffic as result of DDoS, Anomaly Enforcement • Mandates to logically separate varying levels of security (e.g. enclaves)

• Regulation requirements - Health Care – HIPPA | Financial and Transactional – Sarbanes-Oxley, PCI Compliance

• Public Cloud and Key Component of Policy Construct • L3 segmentation for “per tenant” - Leveraged in Intent-based network policies

SD-Access ACI SD-WAN

▪ User & Device Policy ▪ Application Policy ▪ WAN Policy

▪ Micro and Macro ▪ Application Segmentation & ▪ Path selection & QoS Segmentation Micro-Segmentation ▪ Segmentation – port/802.1Q ▪ L3 Segment = VRF ▪ Tenant = VRF ▪ L3 Segment = VRF

WAN Segmentation Segmentation on/of Interconnect Device Device Pooling

WAN Si

“Virtualising” the Extending and Maintaining the “Virtualising” Routing and “Virtualised” Devices/Pools over Multiple Devices Forwarding of the Any WAN Transport Option to Function as a Device Single Device

Device Device Partitioning Pooling

VLANs VRFs Virtual Sw System (VSS) VNI (VXLAN) Virtual Port Channel (vPC) VDC (NX-OS) HSRP/GLBP (Virtual Device Context) Stackwise Cloud Services Router (CSR) ASR 9000v/nV Clustering ISRv “Router per Tenant” Segmentation through Inter-Chassis Control vEdge NFV w/ Orchestration Protocol (ICCP)

Device WAN Segmentation Device Partitioning Interconnect Pooling

WAN Si

VLAN Virtual Sw System (VSS) L2 VPNs L3 VPNs VRF Virtual Port Channel (vPC) VXLAN EVPN/VxLAN MPLS BGP L3 VPN HSRP/GLBP VDC (NX-OS) PW/VPLS L3 VPN over IP Stackwise (Virtual Device Context) OTV BGP EVPN (VXLAN, SR) Inter-Chassis Control Protocol Cloud Services Router (CSR) (ICCP) IOS-XRv 64-bit VXLAN to MPLS Integration

• Segmentation component • Virtual Route Forwarding Instance (VRF)

• Control Plane component • MPLS / Segment Routing (SR): MP-BGP (RFC 4364), E-VPN (L2/L3 VPN) • L3 VPN over IP: MP-BGP (RFC 4364), Overlay Management Protocol (OMP), NHRP (for DMVPN)

• Data Plane component • MPLS / Segment Routing: MPLS, Segment Routing • L3 VPN over IP: MPLS over GRE/IP-UDP (RFC 4023), VXLAN

Service Support of Each Solution: QoS, IPv6 (selective), Encryption, Multicast, etc…

• Introduction - Network Segmentation Drivers and Concepts

• Evaluating WAN Solution Option Criteria for L3 Segmentation

• Evolving Trends for Self Deployed Backbone Designs

• Evolving Trends and Solution Options for L3 Segmentation over IP and to the Public Cloud

• Architecture and Technology Innovations and Trends for the WAN

• End to End WAN Design and Components (Summary of Session)

Options? Which Technology When to use and Where? SD WAN Deployment Options Segmentation Domain Backbone SD-Core Enterprise SD-WAN

SDN SDN Controller/Mgmt Controller/Mgmt

Branch CE Branch Site Site CE CE SP MPLS P Campus P Campus Branch DC DC PE P PE Branch Internet CE Site Site CE CE

Managed Domain Managed Domain Managed Domain Overlay Encap

• Targets Service Provider “like” customers who need • Targets enterprise customers looking to consume to control SLA’s, rapid service turn up times, tighter secure WAN transport, with central mgmt., control, granular service options (SR-TE), end-to-end and application visibility control, provisioning, and visibility • Cisco SD-WAN, MPLS VPN over IP (central • SR, SR-TE, Centralized WAN controller controller and/or open tools for automation)

• Full control of the end to end network and SDN Controller/Mgmt service delivery rate (minutes, not weeks)

• Self determined QoS SLA’s Branch Site • High BW (100G+), High Density Links, High HA CE CE (target 5 9’s) P Campus P DC Branch PE PE • Large scale routing metrics and table size Site P CE Segment Routing • Rich traffic engineer capabilities for granular Backbone control PE Location Managed Domain - DC - CoLocation • Automation transition for network Ops - no longer an option/nice-to-have Solution Enablers: Segment Routing (SR), • Granular telemetry, SLA conformance, billing SR-TE, Centralized WAN controller, high- speed convergence, Dynamic • Line rate encryption (100G+ - MACsec) performance measurements

Secure SD-WAN Core Design: https://xrdocs.io/design/enterprise/2019-07-26-secure© 2020 -Ciscosd- and/orwan its-core affiliates.-design/ All rights reserved. Cisco Public Private SD-WAN - Customer Trends Typical Deployment Reasons for choosing SD-WAN/MPLS over IP

Enterprise SD-WAN (Over the Top) • Customer has no interest in managing massive core WAN transport/HW SDN • Interest in driving down WAN costs using Controller/Mgmt mix of MPLS and Internet Branch • Increase in app transition to public cloud Site – Use of CoLo - Need for application CE SP MPLS Campus aware routing at branch (DIA to O365, DC Branch Internet CE other SaaS) Site CoLo CE CLoud • Does not need granular service levels within the WAN (service per transport) Managed Domain Managed Domain Overlay Encap • Link speeds <10G (most <1G)

• Technology Enabler: SD-WAN, MPLS VPN over mGRE, DMVPN TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 305 “Our rapid change of network requirements can no longer wait 30-60 days for our service provider to modify our segmentation [VRF] requests. We need this change management to be in minutes not days or weeks.” Fortune 50 CIO

Enterprise SD-WAN (Over the Top) • L3 VPN is native to SD-WAN, also offered in standard MPLS over mGRE SDN • Native to OMP, and MP-BGP (RFC 4364) Controller/Mgmt

• Allows customer to control their own L3 Branch segmentation, spin-up, tear-down, Site locations, etc… PE/CE SP MPLS Campus DC • VRF aware routing to client side Branch Internet PE/CE Site CoLo CLoud • Offers L3 VPN over IP with encryption PE/CE

• Technology Enablers: SD-WAN, MPLS Managed Domain Managed Domain VPN over mGRE, DMVPN Overlay Encap VRF’s

East Theater West Theater

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

• Introduction - Network Segmentation Drivers and Concepts

• Evaluating WAN Solution Option Criteria for L3 Segmentation

• Evolving Trends for Self Deployed Backbone Designs

• Evolving Trends and Solution Options for L3 Segmentation over IP and to the Public Cloud

• Architecture and Technology Innovations and Trends for the WAN

• End to End WAN Design and Components (Summary of Session)

• Self Deployed MPLS Backbone (SD-Core) Supporting MPLS BGP IP VPN Services (RFC 4364) WAN

• Self deployed MPLS BGP IP VPNs “over the top” of an SP Offered IP transport

LAN LAN

• Self Deployed MPLS Backbone (SD-Core) Supporting MPLS BGP IP VPN Services (RFC 4364) WAN

• Self deployed MPLS BGP IP VPNs “over the top” of an SP Offered IP transport

LAN LAN

East Theater West Theater

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

• L3 VPN Services • BGP VPN (RFC 4364), VPN over IP, Inter-AS, 6vPE

• L2 VPN Services - PW, VPLS, E-VPN

• Traffic Engineering - Explicit Path Routing • Traffic Engineering, disjoint paths, attributes for best path (latency, packet loss) • Optimisation of bandwidth, shift to Segment Routing TE (SR-TE)

• Bandwidth Protection Services - LFA, TI-LFA (IP FRR), MPLS TE FRR

• IP Multicast (per VPN/VRF, Rosen, LSM, BIER)

• Interworking with new solutions – VXLAN → L2/L3 VPN

• Leverage Segment Routing for Next-Gen Scale, Central Control, optimised services • Offers an “SD-MPLS” solution moving forward

Identity: VLAN, IP address, ACL Intent-based policy, policy follow identity Command Line Interface API / Model-Driven (REST, YANG), SDx, orchestrators (NSO) and controllers as dev platforms Complex backbone, holding entire state of network, Simplified Backbone (IP Fabrics, Segment Routing, multiple protocols (IGP, LDP, RSVP-TE) devices hold minimal state) Physical Devices Software Network Function Virtualization (NFV), and External Cabling Service Chaining, orchestration Best Path Limited to link Cost Enhanced path selection (BW + latency, jitter, loss) In-direct and high-latency Traffic Patterns Shift to CoLo Facilities, moving edge closer to apps Periodic Centralized Polling Model-driven Telemetry and configuration with ML/AI Limited Performance IP Encryption High-Speed encryption (10/100G+) support © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 318 Key Trends and Drivers to Next Gen Backbone

✔Simplified Core Backbone (Segment Routing)

✔Support Massive Scale, High Availability (Multi-Planar Design)

✔Incorporating automation, model-driven API’s, orchestration, and NFV

✔Leverage Co-Location Facilities, Create “Cloud Edge” close proximity to apps

✔Extend QoS and best path selection, beyond link cost (latency, jitter, loss, app)

✔Leverage real-time model-driven telemetry collection for ML/AI benefits (security, optimised network operations (AIOps) )

✔Support line-rate encryption (100/400G) transparent to network protocols

✔Support new transitions – 400G, Massive Devices (IoT), 5G core requirements, Application routing

An IP/MPLS source-routing architecture that seeks the right balance between distributed intelligence and centralized optimization

Path expressed in Data Plane Data the packet

MPLS IPv6 Dynamic path (segment labels) (+SR header)

Control Plane

Routing protocols with extensions SDN controller (IS-IS,OSPF, BGP)

Explicit path

Paths options

Dynamic Explicit (SPF computation) (expressed in the packet)

BRKRST-1124 – Introduction to Segment Routing

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 322 Segment Routing 101 BRKRST-2124 – Introduction to Segment Routing Monday, June 10, 4:00 PM - 5:30 PM | SDCC – Room 4

• Simple to deploy and operate • Leverage existing MPLS forwarding, HW, and services • straight-forward ISIS/OSPF extension to distribute labels • LDP/RSVP not required • exponentially less state in the routing elements for TE • agnostic control-plane also applicable to IPv6 • Provide for optimum scalability, resiliency and virtualization • Tighter integration with application • simpler network, highly programmable • Standards based driven The state is no longer in the network but in the packet

• Simplicity and complexity reduction in the core • Less protocols, reduced state, huge scale, highly programmable

• Protection with integrated TI-LFA FRR

• SR Traffic Engineering made simpler • BW optimization and capacity reaction (WAE + collection) • Disjointed paths (colored topology, SR Flex Algo) • SR-PCE (centralized SR-PCE, end-to-end awareness, multi-domain)

• Low-latency services using Performance Monitoring (PM) • Measure real-time per link delay measurement (loss coming in future) • Allows path selection based on link delay state, rather just cost

• SR On-Demand Next-Hops (BGP focused, SLA-aware per VPN)

• SR IGP Flexible Algorithms • Topology defined by operator, per service

Automate Network Operations Rich Network Data Sources ! 1 4 (Close Loop Implementation)

▪ MDT Change ▪ SNMP Automation ▪ BGP-LS ▪ NetFlow ▪ More Service Deployment Automation Multi-Layer Topology Deploy, Manage, Optimize Optical, Routing, Overlay Derive Actionable Insights Unified Data Collection & Distribution 2 3 (Analytics Apps)

▪ Multi-platform, Multi- vendor support ▪ Data Normalization ▪ Open and Secured Network Health Situation 3rd Party Insights Insights Manager Applications Common Collector Data Analysis Application

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 325 Cisco SP Automation Portfolio* Cisco Crosswork Implement the intent using model-based configuration Improve network SNR with Visualization, reporting Gain visibility with machine learning & service operations routing analytics NSO Network Services Crosswork Crosswork Orchestrator Crosswork Network Situation Active Topology & Manager Inventory Insights

Network planning & real time optimization Device and network health Remediate anomalies with Real time network Closed-loop with proactive KPIs custom playbooks optimization Automation Crosswork WAE Crosswork Change WAN Automation Crosswork Engine (+SR-PCE) Health Insights Automation Optimization

Scalable and Manage a Multi-layer, Multi- Distributed Collection service environment

EPNM Crosswork Evolved Programmable Data Gateway Network Manager Ecosystem Partners

*Portfolio items may be in various stages of development. Please contact your Cisco Account Representative for details © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public Summary of Goals and Targets - Next Gen Architecture • High availability (5 9s+) • Fast converging (targeting now < .5 sec) • Low latency (<50ms) and low jitter for real time communication services • Unicast and multicast traffic (Layer 2 or Layer 3) • Ultra-High Scalability (thousands to 100,000+ nodes, global scale) • Converged applications on a shared network • Traffic Engineering as needed • Fault-domain isolation and service segmentation • Greater Efficiency (higher average utilization) • Secure and Programmable Infrastructure • Maintenance with little to no customer impact BRKSPG-2535 – Next Gen Network Architectures – Cisco Live Barcelona

• Introduction - Network Segmentation Drivers and Concepts

• Evaluating WAN Solution Option Criteria for L3 Segmentation

• Evolving Trends for Self Deployed Backbone Designs

• Evolving Trends and Solution Options for L3 Segmentation over IP and to the Public Cloud

• Architecture and Technology Innovations and Trends for the WAN

• End to End WAN Design and Components (Summary of Session)

Enterprise SD-WAN (Over the Top) • Allows enterprises to deploy simpler-to- Solutions with/without SDN Controller manage MPLS VPN (v4/v6) solutions over IP

• CE owner (“us” ☺ ) controls the L3 VPN deployment

• PE (“SP”) provides transport of IP Branch Site • Key Benefit? PE/CE SP MPLS Campus 1. CE owner can still leverage cost effective L3 transport services, Internet, QoS SLA’s… from the DC Branch Internet PE/CE SP Site CoLo CLoud 2. CE owner controls policy, segmentation, topology, PE/CE encryption… “over the top”

• Target Use cases: simplified “Enterprise Managed Domain Managed Domain controlled” MPLS VPN over IP Transport Overlay Encap VRF’s

• Customer may not control the WAN transport Between MPLS networks

• Cannot depend on “end to end” label forwarding for transport

• Customer requires encryption for their PE to PE MPLS traffic • No native MPLS encryption exists today, must leverage IP

• MPLS over IP allows MPLS VPN solutions to leverage cost effective IP transport

In Summary, the Implementation Strategy Described Enables the Deployment of BGP/MPLS IP VPN Technology in Networks Whose Edge Devices are MPLS and VPN Aware, But Whose Interior Devices Are Not (Source: RFC 4797)

• Segmentation component • Virtual Route Forwarding Instance (VRF)

• Control Plane component • MP-BGP (RFC 4364) • SD-WAN L3 VPN - Overlay Management Protocol (OMP) • DMVPN L3 VPN - NHRP

• Data Plane component • MPLS over GRE/IP-UDP (RFC 4023)

• Service Support of Each Solution: QoS, IPv6 (selective), Encryption, Multicast, etc…

CP: MP-BGP DP: MPLS over IP (GRE/UDP) 1. Self Deployed MPLS Backbone (SD-Core) supporting MPLS BGP IP VPN Services

(RFC 4364) WAN

2. Self deployed L3 VPNs: “Over the top” of an SP Offered IP transport LAN LAN A. MPLS VPN over mGRE / DMVPN B. Cisco SD-WAN (Viptela)

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 342 MPLS VPN over Multi Point GRE (mGRE) GitHub Repo Location https://github.com/netwrkr95/mpls- mgre-configs Private MPLS VPN ”over the top” of SP Offered IP VPN Transport owns CE SP Managed “IP VPN” Service • Offers MPLS-VPN over IP MP-BGP VPNv4 • Inherit spoke-to-spoke communications CE • Uses standard RFC 4364 MP-BGP control Site 1 plane

• Uses standard MPLS over GRE data plane L3 VPN CE Service Site 3 Provider • Offers dynamic Tunnel Endpoint next-hop via Site 2 PE PE BGP CE • Requires only a single IP address for transport over SP network VRF’s Customer • Reduces configuration: Requires No LDP, No Managed Domain GRE configuration setup mGRE Interface

GRE any-to-any

• System dynamically configures mGRE tunnel (via tunnel profile)

• mGRE tunnel is decoupled from physical interface

• User traffic is in VRF/VPNv4 of mGRE payload (hidden from provider)

• Only a single IP address (source GRE/BGP-source) advertised to provider Source IP Address of • VRF, RD, RT mGRE tunnel advertised to provider network

WAN to To user Campus/DC Gold Provider networks with VRF mGRE SP WAN segmentation (802.1Q, Interface Transport port, etc…) Global PHY Blue • VRF, RD, RT Interface

Logical mGRE interface de-coupled from a physical interface

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 347 MPLS VPN over Multipoint GRE (mGRE) Feature Components PE2 2 PE3 4 iBGP View for PE 4 1 172.16.255.2 172.16.255.3 Tunnel Endpoint DB PE4 172.16.255.1 PE1 IP Transport 172.16.255.1 172.16.255.2 172.16.255.4 172.16.255.3 3 172.16.255.5 PE6 PE5 172.16.255.6 172.16.255.5 172.16.255.6 1▪ mGRE is a multipoint bi-directional GRE tunnel ▪ Control Plane leverages RFC 4364 using MP-BGP Multipoint GRE 2 Interface Signalling VPNv4 routes, VPN labels, and building IP next hop (locally) 3▪ VPNv4 label (VRF) and VPN payload is carried in mGRE tunnel encapsulation ▪ New encapsulation profile (see next slide) in CLI offers dynamic endpoint discovery: 4 (1) Sets IP encapsulation for next-hop (2) Installs signaled BGP peer and end-point into “tunnel endpoint database”

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 348 MPLS VPN over Multipoint GRE (mGRE) VPNv4 Configuration Example mGRE PE1 IPv4 PE4 CE2 CE1 Transport eBGP Lo0: 10.0.0.1 Lo0: 10.0.0.4 eBGP Example for PE4 interface Loopback0 ip address 10.0.0.4 255.255.255.255 ! l3vpn encapsulation ip Cisco Sets mGRE Encapsulation transport ipv4 source Loopback0 “Profile” for BGP Next-Hop ! router bgp 100 . . . address-family vpnv4 neighbor 10.0.0.1 activate Apply Route-Map to Received neighbor 10.0.0.1 send-community extended neighbor 10.0.0.1 route-map next-hop-TED in Advertisement from Remote iBGP exit-address-family . . . Neighbour ! route-map next-hop-TED permit 10 Use IP Encap (GRE) for Next-Hop and set ip next-hop encapsulate l3vpn Cisco Install Prefix in VPN Table as Connected IP Tunnel Interface TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 349 MPLS VPN over Multipoint GRE (mGRE) IPv6 (VPNv6) Configuration Example mGRE 2001:db8::2 /64 PE4 PE1 CE2 IPv4 Cloud E 1/0 CE1 eBGP Lo0: 10.0.0.1 Lo0: 10.0.0.4 eBGP Example for PE4 interface Ethernet 1/0 NOTE: Relevant MPLS VPN over mGRE vrf forwarding green Commands That Are Same for IPv4, Are Not ip address 209.165.200.253 255.255.255.224 Shown in This IPv6 Example ipv6 address 2001:db8:: /64 eui-64 ! router bgp 100 . . . IPv6 Address Applied to CE2 address-family vpnv6 Facing Interface neighbor 10.0.0.1 activate neighbor 10.0.0.1 send-community both neighbor 10.0.0.1 route-map next-hop-TED in Apply Route-Map to Received exit-address-family Advertisement from Remote iBGP . . . ! Neighbour (Same as vpnv4) route-map next-hop-TED permit 10 set ip next-hop encapsulate l3vpn Cisco set ipv6 next-hop encapsulate l3vpn Cisco Use IP Encap (GRE) for Next-Hop and Install IPv6 Prefix in VPNv6 Table as Connected Tunnel Interface

• Native Multicast VPN Support • Leverages standards-based Multicast VPN for multicast per VRF

• Encryption Solutions Enabling MPLS VPN over mGRE • Group Encrypted Transport VPN (GETVPN)

• QoS Recommendations • Follow non VRF best-practices • Keep consistent markings enterprise wide, per class

• Dealing with MTU with mGRE • Enhancements for ‘MPLS MTU’ above/below default (MTE = 1476)

• Simple: • Only requires advertising a single IP prefix to SP for mGRE operation • Dynamic Tunnel endpoint discovery is done via iBGP/route-map (no static GRE tunnel) • Solution requires NO manual configuration of GRE tunnels. LDP NOT required! • E-BGP can/is still be used for route exchange (mGRE end-point) with the SP • Standards Based - Leverages standard MP-BGP control plane (RFC 4364) • Flexible - Supports MVPN and IPv6 per MPLS VPN model (MDT and 6vPE respectfully) • Multi-platform support: - ASR 1000 series, ISR/G2, ISR 4xxx, SUP-2T, Cloud Services Router (CSR) • Supports Inter-AS VPN, Multicast VPN (MVPN), standard QoS/H-QoS • Supports IPSec for PE-PE encryption (GET VPN or manual SA) • Scales to 2000 PE’s with ASR 1000 series

http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns431/ns658/white_paper_c11-726689.pdf

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 353 Layer 3 Segmentation Using Cisco SD-WAN Cisco Software Defined WAN (SD- WAN) for L3 Segmentation Cisco SD-WAN (Viptela) L3 VPN Segmentation

IF IF MPLS • VPN 0: Transport (locked) • VPN 512: Mgmt (locked) Service Transport • VPN n: open user VPN (VPN n) (VPN0) IF IF INET

Management • VPNs enabler is VRF’s, each VRF having its (VPN512) own forwarding table IF • vEdge router allocates label to each of it’s service VPNs and advertises it as route attribute in OMP updates - VPN Labels used to identify customer VPN in the incoming packets

VPN 1 Interface VPN1 SD-WAN VPN1 Interface IPSec VPN 2 VLAN VPN2 VPN2 VLAN Tunnel VPN 3 Ingress Egress vEdge vEdge

IP UDP ESP VPN Data 20 8 36 4 …

• Segment connectivity across fabric w/o • Labels are used to identify VPN for reliance on underlay transport destination route lookup • vEdge routers maintain per-VPN routing • Interfaces and sub-interfaces (802.1Q table tags) are mapped into VPNs

▪ Isolated virtual private networks across any transport ▪ VPN isolation is carried over all transports - https://tools.ietf.org/html/rfc4023 ▪ VPN mapping is based on physical vEdge Router interface, 802.1Q VLAN tag or a mix of both

IF Site 1

IF TransportsVPNTransports A IF VPN 802.1q B IF VPN C 802.1q Data Centre

Site 2 IPSec

Full-Mesh Hub-and-Spoke • Each VPN can have it’s own topology - Full-mesh, hub-and-spoke, partial- mesh, point-to-point, etc… VPN1 VPN2 • VPN topology is influenced by leveraging control policies - Filtering TLOCs or modifying next-hop TLOC attribute for routes Partial Mesh Point-to-Point • Customer mission, business, and applications can drive a certain topology: • Applications in single cloud or on- VPN3 VPN4 prem can benefit from hub-spoke • voice takes full-mesh topology • Security compliance - PCI data takes hub-and-spoke topology

VPN TLOC from the remote address From the controller

• State, Country or Global based MPLS VPN where transport option is IP only

• The ”business requirement” mandates segmentation (refer to L3 segmentation use cases) • L3 VPN + encryption • L3 VPN over (e.g. transparently) non-MPLS (e.g. IP) transport, including Internet • L3 VPN Manages Services offering (managed CPE over L3 VPN/IP transport) • L3 VPN over proprietary encryption (external) devices (Government, Defense) • L3 VPN extension into the public cloud (per application segmentation)

• Extend Campus/DC ”policy” over the WAN • Cisco SD-Access = VN / Cisco ACI = VRF / Cisco SD-WAN = VPN

• Targets customers requiring “on-demand, self-deployed” L3 VPN turn-up

IPSec tunnel from DX / ER to Public Direct Connect to Public Internet connection DC Cloud through SP Cloud through co-locations

Branch Branch Branch Branch SP

SP Internet MPLS Internet Data Carrier PE Center Colocation Facility DX / ER Internet IPSec DX / ER

VPC/VNet VPC/VNet VPC/VNet VPC/VNet

IPsec Tunnel from MPLS carriers (L3 VPN carrier) Internet only for customer DC to the offers DX/ER as SP Managed DX/ER from the co-location connectivity. cloud Service to the cloud

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 380 Transit VPC Across regions, accounts/subscriptions VPC VPC VPC Shared A C Services …... • High Scale and Performance Spoke VPC • High Availability: Redundant VPN Tunnels with dynamic routing in a multi-AZ deployment

• Enterprise class routing features in the Transit VPC AZ1 AZ2 Transit VPC • Spoke VPC’s can leverage VGW or VPC CSRs Direct Connect Colocation • Scale-out options allow more Or Internet Facility forwarding when needed on demand ASR Other • See BRKARC-2749 for more Private DC Provider information (Past CL US Events) Hub Site Networks

Standard IPSec + BGP

BGP <-> OMP AZ1 • Fully automated through R vManage wizard VGW AZ2 IGW • Greatly simplifies brownfield AZ1 INET Host VPC vEdge GW integration - MPLS No changes are required on

AZ2 VGW Direct host VPCs vEdge GW Connect • Multipathing, segmentation, AZ1 Gateway VPC R QoS VGW vManage instantiated and AZ2 managed vManage • Fast failover

Host VPC - Speed of BGP convergence

AWS Region

Recommended Sessions: BRKCLD-3440 – Multi-cloud Networking

BGP <-> OMP AZ1 R

VGW AZ2 IGW Enterprise Network AZ1 INET Host VPC vEdge GW Melbourne

MPLS AZ2 VGW Direct ? vEdge GW Connect

AZ1 Gateway VPC R Enterprise Network VGW Perth

AZ2

Host VPC

AWS Region Tenant/Mission 1

BGP <-> OMP AZ1 R

VGW AZ2 IGW Enterprise Network AZ1 INET Host VPC vEdge GW Melbourne

MPLS

AZ2 VGW Direct vEdge GW Connect

AZ1 Gateway VPC R Cisco SD-WAN Enterprise Network VGW Virtual Fabric Perth

AZ2

Host VPC

AWS Region ▪ Extend SD-WAN Fabric into the public cloud Tenant/Mission 1 ▪ Transit VPC is “PE” from fabric to Host VPC ▪ Inter-VRF done through policy (or FW) but stays in region Tenant/Mission 2 TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 387 Summary and Positioning… What? When? Where? Summary of L3 VPN over IP WAN Techniques

Strengths / Weaknesses to help evaluate decision criteria Excellent Option

SubOptimal Option MPLS VPN o mGRE Cisco SD-WAN (Viptela) R3 Bad Option Routers only (no controller req) Controller Based (central) routing calculations Native VPN Multicast (MVPN)

Application Awareness

Transport Agnostic (Internet)

Large Scale VRF (>64)

“SD-WAN” Requirement (RFP)

Per VPN Topology (p2p, mesh)

Enterprise WAN Layer 3 Segmentation Solutions Enterprise WAN L3 Segmentation Solutions Let’s Recap

• Fully understand the application and network service requirements needed • Pace of Service turn-up times, transport available, operational expertise

• Self Deployed MPLS backbone target: • larger-scale, TE required, L2 VPN, tight control

• Layer 3 Segmentation over IP: • MPLS VPN over mGRE: simple MPLS VPN over IP, customer not ready for full- blown SD-WAN yet • Cisco SD-WAN: applications scattered across multiple locations (on-prem, public cloud, SaaS), leverage Internet as transport, cloud managed controller interest

• Assure the solution chosen suits the operational skill set of the IT org

• Keep is simple whenever possible

• Introduction - Network Segmentation Drivers and Concepts

• Evaluating WAN Solution Option Criteria for L3 Segmentation

• Evolving Trends for Self Deployed Backbone Designs

• Evolving Trends and Solution Options for L3 Segmentation over IP and to the Public Cloud

• Architecture and Technology Innovations and Trends for the WAN

• End to End WAN Design and Components (Summary of Session)

• Cloud Ready Network Design and Virtual DMZ

• Enhanced High Speed Encryption solutions for the WAN

• Leveraging automation in SD-WAN for Centralized Policy and Application Control

East Theater West Theater

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

Campus / Branch Campus© 2020 / Branch Cisco and/or its affiliates. All rights reserved. Cisco Public Secure Mobile Secure Mobile Next Generation Fed/Enterprise Architecture Network Architecture Transition in a Multi-Cloud World

Data Center

Cloud Public Cloud SDA Campus / Edge Branch SD-WAN Users

DNA Center vManage SaaS (DNAC) Co-Location Center

Devices Internet Direct Internet Access

Full Security Stack

Deliver Segmentation, Security, Automation, anytime, anywhere, Any transport © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public Cloud Ready Network Architecture Aligning WAN Design w/ Applications and Perimeter DMZ in Co-Location Centers MANAGEMENT

ORCHESTRATION SaaS Customers CONTROL

Secure Employees SD-WAN Physical or Private Internet Fabric Virtual DMZ Data Center Solution MPLS 4G/LTE Partners Public Cloud

Internet IoT

Office / Mobile App Aware SD-WAN Cloud Edge DMZ Management | Security | Policy | Orchestration | Analytics

Cisco Network Cloud Providers MANAGEMENT SAE Hub ORCHESTRATION Zones Customers CONTROL Orchestration and Management WAN

Data Center Internet SD-WAN MPLS Switching CoLo Cloud Employees 4G/LTE Secure Exchange Fabric Service Point

Partners NFV Appliance Internet SaaS

Virtual Network Services CoLo Space Secure DMZ Focus TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 407 Cisco Network Hub Securely Connecting Users Cloud and Application Providers

Cisco vManage / vBond SaaS

Customers Security Agility & Performance Cost Savings Private Data Centre Central policy Rapid provisioning, Lower OpEx and Network enforcement change control and CapEx through NFV. AnyConnect Hub scale-out architecture Reduce circuit costs Employees Branch via NFV fabric. Speed of and number of software with the circuits. performance of hardware. Partners Colocation / DC IaaS Turn-key orchestration and automation of enterprise WAN Service-Chains!

East Theater West Theater

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

• Bandwidth application requirements out- pacing IP encryption capabilities • Bi-directional and packet sizes further impact encryption performance • IPSec engines dictate aggregate performance of the platform (much lower link throughput) BW • Link speed = Encryption Engine Cost per bit for IPSec much more expensive time • Encryption must align with link speed (100G+) to support next-generation Link Speed applications IPSec Encryption Speed TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 419 What is MAC Security (MACsec)? Hop-by-Hop Encryption via IEEE802.1AE

• Hop-by-Hop Encryption model -Packets are decrypted on ingress port -Packets are in the clear in the device

Decrypt at Encrypt at -Packets are encrypted on egress port Ingress Egress 01101001010001001 01101001010001001 • Supports 1/10G, 40G, 100G encryption speeds 128bit AES GCM Encryption 01101001000110001001001000 everything in clear • Data plane (IEEE 802.1AE) and control plane (IEEE through the router

802.1x-Rev) MACsec PHY • Transparent to IPv4/v6, MPLS, multicast, routing • Encryption aligns with Link PHY speed (Ethernet)

128/256 bit AES GCM Encryption 128/256 bit AES GCM Encryption

01001010001001001000101001001110101 011010010001100010010010001010010011101010 01101001010001001

MKA Session

Service Provider Owned Routers/Bridges Data Data Centre Public Carrier Centre Ethernet Service Remote Central Campus/DC Campus/DC

• Leverage MACsec over “public” standard Ethernet transport MACsec MKA Session • Optimise MACsec + WAN features to accommodate running over public Ethernet transport MACsec Secured Path / MKA Session • Target “line-rate” encryption for high-speed applications MACsec Capable Router • Inter DC, MPLS WAN links, massive data projects MACsec Capable PHY

SP Owned Ethernet • Targets 100G, but support 1/10/40G as well Transport Device

• AES-256 (AES/GCM) support – 1/10/40 and 100G rates

• Target Next Generation Encryption (NGE) profile that currently leverages public NSA Suite B • Standards Based MKA key framework

• (defined in 802.1X-2010) within Cisco security development (Cisco “NGE”) • Ability to support 802.1Q tags in clear

• Offset 802.1Q tags in clear before encryption (2 tags is optional) • Vital Network Features to Interoperate over Public Carrier Ethernet Providers

• 802.1Q tag in the clear

• Ability to change MKA EAPoL Destination Address type

• Ability to change MKA Ether-type value

• Ability to configure Anti-replay window sizes • System Interoperability

• Create a common MACsec integration among all MACsec platforms in Cisco and Open Standards

• 10GE → 100GE High speed Site to Site E-LINE / E-LAN - Point to Multipoint Branch n • Campus, WAN, DC→DC, Metro E Branch 2 • Data Centre Interconnect

• High Speed replication and storage transfers Carrier Ethernet • IP/MPLS core/edge links (PE–P, P–P, PE–PE) Service • MPLS labels, VPN, Segment Routing is transparent to MACsec encryption Branch 1 Central • No GRE, simple. Encryption = Link BW Site • High Speed hub-and-spoke • Leverage low-cost/high-speed Metro E transport E-LINE - Point to Point • Simple configuration, no GRE tunnels Carrier Ethernet • Hybrid Encryption Design Options Service

• Ability to leverage BOTH MACsec and IPSec at various Central Central network points Site / DC 1 Site / DC 2

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 431 Hierarchical “Hybrid” MACsec + IPSec Design CSR MACsec IPsec High Throughput Encryption + Lower Scale Sites Lower Throughput Encryption + High Scale Sites

Co Lo Facility Regional IPsec Sites Hub 1 Branch Branch Internet Carrier Ethernet Service Branch Enterprise IPsec Network Branch

Internet Branch Regional Branch MPLS WAN Hub 2 (WAN MACsec) MACsec MACsec Metro E IPsec Branch Regional Hub 3 + DC • “Hybrid” design option for mix of scale, performance, leveraging Ethernet services • MACsec: Backbone/Core – Targets Higher BW, Lower Number of Sites • IPSec: Branch/back-haul – Targets Lower BW, high number of sites, cloud (CSR)

https://github.com/netwrkr95 • Ansible – MACsec Keychain Examples

• Ansible WAN MACsec Playbook and Configs (https://git.io/vQUR3 )

• YANG Models – MACsec Keychain Examples (Using YDK)

• MACsec Key Chain Configuration applications (https://git.io/vH7uD )

• What is YDK? (https://developer.cisco.com/site/ydk/ )

• Ansible Module Using YANG Models with YDK

• Ansible + YDK app (https://git.io/vH7XZ )

BRKRST-2309 – Introduction to WAN MACsec

http://www.cisco.com/c/dam/en/us/td/docs/solutions/Enterprise/Security/MACsec/WP-High-Speed-WAN-Encrypt-MACsec.pdf

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 436 Leveraging Automation for Simplifying Network Operations •“A feature without an API is not a feature. If I can’t automate it, “If it doesn’t have I won’t use it.” • Web 2.0 an API, it does Customer not exist.” Mitchell Hashimoto Co-Founder & CTO - HashiCorp

3rd Party Applications

vManage REST API’s & Webhooks 1

vManage

Cisco vManage Target customer has physical SD-WAN 1 edge appliances without a need for virtual CPE, service orchestration and OSS/BSS from Cisco SP Datacenter ENCS NFVI Cisco Router vEdge cEdge NFVIS (OpenStack, VMware

Challenge

• The ability to rapidly block/rate-limit different traffic types in the WAN • Traffic “match”: IP prefix, DCSP/IPP, protocol, application, etc. • Traffic “action”: drop, rate-limit, divert, re-mark, etc.

• Per box CLI does not scale and increases the “time to react”

• Suspected Vulnerabilities could also be anomalies (infected) detected via third-party tools/applications

• Controlling Non-business applications huge interest (NCAA basketball during March Madness, World Cup, etc. ☺)

Solution Options

• Leverage the centralized control point (vSmart) to push policy enforcement

• Centralized pushing of “match” and “action” policy that blocks or strictly polices a specific application and/or DSCP marking

• Offer the ability for operators to leverage the GUI

• Additionally, leverage API’s (vManage) that allow same capability, with faster deployment. • API’s Allow 3rd party applications (Splunk, ServiceNow), or open source tools (Ansible, Python, etc.) to trigger the enforcement • Eliminates operator intervention, offers more accelerated “action”

2 • vManage 18.3.3 1 • REST API’s on vManage

• Postman for REST API Testing GET PUT vManage MANAGEMENT • Ansible (2.6.12) and Python POST 3

vSmart Demo – Modify RTBH App “Match”

1. Leverage vManage GUI

2. Leverage REST API (using Postman) Policies Policies

3. Leverage REST API + Ansible-playbook A. custom Ansible module (Python)

B. REST API calls to vManage vEdge 10 vEdge 20

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 449 Demo - Dynamic Application Control Option # policy 3rd Party App 2 Driven (API) data-policy _VPN-99-List_RTBH-Spor_508995825 1 vpn-list VPN-99-List Operator Driven sequence 1 match app-list Suspect_Video_Apps GET ! PUT vManage Ansible Driven POST MANAGEMENT action drop (multiple steps count Blocked-Video_347240515 executed) log 3 ! vSmart ! default-action drop ! …

Policies Policies lists app-list Suspect_Video_Apps app foxsports app cbs_video Modify in active policy ! vEdge 10 vEdge 20 site-list ALL-VPN-99-Router-List

https://github.com/netwrkr95 Ansible Playbook – Automate Remote Trigger Black-hole (application list modification)

• https://github.com/netwrkr95/ansible_rtbh_vmanage_api

Ansible Playbooks – Automate MPLS VPN VRF Deployments

• Ansible VRF Creation and Deployment Playbook (https://github.com/netwrkr95/ansible-mpls-vpn

Ansible Playbooks – MACsec Keychain Examples

• Ansible WAN MACsec Playbook and Configs (https://git.io/vQUR3 )

YANG Models – MACsec Keychain Examples (Using YDK)

• MACsec Key Chain Configuration applications (https://git.io/vH7uD )

• What is YDK? (https://developer.cisco.com/site/ydk/ )

Ansible Module Using YANG Models with YDK

• Ansible + YDK app (https://git.io/vH7XZ )

https://developer.cisco.com

East Theater West Theater

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

• Understanding requirements, operational expertise, needs for “control” • Understanding this will dictate solution choices: SR-MPLS or “over-the-top” • Trade-offs: Complexity, Ops complexity, cost, service ”turn-up” times • Cisco SD-WAN offers private L3 segmentation, plus intelligent path control and future intelligence needed as apps are located in diversified locations • The cloud ready network architecture (SD-WAN + CoLo Cloud Edge) offers key elements for intelligent routing + security control closer to applications • WAN MACsec offers simple, high-speed encryption where Ethernet transport or dark fiber are leveraged • Embrace areas where automation and programmability can be leveraged to simplify and accelerate operations and deployment (Day 0 – Day 2+) • ALWAYS “Keep it Simple” when at all possible ☺

East Theater West Theater

Global 1 Tier IP/MPLS Core

In-Theater

IP/MPLS Core Tier 2 Tier

West Region East Region Private DC Co-Lo Center Co-Lo Center Private DC Internet

FTD FTD FTD FTD

SaaS IIaaSaaS 3 Tier

Cloud Services / Internet

Internet Internet Secure Internet Secure SD-WAN Internet SD-WAN Metro Metro MPLS MPLS Fabric Service Fabric Service 4G/LTE 4G/LTE

• The goal is for a simple, modular, hierarchical, structured design

• Business, technical, and physical requirements and constraints must all be considered

• Desired WAN availability and services have design implications

• Evolving technology is driving new WAN designs

• Leveraging Internet, Cloud, and CoLo now fundamental

• Network design should target how the applications survive a variation of outages. • Leverage load sharing capabilities for more resiliency and application performance • End-to-end convergence time is the goal, and can be affected by localized topology changes • Consider IP SLA based monitoring and SD-WAN for real-time path selection • Effective network designs incorporate a combination of convergence techniques

• Understand the application usage before adding services like QoS or Multicast to the WAN

• QoS should be always included in the initial WAN design deployment

• Leverage federated security cloud proxy and localized stack at the branch in a phased approach for consumption

• Don’t look at point solution for automation, rather look and the architecture and then fit the solution.

• Make L3 Segmentation a fundamental element in any new WAN designs • Understand the business and technical criteria for proper next-gen WAN solutions • Incorporate the Cloud Ready Design fundamentals into all new and existing designs moving forward • Leverage high-speed encryption (WAN MACsec) where applicable • Begin to incorporate automation tools into network operations to simplify and error-proof configuration changes • Keep it simple whenever possible!!!

TECCRS-2500 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 461 Q & A Complete your online session • Please complete your session survey survey after each session. Your feedback is very important.

• Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live t-shirt.

• All surveys can be taken in the Cisco Events Mobile App or by logging in to the Content Catalog on ciscolive.com/emea.

Cisco Live sessions will be available for viewing on demand after the event at ciscolive.com.

Demos in the Walk-in labs Cisco campus

Meet the engineer Related sessions 1:1 meetings