Enterprise Network Compute System

#CLUS WAN Architectures and Design Principles Stephen Lynn Consulting Systems Architect [email protected] CCIE 5507 (R&S/WAN/Security) CCDE 2013::56 @netw0rkStlynn BRKRST-2041

#CLUS Agenda . WAN Technologies & Solutions • WAN Transport Technologies • WAN Overlay Technologies • Wide Area Network Quality of Service • WAN Extension into the Cloud • Cisco vBranch with Enterprise NFV • SD-WAN

. WAN Architecture Design Considerations and Best Practices

. Summary

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Live Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKRST-2041 by the speaker until June 18, 2018.

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 WAN Transport Technologies The WAN Technology Continuum Early Networking Early-Mid 1990s Mid 1990s-Late 2000s Today Global Scale Flat/Bridged Multiprotocol Large Scale IP Ubiquity Experimental Networks Business Enabling Mission Critical Business Survival

Architectural Architectural Architectural Planning Lessons Lessons Lessons Protocols required for Route First, Redundancy Scale & Restoration Bridge only if Must ? Build to Scale

DMVPN X.25 Frame-Relay IPv6 NFV Internet 4G/LTE Protocol BGP 1960 1980 GRE 2000 Future

Metro-Ethernet ARPAnet 1970 RIP (BSD) 1990 2010 TCP/IP OSPF, Tag SDWAN ISDN, Switching GETVPN ATM #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 The Challenge

. Build a network that can adapt to a quickly changing business and technical environment

. Realize rapid strategic advantage from new technologies • IPv6: global reachability • Cloud: flexible diversified resources • Internet of Things • Fast-IT • What’s next?

. Adapt to business changes rapidly and smoothly • Mergers & divestures • Changes in the regulatory & security requirements • Changes in public perception of services

IP/MPLS Core Tier 1 Tier

In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region

Internet Cloud

Public Voice/Video Mobility Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

. Use hierarchy to manage network scalability and complexity while reducing routing algorithm overhead

. Hierarchical design used to be… • Three routed layers • Core, aggregation, access • Only one hierarchical structure end-to-end

. Hierarchical design has become any design that… • Splits the network up into “places,” or “regions” • Separates these “regions” by hiding information • Organizes these “regions” around a network core • “hub and spoke” at a macro level

. MPLS WAN is provided by a service provider

. As seen by the enterprise network, every site is one IP “hop” away

. Equivalent to a full mesh, or to a “hubless” hub-and-spoke

VRF VRF VRF VRF VRF VRF

. Virtualization at Layer 3 forwarding ! PE Router – Multiple VRFs ip vrf blue . Associates to Layer 3 interfaces on router/switch rd 65100:10 . Each VRF has its own route-target import 65100:10 route-target export 65100:10 Forwarding table (CEF) ip vrf yellow rd 65100:20 Routing process (RIP, OSPF, BGP) route-target import 65100:20 route-target export 65100:20 . VRF-Lite ! interface GigabitEthernet0/1.10 Hop-by-hop ip vrf forwarding blue . MPLS VPN interface GigabitEthernet0/1.20 ip vrf forwarding yellow Multi-hop

Single Provider Design Dual Providers Design Overlay Network Design

. Enterprise will home all . Enterprise will single or dual home . Overlay tunneling technologies sites into a single carrier to sites into one or both carriers to with encryption for provider provide L3 MPLS VPN provide L3 MPLS VPN connectivity. transport agnostic design connectivity. . Pro: Protects against MPLS service . Pro: Can use commodity . Pro: Simpler design with failure with Single Provider broadband services for lower consistent features cost higher bandwidth service . Pro: Potential business leverage for . Con: Bound to single better competitive pricing . Pro: Flexible overlay network carrier for feature velocity topology that couples from the . Con: Increased design complexity physical connectivity . Con: Does not protect due to service implementation against MPLS cloud failure differences (e.g. QoS, BGP AS . Con: Increased design with Single Provider Topology) complexity . Con: Feature differences between . Con: Additional technology providers could force customer to needed for SLA over use least common denominator commodity transport services features. BRKRST-2041 12 Single Carrier Site Types (Non-Transit)

. Dual Homed Non Transit Only advertise local prefixes (^$) Typically with Dual CE routers BGP design: eBGP to carrier iBGP between CEs Redistribute cloud learned routes into site IGP . Single Homed Non Transit Advertise local prefixes and optionally use default route.

. To guarantee single homed site reachability to a dual homed site experiencing a failure, transit sites had to be elected. . Transit sites would act as a BGP bridge transiting routes between the two provider clouds. . To minimize latency costs of transits, transits need to be selected with geographic diversity (e.g. from the East, West and Central US.)

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 Single vs. Dual Carriers Single Provider Dual Providers Pro: Common QoS support Pro: More fault domains model Pro: More product offerings to Pro: Only one carrier to “tune” business Pro: Ability to leverage vendors Pro: Reduced head end circuits for better pricing Pro: Nice to have a second Pro: Overall simpler design vendor option Con: Carrier failure could be Con: Increased Bandwidth catastrophic “Paying for bandwidth twice” Con: Do not have another Con: Increased overall design carrier “in your pocket” complexity Con: May be reduced to “common denominator” between carriers Resiliency Drivers vs. Simplicity

E-Line (Point-to-Point) E-LAN (Point-to-Multipoint) . Replaces TDM private line and . Offers point to multipoint Frame-Relay or ATM L2VPN connectivity . Point-to-point EVCs offer . Transparent to VLANs and Layer 2 predictable performance for control protocols applications . 4 or 6 classes of QoS support . One or more EVCs allowed per single physical interface (UNI) . Supports service multiplexing (Ex. Internet access and corporate VPN . Supports “hub & spoke” topology via one UNI) #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 MPLS (L3VPN) vs. Metro Ethernet (L2VPN)

MPLS Layer 3 Service MetroE Layer 2 Service

. Routing protocol dependent on the . Flexibility of routing protocol and carrier network topology independent of the carrier . Layer 3 capability depends on carrier offering . Customer manages layer 3 QoS

• QoS (4 classes/6 classes) . Capable of transport IP and non-IP • IPv6 capability traffic.

. Transport IP protocol only . Routing protocol determines scalability in point-to-multipoint . Highly scalable and ideal for large network topology

Layer 2 Overlays Layer 3 Overlays . Layer 2 Tunneling Protocol—Version 3 . IPSec—Encapsulating Security Payload (L2TPv3) (ESP) – Layer 2 payloads (Ethernet, Serial,…) – Strong encryption – Pseudowire capable – IP Unicast only . Other L2 overlay technologies – . Generic Routing Encapsulation (GRE) OTV, VxLAN – IP Unicast, Multicast, Broadcast – Multiprotocol support . Other L3 overlay technologies – MPLSomGRE, LISP, OTP #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 Tunnelling GRE and IPSec Transport and Tunnel Modes IP HDR IP Payload

GRE packet with new IP header: Protocol 47 (forwarded using new IP dst) IP HDR GRE IP HDR IP Payload

20 bytes 4 bytes

IPSec Transport mode 2 bytes ESP ESP IP HDR ESP HDR IP Payload Trailer Auth 20 bytes 30 bytes Encrypted Authenticated

IPSec Tunnel mode 2 bytes ESP ESP IP HDR ESP HDR IP HDR IP Payload Trailer Auth 20 bytes 54 bytes Encrypted Authenticated #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 Cisco Site to Site VPN Technologies Comparison

Features DMVPN FlexVPN GET VPN . Public or Private Transport . Private IP Transport . Public or Private Transport Infrastructure Network . Overlay Routing . Flat/Non-Overlay IP . Overlay Routing . IPv4/IPv6 dual Stack Routing . Large Scale Hub and Spoke with . Converged Site to Site and . Any-to-Any; Network Style dynamic Any-to-Any Remote Access (Site-to-Site)

. Dynamic Routing or IKEv2 Route . Active/Active based on Dynamic . Transport Routing Failover Redundancy Distribution Routing . COOP Based on GDOI . Server Clustering

. Unlimited . Unlimited . 8000 GM total Scalability . 3000+ Client/Srv . 3000+ Client/Srv . 4000 GM/KS

. Multicast replication in IP . Multicast replication at hub . Multicast replication at hub IP Multicast WAN network

. Per SA QoS, Hub to Spoke QoS . Per Tunnel QoS, Hub to Spoke . Transport QoS . Per SA QoS, Spoke to Spoke Policy Control . Locally Managed . Centralized Policy Management . Locally Managed . Tunneled VPN . Tunneled VPN . Tunnel-less VPN Technology . Multi-Point GRE Tunnel . Point to Point Tunnels . Group Protection . IKEv1 & IKEv2 . IKEv2 Only . IKEv1

. Branch spoke sites establish an IPsec tunnel to and SECURE ON-DEMAND TUNNELS register with the hub site

. IP routing exchanges prefix information for each site ASR 1000 Hub . BGP or EIGRP are typically used for scalability IPsec Branch n . With WAN interface IP address as the tunnel source VPN ISR address, provider network does not need to route

customer internal IP prefixes ISR ISR Branch 1 . Data traffic flows over the DMVPN tunnels Branch 2 . When traffic flows between spoke sites, the hub assists the spokes to establish a site-to-site tunnel Traditional Static Tunnels DMVPN On-Demand Tunnels . Per-tunnel QOS is applied to prevent hub site Static Known IP Addresses oversubscription to spoke sites Dynamic Unknown IP Addresses

. Spokes build a dynamic permanent GRE/IPsec tunnel to the Dual DMVPN Design hub, but not to other spokes. They register as clients of the Single mGRE tunnel on Hub, NHRP server (hub) and register their NBMA address two mGRE tunnels on Spokes 192.168.0.0/24 . Active-Active redundancy model—two or more hubs per spoke Physical: 172.17.0.5 Physical: 172.17.0.1 . All configured hubs are active and are routing neighbors Tunnel0: 10.0.1.1 Tunnel0: 10.0.0.1 with spokes . Routing protocol routes are used to determine traffic forwarding Physical: (dynamic) Tunnel0: 10.0.0.12 . A spoke will initially send a packet to a destination (private) Tunnel1: 10.0.1.12 subnet behind another spoke via the hub, and the hub will send it an NHRP redirect. . The redirect triggers the spoke to send an NHRP query for the data packet destination address behind the destination spoke .1 . The destination spoke initiates a dynamic GRE/IPsec tunnel to 192.168.3.0/24 the source spoke (it now knows its NBMA address) and sends Physical: (dynamic) Tunnel0: 10.0.0.11 the NHRP reply. Tunnel1: 10.0.1.11

. The dynamic spoke-to-spoke tunnel is built over the mGRE .1 .1 interface 192.168.1.0 /24 192.168.2.0 /24 . When traffic ceases then the spoke-to-spoke tunnel is removed

Spoke-to-hub tunnels Spoke-to-spoke tunnels

2547oDMVPN tunnels Increase in Scale

Hub and spoke Spoke-to-spoke VRF-lite

Server Load Balancing Hierarchical 2547oDMVPN

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Any-to-Any Encryption Before and After GETVPN Public/Private WAN Private WAN Before: IPSec P2P Tunnels After: Tunnel-Less VPN

WAN

Multicast . Scalability—an issue (N^2 problem) . Scalable architecture for any-to- . Overlay routing any connectivity and encryption . Any-to-any instant connectivity . No overlays—native routing can’t be done to scale . Any-to-any instant connectivity . Limited QoS . Enhanced QoS . Inefficient Multicast replication . Efficient Multicast replication #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Group Security Functions Routing Member Key Server . Forwarding Key Server . Replication . Validate Group Members . Routing . Manage Security Policy . Create Group Keys . Distribute Policy/Keys Group Member Routing Members

Group Member Group Group Member Member . Encryption Devices . Route Between Secure/ Unsecure Regions Group . Multicast Participation Member

Key Encryption Key (KEK) Traffic Encryption Key (TEK) Group Member Routing Members

Group Member Group Member RFC3547RFC6407: Group Domain of Group Interpretation (GDOI) Member

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 GETVPN - Group Key Technology Operation Example GM3 GM4 GM2 . Step 1: Group Members (GM) GM5 “register” via GDOI (IKE) with the GM1 Key Server (KS) GM6 • KS authenticates and authorizes the GM GM9 KS GM8 GM7 • KS returns a set of IPsec SAs GM3 for the GM to use GM4 GM2

. Step 2: Data Plane Encryption GM5 GM1 • GM exchange encrypted traffic using the GM6 group keys GM9 KS • The traffic uses IPSec Tunnel Mode with GM8 GM7

“address preservation” GM3 GM4 GM2 . Step 3: Periodic Rekey of Keys GM5 • KS pushes out replacement IPsec GM1 keys before current IPsec keys expire; GM6 This is called a “rekey” GM9 KS GM7 #CLUS BRKRST-2041 GM8© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 GETVPN Virtualization Deployment Model GETVPN Segmented WAN CE PE PE CE

MPLS VPN

LISP with GETVPN

CE PE PE CE

GET Encrypted LISP

LISP over GETVPN

• Bandwidth application requirements out- pacing IP encryption capabilities

• Bi-directional and packet sizes further impact encryption performance

• IPSec engines dictate aggregate performance of the platform (much lower link throughput) BW

Link speed = Encryption • Cost per bit for IPSec much more Engine expensive

time • Encryption must align with link speed (100G+) to support next-generation applications Link Speed IPSec Encryption Speed BRKRST-2041 31 MACSec – Line Rate L2 Encryption Solution

Authenticated Encrypted 6 Bytes 6 Bytes 8-16 Bytes 4 Bytes 8 Bytes 2 Bytes 8-16 B 4 Bytes DMAC SMAC 802.1AE Header 802.1Q CMD ETYPE PAYLOAD ICV CRC

0x88e5

MACsec EtherType TCI/AN SL Packet Number SCI (optional)

MACsec Tag Format

✓ Frames are encrypted and protected with Advanced Encryption Standard Galios/Counter Mode (AES-GCM-128)

✓ Line Rate Encrypted Ethernet performance of the port (PHY). Speeds 1/10G, 40G, 100G

✓ MACsec Ethertype is 0x88e5

✓ No impact to IP MTU/Fragmentation

✓ Reduced interoperability issues with other L3 Features MACsec Deployment Models

. Data Center Interconnect . Typically seen with utility . Clear-Tag Feature: Option . Point-to-point link dark company for 802.1Q tag in clear for fiber or DWDM  . Point-to-point link dark Metro-Ethernet connection fiber or DWDM  deployment . Provide line rate encryption connection . Option to modify EAPoL with high speed links . Connecting utility stations destination MAC and between DCs for replication together and provide link EtherType to avoid MKA traffic encryption packet being consumed

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 For Your MACsec Example Reference key chain DC1-to-DC2 macsec key 01 key-string 1234567890123456789012345678901234567890123456789012345678901234 cryptographic-algorithm aes-256-cmac lifetime 00:00:00 january 01 2018 infinite ! macsec-policy ACME_100G security-policy must-secure window-size 128 cipher-suite GCM-AES-XPN-256 key-server-priority 0 ! interface HundredGigE0/3/0/0 macsec psk-keychain DC1-to-DC2 policy ACME_100G

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 WAN Quality of Service Quality of Service Operations How Does It Work and Essential Elements Classification and Marking Queuing and Post-Queuing Dropping Operations

. Classification and Marking: • The first element to a QoS policy is to classify/identify the traffic that is to be treated differently. Following classification, marking tools can set an attribute of a frame or packet to a specific value. . Policing: • Determine whether packets are conforming to administratively-defined traffic rates and take action accordingly. Such action could include marking, remarking or dropping a packet. . Scheduling (including Queuing and Dropping): • Scheduling tools determine how a frame/packet exits a device. Queuing algorithms are activated

only when a device is experiencing#CLUS congestionBRKRST and-2041 are deactivated© 2018 Cisco and/or its when affiliates. All the rights congestionreserved. Cisco Public clears.38 Enabling QoS in the WAN Traffic Profiles and SLA Requirements

Voice SD Video Conf Telepresence Data

. Smooth . Bursty . Bursty . Smooth/bursty . Benign . Greedy . Drop sensitive . Benign/greedy . Drop sensitive . Drop sensitive . Delay sensitive . Drop insensitive . Delay sensitive . Delay sensitive . Jitter sensitive . Delay insensitive . UDP priority . UDP priority . UDP priority . TCP retransmits

Bandwidth per Call SD/VC has the Same HD/VC has Tighter Traffic patterns for Depends on Codec, Requirements as Requirements than Data Vary Among Sampling-Rate, VoIP, but Has VoIP in terms of jitter, Applications and Layer 2 Media Radically Different and BW varies based Traffic Patterns on the resolutions . Latency ≤ 150 ms (BW Varies Greatly) . Data Classes: . Jitter ≤ 30 ms . Latency ≤ 150 ms . Latency ≤ 200 ms . Mission-Critical Apps . Loss ≤ 1% . Jitter ≤ 30 ms . Jitter ≤ 20 ms . Transactional/Interactive Apps . Bandwidth (30- . Loss ≤ 0.05% . Loss ≤ 0.10% . Bulk Data Apps 128Kbps) . Best Effort Apps (Default) . Bandwidth (1Mbps) . Bandwidth (5.5- One-Way Requirements One-Way 16Mbps) Requirements #CLUSOne-WayBRKRST Requirements-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 policy-map WAN class VOICE Scheduling Tools priority percent 10 class VIDEO LLQ/CBWFQ Subsystems priority percent 23 class CRITICAL-DATA IOS Interface Buffers bandwidth percent 15 random-detect dscp-based class DATA 1 Mbps VoIP bandwidth percent 19 Policer random-detect dscp-based LLQ class SCAVENGER 5 Mbps RT-Interactive bandwidth percent 5 Policer class NETWORK-CRITICAL bandwidth percent 3 service-policy MARK-BGP class class-default bandwidth percentPackets 25 Packets random-detect Out In CBWFQ Scheduler Tx-Ring

CBWFQ

Without Traffic Shaping Line Rate With Traffic Shaping Shaped Rate

Traffic Shaping Limits the Transmit Rate to a Value Lower Than Line Rate

. Policers typically drop traffic

. Shapers typically delay excess traffic, smoothing bursts and preventing unnecessary drops

. Very common with Ethernet WAN, as well as Non-Broadcast Multiple-Access (NBMA) network topologies such as Frame- Relay and ATM

Two Levels MQC Policy-map PARENT Policy-map CHILD class class-default class VOICE Gig 0/1 shape average 150000000 priority percent 10 Service Level service-policy output CHILD class VIDEO priority percent 23 Interface gigabitethernet 0/1 class CRITICAL-DATA Best Effort service-policy output PARENT bandwidth percent 15 random-detect dscp-based class DATA bandwidth percent 19 Scavenger 150 Mbps random-detect dscp-based Video class SCAVENGER bandwidth percent 5 Critical Data class NETWORK-CRITICAL bandwidth percent 3 Voice Network service-policy MARK-BGP Critical class class-default bandwidth percent 25 random-detect

ToS byte is copied to the

new IP Header IP HDR IP Payloaad ToS

GRE Tunnel GRE

IP HDR IP HDR IP Payload ToS HDR ToS

IPSec Tunnel mode

ESP ESP

IP HDR ESP HDR IP HDR IP Payload ToS ToS Trailer Auth

WAN

Router order Line install Router delivery Router install Online router

Service 1

Appliance WAN

Service order Appliance delivery Appliance install Online appliance

Service 2 Appliance

Appliance WAN

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 What is Cisco vBranch? Network services in minutes, on any platform Cisco DNA Center (DNAC) Cisco Network Service Orchestrator (NSO) / Virtual Managed Services (VMS)

Virtual WAN Virtual Wireless Virtual Router Virtual Firewall Optimization LAN Controller 3rd Party VNFs (ISRv/vEdge) (ASAv) (vWAAS) (vWLC)

Network Functions Virtualization Infrastructure Software (NFVIS)

ISR 4000 + Enterprise Network UCS C-Series UCS E-Series Compute System

Traditional Enterprise NFV

Physical Router Virtual Router Virtual Router Physical Router Ne w ! Virtual Services Virtual Services Virtual Services

4000 Series ISR + Enterprise Network Cisco® 4000 Series ISR UCS® E-Series Compute System (ENCS) UCS C-Series, COTS

Centralized services Upgradable hardware Elastic routing and services Fixed integrated services Deterministic routing Elastic routing and services Performance Conservative performance Router / Server Hybrid Early adopter

Cisco ONE™ Access to Ongoing License Investment Innovation Portability Protection BRKRST#CLUS-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 Branch/Campus Platform Built for Enterprise NFV Colocation Center ENCS 5000 Series for the Branch Public Cloud Best of Routing Complete Open for Third Party & Compute Virtualized Services Services and Apps

Enterprise Network Compute System

ENCS 5100 Series

8 Integrated LAN Ports ENCS 5400 Series with Optional POE USB 3.0 Storage 2 Onboard Gigabit Network Interface Hardware Acceleration for Ethernet ports Module for LTE & 2 HDD or SSD VM Traffic with SFP legacy WAN RAID 0 & 1

ISRv/vEdge ASAv/FTD* vWAAS vWLC Application High Performance Full DC-Class Built for small and Optimization and Featured Functionality medium branches Rich Features Akamai Connect

Windows Server Linux 3rd Party Active Directory, File Network Services Custom Applications Share, Server Management & DNS/DHCP Applications Monitoring

Branch router

IPS/IDS appliance

WAAS appliance N F V I S Patch panel N F V I S

Firewall appliance

A single x86 compute platform housing multiple VNFs

Application Traffic Per-Segment Secure Cloud Cloud Transport SLA Engineering Topologies Perimeter Path Accel Hub

Analytics Application Policies

Routing Security Segmentation QoS Multicast Svc Insertion Survivability Monitoring

Delivery Platform Operations

Broadband MPLS Cellular

Transport Independent Fabric

Orchestration Plane vOrchestrator

vBond MANAGEMENT

Management Plane API (Multi-tenant or Dedicated) ORCHESTRATION

vManage ANALYTICS Control Plane CONTROL (Containers or VMs)

INTERNET MPLS 4G vSmart

Data Plane (Physical or Virtual)

Data Center Campus Branch Home Office vEdge

vSmart . TCP based TLS/DTLS control plane protocol . Runs between vEdge routers and vSmart controllers and between the vSmart controllers . Advertises control plane context, i.e. TLOCs, unicast/multicast destinations, service routes (L4-L7), BFD stats (TE and H-SDWAN) and Cloud onRamp for SaaS probe stats (gateway) vSmart vSmart . Distributes IPSec encryption keys, and data and app-aware policies (embedded NETCONF)

vEdge vEdge

Note: vEdge routers need not connect to all vSmart Controllers

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59 Fabric Operation Walk-Through OMP Update: vSmart OMP . Reachability – IP Subnets, TLOCs . Security – Encryption Keys DTLS/TLS Tunnel . Policy – Data/App-route Policies IPSec Tunnel OMP OMP BFD Update Update Policies OMP OMP Update Update

vEdge Transport 1 vEdge

TLOCs TLOCs

VPN1 VPN2 Transport 2 VPN1 VPN2 BGP, OSPF, BGP, OSPF, Connected, Connected, Static A B C D Static

Subnets Subnets

Single-touch centralized vSmart security policy Controllers - Access Control List App - Application Firewalling Policies

ACL/ TransportsTransports ACL/ App App Transports

User Site Data Center Server vEdge vEdge

. Strong security posture - Regionalized stateful network Regional DC/Colo services vEdge . Multiple network services - Service chaining

Network Service Nodes Data traffic Control Plane #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64 Use Cases & Deployments Supporting a diverse set of topologies and architectures at scale

Technology Use Cases – M&A, Line-of-business Fully Managed WAN With Centralized Control separation, Partner network Segmentation & Multi-Topology Data Center Virtual Fabric Data Center Enterprise NOC & NAC & MDM Access DC User Control

VPN1

t i MPLS CoLo

S VPN2

WAN Video CoLo & DMZ VPN2 Video

Internet

WAN Opt & VPN1 e caching

i Public Cloud User Traffic Branch S User routing & & Network Video Traffic switching Services Viptela vEdge

• Independent and isolated virtual topologies operating at the same time Enterprise Unified Wireless Communications

14 Viptela Confidential 12 Viptela Confidential #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65 SD-WAN TCP Maximum Segment Size Adjustment Packet Fragmentation Avoidance for TCP Traffic

IPSec Link IP MTU Link IP MTU 1500 Bytes 1500 Bytes Host Application vEdge Secure Fabric vEdge Router Router Servers Signaled MSS Signaled MSS 1460B MSS Adjust 1320B Send MSS to 1320B 1320B Signaled MSS Signaled MSS Send MSS 1320B MSS Adjust 1460B 1320B to 1320B . Tunneling introduces additional overhead and increases packet size . Send TCP MSS is min (local link IP MTU - 40B*, signaled MSS value) - Signaled in SYN packets . vEdge Router TCP MSS adjustment overrides signaled MSS value - Each side determines its own MSS value, not synchronized

* 20B IP header + 20B TCP header. Does not include TCP options.

Cloud SaaS Customers Secure Agile Exchange Customers

Colocation Centers

Private Secure Agile Data Center Employees Exchange Employees

Partners DMZ Partners Public Cloud Private Applications Data Center

Corporate DC

AWS Managed VPN

Cisco Internet ISR/ASR

VGW

VLAN A VLAN B CSR 1000V VLAN C Corporate DC AWS Direct Customer Connect POP Cage

Cisco Private VIF Colocation Facility ISR/ASR CSR 1000V

. Represents logically isolated network  Separate IP range, Subnet, Routes, and Security  Overlap IP address allowed

. NO Support for Multicast/Broadcast traffic, Internet GW VLANs, and Transit Routing Subnet Router . Network Controls Available to you: . Internet gateway forward traffic to  Route Tables, aka VRFs outsides and in between VPCs  Internet Gateway (IGW), aka an Internet Router  Security Groups and Network ACLs . Subnet Router forward traffic within  VPNaaS the VPC

. Connect two VPCs together requires network connection – VPC Peering

. VPC Peering enables routing traffic between VPC with IPv4 or IPv6 addresses

. VPC Peering is one-to-one relationship between two VPCs.

. Full Mesh connection between VPCs Cisco ISR/ASR faces scaling challenges.

Corporate DC

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75 Inter-VPN Design using Transit VPCs with CSR1000v . End to end routings between VPCs rd in all regions and other none AWS App VPC Shared Services VPC 3 Party VPC network

. Used for hierarchical designs VGW • Scaling beyond VPC peering limits • Security/Monitoring Services in Transit VPC

. End-to-end encryption with Availability Availability Zone 1 Zone 2 DMVPN or SDWAN Transit VPC

. Redundant CSR1000V across 2xAZ Direct for HA and fast convergence Connect

. Automation brings new spoke VPCs up and into the routing table in

minutes HQ/DC

AWS Network Backbone

AZ 1 AZ 2

Transit VPC AZ 1 AZ 2

US-West Transit VPC

US-East

Corporate Network/DC

Campus/ Data Data Center Center/ Campus WAAS Service

WAN Key Services/ Servers Distribution

VPN Termination

WAN Edge MPLS A MPLS B

Internet

Campus/ Core Layer Data Center

WAN Distribution Layer EIGRP AS 100

Summaries+ Default

DMVPN Hub Routers EIGRP AS = 100 EIGRP AS = 100 EIGRP AS = 100 iBGP Internet Edge

BGP AS = 65511 EIGRP AS = 200 MPLS CE BGP AS = 65511 Layer 2 WAN Routers CE Router

eBGP Layer 2 DMVPN 1 DMVPN 2 Internet MPLS A MPLS B WAN

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86 WAN Edge Connection Methods Compared Recommended Multi-Chassis EtherChannel VSS/3850 Stacks Shared Si Layer 3 LAN P-to-P Link

WAN WAN WAN

. No Static Routes . No First Hop Redundancy Protocols

VSS/3850S tacks

Si Layer 3 P-to-P Link Channel Member Removed IGP recalc

. Link redundancy achieved through . Provide Link Redundancy and reduce redundant L3 paths peering complexity . Flow based load-balancing through . Tune L3/L4 load-balancing CEF forwarding across hash to achieve maximum utilization . Routing protocol reconvergence . No L3 reconvergence required when when uplink failed member link failed . Convergence time may depends on . No individual flow can go faster than routing protocol used and the size of the speed of an individual member of routing entries the link

. ECMP convergence is dependent on the number of Si Layer 3 routes P-to-P Link

. MEC convergence is consistent, independent of the number of routes

2.5

2 ECMP MEC Max 1.5 VSS/3850 Stacks

0.5 secoflostvoice

0 1000 3000 6000 9000 12000 NumberNumber of of Routes Routes - Sup720C

. In principle, redundancy is easy . Any system with more parallel paths through the system will fail less often . The problem is a network isn’t really a single system but a group 2.5 of interacting systems . Increasing parallel paths increases routing complexity, therefore increasing convergence times Seconds

0 Routes 10000

. It is important to force summarization Campus/ Summary at the distribution towards WAN Edge Data Center 10.5.0.0/16 and towards campus & data center . Summarization provides topology change isolation. Summaries + . Summarization reduce routing table Default size. 10.4.0.0/16 0.0.0.0/0.0.0.0 interface Port-channel1 description Interface to MPLS-A-CE no switchport ip address 10.4.128.1 255.255.255.252 ip pim sparse-mode ip summary-address eigrp 100 10.5.0.0 255.255.0.0

MPLS A MPLS B

. Mutual route redistribution between protocols can cause routing loops without preventative measures IGP Domain . Use route-map to set tags and then redistribute (EIGRP/OSPF) based on the tags . Routes are implicitly tagged when distributed from Campus eBGP to EIGRP/OSPF with carrier AS . Use route-map to block re-learning of WAN routes via the distribution layer (already known via iBGP)

router eigrp 100 distribute-list route-map BLOCK-TAGGED-ROUTES in default-metric [BW] 100 255 1 1500 MPLS WAN redistribute bgp 65500

route-map BLOCK-TAGGED-ROUTES deny 10 BGP Domain match tag 65401 65402

route-map BLOCK-TAGGED-ROUTES permit 20

. Run iBGP between the CE routers to exchange prefixes associated with each Campus carrier . CE routers will use only BGP path selection information to select both the primary and 10.5.128.0/2 secondary preferences for any destinations 1 announced by the IGP and BGP iBGP . Use IGP (OSPF/EIGRP) for prefix re- advertisement will result in equal-cost paths at remote-site MPLS A MPLS B bn-br200-3945-1# sh ip bgp 10.5.128.0/21 BGP routing table entry for 10.5.128.0/21, version 71 Paths: (2 available, best #2, table default, RIB-failure(17)) Not advertised to any peer 65401 65402, (aggregated by 65511 10.5.128.254) 10.4.142.26 from 10.4.142.26 (192.168.100.3) Origin IGP, localpref 100, valid, external, atomic- A B aggregate 65402, (aggregated by 65511 10.5.128.254) iBGP 10.4.143.26 (metric 51456) from 10.5.0.10 (10.5.0.253) 10.5.128.0/21 Origin IGP, metric 0, localpref 100, valid, internal, atomic-aggregate, best #CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93 Best Practice - Implement AS-Path Filter Prevent Branch Site Becoming Transit Network

. Dual carrier sites can unintentionally become Campus transit network during network failure event and causing network congestion due to transit traffic . Design the network so that transit path between two carriers only occurs at sites with enough bandwidth . Implement AS-Path filter to allow only locally originated routes to be advertised on the outbound updates for branches that should not MPLS A MPLS B be transit router bgp 65511 neighbor 10.4.142.26 route-map NO-TRANSIT-AS out ! ip as-path access-list 10 permit ^$ A B ! route-map NO-TRANSIT-AS permit 10 iBGP match as-path 10

EIGRP OSPF

. Internal EIGRP – Admin Dist. 90 . Admin Dist. 110

. External EIGRP – Admin Dist. 170 . Route Preference

1. Intra-Area . Metric Calculation metric = bandwidth + delay 2. Inter-Area 3. External E1 (Internal + External Cost) • Bandwidth (in kb/s) 4. External E2 (External Cost) • Delay (in microseconds) . Cost Calculation Cost= Reference BW / Interface BW Default Reference BW = 100Mbps

Campus . eBGP routes are redistributed into EIGRP 100 as EIGRP external routes with default Admin Distance 170 AS100

10.4.128.2 . Running same EIGRP AS for both campus and DMVPN network would result in Internet path

eBGP preferred over MPLS path

MPLS A Internet

EIGRP AS100

Campus D EX 10.5.48.0/21 [170/28416] via 10.4.128.2 . eBGP routes are redistributed into EIGRP 100 as external routes with default Admin Distance 170 EIGRP AS100 . Running same EIGRP AS for both campus and DMVPN 10.4.128.2 network would result in Internet path preferred over

MPLS path eBGP . Multiple EIGRP AS processes can be used to provide control of the routing . EIGRP 100 is used in campus location EIGRP 200 over DMVPN tunnels Internet MPLS A . Routes from EIGRP 200 redistributed into EIGRP 100 appear as external route (distance = 170)

EIGRP . Routes from both WAN sources are equal-cost paths. AS200 To prefer MPLS path over DMVPN use eigrp delay to modify path preference

MPLS CE router#

. eBGP as the PE-CE Routing Protocol Campus

. MPLS VPN as preferred path learned via EIGRP AS100 eBGP

. Secondary path via backdoor IGP link R1 R2 eBGP (EIGRP or OSPF) over tunneled connection IGP (DMVPN over Internet) Backup Link

. Default configuration the failover to MPLS A Internet backup path works as expected

10.4.160.0/24

Campus . After link restore, MPLS CE router receives EIGRP BGP advertisement for remote-site route. AS100 . Does BGP route get (re)installed in the

route table? R1 R2

eBGP IGP

D EX 10.4.160.0/24 [170/3584].... Backup Link

MPLS A Internet

R1# show ip route B 10.4.144.0/24 [20/0] via 10.4.142.2, 01:30:06 B 10.4.145.0/24 [20/0] via 10.4.142.2, 01:30:06 D EX 10.4.160.0/24 [170/3584] via 10.4.128.9, 00:30:06

B 10.4.160.0/24 [20/0].... 10.4.160.0/24

BGP Prefers Path with: 1. Highest Weight 2. Highest Local Preference 3. Locally originated (via network or aggregate BGP) 4. Shortest AS_PATH 5. Lowest Origin type IGP>EGP>INCOMPLETE (redistributed into BGP) 6. Lowest Multi-Exit Discriminator (MED) 7. Prefer Externals (eBGP over iBGP paths) 8. Lowest IGP metric to BGP next hop (exit point) 9. Lowest Router ID for exit point

. Routes redistributed into BGP are considered locally originated and get a default weight of 32768

. The eBGP learned prefix has default weight of 0

. Path with highest weight is selected

ASR1004-1#show ip bgp 10.4.160.0 255.255.255.0 BGP routing table entry for 10.4.160.0/24, version 22 Paths: (3 available, best #3, table default) Advertised to update-groups: 4 5 65401 65401 10.4.142.2 from 10.4.142.2 (192.168.100.3) Origin IGP, localpref 200, valid, external Local 10.4.128.1 from 0.0.0.0 (10.4.142.1) Origin incomplete, metric 26883072, localpref 100, weight 32768, valid, sourced, best

. To resolve this issue set the weights on route learned via eBGP peer higher than 32768

neighbor 10.4.142.2 weight 35000

ASR1004-1#show ip bgp 10.4.160.0 255.255.255.0 BGP routing table entry for 10.4.160.0/24, version 22 Paths: (1 available, best #1, table default) Not advertised to any peer 65401 65401 10.4.142.2 from 10.4.142.2 (192.168.100.3) Origin IGP, metric 0, localpref 100, weight 35000, valid, external, best

ASR1004-1#show ip route .... B 10.4.160.0/24 [20/0] via 10.4.142.2, 05:00:06

East Theater West Theater Global

IP/MPLS Core Tier 1 Tier

In-Theater

IP/MPLS Core Tier 2 Tier West Region East Region

Internet Cloud

Public Voice/Video Mobility Tier 3 Tier

Metro Metro Service Private Service Public IP IP Service Service

. Understand how WAN characteristics can affect your applications. • Bandwidth, latency, loss . A modular hierarchical network infrastructure is the foundation for a solid WAN architecture. Good WAN design have many well-utilized component . Encryption is a foundation component of all WAN designs and can be deployed transparently. . Understand how to build wide area network leveraging Internet transport with SD-WAN. . Design a network with consistent behavior that provides predictable performance. . More is not always better. Keep it simple!

Abstract: Virtual Routing in the Cloud are key enablers of today’s revolutionary shift to elastic cloud applications and low-cost virtualized networking. The book covers every essential building block, present key use cases and configuration examples, illuminate design and deployment scenarios, and show how the CSR 1000V platform and APIs can enable state-of- the-art software-defined networks (SDN). Drawing on extensive early adopter experience, they illuminate crucial OS and hypervisor details, help you overcome migration challenges, and offer practical guidance for monitoring and operations.

http://bit.ly/2l8UAod BRKRST-2041 106 Cisco Webex Teams

Questions? Use Cisco Webex Teams (formerly Cisco Spark) to chat with the speaker after the session How 1 Find this session in the Cisco Events App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKRST-2041 by the speaker until June 18, 2018.

Give us your feedback to be entered into a Daily Survey Drawing. Complete your session surveys through the Cisco Live mobile app or on www.CiscoLive.com/us.

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at www.CiscoLive.com/Online.

#CLUS BRKRST-2041 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 110 Continue Demos in Walk-in Meet the Related your the Cisco self-paced engineer sessions education campus labs 1:1 meetings

#CLUS #CLUS