Implementing VXLAN In a Data Center Lilian Quan, Principle Engineer, INSBU Erum Frahim, Technical Leader, Services Kevin Cook, Solution Architecture
LTRDCT-2223 Agenda
. VxLAN Overview . Flood-&-Learn VXLAN . VXLAN with MP-BGP EVPN Control Plane . VXLAN Design Options . MP-BGP EVPN VXLAN Configuration . Lab Introduction Agenda
. VxLAN Overview . Flood-&-Learn VXLAN . VXLAN with MP-BGP EVPN Control Plane . VXLAN Design Options . MP-BGP EVPN VXLAN Configuration . Lab Introduction Recap – What is VXLAN ?
Tunnel
Ethernet Frames Endpoints
NETWORK OVERLAY Host Host 1 4 IP Addr IP Network IP Addr Host 1.1.1.1 2.2.2.2 Host 2 Switch Switch 5
Host Host
3 IP/UDP Packets 6 PLANE
DATA Outer Outer Outer Outer Outer Outer VXLAN Inner Inner Optiona Original CRC MAC MAC MAC MAC l Inner Ethernet 802.1Q IP DA IP SA UDP ID CRC DA SA (24 bits) DA SA 802.1Q Payload
VXLAN Encapsulation Original Ethernet Frame
• VXLAN uses MAC in UDP encapsulation (UDP destination port 4789)
5 Why VXLAN? VXLAN provides a Network with Segmentation, IP Mobility, and Scale
• “Standards” based Overlay
• Leverages Layer-3 ECMP – all links forwarding
• Increased Name-Space to 16M identifier
• Segmentation and Multi-Tenancy
• Integration of Physical and Virtual
• It’s SDN
6 VXLAN VTEP
VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point). Each VTEP has two interfaces, one is to provide bridging function for local hosts, the other has an IP identification in the core network for VXLAN encapsulation/decapsulation.
Underlay Network (IP Routing)
VTEP VTEP IP Interface IP Interface
Local LAN Segment Local LAN Segment
Host Host Host Host
7 VXLAN Underlay Network – IP Routing
• IP routed Network
• Flexible topologies IP Transport Network
• Recommend a network with redundant paths using ECMP for load sharing
• Support any routing protocols --- OSFP, EIGRP, IS-IS, BGP, etc. • All proven best practices for IP routing network apply
8 VXLAN Underlay Network – Typical DC Topologies
Fabric Design 3-Tier Design DC Spine DC Core
DC Aggregation
DC Access DC Leaf DC Interconnect Collapsed Core/Aggregation 2-Tier Design DC-1 DC-2
DC Core/ Aggregation
DC Access WAN
9 VXLAN BUM Traffic Handling
• BUM Traffic --- Multi-destination traffic • Broadcast • Unknown Layer-2 Unicast
• Multicast VTEP-3
• BUM Traffic transport mechanisms Multicast or / Unicast Replication
• Multicast replication in the underlay network IP Network • Requests the underlay network to run IP multicast • Ingress unicast replication VTEP-1 VTEP-2 • One unicast replica per remote VTEP • Increase traffic load throughout the network
10 Agenda
. VxLAN Overview . Flood-&-Learn VXLAN . VXLAN with MP-BGP EVPN Control Plane . VXLAN Design Options . MP-BGP EVPN VXLAN Configuration . VxLAN Capability on Nexus 9000 Series Switches . Lab Introduction
Flood-&-Learn VxLAN End System End . No VXLAN control plane System End
. Data driven flood-&-learn
VTEP
IP 3 VTEP
-
3
- 3
Multicast/Unicast Replication VTEP-1 VTEP-2 IP Network VTEP 1 VTEP 2 End System A End System B IP-1 IP-2 MAC-A MAC-B IP-A IP-B
12 Challenges with Flood-&-Learn VXLAN Deployments Scale, Mobility and Security Limitations
VXLAN Overlay
VTEP VTEP VTEP VTEP VTEP VTEP VTEP
LIMITED SCALE LIMITED WORKLOAD MOBILITY Security Risk • Flood and learn (BUM)- • Centralized Gateways – Traffic • No authentication for VXLAN Inefficient Bandwidth Utilization Hair-pining devices (VTEPs) • Resource Intensive – Large • Sub-Optimal Traffic Flow MAC Tables
Barrier for Scaling out Large Data Centers and Cloud Deployments13 Agenda
. VxLAN Overview . Flood-&-Learn VXLAN . VXLAN with MP-BGP EVPN Control Plane . VXLAN Design Options . MP-BGP EVPN VXLAN Configuration . VxLAN Capability on Nexus 9000 Series Switches . Lab Introduction What is VXLAN/EVPN? • Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)
• Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
• Forwarding decision based on Control-Plane (minimizes flooding)
• Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
Control- EVPN MP-BGP - RFC 7432 Plane (draft-ietf-l2vpn-evpn)
Provider Backbone Bridges Data- Multi-Protocol Label Switching (MPLS) Network Virtualization Overlay (NVO) (PBB) Plane draft-ietf-l2vpn-evpn draft-sd-l2vpn-evpn-overlay draft-ietf-l2vpn-pbb-evpn
EVPN over NVO Tunnels (VXLAN, NVGRE, MPLSoE) for Data Center Fabric encapsulations Provides Layer-2 and Layer-3 Overlays over simple IP Networks
15 Cisco Supported VXLAN IETF Drafts
ID Title Ver Category Comments
RFC 7348 Virtual eXtensible Local Area Data Plane Flood and Learn based implemented Network RFC 7432 BGP MPLS based Ethernet VPNs EVPN Control Plane Control Plane is based on this RFC draft-ietf-bess-evpn-overlay A Network Virtualization Overlay 0 EVPN Control Plane Implemented Solution using EVPN draft-ietf-bess-evpn-inter- Integrated Routing and Bridging in 0 EVPN Control Plane Implemented subnet-forwarding EVPN draft-rabadan-l2vpn-evpn- IP Prefix Advertisement in EVPN 2 EVPN Control Plane Implemented evpn-prefix-advertisement
Draft-tissa-nvo3-oam-fm NVO3 Fault Management 1 Mgmt Plane Working in progress
16 Functions of VXLAN/EVPN
Host/Network Advertise host/network reachability Reachability information through control protocol (MP- Advertisement BGP) VTEP Security & Authenticate VTEPs through BGP peer Authentication authentication Distributed Anycast Gateway Seamless and Optimal vm-mobility
Early ARP termination ARP Suppression Localize ARP learning process Minimize network flooding Dynamic Ingress Replication Unicast Alternative to Multicast underlay
17 EVPN Primer --- MP-BGP Review
Virtual Routing and Forwarding (VRF) Layer-3 segmentation for tenants’ routing space
BGP advertisement: VPN-IPv4 Addr = RD:16.1/16 Route Distinguisher (RD): BGP Next-Hop = PE1 8-byte field, VRF parameters; unique value to make Route Target = 100:1 eBGP: Label=42 eBGP: VPN IP routes unique: RD + VPN IP prefix 16.1/16 16.1/16 IP Subnet IP Subnet
PE1 P P PE2 Selective distribute VPN routes: CE1 Blue VPN CE2 Route Target (RT): 8-byte field, VRF parameter, unique value to define the import/export rules for ipVRFvrfparameters:blue-vpn VPNv4 routes NameRD 1:100 = blue-vpn RDroute = 1:100-target export 1:100 Importroute- targetRoute import-Target 1:100 = 100:1 Export Route-Target = 100:1 VPN Address-Family: Distribute the MP-BGP VPN routes
18 MP-BGP for VXLAN EVPN Control Plane EVPN Control Plane – Reachability Distribution • EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update Spine • Host-MAC • Host-IP • Internal IP Subnet • External Prefixes
VTEP VTEP VTEP VTEP Leaf
. Use MP-BGP with EVPN Address Family on leaf nodes to distribute internal host MAC/IP addresses, subnet routes and external reachability information . MP-BGP enhancements to carry up to 100s of thousands of routes with reduced convergence time
19 EVPN Control Plane -- Host Advertisement
Install host info to RIB/FIB: Install host info to RIB/FIB: H-MAC-1 MAC table BGP Update: H-MAC-1 MAC table BGP Update: H-IP-1 VRF IP host table 4 H-MAC-1 H-MAC-1 4 H-IP-1 VRF IP host table H-IP-1 Host VNI VTEP H-IP-1 3 Route IP VTEP-1 MAC Host VNI VTEP VTEP-1 3 Reflector VNI-1 IP MAC-1 H-IP-1 VNII-1 VTEP-1 VNI-1 H-MAC-1 H-IP-1 VNII-1 VTEP-1 2 VTEP-2 VTEP-3 BGP Update: H-MAC-1 H-IP-1 VTEP-1 VNI-1
1 VTEP-1
MAC Host VNI VTEP IP
H-MAC-1 H-IP-1 VNII-1 VTEP-1 Local learning of host info: H-MAC-1 H-IP-1 H-MAC-1 (MAC table) VLAN-1 /VNI-1 H-IP-1 (VRF IP host table ) 20 VXLAN BGP Control Plane EVPN Control Plane --- Host Movement
NLRI: Spine • Host H-MAC-1, H-IP-1 • NVE VTEP-1 • VNI 5000
Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf • Cost • Sequence number :0 Host 1 H-MAC-1 H-IP-1 VLAN 10 VXLAN 5000 1. Host 1 attaches to VTEP-1
2. VTEP-1 detects Host1 and advertises H1 with seq #0
3. Other VTEPs learn about the host route of Host 1 MAC IP VNI Next-Hop Encap Seq# H-MAC-1 H-IP-1 5000 VTEP-1 VXLAN 0
21 VXLAN BGP Control Plane EVPN Control Plane --- Host Movement
NLRI: Spine • Host H-MAC-1, H-IP-1 • NVE VTEP-3 • VNI 5000
Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf • Cost • Sequence number: 1 Host 1 H-MAC-1 H-IP-1 VLAN 10 VXLAN 5000 1. Host 1 moves to VTEP-3 from VTEP-1
2. VTEP-3 detects Host 1, sends MP-BGP update for Host 1 with its own VTEP address and a new seq #1
3. Other VTEPs learn about the new route of Host 1 MAC IP VNI Next-Hop Encap Seq# H-MAC-1 H-IP-1 5000 VTEP-3 VXLAN 1
22 VXLAN Routing VXLAN EVPN has two slightly different integrated Route/Bridge (IRB) semantics
Routing ? IP Transport Network SVI A SVI B VTEP-1 VTEP-2 VTEP-3 VTEP-4
Host 1 Host 2 H-MAC-1 H-MAC-2 H-IP-1 H-IP-2 VNI-A VNI-B
Asymmetric Symmetric • Routing on the ingress VTEP and bridging • Routing on both the ingress and the egress on the egress VTEP VTEPs • Requires each VTEP to have all VNIs – • A VTEP only needs to have VNIs in which can result in forwarding table resource they have local hosts. Optimal utilization of wastage. forwarding table resources
• Cisco follows Symmetric IRB 23 Symmetric Integrated Routing & Bridging (IRB) (1)
• Routing on both ingress and egress VTEPs • Layer-3 VNI Layer-3 VNI (VRF VNI) • Tenant VPN indicator • One per tenant VRF • VTEP Router MAC Layer-2 VNI Layer-2 VNI (Network VNI) (Network VNI) • Ingress VTEP routes packets onto the Layer-3 VNI VTEP VTEP VTEP VTEP • Egress VTEP routes packets to the destination Layer-2 VNI
24 Symmetric Integrated Routing & Bridging (IRB) (2)
Egress VTEP routes packets Ingress VTEP S-IP: VTEP-1 from L3 VNI to the routes packets D-IP: VTEP-4 destination VNI- VNI: L3 VNI from source VNI-A 1 2 B/VLAN. to L3 VNI. D-MAC S-MAC: Router-MAC-1 in the inner header D-MAC: Router-MAC-4 is the egress S-IP: H-IP-1 D-IP: H-IP-2 VTEP router MAC S-MAC: MAC-VTEP4 local port D-MAC: H-MAC-2 VNI L3 L3 VNI S-IP: H-IP-1 A VNI VNI B D-IP: H-IP-2
VTEP-1 VTEP-4 VTEP-2 VTEP-3 Router MAC-1 Router MAC-4
S-MAC: H-MAC-1 D-MAC: GW-MAC Host 1 Host 2 S-IP: H-IP-1 H-MAC-1 H-MAC-2 D-IP: H-IP-2 H-IP-1 H-IP-2 VNI-A VNI-B
25 EVPN Control Plane --- ARP Suppression
Minimize flood-&-learn behavior for host learning
MAC IP VNI Next-Hop Encap Seq Spine
H-MAC-2 H-IP-2 5000 VTEP-3 VXLAN 0
2 VTEP VTEP VTEP VTEP VTEP-1 receives and intercepts the ARP 1 2 3 4 Leaf Request. Checks in its own host table.
• If it has an match for H-IP-2, it’ll send ARP Host 1 Host 2 response on behave of Host-2 H-MAC1 H-MAC-2 H-IP 1 H-IP-2 • If it doesn’t have a match for H-IP-2, it’ll VLAN 10 VLAN 10 forward the ARP request to remote VTEPs VXLAN 5000 VXLAN 5000 via multicast encap or head-end replication
1 Host-1 sends ARP Request for H-IP-2
26 EVPN Control Plane -- Head-end Replication Head-end Replication (aka. Ingress replication): Eliminate the need for underlay multicast to transport overlay BUM traffic
Multicast-Free Spine Underlay
VTEP VTEP VTEP VTEP 1 2 2 3 4 Leaf VTEP-1 receives the overlay BUM traffic, encapsulates the packets into unicast VXLAN packets, sends one copy to each remote VTEP peer in the same VXLAN VNI
1 Host-1 sends BUM traffic into the VXLAN VNI
27 Distributed Anycast Gateway in MP-BGP EVPN
# VLAN to VNI mapping vlan 200 vn-segment 5200
# Anycast Gateway MAC, identically configured on all VTEPs fabric forwarding anycast-gateway-mac 0002.0002.0002
# Distributed IP Anycast Gateway (SVI) The same anycast gateway virtual IP # Gateway IP address needs to be identically configured on all address and MAC address are VTEPs configured on all VTEPs in the VNI. interface vlan 200 no shutdown vrf member Tenant-A ip address 20.0.0.1/24 fabric forwarding mode anycast-gateway
SVI SVI SVI SVI GW IP GW IP GW IP GW IP GW MAC GW MAC GW MAC GW MAC VTEP VTEP VTEP VTEP
Host 1 Host 2 Host 3 Host 4 MAC1 MAC2 MAC3 MAC4 IP 1 IP 2 IP 3 IP 4 VLAN A VLAN A VLAN A VLAN A VXLAN A VXLAN A VXLAN A VXLAN A 28 Next-Gen VXLAN Fabric with BGP-EVPN Control Plane Multi-Tenancy, Seamless Host Mobility and Security at Cloud Scale
Route Route Reflector Reflector
BGP-EVPN VXLAN Overlay
BGP Peers VTEP VTEP VTEP VTEP VTEP VTEP VTEP
INTEROPERABLE INCREASED SCALE OPTIMIZED OPERATIONAL SECURITY • Standards Based • Eliminates MOBILITY FLEXIBILITY • VTEP peer • BGP-EVPN Flooding • Distributed • Layer 2 or Layer 3 authentication via Anycast Gwy BGP • VXLAN • Conversational • Controller Choice Learning • Integrated • White-list for • Policy-Based Routing /Bridging unauthenticated Updates • vPC & ECMP VTEPs
Breaking the Traditional VXLAN Scale Barriers 29 Agenda
. VxLAN Overview . Flood-&-Learn VXLAN . VXLAN with MP-BGP EVPN Control Plane . VXLAN Design Options . MP-BGP EVPN VXLAN Configuration . VxLAN Capability on Nexus 9000 Series Switches . Lab Introduction VXLAN Inter-PoD Extension L3 Core L2 Link L3 Link
VTEP VTEP
VXLAN Overlay (VLAN Extension)
Pod 1 Pod 2 IP GW IP GW
Layer-2 VLAN Domain Layer-2 VLAN Domain
31 VXLAN Fabric Design with MP-iBGP EVPN
RR RR Spine MP-iBGP Sessions
VXLAN Overlay MP-iBGP EVPN
VTEP VTEP VTEP VTEP VTEP VTEP Leaf
• VTEP Functions are on leaf layer • Spine nodes are iBGP route reflector • Spine nodes don’t need to be VTEP
32 VXLAN Fabric Design with MP-eBGP EVPN
AS 65000 Spine
MP-eBGP Sessions
VTEP VTEP VTEP VTEP VTEP VTEP Leaf
AS 65001 AS 65002 AS 65003 AS 65004 AS 65005 AS 65006
• VTEP Functions are on leaf layer • Spine nodes are MP-eBGP Peers to VTEP leafs • Spine nodes don’t need to be VTEP • VTEP leafs can be in the same or different BGP AS’s 33 EVPN VXLAN Fabric External Routing
VXLAN Overlay Spine EVPN VRF/VRFs Space
VXLAN Overlay EVPN MP-BGP Border Leaf
VTEP VTEP VTEP VTEP VTEP VTEP Leaf
Routing Protocol of Choice Global Default VRF Or User Space VRFs
IP Routing
34 EVPN VXLAN External Routing with BGP Sample Configuration Router bgp 100 vrf evpn-tenant-1 address-family ipv4 unicast network 20.0.0.0/24 neighbor 30.10.1.2 remote-as 200 RR RR address-family ipv4 unicast Spine prefix-list outbound-no-hosts out VXLAN Overlay Border Leaf interface Ethernet2/9.10 EVPN VRF Instance Space mtu 9216 VTE VTE VTE VTE VTEP P P P P encapsulation dot1q 10 vrf member evpn-tenant-1 ip address 30.10.1.1/30
interface Ethernet1/50.10 mtu 9216 encapsulation dot1q 10 ip address 30.10.1.2/30
IP Routing in the Default router bgp 200 VRF Instance address-family ipv4 unicast network 100.0.0.0/24 network 100.0.1.0/24 neighbor 30.10.1.1 remote-as 100 address-family ipv4 unicast 35 DCI with VXLAN EVPN (Option A)
VXLAN Overlay
RR RR RR EVPNRR VRF/VRFs Space
DC #1 DC #2
EVPN iBGP Border Leaf Border Leaf EVPN iBGP
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP
EVPN Domain #2 EVPN Domain #1
VTEP VTEP VLAN hand-off VLAN hand-off Flood-&-Learn Flood-&-Learn Inter-DC EVPN OVT/VPLS
Inter-DC EVPN Domain
36 DCI with VXLAN EVPN (Option B)
VXLAN Overlay
RR RR RR EVPNRR VRF/VRFs Space
DC #1 DC #2 DCI Border Leaf EVPN iBGP DCI Border Leaf EVPN iBGP
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP
Inter-DC EVPN eBGP (multi-hop)
Global Default VRF Or User Space VRFs
One EVPN Administrative Domain Stretched Across Two Data Centers
37 Agenda
. VxLAN Overview . Flood-&-Learn VXLAN . VXLAN with MP-BGP EVPN Control Plane . VXLAN Design Options . MP-BGP EVPN VXLAN Configuration . Lab Introduction
38 Building your VTEP (VXLAN Tunnel End-Point)
# Features & Globals Enables VTEP (only required on Leaf or Border) feature bgp feature nv overlay nv overlay evpn Enables EVPN Control-Plane in BGP # Spine (S1)
# Leaf (V1) Configure the VTEP interface interface nve1 source-interface loopback0 RR RR RR RR host-reachability protocol bgp Use a Loopback for Source Interface
iBGP 2 1 V Enable BGP for Host reachabilityV
V3 *Simplified BGP configuration; would have 4 BGP peers (RR) IGP not shown 39 Building your EVPN MP-BGP Control-Plane # Features & Globals feature bgp feature nv overlay nv overlay evpn Enables EVPN Control-Plane in BGP # Spine (S1) router bgp 65500 router-id 10.10.10.1 address-family ipv4 unicast address-family l2vpn evpn Activate L2VPN EVPN under each BGP neighborRR RR RR RR neighbor 10.10.10.0/24 remote-as 65500 update-source loopback0 address-family l2vpn evpn iBGP send-community both 2 V1 V route-reflector-client
# Leaf (V1) router bgp 65500 Send Extended BGP Community router-id 10.10.10.10 to distribute EVPN route attributes address-family ipv4 unicast V3 neighbor 10.10.10.1 remote-as 65500 update-source loopback0 address-family l2vpn evpn send-community both * 40 Extend your VLAN to VXLAN
• VLAN to VNI configuration on a per-Switch based • VLAN becomes “Switch Local Identifier” # Features feature vn-segment-vlan-based • VNI becomes “Network Global Identifier” # VLAN to VNI mapping (MT-Lite) • 4k VLAN limitation per-Switch does still apply Vlan 10 VLAN to Layer-2 VNI mapping • 4k Network limitation has been removed vn-segment 5010 • VLAN can be port-significant. The same vlan on # Activate Layer-2 VNI for EVPN evpn different ports can be mapping to different VNIs. vni 5010 l2 Enables EVPN Control- rd auto Plane for Layer-2 route-target import auto Services route-target export auto Alternative is to use VLAN # Activate Layer-2 VNI on VTEP “ingress-replication interface nve1 protocol bgp” ethernet VLAN VNI vxlan source-interface loopback0 host-reachability protocol bgp
Multi-Tenancy Lite (MT-Lite) member vni 5010 mcast-group 239.239.239.100 suppress-arp Enables Layer-2 VNI on VTEP and suppress41 ARP Distributed Anycast Gateway for Extended VLANs
• All VTEPs in a VXLAN are the distributed anycast gateway for its IP subnet. • All VTEPs in a VXLAN need to be configured with an identical anycast gateway virtual MAC address # VLAN to VNI mapping vlan 200 • All VTEPs in a VXLAN need to be configured with vn-segment 5200
an identical anycast gateway virtual IP address # Anycast Gateway MAC, identically configured on all VTEPs fabric forwarding anycast-gateway-mac 0002.0002.0002
# Distributed IP Anycast Gateway (SVI) One gateway virtual MAC per VTEP # Gateway IP address needs to be identically configured on all VTEPs interface vlan 200 no shutdown vrf member Tenant-A ip address 20.0.0.1/24 One gateway virtual IP per VLAN/VXLAN fabric forwarding mode anycast-gateway
42 Routing in VXLAN – Define the Resources
Configuration Example for VRF-A
vxlan # Define VLAN for VRF routing instance Vlan 999 vn-segment 9999 VLAN to Layer-3 VNI mapping
# Define SVI for VRF routing instance interface Vlan999 VNI50000 VLAN to Layer-3 VNI mapping no shutdown mtu 9216 - ip forward required for prefix- vrf member VRF-A based routing ip forward VRF-A # VRF configuration for “customer” VRF vrf context VRF-A vni 9999 VRF context definition rd auto ethernet ethernet - VNI address-family ipv4 unicast route-target both auto - Route-Distinguisher route-target both auto evpn - Route-Targets - IPv4 and/or IPv6
43 Routing in VXLAN – Configure the Routing
Enables Layer-3 VNI on VTEP Configuration Example for VRF-A
and associatevxlan it to VRF # Activate Layer-3 VNI on VTEP interface nve1 source-interface loopback0 host-reachability protocol bgp member vni 5010 mcast-group 239.239.239.100 VNI50000 suppress-arp member vni 9999 associate-vrf
# Route-Map for Redistribute Subnet VRF-A route-map REDIST-SUBNET permit 10 match tag 12345
# Control-Plane configuration for VRF (Tenant) router bgp 65500 ethernet ethernet … vrf VRF-A address-family ipv4 unicast VRF/Tenant definition advertise l2vpn evpn within Overlay Control-Plane redistribute direct route-map REDIST-SUBNET maximum-paths ibgp 2 44 VXLAN Hardware Gateway Redundancy (vPC)
• Redundant connectivity for classic Ethernet hosts
• Extend the IP Interface (Loopback) configuration for the vPC VTEP V5 • Secondary IP address (anycast) is used as the anycast VTEP address V4 • Both vPC VTEP switches need to have the identical secondary IP address configured under the loopback interface Host D VNI 30000
45 VXLAN Hardware Gateway Redundancy (vPC) vPC VTEP Configuration Example
# VLAN to VNI mapping (MT-Lite) vlan 55 interface loopback0 vn-segment 30000 ip address 10.10.10.5/32 # VTEP IP Interface; Source/Destination for all ip address 10.10.10.99/32 secondary VXLAN Encapsulated Traffic. . Primary IP address is used for Orphan Hosts . Secondary IP is for vPC Hosts (same IP on both vPC Peers) interface loopback0 V5 ip address 10.10.10.5/32 Add Secondary IP to VTEP Loopback. ip address 10.10.10.99/32 secondary V4 VXLAN automatically picks up # VTEP configuration using Loopback as source. the secondary IP address as interface nve1 the VTEP address source-interface loopback0 host-reachability protocol bgp interface loopback0 member vni 5010 ip address 10.10.10.4/32 mcast-group 239.239.239.100 ip address 10.10.10.99/32 secondary suppress-arp Host D member vni 9999 associate-vrf VNI 30000
46 VXLAN Hardware Gateway Redundancy (vPC) vPC VTEP Configuration Example # VPC Domain Configuration vpc domain 99 peer-gateway needs to be interface loopback0 peer-switch enabled so that vPC VTEP ip address 10.10.10.5/32 peer-keepalive destination V4-mgmt source v5-mgmt ip address 10.10.10.99/32 secondary peer-gateway switches can forward traffic ip arp synchronize for each other’s router MAC address # VPC Peer-Link interface port-channelXX 5 switchport mode trunk V vpc peer-link V4 # VPC Domain Routing Adjacency Routed Interface (SVI) for routing interface Vlan3999 no shutdown adjacency across VPC Peer-Link ip address 10.254.254.1/30 ip router ospf 1 area 0.0.0.0 interface loopback0 ip ospf network point-to-point ip address 10.10.10.4/32 ip address 10.10.10.99/32 secondary ip pim sparse-mode Host D VNI 30000
47 EVPN Configuration for Stretched Fabric
VXLAN Overlay
RR RR RR EVPNRR VRF/VRFs Space
DC #1 DC #2 DCI Border Leaf EVPN iBGP DCI Border Leaf EVPN iBGP
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP
Inter-DC EVPN eBGP (multi-hop)
Global Default VRF Or User Space VRFs
One EVPN Administrative Domain Stretched Across Two Data Centers
48 EVPN Configuration for Stretched Fabric
VXLAN Overlay
RR RR RR EVPNRR VRF/VRFs Space
DC #1 DC #2 EVPNiBGP iBGP EVPNiBGP iBGP DCI Border Leaf DCI Border Leaf
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP Multi-hop Inter-DC EVPN eBGPeBGP(multi-hop)
Global Default VRF Or User Space VRFs
One EVPN Administrative Domain Stretched Across Two Data Centers
49 eBGP EVPN Configuration (1) Next-hop Unchange
• BGP next-hop is used as the tunnel tail end address. It shall be the advertising VTEP’s address. eBGP configuration on a spine switch • Ensure the next-hop in the BGP route route-map permit-all permit 10 isn’t changed during the route distribution route-map nh-unchange permit 10 set ip next-hop unchanged • eBGP changes next-hop to by default. router bgp 65000 Need to change the policy to next-hop router-id 10.1.1.1 unchanged address-family ipv4 unicast address-family l2vpn evpn nexthop route-map nh-unchange retain route-target all neighbor 192.167.11.2 remote-as 65001 Set next-hop policy not to change address-family ipv4 unicast the next-hop attribute address-family l2vpn evpn send-community extended route-map permit-all out
50 eBGP EVPN Configuration(2) Manually configure import/export route-target
• With eBPG, VTEPs will have different route-targets if using auto RT generation vrf context evpn-tenant-1 • Need to manually configure RTs on vni 9999 rd auto eBGP peers so that they have the same address-family ipv4 unicast RTs route-target import 100:9999 route-target import 100:9999 evpn route-target export 100:9999 route-target export 100:9999evpn Manually configure route-target for evpn vni 5010 l2 VRF rd auto route-target import 100:5010 route-target export 100:5010
Manually configure route-target for L2 VNI under EVPN
51 Agenda
. VxLAN Overview . Flood-&-Learn VXLAN . VXLAN with MP-BGP EVPN Control Plane . VXLAN Design Options . MP-BGP EVPN VXLAN Configuration . Lab Introduction Lab Overview Module Details
• Module 1 – Network Based Overlay DC1: In this module, students will configure a network based overlay with Nexus 9000 switches and use them as a VTEP. Students will also learn how to extend the Anycast gateway using BGP EVPN.
• Module 2 – FW-Security Zone : In this module, students will create the secure zone via placing the FW ( in transparent mode) between the Fabric and External Connectivity.
• Module 3- Stretch Fabric: In this module, students will be able to extend the VLANs from the first DC to the second DC and able to stretch the fabric.
53 Lab Topology
54 Network Overlay DC1 Using BGP EVPN for Control Plane
• Using OSPF for underlay
• IBGP-for overlay-VXLAN EVPN control plane- Route Reflectors are on Spines S11 and S12
• Ingress Replication for the BUM Traffic
• Each student has their own VRF that will be representative of multi- tenancy. Security Module Creating the Security Zone via VXLAN
• The Hosts in the Secured Zone are in VXLAN X53.
• Transparent FW is attached to Edge E11-( No redundancy in this lab).
56 Firewall Usecase Flow
57 DC2- Stretch Fabric Using EBGP EVPN for Underlay/Overlay
• Using EBGP for Underlay/Overlay
• Ingress Replication for the BUM Traffic.
• One of the VLAN X51 will be stretch from DC1 to DC2.
• Each student has their own VRF that will be representative of multi-tenancy.
58 Manual Overview • Manual available at https://cisco.box.com/LTRDCT-2223-Manual
• Password will be provided by proctor
• Any time you see text highlighted in cyan, change the numbers to your pod number – Pod2 is always used as a sample
• Change XX to your two digit pod number (Pod1 = 01, Pod10 = 10)
• Change X to your single/two digit pod number (Pod1 = 1, Pod10 = 10)
• Only type commands that are shown in a box. Commands shown under “Configuration Sample” are not meant to be typed into the devices. Complete Your Online Session Evaluation
• Please complete your online session evaluations after each session. Complete 4 session evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt.
• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Thank you