Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
First Published: 2019-07-20 Last Modified: 2021-09-22
Americas Headquarters
Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS REFERENCED IN THIS DOCUMENTATION ARE SUBJECT TO CHANGE WITHOUT NOTICE. EXCEPT AS MAY OTHERWISE BE AGREED BY CISCO IN WRITING, ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS DOCUMENTATION ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
The Cisco End User License Agreement and any supplemental license terms govern your use of any Cisco software, including this product documentation, and are located at: http://www.cisco.com/go/softwareterms.Cisco product warranty information is available at http://www.cisco.com/go/warranty. US Federal Communications Commission Notices are found
here http://www.cisco.com/c/en/us/products/us-fcc-notice.html.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any products and features described herein as in development or available at a future date remain in varying stages of development and will be offered on a when-and if-available basis. Any such product or feature roadmaps are subject to change at the sole discretion of Cisco and Cisco will have no liability for delay in the delivery or failure to deliver any products or feature roadmap items that may be set forth in this document.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com go trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)
©
2019–2021 Cisco Systems, Inc. All rights reserved.
C O N T E N T S
P R E F A C E
Preface xvii
Audience xvii
Document Conventions xvii Related Documentation for Cisco Nexus 9000 Series Switches xviii Documentation Feedback xviii Communications, Services, and Additional Information xviii
C H A P T E R C H A P T E R
12
New and Changed Information
1
New and Changed Information
1
Overview
7
Licensing Requirements
7
VXLAN Overview Cisco Nexus 9000 as Hardware-Based VXLAN Gateway VXLAN Encapsulation and Packet Format VXLAN Tunnel VXLAN Tunnel Endpoint
7
8
8
9
9
Underlay Network Overlay Network
9
9
Distributed Anycast Gateway
9
Control Plane 10
- C H A P T E R
- 3
Configuring VXLAN 13
Guidelines and Limitations for VXLAN 13 Considerations for VXLAN Deployment 19 vPC Considerations for VXLAN Deployment 21
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
iii
Contents
Network Considerations for VXLAN Deployments 25 Considerations for the Transport Network 26 Considerations for Tunneling VXLAN 27 Configuring VXLAN 29
Enabling VXLANs 29 Mapping VLAN to VXLAN VNI 29 Creating and Configuring an NVE Interface and Associate VNIs 29 Configuring a VXLAN VTEP in vPC 30 Configuring Static MAC for VXLAN VTEP 32 Disabling VXLANs 33 Configuring BGP EVPN Ingress Replication 34 Configuring Static Ingress Replication 34
VXLAN and IP-in-IP Tunneling 35 Configuring VXLAN Static Tunnels 38
About VXLAN Static Tunnels 38 Guidelines and Limitations for VXLAN Static Tunnels 38 Enabling VXLAN Static Tunnels 39 Configuring VRF Overlay for Static Tunnels 40 Configuring a VRF for VXLAN Routing 40 Configuring the L3 VNI for Static Tunnels 41 Configuring the Tunnel Profile 42 Verifying VXLAN Static Tunnels 43 Example Configurations for VXLAN Static Tunnels 43
- C H A P T E R
- 4
Configuring VXLAN BGP EVPN 45
Information About VXLAN BGP EVPN 45
About RD Auto 45
About Route-Target Auto 46
Guidelines and Limitations for VXLAN BGP EVPN 47 Information About VXLAN EVPN with Downstream VNI 50
Asymmetric VNIs 50 Shared Services VRFs 51 Multi-Site with Asymmetric VNIs 51
Guidelines and Limitations for VXLAN EVPN with Downstream VNI 52
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
iv
Contents
Configuring VXLAN BGP EVPN 54
Enabling VXLAN 54 Configuring VLAN and VXLAN VNI 54 Configuring VRF for VXLAN Routing 55 Configuring SVI for Core-facing VXLAN Routing 56 Configuring SVI for Host-Facing VXLAN Routing 57 Configuring the NVE Interface and VNIs Using Multicast 57 Configuring VXLAN EVPN Ingress Replication 58 Configuring BGP on the VTEP 59 Configuring iBGP for EVPN on the Spine 61 Configuring eBGP for EVPN on the Spine 62 Suppressing ARP 63 Disabling VXLANs 63 Duplicate Detection for IP and MAC Addresses 64 Verifying the VXLAN BGP EVPN Configuration 65 Verifying the VXLAN EVPN with Downstream VNI Configuration 66 Example of VXLAN BGP EVPN (IBGP) 69 Example of VXLAN BGP EVPN (EBGP) 80 Example Show Commands 93
- C H A P T E R
- 5
Configuring VXLAN with IPv6 in the Underlay (VXLANv6) 95
Information About Configuring VXLAN with IPv6 in the Underlay (VXLANv6) 95 Guidelines and Limitations for VXLAN with IPv6 in the Underlay (VXLANv6) 96 Information About vPC and VXLAN with IPv6 in the Underlay (VXLANv6) 98 Information About vPC Peer Keepalive and VXLAN with IPv6 in the Underlay (VXLANv6) 98 Configuring the VTEP IP Address 99 Configuring vPC for VXLAN with IPv6 in the Underlay (VXLANv6) 100 Example Configurations for VXLAN with IPv6 in the Underlay (VXLANv6) 101 Verifying VXLAN with IPv6 in the Underlay (VXLANv6) 103
- C H A P T E R
- 6
Configuring External VRF Connectivity and Route Leaking 113
Configuring External VRF Connectivity 113
About External Layer-3 Connectivity for VXLAN BGP EVPN Fabrics 113
VXLAN BGP EVPN - VRF-lite brief 113
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
v
Contents
Guidelines and Limitations for External VRF Connectivity and Route Leaking 114 Configuring VXLAN BGP EVPN with eBGP for VRF-lite 114 VXLAN BGP EVPN - Default-Route, Route Filtering on External Connectivity 118 Configuring VXLAN BGP EVPN with OSPF for VRF-lite 124
Configuring Route Leaking 127
About Centralized VRF Route-Leaking for VXLAN BGP EVPN Fabrics 127 Guidelines and Limitations for Centralized VRF Route-Leaking 127 Centralized VRF Route-Leaking Brief - Specific Prefixes Between Custom VRF 127 Configuring Centralized VRF Route-Leaking - Specific Prefixes between Custom VRF 128
Configuring VRF Context on the Routing-Block VTEP 128 Configuring the BGP VRF instance on the Routing-Block 129 Example - Configuration Centralized VRF Route-Leaking - Specific Prefixes Between Custom
VRF 130
Centralized VRF Route-Leaking Brief - Shared Internet with Custom VRF 131
Configuring Centralized VRF Route-Leaking - Shared Internet with Custom VRF 132
Configuring Internet VRF on Border Node 132 Configuring Shared Internet BGP Instance on the Border Node 133 Configuring Custom VRF on Border Node 133 Configuring Custom VRF Context on the Border Node - 1 134 Configuring Custom VRF Instance in BGP on the Border Node 135 Example - Configuration Centralized VRF Route-Leaking - Shared Internet with Custom VRF
135
Centralized VRF Route-Leaking Brief - Shared Internet with VRF Default 137
Configuring Centralized VRF Route-Leaking - Shared Internet with VRF Default 138
Configuring VRF Default on Border Node 138 Configuring BGP Instance for VRF Default on the Border Node 138 Configuring Custom VRF on Border Node 139 Configuring Filter for Permitted Prefixes from VRF Default on the Border Node 139 Configuring Custom VRF Context on the Border Node - 2 140 Configuring Custom VRF Instance in BGP on the Border Node 141 Example - Configuration Centralized VRF Route-Leaking - VRF Default with Custom VRF 141
- C H A P T E R
- 7
Configuring BGP EVPN Filtering 145
About BGP EVPN Filtering 145
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
vi
Contents
Guidelines and Limitations for BGP EVPN Filtering 146 Configuring BGP EVPN Filtering 146
Configuring the Route Map with Match and Set Clauses 146
Matching Based on EVPN Route Type 147 Matching Based on MAC Address in the NLRI 147 Matching Based on RMAC Extended Community 148 Setting the RMAC Extended Community 148 Setting the EVPN Next-Hop IP Address 149 Setting the Gateway IP Address for Route Type-5 149
Applying the Route Map at the Inbound or Outbound Level 150 BGP EVPN Filtering Configuration Examples 150 Configuring a Table Map 159
Configuring a MAC List and a Route Map that Matches the MAC List 159 Applying the Table Map 160
Table Map Configuration Example 160
Verifying BGP EVPN Filtering 163
- C H A P T E R
- 8
Configuring VXLAN OAM 165
VXLAN OAM Overview 165
Loopback (Ping) Message 166 Traceroute or Pathtrace Message 167
About VXLAN EVPN Loop Detection and Mitigation 169 Guidelines and Limitations for VXLAN NGOAM 170 Guidelines and Limitations for VXLAN EVPN Loop Detection and Mitigation 170 Configuring VXLAN OAM 171 Configuring NGOAM Profile 174 Configuring VXLAN EVPN Loop Detection and Mitigation 175 Detecting Loops and Bringing Up Ports On Demand 176 Configuration Examples for VXLAN EVPN Loop Detection and Mitigation 177
- C H A P T E R
- 9
Configuring vPC Multihoming 179
Advertising Primary IP Address 179 BorderPE Switches in a vPC Setup 180 DHCP Configuration in a vPC Setup 180
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
vii
Contents
IP Prefix Advertisement in vPC Setup 180
C H A P T E R 1 0
Configuring Multi-Site 183
About VXLAN EVPN Multi-Site 183 Dual RD Support for Multi-Site 184 Guidelines and Limitations for VXLAN EVPN Multi-Site 184 Enabling VXLAN EVPN Multi-Site 187 Configuring Dual RD Support for Multi-Site 188 Configuring VNI Dual Mode 189 Configuring Fabric/DCI Link Tracking 190 Configuring Fabric External Neighbors 191 Configuring VXLAN EVPN Multi-Site Storm Control 192 Multi-Site with vPC Support 193
About Multi-Site with vPC Support 193 Guidelines and Limitations for Multi-Site with vPC Support 193 Configuring Multi-Site with vPC Support 193 Configuring Peer Link as Transport in Case of Link Failure 197 Verifying the Multi-Site with vPC Support Configuration 199
Configuration Example for Multi-Site with Asymmetric VNIs 200 TRM with Multi-Site 201
Information About Configuring TRM with Multi-Site 202 Guidelines and Limitations for TRM with Multi-Site 204 Configuring TRM with Multi-Site 206 Verifying TRM with Multi-Site Configuration 207
C H A P T E R 1 1
Configuring Tenant Routed Multicast 209
About Tenant Routed Multicast 209 About Tenant Routed Multicast Mixed Mode 211 Guidelines and Limitations for Tenant Routed Multicast 211 Guidelines and Limitations for Layer 3 Tenant Routed Multicast 212 Guidelines and Limitations for Layer 2/Layer 3 Tenant Routed Multicast (Mixed Mode) 213 Rendezvous Point for Tenant Routed Multicast 214 Configuring a Rendezvous Point for Tenant Routed Multicast 214 Configuring a Rendezvous Point Inside the VXLAN Fabric 215
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
viii
Contents
Configuring an External Rendezvous Point 216 Configuring RP Everywhere with PIM Anycast 218
Configuring a TRM Leaf Node for RP Everywhere with PIM Anycast 219 Configuring a TRM Border Leaf Node for RP Everywhere with PIM Anycast 219 Configuring an External Router for RP Everywhere with PIM Anycast 221
Configuring RP Everywhere with MSDP Peering 223
Configuring a TRM Leaf Node for RP Everywhere with MSDP Peering 224 Configuring a TRM Border Leaf Node for RP Everywhere with MSDP Peering 225 Configuring an External Router for RP Everywhere with MSDP Peering 227
Configuring Layer 3 Tenant Routed Multicast 229 Configuring TRM on the VXLAN EVPN Spine 233 Configuring Tenant Routed Multicast in Layer 2/Layer 3 Mixed Mode 235 Configuring Layer 2 Tenant Routed Multicast 240 Configuring TRM with vPC Support 241 Configuring TRM with vPC Support (Cisco Nexus 9504-R and 9508-R) 244
C H A P T E R 1 2
Configuring Cross Connect 249
About VXLAN Cross Connect 249 Guidelines and Limitations for VXLAN Cross Connect 250 Configuring VXLAN Cross Connect 251 Verifying VXLAN Cross Connect Configuration 253 Configuring NGOAM for VXLAN Cross Connect 254 Verifying NGOAM for VXLAN Cross Connect 254 NGOAM Authentication 255 Guidelines and Limitations for Q-in-VNI 256 Configuring Q-in-VNI 258 Configuring Selective Q-in-VNI 259 Configuring Q-in-VNI with LACP Tunneling 262 Selective Q-in-VNI with Multiple Provider VLANs 263
About Selective Q-in-VNI with Multiple Provider VLANs 263 Guidelines and Limitations for Selective Q-in-VNI with Multiple Provider VLANs 264 Configuring Selective Q-in-VNI with Multiple Provider VLANs 264
Configuring QinQ-QinVNI 267
Overview for QinQ-QinVNI 267
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 9.3(x)
ix
Contents
Guidelines and Limitations for QinQ-QinVNI 267 Configuring QinQ-QinVNI 268
Removing a VNI 269
C H A P T E R 1 3
Configuring Port VLAN Mapping 271
About Translating Incoming VLANs 271 Guidelines and Limitations for Port VLAN Mapping 272 Configuring Port VLAN Mapping on a Trunk Port 274 Configuring Inner VLAN and Outer VLAN Mapping on a Trunk Port 276
C H A P T E R 1 4
Configuring IGMP Snooping 279
Configuring IGMP Snooping Over VXLAN 279
Overview of IGMP Snooping Over VXLAN 279 Guidelines and Limitations for IGMP Snooping Over VXLAN 279 Configuring IGMP Snooping Over VXLAN 280
C H A P T E R 1 5
Configuring Private VLANs 281
About Private VLANs over VXLAN 281 Guidelines and Limitations for Private VLANs over VXLAN 282 Configuration Example for Private VLANs 282
C H A P T E R 1 6
Configuring Policy-Based Redirect 285
About Policy-Based Redirect 285 Guidelines and Limitations for Policy-Based Redirect 286 Enabling the Policy-Based Redirect Feature 286 Configuring a Route Policy 287 Verifying the Policy-Based Redirect Configuration 288 Configuration Example for Policy-Based Redirect 288
C H A P T E R 1 7