Fabricpath/TRILL, OTV, LISP and VXLAN

Flexible Data Centre Fabric - FabricPath/TRILL, OTV, LISP and VXLAN

Ron Fuller– CCIE #5851 (R&S/Storage) Technical Marketing Engineer, Nexus 7000 [email protected] Agenda

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

Port Density Priority Flow Control Adequate Buffer Capacity Early Congestion Notification Adequate Table Sizes FabricPath Multiple Trees Low Latency Switching ECMP L2 & L3 Cut-through Switching Multi-tenancy : : : : Architecture is evolving Rapidly – in the next 24 months L2/L3 Boundary becomes less relevant Clos Topologies dominate new implementations HA models shift Server Edge becomes more intelligent DC Fabric becomes more scalable

L3/L2 L3/L2

L2 L2

East-West traffic – Fate Sharing Domain Larger POD East-West Traffic – Fate Sharing Domain STP is the protocol of choice N+1 redundancy 1+1 redundancy – limited forwarding paths IS-IS is the protocol of choice Broad forwarding paths East-West across L3 boundaries Broader Adjacency Support OSPF/EIGRP are protocols of choice N+1 redundancy – Broad forwarding Paths Same number of physical boxes and links Protocol behavior is L3-like North-South traffic Multi-pathing over L2 and L3 OSPF/EIGRP are protocols of choice More flexible L2 adjacency, better scale capacity N+1 redundancy – Broad forwarding paths Better latency consistency within POD

L2/L3

. The traditional L2 vs. L3 debate has been based on a number of issues . Scalability . Availability

. Requirements for the scalable design moving forward is a scalable, highly available switching fabric with the advantages of both L2 and L3

Advantages of Layer 2 Disadvantages of Layer 2

. Practically “plug-n-play” – No user . MAC address consumption configuration is required to build forwarding database . BPDU generation is CPU intensive with increasing number of VLANs . It makes it simple to support teaming or L2 multicast for clusters . VLAN sprawl causes flooding and broadcasts to propagate even where they are not needed . Easy to segment traffic with VLANs . Half of the links in the topology are blocking

. Very fast movement of end station addresses . Misconfigurations can cause Layer 2 loops which (ability to update MAC address tables after a may make switches unmanageable vMotion-type event)

MAC Table MAC Table A A

Layer 2 Domain

. Layer 3 Routed Topologies alleviate the consumption of L2 tables via route summarization . Layer 3 Routed topologies provide for a degree of fault isolation and

. “Routed Access” provides the logical L3 extension of the design philosophy L2 . “Scaling Up” of the Access Switch via such mechanism as the FEX provide a degree of workload mobility . “L2” domain extension of some form is required for most workload mobility requirements Workload Domain for most Hypervisor and Clustering based solutions is restricted by the Traditional Layer 2/3 boundary © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8 Segment-ID: Scaling Logical Groupings of Connectivity

S1 Web S2 App S3 Database Server Server Server

VLAN ID 802.1Q 802.1Q VLAN ID 802.1ad 12-bits 12-bits standardized frame format VLAN IDSegmentId VLAN ID 12-bits 24-bits 12-bits

L2/L3 Fabric • Identity is mapped to location Location addresses Identity • All these technologies leverage Location/Identity Mapping

FabricPath / VXLAN OTV LISP TRILL Location Switch-ID IP address IP address IP address (IS-IS) (IP protocols) (IP protocols) (IP protocols) Identity Client MAC Client MAC Client MAC Client IP/MAC (Flooding) (Flooding) (IS-IS) (Mapping DB)

Requirement Intra-DC Inter-DC

Layer 2 connectivity FabricPath/TRILL/VXLAN OTV/VPLS

IP Mobility LISP LISP Scale Secure Segmentation VXLAN / Segment-ID VPNs (LISP/MPLS)

LISP IP mobility IP Network DC-west DC-east

POD POD POD POD

App App App OTV/VPLS App App App OS OS OS (Inter-DC x-L3) OS OS OS

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

Switching Routing . Easy Configuration . Multi-pathing (ECMP) . Plug & Play . Fast Convergence . Provisioning Flexibility . Highly Scalable FabricPath “FabricPath brings Layer 3 routing benefits to flexible Layer 2 bridged Ethernet networks”

MAC-in-MAC Optimal MAC Learning IS-IS

• Creates hierarchical layer 2 • Prevent potential MAC table • Scalable routing protocol with address scheme with additional MAC overflow in large scale L2 domain proven implementation for fast header • Traditional source-learning only on convergence upon network changes • Source and destination Switch_ID Edge port for locally connected MAC • Link-state protocol ensures optimal written into outer MAC header at addresses path between any 2 nodes L2MP edge • Learning is disabled on Core port to • Built-in authentication mechanism • Forwarding inside L2MP core reduce MAC table utilization enhances network security and network is based on destination • Non-local source-MAC only learned stability Switch_ID if destination-MAC is already learned • Inherent support for ECMP and • Embedded path selector (FTAG) as local entry multi-topology maximize link provides multi-pathing for even utilization broadcast and multicast • Built-in protections (TTL and multicast RPF) minimize impact of transient network issues

• IS-IS assigns addresses to all FabricPath switches automatically • Compute shortest, pair-wise paths • Support equal-cost paths between any FabricPath switch pairs

S10 S20 S30 S40 FabricPath Routing Table

Switch IF S10 L1 S20 L2 S30 L3 FabricPath S40 L4 L1 L2 L3 S200 L1, L2, L3, L4 L4 … … S400 L1, L2, L3, L4 S100 S200 S300 S400

• The association MAC address/Switch ID is maintained at the edge

S10 S20 S30 S40

Switch ID space: S300: FabricPath Routing decisions A  B S100  S300 Routing Table are made based on Switch IF the FabricPath … … routing table S100 FabricPath S200(FP) S300 S100 L1, L2, L3, L4

MAC adress space: 1/1 1/2 S300: CE MAC Address Table Switching based on Classical Ethernet (CE) MAC IF MAC address tables A B B 1/2 …A S100…

• Core fabric leverages an independent routing topology from the edge • Scales MAC learning • Scales Core topology state © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16 New Control and Data Plane

• Edge switches maintain both MAC address table and Switch ID table

• Ingress switch uses MAC table to determine destination Switch ID

• Egress switch uses MAC table (optionally) to determine output switchport

S10 S20 S30 S40

FabricPath MAC Table on S100

MAC IF/SID Local MACs point A e1/1 S100 S101 FabricPath S200 to switchports B e1/2 Remote MACs point C S101 to Switch IDs D S200

• FabricPath IS-IS manages Switch ID (routing) table

• All FabricPath-enabled switches automatically assigned Switch ID (no user configuration required)

• Algorithm computes shortest (best) paths to each Switch ID based on link metrics

• Equal-cost paths supported between FabricPath switches

S10 S20 S30 S40

FabricPath Routing Table on S100 Switch IF One „best‟ path S10 L1 to S10 (via L1) S20 L2 S30 L3 L1 L2 L3 L4 S40 L4 Four equal-cost S101 L1, L2, L3, L4 paths to S101 … … FabricPath S200 L1, L2, L3, L4

MAC IF MAC IF A e1/1 A s1,e1/1 … … FabricPath … … B s8, e1/2 B e1/2 s3 s5 s8 e1/1 e1/2 A B MAC IF … …

• Edge switch only learn the MAC of remote hosts when there are two way communications between remote hosts and local hosts

• Unknown unicast flooding alone won‟t have all switches within VLAN learn the source MAC

• Intermediate switches don‟t learn the MAC

• Hardware based MAC learning

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19 Cisco FabricPath Terminology . Interface connected to another FabricPath device . Sends/receives traffic with FabricPath header . Does not run spanning tree . Does not perform MAC learning! . Exchanges topology info through L2 ISIS adjacency FP Core Ports . Forwarding based on „Switch ID Table‟ S10 S20 S30 S40

Spine Switch

FabricPath (FP) S100 S200 S300

Leaf Switch

1/1 1/2 Classical Ethernet (CE) A B

CE Edge Ports . Interface connected to traditional network device . Sends/receives traffic in standard 802.3 Ethernet frame format . Participates in STP domain . Forwarding based on MAC table

Automatically handled by IS-IS FabricPath

V10 V20 V30 V30 V10 V20 V10 V30

V10 V20 V30

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21 • Multidestination traffic constrained to Root for Root for loop-free trees touching all FabricPath Tree 1 Tree 2 switches S10 S20 S30 S40 • Root switch assigned for each multidestination tree in FabricPath domain

• Loop-free tree built from each Root and assigned a network-wide identifier (Ftag)

• Support for multiple multidestination S100 S101 FabricPath S200 trees provides multipathing for multidestination traffic Two trees supported in NX-OS release 5.1 S100 S20 S100 S10

S10 S101 S30 S40 S101 S20

FabricPath Topology „0‟ VLAN 20 (DC Wide) Common across entire Data Center

FabricPath Topologies FabricPath FabricPath Topology Topology „1‟ „2‟ VLAN 20 – DC Wide VLAN 20 – DC Wide VLAN 30 – POD Local (and non-unique) VLAN 30 – POD Local (and non-unique) VLAN 10 – POD Local (and unique) VLAN 40 – POD Local (and unique)

• Extending FabricPath to the edge switches without requiring a redesign of the VLAN topology • Each FP switch can have up to 2 Topology ID‟s defined (Topology ID‟s does not have to be unique). • Each Topology will have 2 Multi-Destination Trees defined

Classical Ethernet Frame DMAC SMAC 802.1Q Etype Payload CRC

16 bytes Original CE Frame

Outer Outer FP CRC Cisco FabricPath DA SA Tag DMAC SMAC 802.1Q Etype Payload (new) Frame (48) (48) (32)

6 bits 1 1 2 bits 1 OOO/DL 1 12 bits 8 bits 16 bits 16 bits 10 bits 6 bits

RSVD U/L Endnode ID I/G Endnode ID Sub Etype

Switch ID LID Ftag TTL

(5:0) (7:6) Switch ID 0x8903

• Switch ID – Unique number identifying each FabricPath switch • Sub-Switch ID – Identifies devices/hosts connected via VPC+ • LID – Local ID, identifies the destination or source interface • Ftag (Forwarding tag) – Unique number identifying topology and/or distribution tree • TTL – Decremented at each switch hop to prevent frames looping infinitely

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24 Putting it all together – Host A to Host B (1) Broadcast ARP Request Root for Root for Multidestination Tree 1 Tree 2 Trees on Switch 10 S10 S20 S30 S40 4 Tree IF DA→FF Ftag → 1 po100,po200,po300 Ftag→1 po300 2 po100 SA→100.0.12 DA→FF po100 po200 DMAC→FF Ftag→1

SMAC→A SA→100.0.12

Multidestination Payload DMAC→FF

Trees on Switch 100 SMAC→A po10 po20 po30 po20 po30 po40 3 Tree IF po40 po10 Payload Broadcast → 1 po10 S100 S200 Multidestination S300 2 po10,po20,po30,po40 Trees on Switch 300 5 Tree IF 6 FabricPath e1/13 Ftag → 1 po10,po20,po30,po40 e2/29 Payload MAC Table on S100 DMAC→FF 2 po40 SMAC→A MAC IF/SIDIF/SID SMAC→A DMAC→FF A e1/13 (local) 2 Payload FabricPath MAC A MAC B 1 MAC Table on S200

• S100: S100# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID MAC A learned as ------+------+------+------+------+----+------local entry on e1/13 * 10 0000.0000.000a dynamic 0 F F Eth1/13

S100#

• S10 (and S20, S30, S40, S200, S300): S10# sh mac address-table dynamic MAC A not learned Legend: on other switches * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID ------+------+------+------+------+----+------

10 Tree IF Ftag → 1 po100,po200,po300 po300 2 po100 DA→MC1 DA→MC1 Ftag→1 Ftag→1 po100 po200 SA→300.0.64 SA→300.0.64 DMAC→A Multidestination DMAC→A SMAC→B Trees on Switch 100 SMAC→B po20 po30 po40 po10 po20 po30 Payload Payload 11 Tree IF po40 po10 Ftag → 1 po10 S200 Multidestination S300 2 po10,po20,po30,po40 Trees on Switch 300 9 Tree IF 7 FabricPath e1/13 e2/29 Unknown → 1 po10,po20,po30,po40 DMAC→A MAC Table on S100 Payload 2 po40 SMAC→B MAC IF/SIDIF/SID SMAC→B Payload 12 DMAC→A A e1/13 (local) FabricPath MAC Table on S300 MAC A MAC B B 300.0.64 (remote) 8 MAC IF/SID MISS © 2010 Cisco and/or its affiliates. All rights reserved. B e2/29 (local) Cisco Confidential 27 Putting it all together – Host A to Host B MAC Address Table after the first ARP frame

• S100: S100# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID ------+------+------+------+------+----+------S100 learns MAC B as * 10 0000.0000.000a dynamic 90 F F Eth1/13 remote entry reached 10 0000.0000.000b dynamic 60 F F 300.0.64 through S300

S100#

• S300: S300# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link MAC B learned as VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID local entry on e2/29 ------+------+------+------+------+----+------• 10 0000.0000.000b dynamic 0 F F Eth2/29 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28 FabricPath Routing Table on S30 S10 S20 S30 S40 Switch IF … … S300 → S300 po300 16 po300 DA→300.0.64 DA→300.0.64 FabricPath Routing Ftag→1 Ftag→1 Table on S100 SA→100.0.12 SA→100.0.12 Switch IF DMAC→B DMAC→B S10 po10 SMAC→A SMAC→A S20 po20 po10 po20 po30 po20 po30 po40 Payload Payload Hash po40 po10 S30 po30 S200 S300 S40 po40 S100 FabricPath Routing po10, po20, Table on S300 S200 17 po30, po40 15 Switch IF po10, po20, S300 e1/13 … … e2/29 S300 → po30, po40 Payload S300 Use LID (64) S300 → SMAC→A FabricPath DMAC→B DMAC→B MAC Table on S100 SMAC→A FabricPath MAC A MAC Table on S300 MAC B MAC IF/SID Payload MAC IF/SID A e1/13 (local) 14 13 18 A S100.0.12 (remote) B → B 300.0.64 (remote) If DMAC is known, then B e2/29 (local) © 2010 Cisco and/or its affiliates. All rights reserved. learn remote MAC Cisco Confidential 29 Putting it all together – Host A to Host B Unicast forwarding

S100# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID ------+------+------+------+------+----+------* 10 0000.0000.000a dynamic 90 F F Eth1/13 10 0000.0000.000b dynamic 60 F F 300.0.64

S300# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID ------+------+------+------+------+----+------S100 learns MAC A as remote entry reached 10 0000.0000.000a dynamic 30 F F 100.0.12 through S100 • 10 0000.0000.000b dynamic 90 F F Eth2/29

S100# sh fabricpath route FabricPath Unicast Route Table 'a/b/c' denotes ftag/switch-id/subswitch-id '[x/y]' denotes [admin distance/metric] ftag 0 is local ftag Topology (ftag), Switch subswitch-id 0 is default subswitch-id ID, Sub-Switch ID

FabricPath Unicast Route Table for Topology-Default Administrative distance, routing metric 0/100/0, number of next-hops: 0 via ---- , [60/0], 0 day/s 04:43:51, local 1/10/0, number of next-hops: 1 Route age via Po10, [115/20], 0 day/s 02:24:02, isis_fabricpath-default 1/20/0, number of next-hops: 1 via Po20, [115/20], 0 day/s 04:43:25, isis_fabricpath-default Client protocol 1/30/0, number of next-hops: 1 via Po30, [115/20], 0 day/s 04:43:25, isis_fabricpath-default Next-hop interface(s) 1/40/0, number of next-hops: 1 via Po40, [115/20], 0 day/s 04:43:25, isis_fabricpath-default FabricPath 1/200/0, number of next-hops: 4 via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-default S10 S20 S30 S40 via Po20, [115/40], 0 day/s 04:43:06, isis_fabricpath-default via Po30, [115/40], 0 day/s 04:43:06, isis_fabricpath-default via Po40, [115/40], 0 day/s 04:43:06, isis_fabricpath-default po10 1/300/0, number of next-hops: 4 po20 via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-default po30 via Po20, [115/40], 0 day/s 04:43:25, isis_fabricpath-default po40 S100 S200 S300 via Po30, [115/40], 0 day/s 04:43:25, isis_fabricpath-default via Po40, [115/40], 0 day/s 04:43:25, isis_fabricpath-default © 2010 Cisco and/or its affiliates. All rights reserved. A B Cisco ConfidentialC 31 FabricPath Design STP Interaction

FabricPath (no STP) FabricPath

Classical Ethernet STP Domain (STP) STP STP Domain 1 BPDU BPDU✖ Domain 2 CE Edge Ports . FabricPath domain appears as single Spanning-Tree bridge . All FabricPath bridges share a common (static) bridge ID Cisco reserved MAC c84c.75fa.6000 . STP BPDUs are not carried through the FabricPath network . Configure all FabricPath edge switches using “spanning-tree vlan root primary” (or manually configure bridge priority lower than any STP bridge) Each FabricPath edge switch must be the root for all connected STP domains Strongly recommended to use the same bridge priority on all FabricPath edge switches © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32 FabricPath L2/L3 Boundary Location

Layer 3 Boundary at the Spine Layer 3 Integration at the Leaf/Edge

. Straightforward with two spine switches . Provides a “cleaner” spine design

. Considerations with more than two spines: . Traffic distributed equally across spines (no hot . HSRP: Traffic polarized to spines on a per VLAN basis spot) (South-North) . Increased number of hops to reach gateway . GLBP to distribute servers to different default gateways (latency) . Anycast FHRP future solution

L3 FabricPath

FabricPath

L3 L3

• Simplest migration from most existing designs L3 Domain • The spine is also used for routing with

M1/F1 in the same VDC L3

• Consideration – MAC Learning and Scaling edge/spine id based id

M1+F1 M1+F1 - • Compared to classic ethernet designs you s

gain:

forwarding for learning MAC + traffic routed Ease of configuration Switch MAC address table increased scalability and more efficient learning Traffic distribution on all uplinks edge Possibility to offload the spine by providing direct communication paths between the edge layer devices […] Conversational Learning Conversational Learning

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34 FabricPath L2/L3 Boundary Location Leaf/Spine/Boundary Architecture L3 Domain • By separating the L3 function from the spine, the F1 card in L3 edge the spine performs pure switch- id forwarding M1/F1 M1/F1 FP port FP port • The L3 edge will need both M1/F1 in order to connect with Fabricpath ports to the spine

spine spine

• The M1/F1 L3 edge will need to id based id perform learning for the remote -

mac addresses

Switch forwarding • L3 edge and spine can be combined in the same chassis by means of VDCs edge Conversational Conversational Learning

Large Scale Fabric 4K VLAN’s, 128K MAC Address, 512K Routes

blade1 blade1 blade1 blade1 blade1 blade1 blade2slot 1 blade2slot 1 blade1 blade1 blade2slot 1 blade2slot 1 blade2slot 1 blade2slot 1 blade3slot 2 blade3slot 2 blade1 blade1 blade1 blade1 blade2slot 1 blade2slot 1 blade3slot 2 blade3slot 2 blade3slot 2 blade3slot 2 blade4slot 3 blade4slot 3 blade2slot 1 blade2slot 1 blade2slot 1 blade2slot 1 blade3slot 2 blade3slot 2 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade5slot 4 blade5slot 4 blade3slot 2 blade3slot 2 blade3slot 2 blade3slot 2 slot 3 slot 3 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 blade6slot 5 blade6slot 5 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 4 blade4slot 4 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade7slot 6 blade7slot 6 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 blade5 blade5 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 slot 7 slot 7 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 8 blade8slot 8 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 slot 8 slot 8 slot 8 slot 8 slot 7 slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 8 blade8slot 8 slot 8 slot 8 slot 8 slot 8 blade1 blade1 blade1 blade1 blade1 blade1 blade2slot 1 blade2slot 1 blade1 blade1 blade2slot 1 blade2slot 1 blade2slot 1 blade2slot 1 blade3slot 2 blade3slot 2 blade1 blade1 blade1 blade1 slot 1 slot 1 blade3slot 2 blade3slot 2 blade3slot 2 blade3slot 2 blade4slot 3 blade4slot 3 blade2slot 1 blade2slot 1 blade2slot 1 blade2slot 1 blade2slot 2 blade2slot 2 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade5slot 4 blade5slot 4 blade3slot 2 blade3slot 2 blade3slot 2 blade3slot 2 blade3 blade3 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 slot 5 slot 5 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade6 blade6 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 slot 5 slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 slot 7 slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade6 blade6 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 blade8slot 8 blade8slot 8 slot 8 slot 8 slot 8 slot 8 blade7slot 6 blade7slot 6 slot 7 slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 8 blade8slot 8 slot 8 slot 8 blade1 blade1 slot 8 slot 8 blade1 blade1 blade1 blade1 blade2slot 1 blade2slot 1 blade1 blade1 blade2slot 1 blade2slot 1 blade2slot 1 blade2slot 1 blade3slot 2 blade3slot 2 blade1 blade1 blade1 blade1 blade2slot 1 blade2slot 1 blade3slot 2 blade3slot 2 blade3slot 2 blade3slot 2 blade4slot 3 blade4slot 3 blade2slot 1 blade2slot 1 blade2slot 1 blade2slot 1 blade3slot 2 blade3slot 2 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade5slot 4 blade5slot 4 blade3slot 2 blade3slot 2 blade3slot 2 blade3slot 2 slot 3 slot 3 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 blade6slot 5 blade6slot 5 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 3 blade4slot 4 blade4slot 4 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade7slot 6 blade7slot 6 blade5slot 4 blade5slot 4 blade5slot 4 blade5slot 4 blade5 blade5 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 slot 7 slot 7 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade6slot 5 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 8 blade8slot 8 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 blade7slot 6 slot 8 slot 8 slot 8 slot 8 slot 7 slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 7 blade8slot 8 blade8slot 8 slot 8 slot 8 slot 8 slot 8 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36 Standards Based + Cisco Extensions

• Nexus 5500, F1, F2 3 3 Cisco Forwarding 0 TRILL Forwarding 0 and all future HW are 1 1 capable of IETF Outer CDCE DA Outer MAC DA standards TRILL Outer CDCE DA Outer CDCE SA Outer MAC DA Outer MAC SA NextHop Header Outer CDCE SA Outer MAC SA • Support for TRILL in TTL NX-OS is pending ET = DTAG FTAG ET = 802.1Q Outer VLAN completion of Inner MAC DA ET = TRILL V/R/M, HopCnt TRILL extensions to the Inner MAC DA Inner MAC SA Egress RB Ingress RB Header baseline protocol Inner MAC SA Inner MAC DA

• Multi-topology, VRRP ET = 802.1Q Inner VLAN Inner MAC DA Inner MAC SA Ethernet interaction, … Inner MAC SA Header Payload… ET = 802.1Q Inner VLAN Payload...

L3 Core L2+L3 FabricPath Core

FabricPath POD vPC POD

vPC+ POD vPC+ POD

Path Fabric

Site 1

FabricPath FabricPath FabricPath FabricPath

Site 4 Site 2

Path Fabric

Site 3

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

• Solution: VXLAN Web vApp1 vApp2 Web Millions of dedicated LAN segments VM VM Security at Scale App App VM VM vApp mobility across data centers & clouds DB DB VM • VXLAN is network friendly VM Efficient load sharing of links (port channel) Supports NAT; better security controls

VXLAN IETF Draft: http://datatracker.ietf.org/doc/draft-mahalingam-dutt-dcops-vxlan/

. Ethernet in IP overlay network . Tunnel between VEMs . Entire L2 frame encapsulated in UDP . VMs do NOT see VXLAN ID . 50 bytes of overhead . IP multicast used for L2 broadcast/multicast, . Include 24 bit VXLAN Identifier unknown unicast . 16 Million logical networks . Technology submitted to IETF for standardization (Cisco, VMware, Citrix, Red . VXLAN can cross Layer 3 (IPv4 currently) Hat, Broadcom, Arista, and Others)

VXLAN Encapsulation Original Ethernet Frame

Outer Outer VXLAN Inner InnerM Optional Original Outer Outer Outer Outer MAC MAC Header (8 MAC AC Inner Ethernet CRC 802.1Q IP DA IP SA UDP DA SA bytes) DA SA 802.1Q Payload

VXLAN Flags8 Networker Reserved Res. bits Identifier (VIN) 24 bits 8 bits © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41 24 bits • The Nexus 1000V VEMs act as the VXLAN Tunnel Endpoints (VTEP)

• Nexus 1000V uses a VMKNIC to terminate VTEP traffic

• VM to VM traffic on different access switches is encapsulated in a VXLAN header + UDP + IP

• VTEPs use multicast to deliver unknown destination VM MAC addresses to all VTEPs participating in a given VXLANs

• VM MAC to VTEP IP address mappings are gleaned from encapsulated packets Similar to Ethernet bridge flood and learn behavior

• Known destination VM MAC addresses are carried over point to point tunnels between VTEPs

End End Bridge System Bridge System Domain Domain VTEP Switch VTEP IP Multicast Switch Enabled Underlying Network End End System System

Direct Unicast tunnels between VTEPs VTEP = VXLAN Tunnel End Point (Carries known unicast frames) VNI = VXLAN Network Identifier

VTEP VTEP VXLAN‟s IP Any Source Multicast Group (*,G) acts as a bus for delivery to all relevant VTEPs for a given VNI (Carries unknown/broadcast/multicast frames)

VTEP VTEP

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43 43 © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44 VTEP Use Of IGMP IGMP Used to Join Each VXLANs Assigned Multicast Group on Demand

Web DB DB Web VM VM VM VM

Join Multicast Join Multicast Group 239.1.1.1 Group 239.2.2.2 Join Multicast Join Multicast Group 239.2.2.2 Group 239.1.1.1

VM 1 MAC: VM 2 MAC: VM 3 abc xyz

VXLAN VXLAN VXLAN VMKNIC VMKNIC VMKNIC 1.1.1.1 2.2.2.2 3.3.3.3 VEM 1 VEM 2 VEM 3 Multicast Multicast Multicast

VM 1 MAC: VM 2 MAC: VM 3 abc xyz

VXLAN VXLAN VXLAN VMKNIC VMKNIC VMKNIC 1.1.1.1 2.2.2.2 3.3.3.3

Unicast

MAC Table: VEM 2 VM Source MAC Remote Host VXLAN IP Layer 3 VM1:abc 1.1.1.1

VM 1 MAC: VM 2 MAC: VM 3 abc xyz

VXLAN VXLAN VXLAN VMKNIC VMKNIC VMKNIC 1.1.1.1 2.2.2.2 3.3.3.3 VEM 1 VEM 2 VEM 3

MAC Table: VEM 1 MAC Table: VEM 2 VM Source MAC Remote Host VM Source MAC Remote Host VXLAN IP VXLAN IP VM2:xyz 2.2.2.2 VM1:abc 1.1.1.1

VM 1 MAC: VM 2 MAC: VM 3 abc xyz

VXLAN VXLAN VXLAN VMKNIC VMKNIC VMKNIC 1.1.1.1 2.2.2.2 3.3.3.3

Unicast

MAC Table: VEM 1 MAC Table: VEM 2 VM Source MAC Remote Host VM Source MAC Remote Host VXLAN IP VXLAN IP VM2:xyz 2.2.2.2 VM1:abc 1.1.1.1

Web App DB App VM VM VM VM

• Encapsulate with Blue VXLAN ID VEM Discards Since No VM with • Multicast to Servers Registered for Blue VXLAN ID 239.1.1.1 Multicast Group

VM Broadcast Frames Sent to More Servers

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51 Single Network Architecture Delivers: . VM Mobility (topology independent addressing) . Security: VPNs/Multi-tenancy . Route Scalability (on demand routing) . IPv6 enablement, . Routing Policy simplification

Benefits Use-Cases

. Services integrated in a single architecture . DCI route optimization/mobility . Services can be offered across organizational . Workload Portability to Cloud boundaries (multiple providers) . Secure Multi-tenancy across organizations . Very large scale . Rapid IPv6 Deployment . Open model to integrate with cloud orchestrators . Route scaling

Efficient Multi-Homing IPv6 Transition Support v6 LISP v6 Services LISP Router Internet Router IPv4 IPv6 Internet Internet LISP LISP v4 Site v6 v6 Routers

. IP Portability . v6-over-v4, v6-over-v6 . Ingress Traffic Engineering without BGP . v4-over-v6, v4-over-v4

Multi-Tenancy and VPNs Host-Mobility LISP Site LISP Site

IP Network IP Network

West-DC East-DC West-DC East-DC

. Reduced CapEx/OpEx . Cloud / Layer 3 VM moves . Large scale Segmentation . Segmentation © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53 Location Identity Separation Protocol What Do We Mean by “Location” and “Identity”?

Today‟s IP Behavior Loc/ID “Overloaded” Semantic IP core 10.1.0.1 When the Device Moves, It Gets a New IPv4 or IPv6 Address for Its Device IPv4 or IPv6 New Identity and Location Address Represents 20.2.0.9 Identity and Location

LISP Behavior Loc/ID “Split” IP core 10.1.0.1 When the Device Moves, Keeps Device IPv4 or IPv6 1.1.1.1 Its IPv4 or IPv6 Address. 2.2.2.2 It Has the Same Identity Address Represents 10.1.0.1 Identity Only. Its Location Is Here! Only the Location Changes

3 EID-prefix: 10.2.0.0/24 Mapping Locator-set: Entry Non-LISP site This Policy Controlled 1 Non2.1.-1LISP.1, sitepriority: 1, weight: 50 (D1) DNS Entry: by Destination Site 2.1.2.1, priority: 1, weight: 50 (D2) D.abc.com A 10.2.0.1 10.1.0.0/24 LISP Site S ITR PITR 2 1.1.1.1 5.4.4.4

10.1.0.1 -> 10.2.0.1 IP Network 5.3.3.3 EID-to-RLOC 4 mapping 1.1.1.1 -> 2.1.1.1 5.1.1.1 5.2.2.2 10.1.0.1 -> 10.2.0.1 2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1 ETR 5 West-DC East-DC 10.1.0.1 -> 10.2.0.1 D 10.2.0.0/24 10.3.0.0/24

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55 A LISP Packet Walk 3 How About Non-LISP Sites? EID-Prefix: 10.2.0.0/24 Mapping Locator-Set: 1 Entry DNS Entry: 2.1.1.1, priority: 1, weight: 50 (D1) D.abc.com A 10.2.0.1 2.1.2.1, priority: 1, weight: 50 (D2) Non-LISP Site Non-LISP Site S

2 192.3.0.1 -> 10.2.0.1 PITR 4.4.4.4

4 5.3.3.3 4.4.4.4- > 2.1.2.1 EID-to-RLOC 192.3.0.1 -> 10.2.0.1 mapping IP Network 5.1.1.1 5.2.2.2 2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1 ETR 5 West-DC East-DC 192.3.0.1 -> 10.2.0.1 D 10.2.0.0/24 10.3.0.0/24

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56 LISP Roles and Address Spaces Mapping EID RLOC What Are the Different Components Involved? a.a.a.0/24 w.x.y.1 DB b.b.b.0/24 x.y.w.2 c.c.c.0/24 z.q.r.5 d.d.0.0/16 z.q.r.5 EID RLOC a.a.a.0/24 w.x.y.1 b.b.b.0/24 x.y.w.2 c.c.c.0/24 z.q.r.5 LISP Roles EID Space d.d.0.0/16 z.q.r.5 EID RLOC a.a.a.0/24 w.x.y.1 b.b.b.0/24 x.y.w.2 • Tunnel Routers - xTRs c.c.c.0/24 z.q.r.5 d.d.0.0/16 z.q.r.5 • Edge devices in charge of ITR encap/decap Non-LISP ALT Prefix Next-hop w.x.y.1 e.f.g.h x.y.w.2 e.f.g.h • Ingress/Egress Tunnel Routers z.q.r.5 e.f.g.h (ITR/ETR) z.q.r.5 e.f.g.h

• EID to RLOC Mapping DB PxTR RLOC Space • Contains RLOC to EID ETR mappings • Distributed across multiple Map EID Space Servers (MS) • MS may connect over an ALT network Address Spaces • Proxy Tunnel Routers - PxTR • EID = End-point Identifier • Coexistence between LISP and • Host IP or prefix non-LISP sites • RLOC = Routing Locator • Ingress/Egress: PITR, PETR • IP address of routers in the backbone

LISP Site Mapping Cache Entry (on ITR): 10.2.0.0/16-> (2.1.1.1, 2.1.2.1) ITR

Map Server / Resolver: 5.1.1.1

Map-Reply 10.2.0.0/16 -> (2.1.1.1, 2.1.2.1)

2.1.1.1 2.1.2.1 3.1.1.1 3.1.2.1 ETR ETR ETR ETR Database Mapping Entry (on ETR): Database Mapping Entry (on ETR): 10.2.0.0/16 -> (2.1.1.1, 2.1.2.1) 10.3.0.0/16 -> (3.1.1.1, 3.1.2.1)

West-DC East-DC 10.2.0.0 /16 10.3.0.0/16 Y X Y Z

© 2010 Cisco and/or its affiliates. All rights reserved. 10.2.0.2 Cisco Confidential 58 Basic LISP Configuration Servers ip lisp map-resolver ip lisp map-server lisp site west-DC authentication-key 0 s3cr3t eid-prefix 10.2.0.0/24

Border Routers Between Backbones ip lisp proxy-itr ip lisp ITR map-resolver 5.3.3.3

Non-LISP Sites LISP Site PITR ITR Branch Routers 5.3.3.3 ip lisp itr-etr 1.1.1.1 Mapping DB ip lisp ITR map-resolver 5.3.3.3 5.1.1.1 DC Aggregation Routers 5.2.2.2 IP Network ip lisp itr-etr ip lisp database-mapping 10.2.0.0/24 2.1.1.1 p1 w50 ip lisp database-mapping 10.2.0.0/24 2.1.2.1 p1 w50 2.1.1.1 2.1.2.1 ip lisp ETR map-server 5.1.1.1 key s3cr3t ip lisp ETR map-server 5.2.2.2 key s3cr3t ETR West-DC East-DC 10.2.0.0/24 Usually Devices Will Be Configured as ITRs and ETRs to Handle Traffic in Both Directions; We Illustrate Only One Direction for Simplicity © 2010 Cisco and/or its affiliates. All rights reserved. RLOC EID LISP EncapCisco/ ConfidentialDecap 59 Agenda

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60 LISP Host-Mobility Needs: • Global IP-Mobility across subnets • Optimized routing across extended subnet sites Non-LISP Sites LISP Site LISP Solution: PxTR xTR • Automated move detection on xTRs Mapping DB • Dynamically update EID-to-RLOC mappings IP Network • Traffic Redirection on ITRs or PITRs Benefits: LAN Extensions

• Direct Path (no triangulation) LISP-VM (xTR) • Connections maintained across move West-DC East-DC • No routing re-convergence • No DNS updates required • Transparent to the hosts RLOC EID LISP Encap/Decap • Global Scalability (cloud bursting) • IPv4/IPv6 Support © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 61 Host-Mobility Scenarios

Moves Without LAN Extension Moves With LAN Extension

LISP Site Non-LISP LISP Site xTR Site xTR

DR Location or Mapping DB Mapping DB Cloud Provider IP Network Internet or DC Shared WAN LAN Extension

LISP-VM (xTR) LISP-VM (xTR) West-DC East-DC West-DC East-DC

IP Mobility Across Subnets Routing for Extended Subnets

Disaster Recovery Active-Active Data Centers

Cloud Bursting Distributed Clusters Application Members in One Location Application Members Distributed (Broadcasts across sites) © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 62 LISP Host-Mobility - Move Detection Monitor the Source of Received Traffic • The new xTR checks the source of received traffic • Configured dynamic-EIDs define which prefixes may roam lisp dynamic-eid roamer Received a Packet … database-mapping 10.2.0.0/24 p1 w50 database-mapping 10.2.0.0/24 p1 w50 … It’s from a “New” Host map-server 5.1.1.1 key abcd … It’s in the Dynamic-EID Allowed interface vlan 100 Range lisp mobility roamer Mapping DB

5.1.1.1 5.2.2.2 …It’s a Move! A B C D Register the /32 with LISP LISP-VM (xTR)

West-DC East-DC 10.2.0.0 /16 10.3.0.0/16 Y X Y Z

• When a host move is detected, updates are triggered: The host-to-location mapping in the Database is updated to reflect the new location The old ETR is notified of the move ITRs are notified to update their Map-caches

• Ingress routers (ITRs or PITRs) now send traffic to the new location 10.2.0.0/16 – RLOC A, B LISP Site xTR Mapping DB 10.2.0.2/32 – RLOC C, D A B C D LISP-VM (xTR)

West-DC East-DC 10.2.0.0 /16 10.3.0.0 /16 Y X Y Z

© 2010 Cisco and/or its affiliates. All rights reserved. 10.2.0.2 Cisco Confidential 64 LISP Host-Mobility - First Hop Routing Across Different Subnets • SVI (Interface VLAN x) and HSRP configured as usual (Consistent GWY-MAC configured across all dynamic subnets)

• The lisp mobility command enables proxy-arp functionality on the SVI The LISP-VM router services first hop routing requests for both local and roaming subnets

• Hosts can move anywhere and always talk to a local gateway with the same MAC

interface vlan 100 interface vlan 100 interface vlan 200 ip address 10.3.0.7/24 ip address 10.2.0.5/24 ip address 10. 2 lisp.0.8 /mobility24 roamer lisp mobility roamer interface Ethernet2/4 lisp mobility roamerip proxy -arp ip proxy-arp ip address 10.1.0.6/24 ip proxy-arp hsrp 201 hsrp 101 lisp mobility roamer hsrp 201 mac-address 0000.0e1d.010c mac-address 0000.0e1d.010c ip proxy-arp mac-address 0000 ip 10.0.3e.10d..1010 c ip 10.2.0.1 hsrp 101 ip 10.3..0.1

mac-address 0000.0e1d.010c A B C D ip 10.2.0.1

LISP-VM (xTR) HSRP Active HSRP Active West-DC East-DC 10.2.0.0 /24 10.3.0.0 /24 HSRP HSRP ARP ARP GWY-MAC GWY-MAC © 2010 Cisco and/or its affiliates. All rights reserved. 10.2.0.2 Cisco Confidential 65 Null0 host routes indicate the host is “away”

10.2.0.0/16 – RLOC A, B 6 10.2.0.2/32 – RLOC C, D

Map-Register 10.2.0.2/32 Map-Notify Mapping DB 10.2.0.2/32 5.1.1.1 5.2.2.2

Routing Table: Routing Table: 7 5 10.2.0.0/16 – Local 10.3.0.0/16 – Local 10.2.0.2/32 – Null0 10.2.0.2/32 – Local 4 10 A B Routing Table: C D 10.3.0.0/16 – Local 2 10.2.0.2/32 – Local

Routing Table: 9 3 10.2.0.0/16 – Local 10.2.0.0 /16 10.3.0.0 /16 8 10.2.0.2/32 – Null0 1 East-DC West-DC Y

10.2.0.0/16 – RLOC A,B 1. ITRs and PITRs with cached mappings continue to send traffic to the old locators LISP site The old xTR knows the host has moved (Null0 route). ITR 10.2.0.2/32 – RLOC C,D 2. Old xTR sends Solicit Map Request (SMR) messages to any encapsulators sending traffic to the moved host Mapping DB

3. The ITR then initiates a new map request process

4. An updated map-reply is issued from the A B C D new location LISP-VM (xTR)

5. The ITR Map Cache is updated West-DC East-DC 10.2.0.0 /16 10.3.0.0 /16 • Traffic is now re-directed Y X Y Z • SMRs are an important integrity measure to 10.2.0.2 avoid unsolicited map responses and spoofing © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 67

LISP Host-Mobility Configuration Across Subnets (No LAN Extensions)

ip lisp ITR-ETR ip lisp ITR-ETR ip lisp database-mapping 10.3.0.0/16 ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.3.0.0/16 ip lisp database-mapping 10.2.0.0/16 lisp dynamic-eid roamer lisp dynamic-eid roamer database-mapping 10.2.0.0/24 database-mapping 10.2.0.0/24 database-mapping 10.2.0.0/24 database-mapping 10.2.0.0/24 map-server 1.1.1.1 key abcd map-server 1.1.1.1 key abcd map-notify-group 239.2.2.2 interface vlan 100 map-notify-group 239.1.1.1 ip address 10.3.0.11 /16 interface vlan 100 lisp mobility roamer ip address 10.2.0.10 /16 ip proxy-arp lisp mobility roamer hsrp 201 ip proxy-arp mac-address 0000.0e1d.010c hsrp 101 ip 10.3.0.1 mac-address 0000.0e1d.010c ip 10.2.0.1 Mapping DB A B C D LISP-VM (xTR)

West-DC East-DC 10.2.0.0 /16 10.3.0.0 /16

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

• Ethernet LAN Extension over any Network Works over dark fiber, MPLS, or IP Many Physical Sites – Multi-data center scalability One Logical Data Center

• Simplified Configuration & Operation Seamless overlay - No network re-design Single touch site configuration

• High Resiliency Failure domain isolation Seamless Multi-homing Any Workload, Anytime, Anywhere • Maximizes available bandwidth Unleashing the Full Potential of Compute Virtualization Automated multi-pathing Optimal multicast replication

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 70 OTV Data Plane Inter-Site Packet Flow 1. Layer 2 lookup on the destination MAC. 4. The Edge Device on site East receives MAC 3 is reachable through IP B and decapsulates the packet 2. The Edge Device encapsulates the frame 5. Layer 2 lookup on the original frame. 3. The transport delivers the packet to the MAC 3 is a local MAC Edge Device on site East 6. The frame is delivered to the destination

3 MAC TABLE Transport MAC TABLE VLAN MAC IF VLAN MAC IF Infrastructure Decap 100 MAC 1 Eth 2 IP A 2 4 IP B 100 MAC 1 IP A 1 OTV OTV OTV OTV 5 100 MAC 2 Eth 1 Encap 100 MAC 2 IP A MAC 1  MAC 3 IP A  IP B Layer 2 100 MAC 3 IP B MAC 1  MAC 3 IP A  IP B 100 MAC 3 Eth 3 Layer 2 Lookup 100 MAC 4 IP B 100 MAC 4 Eth 4 Lookup

• OTV proactively advertises MAC reachability (control-plane learning)

• MAC addresses advertised in the background once OTV has been configured

• IS-IS is the OTV Control Protocol running between the Edge Devices

• No specific configuration is required

OTV MAC Addresses OTV Advertisements

IP A IP B West East

IP C OTV

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

• A subnet usually implies location

• Yet we use LAN extensions to stretch

subnets across locations LISP site

Location semantics of subnets are lost xTR

• Traditional routing relies on the location semantics of the subnet IP Network Can‟t tell if a server is at the East or West location of the subnet LAN Extension • More granular (host level) information is required

LISP provides host level location semantics West-DC East-DC

10.2.0.0 /24 is the dyn-EID 10.2.0.0/16 – RLOC A, B 6 10.2.0.2/32 – RLOC C, D

Map-Register 10.2.0.2/32 Mapping DB 5.1.1.1 5.2.2.2 Routing Table: Routing Table: 10.2.0.0/16 – Local 10.2.0.0/16 – Local Routing Table: 5 10.2.0.0/24 – Null0 10.2.0.0/16 – Local 10.2.0.0/24 – Null0 4 10.2.0.2/32 – Null0 10.2.0.0/24 – Null0 4 10.2.0.2/32 – Local A B 2 10.2.0.2/32 – Local C D Routing Table: 10.2.0.0/16 – Local 10.2.0.0/24 – Null0 4 10.2.0.2/32 – Null0 3 10.2.0.0 /16 3 10.2.0.0 /16 1 OTV East-DC West-DC Y

Map Cache @ ITR Refreshing the map caches 10.2.0.0/16 – RLOC A,B

1. ITRs and PITRs with cached mappings continue to LISP site send traffic to the old locators ITR 1. The old xTR knows the host has moved (Null0 route). 10.2.0.2/32 – RLOC C,D

2. Old xTR sends Solicit Map Request (SMR) messages to any encapsulators sending traffic to Mapping DB the moved host

3. The ITR then initiates a new map request process

4. An updated map-reply is issued from the new location A B C D 5. The ITR Map Cache is updated LISP-VM (xTR)

• Traffic is now re-directed West-DC OTV East-DC 10.2.0.0 /16 10.2.0.0 /16 • SMRs are an important integrity measure to avoid unsolicited map responses and spoofing Y X Y Z 10.2.0.2

• Consistent GWY-IP and GWY-MAC configured across all sites Consistent HSRP group number across sites  consistent GWY-MAC

• Servers can move anywhere and always talk to a local gateway with the same IP/MAC

interface vlan 100 interface vlan 100 ip address 10.2.0.5/24 interface vlan ip 200address 10.2.0.7/24 ip address 10.2.0.8/24 interface Ethernet lisp2 mobility/4 roamer lisp mobility roamer lisp mobility roamer ip address 10 lisp.2.0 .extended6/24 -subnet-mode lisp extended-subnet-mode lisp extended-subnet-mode lisp mobility hsrp roamer 101 hsrp 101 LAN Ext. hsrp 101 lisp extended ip-subnet 10.2.0-.mode1 ip 10.2.0.1 ip 10.2.0.1 hsrp 101 A B C D ip 10.2.0.1 LISP-VM (xTR)

HSRP Active HSRP Active West-DC East-DC 10.2.0.0 /24 10.2.0.0 /24 HSRP HSRP ARP ARP GWY-MAC GWY-MAC © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 77 LISP VM-Mobility Configuration With Extended Subnets  Use “Extended-Subnet-Mode” ip lisp ITR-ETR ip lisp ITR-ETR ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.2.0.0/16 ip lisp database-mapping 10.2.0.0/16 lisp dynamic-eid roamer lisp dynamic-eid roamer database-mapping 10.2.0.0/24 … database-mapping 10.2.0.0/24 database-mapping 10.2.0.0/24 database-mapping 10.2.0.0/24 map-server 1.1.1.1 key abcd map-server 1.1.1.1 key abcd map-notify-group 239.10.10.10 map-notify-group 239.10.10.10 interface vlan 100 interface vlan 100 ip address 10.2.0.10 /16 ip address 10.2.0.11 /16 lisp mobility roamer lisp mobility roamer lisp extended-subnet-mode lisp extended-subnet-mode hsrp 101 hsrp 101 ip 10.2.0.1 ip 10.2.0.1

LAN Ext. 1.1.1.1 2.2.2.2 Mapping DB A B C D

LISP-VM (xTR)

West-DC East-DC 10.2.0.0/16

• Clients (192.168.0.1 & 192.168.2.1 CLIENT 192.168.2.1 communicate with Server 10.2.0.2 Non-LISP Sites CLIENT 192.168.2.1  10.2.0.2 • Client-server traffic is LISP 10.1.0.1 LISP Site xTR PxTR encapsulated at the ITRs or PITRs 10.1.0.1  10.2.0.2 G F Mapping DB Client-to-server: to ETRs C or D GD 192.168.2.1  10.2.0.2 Server-to-client: FC 10.1.0.1  10.2.0.2 to ETR (F) for LISP sites to PETR (G) for non-LISP sites A B C D • Server-Server off-subnet traffic across LISP-VM (xTR) sites is also LISP encapsulated West-DC East-DC 10.2.0.0 /16 10.3.0.0 /16 10.1.0.1  10.2.0.2 Y 192.168.2.1  10.2.0.2 X Y

© 2010 Cisco and/or its affiliates. All rights reserved. 10.2.0.2 Cisco Confidential 79 On-Subnet Server-Server Traffic On Subnet Traffic Across L3 boundaries With LAN Extension Without LAN Extensions • Live moves and cluster member • Cold moves, no application dispersion dispersion • X- Y traffic is sent to the LISP-VM router & LISP encapsulated • Traffic between X & Y uses the LAN Extension • Need LAN extensions for link-local multicast traffic • Link-local-multicast handled by the LAN Extension BC 10.2.0.3  10.2.0.2 Mapping DB LAN Ext. 10.2.0.3  10.2.0.2 A B C D A B C D

LISP-VM (xTR) LISP-VM (xTR) West-DC West-DC 10.2.0.0/16 East-DC East-DC 10.2.0.0/16 10.3.0.0/16

10.2.0.3 Y 10.2.0.3 Y X Y Z X Y Z

. The Evolving Data Centre Fabric

. FabricPath

. VXLAN 1K Cisco Nexusx8 . LISP 6

. LISP Host Mobility

. OTV LAN Extension

. Mobility with Extended Subnets

. Nexus Fabric

• Distance limited by application latency budget and storage replication

• Two types of traffic specific to the cluster: Non-routable heartbeats: FabricPath (Intra-DC) & OTV (Inter-DC) provide LAN connectivity Front-end IP connectivity: LISP provides mobility for cluster virtual-IP failover

LISP IP mobility IP Network DC-west DC-east OTV POD POD (Inter-DC) POD POD

App Cluster Distributed App (GeoCluster) OS OS OS

• OTV extends the LAN across DC sites without compromising network stability

• LISP integrates with SLBs and balances traffic across the SLBs (Future)

Intra-DC Inter-DC Virtual Machines VXLAN (x-L3), FabricPath (L2) OTV (x-L3) Physical Machines FabricPath (L2), VXLAN GWY (future) OTV (x-L3)

LISP IP mobility IP Network DC-west DC-east

POD POD POD POD

App App App App OTV OS OS OS (Inter-DC x-L3) OS

Fabric Path VXLAN SLB © 2010 Cisco(Intra and/or-DC its affiliates. L2) All rights(Intra reserved.- DC x-L3) Cisco Confidential 83 • Reduce Disaster Recovery Bring-up times - Less Network Changes/Operations = Faster recovery times

• Preserve IP addressing with LISP host mobility No reconfiguration of applications or network service policies No routing re-convergence Automatic routing re-localization (upon application bring-up at DR)

• VXLAN segments move along with the applications (vApps)

LISP IP mobility IP Network DC-west DC-east

POD POD POD POD

App App App App App App

OS OS OS OS OS OS

vxlan 1 • Move virtual Applications (vApps) to private cloud PODs V web M Move VMs and virtual Segments (VXLANs) vxlan 2

V VSG • LISP host mobility allows the vApp GWY to roam app M

Maintain GWY IP address and optimal reachability vxlan 3 V db M • VXLAN segments move along with the applications (vApps) Very large scale of virtual segments can move and extend across L3 boundaries vApp = Collection of VMs and segments LISP with a GWY IP mobility IP Network DC-west DC-east

POD POD POD POD

GWY GWY GWYvxlan 1 GWY GWY vxlan 1 VM vxlan 1 vxlan 1 vxlan 1 web VM vxlan 2 web VM vxlan 2 VM VM web VM vxlan 2 web vxlan 2 web vxlan 2 app VM vxlan 3 app VM vxlan 3 VM VM appVM vxlan 3 app vxlan 3 app vxlan 3 db VM © 2010 Ciscodb and/or itsVM affiliates. All rights reserved. VM VM Cisco Confidential 85 db db db Complimentary Capabilities FabricPath, VXLAN, LISP

Requirement Intra-DC Inter-DC

Layer 2 connectivity FabricPath/TRILL/VXLAN OTV/VPLS

IP Mobility LISP LISP Scale Secure Segmentation VXLAN / Segment-ID VPNs (LISP/MPLS)

LISP IP mobility IP Network DC-west DC-east

POD POD POD POD

App App App App App App

OS OS OS OTV/VPLS OS OS OS (Inter-DC x-L3) Fabric Path VXLAN/OTV Fabric Path VXLAN/OTV © 2010 Cisco(Intra and/or-DC its affiliates. L2) All rights(Intra reserved.- DC x-L3) (Intra-DC L2) (Intra-DC x-LCisco3) Confidential 86 Q&A

#CiscoPlusCA

We value your feedback. Please be sure to complete the Evaluation Form for this session.

Access today‟s presentations at cisco.com/ca/plus

Follow @CiscoCanada and join the #CiscoPlusCA conversation