TRILL vs. SPB

Mikael Holmberg Senior Principal Corporate Systems Engineer EMEA [email protected] Introduction – Data Center Switching Evolving!

Host A

Host B

Host C

Spanning Tree Challenges § STP introduced Blocked Ports leading to Inefficient Paths § STP has slow (in seconds) and is disruptive § Less Aggregate § MAC address tables don’t scale § Instability with Optimization § Could IP help?... Yes… but…. Optimum Forwarding

• In the following 6 Bridge Network: B C – Optimum forwarding would use all 8 links E F A D

• Loop avoidance protocols reduce B C available links E F • Traffic limited to only one path – (STP, ERPS, EAPS) A D

3 Multi-Pathing

B C

E F

A D

4 Multi-Pathing

• Bridges limit traffic to one path

B5 B6 B7

B1 B2 B3 B4

5 Multi-Pathing

• You want something that would support multi-path for higher throughput…

RB5 RB6 RB7

RB1 RB2 RB3 RB4

6 Path Computation (IS-IS) UNICAST PATH CALCULATION

• TRILL uses the Dijkstra Algorithm, to calculate the best path route 8 8 8 based on link cost to every node in E F G H the network 8 12 12 14 10 10 • Each node makes an independent 10 decision on where to send a packet I 12 A B 12 J based on the packet’s destination 8 10 10 8 egress node 12 12 • F to H: K C 10 D L – F-G-H = path cost 16 10 10 8 12 12 8 • F to N: M N O P – F-I-K-N = path cost 28 8 8 8

8 (IS-IS) MULTIPATH CALCULATION

• A link state algorithm allows multipath forwarding • Multipath forwarding allows the ingress 8 8 8 node to forward packets along multiple E F G H paths to reach the destination, so long 8 12 12 14 as they are all considered to be the best 10 10 path 10 I 12 A B 12 J • The ingress node uses a hashing algorithm to select the next peer. 8 10 10 8 – The hashing algorithm operates on the 12 12 encapsulated packet header so that K C 10 D L individual flows always follow the same 10 10 path 8 12 12 8 – This can lead to bi-directional traffic M N O P flows taking different paths based on the 8 8 8 hash • I to L: – I-A-B-J-L= cost 42 – I-K-C-C-L= cost 42 9 (IS-IS) PER HOP MULTIPATH CALCULATION

• Each hop along the path performs its own next hop look-up 8 8 8 independently of the previous hops E F G H • At each hop along the path, there 8 12 12 14 may be multiple paths that were not 10 10 available to the previous hops 10 I 12 A B 12 J • This provides another level of load sharing not available to Layer 2 8 10 10 8 networks 12 12 K C 10 D L – This is not currently supported in 10 10 Service Provider (SPB). 8 12 12 8 • M to B: M N O P – Shortest path is via C 8 8 8 – C to B: • C-A-B = path cost 20 • C-D-B = path cost 20

10 Multi-Destination Trees (TRILL)

VLAN X VLAN X VLAN X • Broadcast, Multicast and Unknown Unicast packets are forwarded 8 8 8 using Multicast Distribution Tress E F G H • RBridges compute a single shared 8 12 12 14 10 10 tree based on LSP database for all 10 multi-destination traffic I 12 A B 12 J • Multiple trees can be computed to 8 10 10 8 load-share across multiple equal 12 12 cost links K C 10 D L 10 10 • RBridge with highest priority 8 12 12 10 becomes the “TREE Root” and all M N O P distribution trees are rooted from 8 8 8 here VLAN X VLAN X

11 Multicast Distribution Trees (TRILL)

• VLAN X attached at F,E,H,M and O • RBridge F has been configured with the highest priority Root Distribution Tree • Rbridge forwards packets with VLAN tags to only those tree adjacencies that have downstream matching Access VLANs • RBridges K, G, and L are not required to forward traffic to some or all of the distribution tree adjacencies. – This effectively prunes the distribution tree and reduces packet replication and unnecessary traffic forwarding.

12 TRILL TRILL: Transparent Interconnection of Lots of Links

• IETF standard for L2 scalability • Inventor of STP is inventor of TRILL – Radia J. Perlman • Many RFCs: – RFC 5556: Problem & Applicability Statement – RFC 6325: Bridges (RBridges): Base Protocol Specification – RFC 6326: TRILL use of IS-IS – RFC 6327: Routing Bridges Adjacency – RFC 6439: Routing Bridges Appointed Forwarders WHY IS-IS …?

• The IS-IS (Intermediate System to Intermediate System) link state was chosen for SPB over OSPF (), the only other plausible candidate, for the following reasons: – IS-IS runs directly at Layer 2. Thus no IP addresses are needed, as they are for OSPF, and IS-IS can run with zero configuration. – IS-IS uses a TLV (type, length, value) encoding which makes it easy to define and carry new types of data.

15 Introduction – Best of the Both Worlds!

L2 Switching TRILL L3 Routing

ü Plan & Play ü Minimal Configuration ü ü Minimal Configuration Fast Convergence ü Plug & Play ü ü Plug & Play Multiple Paths ü Fast Convergence ü ü Load Balancing Flat Addressing ü Multiple Paths ü ü Hierarchical Slow Convergence ü Load Balancing Forwarding ü Single Path ü Hierarchical ü Multiple Multicast ü Single Multicast Tree Forwarding Trees ü Constrained Scalability ü Highly Scalable ü Highly Scalable TRILL: Transparent Interconnection of Lots of Links

TRILL Header (8 bytes including TRILL Ethertype)

OP- Egress RBridge Ingress RBridge Hop Count Options V R M Length Nickname Nickname

ü M (1-bit): Multi-destination bit (0 = Unicast, 1 = Multi-destination)

ü Hop Count (6-bit): Mitigates Loop issues

ü Nicknames (16-bit): Dynamically assigned through nickname acquisition protocol

Dynamic Nickname Acquisition Protocol ü Nicknames are manually configured or dynamically assigned

ü Dynamic nicknames based on hashing parameters (System ID, time, date etc.)

ü RBridge Nicknames advertised using Link State PDUs (LSP)

ü Priority of the nickname is advertised in the LSP

ü Nicknames are persistent across reboots TRILL Basic Interworking

TRILL Campus VLAN X

VLAN Y

ü RBridges exchanges TRILL IS-IS Hello frames

ü Hellos establish IS-IS connectivity on RBridge port

ü RBridges elect Designated RBridge (DRB) for each link ü RBridges exchanges LSP to have a global link state database

ü Includes information such as VLAN, Nicknames, link cost etc.

ü Calculates optimal paths for unicast and multi-destination traffic ü DRB specifies the Appointed Forwarder for each VLAN

ü Appointed Forwarders encapsulate/decapsulates TRILL data frames TRILL Packet Encapsulation (Unicast Frames)

RBridge1 RBridge2 RBridge3

Host X Host Y MAC A MAC B MAC C MAC D MAC E MAC F

Host X MAC MAC C MAC E Host X MAC MAC D Host Y MAC MAC B Host Y MAC Outer VLAN VLAN Outer VLAN VLAN RBridge3 RBridge3 Payload Payload Nickname Nickname FCS RBridge1 FCS RBridge1 Nickname Nickname Hop Count Outer Header Hop Count

TRILL Header Host X MAC Host X MAC

Original Frame with Host Y MAC Host Y MAC Inner Ethernet Header, Payload & new FCS Inner VLAN Inner VLAN

Payload Payload

FCS FCS SPB What is SPB? (1)

• IEEE protocol builds on 802.1 standards • A new for Q-in-Q and M-in-M – Leverage existing inexpensive ASICs – Q-in-Q mode called SPBV – M-in-M mode called SPBM • Backward compatible to 802.1 – 802.1ag, Y.1731, protocols • Multiple loop free shortest paths routing – Excellent use of mesh connectivity – Currently 16 equal cost paths. – Optimum multicast head end or tandem replication

21 What is SPB? (2)

• Light weight form of traffic engineering – Head end assignment of traffic to 16 shortest paths. – Deterministic routing - offline tools predict exact routes. • Scales to ~1000 or so devices – Uses IS-IS already proven well beyond 1000. – Huge improvement over the STP scales. • Good convergence with minimal complexity – sub second (modern processor, well designed) – below 100ms (use of hardware multicast for updates) – Includes multicast flow when replication point dies.

22 What is SPB? (3)

• Service membership advertised in same protocol as Topology – Minimizes complexity, near plug-and-play – Support E-LINE/E-LAN/E-TREE – Just variations on membership attributes • Address learning restricted to edge (M-in-M) – FDB is computed and populated just like a . – Unicast and Multicast handled at same time. • Computations guarantee unicast/multicast: – Symmetry (same in both directions) – Congruence (unicast/multicast follow same route) – Tune-ability (currently 16 equal costs paths)

23 SPBM Packet Encapsulation

BEB A BCB B BEB C VLAN X VLAN X CVLAN=20 ISID=100 CVLAN=20 ISID=100 Destination C Destination A

Data ISID ISID ISID Data Ethertype Ethertype Ethertype Ethertype Ethertype CTAG CTAG BTAG BTAG BTAG SRC MAC SRC MAC BSA DST MAC BSA BSA DST MAC

BDA BDA BDA

Data SPB Header Data Data Ethertype Ethertype Ethertype

CTAG CTAG CTAG Original Frame with Inner Ethernet SRC MAC SRC MAC SRC MAC Header, Payload etc. DST MAC DST MAC DST MAC Comparison to MLAG, SPB, VPLS/MPLS M-LAG for Active-Active Paths Efficient Bandwidth Usage

• LAG allows combining of ports effectively Core Network increasing the bandwidth. Up to 64 ports in a LAG Group. Inter- Switch • M-LAG allows combining of ports on ‘two’ Connection switches to form a single logical (ISC) connection to another network device • Active-active paths. No STP port blocking • Fast Failover • For both Layer-2 and Layer-3

deployments MLAG MLAG Group 1 Group 2 • Interoperates across tiers • Works with servers, switches, storage, and other network appliances

Data Center

26 Virtual Private LAN Service (VPLS) – RFC 4761/4762

Emulated LAN

• L2 Ethernet VPN providing multi-point communication over IP/MPLS networks • All tenants sites appear to be on the same LAN regardless of location • VPLS provides VLAN extensions over IP/MPLS networks • Each tenant VLAN is mapped to a virtual switch instance or VPN ID

27 Shortest Path Bridging (SPB) – IEEE 802.1aq

• Equal Cost Multi-Path (ECMP) solution (up to 16 trees) • Large L2 bridging topologies (up to 16 million) based on IS-IS as link state control protocol • Service & Infrastructure separation

Core/Backbone VLANs (4096 VLANs)

Service Instance IDs (16 million)

Tenant VLANs (4096 VLANs)

28 IEEE Standards Evolution to scale L2 Fabrics

Payload

VLAN ID (12 bit)

Source Address

Payload Destination Address

Payload VLAN ID (12 bit) Service ID (24 bit)

Payload VLAN ID (12 bit) SVLAN ID (12 bit) B VLAN ID (12 bit)

Source Address Source Address Source Address Source Address

Destination Address Destination Address Destination Address Destination Address

IEEE 802.1 IEEE 802.1Q IEEE 802.1ad IEEE 802.1aq

29 Comparison

TRILL MLAG SPB VPLS Standard Body IETF Vendor-specific IEEE IETF New (Variant of Technology New Matured Matured PBB) Yes Minimal Yes Yes B-VID needs to be No Configuration configured for each ECMP

Yes Yes Yes 16 active links with Yes 16 active links with ECMP 16 ECMP LSPs true hop-by-hop 2 active links ingress ECMP can be achieved ECMP decisions decisions Loop Yes Yes Yes Yes Prevention TTL and RPC RPC only Virtualization Higher scale Higher scale 4K networks 4K networks Scale with mac-in-mac with VPN ID