CarrierCarrier . EthernetEthernet Raj Jain Washington University in Saint Louis Saint Louis, MO 63130 [email protected] These slides and audio/video recordings of this class lecture are at: http://www.cse.wustl.edu/~jain/cse570-13/ Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-1 OverviewOverview 1. Technologies for Data Center Interconnection: PDH, SDH, MPLS, PWE3 2. Metro Ethernet Services: E-Line, E-LAN, E-Tree, E-Access 3. Administration and Management Ethernet Services 4. Extensions to Ethernet for carriers: PB (Q-in-Q), PBB (MAC-in-MAC), PBB-TE Note: Although these technologies were originally developed for carriers, they are now used inside multi-tenant data centers (clouds) Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-2 OptionsOptions toto ConnectConnect TwoTwo DataData Centers?Centers? Danforth Campus Medical Campus 1. Dedicated Optical fiber (leased from the phone company) 2. Ethernet over Optical Transport Network (all-Optical Switches) 3. Ethernet over Wavelength Division Multiplexing (DWDM) 4. Ethernet over Synchronous Digital Hierarchy (SDH) 5. Ethernet over Plesiochronous Hierarchy (PDH) 6. Ethernet over Pseudo-wire over MPLS 7. Ethernet over Micro-wave 8. Single Pair High-Speed Digital Subscriber Line (SHDSL) 9. Ethernet with enhancements Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-3 PlesiochronousPlesiochronous DigitalDigital HierarchyHierarchy (PDH)(PDH) Plesios + Synchronous = Near synchronous Phone Line = 64 kbps = 1 User channel North America T1 = 1.544 Mbps = 24 User channels T1 T2 = 6.312 Mbps = 96 Channels T3 = 44.736 Mbps = 480 Channels Europe: E1 = 2.048 Mbps = 32 Channels E2 = 8.448 Mbps = 128 Channels E3 = 139.264 Mbps = 2048 Channels Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-4 SONET/SDHSONET/SDH E S S E Ethernet S S SDH SONET=Synchronous optical network Standard for digital optical transmission Standardized by ANSI and then by ITU Synchronous Digital Hierarchy (SDH) Protection: Allows redundant Line or paths Fast Restoration: 50ms using rings Sophisticated management Ideal for Voice: No queues. Guaranteed delay Fixed Payload Rates: OC1=51.84 Mbps, OC3=155M, OC12=622M, OC48=2.4G, OC192=9.5G Rates do not match data rates of 10M, 100M, 1G, 10G Static rates not suitable for bursty traffic One Payload per Stream High Cost Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-5 MultiprotocolMultiprotocol LabelLabel SwitchingSwitching (MPLS)(MPLS) PBXPBX PBXPBX 1 3 5 2 3 Allows virtual circuits in IP Networks (May 1996) Each packet has a virtual circuit number called ‘label’ Label determines the packet’s queuing and forwarding Circuits are called Label Switched Paths (LSPs) LSP’s have to be set up before use Allows traffic engineering Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-6 LabelLabel SwitchingSwitching ExampleExample Ethernet Header IP Header Payload Ethernet Header Label IP Header Payload 64 3 5 <64> A R1 <3> <5> R3 C B R2 <2> <5> <3> 5 3 2 Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-7 IPIP overover MPLSMPLS overover EthernetEthernet PE= Provider Edge CE PE Carrier Network PE CE CE = Customer Edge Dest. Adr Src. Type MPLS Customer Customer L3 CRC Next Hop Adr /Len Tag Dest. MAC Adr Src. MAC Adr PDU 48b 48b 16b 32b 48b 48b 32b Label Experimental Stack Position Time CoS/Drop-Preced. 1 Bottom to Live 20b 3b 1b 8b Allows 220 Label switched paths (LSP) Each path can have reserved capacity MPLS became a very popular Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-8 MartiniMartini DraftDraft 1995-1999: IP over ATM, IP over Ethernet L3 IP L2 Ethernet ATM PPP 2000+: ATM over IP, Ethernet over IP SONET over IP L2 Ethernet ATM PPP L3 IP Idea proposed by Luca Martini of Level 3 Communications and then Cisco Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-9 PseudoPseudo Wire:Wire: L2L2 CircuitsCircuits overover IPIP CE1 PE1 PE2 CE2 Pseudo Wire Emulated Service ATM ATM Network Network A B Frame Relay Frame Relay Network Network IP Network Ethernet Ethernet Tunnel Hdr Demux Field [Control] ATM ATM ATM ATM Payload Type How to de-assemble payload MPLS/GRE/L2TP - How to get to egress Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-10 EthernetEthernet overover PWE3PWE3 overover MPLSMPLS MPLS Label PW Label [Control] Ethernet Frame w/o FCS PID Flags FRG Length Sequence # PW1 4b 4b 2b 6b 16b MPLS PW2 Pseudo-Wire Emulation Edge-to-Edge (PWE3) LSP Multiple pseudo-wires per MPLS LSP PW3 Payload ID (PID): 5=Untagged Ethernet, 4=VLAN tagged, … 4 VLAN tag put by carrier and customers may or may not be relevent for forwarding. Determined administratively by PE. Flags: Payload specific. FRG: Used for fragmentation Pause frames are obeyed locally. Not transported. May put 802.1p priority in exp field of MPLS label Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-11 VirtualVirtual PrivatePrivate LANLAN ServiceService (VPLS)(VPLS) Allows multi-point Ethernet services over MPLS networks using pseudo-wires Non-Hierarchical VPLS: Edge routers do complete routing table, label distribution, multicast/broadcast replication Suitable for small deployments Hierarchical VPLS (H-VPLS): Edge routers communicate with only other edge devices and do not have large routing tables Suitable for large deployments A CE PE PE CE B CE: Customer Edge B CE PE PE CE A PE: Provider Edge A CE PE Service PE CE B Provider Ref: H. Saboowala, M. Abid, S. Modali, "Designing Networks and Services for the Cloud: Delivering business-grade cloud applications and services," Cisco Press 2013, ISBN:1587142945 Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-12 DifferentiatedDifferentiated ServicesServices A way for IP routers to provide QoS Expedited Forwarding (EF): Also known as Premium Service Virtual leased line Guaranteed minimum service rate Policed: Arrival rate < Minimum Service Rate Not affected by other forwarding classes Assured Forwarding (AF): Four Classes: No particular ordering Three drop preference per class: Low, Medium, High Best Effort Service Differentiated Service Code Point (6 bits) encode the service, E.g., 101110 = EF Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-13 CarriersCarriers vs.vs. EnterpriseEnterprise We need to exchange topology for Sorry, We can’t tell you optimal routing. anything about our internal network. Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-14 Issue:Issue: UNIUNI vsvs PeerPeer--toto--PeerPeer SignalingSignaling Two Business Models: Carrier: Overlay or cloud Network is a black-box User-to-network interface (UNI) to create/destroy light paths (in OIF) Enterprise: Peer-to-Peer Complete exchange of information UNI UNI Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-15 UNIUNI vs.vs. ENNIENNI User to Network Interface (UNI): Separates responsibilities between the user and the provider. (Troubleshooting, failures etc). Like the wired phone box outside your home. Only one customer’s traffic. External Network to Network Interface (ENNI): Separates responsibilities between two providers. Many customer’s traffic passes through an ENNI Tier 2 operators sell services to Tier 3 service providers. Customer UNI Provider 1 ENNI Provider 2 Tier 3 Tier 2 Ref: Fujitsu, “Carrier Ethernet Essentials,” http://www.fujitsu.com/downloads/TEL/fnc/whitepapers/CarrierEthernetEssentials.pdf Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-16 OperatorOperator VirtualVirtual ConnectionConnection (OVC)(OVC) Between UNI and ENNI or between two ENNIs. For wholesale service providers Two types: Point-to-Point and Multipoint-to-Multipoint Untagged or single tagged frames at NNI. Q-in-Q at ENNI UNIs may be 10 to 100 Mbps. ENNIs at 1 to 10 Gbps. UNI UNI ENNI ENNI UNI ENNI ENNI UNI Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-17 MetroMetro AccessAccess EthernetEthernet PrivatePrivate LineLine Access Ethernet Private Line (Access-EPL): Port-based service for Internet access Like the service at your home. Ends at your access provider, where many other Access- EPLs may end Access provider has only one interface Shared by many Access-EPLs Different from p2p EPL. Customer 1 ENNI Customer n E-Access Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-18 EndEnd--toto--EndEnd MetroMetro EthernetEthernet ConnectionConnection An EC may go through multiple service providers Multiple OVCs can be concatenated to create an EC ENNI ENNI Wholesale Wholesale Retail Access Provider Transport Provider Service Provider UNI OVC1 OVC2 OVC3 UNI Subscriber Subscriber EC Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-13/ ©2013 Raj Jain 7-19 EthernetEthernet VirtualVirtual ConnectionsConnections (EVCs)(EVCs) Port-based ECs: Forwarding not based on VLANs. Frames delivered to remote UNI/ENNI for P2P or Based on destination address for P2MP VLAN-based ECs: Forwarding based on VLAN tag. Multiple Virtual UNIs Ethernet Virtual Connection (EVC) More cost-effective for Enterprise customers EVC Types of EVCs: UNI EVC 1. Ethernet Virtual Private Line (EVPL) 2. Ethernet Virtual Private Tree (EVP-Tree) EVC 3. Ethernet Virtual Private LAN (EVPLAN) 4. Access Ethernet Virtual Private Line (Access EVPL) Note: Service providers always share an ENNI for multiple connections OVCs are always virtual No OCs Washington University in St.