Ethernet at New Speeds, Deterministic Networking, and Power Over Everything!

Ethernet Evolving: Ethernet at New Speeds, Deterministic Networking, and Power over Everything!

Dave Zacks Distinguished System Engineer @DaveZacks

Peter Jones Principal Engineer @petergjones

#NBASET BRKCRS-3900 #HighBitrate Ethernet Evolving: Ethernet at New Speeds, Deterministic Networking, and Power over Everything! BRKCRS-3900 – Session Overview We have a great session for you today … so sit back, buckle in, and let’s get started! Come to this session to get geared up on the latest developments in Ethernet for Campus and Branch networks.

The Ethernet we all know and love is evolving at a rapid pace. With the release of MultiGigabit Ethernet on Cisco's latest Catalyst access switching platforms, you can connect new devices that support MultiGigabit connections at 2.5Gb/s and 5Gb/s over Cat5e/6 copper. No longer are 1Gbps and 10Gbps the only connectivity options – intermediate speeds are now possible. Cisco’s Multigigabit Technology is a key enabler for upcoming 802.11ac Wave 2 wireless deployments, where the Ethernet bandwidth required per AP exceeds 1Gb/s. Multigigabit switches also support other high-bandwidth devices in the Ethernet access layer without requiring expensive and time-consuming re-cabling for 10GBASE-T. We will talk about how Cisco MultiGigabit switches work, and how they will change your network. As well, we will discuss work being done in (and out of) the standards bodies to broaden the footprint for Ethernet, and enable the Internet of Everything. This includes additional Ethernet speeds, capabilities for deterministic networking functionality over an Ethernet infrastructure (precision timing, pre-emption, time scheduling, etc), and of course we need to provide more Power

over Ethernet to more devices, in more places, than ever. Come to the session to get the latest scoop!3 Ethernet Evolving: Ethernet at New Speeds, Deterministic Networking, and Power over Everything! Your Instructors Today … Peter Jones and Dave Zacks

•

Peter is a Software Principal Engineer, and has been with Cisco for 10 years. Peter works on system architecture (ASIC, hardware & software) for Cisco Campus switching, as well as standardization of 2.5G / 5G BASE-T Ethernet as NBASE-T Alliance chair and in IEEE 802.3bz.

Dave is a Distinguished System Engineer, and has been with Cisco for 16 years. Dave works primarily with large, high-performance Enterprise network architectures, designs, and systems. Dave has over 20 years of experience with designing, implementing, and supporting solutions with many diverse network technologies.

We both have a strong interest in the evolution of Ethernet – a passion we hope to share with you via this presentation!  Peter Jones Principal Eng.

Dave Zacks4 Dist. Sys. Eng. Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

The Evolution of Ethernet

Cisco MultiGigabit – Platforms and Applications Dave Cisco MultiGigabit – Detailed Review

Cisco MultiGigabit – Interoperability Peter Power over Ethernet – Advances

Other Areas of Evolving Ethernet Dave Deterministic Ethernet – Overview and Applications

Wrap-up – Final Thoughts Peter

The Evolution of Ethernet Cisco MultiGigabit – Platforms and Applications Cisco MultiGigabit – Detailed Review Cisco MultiGigabit – Interoperability Power over Ethernet – Advances Other Areas of Evolving Ethernet Deterministic Ethernet – Overview and Applications Wrap-up – Final Thoughts Dave

BRKCRS-3900 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 Ethernet Evolution Ethernet has a long history since it was invented in 1973 … Past and Future • Robert Metcalfe – May 22nd, 1973 – first memo describing Ethernet

LAN standards work undertaken in 1980 by IEEE, as Project 802 Originally based on CSMA/CD, Ethernet has evolved over many media types, over many years, and many speeds … and it continues to evolve … 7 "Biblioteca Popular del Paraná - Interior 04" by Mauriciocaminos - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Biblioteca_Popular_del_Paran%C3%A1_- _Interior_04.jpg#/media/File:Biblioteca_Popular_del_Paran%C3%A1_-_Interior_04.jpg Ethernet Evolution Past and Future

Existing standards … Evolving new standards … Further standards under development …

available 8 http://www.ethernetal liance.org/roadmap/ … and possible futures … Ethernet Evolution Ethernet has over time not only seen an evolution Media – Serial to Parallel in speeds, but in feeds (media types and support) as well …

9 Ethernet Evolution Ethernet has over time not only seen an evolution Media – SMF in speeds, but in feeds (media types and support) as well …

10 Ethernet Evolution Ethernet has over time not only seen an evolution Media – MMF in speeds, but in feeds (media types and support) as well …

11 Ethernet Evolution Ethernet has over time not only seen an evolution Media – Twinax in speeds, but in feeds (media types and support) as well …

12 Ethernet Evolution Ethernet has over time not only seen an evolution Media – Twisted Pair in speeds, but in feeds (media types and support) as well …

MultiGigabit! Over Cat 5e/6 cabling

13 • What uses BASE-T? Twisted Pair (aka BASE-T) Desktop computers, servers, Market Usage and Success IP Phones, Wireless Access Points Consumer electronics • How much has been sold? • More than 4 Billion 100BASE-T and 1000BASE-T switch ports shipped in the last 20 years (similar # for end points) • BASE-T port total is approaching 1B/year

• Why choose BASE-T? • Ease of use and backward compatibility • Structured cabling • Incremental speed upgrades • Multi-vendor interoperability • PoE for power and data on single cable. • Price / performance

14 Wireless Standards

Early 2000 C L I E N T S / B ANDWIDTH 11 Mbps 802.11a, 802.11b 11Mbps 2002 Have Nice to 54 Mbps 54 802.11g 2004 – 2006 Past, Present, and Future Pervasive 450 450 Mbps 2008 802.11n 2010 Applications Rich Media 2012 802.11ac 1 Gbps Wave1 2014 Critical Mission 6.8 Gbps 6.8 Gbps 3.5 802.11ac Wave2 2015 10 Gbps 10 Future 15 Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

Maintain Switch to AP Reach at Higher Speeds Adaptive Rate Technology (FE, 1G, 2.5G, 5G, and 10G)  Future proofed for higher speeds

Infrastructure Investment Protection Supports up to 100m distance with Cat5e cabling up to 5G speeds for Brownfield Supports Cat6a cabling for Greenfield deployments for higher speeds

PoE / PoE+ / UPoE Cisco Innovation over 10GBASE-T Standard to support high end point power needs

Standards Compliant 1GBASE-T and 10GBASE-T IEEE standards, IEEE P802.3bz 2.5/5GBASE-T standard started 17 Cisco MultiGigabit Product Family – Overview

NBASE-T AllianceSM

4500E 3850 3560CX

• Best In Class Modular Access • Industry leading Fixed Access • NG Workspace switch • New 48 Ports Line Card • 24 & 48 Port Stackable Switches • MultiGigabit in smallest form factor • 12 Ports of Multigigabit per slot • 24 & 12 MultiGigabit Ports • PoE / PoE+ • Up to 96 MultiGigabit ports per system • New Uplinks • Instant Access support

18 Cisco MultiGigabit Platform Details – Catalyst 4500E

Mode mGig Lite Mode 1 48p 1GE UPOE (First 12p usable as mGig)

Mode mGig Enhanced Mode 2 12p mGig UPOE + 24p 1GE UPOE

Mode mGig Performance Mode 3 12p mGig UPOE C4500E Multigigabit Line card

Innovation with Investment Protection 19 Supported with Supervisor Engine 7 and 8 on all 4500E chassis Cisco MultiGigabit Platform Details – Catalyst 3850

48-port with 12 ports of mGig High Performance 24-port mGig

# of mGig Ports 12 mGig ports 24 mGig ports

Port Capabilities UPoE, EEE, MACsec UPoE, EEE, MACsec

New uplink New 2x40G and 8x10G New 2x40G and 8x10G Modules (existing modules are supported) (existing modules are supported)

Stackable with all other 3850 Switches 20 Catalyst 3850 – Under the Covers

UADP ASICs PoE+ Downlink Controllers (x2) Phys (x12)

Cavium CPU Ampere / Stack Power Controller

FRU Uplink Module

Ethernet And Power Stack Conn (x2) Console Port Redundant Power Supplies Fan FRU (x3) Back Stack Conn (x2) 21 Want to know Want to know more about more about Catalyst 3850 – 3850 / 3650? Review UADP? Attend BRKARC-3438, BRKARC--3467, Schematic Layout – 24-port MultiGigabit “Catalyst 3850 / 3650 “Cisco Enterprise Architecture”! Silicon”! 480G STACK INTERFACE UADP ASIC 500 MHz / 80Gbps

Packet Buffer Packet Buffer Packet Buffer Packet Buffer CPU

Packet Buffer Packet Buffer Packet Buffer Packet Buffer

Reassembly Reassembly Reassembly Reassembly FPGA Crypto Crypto Crypto Crypto

Ingress Egress Ingress Egress Ingress Egress Ingress Egress FIFO FIFO FIFO FIFO FIFO FIFO FIFO FIFO SDRAM

Network Interface Network Interface Network Interface Network Interface Flash

USB

10G Quad 10G Quad 10G Quad 10G Quad 10G Quad 10G Quad Dual PHY Dual PHY PHY PHY PHY PHY PHY PHY

12 Port UPoE 12 Port UPoE EMP Console

12 x 10G 100M/1G/2.5G/5G/10G 2x40G, 8x10G, 4x10G, 22 12 x 10G 100M/1G/2.5G/5G/10G 2x1G 2x10G, 4x1G Cisco MultiGigabit Platform Details – Catalyst 3560CX

6 x 1G/PoE+ 2 x mGig PoE+ 2 x 10G SFP+ MULTIPLE USE CASES

Maintain Switch Infrastructure PoE+ Standards mGig for 11ac mGig as Uplinks to AP Reach at Investment Cisco Innovation Over 10GT Compliant AP Deployments Connected to Higher Speeds Protection Standard to Support High 1G and 10G BaseT IEEE Access Switches End Point Power Needs Adaptive Rate Technology Supports 100m Distance with Standards (Cat 3K/4K) (FE, 1G, 2.5G, 5G, and Cat5e Cabling up to 5G 10G)  Future Proofed for Speeds Higher Speeds for Brownfield Supports Cat6a/6e Cabling for Greenfield Deployments 23 for Higher Speeds Cisco MultiGigabit New Announcing the Cisco 3800 – MultiGigabit-Enabled Access Point!

• Industry leading 4x4 MIMO:3 spatial streams (SS) Wave 2 802.11ac access points • Dual radio, 802.11ac Wave 2, 160 MHz • Combined Data Rate of 5.2Gbps • 2 x 5 GHz: 4x4: 3SS supporting - SU-MIMO / MU-MIMO - Flexible Radio Assignment: 2.4GHz, Dual-5GHz, Wireless Security Monitoring, or Wireless Service Assurance • Gigabit Ethernet and MultiGigabit Ethernet (1G, 2.5G, 5G) • HDX Technology • USB 2.0 • Internal and external antenna models Cisco Aironet® 3800 Series • Smart Antenna Connector - 2nd Antenna Connector

• Modularity: Side Mount Modular * Planning Gigabit Wi-Fi has fully arrived. 24 NBASE-T Use Cases Campus Connectivity

Campus Servers • Manufacturing Floors • Enterprise Workspace • Content Editing • CAD/CAM

High End Research Workstations • Universities • Research Labs

25 NBASE-T Use Cases Access Extension

Aggregation Connectivity • New Workspaces with older Cabling Catalyst 3850 2.5G/5G BASE-T

Compact Switch with NBASE-T Compact • Guest Cabins • Conference Rooms • Cruise ships • Retail Stores

26 NBASE-T Use Cases Enterprise, IoT, and Home

• Network Attached Storage • Signage • Service Provider gateways • Medical devices with PON / DOCSIS • Content editing • Small Cells • Carrier Ethernet (LAN side) • Security cameras • Desktop & Gaming PCs

27 Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

• Between 2003 to 2014 ~70 billion meters of Cat 5e and Cat 6 cabling were sold…. • ~90% of installed base • Enormous network infrastructure investment

• Existing specifications support 1Gb/s over this cable, but faster data rates are possible • BASE-T allows incremental upgrade

• Lets get more value from this investment! 10 meters of cabling • What can we enable? for every man, woman, and child on Earth …

29 Who Does What? For Your Ethernet Village – we need all of them Reference

IEEE 802.3 Ethernet Working Group (link) “develops standards for Ethernet networks”

Ethernet Alliance (link) “supports activities from Ethernet technology incubation to interoperability demonstrations, and education.”

IETF (link) “make the Internet work better.”

Optical Internetworking Forum (OIF link) Implementation Agreements (IAs) & Benchmarks, perform worldwide interoperability testing, build market awareness and promote education for technologies, services and solutions

TIA TR-42 (link) These groups define cabling standards and best practices. ISO/IEC JTC1/SC25 WG3 (link) NBASE-T Alliance (link) “NBASE-T enables data rates of 2.5 and 5 Gbit/s on 100m of cat5e/6 cable.”

System Vendors We build product (e.g. switches, routers, APs, etc) to sell. Customers You buy our product to build networks to …

30 2015 BASE-T Standards Roadmap – Review

Cat 6a

Cat 6

Cat 5e

31 The Application Spaces of BASE-T – Overview ENTERPRISE FLOOR Office space, for example DATA CENTER

Floor or

Room-based

Reach T

T Row-based

- -

(MoR/EoR)

10GBASE

1000BASE 2.5/5GBASE

5m 30m 100m 30m 5m Rack-based (ToR)

25GBASE 40GBASE

Data Rate

Source: George Zimmerman, CME Consulting 32 Enterprise Network Structure – Today

Core • Typically built using this three tier design • This design is proven, Distribution stable, and durable Access • Uses BASE-T copper links between Access switches AP AP AP and APs / End Devices APs and End Devices

33 802.3 Ethernet and 802.11 WLAN – Deployments Today and Tomorrow

1000BASE-T Power over Ethernet

Access Switch Cabling Access Point Mostly 1000BASE‐T ports Cat 5e/6/6A up to 100M Connects 802.11 to 802.3 PoE PSE (15/30/60W, 4PPoE) New installs using Cat 6A for 10+yr life PoE PD (Powered Device) Footprint sensitive (e.g. power, heat, etc.) 11acWave 2 drives backhaul traffic > 1 Gb/s. No easy way to get above 1Gb/s.

34 Enterprise Access Links – 1% Today 8%

World wide BASE-T cabling is dominated by Cat 5e & Cat 6. Cat 5e 38% • Based on cable sales data, average drop lengths and replacement rates Cat 6 • Source: BSRIA December 14 Cat 6a Ethernet Switch ports by speed Dell’Oro Group Ethernet Switch 5-year Forecast Jan 2016 53% Cat 7

100 Gbps Installed Base Cable by Type 40 Gbps

10 Gbps Ports 1000 Mbps Enterprise Access port types are 100 Mbps dominated by 1000BASE-T.

More than 4 Billion 100BASE-T & 1000BASE-T switch ports shipped in 2000 2005 2010 2015 2020 last 20 years.

35 Enterprise Access Links – Installed Cable Plant over Time

1800 1600 2014 Cat 5e Outlets 539 M 1400 Cat 6 Outlets 743 M 1200 Cat 6a Outlets 111 M 1000 Cat 7 Outlets 10 M Total 1.4 B 800 600 400 200 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Cat 5e outlets Cat 6 outlets Cat 6a outlets Cat 7 outlets

Source: BSRIA December 14 36 Enterprise AP Radio Bandwidth – Evolution 10000

Period 802.11 Max Standard (Mb/s) 1000 1997-1998 11 2 Max 1999-2001 11b 11 High 2002-2006 11a/g 54 2007-2011 11n 600 Mid 100 2013-2015 11ac Wave1 3470

Low 2015-2017 11ac Wave2 6930 AP Radio Radio (Mb/s) rate AP

10 Graph data: Cisco 802.11ac White Paper (link) 2002+ 2007+ 2013+ 2015+ 802.11 is clearly outgrowing 1000BASE-T AP to switch bandwidth should be ~75% of radio bandwidth (avoids Ethernet as system bottleneck) APs upgraded faster than switches, switches faster than cabling 802.11ax (High Efficiency WLAN) is coming, targets 4x throughput per station.

37 What’s Happening with 802.11?

Enterprise AP Volume Split Source: Dell’Oro Group Wireless LAN 5-year Forecast Jan 2016 Rapid transition from 11n to 11ac

802.11ac Enterprise 802.11ac AP Transition APs 802.11n Source: Dell’Oro Group Wireless LAN 5-year Forecast Jan 2016

2013 2020

Wave 1 APs Wave 2 Wave 2 starts ramping in 2016

2013 2020

38 What Does This Mean for Ethernet? Enterprise Edge: Ethernet and 802.11 connections Source: Dell’Oro Group Enterprise Edge 5-year Forecast Jan 2015

In 2015 more users connecting via WiFi than Ethernet

Ethernet 100M/1000M to 2.5G/5G transition

Source: Dell’Oro Group Ethernet Switch 5-year Forecast Jan 2016 802.11 Active Connections Active

1999 2014 2019 5.0 Gbps

2.5 Gbps Ports 1000 Mbps 100 Mbps 2.5G / 5G BASE-T Ethernet is expected to be rapidly adopted

2014 2020

39 Technology

Patch Patch End Switch Stranded Panel Solid Panel Patch Device

• 90 meters of solid horizontal cabling

• Total of 10 meters of stranded patch cable

Panel

Panel Panel End Switch Device Stranded Solid Patch

• 2 or more intermediate connections (patch panels etc.) • 802.3 defines “Link Segment”, includes cable and all connectors

41 Cable, Channel & Link Segment – Victims, Disturbers, and Noise

• The 90 meter run is bundled “6 around 1” • The cable in the middle is the victim cable • It’s got noisy neighbors, 6 disturber cables • Noise types • Near end crosstalk (NEXT) • Far end crosstalk (FEXT) • Alien crosstalk (AXT) • Impulse noise (aka EMI)

Net/Net “100 Meters of Cat 5e/6/6a” is shorthand for “A channel of nnnMhz”

42 10GBASE-T – Technology Overview

1000BASE-T and 10GBASE-T already share many common attributes – • RJ45 connector • 4-pair Category cabling • Full duplex communication • Transmit & receive on same wires simultaneously Receive Echo NEXT FEXT ISI Error • Support for PoE, EEE, Autonegotiation… Signal Cancel Cancel Cancel Removal Correction 10GBASE-T technologies can be used to squeeze more usable bandwidth out of installed cabling Advanced coding and signal processing allow 2.5Gb/s and 5Gb/s operation of Cat5e cabling developed for 1Gb/s

43 BASE-T details – For Your 1000, 10G, 2.5G, 5G Reference 1000 BASE-T 10G BASE-T 2.5G BASE-T 5G BASE-T

Pairs 4 4 4 4

Coding Manchester / Manchester / Manchester / Manchester / PAM-5 PAM-16 /DSQ-128 PAM-16 PAM-16 FEC TCM LDPC LDPC LDPC

Bits per symbol 2 3.125 3.125 3.125

Nyquist Rate 62.5M 400M 100M 200M

Signaling rate 125 Mbaud 800 Mbaud 200 Mbaud 400 Mbaud

Data Rate 4p*4b*62.5M = 4p*6.25b *400M = 4p* 6.25b*200M = 4p* 6.25b*200M = 1000M 10G 5G 5G 44 There’s Lots of Room for BASE-T PHYs –

Between 1G and 10G

Cat Cat Spec 5e (100 MHz @100M) Cat Spec 6 (250 MHz @100M) Spec Cat 6A (500 MHz @100M) 6.25

2.5G BASE-T 5G BASE-T

4 10G BASE-T 1000

BASE-T Encoding: Encoding: bits/sec/Hz (per Pair) 62.5 100 200 400 Nyquist Frequency (MHz) for Ethernet Data Rates Source: George Zimmerman, CME Consulting 45 Status

• NBASE-T Alliance (www.nbaset.org/) • Vendor alliance for 2.5G/5G BASE-T with 40+ participant companies. • Who is in the Alliance? • Components, silicon, systems, cabling, test, etc • Alliance role? • Enable widespread deployment, evolve specifications, facilitate interoperability.

47 IEEE 802.3 Status – The Road to a Standard

• November 2014: “Call For Interest” (link) “Next Generation Enterprise Access BASE-T PHY” Study Group formed

• March 2015: IEEE 802.3bz 2.5G/5GBASE-T Task Force Approved (link) • Approved objectives, 2.5 Gb/s & 5 Gb/s over Cat 5e and Cat 6 cabling

• May 2015: First Task Force meeting (link) • Adopts technical baselines consistent with NBASE-T specifications • Press releases from IEEE SA and Ethernet Alliance, webinar from Ethernet Alliance

• July 2015 (link) - January 2016 (link): Task Force creates/reviews draft • January 2016, draft technically complete, Working Group ballot

• What happens next? • Meetings in March, May, July 2015.

• Working Group ballot, Sponsor Ballot, approval from RevCom, publication, DONE! 48 Legend IEEE 802 Plenary IEEE 802.3bz Status – IEEE 802.3 Interim The Way IEEE Shows It IEEE-SA Standards Board

Task Force Approved Standard

1st Study 1st Task Working Sponsor Sponsor Group Mtg Force Mtg Now Group Ballot Ballot Ballot Starts Starts complete Objectives Baselines Adopted Adopted

J F M A M J J A S O N D J F M A M J J A S O N D

2015 2016

D0.1 D1.0 D1.1 D1.2 D2.0 D2.1 D3.0 D3.1 D3.2

TF Reviews WG Ballots Sponsor Ballots

49 NBASE-T – What are the Analysts Saying? We saw the very first 2.5/5.0 GE port shipments begin to ramp at the end of June, and expect volumes to begin rising significantly in the third quarter of this year as multiple product lines begin to ramp. The benefit of increased throughput without replacing cable is creating an entirely new Ethernet market to serve the demands of the campus environment. This benefit could push port counts over 1 million in only a year. Chris DePuy, VP of Enterprise Edge Research, Dell’Oro Group http://www.nbaset.org/nbase-t-alliance-hosts-first-plugfest/

50 NBASE-T – What are the Analysts Saying? The transition to next generation 802.11ac Wave 2 access points will drive significant demand for 2.5G ports in the coming years as this technology will allow enterprises to cost-effectively deliver more bandwidth, while re-using parts of their existing infrastructure. 2.5G ports will be one of the bright spots in the switching market, growing at a 175 percent compound annual growth rate through CY19. Matthias Machowinski - Director, Enterprise Networks and Video, IHS http://www.nbaset.org/nbaset-alliance-celebrates-one-year-anniversary/

51 NBASE-T – What are the Analysts Saying? In the standards process, the .bz NBASE-T flavor won out in terms of modulation being used. NBASE-T is essentially becoming the IEEE standard. There is no real competing specification at this point for anything between 1 Gig and 10 Gig. Linley Group Principal Analyst Bob Wheeler http://www.eetimes.com/document.asp?doc_id=1328455

52 Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

Panel session: Technology & Use cases PNLCRS-1000 - Introducing NBase-T Alliance 54 NBASE-T Alliance at Interop 2015 Las Vegas –

• First alliance multivendor demonstration at Interop 2015 Las Vegas

• Participants included Aquantia, Cavium, Centec Networks, Cisco, CommScope, Fluke, Freescale, Intel, Marvell, Microsemi,Tehuti Networks, Xilinx

http://www.nbaset.org/nbase-t-alliance-demos-end-to-end-enterprise-infrastructure-solutions-at-interop/

55 Cisco Live San Diego 2015 – Cisco and Intel

https://twitter.com/petergjones/status/608434210521640960 https://twitter.com/petergjones/status/609101309883514880 56 NBASE-T Interoperability – First Industry Plugfest September 15 • Wide Industry Representation including Aquantia, Cisco, CommScope, Fluke Networks, Freescale, Intel, Microsemi, Panduit, Spirent and Tehuti Networks • Cross vendor testing • Testing 2.5G & 5G BASE-T • Linkup time, auto negotiation and bit error rate. • Cat5e, Cat6 at 40m, 65m and 100m • All tests passed • 2.5G/5G BASE-T Speeds – Link up Diagnostics • Cables – Cat5e, Cat6 at 40m, 65m and 100m See http://www.nbaset.org/smooth-sailing-at-first-nbase-t-alliance-plugfest/

57 NBASE-T Fundamentals – TechWise TV

http://www.cisco.com/c/m/en_us/training-events/events-webinars/techwise-tv/fundamentals-nbase.html

58 Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

Ethernet used to provide datagrams and collisions

In 2000 Cisco gave you the option of one cable for power and data

Cisco has led every PoE transition

60 UPoE / 4PPoE – Overview

Next standard (802.3bt 4PPoE) PoE PSE Power numbers targets 100W 100 at the PSE 80

60 More efficient to send power Cisco down 4 pairs for same load 40 IEEE 802.3 Can send 2 x Power 20 compared to 2 pair PoE / PoE+

0 Backward compatible 2000 2003 2007 2009 2011 2016 to PoE / PoE+

61 Power over Ethernet – Interesting Uses Today RBC WaterPark Place III in Toronto, Ontario is one of the most intelligent, connected, and secure buildings in the world.

62 Power over Ethernet – Interesting Uses Today

Combined achievement of Cisco, the IoE Innovation Centre Toronto & ecosystem partners

One network for lighting, HVAC, metering, security, blinds Cisco Integration Platform (CIP) orchestrates different systems for improved visualization, control, analytics and cost savings CIP allows for increased flexibility and personal control of tenant spaces

63 Disparate Networks – Converged by IP

Building systems Traditional Design – Smart Design – are networks Limited Capabilities New Capabilities

64 Building Features – Overview

1440 PoE LED light fixtures

140 new 8-port PoE+ ceiling switches

2300+ PoE HVAC controllers from Delta Controls

65 Building Features – Overview

Lights Powered by the IP Network

PoE and IP-Controlled HVAC

IP Sensors built into fixture 66 Building Features – PoE-based Infrastructure Makes It All Cisco 3560-CX Fanless PoE+ Switches in Ceiling-Mounted Rack Possible Building Automation From network closet To Light fixtures

Lighting

67 Further Evolution of PoE – PoDL – Power over Data Line PoDL is power for one-pair Ethernet (100/1000 BASE-T1)

Main goals – Use with BASE-T1 (one pair) media Can work just as power distribution Fast startup – BASE-T1 wants startup < 100ms Supports wide range of voltages and power levels

68 Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

We are not in Kansas any more

Standards active in – 100M, 1G, 2.5G, 5G 25G, 40G, 100G 400G FlexEthernet (N by M)

70 Ethernet Places –

Ethernet used to live in the office …

Active development for – Campus – 2/5G/5G BASE-T SP – 400G & FlexEthernet

DC – 25G DAC / Twinax

Home - GE POF Auto – 100 / 1000 BASE-T1

Factory – 1000BASE-T1 71 Ethernet Media –

Ethernet used to run over fat yellow coax …

Active development of – Category 8: 25/40G BASE-T for DC DAC/Twinax: 25G for DC

POF: 1G for Home/auto BASE-T1: Auto & Industrial 100BASE-T1 over 15 meters 1000BASE-T1 over 15 or 40 meters

72 Ethernet Services –

Ethernet used to provide datagrams and collisions …

Services on the way / improving – Preemption Precision Time Defined latency and jitter Seamless redundancy

Why? To serve new markets for Deterministic Networking.

73 Ethernet Activity – From IEEE Interim, Jan 2016, Atlanta GA

25GSMF (single lane) Study Group objectives 50G (single lane) Study Group objectives NG100G and 200G (Multi-Lane) Study Group objectives 802.3bq (25G / 40GBASE-T) going to Sponsor Ballot 802.3br (Preemption) going to Sponsor Ballot Next Generation Enterprise / Campus / Data Center (NG-ECDC) – possible new work? • YANG Models for IEEE 802.3 • 400 GbE over 40km • Single Twisted Pair - 1000m

74 IEEE 802.3 – Some New Work For Your Reference

• 25GbE SMF 25GbE can be natural successor to 10GbE, but is missing a story for longer than 100 meters (e.g. looking at enterprise and metro use cases). The 25GbE SMF Study Group was approved November 2015 (link).

• 50GbE 50 Gb/s is the next “natural” signaling rate that makes sense. Two Study Groups were approved in November 2015 (link) to look at using this signaling. They are investigating single lane 50GbE (1x50GB/s) and also 100GbE (2x50GB/s) / 200GbE (4x50GB/s)

• 2.5G/5G Backplane The 802.3cb 2.5 Gb/s and 5 Gb/s Backplane and Copper Cables Task Force was approved November 2015 (link). The initial use case is Ethernet Object Drives (HDDs) for “Cheap and Deep” storage.

75 Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

Support for the needs of critical data streams:

1. Time synchronization for network nodes and hosts to better than 1 µs.

2. Resource reservation for a flow (e.g., buffers and link bandwidth links).

3. Extremely low packet loss ratios, starting at 10–6 and extending to 10–10 or better

4. Guaranteed end-to-end latency for a flow.

The Goal? To move real-time digital data carried on dedicated wires and proprietary packet networks onto a Converged Enterprise network

78 Who needs Deterministic Networking?

• Two classes of demanding customers, Industrial and Audio/Video. Both have moved into the digital world, and some are using packets. They realize they must move to Ethernet, and most will move to IP.

1. Industrial: process control, machine control, and vehicles. • At Layer 2, this is IEEE 802.1 Time-Sensitive Networking (TSN). • Data rate per stream low, but can be large numbers of streams. • Latency and jitter critical to meeting control loop frequency requirements.

2. Audio / video: streams in live production studios. • At Layer 2, this is IEEE 802.1 Audio Video Bridging (AVB). • Not so many flows, but one flow is 3 Gb/s now, 12 Gb/s tomorrow. • Latency and jitter are important, as buffers are scarce at these speeds.

79 Why such a Low Packet Loss Ratio? Back-of-the-envelope calculations:

1. Industrial: • Automotive factory floor: • 1000 networks • 1000 packets/s/network • 100,000 s/day = 1011 packets/day. • Machine fails safe when 2 consecutive packets are lost. • At a random loss ratio of 10–5, 10–10 is chance of 2 consecutive losses. • 1011 packets/day • 10–10 2-loss ratio = 10 production line halts/day. • In extreme cases, lost packets can result in damage.

2. Audio video production: (not distribution) • 1010 b/s • 10 processing steps • 1000 s/show = 1014 bits = 1010 packets. • Waiting for ACKs and retries = too many buffers, too much latency. • Lost packets result in a flawed master recording, which is the user’s end product.

80 How such a Low Packet Loss Ratio?

1. Zero congestion loss Reserve resources along the path (e.g. IntServ, RSVP) with admission control. This requires hardware in the form of buffers, shapers, and schedulers. • Circuits only scale by aggregation in to larger circuits. ( MPLS? Others?) • Zero congestion loss goes hand-in-hand with finite guaranteed latency.

2. Seamless redundancy • 1+1 redundancy: Serialize packets, send on 2 (or more) fixed paths, then combine and delete extras. • Zero congestion loss means packet loss is failed equipment or transmission errors. • Some controlled networks only need zero congestion loss. In a converged network using seamless redundancy, congestion is likely on both paths.

81 IEEE 802 Standards – For Your Around Deterministic Ethernet Reference

• 802.1AS/AS-Rev Time: A plug-and-play Precision Time Protocol (PTP) profile that allows network devices distribute time in a physical network, regardless of L2 / L3 boundaries. (AS complete, AS-Rev in TG ballot)

• 802.1Qat /Qcc Reservation: MSRP protocol to reserve bandwidth along an L2 path determined by L2 topology protocol, e.g. ISIS. (Qat complete, Qcc TG ballot)

• 802.1Qca Path distribution: ISIS TLVs to compute and distribute multiple paths through a network. (Standard)

• 802.1CB Seamless Redundancy: 1+1 duplication for reliability. (WG ballot) 82 IEEE 802 Standards – continued For Your Around Deterministic Ethernet Reference

• 802.1Qat AVB Credit-Based Shaper: Similar to the typical run rate/burst rate shaper, but with really useful mathematical properties. (Standard)

• 802.3br/802.1Qbu Transmission preemption: Interrupt (1 level only) transmission of an Ethernet frame with a frame with tight latency requirements, then resume the interrupted frame. (3br starting Sponsor, 1Qbu complete)

• 802.1Qbv Time scheduled queues: Every bridge port runs a synchronized, repeating schedule that turns on and off each of the 8 queues with up to nanosecond precision. (Revcom)

• 802.1Qci Per-Stream Filtering and Policing: Packets accepted only from the right port only at the right time or at the right rate. (TG Ballot)

• 802.1CM Time-Sensitive Networking for Fronthaul: Profiles of Deterministic Ethernet to build networks for mobile fronthaul (TG Draft) 83 Haven’t We Been Here Before? Ethernet won against Token Bus (802.4), Token Ring (802.5), DQDB (802.6), ATM, RPR (802.17), … Isn’t this the same thing? NO! “Standard” networking didn’t address needs for industrial control, vehicle control, video studios, …. Other network use cases have adapted to delay/loss (e.g. Voice over IP), the Deterministic Networking target market has not. Just moving to faster links doesn’t solve the problem. • How can we serve this market?

84 Thinking about the network

Physical Controller connectivity

Network sizes vary from ~home to ~large but within one administrative domain. Listener La L2 L2 Lc T Talker MultiLink subnet L2 L3 Ld Routers Lb Bridges Lots of complex protocols 86 Reference Network – Functional View Physical X connectivity Queue

Clock Listener La

Lc T Talker

Ld Lb

Just network devices, queues, clocks, and links!!

87 IEEE / IETF Cooperation – Needed for Mixed L2 / L3 Networks

• Networks include both bridges (L2) and routers (L3) • Every node along a path must participate in the reservation protocols and reserve resources. • Reservations could come from pre-configuration, management, or protocol. • There are valid use cases for application-driven peer-to-peer control flow models for centrally controlled models, and for mixed scenarios. • End devices (and applications) can participate in the protocols. • Hosts and operations managers don’t know or care whether network is bridged or routed. • One UNI, one operator view. To succeed, we need both groups to co-operate. In IEEE, this is 802.1 (link) In IETF, this is DetNet (link) 88 What’s a Cycle? Definition

• Many deterministic systems / technologies (e.g., EtherCAT, PROFINET, TTEthernet) support the concept of a cycle.

• A cycle is a fixed, repeating period of time that defines when different types and streams of traffic are sent.

• The cycle is designed to support the requirements of the control loops running on the network.

• Cycle time varies widely depending on the type and length of the control loop.

• e.g., EtherCAT targets ≤ 100 µs, PROFINET is <1ms to ~100ms depending on use case

• Traffic is scheduled within a cycle, and all sources/sinks are locked to the same clock.

• The Cycle is a key concept to help enable determinism in a system.

Cycle0 start Cycle1 start Cycle2 start Cycle3 start Increasing time 89 Primitives: Queueing & Scheduling

• IEEE 802.1 does architecture (queueing, internetworking, security, etc) for all the IEEE 802 media (.3, .11, .15, etc) • 802.1 standards define an integrated set of queuing and scheduling capabilities. The “integrated” part is important – Interactions among these capabilities are well-characterized and mathematically sound.

Lets look at these building blocks

91 Priority Queuing and Weighted Queuing – Overview

• Priority queueing (including weighted round robin) from 802.1Q-1998

1 0 2 3 4 5 6 7 Priority selection

• Weighted queues from 802.1Q-2012 (802.1Qaz)

Weighted 1 0 2 3 4 5 6 7 Priority selection

92 AVB (Audio Video Bridging) Shapers – Overview

• Shapers from 802.1Q-2012 (802.1Qat). • Shaped queues have higher priority than unshaped queues.

Weighted  Highest priority for shaped queues 1 0 4 5 6 7 2 3 Priority selection

• The AVB shaper is similar to the typical run rate / burst rate shaper, but with really useful mathematical properties. • Only parameter is bandwidth. • Impact of any number of adjacent shapers on other queues is the same as the impact of one shaper with the same total bandwidth.

• Time-gated queues from 802.1Qbv. A circular schedule of {time, bit-mask} tuples controls queue transmission eligibility.

Weighted 1 0 4 5 6 7 4 5

T T T T T T T T Operated by a repeating schedule Gates eligibility of queue to be selected Priority selection "AdlerClock (1)" by Livinghistorynola - Own work. Licensed under CC BY-SA 4.0 via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:AdlerClock _(1).jpg#/media/File:AdlerClock_(1).jpg

94 Cyclical Queuing and Forwarding – Overview

• Coordinating input and output traffic handling within network devices enables better traffic control with less buffering in the network. • Use time gated queues to create double or triple buffers (two pairs, 2–3 and 4–5, shown in this example)

Shapers ensure fair access for 0, 1, 6, 7 traffic 1 0 6 7 2 3 4 5 T T T T T T T T Alternate green and purple Priority selection • If per-hop latency is low (compared to cycle time), double buffers (see below) are sufficient. If not, triple buffers are required to avoid output starvation.

Dead-time pad  Frames being received

For next cycle   Output in progress 95 Per-Stream Input Filtering and Policing – Overview

Applies to frames G G G G G G G G G G G coming up the stack, not down. Priority + circuit_identifier demux

• Priority and a flow ID select an input Gate (similar to a policer)

• Each Gate has some of the following: 0 0 1 0 0 1 1. A pass / drop switch. (May be time scheduled) P 2. A standard policing function. 3. Frame Counters. 0 0 1 4. An output class. 5. Filters, e.g. max frame size. 0 0 1

G 96 802.3br / 802.1Qbu Preemption – Overview Scheduled rocks of critical packets in each cycle:

Conflict excessively with not-guaranteed packet rocks: … …

Preemption lets you create sand between the rocks.

e e … …

97 What Does This Provide?

• Guaranteed zero congestion loss: Provides finite latency for reserved flows.

• Cut-through forwarding: These scheduling tools enable guarantees for predictable cut-through forwarding for ultra-low-latency packets.

• Intentional buffering delays: Intentionally delaying transmissions to minimize jitter for the critical traffic (helps many control loop systems) .

Enabling convergence of critical data streams onto a single shared network

98 Ethernet Evolving – Ethernet at New Speeds, Deterministic Agenda Networking, and Power over Everything - BRKCRS-3900

100 Ethernet Dominates Wired Networking – Used Worldwide, in Many Different Places and Applications

1% 8% Ethernet Switch ports by speed Dell’Oro Group Ethernet Switch 5-year Forecast Jan 2016 Cat 5e 38% Cat 6 Cat 6a 100 Gbps 53% Cat 7 40 Gbps

10 Gbps Ports 1000 Mbps

100 Mbps Source: http://www.ieee802.org/3/NGEBASET/public/jan15/jones_ngeabt_04c_0115.pdf

2000 2005 2010 2015 2020

101 Ethernet Innovations Continue – At Both High Speed and Low Speed

102 BRKCRS-3900 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 102 Ethernet is the foundational Ethernet Innovations Continue – network technology for the world! Spreading to New Markets, and Adding New Services And it will continue to evolve …

e c

p e

z z z

) ) )

H H H

M M M

e A

M M M

0 0 0 Weighted

5 6 6

0 0 0

t t t

1 1 1

0 5 0

a a a

1 2 5 1 0 4 5 6 7 4 5

C ( @ C ( @ C ( @

)

r i 6.25 a T T T T T T T T

r 2.5G BASE-T 5G BASE-T e Priority selection

( 802.1Qat / Qcc

z “Reservation”

H 802.1AS / ASbt / 4

e “Precision Time”

/ 10G BASE-T

t i 802.1QCB

1000

g BASE-T 802.1Qca “Redundancy”

d “Path Distribution”

E 62.5 100 200 400 Nyquist Frequency (MHz) for Ethernet Data Rates

PoE PSE Power numbers 100

60 Cisco 40 IEEE 802.3 20

103 0 BRKCRS-3900 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 103 2000 2003 2007 2009 2011 2016 Ethernet Evolving: Ethernet at New Speeds, Deterministic Networking, and Power over Everything! Did We Achieve Our Objectives?

Do You Have a Better Understanding …

of how Ethernet is evolving …

of new Ethernet capabilities …

and how Ethernet will continue to evolve and grow?

Don’t Forget to fill out your evaluations!

104 Continue Your Education Continue Your Education TECCRS-2678 Converged Access - Wired-Wireless System Architecture, Design and Operations Monday – 9:00 a.m 8 hours TFDX Multigigabit, NBASE-T & IEEE 802.3 – Feb 16 Monday – 8:00am 1 hour LTRRST-2006 Next Generation Workspace - Lab Tuesday – 9:30 a.m. 4 hours BRKARC-3438 Cisco Catalyst 3850 and 3650 Series Switching Architecture Tuesday – 2:15 p.m 2 hours BRKCRS-2501 Campus QoS Design-Simplified Tuesday – 2:15 p.m 2 hours BRKIOT-2113 Intermediate - Internet of Things for the Enterprise Tuesday – 11:15 a.m. 90 minutes BRKCRS-2888 Advanced Enterprise Campus Design: Converged Access Wednesday – 9:00 a.m 2 hours PSOEWN-1004 Cisco Unified Access: Enabling Simple, Smart, Secure Access for the Digital Enterprise Wednesday – 1:15 p.m. 1 hour BRKARC-3467 Cisco Enterprise ASICs - Delivering Innovation for Advanced Routing and Switching Wednesday – 2:30 p.m. 90 minutes BRKCRS-3438 High Availability in the Access Same speakers! Thursday – 9:00 a.m. 2 hours BRKCRS-3146 Troubleshooting Cisco Catalyst 3650 / 3850 Series Switches Thursday – 9:00 a.m 2 hours

Additional sessions that may interest you!

105 Call to Action

• Visit the World of Solutions for • Cisco Campus • Walk in Labs • Technical Solution Clinics

• Meet the Engineer

• Lunch and Learn Topics

• DevNet zone related sessions

106 Complete Your Online Session Evaluation

• Please complete your online session evaluations after each session. Complete 4 session evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt.

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

107 Thank you

108