High Speed Optics – the Road to 400G and Beyond
Total Page:16
File Type:pdf, Size:1020Kb
#CLUS High Speed Optics – The road to 400G and Beyond Errol Roberts – Distinguished Systems Engineer Mark Nowell – Cisco Fellow, Engineering BRKOPT-2005 #CLUS Agenda • Optics Landscape • Optics and Optical technology • Standards and Solutions – 400G • Future developments on 400G and beyond • Conclusion #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Cisco Webex Teams Questions? Use Cisco Webex Teams to chat with the speaker after the session How 1 Find this session in the Cisco Live Mobile App 2 Click “Join the Discussion” 3 Install Webex Teams or go directly to the team space 4 Enter messages/questions in the team space Webex Teams will be moderated cs.co/ciscolivebot#BRKOPT-2005 by the speaker until June 16, 2019. #CLUS © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 Optics Landscape Setting the stage...... Optics deployment context Inter-DC/WAN/Metro/Long Haul Intra-DC / CO Transport Network IEEE carrying Ethernet Traffic defined Regen Ethernet OA TxP Mux/Demux ROADM DC & Client Optics Line/Transport Optics • Fiber is plentiful • Fiber is scarce • Distances are short (<10km) • Distances are long (>80km) • “Direct Detect” technology is • Coherent (DWDM) technology required sufficient (for now) • Figure of Merit: $$$/Gb-Km • Figure of Merit: $$$/Gb (cost) #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 Ethernet Roadmap 6 new speeds in 1st 35 years of Ethernet 6 new speeds in a 2 year span • 2.5 GbE – 2016 • 5 GbE – 2016 • 25 GbE – 2016 • 50 GbE – late 2018 • 200 GbE – 2017 • 400 GbE – 2017 • Proliferation of MSAs/Consortia • QSFP-DD, OSFP • 100G Lambda • COBO https://ethernetalliance.org/the-2018-ethernet-roadmap/ #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 Development and deployment at Ethernet Alliance Roadmap points all along the Ethernet value chain have exploded!! #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Ethernet Switch Port Speed Transitions Millions 70 400 GbE • 100-200 GbE 40 GbE 50 GbE 10 GbE 25 GbE • 1 GbE 0 Source: #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 400 GbE Optical Module market projections Year 5 of 400 GbE is Normalized Optical Module Shipping approx. 20x larger than 20 Volumes Year 5 of 100 GbE 15 Driven by multiple markets Total 100 GbE looking to adopt at same 10 Total 400 GbE time. 5 This provides an exciting challenge for equipment - and component 1 2 3 4 5 Years after initial adoption manufacturers Courtesy: LightCounting Caveat when assessing market analyst data: A 400 GbE module might be used for dense 100 GbE #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Importance of Interconnect Data Center to User 14.9% Interconnect is #2 HW cost Data Center Within category in Cloud DC (behind to Data Center servers) Data Center 71.5% 13.6% 20.6 ZB Cost of interconnect can in 2021 accelerate or delay upgrade Some of East- transition West Traffic interconnects buildings (DCI) We need to focus on lowering Total East-West Traffic Will Be 85% optics cost to accelerate transition (Rack-local traffic would add another slice to 400 GbE twice the size of “Within Data Center”) Source: Cisco Global Cloud Index, 2016–2021 #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 Data Center Fabric • High bandwidth Fabric • Hash Efficiency Improvements w/high • Copper Cables • Redundancy Model speed links • Multimode • Highly Scalable • 400G deployments on horizon • Single-mode • Distributed Data Center • DWDM (DCI) Multiple Spines 100G 400G 100G 400G Wider Spine 25G 50G 100G Higher Scale Compute #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 Ethernet Speed Migration: Uplinks / Core 2017 2018 2019/2020 40 GE 100 GE 100 GE 100 GE 200/400 GE 40 GE 100 GE 40 GE 100 GE 100 GE (40 GE optics for cost 40 GE 100 GE 40 GE 100 GE 100 GE/400 GE (40 GE optics for cost) 40 GE 40 GE 100 GE 100 GE 40 GE optics for cost 100 GE 100 GE 400 GE Source: Dell’Oro Jan 2018 #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Data Center Trends • Interconnect costs affect DC physical architecture A distributed data center • HyperScale DC focused dominantly on Single-mode long haul fiber (duplex and parallel). • Presence of Multimode fiber requires consideration of optics to enable forward and backward connectivity as building building migration from 10GbE 40/50 GbE 100 GbE 400 GbE DCI • DC’s expanding beyond single physical building ≤100 km • Future capacity, fiber, reach, optics cost, system architecture are all considerations building building building building #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 Network Trends • Services, content and apps are moving to An Access Network the edge Metro/ long haul • Traffic is growing faster at the Metro • Higher capacity systems are being deployed Co-lo DCI • High speed optics are being adopted in client form factors • Networks are being simplified, optimized and automated Router / Switch #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 Optical Transport Coherent market trends DCI drove edge growth in 2017 • Next wave coming in 2020 Right • 100G/400G “ZR” pluggable axis market Lessons from 10G Migration • 10G Long Haul migration to Left axis Coherent complete • 45% of all 10G links are <80km. (ZR) Cignal AI: 4Q18 Optical Applications Report • The 10G-ZR application will migrate to 100G coherent at 100G and above, in 100G equivalents the right price Edge: < 120km Metro: < 600km LH: >600km Courtesy of Andrew Schmitt, Cignal AI #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 ASIC Trends: Capacity increases continue • Historical curve fit to highest rate switch products introduced to Total Switch market (blue squares) IO BW • Single ASIC IO capacity doubling every ~ 2 years • IO speed has to increase due to package limitations High Speed IO Electrical Ratification #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 Switching ASIC IO Escape: forcing transition to higher lane speeds Existing Switch Devices • ~ 70mm package is a current BGA practical maximum (due to coplanarity / warpage) • BGA devices with > 14 Tb/s of aggregate bandwidth are forced to transition to lane rates beyond 50 Gb/s #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 Ethernet’s consistent trend: Narrower/Faster and re-use Higher speed Fewer lanes Fewer 16x 10x10 8x # lanes 4x 2x 1x1 1 2.5 1010 25 50 100100 Gb/s Gb/s Gb/s Gb/s Gb/s * At the right time IO (SERDES) speeds #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 Optics and Optical Technologies Setting the stage...... Optics deployment context Inter-DC/WAN/Metro/Long Haul Intra-DC / CO Transport Network IEEE carrying Ethernet Traffic defined Regen Ethernet OA TxP Mux/Demux ROADM DC & Client Optics Line/Transport Optics • Fiber is plentiful • Fiber is scarce • Distances are short (<10km) • Distances are long (>80km) • “Direct Detect” technology is • Coherent (DWDM) technology required sufficient (for now) • Figure of Merit: $$$/Gb-Km • Figure of Merit: $$$/Gb (cost) #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 Optical technology… …is the key enabler for systems to maintain pace with the requirements of the bandwidth pressures Challenge Goal Form Factor Size Maintain/Increase Port Density Power reduction Maintain/Increase Port Density Reach System/network scaling Packaging simplification Increased Yield, lowers cost The next higher speed System Scale, lower costs #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 Anatomy of a module Host-Module High Speed optical SERDES Optimum when n = m side SERDES interface interfaces TIA PHY/CDR m n n:m Gearbox LD • Ethernet’s architecture allows these sets of interfaces to evolve independently. • Enables continual optimization of cost to occur. #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 Client Optics Goals – how do get there? • Less lanes – higher Fewer components: speed per lane • Less lanes Lower • More integration • More integration Smaller Cost Economies of Scale Size • Volume Lower Power • More integration • Advanced technology Regardless of data rate these goals have always been and will always be true #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Optical Technology Trends Better materials e.g. Silicon Photonics for integration Integration Goals: Lower Faster Optics e.g. PAM modulation Cost (fewer needed) Size Power Higher Yields Relaxed e.g. Forward Error specifications Correction #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 Silicon Photonics – advancing integration Silicon Photonic Die Processed 8” SOI wafer (Image from Samsung website) • Ability to control optical signals in silicon • Utilizes CMOS manufacturing infrastructure and capabilities • Wafer scale manufacturing of optics! • Promise to meet current and future optical communication requirements with: Higher BW (density), Lower Power, and Lower Cost #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 Silicon Photonics Potential for “transceiver on a chip” • Optical devices can be made cheaply using standard semiconductor CMOS fabrication techniques • Optics can be integrated with microelectronic chips. • Silicon integrated optical chips that can generate, modulate, process and detect light signals Silicon Photonics is the most promising optical technology for: Solving for Cost, Power, Density for DC & Client optics Solve for Cost, Power, Density while maintaining Performance for DCI & Long Haul optics #CLUS BRKOPT-2005 © 2019 Cisco and/or its affiliates.