Router Energy Per Bit Router Energy Efficiency 10000
Total Page:16
File Type:pdf, Size:1020Kb
Energy Considerations in Data Transmission and Switching Rod Tucker Department of Electrical & Electronic Engineering University of Melbourne Summary • Big‐picture view of data and energy in network & DCs • Snapshot of 2017 and projections for 2021 • Key energy bottlenecks – Access networks – Servers in data centers – Data storage – Switches and routers – Global optical transmission • Gap between technology limits and reality The Internet Storage Servers Switches Router Routers etc Global Data Core Network Base Center Station Switch Data Center Data Networks Center Metro/Edge Access Network Network Data Center Content Distribution OLT ONU Data Networks Router Center This talk Fiber Optical TX/RX Router Data usage in 2017 ‐ 1.5 Zettabytes >15 ? 12 Zettabytes/year 200 Tb/s 3200 Tb/s 800 Tb/s (40 %) 1.5x1021 bytes/year* Global 500 Tb/s (avg.) Data Core Network 135 kb/s (avg.) Center Data Center Data Networks Center Metro/Edge Access Network Network Data Center 300 Tb/s Content Distribution 3.7 billion users Data Networks (50 % of world pop.) Center 300 Tb/s (60 %) Sources: Cisco, VNI*, 2016‐2021, (2017); Google; Facebook; Akamai 2021 ‐ Twice as much data as 2017 > 40 ? 25 Zettabytes/year 300 Tb/s 6.4 Pb/s 1.6 Pb/s (30 %) 3.3x1021 bytes/year* Global 1Pb/s (avg.) Data Core Network 220 kb/s (avg.) Center Data Center Data Networks Center Metro/Edge Access Network Network Data Center 600 Tb/s Content Distribution 4.5 billion users Data Networks (60 % of world pop.) Center 700 Tb/s (70 %) Sources: Cisco, VNI*, 2016‐2021, (2017); Google; Facebook; Akamai Data, energy and efficiency Data (~ 25 % p.a. Increase) Index Net increase in energy consumption 1 Time Energy intensity 1/ɳ (Joules per bit) Energy efficiency of network equipment (bits per Joule) Network energy consumption Inventory‐based estimate Van Heddeghem et al., Computer. Comms., 2014 Power consumption of the global Internet 1012 Global electricity supply ~ 4 % ~ 3 % ~ 2% Access Network 1011 Network Total 1010 Data Centers* Power Consumption (W) 1 Nuclear power 109 station 2010 2015 2020 Year *Scaled to global Sources: Tucker, OFC 2011 (updated), Van Heddeghem et al., 2014 ( ), Shehabi et al*., 2016 8 Power consumption of the Internet (2017) 5x109 W 5x1010 W Global 10 Data Core Network 6x10 W Center Data Data Center Networks Metro/Edge Center Access 33 W/user Network Network Data Data centers: Content Center 14 W/user Distribution 3.6 billion Network: Data Networks users 19 W/user Center ? W 1.2x1011 W Energy per customer bit ~ = 240 μJ 180 μJ (2021) 500x1012 b/s 4 kb/J Routers Cisco Juniper 10 Router energy per bit Router Energy Efficiency 10000 Cisco AGS 30% p.a. improvement 1000 Wellfleet BCN 100 Cisco GSR 12000 15% p.a. improvement (nJ per bit) Cisco GSR 12000b Cisco CRS‐1 Avici TSR Energy Consumption 10 Cisco CRS‐3 nano‐Joules per bit ALU7750 CRS‐X Cisco NCS 6000 Juniper T4000 1 1985 1990 1995 2000 2005 2010 2015 Year Sources: Nielsen, 2010, Cisco and Juniper Data Sheets, 2015 11 High‐end core router Line Card Buffer Forwarding Engine Routing O/E Switch Fabric Tables I/O Line Card Buffer Power supply inefficiency Buffer Forwarding Engine Routing O/E Switch Engine Fans and Control blowers Control Plane Data Plane Energy/bit 0.1 nJ 0.5 nJ 0.1 nJ 0.6 nJ 0.1 nJ 0.1 nJ Fraction of Total Software‐Defined Optical Switching? Networking? Source: G. Epps, Cisco, 2015 12 Some (ns) switch fabrics m x m Benes SOA Gate Array CMOS 1 1 2 2 3 144 x 144 @11 Gb/s SOA Gate 3‐dB Coupler m m Optical Dilated ring resonator switch 1 1 2 2 Wavength CMOS m Converters driver m Optical Energy per bit per crosspoint 10‐8 Semiconductor optical amplifier array 10‐9 CMOS (Including SERDES) Ring resonator Wavelength routed (AWG) 10‐10 VSC3144 ‐11 * Broadcom 256x256 Switch Fabric Energy per Bit per Crosspoint (J) 10 10 100 1000 Bit Rate per Wavelength (Gb/s) Source: Tucker, OFC, 2012; Stone, Yesterday Optical transmission > 100 million km Transmission energy consumption 10 5 Marconi Wireless Trans‐Atlantic 10 3 Telegraphy Fessenden Trans‐Atlantic Coax 10 Optical + Regen NY ‐ Paris First Trans‐ Atlantic Optical + EDFA 10 ‐1 Newhaven ‐ Azores TAT‐1 ‐3 10 Key West ‐ Havana TAT‐3 TAT‐8 10 ‐5 TAT‐5 Energy/Bit/1000 km (J) TAT‐10 ‐7 ~15% improvement p.a. TAT‐9 10 TAT‐11 TAT‐12/13 10 ‐9 Terrestrial systems Short‐range 100 Gb/s link 10 ‐11 1840 1860 1880 1900 1920 1940 1960 1980 2000 2020 2040 Year Source: Tucker, JSTQE 2011 16 Gap between limits and reality 10‐4 ↓ 15 % per annum 10‐5 “Reality” 10‐6 Transmission 10‐7 Global Routers B x 103 10‐8 Technology “Limits” Network X ‐9 X Energy per Bit (J) 10 Transmission 10‐10 A x 102 ‐11 Devices 10 Crosspoint Sub‐systems 10‐12 2010 2015 2020 2025 Year Source: Tucker, JSTQE 2011. (adapted) 17 Gap between limits and reality (A) • Transmission: ‐ SERDES, Clock recovery, DSP, amplifiers, hop count, protection and restoration, etc • Routers and Switches: ‐ Routing plane, switch controllers, buffers, hop count, protection and restoration, etc. 18 Gap between limits and reality (B) Protocols, DSP, device efficiency, Real interconnects, system margins and penalties, etc PTotal Ideal Poverhead Power consumption Inefficiency Traffic (b/s) P function Overhead Function Subsystem Management and power Energy per bit Ideal overheads, interconnects, etc. Traffic (b/s) 19 Diurnal traffic variations Italy North America * Real Busy‐hour traffic growth: 51% p.a. Average traffic growth: 32% p.a. Ideal * Cisco, VNI*, 2016‐2021, (2017) Power consumption 20 Data centres • Servers • Switches and routers • Interconnects • Data Storage http://www.google.com/about/datacenters/gallery/#/all Zoom in to server chip Intel 14‐nm chip • Energy dominated by 1 μm ‐ wires at high speed ‐ leakage currents at low speed Wire M12 Energy Power per op. M9 leakage M1 Clock/data rate • Chip‐level optical interconnects ‐ limited to upper layer(s) ‐ alleviate the pin‐out problem Chip energy 10‐10 12‐m 10‐11 PMOS SiGe VCSEL 50 Gb/s* 10‐12 FinFET RX 64 Gb/s** 10‐13 (1‐m)3 laser*** ‐14 10 Total CMOS chip energy including wires ↓ 25 % p.a. 10‐15 45 130 32 22 ‐16 CMOS gate 10 90 11 energy 65 ‐17 45 (100‐nm3) laser*** 10 32 22 Link goal Energy per Operation (J) ‐18 10 Feature 11 size in nm 10‐19 10‐20 1970 2000 2010 2020 2030 1980 1990 Year Sources ITRS ’97‐’10 Roadmaps; Hinton et al., JSTQE 2008; Tucker, JSTQE, 2010, Miller, OFC, 2016*** https://www.e3s‐center.org/ ; Belfiore, JSSC, 2017* ; Cevrero, ISSCC, 2017** Zoom out to data centers (2017) 5x109 W 3200 Tb/s 2x1010 W 25+ Gb/s 100+ Gb/s Servers: 2x1010 W Energy per bit = = 6.3 μJ Energy per customer bit = 40 μJ 3200 Tb/s Network: Energy per network bit = 1.5 μJ Energy per customer bit = 10 μJ 24 Gap between limits and reality – Data Center 10‐4 ↓ 15 % per annum Current 10‐5 Trends 10‐6 10‐7 Switches Data Technology “Limits” Centers 10‐8 ‐9 Links Energy per Bit (J) 10 x 106 ‐10 10 Crosspoint 10‐11 Devices 10‐12 Links Sub‐systems 2010 2015Year 2020 2025 25 Take home • Energy is an important issue – Network consumption: 19 W per user – Data centers: 14 W per user • Access networks dominate, followed by data centers • Technology improvements barely keeping up with data growth • Potential for 3+ orders of magnitude improvement – Device level – Circuit level – System level.