The Findings of the IEEE 802.3 Industry Connections Ethernet

The Findings of the IEEE 802.3 Industr y Connections Ethernet Bandwidth Assessment Ad Hoc

IEEE 802 Plenary San Diego, CA, USA July 16, 2012 Presenters

• John DAmbrosia,D’Ambrosia, Dell, IEEE 802.3 BWA Chair • Peter Anslow, Ciena, IEEE 802.3 BWA Editor • Mark Nowell, Cisco • Scott Kipp, Brocade • Peter Stassar, Huawei

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 2 Agenda

• Introduction (John DDAmbrosia)’Ambrosia) • Findings – Overview (Mark Nowell) – The Data Center (Scott Kipp) – Transport Networks (Peter Stassar) • Summary (Peter Anslow)

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 3 Disclaimers

• This presentation is a supplement to the IEEE Industry Connections Ethernet Bandwidth Assessment D1.2, which is pending final approval (this week) by the IEEE 802.3 Working Group • All contributed information is solely the perspective of the respective contibttributors. • The views expressed in the Assessment solely represent the views of the IEEE 802.3 Working Group, and do not necessarily represent a position of the IEEE, the IEEE Standards Association, or IEEE 802.

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 4 INTRODUCTION

JOHN D’AMBROSIA, DELL

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 5 The 2006 HSSG Call‐For‐Interest

The Ethernet Ecosystem

Consumer Broadband Access Content Providers

Broadband Internet Backbone ANtkAccess Networks NtNetwork s Content Networks

Internet Backbone Networks Research Enterprise Networks Networks

Corporate Data Centers and Research, Education Enterprise and Government Facilities Internet eXchange and (High Performance (High Performance Interconnection Points Computing) Computing)

July 18, 2006 Higher Speed Study Group CFI, V 1.01 20 San Diego, CA

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 6 The 2007 HSSG Tutorial

Why Higher Speed Ethernet?

Fundamental bottlenecks are happening everywhere

Increased # Increased Increased Bandwidth +=+ of users access services explosion rates and methods everywhere

As demonstrated EFM, xDSL, YouTube, by the numb er of WiMax, BitT orrent, ISPs: Comcast, xPON, VOD, AOL, YahooBB, Cable, WiFi, Facebook, NTT, Cox, 3G/4G… Kazaa, Netflix, Easyy,g,Net, Rogers, iTunes, 2nd BT, ... life, Gaming…

IEEE 802.3 Higher Speed Study Group - TUTORIAL 18

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 7 Life after IEEE P802.3ba

• End‐users through the prior HSSG: The next speed of 1,000,000 Ethernet must begin when 100GbE done! 100 Gigabit Ethernet 100,000

• HSSG Bandwidth Forecast for Core Networking 40 Gigabit Ethernet “Core Networking” Doubling ≈18 mos s // 10 Gigabit Ethernet

– 2013: 400 Gb/s Mb 10,000

– 2015: 1 Tb/s Rate

• Other bandwidth trends? Gigabit Ethernet 1,000

Server I/O • 2011 Formation of: Doubling ≈24 mos IEEE 802.3 Industry Connections 100 Ethernet Bandwidth Assessment Ad 1995 2000 2005 2010 2015 2020 Date Hoc

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 8 IEEE 802.3 BWA Web & Reflector Information

• Charter and Scope – Evaluate Ethernet wireline bandwidth needs of the industry – Reference material for a future activity – The role of this ad hoc is to gather information, not make recommendations or create a CFI

• Webpage ‐ http://www. ieee802. org/3/ad_ hoc/bwa/index. html • Reflector ‐ http://www.ieee802.org/3/ad_hoc/bwa/reflector.html • Public request for data ‐ http://www.ieee802.org/3/ad_hoc/bwa/public/anslow_01a_0411.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 9 Summary of Data Submissions

• Scott Kipp, Brocade, “Data Center Bandwidth Scenarios” – httpp//://www.ieee802.org/3/ ad _hoc /bwa /public /ma y11 /ki pp_01 _0511. pdf • Andy Bach, NYSE Euronext, “Bandwidth Demand in the Financial Industry ‐ The Growth Continues” – http://www.ieee802.org/3/ad_hoc/bwa/public/jun11/bach_01a_0611.pdf • Kimball Brown, LightCounting, “Server Bandwidth Scenarios ‐ Signposts for 40G/100G Server Connections” – httpp//://www.ieee802.org/3/ ad_ hoc///p/j/bwa/public/jul11/brown__ 01a 0711.pdf • Tom Cloonan, Arris, “Bandwidth Trends on the Internet... A Cable Data Vendor's Perspective” – http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/cloonan_01a_0911.pdf • Scott Kipp, Brocade, “Storage Growth and Ethernet” – http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/kipp _01a _0911.pdf • Mark Nowell, Cisco, “Cisco Visual Networking Index (VNI) Global IP Traffic Forecast Update; 2010 ‐ 2015” – http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/nowell_01_0911.pdf • Petar Pepeljugoski and Paul Coteus, IBM, “Bandwidth needs in HPC taking into account link redundancy” – http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/pepeljugoski hoc/bwa/public/nov11/pepeljugoski_01 _1111.pdf • Huang Xi, Huawei, “Bandwidth Needs in Core and Aggregation nodes in the Optical Transport Network” – http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/huang_01_1111.pdf • Henk Steenman, AMS‐IX / Euro‐IX, “The European Peering Scene” – http://www. ieee802.org/3/ad _hoc/bwa/public/nov11/steenman _01 _1111 . pdf • Lone Hansen, BSRIA, “Global Data Centres Presentation IEEE” – http://www.ieee802.org/3/ad_hoc/bwa/public/dec11/hansen_01_1211.pdf • Eli Dart, ESnet, “Data Intensive Science Impact on Networks” – http://www. ieee802.org/3/ad _hoc/bwa/public/dec11/dart _ 01_ 1211. pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 10 Assessment Limitations

• Assessment Duration: 18 months maximum – Limited study time – Prevent data from becoming dated – IfInforma tion provide d snapshthot at time of subiibmission • Past trends may not be an accurate predictor of the future – Emerging applications – Technology – Standardization Efforts – Will Ethernet cost per gigabit continue to decrease? • Underlying assumptions – Market adoption – Continuation of applications that require increasing bandwidth

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 11 FINDINGS

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 12 OVERVIEW

MARK NOWELL, CISCO

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 13 The Equation Remains the Same

Increased # Increased Increased BdidthBandwidth More Devices+=+ Faster Broadband Speeds of Access Key Services SpeedExplosion Increasing Users Rates and Growth MthdMethods Everywhere More Internet Users Factors More Rich Media Content

Source: nowell_01_0911.pdf citing Cisco Visual Networking Index (VNI) Global IP Traffic Forecast, 2010–2015, http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/nowell_01_0911.pdf IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 14 2015 Global Users and Network Connections

North America Western Europe Central/Eastern Europe 288 Million Users 314 Million Users 201 Million Users 222.2 Billion Networked Devices 232.3 Billion NNtetwork kded DDievices 902 Million NtNetwork kded DiDevices

Japan 116 Million Users 727 Million Networked Devices

LiLatin AAimerica Middle East & Afifrica Asia Pacific 260 Million Users 495 Million Users 1330 Million Users 1.3 Billion Networked Devices 1.3 Billion Networked Devices 5.8 Billion Networked Devices

North America Western Europe Central/Eastern Europe 3.7‐Fold growth 3.9‐Fold growth 3.3‐Fold growth 7.5 to 27 Mbps 9.2 to 36 Mbps 6.1 to 20 Mbps

Japan 4.1‐Fold growth 15.5 to 64 Mbps

Latin America Middle East & Africa Asia Pacific 2.9‐Fold growth 2.5‐Fold growth 4.6‐Fold growth 2.8 to 8 Mbps 2.8 to 7 Mbps 5.5 to 25 Mbps

100G The Era of Wideband 10G The past 29‐years show a Cable Modems

1G constant bandwidth increase of ~300 Mb/s ? ~1.5x every year... 200 Mb/s

(b/s) 100M 50 Mb/s 12 Mb/s 10M Modems

rr 5 Mb/s

fo 1M 1 Mb/s 256 kb/s 512 kb/s 100k 56 kb/s 28 kb/s 128 kb/s 33 kb/s Bandwidth 10k 9.6 kb/s dd 14.4 kb/s The Era of Cable Modems 2.4 kb/s 1k 1.2 kb/s Permitte 300 b/s The Era of DS

100 Dial‐Up Modems ax

MM 10

1 Year 1982 1986 1990 1994 1998 2002 2006 2010 2014 2016

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/cloonan_01a_0911.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 17 Example: Traffic Generation Comparison

Bandwidth Generation Compared to a 32 bit based laptop

Device Traffic multiplier Tablet 1.1 64‐bit Laptop/PC 1.9 Internet enabled HDTV 2.9 Gaming console 3.0 Internet enabled 3D TV 3.2

North America Western Europe Central/Eastern Europe 22.3 EBEB/Month/Month by 2015 18.9 EBEB/Month/Month by 2015 3.7 EBEB/Month/Month by 2015 26% CAGR, 3X GGthrowth 32% CAGR, 4X GGthrowth 39% CAGR, 5X GGthrowth

Japan 4.8 EB/Month by 2015 27% CAGRCAGR,, 3X Growth

Latin America Middle East & Africa Asia Pacific 4.7 EBEB/Month/Month by 2015 2.0 EB/Month by 2015 24.1 EBEB/Month/Month by 2015 48% CAGRCAGR,, 7X Growth 52% CAGR, 8X Growth 35% CAGR, 4X Growth

5 M Order Traffic Equities Quotes EitiEquities TdTrades OtiOptions DDtata second

er Bandwidth Growth pp

3T essages M M

2T 0 2000 2005 2010 Usage growth 1T Source Data Industry Distribution

0 2006 2005 2011

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/jun11/bach_01a_0611.pdf IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 21 Bandwidth Growth is throughout the Eco‐system

Networking equipment, compute (servers) equipment and storage equipment all required to scale to match application requirements

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/jun11/bach_01a_0611.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 22 THE DATA CENTER

SCOTT KIPP, BROCADE

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 23 Data Center Growth

Increased + Increased + Increased = Bandwidth Storage Processing Bandwidth Explosion

Entered the 100GbE era in 2010 Networking Individual switches have Tb/s of bandwidth

First petaflop supercomputers in 2011 Compute IdiidlIndividual servers ddlieliver ing 10s of Gb/s of I/O PCIe 3.0 supports 2 x 40GbE NICs now

Entered the zettabyte (1 billion terabytes) era in 2010 Storage Individual disk drives over 1 terabyte 1000 disk drive storage subsystem equals 1 Petabyte

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 24 Cumulative Data

GhGrowth over Next Total Digital Data Decade 7910 # of Servers x10 tes) yy Entered the Storage x50 Zettabyte Era (Exab # of Files x75 e gg 1227 Consider Stora 130 the implications! 2005 2010 2015

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/kipp_01a_0911.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 25 Storage Access Methods

SAN DAS Ethernet Switch NAS iSCSI Storage Arrays

iSCSI FCoE Switch Adapter Fibre Channel NIC Over Ethernet LAN Storage Arrays CNA 8G, 16G, LOM 32G Fibre Channel Storage Arrays Ethernet HBA FC Fibre Channel Sour ce: Fibre Channel Tape Library http://www.ieee802.org/3/ad_hoc/bwa/public/sep11 /kipp_01a_0911.pdf Switch IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 26 Growth in External Storage Subsystems

Ethernet Based Storage NAS, iSCSI and FCoE Over 50% of 35 r storage

Yea 30

expected to be 25 per Ethernet‐

dd 20 bdbased in 2015 15 2011 tes Sol yy 10 2015

Exab 5 0 NAS Fibre iSCSI DAS FCoE Channel

Source: kipp_01a_0911.pdf citing IDC Worldwide Enterprise Storage Systems 2011‐2015 http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/kipp_01a_0911.pdf IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 27 What’s Happening With Servers?

• Moore’s Law keeps increasing transistor count and improving performance • 2012 servers bibegin using PCIe 303.0 to support 40GbE Speeds of Network Year Released Gigabyte/s Interface Cards (NICs) PCIe 1.x 2003 4 2 x 10GbE PCIe 2.x 2005 8 4 x 10GbE PCIe 303.0 2012 16 2 x 40GbE PCIe 4.0 2016 32 2 x 100GbE

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/jul11/brown_01a_0711.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 28 Server Port Speed d hippe SS

orts PP

rver ee S

ual nn An

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/jul11/brown_01a_0711.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 29 Server Aggregation in a Cluster

Each server Rack of 40 Each cluster of 25 producing servers producing racks producing 10‐80 Gb/s 0.4 ‐3.2 Tb/s 10‐80 Tb/s

I/O per server (Gb/s) 5 10 20 40 80 Each 1,000 server Servers / rack 40 40 40 40 40 cluster sends fraction of possible Bandwidth / rack (Gb/s) 200 400 800 1600 3200 bandwidth to Interconnect Fabric Source: http://www.ieee802.org/3/ad_hoc/bwa/public/may11/kipp_01_0511.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 30 Cluster Aggregation in Data Center

Cluster or Cluster Traffic Data Center POD of RkRacks to Core in Tb/s Containers

Cluster 2 Cluster 3 Cluster 4

Cluster 1 CORE Cluster 5

Cluster bandwidth to core (Tb/s) 0.4 1 2 4 Clusters 10 10 10 10 Core Bandwidth to core (()Tb/s) 4 10 20 40 SihSwitches / Bandwidth to WAN (Gb/s) 20 40 200 400 Routers Oversubscription to WAN 200 250 100 100

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/may11/kipp_01_0511.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 31 Example: Impact of Data Migration

Application migration are between servers, clusters or data centers

Primary Data Center Cloud Provider or WAN Secondary Data Center WDM Router Router WDM Application A data needs to be mirrored before the Server application A can move. A A A SAN SAN Ethernet Switch Switch FC

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/kipp_01a_0911.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 32 TRANSPORT NETWORKS

PETER STASSAR, HUAWEI

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 33 Transport Network Classifications

AAtiggregation NNdode Core Node Aggregation Node Core Node

Access Node Access Node

Feeding the Networks • Consumers • Businesses (data centers)

Note: “Aggregation Nodes” in single carrier networks are equivalent to “IXPs“ in multi carrier networks

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/huang_01_1111.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 34 Bandwidth Needs Per Wavelength: Core Nodes in Single Carrier Networks

1000 1000

900 900 node node ee 800 ee 800

700 700

600 600 in a cor in in a cor λ λ 500 500

400 400 per per

300 300 Gb/s Gb/s

200 200

100 100

0 0 2007 2009 2011 2013 2015 2020 2007 2009 2011 2013 2015 2020

1000

900 Notes:

e node 800 rr • Single carrier networks 700 • 600 More than one carrier in overview in a co λ 500 • 2015: Range 100G – 400G per λ 400 s per // • 2020: Range 400G – 1T per λ 300 Gb 200 • Africa: same trend, a bit delayed 100

0 2007 2009 2011 2013 2015 2020

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/huang_01_1111.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 35 Bandwidth Needs Per Wavelength: Aggregation Nodes in Single Carrier Networks

400 400

350 350 node node nn

300 300

250 250

200 200 naggregatio n aggregation aa 150 aa 150 in in in in

λ 100 λ 100

50 50

0 0 Gb/s per per Gb/s Gb/s per per Gb/s 2007 2009 2011 2013 2015 2020 2007 2009 2011 2013 2015 2020

400

350 Notes:

300 ion node • Single carrier networks tt

250 • More than one carrier in overview 200 • 2015: Range 50G – 100G per λ 150 • in an aggrega 2020: Range 100G – 400G per λ

λ 100 • Africa: same trend, a bit delayed 50

0 Gb/s per per Gb/s 2007 2009 2011 2013 2015 2020

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/huang_01_1111.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 36 Euro‐IX IXP Locations

132 IXPs 36 Countries 140 Cities 420 Locations

Euro‐IX affiliated IXPs

Soon to be affiliated IXPs

Note: Global 321 IXP’s (100%), EU 41%, America’s 33%, Asia‐Pacific 19%, Africa 7% Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/steenman_01_1111.pdf IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 37 Global Annual IXP Peak Traffic Growth Rates: By Region (for 2010)

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/steenman_01_1111.pdf IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 38 Five Year Peak European IXP Traffic Projections Tb/s Tb/s

100 Mb Mb 1 GigE

10 GigE

10 Mb

Notes: 10Mb almost zero, 100Mb strongygly ↘, 1G starting ↘, 10G strongygly ↗

LONAP (London) 8.5 Gb/s peak traffic on a typical 2010 weekday

Wed. 29. 5 Gb/s peak afternoon World Cup 2010 England vs Slovenia

• CERN: Atlas detector in LHC ((gLarge Hadron Collider) generates ~1 petabyte/sec • Genome seqqguencing: – Per‐instrument data rate strongly ↗ (~10x over 5 yy)ears) – Cost of sequencers strongly ↘ (10x over 5 years) • Futures: Square Kilometer Array (SKA) – ~2800 receivers in telescope array – 2 petabytes/sec to central correlator • sending @ ~100 Gb/s to analysis centers

Source: http://www.ieee802.org/3/ad_hoc/bwa/public/dec11/dart_01_1211.pdf IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 42 ESnet Accepted Traffic (Petabytes/month)

Exppgecting 100 Petabytes/month of data in 2015

PETE ANSLOW, CIENA

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 44 Summary method

• Relative growth of the various sectors plotted on a single chart – The growth of each sector was normalized to 2010 (the year IEEE Std 802.3ba was approved) • This growth is a predictor of the future only if downward cost per bit trend is continued – Ethernet cost per bit has to fall with time or the predicted exponential rise in traffic will result in unsupportable costs

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 45 Variation Factors

• Reported growth trends are “sector” averages – There is considerable variation by region or by market segment • Example 1 – Predicted Regional Growth of IP traffic (2010 – 2015) – Minimum: 26 % in North America – Maximum: 52 % in the Middle East and Africa – Reported average: 32% • Example 2 –Growth in IXP peak Traffic in 2010 – Minimum: 45.88 % in small IXPs – Maximum: 78.18 % in medium IXPs – Reported Average: 64%

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 46 Growth Rate Trends

100 Financial sector fit to Figure 15 Science CAGR = 95% fit to Figure 13 ESnet 2004 to 2011 e 10 uu CAGR = 70% Peering val fit to Figure 39 HSSG tutorial CAGR = 64% 2010

oo Slide 22 core t

Cable 1 CAGR = 58% Figure 39 Euro‐IX Figure 20 historical data CAGR = 50% relative

cc HSSG tutorial Slide 22 server I/O IP traffic Traffi 0.1 CAGR = 36% Figure 2 CAGR = 32% Figure 15 NYSE historical data 0.01 2004 2006 2008 2010 2012 2014 2016 2018 2020 IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 47 Summary

• The exponential rise in traffic is predicted to continue

Bandidhdwidth Increased # +=Increased + Increased of users access services explosion rates and methods everywhere

• Servicing dddemand with existing rates or new ones > 100 Gb/s will depend on the cost effectiveness of the solution

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 48 Publication Information

• IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc Report is pending final approval this week.

• Upon approval final report to be published: – httpp//://www.ieee802.or g//g/3/ad_ hoc///bwa/BWA_ Reppport.pdf

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 49 The IEEE 802.3 Ethernet Bandwidth Assessment Ad Hoc would like to thank all of the individuals who contributed data to this effort.

THANK YOU! QUESTIONS?

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 50 ABBREVIATIONS

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 51 1GbE 1 Gb/s Ethernet 10GbE 10 Gb/s Ethernet 40GbE 40 Gb/s Ethernet 100GbE 100 Gb/s Ethernet 3D TV three‐dimensional television BW bandwidth CAGR compound annual growth rate CMTS cable modem termination system CNA Converged Network Adapter DAC digital‐to‐analog converter DAS direct attached storage DOCSIS Data Over Cable Service Interface Specification DS downstream EPON Ethernet passive optical network FCoE Fibre Channel over Ethernet HBA Host Bus Adapter HHP house‐holds passed HPC high performance comppguting HSSG Higher Speed Study Group I/O input/output IP Internet Protocol iSCSI Internet small computer system interface ISP Internet service provider

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 52 IXP Internet exchange point LAN local area network LAG link aggregation LHC Large Hadron Collider LOM LAN on motherboard MAN metropolitan area network MSO multi‐system operator NAS network attached storage NIC network interface card OOMEM original equipment manufacturer OTN Optical Transport Network P2P peer‐to‐peer PC personal computer PCIe Peripheral Component Interconnect Express QAM quadrature amplitude modulation RFOG radio frequency over glass SAN storage area network SMB small and medium business US upstream VOD video on demand WAN wide area network x86 a family of architectures based on the Intel 8086 CPU

IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc July 2012 IEEE 802 Plenary, San Diego, CA, USA Page 53