Demystifying 40 Gigabit Interfaces in Data Centers

Rita Horner Synopsys

Santa Clara, CA USA April-May 2014 1 Evolution of Ethernet Speeds

100 GE Data Rate (4x25 GE) (Mbps) 100 GE (10x10 GE) 100,000 10 10,000 Gigabit 40 GE Ethernet (4x10 GE) 1000 FDDI 100 Ethernet 10

1 1980 1990 1995 1998 2002 2010 2014

Santa Clara, CA USA April-May 2014 2 Different Interfaces in a Chassis

1. Chip-to-Chip Port side 2. Chip-to-Module (direct) 3. Chip-to-Backplane 3

2 1 2 3 1

2 1 3

Santa Clara, CA USA April-May 2014 3 Evolution of 10 GE Port Side Aggregated 10 GE (XAUI)  Serial 10 GE

. IEEE 802.3ae-2002 – supporting both port side and backplane using aggregated XAUI (3.125 Gbps per lane in a x4 configuration) with 8b/10b encoding

10 GE Module form factor migration Gen1 X2 Gen2 Gen3 4-lanes XAUI 300 pins 16-lanes XSBI

Xenpak SFP+ 1-lane 10G SFI 4-lanes XAUI XFP 1-lane 10G XFI

Modules with Gearbox, a.k.a. MUX/De-MUX Module with Retimer Un-retimed

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Santa Clara, CA USA April-May 2014 4 10GE Port Side Module Interfaces Size Reduction Requires Power Reduction

4x 4 XENPAK Port 3.125 Gbps Gearbox ASIC SerDes XAUI or 10 GE PHY (4:1) 4 X2

4x 4 Port 3.125 Gbps Gearbox XAUI XFI Retimer XFP 10 GE ASIC SerDes (4:1) PHY 4 10 GE

Port 10.3 Gbps SerDes 10 GERetimer 10 GE SFP+ 10 GE ASIC PHY XFI SFI

Santa Clara, CA USA April-May 2014 5 Evolution of Port Densities 10GE  40GE Single row 24 x 10 GE SFP+ ports  240 Gbps Two rows 48 x 10 GE SFP+ ports  480 Gbps

Single row 4 x 40 GE CFP ports  160Gbps

Single row 12 or 16 x 40 GE QSFP ports  480 to 640 Gbps Two rows 12 or 16 x 40 GE QSFP ports  960 to 1280 Gbps

Santa Clara, CA USA April-May 2014 6 40GE Port Side Module Interfaces Size Reduction Requires Power Reduction

4x 4 Port 10.3 Gbps ASIC SerDes XLAUI Retimer CFP 40 GE PHY 4

4x Port 10.3 Gbps 40 GERetimer 40 GE QSFP 40 GE ASIC SerDes PHY XLAUI XLPPI

4x Port 10.3 Gbps 40 GE QSFP 40 GE ASIC SerDes PHY Retimer XLPPI

Santa Clara, CA USA April-May 2014 7 Different Interfaces in a Chassis

1. Chip-to-Chip Port side 2. Chip-to-Module (direct) 3. Chip-to-Backplane 3

2 1 2 3 1

2 1 3

Santa Clara, CA USA April-May 2014 8 10GE and 40GE Interfaces

Port Side Rate Interface Chip-to-Chip Chip-to-Module Backplane Fiber Optic Fiber Optic Copper Electrical XFI SFI (limiting & linear) SFI linear 10GBASE-KR

SFP+, 10GBASE-SR SFP+, DAC XFP, 10GBASE-SR 10 GE Module SFP+, 10GBASE-LR (Direct XFP, 10GBASE-LR - types SFP+, 10GBASE-LRM Attached XFP, 10GBASE-ER SFP+, 10GBASE-ER Copper)

Electrical XLAUI XLPPI 40GBASE-KR4

CFP, 40GBASE-SR4 QSFP, 40GBASE-SR4 QSFP, 40 GE Module CFP, 40GBASE-LR4 QSFP, 40GBASE-LR4 40GBASE- - types CFP, 40GBASE-ER4 QSFP, 40GBASE-ER4 CR4

Santa Clara, CA USA April-May 2014 9