Facebook Server Intel Motherboard v3.0 Author: Jia Ning, Engineer, Facebook Contents Contents .......................................................................................................................................... 2 1 Scope ......................................................................................................................................... 5 2 Overview ................................................................................................................................... 5 2.1 License ............................................................................................................................. 5 3 Efficient Performance Motherboard v3 Features ..................................................................... 6 3.1 Block Diagram .................................................................................................................. 6 3.2 Placement and Form Factor ............................................................................................ 6 3.3 CPU and Memory ............................................................................................................. 7 3.4 Platform Controller Hub .................................................................................................. 8 3.5 Printed Circuit Board Stackup (PCB) ................................................................................ 8 4 Basic Input Output System (BIOS) ........................................................................................... 10 4.1 BIOS Chip ....................................................................................................................... 10 4.2 BIOS Source Code .......................................................................................................... 10 4.3 BIOS Feature Requirements .......................................................................................... 10 5 Baseboard Management Controller (BMC) ............................................................................. 14 5.1 Management Network Interface ................................................................................... 14 5.2 Local Serial Console and Serial-Over-LAN (SOL) ............................................................ 14 5.3 Graphic and GUI ............................................................................................................. 15 5.4 Remote Power Control and Power Policy ...................................................................... 15 5.5 Port 80 POST .................................................................................................................. 15 5.6 Power and System Identification LED ............................................................................ 15 5.7 Platform Environment Control Interface (PECI) ............................................................. 16 5.8 Power and Thermal Monitoring and Power Limiting ..................................................... 16 5.9 Sensors .......................................................................................................................... 16 5.10 System Event Log (SEL) ................................................................................................ 21 5.11 Fan Speed Control in BMC ........................................................................................... 23 5.12 BMC Firmware Update ................................................................................................ 24 5.13 BMC Update: Dual PCH flash ....................................................................................... 24 5.14 BMC Update Complex Programmable Logic Device (CPLD) ........................................ 26 6 Thermal Design Requirements ................................................................................................ 26 2 November 5, 2013 Open Compute Project Intel Motherboard Hardware v3.0 6.1 Data Center Environmental Conditions ......................................................................... 26 6.2 Server Operational Conditions ...................................................................................... 27 6.3 Thermal Kit Requirements ............................................................................................. 28 7 Midplane Interface and Midplane ........................................................................................... 29 7.1 Connector Modules ....................................................................................................... 29 7.2 Pin Definition ................................................................................................................. 29 7.3 Population options ........................................................................................................ 33 7.4 Midplane for Open Rack V1 ........................................................................................... 34 8 Motherboard Power system ................................................................................................... 35 8.1 DC Cable Assembly ........................................................................................................ 35 8.2 Input Voltage ................................................................................................................. 35 8.3 Hot Swap Controller (HSC) Circuit ................................................................................. 35 8.4 CPU Voltage Regulator (VR) ........................................................................................... 36 8.5 DIMM Voltage Regulator ............................................................................................... 37 8.6 Voltage Regulator Module Design Guideline ................................................................. 37 8.7 Hard Drive Power .......................................................................................................... 38 8.8 System VRM Efficiency .................................................................................................. 39 8.9 Power On ....................................................................................................................... 39 9 I/O System ............................................................................................................................... 40 9.1 PCIe* x24 Slot Connector/Riser Card ............................................................................. 40 9.2 DIMM Connector ........................................................................................................... 41 9.3 PCI-E Mezzanine Card .................................................................................................... 41 9.4 Network ......................................................................................................................... 43 9.5 USB ................................................................................................................................ 43 9.6 SATA ............................................................................................................................... 44 9.7 Next Generation Form Factor (NGFF) Connectors ......................................................... 44 9.8 Debug Header ................................................................................................................ 44 9.9 Switches and LEDs ......................................................................................................... 47 9.10 Fan Connector ............................................................................................................. 49 9.11 TPM Connector and Module ....................................................................................... 49 10 Mechanical .............................................................................................................................. 51 10.1 Shelf ............................................................................................................................. 51 10.2 Tray and Sled ............................................................................................................... 51 10.3 PCIe and HDD bracket ................................................................................................. 52 10.4 Fixed Locations ............................................................................................................ 52 http://opencompute.org 3 10.5 PCB Thickness .............................................................................................................. 52 10.6 Heat Sinks and Independent Loading Mechanism (ILM) ............................................. 52 10.7 Silk Screen ................................................................................................................... 52 10.8 DIMM Connector Color ............................................................................................... 52 10.9 PCB Color ..................................................................................................................... 52 11 Environmental Requirements ................................................................................................. 53 11.1 Vibration and Shock .................................................................................................... 53 12 Prescribed Materials ............................................................................................................... 54 12.1 Disallowed