Managing Cloud Hardware
Alfie Lew, Inspur John Leung, Intel Sai Dasari, Facebook Agenda
• Hardware Management Overview • Leverage Existing Standards Open Software • Open Source Projects • OpenBMC firmware • Open Source Firmware Open Hardware • OpenRMC Summary • Open Firmware • Q&A
2 Hardware Management (Legacy)
Remote • In-band Out-of-band Client ▪ Uses controller (BMC) ⎻ Uses main CPU, primary In-band memory, etc. ▪ May operate at low power states SW OS Agent Out-of-band • Off-platform On-platform (IPMI) ⎻ Manageability of the ▪ Manageability of devices BIOS CPU NIC system to a remote client within the platform NC-SI ⎻ IPMI over RMCP+ ▪ IPMI over various BMC FW BMC protocols and connections IPMI On-platform device other Issues: BMC firmware is not open • device • Security concerns with RMCP+ Platform DMTF Manageability Standards
Remote Client • Off-platform (Redfish) ⎻ A RESTful interface for manageability (HTTP, JSON) In-band
Includes resource models for systems, chassis, SW ⎻ OS managers. Other SDO's have extended Redfish to Agent Out-of-band manage other type of platforms and services (Redfish)
• On-platform (PLDM over MCTP) BIOS CPU NIC
⎻ MCTP - A route-able transport protocol for manageability NC-SI messages (connection independent) BMC FW BMC
⎻ PLDM - A bit-based request/response command model PLDM MCTP ⎻ RDE - A PLDM model designed for devices to support device other Redfish semantics device Platform Redfish Overview ⎻ Uses cloud/web protocols, structures, security models • HTTP, JSON (leverage existing tool chains) ⎻ Resource schemas are accessible, to enable interface programmatically • json-schema, OpenAPI/swagger schema
HTTP/S HTTP GET https://
rawData = urllib.urlopen(‘https://
Output 1A87CA442K Redfish Computer System Model
Service Root Collection Resource Single Resource HTTP GET /redfish/v1/Systems/CS_1/Processors/2 /redfish/v1 Root /redfish/v1/Systems ./Systems/
Schemas Managed By Platform HW Mgmt /redfish/v1/Managers Collection of ./Managers/
6 Managing Beyond Computer Systems
The Redfish Forum works with other SDOs to Redfish Client create models for new management domains Redfish Off-platform interface ⎻ Storage services and networked storage Aug 2015 Aug 2016 Internet Draft 2017 System Storage Network ⎻ Ethernet Switch - map YANG to Redfish YANG ⎻ Industrial IoT ⎻ Telco - Customer Premise Equipment IIoT Telco
⎻ DCIM - DC Infrastructure Mgmt (e.g. CRACs, (Starfish) PDUs, chillers) DCIM (facilities)
With the number of models increasing, how would OCP specify the required manageability of OCP platforms? OCP Platform Manageability Direction
• OCP will use Redfish profiles to specify platform manageability • The OCP Hardware Management project Interface ⎻ Will specify the manageability common across OCP platforms Server Storage Network Telco Rack & ⎻ Has approved the "OCP Baseline Hardware Profile(s Profile(s Profile(s Profile(s Power Management Profile"1 ) ) ) ) Profile(s) • Other OCP projects Hardware Mgmt Baseline Profile ⎻ Will create platform specific profiles ⎻ Profiles will extend the "Baseline Hardware Management Profile" ⎻ "OCP Server Profile" is available
1opencompute.org/wiki/Hardware_Management/SpecsAndDesigns OCP San Jose Compute Node
➢ Successfully ran the Redfish Interop Validator testing against the "OCP Baseline Hardware Management Profile" ➢ Demonstrated at OCP Summit (March 2018)
OCP Server Profile Enabling Interoperable Manageability
• OCP will prescribe platform manageability in machine readable JSON profile • Implementers can use the profile to test conformance of an implementation • Linux Foundation governs the open-source implementation of BMC firmware (OpenBMC)
10 OCP San Jose Compute Node
➢ Incorporated OpenBMC
➢ The implementation passed the OCP Baseline HW Manageability Profile
➢ Demo’ed at OCP Regional Summit (Oct 2018, Amsterdam)
➢ Changes will be upstreamed to the OpenBMC repository1
➢ Target Date: March 2019 available on GitHub
1The Linux Foundation project A Brief History of OpenBMC Project
Two independent implementations
Facebook IBM
Linux Foundation Project (IBM, Google, Intel, Microsoft, Facebook)
12 OpenBMC Development Model Traditional OpenBMC
3rd Party 3rd Party BMC 3rd Party 3rd Party BMC HW Vendor FW Vendor HW Vendor HW Vendor (AIC/PS/NVMe) (Aspeed/Nuvoton) (AIC/PS/NVMe) (Aspeed/Nuvoton)
3rd Party BMC 3rd Party BMC OpenBMC ODM FW Vendor FW Vendor Project Server Vendor
BMC FW BMC FW Source Code Source Code
13 OpenBMC Architecture
CHASSIS CONFIGURATION FIRMWARE RMCP+ REDFISH INVENTORY CONTROL MANAGER UPDATE ENDPOINT ENDPOINT MANAGER
Applications SYSTEMD D-BUS
bridge bridge GPIO/ EWS FSC BT MBOX KCS IPMI IPMB ADC ENDPOINT
Linux Kernel/Drivers Bootloader & Kernel Boot Loader (e.g. U-Boot)
Hardware BMC SoC / SPI Flash / TPM
14 Additional Resources
• Source Code: • Contact • https://github.com/openbmc • Mail: [email protected] • IRC: #openbmc on freenode.net • Continuous Integration Testing • Riot: #openbmc:matrix.org • https://openpower.xyz • Web Page: • Code Reviews • www.openbmc.org • http://gerrit.openbmc-project.xyz
15 Mission: Develop an open source philosophy based ‘system firmware’ (BIOS) modules, to support different Operating Systems and different CPU silicon vendors.
Major Companies Contributing to OSF development: Microsoft, Intel, Infrastructure, Google, Facebook, Lenovo, IBM, Two Sigma, ITRenew, 9 Elements, Tools & Cavium, AMD and many more. Eco-system support Major Work Streams: Open EDKII DXE core, Linux Boot, Core Boot, Silicon Interface Firmware Module, Intel FSP, AMD AGESA, ARM boot code, HW platform module, Build tools, Automated test support, HW requirements, etc.
GitHub Repositories Collateral link: https://github.com/opencomputeproject/OSF
Bi-weekly OSF discussions : Architectural reviews, workstream progress, design reviews, agenda setting and other collaborative discussions on OSF development. Contact the following to subscribe, participate and contribute: [email protected] [email protected] [email protected] Collaborative Community development model Proposed OpenRMC sub-project
• Rack ⎻ The rack is the fundamental building block for the Data Center • Rack Manager Controller (RMC) ⎻ The RMC is fundamental to Data Center Management hierarchy ⎻ The RMC can be located standalone, within power shelf, or within the switch • Opportunity ⎻ Common language/model between RMC and nodes (aggregation, etc.) ⎻ Provide an open source reference RMC
nodes Open Source Firmware for OCP Ecosystem
OCP Design Binary Blobs Collateral
Product/Company Specific Presentations Specifications Source Repo
Firmware (Build Meta Data) Open Source Community Repo (e.g. OpenBMC, OpenRMC, LinuxBoot, Schematics Data Sheets Coreboot, OpenEDKII)
OCP Specific Source Repo Design (e.g. LED Config, Redfish 3D/CAD Files Packages Profiles)
18