Openpower Foundation Opportunities for Education, Research and Collaboration

OpenPOWER Foundation Opportunities for Education, Research and Collaboration

June 2015

Misty Decker Strategic University Alliances misty@us.ibm.com

• Moore’s law no Open Development • Rich software longer satisfies open software, open hardware ecosystem performance gain • Spectrum of power • Growing workload Collaboration of servers demands thought leaders • Multiple hardware • Numerous IT simultaneous innovation, options multiple disciplines consumption models • Derivative POWER • Mature Open Performance of chips software ecosystem POWER architecture amplified capability

Feeds back … resulting in client choice

OpenPOWER is an open development community, using the POWER Architecture to serve the evolving needs of customers.

The year • Collaborative solutions, standards, and reference designs available • Independent members solutions and systems ahead • Sector growth in technical computing and cloud • Global growth with increasing depth in all layers • Broad adoption across hardware, software, and end users

Arizona State Oregon State University ASTRI POSTECH (Gyungbuk, Korea) Bauman Moscow State Tech University Rice University Cineca Sandia National Lab Combatore Institute Seoul National University Forschungszentrum Juelich Shanghai Jiao Tong University Freebsd Foundation Symbiosis Inst Comp (SICSR) Hartree Centre Science & Technologies Tohoku University IIIT Bangalore Tsinghua University IIIT Bombay University of Arkansas IIIT Roorkee University of Campinas (Unicamp) INAF Instituto Nazionale di Astrofisin University of Central Florida Institute of Communication and Computer Systems University of Hawaii Lawrence Livermore Labs University of Hyderabad Louisianna State University University of Illinois Max Planck Society University of Oregon Northeastern Univ (S/W College) in Liaoning, China University of Patras Oak Ridge National Laboratory University of Southern California Waseda University

Platinum Gold Silver (partial list) Google Avnet Algo-Logic Systems IBM Chuanghe Telco Tech Asetek Nvidia Hitachi Celestica LLC Altera Huaxun Tech Company Chelsio Communications Canonical Inspur Cirrascale Mellanox Res Inst of Jiangsu Industrial Convey Computer Corporation Micron Neu Cloud Orientale DataDirect Networks Samsung VeriSilicon Exablaze SK Hynic Wistron Fusion IO Suzhou Power Core Zoom Networks GIS Federal Tyan ZTE IDT Inventec Corporation KNS Group Memblaze Penguin Computing Qlogic Rackspace Redis Labs Ltd. Synapse Design Xilinx

Venue How to access Technical Details

SuperVessel Intended for Chinese Universities, Individual Developers FPGA and Ecosystem Partners GPU (future) Request access: ptopenlab.com Priority Access given to Facebook group: SuperVessel OpenPOWER Chinese applicants QQ group: SuperVessel

Brno University of Intended for Universities, Individual Developers and Power8 Technology, Ecosystem Partners Czech Republic Request access: https://fit-rhlab.rhcloud.com/powerlinux- openpower-development-hosting/ UNICAMP Intended for Universities, Individual Developers and Duration: 1 month, can be São Paulo, Brasil Ecosystem Partners extended upon request Request access: OS: Ubuntu, Fedora http://openpower.ic.unicamp.br/minicloud/index.html OSU Open Intended for Universities, Researchers, Open Source VM Size: 1-8 CPU, 2-16GB Source Lab Developers RAM, 20-120GB disk IBM Linux Technology Center sponsored projects Duration: 1 day to indefinite Request access: OS: Ubuntu, Fedora, Debian, osuosl.org/services/powerdev/request_hosting OpenSUSE

Coming Soon: Indian Institute of Technology, NC State University, Texas Advanced Computing Center Contribute to Workgroups

Work Group Projects Participants 25G IO Interoperability Mode Specification of a 25 Gbps operating mode Member

PSL to AFU specification Accelerator OpenCL SDK to OpenPower Platform interface Member API for access to system interface services via CAIA

System Operating Environment Application Software OpenPOWER Software Ecosystem enablement Public (Open Source) — Toolchain Compliance Compliance Member OpenPOWER profile of architecture Hardware Architecture — Power8 ISA Book 1, 2, 3 Member Coherent Accelerator Interface Architecture (CAIA) I/O related specifications required for OpenPOWER System Design IO Work Group Member I/O Enablement guides and tools 1. Define memory bus protocol/interface f Memory Member 2. Define Memory component potential line up Open Server Development POWER 8 Developer Board Member Platform POWER 8 Reference Design Linux LE, KVM System Software Firmware (OpenPOWER FW Interface) Public (Open Source) POWER LE ABI Develop OpenPOWER FSI Specification version 1.0 and version 2.0 FSI (Full Service Interface) WG Establish context documents referencing to system control options using Member (Open Source) OpenPOWER FSI Source: http://openpowerfoundation.org/technical/working-groups/

Documentation Future OpenPOWER Webinars - Available without an NDA .Performance analysis - Request specific materials .Performance counters Discussion Groups .Multi-threading optimization - Access to experts .Vectorization of codes - Ask your questions .Mathematical libraries - Find colleagues with ideas .Optimization for memory bandwidth Proposed Workshop .XL compilers - Nov 12-13 in Austin TX .Optimization of application XY - Prior to Supercomputing .Optimization of GPU computing Conference .Introduction to CAPI - Technical Knowledge - Explore Collaboration

Information accessible from Members area of OpenPowerFoundation.org

The level of support behind the OpenPOWER Both the current results and future potential Foundation leads me to believe that IBM has are so promising that we are preparing to a real chance at ending Intel's server chip build an OpenPOWER-based, Open “ monopoly.” [IBM is the real threat to Intel’s “ server dominance, Motley Fool] Compute platform. And it will run OpenStack services.

Up to 100x http://www.rackspace.com/blog/openpower-opening-the-stack-all-the- the power of a way-down/ “ classic x86 setup http://labs.runabove.com/power8/ The new systems incorporate technologies from IBM and other providers that are part of “ OpenPOWER…which allow you to achieve … doubling the performance of its already unprecedented computing performance. powerful predecessor, Power7+. The [Cloud Times] “ POWER8 specs are mind boggling. [Microprocessor report] IBM’s huge advantages in multithreading Data-centric supercomputers based on POWER and memory bandwidth favor POWER8 for Department of Energy, Oak Ridge “ when running larger test suites that more National Labs & Lawrence Livermore closely reflect real- to advance innovation and discovery world enterprise in science, engineering and national applications.

defense in $325M deal http://www.linleygroup.com/newsletters/newsletter_detail.php?num=5275

http://openpowerfoundation.org/membership/how-to-join/

Review the Membership Kit

Signature of the Membership Agreement

IP Agreement Any IP submitted is shared with Foundation Members IP developed outside of Foundation owned by University

Individual Membership option also available

DMI connection between an Altera Stratix V FPGA accelerator and a POWER8 CPU Convey’s CAPI developer kit based on the First China “local” POWER company’s Xilinix-based co-processors derivative chip, CP1

Components & Systems

First commercially available OpenPOWER New CAPI-based solution: the ConnectX-4 Nallatech’s OpenPOWER CAPI third-party server adapter card by Mellanox Developer Kit

First Open server specification and motherboard combining OpenPOWER, First GPU-accelerated OpenPOWER Prototype of a new high-performance server New Systems OpenCompute and OpenStack (mock-up) developer platform on the path to exascale & Platforms

RedPower, the first China First commercially available OpenPOWER 2-socket system Inspur 2-socket ChuangHe China-branded OpenPOWER server coming to market in 2015 POWER8 Server OpenPOWER system with POWER8

Data Engine for NoSQL with 40TB CAPI-attached flash 24:1 Server consolidation for 3x lower cost per user Bringing It All Together

Clustering Open Source 192 Vcores 40TB in 2U Redis + CAPI Altera FPGA acceleration and IBM CAPI

Monte Carlo 250x faster

than POWER8 core alone, reduced C code 40x over non-CAPI FPGA

• Support for little endian applications • PoCs available through the Power Development Platform • 50 IBM Innovation Centers and Client Centers Worldwide

8x faster than x86 Ivy Bridge on pattern extraction

10x faster text search than CPU only

Turbo LAMP Stack for mobile and web apps (Ubuntu) (Ubuntu) Orchestration PHP (Zend)

- Juju - PHP (Zend)

Apache Web Server ariaDB (MariaDB) (Ubuntu) M PHP (Zend)L inux (Ubuntu)

High Speed Data Transfer (Mellanox) Turbo LAMP Turbo

Future Dramatically less data center infrastructure Dramatically faster responsiveness to customers

2-3x Port Density with 50% 10x Performance over typical Lower Power SATA SSD’s

Series 7 & 8 – RAID/HBA Portfolio Ecosystem of Industry Solutions -24 port Density Industry’s only 16 • native SAS/SATA adapters

• Flexibility HBA, Caching HBA, or RAID PMC Flashtec Samsung SSD Memblaze SSD • Performance Leading Throughput & IOPS NVRAM 2.5” PCIe Card Up to 10M IOPS 700K IOPS Up to 8TB

• Accelerated appliance with novel data-flow implementation of Memcached on FPGA. • Up to 36x improved performance and power response times in microsecond range. • CAPI integration of memory allows both host memory and coherent- attached flash to be used as value store.

Before: NoSQL in memory (x86) After: NoSQL POWER8 + CAPI Flash

WWW WWW

10Gb Uplink 10Gb Uplink

POWER8 Server Load Balancer 4U 24U Flash Array w/ up 500GB Cache to 40TB 500GBNode Cache 500GBNode Cache 500GBNode Cache 500GBNode Cache Node

Backup Nodes Less is More 24:1 physical server consolidation = 6x less rack space

Best-in-class ingredients • IBM POWER8 • IBM Java • NVIDIA CUDA GPU acceleration • Ubuntu Little Endian Linux for POWER

Web, Java Apps and Infrastructure Analytics & Research

HPC applications for Life Sciences

• Highly threaded • Compute intensive • Throughput oriented • High memory bandwidth • Scale out capable • Floating point • High quality of service • High I/O rates

Business Applications Database

• High quality of service • Handle peak workloads • Scalability • Scalability • Flexible infrastructure • High quality of service • Large memory footprint • Resiliency and security

• Single socket ATX form factor board, BMC based evaluation board • Service subsystem identical to x86 • Reference System for ISV application and OpenPOWER partners • OpenPOWER Software Stack

4x reduction in storage 24:1 server footprint POWER8 reduction through CAPI Flash through GPFS and compression Scale-out Systems for BD&A - Hadoop for BD&A – no SQL

Power S822L

5x Watson scaling through Non-disruptive Software Optimization and Power S824L Application mobility from POWER8 for Cognitive P6/P7 to P8 Power S822

10x ops/sec via Mellanox Fabric exploitation for Data- Power S824 65% Guaranteed system centric workloads utilization without application degradation 8x IBM Java advantage through NVIDIA GPU acceleration

New: SoftLayer Bare Metal POWER cloud New: Rackspace joins OpenPOWER, declares Open Compute OpenPOWER system

New: Docker for Power Systems

New: Bluemix Cloud Integration and SQLDB Services with Power Systems for SOE/SOR integration

Update: OpenStack HEAT and Chef Server for Power

Up to 100x the power of a classic x86 setup

Two super computers for Oak Ridge and Current DOE Leadership Computers Lawrence Livermore Labs in 2017. Titan (ORNL) Sequoia (LLNL) Mira (ANL) 2012 - 2017 2012 - 2017 2012 - 2017

5x – 10x Higher Application Performance versus Current Systems

>100 PF, 2 GB/core main memory, local NVRAM, Mellanox EDR 100Gb/s InfiniBand, IBM POWER CPUs, NVIDIA Tesla GPUs

• ISV community of 1,400+ • All major Linux distros • Open sourced POWER8 Cloud firmware stack Software Existing Standard Operating Open • Resources for porting Environment Source (System Mgmt) Software and optimizing: Communities Operating OpenPOWERFoundation.org System / KVM

New OSS Firmware OpenPOWER Firmware Community

Hardware OpenPOWER Technology

2014 2015 2016 Work Group CAPI Linux SDK System SW Charter 64b ABI Platform Ref

25g IO Compatibility Charter 25g IO Spec

OpenPOWER I/O Charter Sys I/O Enablement Guide CAPI AFU Intf Spec V1 CAPI AFU Intf Spec V2 Accelerator Charter OpenCL SDK

Memory Charter OPMB Intf. Spec V1

P8 SP010 P8 2U2S P8+ 1U1S P8+ 2U2S Charter Developer Platform Data Reference Reference Reference OpenPOWER ISA Profile V1 OpenPOWER ISA Profile V2 Charter IO Device Architecture V2 IO Device Architecture V3 HW Architecture Coherent Accel Intf Arch Coherent Accel Intf Arch

Compliance Specification Comp Compliance Charter Draft Review WG Spec STD

FSI Specification Charter FSI Spec Proposed Integrated Solutions Work Groups Pers Med

SP010 – Tyan OpenPOWER Customer Reference System AFU – Accelerator Function Unit OPMB – OpenPOWER Memory Bus CAPI – Coherent Accelerator Processor Interface FSI – Field Replaceable Unit (FRU) Service Interface ABI – Application Binary Interface SDK – Software Developer Kit

Technology and Innovators, Integrators, End Users Software Innovators and Partners INTEREST

Discuss areas of Deep understanding of Strategic imperatives and collaboration and synergy workload demands and workload demands for in OpenPOWER. consumption preferences performance and cost of end users. optimization. Sign up for membership Require open software ACTION ACTION and join a work group. Team with innovators and end users to drive and systems with choice Build technical and requirements, engage in and flexibility. business relationships. specific projects. Engage directly on system design options.

Develop Deliver Deploy collaborative innovations collaborative innovations collaborative innovations

OUTCOME with compelling value with compelling value with compelling value

Applications and Services

Systems Management & Processors Cloud Deployment Systems Acceleration & Semiconductor Technology HW/SW Optimization Firmware, Operating System and Hypervisor

Processors

Semiconductor Technology Use Cases

Workload Acceleration Services Delivery Model Advanced Memory Tech POWER8 Linux Network & I/O Accel NEW OpenPOWER

System stack innovations are required to drive Cost/Performance

Anyone may participate in OpenPOWER. Membership levels are designed for those that are investing to grow and enhance the OpenPOWER community and its proliferation within the industry.

. The OpenPOWER Foundation is a Not-for-profit entity with a Board of Directors and a Technical Steering Committee. • Membership levels provide either a default Board of Director position (Platinum) or an opportunity to be elected to the Board (Gold, Silver, and Assoc/Academic members). The Bylaws include additional governance detail. • Technical Steering Committee is formed from Work group Leads and Platinum members.

. Membership options include Platinum, Gold, Silver, and Associate / Academic memberships • Annual fee and dedicated full-time equivalent (FTEs) - verification of FTEs on honor system • Contributors, committers, Work group leads and project leads influence Technical Steering Committee • Associate / Academic level is not available to corporations

Membership Annual Fee FTEs Technical Steering Committee Board / Voting position Level $ USD One seat per member not otherwise Includes board position Platinum $100k 10 represented Includes TSC position Gold members may elect one board May be on TSC if representative per three gold Gold $60k 3 Work group lead members

$20k May be on TSC if Sliver members may elect one board Silver $5k if <300 0 employees Work group lead representative for all silver members Associate & $0 0 May be on TSC if May be elected to one community Academic Work group lead observer, non-voting Board seat

Membership agreement, Bylaws, and IP Rights Policy available for review www.openpowerfoundation.org

Venue ISV OSD OS How to access Server size & access duration PDP - Power P P Ubuntu Intended for IBM PartnerWorld Members VM Size: 1-2 vCPU, 2- Developmen RHEL and Open Source Developers (beta) 4GB RAM, 35GB disk t Platform SLES Virtual Machines for porting and testing Duration: (Cloud) Offering Overview: ibm.com/partnerworld/pdp 2 weeks+ Request access: Longer when ibm.com/partnerworld/wps/ent/pdp/web/MyPro sponsored gramAccess Send a note to [email protected] if questions or issues IIC - IBM P S Ubuntu Intended for ISVs and Systems Integrators VM or bare metal, Innovation RHEL For porting, integration and performance Hypervisor access, Center SLES testing IBM Storage (e.g. One-on-one engagement. Multiple and large Flash, Spectrom) systems possible Duration: 2 weeks+ Request access: ibm.com/partnerworld/iic OSU Open N P Ubuntu Intended for Universities, Researchers, VM Size: 1-8 CPU, 2- Source Lab Fedora Open Source Developers 16GB RAM, 20-120GB IBM Linux Technology Center sponsored disk projects Duration: 1 day to Request access: indefinite ISV=Independent Softwareosuosl.org/services/powerdev/request_hosting Vendor OSD=Open Source Developer P=Primary S=Secondary Current Fee Based or Sponsored Access to Linux on Power

Venue OS How to access Server size & access duration

Site Ox Ubuntu Intended for anyone 1 cpu, 2GB, 20GB Storage Virtual Machine rental for porting and testing $2/day, $32/month Request access: www.siteox.com/lop 2-week free trial OVH Ubuntu Intended for anyone S=8 threads, 4GB RAM, 10GB RAID Fedora Request access: cloud.runabove.com/signup/?launch=power8 $0.05/hour, capped at $32/month 2XL=178 threads, 48GB, 480GB RAID $1.08/hour, capped at $700/month Center Ubuntu Intended for China users only with sponsor by an IBMer Express RHEL Virtual Machine access for porting and testing 1st 2 weeks free. Thereafter needs SLES Request access: https://w3- business justification. 31.ibm.com/support/techline/gcg/tssapp/cscexpress/ Client Ubuntu Intended for IBM Customers VM or bare metal, Hypervisor Centers RHEL One-on-one engagement. Multiple and large systems possible access, IBM Storage (e.g. Flash, SLES Requires an IBM sponsor (i.e. Request form is available inside Spectrom) IBM only) Duration: 2weeks+, defined by Request access: projects http://d27db001.rchland.ibm.com/b_dir/bcdcweb.nsf/mywork?O Fee paid by Sales team or Clients penForm Send a note to [email protected] if questions or issues Lease & Ubuntu Intended for IBM PartnerWorld Developers Physical systems Discount RHEL Request access: Up to 50% discount on hardware SLES ibm.com/partnerworld/wps/servlet/ContentHandler/isv/hardware and significant discount on software mall www.ptopenlab.com SuperVessel: The Open Cloud Service for OpenPOWER Yonghua Lin, Ling Shao

IBM SuperVessel WeChat group Facebook group: SuperVessel OpenPOWER QQ group: SuperVessel Why SuperVessel Cloud

Motivation: To grow the number of developers on OpenPOWER in the world

. Freely opened to all the students in universities, individual developers and ecosystem partners for OpenPOWER.

. Built on the POWER7/POWER8 resource pool

. Built with open technologies, such as OpenStack

. Support advanced technology services, e.g. Docker service, big data service, cognitive computing service 37 Service Roadmap of SuperVessel

Super Marketplace (Online) (Online) (Online) (Preparing) SuperVessel SuperVessel OpenPOWER Super Super Project Cloud Big Data and Enablement Class Team Service HPC Service Service Service Service 1.VM and 1.Big Data 1.X-to-P migration 1.On-line video 1.Project container service service: 2.OpenPOWER courses management 2.Storage service MapReduce new system test 2.Teacher course service 3.Network service (Symphony), SPARK service management 2.DevOps 4.Accelerator as 2.Performance 3.User automation service tuning service contribution 5.Image service management

Docker SuperVessel Cloud Infrastructure

Storage IBM POWER servers OpenPOWER server FPGA/GPU

38 Architecture of SuperVessel

OpenPOWER-SuperVesselCloud Unified Management Platform User account & Admin Virtual point User interface authentication Statistic and analysis interface management management

Cloud Infrastructure Big Data Service OpenPOWER Super Class Super Project Service enablement service service Service

Services for cloud admin OpenStack System Image controller System monitoring Horizon HEAT Nova maintenance management Resource usage Baremetal System analysis Glance Cinder Neutron metering management (IBM Cloud Management)

KVM KVM LxC/ LxC Docker / Docker KVM

Nova Neutron Cinder Nova Neutron Cinder Nova Neutron Cinder Nova Neutron Cinder Nova Neutron Cinder

KVM pool for POWER8 LE KVM pool for POWER8 BE Container pool for Container pool for KVM pool for x86 POWER8 LE POWER8 BE GPFS

OpenPOWE FPGA/GPU R server

39 To expand the SuperVessel via collaboration with OpenPOWER partners

. Multi-site design : Using the multi-region in OpenStack to support multi-site management of SuperVessel cloud. . We are designing the first multi-site for Jiangsu Development and Test cloud on POWER – Shared dashboard, Shared keystone, Shared object store (in future) – Enhance the cache design for performance optimization – Enable the regional administration Unified Management of SuperVessel

Dashboard for cloud Dashboard for Big data …

Beijing Site Partner Site

OpenStack controller OpenStack controller Users Users V V V V V V V V M M M M M M M M

40 SuperVessel Cloud Services

• Virtual machine and Docker services • VPN management • Pre-installed image service We use virtual points for • Multi-site support will be provided in next step resource management

41 SuperVessel Cloud Services

• Pre-installed image service: It could very much make user easier to start development on POWER • Docker is used to provide the pre-installed environment.

42 SuperVessel Big Data Services

• Support MapReduce and SPARK services • Allow user to request different size of cluster

43 Shared file system for Cloud and Big Data service Select Big data computing Cloud Infrastructure Big Data Service • Service framework (Mapreduce, SPARK • Select cluster size HEAT template for big data cluster • Select data folder size

OpenStack controller Servers KeyStone Horizon Nova HEAT Servers Glance Cinder Manila Neutron GPFS FPOGPFS FPO User A User B Docker User A DockerDocker DockerDocker Docker KVM/Docker (Symphony (Symphony(Symphony (Symphony(SPARK) ) ) (Symphony (Web app) ) ) ) POWER7/POWER8 GPFS Folder A Folder B FPO POWER7/POWER8

• HEAT will orchestrate docker instances, subnet and data folder based on user’s request • Manila provides the NFS service using GPFS as backend, and the folder will be mounted via nova- docker (with –v support) • Folder created by Manila could be accessed by the KVM/docker instances created for big data and other purpose

44 SuperVessel Accelerator as Service • Accelerator MarketPlace for developers to Fig.2 Cloud users could apply upload and compile the accelerators for accelerator when creating VM SuperVessel POWER cloud • Allow user to request different size of cluster Fig.1 Accelerator MarketPlace for SuperVessel Cloud

45 Developer quickly grew in the past 100 days

• Users from 30 universities (including 3 from US) • 6 university joint research projects  Supported several courses from universities, e.g.  The Big Data course of Beijing Institute of Technology: 100 students with 38 big data clusters  The virtualization technology 46 course from University of Delaware (US)

Our target is 100,000 users and 100 universities in 2015/2016.

46 Thank You !!

Call for your collaboration to build this CLOUD together