SPECIAL STUDY D2 Interim Report: Development of a Supercomputing Strategy in Europe (SMART 2009/0055, Contract Number 2009/S99-142914) Earl C. Joseph, Ph.D. Steve Conway Chris Ingle Gabriella Cattaneo Cyril Meunier Nathaniel Martinez Author(s) IDC: Earl C. Joseph, Ph.D., Steve Conway, Chris Ingle, Gabriella Cattaneo, Cyril Meunier and Nathaniel Martinez Deliverable D2 Interim Report Date of July 7, 2010 delivery Version 2.0 Addressee Bernhard FABIANEK officers Performance Computing Officer A P.508.872.8200 F.508.935.4015 www.idc.com www.idc.com F.508.935.4015 P.508.872.8200 A European Commission DG Information Society Unit F03 — office BU25 04/087 25 Avenue Beaulieu, B-1160 Bruxelles Tel: Email: Contract ref. Contract Nr 2009/S99-142914 Global Headquarters: 5 Speen Street Framingham, MA 01701 US Framingham, MA Street Global Headquarters: 5 Speen Filing Information: April 2010, IDC #IDCWP12S : Special Study IDC OPINION This is the Interim Report (Deliverable D2) of the study, "Development of a Supercomputing Strategy in Europe" by IDC, the multinational market research and consulting company specialized in the ICT markets, on behalf of DG Information Society and Media of the European Commission. This D2 Interim Report presents the main results of the supercomputer market and industry analysis and the overview of main the technology trends and requirements (WP1 and WP2 of the study workplan). With the terms supercomputers or HPC (High Performance Computers) in this study, we refer to all technical computing servers and clusters used to solve problems that are computationally intensive or data intensive, excluding desktop computers. The results presented in this report will feed into the next phase of the study, which is the development of the Strategic Agenda for Supercomputing in Europe, to be presented in the Final Report of the study. We greatly thank the Technical Working Group contributors for their ideas, insights, and suggestions, which helped to crystallize the findings in this report. The Technical Working Groups are made up of members of this project's Technical and Strategy Committee, along with IDC representatives. External team members included: Hervé Mouren, TER@TEC and Christian Saguez, TER@TEC; Richard Blake, STFC Daresbury Laboratory; Arndt Bode and Herbert Huber, Leibniz-Rechenzentrum/LRZ Munich; and Friedel Hossfeld, Forschungszentrum Jülich. Findings include: ` Europe is under-investing in HPC, while other nations grew their supercomputer Europe is under investing in HPC, investments dramatically even in 2009, the most difficult year of the global while other nation’s economic recession. investments grew their supercomputer investments ` HPC use is indispensable for advancing both science and industrial dramatically competitiveness. ` Supercomputing revenues (annual spending on systems priced above €375,000, or $500,000) increased by 25% worldwide in 2009, but decreased 9% in Europe. ` HPC research funding in Europe includes a diversity of EU, national and regional programs, and few countries have a coherent HPC development strategy. ` HPC stakeholders from research, industry and academia rank U.S. and Japanese HPC research programs ahead of Europe's research programs. ` There is strong support for expanding PRACE (Partnership For Advanced Computing in Europe) to respond to growing industry demand for HPC capacity. ` The transition to petascale and exascale computing creates opportunities for Europe's scientific and computing communities to return to the forefront of development for the next generation of research and HPC software technologies. #IDCWP12S ©2010 IDC EXECUTIVE SUMMARY Supercomputing has become a key element for the competitiveness of knowledge- European stakeholders from based economies. But in recent years, Europe has under-invested in High industry, research and Performance Computing (HPC), both in annual spending on computing resources and academia believe that in research investments, while other nations' investments grew even during the Europe has a chance to jump back at the economic recession. European stakeholders from industry, research, and academia forefront of the believe that Europe has a chance to jump back to the forefront of development for the development of the next generation of next generation of HPC-based research, and for the applications and other software HPC technologies required for the transition to petascale and exascale computing. This section presents key findings on the HPC market in Europe, based on the research carried out by IDC on behalf of DG Information Society and Media of the European Commission. Again, the main goal of this project is to develop a supercomputing strategy for Europe, and this Interim Report presents the results of IDC's field research carried out in the period December to March, 2010. The Final Report due out in a few months will present the recommended strategy. The research conducted for the Interim Report included a broad survey of the European HPC stakeholders (scientific and engineering end users, vendors, and others); in-depth interviews with supercomputing stakeholders from funding agencies and research centers in Europe, the U.S. and Japan; and case studies of four major HPC centers in Europe, to illustrate the situations of centers of this kind. Although the main focus of this report is on the HPC market and industry in Europe, the supercomputing market is global in scope. To help put the European HPC market in perspective, this report also presents research findings and other information about HPC in other major world regions, particularly North America, Japan, and other areas of the Asia-Pacific region, including developments in China and India. HPC Market Size According to IDC's HPC tracking research, the worldwide market for HPC systems was worth about €6.45 billion in 2009. And although the recession pummeled lower- priced HPC systems, the market for high-end HPC systems grew substantially even during the difficult global recession year of 2009, when revenue for HPC systems priced above $500,000 (€375,000) increased by 25% and revenue for HPC systems priced above $3 million (€2.25 million) jumped an impressive 65%. EU HPC system revenue in 2009 amounted to about €1.9 billion ($2.5 billion), or 29.4% of worldwide revenue, compared with 49.5% for North America, 10.5% for the Asia-Pacific region without Japan, and 9.5% for Japan. EU's share of the worldwide market slipped nearly 2% from the pre-recession 2007 high of 33.1%. During the same period (2007-2009), North America's market share grew nearly 2%, from 47.8% to 49.5%. IDC forecasts that the worldwide market for HPC systems will expand at a healthy 6.3% compound annual growth rate to surpass €8.3 billion ($11 billion) in 2013. The Broader HPC Ecosystem The addition of the other categories, including storage, service, application software, and middleware, to system revenue pushed the aggregate value of the worldwide HPC market in 2009 to €12.9 billion, and to €17.3 billion as forecast for 2013. The ©2010 IDC #IDCWP12S revenue growth rate for the non-computer categories has been, and is projected to remain, higher than for the HPC systems. Storage in particular has been growing at a pace several percentage points higher than HPC systems, owing to the "data explosion" associated with running increasingly large, complex HPC problems and workloads. Aggregate HPC revenue (spending) within the EU amounted to €3.3 billion in 2009. IDC forecasts that total HPC spending IDC forecasts that total HPC spending within the EU will more than double to surpass within the EU will €4.5 billion by 2013. As with the worldwide forecast, the compute category is more than double to expected to experience the lowest growth rate among the revenue categories. surpass €4.5 billion by 2013 "Supercomputers" Segment For Systems Priced at $500,000/€375,000 and Higher In the "supercomputer" segment for HPC systems priced at $500,000/€375,000 and up, IBM was most often the EU market share leader in 2005-2009, with HP jumping ahead of IBM only in 2008 and otherwise remaining a close second. Together, IBM and HP captured 78% of the EU market in this price band in 2009. The third-place vendor, Bull, accounted for only about 5% of the market. The Very High End Bracket For Systems Priced at $3 Million/€2.25 Million and Up The topmost price band, for HPC systems sold for more than $3 million/€2.25 million, is especially relevant for HPC leadership initiatives. In 2009, the most recent historical year, IBM easily led all other vendors by capturing nearly half (46%) of EU revenue for HPC systems in this price band. HP finished second, with about half of IBM's market share (23%), followed by Cray (11%), Bull (9%), and SGI (6%). Only about 5% of the revenue in this highly competitive segment went to "others." Over the last five years (2005 to 2009), the EU market for these high-end supercomputers grew at a rate of only about 2.7% a year, while the U.S. grew by 14% and the world as a whole grew by 13%. Clearly, the EU is not investing at the same level as many other nations in the critical over-$3 million supercomputer category. HPC Server Market Suppliers During the period 2005 to 2008, HP was consistently the leading supplier of HPC systems in the EU region, with IBM running second and Dell a more distant third. In 2009, IBM exceeded HP by a small amount. In 2009, the EU HPC market share for these three vendors totaled 77%, leaving less than one-quarter of the market for all others to share. The only EU-based vendor on the list, Bull, had 2009 EU HPC market share of only 1.8%. Note, however, that the "Other" category was really the third-place finisher, ahead of Dell with 11.6% market share.