High Productivity Computing With Windows Windows HPC Server 2008 Product Unit Manager Windows HPC Server Microsoft Corporation Trends affecting HPC Make HPC Broadly Accessible Accelerate Innovation • Upgrade IT skills • Multi-core platforms • Easy access for end • Heterogeneous systems users
Trends Impacting HPC Landscape
Data Proliferation Management Control & Align Costs
• Data Explosion • Enmeshing of Enterprise IT and HPC • Managing data • System Management Costs Today’s HPC Environment
Clusters/super High speed Corporate infrastructure computers networking Storage
Financial Information workers Scientists Engineers analysts
Specialized Mainstream technologies Compilers languages Debuggers High Productivity Computing
HPC and IT data centers merge, streamlined cluster management Combined infrastructure
Users with broad access to multiple cores and servers Integrated desktop and HPC environment
Simplified parallel development Unified development environment Microsoft Entry into HPC
Personal Super Computing Built on Windows Server 2003 Microsoft Entry into HPC Addressing Personal And Workgroup Needs End User Applications available for Windows Parallel and HPC Development Tools Ease of Management and Deployment Microsoft HPC Server Today
Broad Reaching HPC Built on Windows Server 2008 Support Traditional & Emerging HPC Larger Cluster support & Top500 Range Better integration for Windows-based Users Broader Developer support with tools and SOA Improved Management and Deployment HPC Pack 2008 Overview
New System Center UI Support for SOA and WCF PowerShell for CLI Management Granular resource scheduling High Availability for Head Nodes Improved scalability for larger Windows Deployment Services clusters Diagnostics/Reporting New Job scheduling policies Support for Operations Manager Interoperability via HPC Profile Systems Job Management Scheduling
Networking & MPI Storage NetworkDirect (RDMA) for MPI Improved iSCSI SAN & parallel Improved Network file system Support in Win2008 Configuration Wizard Improved Server Message Shared Memory MS-MPI for Block ( SMB v2) multi-core New 3rd party parallel file MS-MPI integrated with system support for Windows Windows Event Tracing New Memory Cache Vendors Microsoft HPC MOMENTUM
Wall Street Journal: “The Cray HPC brand name and experience, Gordon Bell: combined with Microsoft's strategy of “Congratulations on making it extending the familiar Windows into the HPC club! You guys environment upward to the server have come a long way, level, gives the Cray CX1 solution #10 especially since Bill’s Keynote strong potential for exploiting the at Supercomputing 2005 in anticipated growth of this market Seattle.” segment.” Ansys: “We have achieved Ferrari: “We are always looking some impressive performance for the most advanced gains on the latest solution technological solutions and, of from Microsoft. The course, the same applies for combination of ANSYS software and engineering. To software and Windows HPC achieve industry leading power- Server 2008 enhances cluster to-weight ratios, reduction in gear computing as an option for our change times, and revolutionary customers who need more aerodynamics, we can rely on HPC capacity.” Windows HPC Server 2008.”
Gartner: “Windows HPC HPCWire Editors’ Choice Server 2008 should be Award: “Best HPC software seriously considered as the product or technology” – sole operating environment or Windows HPC Server 2008 as an alternative mode side- by-side with Linux.”
Improved Efficiency for the Systems Admin
• Simple to setup and manage in a familiar environment – Turnkey cluster solutions through OEMs – Simplify system and application deployment • Base images, patches, drivers, applications • Focus on ease of management – Comprehensive diagnostics , troubleshooting and monitoring – Familiar, flexible and “pivotal” management interface – Equivalent command line support for unattended management • Scale up – Scale deployment, administration, infrastructure – Head node failover – Cluster usage reporting – Compute node filtering • Better integration with enterprise management – Patch Management – System Center Operations Management – PowerShell – Windows 2008 high Availability Services NetworkDirect
A new RDMA networking interface built for speed and stability • Priorities Socket-Based – Comparable with hardware-optimized MPI MPI App stacks App
• Focus on MPI-Only Solution for version 2 MS-MPI – Verbs-based design for close fit with native, high-perf networking interfaces Windows Sockets (Winsock + WSD) RDMA – Coordinated w/ Win Networking team’s Networking long-term plans Networking Networking Networking WinSockHardware Direct NetworkDirectNetworking ProviderHardware Hardware • Implementation TCP/Ethernet ProviderHardware Networking Hardware – MS-MPIv2 capable of 4 networking paths: Networking UserNetworking Mode Access Hardware Layer User • Shared Memory Mode between processors on a motherboard Kernel TCP Mode • TCP/IP Stack (“normal” Ethernet) IP Pass • Winsock Direct (and SDP) - for sockets-based RDMA NDIS Networking • New RDMA networking interface NetworkingMini-port HardwareHardware – HPC team partners with networking IHVs Driver By Kernel to develop/distribute drivers for this new Networking Hardware NetworkingHardware Hardware Driver interface Networking Hardware NetworkingNetworking Hardware Hardware
CCP OS IHV (ISV) App Component Component Component Scenario: Broaden Application Support
V1 (focusing on batch jobs) V2 (focusing on Interactive jobs)
Engineering Oil & Gas Life Science Financial Services Excel Interactive Applications Applications Applications Cluster Applications Structural Analysis Reservoir simulation Structural Analysis Portfolio analysis Pricing Crash Simulation Seismic Processing Crash Simulation Risk analysis Modeling Your applications Compliance here Actual
Job Scheduler WCF Service Router
Resource allocation WS Virtual Endpoint Reference Process Launching Request load balancing Resource usage tracking + Integrated Service activation Integrated MPI execution Service life time management Integrated Security Integrated WCF Tracing
Service Service Service Service App.exe App.exe App.exe App.exe (DLL) (DLL) (DLL) (DLL) Tuning Wizard for Linpack Single pack with HPL 1.0a + Intel math Kernel Lib Demo Microsoft in HPC, A Solution Component
Boosting Developer Productivity - Visual Studio for Parallel Programming Boosting IT Pro Productivity - Range of compilers/ libraries / debuggers - HPC Server built in tools and profilers for Visual Studio - Enterprise management tools - Program SOA based workloads and deploy - MOAB Adaptive HPC Suite on Windows HPC using Visual Studio
Developer IT Pro Productivity Productivity
End User
Boosting End User Productivity Productivity - Rich Ecosystem of ISV applications 100’s of applications are now integrated with Windows HPC Server -Sharepoint HPC oriented web parts for customizing Sharepoint integration with HPCS -XLRunner and UDF Offloading Scale long running Excel spreadsheets using Windows HPC Server Parametric Sweeps End user scheduling of long running calculations
Schedule Job
Jobs
Head Node Job Cluster Mgmt Mgmt Results Resource Scheduling Mgmt
Excel 14 using Excel Office SharePoint Runner (Document Libraries) Customer Momentum Windows HPC Tomorrow
2010
Parallel Extensions
Highly Scalable, Efficient HPC Built on Windows Server 2008 R2 Scalable HPC Infrastructure for 1000+ nodes Customizable management elements for superior control Evolved SOA support for scale and programmability Programming Patterns for HPC with Extensions .NET Continued support for integration with Microsoft Infrastructure What’s coming in R2 Beta 1
Windows HPC Server 2008 R2 dramatically increases the scale of HPC clusters with an out-of-the-box experience to deploy, manage and schedule large scale HPC systems.
Scalability Ease of Use Efficiency
• Reliable deployment at scale • Heat map customization • iSCSI boot for diskless compute Systems • Admin tool experience at scale • Updated node template nodes Management • Support for enterprise and software patching • Saved views and filtering databases wizard • Extensible diagnostics • Filter by node location • Support for custom reporting
• High volume job & task • Job progress tracking • Service job resource balancing Job scheduling • Node prep and release tasks • Activate GPGPU jobs Scheduler • Dynamic parametric tasks • Streamlined job troubleshooting • Improved password management
• Enhanced environment diagnostics • Duplex Message Exchange • Fire & Recollect SOA jobs • Service list and configuration Pattern • Out-of-the-box setup for broker • Graceful service shutdown SOA • Message persistence • Unified WCF configuration Model • Automated broker restart or • SOA job monitoring failover Demo Demo Release Schedule
Q2 2009 Q3 2009 Q4 2009 Q1 2010 Q2 2010 May June July Aug Sept Oct Nov Dec Jan Feb Mar
Community CTP 2 Beta RC RTM Technical Preview Sept 2009 Nov 2009 Feb 2010 H1 2010 May 2009
Windows HPC Server 2008 SP1
RTM Spring 2010 English, Chinese, Japanese Interoperability
• Microsoft’s strategy for growing HPC market – Support for non-CS domain experts • Matlab, R, Excel users doing data analytics – Integration with the enterprise data center – Rich programming models • MPI, SOA, and the new Parallel Computing Platform – Reduce support costs and provide great performance for existing HPC ISVs Interoperability
• Present – New categories of apps on HPC Server Windows – Horizontal applications starting to use clusters – Major HPC apps on Windows
Linux Interoperability
• Future – Windows support helps Windows grow HPC market – New application types flourish – Horizontal applications regularly leverage clusters – Major HPC apps on Windows
Expands HPC reach within your Linux organization – Take advantage of new Windows HPC capabilities for new HPC users Interoperability
• Dynamic workloads require a dynamic datacenter – Moab Adaptive HPC Suite to dynamically change OS • Sophisticated policies and SLAs to determine OS mix – Applications still integrate directly with HPC Server job scheduler • Progress, troubleshooting, integrated security provided by HPC Server
T = 1 T = 2 T = 3 T = 4 Dynamic Adaptation
Expand Cluster A New Model: Utilization by Hybrid Windows/Linux HPC Clusters enabling both Allows you to address both Windows and Windows HPC and Linux demand Linux users to as needed co-exist Enables new HPC cluster users stuck on Windows Workstations Can still take direct advantage of Windows HPC benefits (Excel, ...)
Provides a framework for your cluster to dynamically respond to user demand over time Working Together
http://www.clusterresources.com
http://www.microsoft.com/hpc/ Rocky Mountain Supercomputing Centers Windows HPC in the Future
Seamless Parallelism Addressing the data explosion with petabyte data analytics tools Tackling scalable applications from workstation to cluster to cloud Dynamic and elastic virtualized clusters Uniform programming model for scalable applications Easily deploy and manage scalable applications Capacity without limitations for users of those applications Industry Focused Partners Additional Information
• Microsoft HPC Web site – Evaluate Today! – http://www.microsoft.com/hpc • Windows HPC Community site – http://www.windowshpc.net • Windows HPC TechCenter – http://technet.microsoft.com/en-us/hpc/default.aspx • HPC on MSDN – http://code.msdn.microsoft.com/hpc • Windows Server Compare website – http://www.microsoft.com/windowsserver/compare/default.mspx © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.