IBM United States Software Announcement
212-204, dated June 4, 2012
IBM Platform Symphony V5.2 enables scalable, high- performance grid services for a variety of parallel compute and data intensive applications
Table of contents
122246
78
Planned availability date Description Product positioning Program number
13 Ordering information 16 Terms and conditions
At a glance
IBM® Platform SymphonyTM V5.2 helps you exceed your performance goals with a fast, efficient grid computing environment.
You can realize: •••••
Faster throughput and performance Higher levels of resource utilization Reduced infrastructure and management costs Reduced application development and maintenance costs The agility to respond instantly to real-time demands
For ordering, contact Your IBM representative or an IBM Business Partner. For more information contact the Americas Call Centers at 800-IBM-CALL (426-2255).
Reference: YE001
Overview
IBM Platform Symphony V5.2 is an enterprise-class grid manager for running distributed application services on a scalable, shared, heterogeneous grid. It accelerates a wide variety of compute and data-intensive applications, quickly computing results while making optimal use of available infrastructure.
Platform Symphony's efficient low-latency middleware and scheduling architecture is designed to provide the performance and agility required to predictably meet and exceed throughput goals for the most demanding analytic workloads.
Designed for reliability and having advanced management features, IBM Platform Symphony helps organizations realize improved application performance at a significantly reduced total cost of ownership.
Platform Symphony can help you: •••
Obtain higher quality business results faster Reduce infrastructure and management costs Combine compute and data intensive applications on a single shared platform
- IBM United States Software Announcement 212-204
- IBM is a registered trademark of International Business Machines Corporation
1
Platform Symphony runs on a variety of hardware and operating environments including the latest generation of IBM System x® servers. By pre-qualifying and certifying these platforms in large-scale environments, you take the risk out of deploying mission-critical grid computing applications.
Key features of Platform Symphony : ••••
Ultra-fast, low-latency grid scheduler (less than 1 millisecond overhead) Scalable to 10,000 cores per application and 40,000 cores per cluster Heterogeneous application and platform support Unique resource sharing model that enables lending and borrowing for maximum efficiency
••
Optimized, low latency MapReduce implementation Support of both compute and data intensive problems on a single shared grid of resources
Platform Symphony is available in four different editions tailored to meet your different business requirements.
•
•••
IBM Platform Symphony Developer Edition - Build and test applications without the need for a full-scale grid. IBM Platform Symphony Express® Edition - The ideal solution for departmental clusters. IBM Platform Symphony Standard Edition - Enterprise class performance and scalability. IBM Platform Symphony Advanced Edition - Use for distributed compute and data intensive applications.
Key prerequisites
- •
- A physical grid computing environment comprised of IBM System x or similar
servers.
•••
Cluster nodes should be preinstalled with supported operating environments. Cluster nodes should be connected via a fast TCP/IP network infrastructure. Management hosts on the cluster should share a common network file system.
Planned availability date
June 15, 2012
Description
IBM Platform Symphony V5.2 is a high-performance grid middleware and management solution that runs on your choice of hardware and operating environments. You can use Platform Symphony to run preintegrated applications available from a variety of independent software vendor (ISVs), or easily adapt and accelerate their own compute and data intensive parallel workloads on a grid making them agile and flexible.
Performance and scalability
Platform Symphony is one of the industry's fastest and most scalable serviceoriented architecture (SOA) grid management software. IBM Platform Symphony provides organizations with a competitive advantage by solving a wide range of parallelizable compute and data-intensive business problems.
••
Scale to 10,000 cores per application Sub-millisecond latency for grid services
- IBM United States Software Announcement 212-204
- IBM is a registered trademark of International Business Machines Corporation
2
••
Throughput to 17,000 tasks per second Reallocate up to 1,000 grid services per second
Platform Symphony environments can scale by linking multiple clusters to environments comprised of hundreds of thousands of cores. For organizations that need a fast service-oriented computing infrastructure, Platform Symphony's performance and scalability provides a clear competitive advantage.
Flexible resource sharing
A key advantage of Platform Symphony is its sophisticated hierarchical resource sharing model. Platform Symphony's ability to lend and borrow resources, based on policy, guarantees users access to owned resources when needed. Departments can also easily share resources not in use with others, and borrow resources dynamically as needed. By avoiding the need for application-specific siloes, users enjoy a better quality of service with reduced infrastructure and management costs. Whether you are deploying on LinuxTM , WindowsTM , Windows HPC Server, UNIXTM , or all of the above, you can manage your environment as a single shared resource pool with IBM Platform Symphony .
Support for compute and data intensive workloads
IBM Platform Symphony is optimized for both compute and data intensive parallel workloads. It provides a rich set of client and server side APIs for a variety of parallel computing problems alongside an optimized Hadoop compatible MapReduce implementation along with other data handling innovations to accelerate the processing of large distributed data sets. Both interfaces take advantage of Platform Symphony's low-latency middleware and resource orchestration capability to deliver dramatic gains for both compute and data intensive applications. By incorporating the function of scheduling both compute and data related tasks into a single smart scheduler, tasks can be dispatched with an awareness of data locality resulting in dramatic gains in performance and efficiency as applications avoid the needless movement of data across congested networks.
Rapid response for business critical workloads
IBM Platform Symphony is able to react instantly to changes in application demand. With the flexibility to adapt when priorities change, IBM Platform Symphony can reallocate over 2,000 compute engines per second to different workloads depending on configurable sharing policies and application priorities. This translates into better application performance, better utilization, and an ability to respond quickly to business critical demands.
Ease of application and data integration
Regardless of your development environment, IBM Platform Symphony has you covered. Well documented APIs enable fast integrations for applications written in the following languages:
•••••
C++ C#, .NET JavaTM Excel COM Native binaries
Extensions provided for MicrosoftTM Visual Studio Professional support compilation free integration of .NET assemblies. Developers can use a step-by-step wizard to integrate and test applications end-to-end without needing to be expert in the Platform Symphony APIs. For Java developers, the Eclipse integrated development environment (IDE) is supported as well.
For data intensive applications, Platform Symphony Developer Edition and Platform Symphony Advanced Edition provide an optimized Hadoop compatible MapReduce implementation enabling most Hadoop applications to run without modification.
- IBM United States Software Announcement 212-204
- IBM is a registered trademark of International Business Machines Corporation
3
Plug-ins are also provided for distributed file systems including IBM General Parallel File System ( GPFSTM ), Apache HDFS, and Appistry. Platform Symphony also supports integrations with third-party in-memory data grid solutions.
A free downloadable IBM Platform Symphony Developer Edition makes developers productive immediately, enabling them to develop and test CPU or graphics processing unit (GPU) based applications on their operating system of choice without the need for a production grid. Platform Symphony Developer Edition also enables you to develop and exercise data intensive MapReduce applications with just one or two compute hosts. It is available from developerWorks®
http://www.ibm.com/developerworks/
Accessibility by people with disabilities
A US Section 508 Voluntary Product Accessibility Template (VPAT) containing details on accessibility compliance can be requested at
http://www.ibm.com/able/product_accessibility/index.html
Product positioning
Platform Symphony is part of a family of cluster and grid workload management solutions from IBM .
These products include: ••
IBM LoadLeveler® IBM Platform LSF®
IBM LoadLeveler is a job scheduler that controls the scheduling of batch jobs. LoadLeveler matches job requirements to the best available compute resources for execution and runs on IBM POWER® machines and x86-based Linux systems. LoadLeveler is an excellent choice as a workload management system when you are running applications already integrated with or supported by LoadLeveler on POWER or Linux based systems.
IBM Platform LSF family is comprised of a batch workload scheduler similar to LoadLeveler . A number of other components are also available for Platform LSF including graphical workflow management tools, application-centric portals, monitoring and analysis tools, and tools for scheduling commercial software license features. While there is some intersection in functionality between LoadLeveler and Platform LSF , Platform LSF may be an appropriate choice when you need these specific additional capabilities or when you wish to deploy a Windows , or mixed Windows , Linux , or UNIX (including AIX® ) environment involving hardware or operating systems not supported by IBM LoadLeveler .
Platform Symphony is different from all of the solutions mentioned above. Key differences: •
••
Platform Symphony is most often used with programmatic (API driven) workloads. Whereas a batch scheduler such as Platform LSF can schedule jobs in seconds or minutes, Platform Symphony can schedule tasks in milliseconds. Because of this difference, Platform Symphony is frequently described as supporting online or near real-time requirements. Whereas Platform LSF jobs are typically command line driven or integrated via a C language API, Platform Symphony offers a rich set of client and server side APIs including C++, C#, .NET, Java , Excel COM, and other programming interfaces. Platform Symphony incorporate service-oriented middleware specifically optimized for HPC environments. With a batch workload scheduler, while there are exceptions, the resources assigned to a job are normally static and do not adjust at run time. With Platform
- IBM United States Software Announcement 212-204
- IBM is a registered trademark of International Business Machines Corporation
4
Symphony , resource assignments are fluid and service instances can be reallocated very quickly.
•
•
Job runtimes in a batch computing context often run for minutes or even hours. Task runtimes in Platform Symphony , by contrast, are often very short and execute in milliseconds. As a point of reference, the largest Platform LSF clusters complete a few million jobs per day while the largest production Platform Symphony clusters run in excess of 400,000,000 tasks per day. Another key differentiator is related to data handling. Batch workload schedulers generally rely on a shared file system to move data between hosts or mechanisms such as pre-execution scripts to move data by other means. Platform Symphony by contrast supports fast in-band data movement and a variety of optimizations for data handling such as direct-data transfer, data compression, and data-aware scheduling.
- •
- Platform Symphony dynamically provisions services in response to application
demand, however it does not provision the actual cluster nodes or host operating environments. Because of this, Platform Symphony will normally be sold along with cluster management tools or into environments that already have cluster management tools in place.
Because of these key differences, Platform Symphony typically runs different types of analytic workloads than Platform LSF . Workloads that require very fast scheduling, low latency, high throughput and scale, and fast, agile resource sharing are likely to be best served by Platform Symphony .
Some unique characteristics of Platform Symphony relative to other highperformance SOA grid middleware solutions are:
- •
- Heterogeneity - Platform Symphony is agnostic of operating systems,
programming languages, and software frameworks making it easier to integrate heterogeneous environments. Platform Symphony can run not only multiple compute-oriented workloads, but also data-oriented MapReduce workloads on the same grid sharing resources between them.
•
•
Scale and performance - Platform Symphony employs a unique "push" based scheduling model that helps it deliver better scalability and faster response times. Sharing - A unique resource sharing model at the heart of Platform Symphony provides users or lines of business instant access to owned resources when needed, but also facilitates dynamic resource sharing to help ensure that compute hosts are fully utilized at all times.
As mentioned earlier, Platform Symphony is also available to run MapReduce jobs (a component of Apache Hadoop) via an optimized, low-latency MapReduce capability built into the Platform Symphony Advanced Edition.
If you deploy Hadoop clusters, you may want to consider IBM InfoSphere® BigInsights for specific requirements where you want to run service-oriented workloads on the same cluster as Hadoop jobs, or where you have special requirements that demand low-latency scheduling to speed up short duration MapReduce jobs, Platform Symphony may be an appropriate solution.
Customers may choose to deploy Platform LSF and Platform Symphony on the same shared cluster infrastructure but there are some limitations. If deploying mixed Platform Symphony and Platform LSF clusters contact a Platform Symphony product specialist for configuration advice and assistance.
Reference information
Refer to Preview Announcement 212-211, dated June 04, 2012 .
- IBM United States Software Announcement 212-204
- IBM is a registered trademark of International Business Machines Corporation
5
Program number
Program number
Program
- name
- VRM
- 5725-G86
- 5.2.0
- IBM Platform Symphony
Additional information
Recommended platforms
Platform Symphony is supported on most IBM and third-party x86 and x64 hardware platforms. Platform Symphony installations are typically large and span multiple servers. Because of this, rack-based solutions tend to be the most practical.
Recommended IBM hardware platforms for Platform Symphony include IBM System x racks, blades, and iDataPlex® systems.
For customers running data intensive workloads, such as MapReduce on the Plaform Symphony cluster, the IBM systems noted above, supporting larger numbers of independent disk drives, tend to yield the best application performance. The best system should be selected in consultation with System x and Platform Symphony specialists. The optimal configuration depends on the scale of the cluster and the nature anticipated application workloads.
IBM Power® System servers running AIX V5 are compatible with Platform Symphony but from a client perspective only. This means that client applications running on AIX can call Platform Symphony compute services, but the services themselves are assumed to be running on a grid comprised of System x or compatible systems.
Platform Symphony installations usually involve a predefined number of management hosts and a larger number of compute hosts. Management hosts typically have more robust configurations in terms of CPU, memory, and disk since management hosts run the session management, a key component in the distributed middleware. As requirements changed, the logical set of management hosts can easily be changed without physical reconfiguration of the cluster.
A typical installation might involve half a dozen management hosts and several hundred compute hosts. The number of management hosts is a function of the number of discrete application types that will be run on the grid concurrently. Each application will generally start its own session manager process requiring significant resources on the management host. Part of the reason for separating management functions across multiple hosts is to ensure adequate performance of key system services during periods of high load. For example, services such as the web-based console service and service director typically run on their own nodes so that they are not affected by session managers that can easily consume all of the physical resources of a management host.
Large grids also benefit from running multiple instances of repository services across multiple management hosts and across multiple geographies so that as applications demands change, new services can be rapidly propagated to compute hosts from multiple repository service instances running in parallel.
To facilitate automatic failover between services on these management nodes, a shared filed system is typically implemented. Depending on the scale of the grid and the anticipated network file system I/O, this shared file system may be implemented as a Network File System (NFS) export from one of the compute hosts or as a separate dedicated filer.
Platform Symphony session managers journal their activities as they run so that in the case of a software failure, workloads can be recovered from where they left off without the need to recompute results already obtained. For large simulations
- IBM United States Software Announcement 212-204
- IBM is a registered trademark of International Business Machines Corporation
6
involving millions of tasks, this journaling activity can become a performance bottleneck and these types of environments benefit from solid-state drives or other technologies to accelerate synchronous writes.
Education support
The following classes are available:
Course code
Course title
Classroom
H050G H060G H070G H071G H072G
IBM Platform Symphony 5.x Administration IBM Platform Symphony 5.x Application Programming IBM Platform Symphony-MapReduce Administrator IBM Platform Symphony-MapReduce Developer IBM Platform Symphony-MapReduce Data Management
Instructor-led online (ILO)
H150G H160G H170G H171G H172G
IBM Platform Symphony 5.x Administration IBM Platform Symphony 5.x Application Programming IBM Platform Symphony-MapReduce Administrator IBM Platform Symphony-MapReduce Developer IBM Platform Symphony-MapReduce Data Management
IBM training provides education to support many IBM offerings. Descriptions of courses for IT professionals and managers are on the IBM training website
http://www.ibm.com/services/learning/
Call IBM training at 800-IBM-TEACH (426-8322) for catalogs, schedules, and enrollments.
Offering Information
Product information is available via the Offering Information website
Also, visit the Passport Advantage® website
http://www.ibm.com/software/passportadvantage
Publications
The following publications are shipped with the product and available on the web. The following publications and documentation are provided for Platform Symphony in both web-friendly and PDF formats and are included in the Platform Symphony knowledge center accessible from the Platform Symphony web-based console.
••••••••
Easy Cluster Verification Steps to Get Started Connector for Microsoft Excel User Guide IBM Platform Symphony Foundations GPU Application Development Guide Installing the IBM Platform Symphony Client on UNIX Installing the IBM Platform Symphony Client on Windows Installing your IBM Platform Symphony Cluster Cluster and Application Management Guide