IBM Grid Technical Strategy & Related Research Projects

Grid Computing

Jean-Pierre Prost Senior Technical Staff Member IBM Montpellier, France

Outline

Grid and Virtualization

Examples of Commercial Grids

Grid Components and IBM Grid Strategic Directions

IBM Grid Research Projects

ApplicationApplication ApplicationApplication ApplicationApplication ApplicationApplication ...... ApplicationApplication 1 1 2 2 3 3 4 4 N N

Grid Middleware

“Virtualized”“Siloed” infrastructure: architecture: • Creates• Higher a virtual costs application (capital + operational)operating, storthrouagegh limited& collaboration pooling of environmentIT assets across silos • Virtualizes• Challenging application cross services organization execution collaboration • Dynamically• Limited fulfills responsiveness requests over due a virtualto more pool manual of system scheduling resources and provisioning • Offers an adaptive, self-managed operating environment that offers high availability

3 IBM Grid Technical Strategy & Research Projects © 2005 IBM Corporation Grid Computing Grid and Virtualization Virtualize Outside the Enterprise IBM is focused on solutions that help clients realize value from the full spectrum of grid computing solutions Virtualize the Enterprise Suppliers, partners, customers and external resources

Virtualize Unlike Resources Enterprise wide Grids, Information Insight, and Virtualize Like Global Fabrics Resources Heterogeneous systems, storage, and networks; Application-based Grids Cluster

Single System (Partitioning) Simple Sophisticated (2-4) (4+) Homogenous systems, storage, and networks Homogenous Heterogeneous Single Organization Multiple Organizations Tightly Coupled Loosely Coupled

4 Grid Computing Realizing Increasing Levels of Business Value Virtualize Outside the Enterprise

Virtualize the Enterprise

IT Simplification for Virtualize Unlike Enterprise Optimization Resources Asset Utilization Workload Prioritization Virtualize Like Resources Infrastructure Flexibility Enabling Business Resiliency

Cluster Aggregating Information Business Insight Single System (Partitioning) and Collaboration Simple Sophisticated (2-4) (4+)

Application Acceleration Time to Results and Higher Quality

5 Grid Computing

Characteristics of Grid Evolution Virtualize Outside the Enterprise

Virtualize the Enterprise

Virtualize Unlike Resources • Multiple Organizations Virtualize Like • Meta-scheduling Resources • Workload Prioritization • SLA Attainment • Advanced Scheduling • Orchestration • Multiple Apps/Resources • Information On Demand Cluster • Integrated Security Single System • Provisioning (Partitioning) • Workload Management Simple Sophisticated (2-4) (4+) • Information Virtualization • Cluster Scheduling • Single Application • Homogenous • Storage/File Virtualization

6 Grid Computing Across the Spectrum: IBM Customer References Virtualize Outside the –Range of technologies based on best fit for task Enterprise –Heterogeneous implementations address customer requirements Virtualize the –Open approach using standards-based tools, Enterprise IBM products and partner products ZJ Transportation Virtualize Unlike Resources

China Grid Virtualize Like Resources

Cluster

Single System (Partitioning) Simple Sophisticated (2-4) (4+) National Digital Mammography Archive IBM

Northeast US Financial Services

7 Grid Computing

EngineeringEngineering andand DesignDesign MAGNA STEYR AutomotiveAutomotive

Challenge Too much time required to effectively run clash tests between complex sub- assemblies which impacts quality of the end product and getting the product to market on time. Benefits: Significant performance improvement (72 – 4hrs) Too much administrative time required Risk and Error Reduction from design engineers Cost Reduction Solution Increased accuracy of data improves quality and reduces late changes) Grid enabled clash environment. IBM developed code and services using Improved Time to Market Platform Computing LSF w/ Dassault Faster evaluation of design alternatives Systemes CATIA & ENOVIA DMU More accurate and timely product applications providing clash detection development analysis.

“Grid“Grid technology technology from from IBM IBM and and Pl Platformatform Computing Computing reduced reduced the the ti timeme required required for for our our clash clash testing testing fromfrom 72 72 – – 4 4 hours hours and and contributed contributed significantly significantly to to enhancing enhancing our our design design quality, quality,”” said said Dr. Dr. Heinz Heinz Mayer,Mayer, MAGNA MAGNA STEYR. STEYR.

8 Grid Computing

Enterprise Optimization Audi/MSC/IBM Proof of Concept Automotive Distributed Simulation Data and Process Management Challenge Reuse defined simulation schemes to enhance engineer productivity and innovation. Improvement of integrated, secure and collaborative design environment across the distributed enterprise Technology Benefits: Shared access for engineers from all enterprise sites to all A robust, scalable multi site Simulation Data and Process simulation data from cars. Central coordination point for data Management (SDM) infrastructure and content Usage of standard applications to design a multi site collaboration environment Data sharing and data exchange capability for multi site collaboration to be implemented between Audi and SEAT Business Opportunities: Solution Flexibility to handle more projects, scale up engineer and design work An information grid enabling collaborative engineering Global view of all CAE data to assess entire vehicle at any Multi site engineering data sharing based on: given time in the design process – MSC.SimManager Reuse design work, maintain historical data on all analysis – IBM eServer and TotalStorage generated – IBM Data Management replication and federation More accurate and timely product development technology based on WebSphere Information Integrator Improved productivity of the Engineering Analysis Office and WebSphere MQ Improved Time to Market and faster evaluation of design – IBM integration services alternatives in collaborative tasks

“Collaborating with our engineering team at SEAT will become more and more important due to the common activities inside the Audi “Collaborating with our engineering team at SEAT will become more and more important due to the common activities inside the Audi brand group. The new technology we are developing with IBM and MSC.Software will help us collaborate more effectively and build a real brand group. The new technology we are developing with IBM and MSC.Software will help us collaborate more effectively and build a real global engineering collaboration, reducing travel and taking both time and cost out of the product development process” global engineering collaboration, reducing travel and taking both time and cost out of the product development process” -- Dr.-Ing. Ulrich Widmann, head of functional design crash, occupant safety, NVH analysis AUDI AG -- Dr.-Ing. Ulrich Widmann, head of functional design crash, occupant safety, NVH analysis AUDI AG

9 Grid Computing

Business Analytics Sal. Oppenheim Financial

Challenge:

► Sal. Oppenheim is one of the leading independent private banks in Europe. Sal. Oppenheim is Technology Benefit upgrading its pricing system for equity derivative trading. High performance grid cluster based on latest

► Traditional dedicated compute resources handle Platform Symphony grid software and IBM the dynamic structure of the large amount of risk cluster technologies. and pricing calculations.

► Unbalanced peak performance sized solutions Solution is integrated into the IT management would be too inefficient in terms of price concept of Sal. Oppenheim performance.

► The grid has to deliver a service level based risk Business Benefit and price calculation solution. Product development for equity derivatives Solution: Flexible high speed grid solution which can ► one of a kind flexible grid system developed by Sal. Oppenheim, Platform grow with the demands of trading volumes. Computing, the IBM business partner Morse and the IBM Grid computing team. The service level based risk and pricing solution guarantees performance and availability of the – IBM x336 critical business application – IBM Cluster Management Software – Platform Symphony

“„Our financial trading teams are working in a very volatile and extremely demanding market. Processing of large amounts of data and numerical intensive simulations in highest quality and speed decide between win and loss. With the introduction of the grid solution we believe to be able to fulfill the demand we are getting from the business side. With the grid solution, we will have in the mid-term an IT infrastructure that will allow us to react flexibly and grow as demand grows with acceptable cost“, said Arno Radermacher, Head of IT Investment10 Banking at Sal. Oppenheim. Grid Computing

Business Analytics Hewitt Associates LLC Financial Challenge Create Grid Computing environment to: – Contain expenses for CalcEngine valuations – Maintain or improve availability, response time & scalability Benefits: – Insure personal-data security • Efficiently uses of the combined – Capitalize on existing application code processing power of their – Cooperate with z/OS Sysplex CICS Calling heterogeneous environment Environment • Experienced an immediate 10% faster – Enable smooth and orderly migration to response time with the first change application deployment Solution • Open architecture enables Hewitt to Development of cross z/OS and x-Series platform easily deploy additional applications environment to enable more efficient and faster • Increased processing speed reduced running of application with compute-intensive cost per transaction process. • Reduced operational costs improves Includes DataSynapse GridServer competitiveness in their industry segment

“IBM“IBM the the BladeCenters BladeCenters running running Linu Linuxx allowed allowed us us to to quick quicklyly migrate migrate proprietary proprietary applications applications using using Smalltalk.Smalltalk. In In the the first first day, day, we we were were able able to to handle handle 8,000 8,000 client client pension pension calls--on-demand.” calls--on-demand.” --Perry--Perry Cliburn, Cliburn, CIO CIO of of Hewitt Hewitt Associates Associates

11 Grid Computing

Engineering and Design SIEMENS Mobile Communications Challenge:

► Increasing demand for rapid development of mobile phone software caused a Benefits: continuous growth of compute and storage infrastructure Siemens Mobile can now set up a scalable system which with the potential to adapt to their ► The existing IT architecture and the current current and future demands for mobile phone set up of the software revision system were software development. not scalable enough for a predictable quality of service. Potential for better manageability and failure detection as the critical set up of the individual Solution: development environments is now decentralized and distributed to the individual grid nodes ► IBM Global Services to implemented a “Compile Grid” based on industry standards Grid software The compute intensive compilations are separated from the centralised CVS installation. ► IBM xSeries e325 dual Opteron servers This allows the compute intensive compilation workload to be distributed to the least used ► SUSE Linux resource in the Grid ► Platform LSF

"We chose a Linux grid on an IBM eServer system for our phone software development to help us increase our quality of service levels. This fits to our continuous growth perfectly" Ulf van Laak, Engineer of Development at Siemens ICM Kamp-Lintfort

12 Grid Computing

Yamanouchi Pharmaceutical Co., Ltd Research & Development Pharmaceutical Challenge Increase the speed of analysis processing for in silico drug discovery

Solution Technology Benefits: Virtualize a large number of office PCs by using the Shorten the process time from Grid Middleware. This provides a distributed high- 3-4 weeks to 3-4 days speed , large volume processing system for the drug discovery research system. Business Benefits: Expedite the research into drug ►H/W: IBM xSeries Linux Cluster, ThinkPad® discovery

► ： Grid Middleware Platform ActiveCluster Improve TCO by utilizing existing ►Application : in-silico screening IT resources.

“The“The IBMIBM GridGrid ComputingComputing implementation,implementation, bbasedased onon existingexisting ITIT resources,resources, allowsallows usus toto accelerateaccelerate ourour researchresearch inin thethe drugdrug discoverdiscoveryy processprocess throughthrough largelarge scalescale analysis.”analysis.” MasayaMasaya Orita, Orita, Ph.D.Ph.D. InstituteInstitute forfor DrugDrug DiscoveryDiscovery ResearchResearch YamanouchiYamanouchi PharmaceuticalPharmaceutical Co.,Co., Ltd Ltd

13 Grid Computing

IT Frontier Corporation Enterprise Optimization Grid Delivery Technology realized higher quality contents distribution system with minimum investment zChallenge Verification and utilization of the higher quality contents distribution system which used the existing IT infrastructure effectively. Technology Benefits: zSolution By IBM P2G “IBM Peer-to-Group Media Broadcast （IBM ¾ Dynamic load sharing of Network P2G）” virtualized and optimized the existing and distribution server network and the distribution server and enabled ¾ Distribute high quality and large higher quality volume contents with existing IT contents distribution infrastructure. 9IBM Peer-to-Group Media Broadcast （IBM P2G Business Benefits: ） 9Client :100 office PCs ¾ Improve internal communication 9Contents : Education Materials (Movie) and responsiveness of Enterprise Live Broadcasting (400Kbps) by real time information sharing. IBM P2G which enable quality animation distribution using the existing infrastructure have a potential of changing fundamentally the way that should be information shared in an organization. IT Frontier Corporation Network Division General Manager Mr. Niwa

14 Grid Computing

IBM Grid Components

ProvisioningTivoli Provisioning & Orchestration Manager Tivoli Intelligent Orchestrator Workload Information WebSphere XD Virtualization Application Application Application Application Application Virtualization WebSphere Information Integrator 1 2 3 4 … N Tivoli Workload Scheduler (Distributed Data, Caching, Replication (Dynamic Scheduling) DB2 Content Manager IBM LoadLeveler Federation, Transformation)

Grid Middleware System Storage eWLM GPFS, SANFS Virtualization VirtualizationNFS V4, SAN VC (WorkloadPartitioning Management, Partitioning) (Virtual Volume Management)

Web Service Foundation & Resource Model

15 Grid Computing Workload Virtualization Distributed Network of Schedulers

Policy driven brokering

Scheduling scheduling exceptions Provisioning &

submissions W W Orchestration W W

te e l re e u d g / fi d n d o W W W W a c re W Resource Resource service level W W agreements Resource W W Resource resource pool

Workload service level exceptions Management

Policy-driven cache and replica management Autonomic data placement Federation of data stores to utilize existing resources

App. App. App. App.

Request & Response & QoS Delivered QoS Virtual Data Placement Store Data Manager Policy

AutomaticAutomatic Value SenseSense andand ManagingManaging thethe demanddemand ResponseResponse Real-timeReal-time andand supplysupply ofof gridgrid semi-semi- resourcesresources automaticautomatic responseresponse RespondRespond toto Tivoli Intelligent alertsalerts Orchestrator ScheduleSchedule withwith inputinput --OfflineOffline performanceperformance analysisanalysis -- Application Application instrumentationinstrumentation Tivoli Provisioning ScheduleSchedule Manager GridGrid From deployment to ConfigureConfigure GridGrid automatically sensing and responding to the DeployDeploy GridGrid environment Time

Standards Body Cooperation Common Resource Model

Web Service Foundation

Web Services

Aligning IBM Research with the Grid Strategy, Product Development, and Customer Needs

Common Resource Model

Provisioning Web Service Foundation Workload & Virtualization Orchestration

Application IBM Research Information Grid Computing Institute Development Virtualization

Product Customers Design Centers Development for (SWG,STG, IGS) on demand business

Web Local WebSphere Servers Application Server Grid DB2 Trans. app. 1 Database Server Grid Trans. app. 2 Internet ParallelApplication

Remote Data WAS Center / enabled Controller / MAPE On Demand Grid Servers AnalyzeAnalyze PlanPlan Resource Manager Parallel App

Monitor Execute Serv# Trans Pri Util Knowledge Monitor Execute 1 Trade 10 70% Fore- 2 80 80% caster SLA Scheduler Spec Distributed Multi-Site Controllers

Dynamic on-demand provisioning and resource allocation for Transactional Grids Dynamically use servers needed for current load and SLA Detect a surge, and reconfigure system to handle surge Use capacity available when applications have different times for surge Use multi-site / multi-datacenter resources to support load Allows configuration close to average rather than peak traffic for each workload

BE BE BE

WS II as Shared Cache Node

Browser/ Network Application Web Service Client Dispatcher Server Caching To meet QoS goals

Challenge: What to Cache? Approach: Intelligent and Adaptive Caching (closer to App srvr)

Components: • Data Placement Advisor (DPA) for Intelligent Caching • Monitoring for Adaptive Caching • Autonomic Replication Management Service for maintaining Cache Consistency

DPA: determine what to cache Workload DPM: setup cache and replication Frontend Database QCC: load monitor for adaptive caching 1 (Cache) ARMS: maintain data consistency WS II 2 Query/ Cache (MQTs) Result 3 Read only query log

10 Sync.

MQT 4 Simulated catalog Base tables Advisor 5 Write only query log • MQT & query mapping •MQT size Backend 6 • (MQT, benefit, overhead) Database • MQT dependency

• Synchronization cost estimation QCC DPA 7 • What if analysis for data placement

8 MQTs ARMS 9 Setup MQTs DPM

Abstract Component Diagram RDBMS

LDAP

Resource Namespace Service (other)

Datastore Single Namespace Resource Targets Object Schema Object Schema Virtual Directories NFS

Resource Junctions Service Interface Adapter CIFS

RNS Referrals GridFTP

Aliases Database (other) Web Services LDAP

Application Client Filesystem Client

26 IBM Grid Technical Strategy & Research Projects © 2005 IBM Corporation Grid Computing Provisioning Resilient Grid Service Provisioning & Orchestration Grid Administrator Autonomic GS Grid Customer GS Migration Migration Service Policy Service Migration Plan Grid Resources Grid Service Registry Broker Migration Migration Grid Services Analyzer Scheduler Registry

MA LRM SM LRM SM MA OGSA Middleware OGSA Middleware MA LRM SM

OGSA Middleware LRM Local Resource Monitor

SM Service Monitor Grid Resources MA Migration Agent

Software Solution Solution DataCenter Developer Integrator Deployer Operator

COMFORT

Software Solution Solution Configuration Configuration Configuration Configuration Configuration Conflict Conflict Template Template Wizard Detector Resolver Builder Builder

Software Resource Configuration Model (UML and Schema)

Software Solution Configuration Configuration Conflict-free Configuration Configuration Item Value Conflict Report Configuration Templates Templates DB2 UDB 8.2 Server

DB2 J2EE Client 8.1 Application

WAS

Software TPM/TIO Repository Configuration Management DB

Capacity Business Management process for Yield UI selling the management inventory Advance reservation

Provisioning Triggers Reservations Resource description - description - past and capacity and present prices

TPM Resource Monitoring

BCS and IGS Grid Practices Technical Support Grid & Virtualization Capacity Design On Demand Centers Centers

Grid Whitepapers and Redbooks Grid Technical and Sales Specialists

Grid Integration Center

Products www.ibm.com/grid

Grid Enabled Grid Computing Institute ISVs & Middleware Partners IBM Research

[email protected]