Getting The Most Out of DB2 in Your New Applications Curt Cotner IBM DB2 Development, IBM Fellow and VP

Session Code: S03 Tue May 3, 10:15-11:45 | Platform: z/OS, LUW IBM Information Management

Optimizing Your Java and WebSphere Applications with Data Studio and Optim Tools

© 2007 IBM Corporation IBM Information Management

Data Studio -- pureQuery tooling is in the box! Speed up problem isolation for developers – even when using frameworks

ƒ Capture application-SQL-data object correlation (with or without the source code) ƒ Trace SQL statements to using code for faster problem isolation ƒ Enhance impact analysis identifying application code impacted due to database changes ƒ Answer “Where used” questions like “Where is this column used within the application?” ƒ Use with modern Java frameworks e.g. Hibernate, Spring, iBatis, OpenJPA

© 2007 IBM Corporation IBM Information Management Data Studio -- Code/Debug Oracle PL/SQL or Sybase T-SQL

© 2007 IBM Corporation IBM Information Management Data Studio -- OPM Performance View

© 2007 IBM Corporation IBM Information Management

Data Studio -- OPM Performance View

Table Columns Table Actions Golf score Export – Exports the data to file SQL statement Remove All – clears the table of all rows Annotation Inputs for host variables Row Actions Total Server Time Open in SQL Editor – opens SQL editor Average Server Time with selected SQL statements Number of Rows Filter – Hides all but the selected rows Number of Rows Examined Remove – removes selected row(s) Average Number of Row Returned CPU time Number of Sorts Number of RSCANs Number of ISCANs Number of physical IOs Number of logocial IOs

© 2007 IBM Corporation IBM Information Management

Data Studio -- New Routine Creation Wizard

© 2007 IBM Corporation IBM Information Management

Data Studio -- Templates Management

© 2007 IBM Corporation IBM Information Management

Data Studio -- Server Profile Management

© 2007 IBM Corporation IBM Information Management

Data Studio -- Deployment Management

© 2007 IBM Corporation IBM Information Management pureQuery Runtime – every Java application benefits!

ƒ JDBC – acceleration for any JDBC application ̛ Convert dynamic SQL to static SQL ̛ Replace problem queries without changing the source ̛ Remove literals from SQL to get better statement cache hit ratios ƒ Hibernate/OpenJPA/iBatis – acceleration for persistence layers ̛ Improved SQL “batch” peformance ̛ Auto-tuning of Hibernate and OpenJPA peristence options ƒ SQL-friendly APIs for OO access to relational ̛ Object to relational mapping ̛ APIs that can be tailored to return XML, JSON, arrays, etc. ƒ Improved management, monitoring, problem determination ̛ Tracks SQL back to the Java class file and line number ̛ Enables performance monitors to report by application name ƒ Provides the foundation for improved developer tooling ̛ Syntax assist, code generation, performance reporting, etc.

© 2007 IBM Corporation IBM Information Management What’s so Great About DB2 Accounting for CICS Apps?

CICS AOR1 z/OS LPAR Txn1 - Pgm1 - Pgm2

CICS AOR2 TxnA DB2PROD - PgmX - PgmY CICS AOR3 Txn1 - Pgm1 - Pgm2

App CPU PLAN DB2 Accounting for CICS apps allows you to study performance data from many perspectives: Txn1 2.1 • By transaction (PLAN name) TN1PLN • By program (package level accounting) TxnA 8.3 • By address space (AOR name) TNAPLN • By end user ID (CICS thread reuse)

This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.

© 2007 IBM Corporation IBM Information Management JDBC Performance Reporting and Problem Determination – Before pureQuery Application Server

E DB2 or IDS D J P D B a e

B t r Q USER1 a s 2

A1 i u A s

J e t c

A4 e a r c y n v e

a c

L USER1 s

A3 e a s D

A2 n

L r L g i a v o u y

e USER1 A5 g e a r i c r A6 g e

What is visible to the DBA? - IP address of WAS app server - Connection pooling userid for WAS User CPU - app is running JDBC or CLI PACKAGE USER1 2.1 JDBC What is not known by the DBA? USER1 8.3 JDBC - which app is running? USER1 22.0 JDBC - which developer wrote the app? - what other SQL does this app issue? - when was the app last changed? - how has CPU changed over time? - etc.

© 2007 IBM Corporation IBM Information Management What’s so Great About Optim pureQuery Accounting for WebSphere Applications?

z/OS LPAR Unix or Windows

WAS 21.22.3.4 CICS AOR2 TxnA (Set Client App=TxnA) TxnA (PLANA) - ClassX - PgmX - ClassY - PgmY

Data Studio and pureQuery provide the same granularity App CPU for reporting WebSphere’s DB2 resources that we have TxnA 2.1 with CICS: TxnB 8.3 • By transaction (Set Client Application name ) • By class name (program - package level accounting) • By address space (IP address) • By end user ID (DB2 trusted context and DB2 Roles)

This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.

© 2007 IBM Corporation IBM Software Group – Information Management DB2 Java Data Access Frameworks Acceleration

• Hibernate: http://www.ibm.com/developerworks/data/library/techarticle/dm- 1008hibernateibatispurequery1/index.html?ca=dnw-1133&ca=dth-i • iBatis: http://www.ibm.com/developerworks/data/library/techarticle/dm- 1009hibernateibatispurequery2/index.html • Spring: http://www.ibm.com/developerworks/data/tutorials/dm0806hsing/index.html

© 2011 IBM Corporation IBM Software Group – Information Management Accelerate Java frameworks: Hibernate & iBatis

• Generate SQL and bind as Static Packages before deploying application • Improve performance with heterogeneous batching & Static Execution • Identify and replace problematic SQL with hand-tuned alternative SQL • Track SQL requests back to the framework query, including java source file/line #

© 2011 IBM Corporation IBM Information Management Object/Relational Mapping pureQuery can monitor your Java application’s object access patterns and class Customer { public String Name; automatically select the optimal eager/lazy public String mailingAddress; fetch setting for each SQL statement!!! public String daytimePhone; public Order[] recentOrders; Table Column Type public Complaint[] complaintHistory CUST NAME CHAR(64) … CUST ADDRESS CHAR(128) } CUST PHONE_NUM CHAR(10)

Table Column Type COMPLAINTS CUST_NAME CHAR(64)

COMPLAINTS COMP_ID CHAR(18) Table Column Type COMPLAINTS DESC VARCHAR(32K) ORDERS CUST_NAME CHAR(64) ORDERS ORDER_NUM CHAR(12)

ORDERS DATE_ORD DATE Table Column Type CREDIT_DATA CUST_NAME CHAR(64)

CREDIT_DATA CARD_NUM CHAR(18) Table Column Type CREDIT_DATA VALID_UNTIL DATE ORDER_ITEMS ORDER_NUM CHAR(12) ORDER_ITEMS ITEM CHAR(128)

ORDER_ITEMS QUANTITY SMALLINT

© 2007 IBM Corporation IBM Information Management Eager vs. Lazy Fetch class Customer { public String Name; “Select object(customer) WHERE…” public String mailingAddress; public String daytimePhone; public Order[] recentOrders; public Complaint[] complaintHistory … } “SELECT CUST.NAME, CUST.ADDRESS,CUST.ADDRESS … … FROM CUST WHERE…” FROM“SELECT CUST, ORDERS.ORDER_NUM COMPLAINTS, … WHERE …” Table Column Type CUST NAME CHAR(64) “SELECTCOMPLAINTS.COMP_IDORDERS, ORDER_ITEMS … WHERE …” CUST ADDRESS CHAR(128) WHERE…”O CUST PHONE_NUM CHAR(10) O Table Column Type O Table Column Type COMPLAINTS CUST_NAME CHAR(64) COMPLAINTS COMP_ID CHAR(18) ORDERS CUST_NAME CHAR(64) COMPLAINTS DESC VARCHAR(32K) ORDERS ORDER_NUM CHAR(12) ORDERS DATE_ORD DATE

Table Column Type Table Column Type CREDIT_DATA CUST_NAME CHAR(64) ORDER_ITEMS ORDER_NUM CHAR(12) CREDIT_DATA CARD_NUM CHAR(18) ORDER_ITEMS ITEM CHAR(128) CREDIT_DATA VALID_UNTIL DATE ORDER_ITEMS QUANTITY SMALLINT

© 2007 IBM Corporation IBM Software Group – Information Management Hibernate AutoTuning • Automatically identify and fix common problems with Java Persistence applications • hundred’s of SQL per transaction • tens of unwanted joins per SQL https://www.ibm.com/services/forms/preLogin.do?source=swg-iopahb

© 2011 IBM Corporation IBM Information Management How Optim Development Studio Helps

© 2007 IBM Corporation IBM Information Management Client Optimization Improve Java data access performance for DB2 – without changing a line of code

ƒ Captures SQL for Java applications ̛ Custom-developed, framework-based, or packaged applications ƒ Bind the SQL for static execution without changing a line of code ̛ New bind tooling included ƒ Delivers static SQL execution value to existing DB2 applications ̛ Making response time predictable and stable by locking in the SQL access path pre- execution, rather than re-computing at access time ̛ Limiting user access to tables by granting execute privileges on the query packages rather than access privileges on the table ̛ Aiding forecasting accuracy and capacity planning by capturing additional workload information based on package statistics ̛ Drive down CPU cycles to increase overall capability ƒ Choose between dynamic or static execution at deployment time, rather than development time

© 2007 IBM Corporation IBM Information Management Optim pureQuery Runtime for z/OS

ƒ In-house testing shows double-digit reduction in CPU costs over dynamic JDBC % increase/reduction in CPU per transaction compared to JDBC using Type 4 driver Normalized Throughput by API for JDBC Type 4 Driver

500 r e p

C

U 25% B P ) 400 D C 15% J R

n T i o I

t (

n t 6% d o u i e

p 300 t r c a h

ƒ g 524 u p u d

485 m e o

r 446 r o / c h 420 e T

200 s 360 n

a -14% s d e e n r z a i 274 c r l t n a i

m 100 r

% -35% o

N -50%

c 2 c i c A i t ti B P a 0 J J m t a a t E S 2 n S ƒ C c c c . i i i A t t y t d B B P m a a D p o J J D t t a O h E J n S S d t t y o e n d h n D z o t M i ie h e l d m t Q i C o t e M p h t p M Q e O p Q t M p n Q e i ƒ IRWW – an OLTP workload,p l Type 4 driver C ƒ Cache hit ratio between 70 and 85% ƒ 15% - 25% reduction on CPU per txn over dynamic JDBC

© 2007 IBM Corporation IBM Information Management What Is Heterogeneous Batching? Heterogenous Batching – multiple operations across different tables all execute as one batch

Table 1, operation 1 Data Server Table 1, operation 2 Table 1, operation 3

Table 2, operation 1 Table 2, operation 2

Table 3, operation 1 Table 1, operation 4

© 2007 IBM Corporation IBM Information Management JDBC Batching v/s pureQuery Heterogeneous Batching

ƒ JDBC batching used by Hibernate Batcher is currently limited The advantage of Heterogenous Batching ̛ Cannot batch entities that map to multiple tables • Primary and Secondary tables. 500 • Inheritance Join and Table per class

) 450

strategies c ̛ Cannot batch different operations e 400 against same table S 350 m ( • Field level updates

e 300 • Insert, update m HeteroBatching i 250

̛ Cannot batch different entities T 200 No Batching ̛ Each batch is a message to the d e

database s 150 JDBC Batching p

a 100 l

ƒ pureQuery heterogeneous batching E 50 plug-in for Hibernate on the other 0 hand ̛ Can batch entities that map to multiple 0 20 40 60 80 100 tables ̛ Can batch different operations against # of operations per transaction the same table ̛ Can batch different entities into a single batch * Preliminary findings based on validation with a test designed to demonstrate ̛ Combines insert, deletes, updates into heterogeneous batching differences. This is not intended to be a formal single batch benchmark.

© 2007 IBM Corporation IBM Information Management pureQuery – Stripping Literals from SQL

JDBC app

INSERT INTO T1 VALUES(‘ABC’,2,’DEF’) pureQuery INSERT INTO T1 Runtime conversion VALUES(:h1,:h2,:h3)

• pureQuery can identify statements that use no parameter markers, and strip the literals out at runtime • significant performance gains: • less CPU cost at PREPARE • better use of dynamic statement cache

© 2007 IBM Corporation IBM Information Management

WebSphere – a first class OPM citizen

WebSphere support

has a built-in support for OPM (starting with V6.0.21), allowing to ... 47 57.67 47 57.67 • identify problems with 52 0.34WAS connection pool • identify differences in the configuration of nodes in a WAS cluster • check if a node in a WAS cluster has a system or network problem

© 2007 IBM Corporation IBM Information Management

OPM can tell you where the query came from …

Application source

Capture SQL with pureQuery runtime

Upload collected metadata into OPM

SELECT * FROM DB2ADMIN.I…

SELECT * FROM DB2ADMIN.INVENTORY

TestOPM my.test main 13 blahVer N/P N/P blah capture…

© 2007 IBM Corporation IBM Information Management

OPM Extended Insight (EI) Overview dashboard

© 2007 IBM Corporation IBM Information Management

OPM Extended Insight Dashboard – Client Details

© 2007 IBM Corporation IBM Information Management

Open Source Scripting Languages

© 2007 IBM Corporation IBM Software Group – Information Management DB2 Scripting languages support

Application

Django Ruby on Zend/Drupal Rails

Perl Python PHP Ruby

DB2 CLI

DB2

• Up-to-date with latest Django/Rails/Zend releases. • All open source drivers and adapters are available on the DB2 media • Python: http://code.google.com/p/ibm-db/ • Ruby: http://rubyforge.org/projects/rubyibm/ • PHP: http://pecl.php.net/package/ibm_db2/ , http://pecl.php.net/package/PDO_IBM • Perl: http://search.cpan.org/~ibmtordb2/

© 2011 IBM Corporation IBM Software Group – Information Management In-the-works

• DB2 Drupal Support – A widely used PHP based Web Content Management System – DB2 support for Drupal 6 publicly available shortly – Drupal 7 support to follow

• SQL Generation for Java API based Query Systems before deployment • Complete Accelerator support for Hibernate / JPA Criteria Queries

© 2011 IBM Corporation NoSQL Datastores

© 2011 IBM Corporation Document Stores 12 + Graph Stores XML Stores 11+ 9 +

Current NoSQL Landscape Tabular Stores Key Value Stores 6 + 23 +

Object Stores Others 12 +

Currently there are more than 100+ noSQL systems Not clear which of these will survive

© 2011 IBM Corporation Why did NoSQL Datastores Arise?

‰ Some apps need very few database features, but need high scale

‰ Desire to avoid data/schema pre-design altogether for simple apps

‰ Need for a low-latency, low-overhead API to access data

‰ Simplicity – I don’t need fancy indexing - just fast lookup by primary key

© 2011 IBM Corporation NoSQL Datastores

Focus on Give up

‰ Commodity servers, networking, disks ‰ Relational data model ‰ Easy elasticity and scalability to ‰ SQL APIs multiple racks (10s to 100s of servers) ‰ Complex queries (joins, secondary ‰ Fault-tolerance and high availability indexes, ACID transactions)

Two Worlds

Transactional Analytics ‰ Custom high-end OLTP for financial ‰ Managing updates applications ‰ Support for random access and indexing ‰ Scaleout datastores for Cloud/Web 2.0 ‰ Scaleout content store

‰ Examples ‰ Examples ¾ MemcacheDB, Cassandra, Dynamo, Voldemort, ¾ Bigtable, HBase, Hypertable SimpleDB, Gigaspaces, Websphere eXtreme Scale

© 2011 IBM Corporation Graph Stores ‰ Datastores which can handle ‰ Large but sparse predicate set for a subject ‰ Can do link following ‰ Optionally support other graph like operations ‰ Transitive closure computation, Shortest paths ‰ Common Model : RDF (Resource Description Framework)

‰ A way of expressing linked data : Subject – Property – Object (Value) Natural graph structure. Ideally suited for representing information extracted from unstructured sources or graph-based information Example : IBM hasLocation Hawthorne Hawthorne LocatedIn New York ‰ Applications include semantic web, data integration, metadata access, inferencing applications in healthcare, government, telecom,oil

‰Open source examples : Jena, Sesame

© 2011 IBM Corporation Key Value Stores

‰ Allows one to store and query values by key

‰ Quick response time, distributed, with optional transaction support

‰ Often used to cache data and objects in RAM for Web 2.0 applications

‰ Open source examples : MemcacheD, Redis

‰ Common Interface : ‰ Get ‰ Put ‰ Remove

© 2011 IBM Corporation Document Stores

‰ Scalable, high-performance, schema-free, document-oriented databases.

‰ Manages collection of JSON like documents.

‰ Used in Web 2.0 scenarios

‰ Open source examples : CouchDB, MongoDB

‰ Details of CouchDB : ‰ A document could be one or more field/value pairs expressed as JSON. ‰ The field value pairs might be very sparse across documents ‰ Every document in a CouchDB database has a unique id.

Example of a document: "Subject": “Cricket World Cup“ "Author": "Rusty“ "PostedDate": “2/23/2011“ "Tags": [“cricket", “world cup", “overs“] "Body": “The Cricket World Cup is currently ongoing. It is played over 50 overs”

© 2011 IBM Corporation XML Stores

‰ Data repository for XML data. Storage scheme could be in native form

‰ Useful when there is variable schema in the data or transmission in XML

‰ Transaction support . Support for indexing and query processing

‰ Applications include creating business reports, SOA, webservices, forms

‰ Open source examples : Sedna, BaseX

‰ Interface : XQuery/XPath

© 2011 IBM Corporation Tabular Stores

‰ Scalable, distributed repository for structured data

‰ Various levels of transaction support

‰ Richer support for indexing and query processing

‰ Applications include analytics, storage of user metadata information

‰ Open source examples : HBase, Cassandra

‰ Interface : APIs vary from implementation to implementation ‰ Thrift ‰ Java

© 2011 IBM Corporation Past Proclamations of the Imminent Demise of DB2

‰Object databases vs. relational ¾ Gemstone, ObjectStore, Objectivity, etc. ‰In-memory databases vs. relational ¾ TimesTen, SolidDB, etc. ‰Persistence frameworks vs. relational ¾ Hibernate, OpenJPA, etc. ‰XML databases vs. relational ¾ Tamino, BaseX, etc. ‰Open Source vs. commercial relational ¾ MySQL, PostgreSQL, etc. ‰Column-store databases vs. relational ¾ Sybase IQ, Vertica, etc.

© 2011 IBM Corporation Key Attributes of an Enterprise-class Data Repository

‰ A very broad set of attributes must be available: ¾ Ability to scale up to large SMPs ¾ Ability to scale out to multiple boxes ¾ Ability to exploit evolving microprocessor and memory architectures (SMP, multi-core, instruction pipeline, FGPA, SMT, etc.) ¾ Parallelism at many levels in the architecture ¾ High availability solutions (replication, failover, high speed restart/recovery, online schema change) ¾ Security/Auditing/Compliance ¾ Compression, archiving, and overall management of storage ¾ Performance/Monitoring/Tuning tool suite ¾ Modeling/Design/Development tool suite ¾ Programming interfaces to the constantly changing application space (Cobol, C/C++, Java, .NET, Ruby, PHP, Python, etc.) ¾ Adoption by packaged ISV applications ¾ Wide range of end-user reporting tools ¾ Enterprise-level problem determination, service, and support This massive infrastructure is needed regardless of the APIs you choose to expose to the end users and applications, or the logical/physical data model of the data. History has shown that we can add a new set of APIs or a new data model to DB2 much faster than the alternative solutions can build out a full solution with all the above attributes.

© 2011 IBM Corporation RDF and Jena Built on Top of DB2 Infrastructure

Immediately takes advantage of: • DB2 storage infrastructure • DB2 backup/recovery • DB2 pureScale technology • DB2 performance monitoring Data Analyzer RDF queries • DB2 security and auditing (SPARQL) • DB2 high-volume utilities SchemaCreator • etc.

We’ve found that Jena using DB2 Query translator RDF Loader out-performs the open source SPARQL-to-SQL Jena implementation by up to 300%.

RDF query Relational store uses Relational store Processing Query processor Functions Indexing

© 2011 IBM Corporation DB2 is making investments to support Key Value data (Redis)

Put (Key, Value)

Get (Key) Remove(Key)

Value

Data Store

Key/value access is very well optimized with the recently GA support for hash data access in DB2 11 for z/OS. Range partitioning and DPSIs also help optimize for key/value access patterns.

© 2011 IBM Corporation DB2 is already providing XML support

Applications include creating business reports, SOA, webservices, forms etc

© 2011 IBM Corporation What good is Hadoop for compared to warehousing ? Person -name, address, age, … -Behavioral model, url visits, network of friends

Product vs -SKU, price, … -Color, web ratings, green ratings, …

Merchant -StoreID, address, volume sales, profit… Warehousing -Employees, Neighboorhood, … domain Events -Person x bought product y at merchant z -Financial trace (for fraud), Clickstream, Hadoop domain Complaints, Location Based Services (smart phone customer tracking), hospital records

-non traditional, semi-/un-structured data, with no heavy DBA work up front -Algorithmic analytics that is not just SQL programming– requires high-scale analytics server

© 2011 IBM Corporation IBM Database Provider and Add-Ins for .NET framework 4.0 & Visual Studio .Net 2010

Microsoft .Net 4.0 and VS 2010 Integration

© 2009 IBM Corporation IBM Database Add-Ins for Visual Studio .Net

ƒ Single Offering of VSAI for Visual Studio 2005, 2008 and 2010 Install – Single VSAI installer for all installed versions of Visual Studio – Integrated VSAI Help with VS 2010 using the new VS 2010 help integration processes – New program group enabling registration of VSAI for versions of VS installed after VSAI install. The program group also has links to online documentation and the .Net Development with DB2 and Informix forums. ƒ Seamless integration of VSAI functionality into VS 2010 development environment – Support for Microsoft Windows® Presentation Foundation (WPF) and Extensible Application Markup Language (XAML) with the ability to generate sample XAML application from an IBM database table, view, or procedure – Ability to access and manage DB2 and Informix data connections – Create and modify stored procedures, functions, tables, views, indexes, triggers, generate web services etc. – Support for debugging SQL and SPL procedures. – XML support including support for XML data type, XML schema repository and annotated XML schemas. – Support for creation of IBM Database Projects, Windows or ASP.NET Web applications with DB2 and Informix Data Sources – Support for SQL capture and running SQL statically

© 2011 IBM Corporation IBM .NET Data Provider for .NET Framework 4.0

ƒ A new .IBM Database Provider for .NET Framework 4.0 – Includes all previously supported functionality for .NET framework 1.1 and 2.0 including but not limited to XA transactions, sysplex and XML support – Ability to configure .NET framework through client installer for automatic re- direction of 1.1 and 2.0 .NET applications to run against the IBM 4.0 provider – Works with full version of .NET Framework 4.0 as well as its smaller- footprint version known as the ‘Client Profile” ƒ A new Entity Provider for Entity Framework 4.0. – Support for optimized and seamless mapping of .NET canonical functions in LINQ queries into SQL functions on IBM DB2 and IDS servers. – Support for translation of LINQ queries into SQL queries that exploit the database features (e.g. indexes) on IBM DB2 and IDS servers.

© 2011 IBM Corporation Overview of Optim Configuration Manager V1.2 What is it?

• A data client and server configuration management solution providing centralized view and control of data client and server relationships

What is its value to DB2 customers?

• Gain insight into IT environments through automated discovery of data client and data server configurations and relationships

• Reduce database outages and enforce compliance with centralized control and analysis of distributed database access

• Help maintain SLAs of database applications with configurable client connection properties and connection-level load balancing

Functions and features

• Registry of server connection properties for the enterprise

• Links to the correct IDA model, pureQuery metadata, and OPM performance repository for each database server

• Historical view of data client and server configuration changes

• Push down client configuration changes to data clients

• Isolate or roll-out apps to specific data server(s) through connection-level load balancing

• Workload Manager classification can be updated and validated against expected SLAs

51 © 2010 IBM Corporation SYSTEM 2 Discovery IOCM WEB UI Applications SYSTEM 1 DB2 A IOCM Server DB2 B DB2 DB2 C DB2 JCC DRIVER Repository Catalogs M1 M2 M3 CMX Controller Discovery Tasks •DB2 z/OS DataSharing DB2 z •DB2 z/OS 3-way •DB2 Distributed Data Sharing •DB2 DPF •DB2 pureScale • Automatic discovery of clients connected to database servers • IP address, connection properties, client driver software level, etc. • Automatic discovery of database server changes • Server name, system parameters, database schema, etc. • Repository contains a history of how these things change over time: • Which clients are using client software older than DB2 9.7.1? • What changes have occurred on DB2A since Sept 1, 2010?

52 © 2010 IBM Corporation Enterprise Connection Management SYSTEM 2 IOCM WEB UI Applications SYSTEM 1 DB2 A IOCM Server DB2 B DB2 DB2 C DB2 JCC DRIVER Repository Catalogs M1 M2 M3 •DB2 z/OS DataSharing CMX Controller Discovery Tasks •DB2 z/OS DB2 z •DB2 Distributed 3-way •DB2 DPF Data Sharing •DB2 pureScale • Centrally manage inventory of database connection properties • Automates connection definition for clients • Tailored list of database connection “folders” for Web and Eclipse users • Centrally manage the relationships between systems and IDA models: • Record with IDA model file is current for each of the database servers • Identify which OPM repository should be used for each database server • pureQuery metadata for each database server • Centrally manage client connection properties: • Provide defaults or overrides for IP address, port, isolation level, trace on/off, etc. • Penalty boxing – force misbehaving app to use exclusively member X of system

53 © 2010 IBM Corporation Capture Replay Technology Preview

© 2009 IBM Corporation Capture Replay Validation report enables drill- down on failed replays, like Different Return Codes Open | d Welcome X Capture / Replay X Move Diff Rows Returned Create Test Database SQL Workloads Validation Report X Adjustable >= 5% to 10% Validate that the replay matches the original capture. Remove failed SQL and related transactions.

Overview

Replay Success Response Time

Successful SQL Replays 9000 / 10000 90% JJJJJJJJJJJJ

Failed SQL Replays 1000 / 10000 10% JJ PeakOrders y Different Return Codes 300 / 10000 3% J PeakOrders[5]

y Different # Rows Returned 200 / 10000 2% J

J y Different # Rows Updated 300 / 10000 3% 0 1 2 3 4

y Missing SQL 0 / 10000 0% Elapsed Time (Hours)

Successful Transaction Replays 500 / 800 63% JJJJJJJJ PeakOrders[0] Total 240:35 JJJJJJJJJJJJJJJJJJJJ

Failed Transaction Replays 300 / 800 27% JJJJ PeakOrders[5] Total 220:25 JJJ JJJ JJJ JJJ JJJ J

y Different Return Codes 100 / 800 12% JJ Total Improvements 25:30 10% JJJ

y Different # Rows Returned 60 / 800 7% J Total Regressions 5:20 2% J

y Different # Rows Updated 70 / 800 8% J SQL with >= 5% Improvement 300 / 10000 3% J

y Missing Transactions 0 / 800 0% SQL with >= 5% Regression 200 / 10000 2% J

New SQL 50 Trans with >= 5% Improvement 10 / 250 3% J

New Transactions 2 Trans with >= 5% Regression 2 / 250 1% J

SQL Execution (1000 / second) Rows Returned (10,000 / second)

© 2011 IBM Corporation Capture Replay

Open | d Welcome X Capture / Replay X

Create Test Database SQL Workloads Performance Report X

Top ‘N’ SQL Statements Comparison

Sort by: Total Response Time Change | d Number of Statements: 5 | d Show: Both Regressions and Improvements | d

SQL Regressions

Total Response Time Average Response Time Rows Rows Return Baseline Change in Baseline Change Change Baseline Change Change Updated Returned Code Statement Text Executions Executions (sec) (sec) d (%) (sec) (sec) (%) (changes) (changes) (Changes)

UPDATE DBPARTITION… 10050 0 200.849 +100.427 +50% 0.059 +0.027 +50% 0 0 0 JJJJJJJJJJJJJJJ JJJJJJJJJ JJJJ JJJJJJJJJ

INSERT T1.AGENT_ID … 25 0 896.433 +90.708 +10% 12.433 +1.208 +10% 0 0 0 JJJJJJJJJJJJJ JJJ JJJJJJJJJJJJJ JJJ UPDATE DBPARTITION… 2234 0 1765.623 +85.676 +5% 1.223 +0.176 +5% 0 0 0 JJJJJJJJJJJJ J JJJ J INSERT T2.AGENT_ID … 307 0 248.321 +78.786 +32% 0.821 +0.286 +32% 0 0 0 JJJJJJJJJJJ JJJJJJJ JJJJJJJJJJJ JJJJJJJ SELECT * FROM T3 … 529 0 215.765 +75.653 +27% 0.565 +0.133 +27% 0 0 0 JJJJJJJJJJ JJJJJJ JJJJJJJJJJ JJJJJJ

SQL Improvements

Total Response Time Average Response Time Rows Rows Return Baseline Change in Baseline Change Change Baseline Change Change Updated Returned Code Statement Text Executions Executions (sec) (sec) d (%) (sec) (sec) (%) (changes) (changes) (Changes)

SELECT T2.AGENT_ID … 100 0 1874.321 -195.427 -12% 10.874 -22.337 -12% 0 0 0 JJJJJJJJJJJJJJJ JJJ JJJJJJJJJJJJJ JJJ

SELECT T1.AGENT_ID … 345 0 135.987 -120.7083 -95% 0.421 -0.398 -95% 0 0 0 JJJJJJJJJJJ JJJJJJJJJJ J JJJJJJJJJJ SELECT DBPARTITION… 15454 0 1201.787 -55.676 -5% 0.123 -0.059 -5% 0 0 0 JJJJJ J J J SELECT T2.AGENT_ID … 4443 0 86.874 -20.786 -23% 0.013 -0.007 -23% 0 0 0 JJJ JJJJJ JJJ JJJJJ SELECT DBPARTITION… 56 0 753.765 -15.653 -2% 15.345 -1.334 -2% 0 0 0 JJ J JJ J

© 2011 IBM Corporation SQL Statement Comparison Drill-down

Optim Performance Manager Compare performance details of this statement across the two workload runs SQL Statement Comparison Report SQL Statement

SELECT B.COL1, B.COL3, B.COL5, B.COL6, B.COL12 FROM T1.SETLMNT, BRANCH B, ADDR A WHERE S.TRANS_NO = ?, AND S.TRANS_PROC_DT < '9999-12-31‘ AND YEAR (S.TRANS_TARGET_DT) = ‘2002’ AND S.TRANS_TYPE IN (‘A1', Tune SQL ‘A2', ‘A3', ‘Z9') AND S.TRANS_CD IN ('EOD', 'IMD', ‘UGT') AND S.TRANS_SETL_DT = ? AND B.BRANCH_EFF_DT <= ? AND B.BRANCH_INACTIVE_DT > ?

Metric Test Replay 1 Test Replay 2 % Change Average Elapsed Time (seconds) Executions 508 508 0% 2 Average Elapsed Time (sec) 0.567 0.876 +45% 1.8 JJJJJ 1.6 1.4 Test Replay 2 Total Elapsed Time (sec) 254.453 367.463 +45% JJJJJ 1.2 1 Test Replay1 Average CPU Time (sec) 0.0567 0.1376 +275% 0.8 JJJJJJJJJJJ 0.6 Total CPU Time (sec) 25.4567 69.876 +275% 0.4 JJJJJJJJJJJ 0.2 Average System CPU Time (sec) 0.0062 0.0121 +175% 0 JJJJJJJJJJJ 0 1 2 3 4 Execution Time (Hours) Total System CPU Time (sec) 2.3445 6.6503 +175% JJJJJJJJJJJ Average User CPU Time (sec) 0.0434 0.1221 +275% Average CPU Time (seconds)

JJJJJJJJJJJ Total User CPU Time (sec) 20.432 57.876 +275% 0.35 JJJJJJJJJJJ 0.3 Average Get Pages 4.01 4.40 +15% 0.25 Test Replay 2 JJ Total Get Pages 2000 2300 +15% 0.2 Test Replay1 JJ 0.15 Sorts 0 0 0% 0.1 0.05 Table Scans 0 0 0% 0 0 1 2 3 4 Execution Time (Hours)

© 2011 IBM Corporation Capture Replay

Open | d Welcome X Capture / Replay X

Create Test Database SQL Workloads Performance Report X

Top ‘N’ Transaction Comparison

Sort by: Total Response Time Change | d Number of Statements: 5 | d Show: Both Regressions and Improvements | d

Transaction Regressions

Total Response Time Average Response Time Rows Rows Return SQL Baseline Change Change Baseline Change Change Updated Returned Code Transactions Type Statements (sec) (sec) d (%) (sec) (sec) (%) (changes) (changes) (Changes)

APPNAME23 App Name 25 200.849 +100.427 +50% 0.059 +0.027 +50% 0 0 0 JJJJJJJJJJJJJJJ JJJJJJJJJ JJJJ JJJJJJJJJ

ACCTSTR456 App Name 5 896.433 +90.708 +10% 12.433 +1.208 +10% 0 0 0 JJJJJJJJJJJJJ JJJ JJJJJJJJJJJJJ JJJ ACCTSTR789 Acnt Str 73 1765.623 +85.676 +5% 1.223 +0.176 +5% 0 0 0 JJJJJJJJJJJJ J JJJ J PKGNM123 Package 15 248.321 +78.786 +32% 0.821 +0.286 +32% 0 0 0 JJJJJJJJJJJ JJJJJJJ JJJJJJJJJJJ JJJJJJJ SQL_SEQ_567 SQL Seq 75 215.765 +75.653 +27% 0.565 +0.133 +27% 0 0 0 JJJJJJJJJJ JJJJJJ JJJJJJJJJJ JJJJJJ

Transaction Improvements

Total Response Time Average Response Time Rows Rows Return SQL Baseline Change Change Baseline Change Change Updated Returned Code Transactions Type Statements (sec) (sec) d (%) (sec) (sec) (%) (changes) (changes) (Changes)

SQL_SEQ_765 SQL Seq 15 1874.321 -195.427 -12% 10.874 -22.337 -12% 0 0 0 JJJJJJJJJJJJJJJ JJJ JJJJJJJJJJJJJ JJJ

SQL_SEQ_988 SQL Seq 43 135.987 -120.7083 -95% 0.421 -0.398 -95% 0 0 0 JJJJJJJJJJJ JJJJJJJJJJ J JJJJJJJJJJ ACCTSTR333 Acnt Str 20 1201.787 -55.676 -5% 0.123 -0.059 -5% 0 0 0 JJJJJ J J J ACCTSTR555 Acnt Str 1 86.874 -20.786 -23% 0.013 -0.007 -23% 0 0 0 JJJ JJJJJ JJJ JJJJJ APPNAME767 App Name 56 753.765 -15.653 -2% 15.345 -1.334 -2% 0 0 0 JJ J JJ J

© 2011 IBM Corporation Capture Replay

SQL list for selected transaction. Open | d Welcome X Capture / Replay X

Create Test Database SQL Workloads Performance Report X

Top N Transactions Report > SQL List for Transaction APPNAME23

SQL List for Transaction APPNAME23

Total Response Time Average Response Time Rows Rows Return Updated Returned Code Baseline Change in Baseline Change Change Baseline Change Change (changes) (changes) (Changes) Statement Text Executions Executions (sec) (sec) d (%) (sec) (sec) (%)

UPDATE DBPARTITION… 10050 0 200.849 +100.427 +50% 0.059 +0.027 +50% 0 0 0 JJJJJJJJJJJJJJJ JJJJJJJJJ JJJJ JJJJJJJJJ

INSERT T1.AGENT_ID … 25 0 896.433 +90.708 +10% 12.433 +1.208 +10% 0 0 0 JJJJJJJJJJJJJ JJJ JJJJJJJJJJJJJ JJJ UPDATE DBPARTITION… 2234 0 1765.623 +85.676 +5% 1.223 +0.176 +5% 0 0 0 JJJJJJJJJJJJ J JJJ J INSERT T2.AGENT_ID … 307 0 248.321 +78.786 +32% 0.821 +0.286 +32% 0 0 0 JJJJJJJJJJJ JJJJJJJ JJJJJJJJJJJ JJJJJJJ SELECT * FROM T3 … 529 0 215.765 +75.653 +27% 0.565 +0.133 +27% 0 0 0 JJJJJJJJJJ JJJJJJ JJJJJJJJJJ JJJJJJ SELECT T2.AGENT_ID … 100 0 1874.321 -195.427 -12% 10.874 -22.337 -12% 0 0 0

JJJJJJJJJJJJJJJ JJJ JJJJJJJJJJJJJ JJJ SELECT T1.AGENT_ID … 345 0 135.987 -120.7083 -95% 0.421 -0.398 -95% 0 0 0

JJJJJJJJJJJ JJJJJJJJJJ J JJJJJJJJJJ SELECT DBPARTITION… 15454 0 1201.787 -55.676 -5% 0.123 -0.059 -5% 0 0 0

JJJJJ J J J SELECT T2.AGENT_ID … 4443 0 86.874 -20.786 -23% 0.013 -0.007 -23% 0 0 0

JJJ JJJJJ JJJ JJJJJ SELECT DBPARTITION… 56 0 753.765 -15.653 -2% 15.345 -1.334 -2% 0 0 0

JJ J JJ J SELECT T2.AGENT_ID … 100 0 1874.321 -195.427 -12% 10.874 -22.337 -12% 0 0 0

JJJJJJJJJJJJJJJ JJJ JJJJJJJJJJJJJ JJJ SELECT T1.AGENT_ID … 345 0 135.987 -120.7083 -95% 0.421 -0.398 -95% 0 0 0

JJJJJJJJJJJ JJJJJJJJJJ J JJJJJJJJJJ SELECT DBPARTITION… 15454 0 1201.787 -55.676 -5% 0.123 -0.059 -5% 0 0 0

JJJJJ J J J

© 2011 IBM Corporation IBM Information Management

ƒIBM Data Studio ̛www.ibm.com/software/data/studio • FAQs / Tutorials • Downloads • Forum / Blogs • Join the IBM Data Studio user community

ƒData Studio Book ̛http://bit.ly/dstudiobook

© 2007 IBM Corporation IBM Information Management

© 2007 IBM Corporation