Using DB2 Monitors to Analyze and Improve SQL & Query Performance

IBM System i™

Session: 440101 (47MA)

Using DB2 Monitors To Analyze and Improve SQL & Query Performance

Gene Cobb IBM Rochester, MN

8 Copyright IBM Corporation, 2006. All Rights Reserved. i want stress-free IT. This publication may refer to products that are not currently available in your country. IBM makes no commitment to make i want control. available any products referred to herein. i want an i.

IBM System i Query Optimization and Execution Feedback

V5R4 delivers!

Many new and exciting tools for monitoring and analyzing DB2 UDB for iSeries SQL requests and SQL performance

1 IBM System i

Query Optimization and Execution Feedback V5R4 V5R4 V5R4 Visual Explain Index SQE Plan SQE Cache Advised PlanCache Snapshots

Detailed DB Monitor Data SQL request Summarized DB Monitor Query Data Optimization Debug Print SQL Job Log Information Messages Messages

IBM System i What are the DB2 SQL Performance Monitors?

• Integrated tool set used to gather database performance related statistics for SQL-based requests running on the iSeries – i5/OS – Applicable to both batch jobs and interactive work – Only collect information on the DB2 engine processing - does not include the time that it took for the requests to get to the DB2 engine (e.g., JDBC driver processing)

• Monitor data dumped into table(s) where it can be queried to help identify and tune performance problem areas – Collected data can be analyzed to look at SQL performance at a global system level, job-level or, specific SQL statement level

• Monitor data most useful given a basic understand of DB2 UDB engine and query optimization techniques

• Monitor data does not tell you what to do – The science and art of query performance needs to be applied

2 IBM System i Database Monitor Information

• Provides information such as: – System, job and user names – SQL statement text – Start and end timestamps – Estimated processing time – Total rows in table queried – Number of rows selected – Estimated number of rows selected – Estimated number of joined rows – Key columns for advised indexes – Total optimization time and run time – Open data path (ODP) implementation – QAQQINI settings – Many others

IBM System i

Database Monitor Information

• Collected information can be analyzed later: – Which queries are the most time consuming? – How many queries rebuild access plans? – How many queries performed a full table scan? – How many temporary indexes have been created over a particular table? – Which user is running the longest running queries? – Has the implementation of a particular query changed with the application of a PTF or a new release? • Before and After comparison of the monitor data – Visually explain a given query plan • From detailed data captured by the query optimizer

3 IBM System i

Measuring & Monitoring DB2 Performance

END Output Results

Disk I/O ƒJournaling

RunTime ƒIndex Maintenance

ƒConstraint Enforcement

Communications ƒLocking

ƒTrigger Processing

Open ƒODP Creation Database Processing ƒDatabase Authentication

Authentication ƒAccess Plan Creation

Optimization ƒIndex Selection

ƒIndex Estimates User Display I/O Process Request BEGIN

IBM System i Measuring & Monitoring DB2 Performance

• iSeries Navigator GUI interface – SQL Performance Monitors New! – SQE Plan Cache Snapshots – Start, Stop, Import, Capture, Manage, Analyze

• Summarized Database Monitor

• SQE Plan Cache Snapshot

• Detailed Database Monitor

• The collection and capture mechanism will depend on how wide a net you need to cast, and how fine the net should be woven

4 IBM System i

Monitor Interfaces via iSeries Navigator

IBM System i

Summarized Database Monitor

5 IBM System i

Summarized Database Monitor

• Few enhancements in V5R4 • Summarized information collected by the SQL “monitoring” facility – Data is placed into multiple DB2 tables – Memory based – Low overhead • CQE and SQE support • API interface • GUI interface via iSeries Navigator – Access – Pre-filtering and Post-filtering – Analysis by time, user, job, statement, etc. – No Visual Explain • Data is not volatile – Information from the optimizer and engine is “captured” and summarized at a point in time

IBM System i Summarized Database Monitor - New

6 IBM System i Summarized Database Monitor - New

IBM System i Summarized Database Monitor - New

7 IBM System i Summarized Database Monitor - New

IBM System i

Summarized Database Monitor - Import

8 IBM System i Summarized Database Monitor - Import

IBM System i Summarized Database Monitor - Properties

9 IBM System i Summarized Database Monitor - Properties

IBM System i Summarized Database Monitor - Analyze

10 IBM System i Summarized Database Monitor - Analyze

IBM System i

SQE Plan Cache Snapshots

11 IBM System i

SQE Plan Cache Snapshots

• New V5R4 feature • System wide information materialized from the SQE Plan Cache – No overhead – data capture is part of normal query optimization and execution – Pre-filtering – Data is placed into a single DB2 table • SQE support only • SQL interface (CALL QSYS2/DUMP_PLAN_CACHE ‘schema’, ‘snapshot’) • GUI interface via iSeries Navigator – Access – Filtering – Analysis by time, user, job, statement, etc. – Visual Explain • Data is not volatile – Information from the SQE Plan Cache is “captured” at a point in time • SQE Plan Cache is always available – No need to “start and stop” a tool or utility • Additional analysis methods available like “before and after” comparisons

IBM System i SQE Plan Cache Snapshots - New

12 IBM System i SQE Plan Cache Snapshots - New

IBM System i

SQE Plan Cache Snapshots - Import

13 IBM System i SQE Plan Cache Snapshots - Properties

IBM System i SQE Plan Cache Snapshots - Properties

14 IBM System i SQE Plan Cache Snapshots - Analyze

IBM System i SQE Plan Cache Snapshots - Analyze

“Dashboard” provides a quick overview of information

15 IBM System i SQE Plan Cache Snapshots - Analyze Summary

IBM System i SQE Plan Cache Snapshots - Analyze Statements

16 IBM System i SQE Plan Cache Snapshots - Show Statements

IBM System i

SQE Plan Cache Snapshots - Show Statements

List is initially empty, the statements are filled after Retrieve or Refresh

17 IBM System i

SQE Plan Cache Snapshots - Compare

IBM System i SQE Plan Cache Snapshots - Compare

18 IBM System i SQE Plan Cache Snapshots – Compare VE

IBM System i

Detailed Database Monitor

19 IBM System i Detailed Database Monitor

• Enhanced in V5R4 • Detailed information collected - SQL “trace” – Data is placed into a single DB2 table – Potentially high overhead • CQE and SQE support • Command interface – STRDBMON / ENDDBMON • Connection attributes interface • GUI interface via iSeries Navigator – Access – Pre-filtering and Post-filtering – Analysis by time, user, job, statement, etc. – Summary information via “dashboard” – Visual Explain • Data is not volatile – Information from the optimizer and engine is “captured” at a point in time • Additional analysis methods available like “before and after” comparisons

IBM System i

Detailed Database Monitor - STRDBMON command

Type of records = *SUMMARY is NOT the memory based “summary” monitor

20 IBM System i

Detailed Database Monitor - STRDBMON command

New types of filtering to limit the scope and overhead of data collection

IBM System i

Detailed Database Monitor - STRDBMON command

• New (late) filtering option based on the “query governor” thresholds – Query governors: query time limit and/or query storage limit • Set via System value, QAQQINI, CHGQRYA – Query storage limit exit point program called when a governor threshold reached • QIBM_QQQ_QUERY_GOVR – Filter capability added to V5R4 late – use the Comment parameter to specify the filter – Query governor filter is combined with other filters • FTRQRYGOVR parameter – Parameter values are *NONE, *ALL or *COND – Filtering parameter value *NONE - never react to the thresholds – Filtering parameter value *ALL - always react to the thresholds – Filter parameter value *COND - react based on the return code of the exit point

21 IBM System i Detailed Database Monitor - ENDDBMON command

Ends a specifically named and numbered monitor – REMEMBER the name or ID

IBM System i Detailed Database Monitor - New

22 IBM System i

Detailed Database Monitor - New via Run SQL Scripts

•A detailed database monitor can be started within Run SQL Scripts

IBM System i Detailed Database Monitor - New

23 IBM System i Detailed Database Monitor - New

IBM System i

Detailed Database Monitor - Import

24 IBM System i Detailed Database Monitor - Properties

IBM System i Detailed Database Monitor - Properties

25 IBM System i Detailed Database Monitor - Analyze

IBM System i Detailed Database Monitor - Analyze

“Dashboard” provides a quick overview of information

26 IBM System i Detailed Database Monitor - Subset

IBM System i Detailed Database Monitor - Show Statements

27 IBM System i Detailed Database Monitor - Show Statements

List is initially empty, the statements are filled after Retrieve or Refresh

IBM System i

Detailed Database Monitor - Compare

28 IBM System i Detailed Database Monitor - Compare

IBM System i

iSeries Navigator Analysis - Preferences

•Allows the analysis job to pick up and use a QAQQINI file

29 IBM System i

The Details

IBM System i

Detailed Database Monitor Row Types

• Detailed Database Monitor data and SQE Plan Cache Snapshot data are very similar • Most Row types are found in the snapshot data • Similar analysis methods can be used for both the detailed database monitor data and the snapshot data • Even though similar, snapshot data cannot be imported as detailed monitor data, nor compared to detailed monitor data • Additional information on rows and columns in publication: –DB2 Universal Database for iSeries Database Performance and Query Optimization

30 IBM System i Custom Queries or Analysis Reports?

• Detailed Database Monitor data and SQE Plan Cache Snapshot data can be analyzed with custom queries or iSeries Navigator reports • The reports provide a way to navigate to different areas of interest, and seamlessly identify a given query • The reports provide a way to narrow down focus to a given query, then seamlessly Visually Explain that query • The reports are robust; containing many, many columns • Custom queries provide a way to narrow down the set of columns and rows of interest • Custom queries provide alternate methods of analysis • Custom queries require a good understanding of the underlying data

IBM System i

Detailed Database Monitor Row Types

• Row types most often used in analysis (QQRID value) – 1000 – SQL request summary – 3000 – Table scan – 3001 – Index scan/probe – 3002 – Index created – 3003 – Query sort – 3004 – Temporary results – 3006 – Access plan rebuilt – 3007 – Index evaluation – 3010 – Host variable and ODP implementation – 3014 – General environment (including QAQQINI settings) – 3015 – SQE statistics advised – 3020 – Index advised (V5R4)

31 IBM System i

Detailed Database Monitor Row Types • Other row types – 3005 – Table locked – 3008 – Sub query processing – 3018 – STRDBMON & ENDDBMON attributes – 3019 – Rows retrieved and I/O statistics (only with *DETAIL option) – 3021 – Bitmap created – 3022 – Bitmap merge (index And/Or) – 3023 – Temp hash table created – 3025 – Distinct processing – 3026 – Set processing (V5R4) – 3027 – Sub query merge – 3028 – Grouping – 3030 – Materialized query table (MQT) – 3031 – Recursive common table expression (V5R4) – 5002 – Internal use only - SQE VE

IBM System i

Global DB Monitor Columns

• Columns common to all row types – QQRID: Row Type ID (1000, 3000, 3001, etc.) – QQUCNT: Unique Query/Request Identifier - surrogate key – QQJOB: Job name – QQUSER: Job user name – QVC102: CURRENT job user name – QQJNUM: Job number • Useful when multiple jobs collected in one DB monitor table – QQTIME: Time that the row was created • Useful for identifying queries in a given time period – QQJFLD: Join column • combination of QQJOB, QQUSER, QQJNUM, QQUCNT – QQI9: Thread Identifier • Useful for multi-threaded apps

32 IBM System i

Global DB Monitor Columns

Column used for Linking rows Row different data based together in the Type monitor on row type

IBM System i

Global DB Monitor Columns

• Monitor data is arranged chronologically – It is not guaranteed to be ordered by statement execution when multiple jobs monitored • Use QQUCNT and QQI5 columns to view and link together all the row types for a database request – QQUCNT is the surrogate key used to identify each unique query within a job • If analyzing monitor data for multiple jobs, then need to combine QQUCNT with QQJNUM – QQI5 used in conjunction with QQUCNT to identify the instance of that query within the job. • QQI5 and QQUCNT are also the linkage between the 3010 and 1000 row for SQL statements with host variables or parameter markers.

33 IBM System i Global DB Monitor Columns • QQ1000 and QQ1000L columns provide additional information • QQ1000 is a VARCHAR(1000) column • QQ1000L is a new CLOB column to hold all the information in a single row • QQ1000 and QQ1000L may not always contain the same information for the same row type; check the publication

IBM System i 1000 Row - SQL Statement Summary

• Row generated for each SQL operation (Update, Commit, etc.) • General SQL request info – QQ1000: First 1000 bytes of prepared text of SQL statement – QQ1000L: CLOB column containing all the text – QQC21: Type of SQL operation (OP, FE, CL, UP, IN, DL, ...) • 'MT' in this column indicates continuation row for SQL statements that exceed 1000 characters – no longer appears in V5R4 monitor data or after import to V5R4 • 'FE' is a Fetch summary row, NOT the actual number of fetch operations • ODP-related operation types: – SI, SV, OP, IN, UP, DL – QQI2: Number of rows updated/inserted/deleted – QQI3: Number of rows fetched (only on FE rows) • Actual number of rows fetched, not fetch attempts – QQI6: Elapsed time for this operation in microseconds (1/1,000,000) • Time to fetch all rows may not be included in with Open & Select operations, need to look at time on Fetch operation rows

34 IBM System i 1000 Row - SQL Statement Summary

• Access Plan Information – QQC103 & QQC104: Package/Program Name & Library – QVC18: Dynamic SQL Statement Type • 'E' - Extended Dynamic // 'S' – System Wide Statement Cache // 'L' - Prepared Stmt – QQC22 & QVC22: Access Plan Rebuild Code & Sub code • Sub code useful for IBM debug purposes – QVC24: Access Plan Save Status • 'Ax' values mean access plan could not be saved • Blank & 'Bx' values mean access plan was saved successfully • ODP Information – QQI5: Query Instance counter, 0 value means full open occurred – QQC15: Hard Close Reason Code (for 'HC' operation type) – QVC12: Pseudo Open Indicator – QVC13: Pseudo Close Indicator – QQC181 & QQC182: Cursor & statement name

IBM System i

1000 Row - Identifying "Problem" Jobs • Identify the most time consuming jobs: SELECT SUM(qqi6) "Total Time in mics", COUNT(*) "Total SQL Requests", qqjnum, qquser, qqjob FROM –monitor table name-- WHERE qqrid = 1000 GROUP BY qqjnum, qquser, qqjob ORDER BY 1 DESC;

• Analyze results to determine if a specific job should be target for further analysis • Use QQJNUM value to filter additional monitor data analysis

35 IBM System i

1000 Row - Identifying "Problem" SQL • Which SQL statements account for the most run time: SELECT SUM(qqi6) "Total Time in mics" , COUNT(*) "No. Times Run", qq1000l FROM –monitor table name-- WHERE qqjnum = ‘--job number--' AND qqrid = 1000 AND qqucnt <> 0 GROUP BY qq1000l ORDER BY 1 DESC;

•QQUCNT<> 0 eliminates non-ODP operations (Prepare, Commit, etc.) •Report does not capture Open, Fetch or Close time

IBM System i

1000 Row - Identifying "Problem" SQL -continued • Previous report does NOT include the Open, Fetch and Close time: WITH OpenFetchClose AS (SELECT qqjfld, sum(qqi6) AS qqi6_total FROM –monitor table name-- WHERE qqrid=1000 AND qqc21 IN ('SI','OP','FE','CL','IN','UP','DL‘, ‘HC’) GROUP BY qqjfld) SELECT sum(x.QQI6_Total) as "Total Time -mics", COUNT(*) "No. Times Run", z.qq1000l FROM OpenFetchClose x, –monitor table name--z WHERE x.qqjfld = z.qqjfld AND z.qqrid=1000 AND z.qqc21 IN ('SI','OP','IN','UP','DL') AND qqi5=0 GROUP BY qq1000l ORDER BY 1 DESC;

36 IBM System i

30xx - Query Optimization Row Types • Row types 30xx are referred to as Optimization rows – Contain information to construct the access plan for an SQL request – Presence of Optimization rows indicate that a Full Open occurred – Data Access Method columns contain details on the optimization process • QQEPT: Optimizer's estimated processing time • QQREST: Estimated number of rows selected • QQAJN: Estimated number of rows joined • QVPARD: Parallel degree requested • QVPARU: Parallel degree used

IBM System i

3000 Row - Table Scan • Table Information – QVPTBL - Table Name – QVPLIB - Schema Name – QQTOTR - Number of rows in table • Query Optimization Details – QQRCOD - Reason code, Why Table Scan chosen • T1 - No indexes exist. • T2 - Indexes exist, but none can be used. • T3 - Optimizer chose table scan over available indexes. – Index advised information – Obsolete in V5R4, refer to the 3020 row • QQIDXA – Index Advised • QQI2 - Number of primary keys • QQIDXD - Suggested keys for index

37 IBM System i 3000 Row - Table Scan Analysis

• Which queries involve table scans and show the "Estimated rows selected" versus "Total rows in tables": WITH tablescans AS (SELECT DISTINCT qqjfld, qqucnt, qqrest, qqtotr FROM –monitor table name-- WHERE qqrid = 3000) SELECT SUM(qqi6) "Total Time in mics", COUNT(*) "Times Run", a.qqucnt, integer(avg(b.qqrest)) "Est Rows Selected", integer(avg(b.qqtotr)) "Total Rows in Table", qq1000l FROM –monitor table name-- a, tablescans b WHERE qqrid = 1000 AND a.qqjfld = b.qqjfld GROUP BY a.qqucnt, qq1000l ORDER BY 1 DESC;

IBM System i 3001 Row - Index Analysis

• What indexes are used most often? SELECT count(*) "No. of Uses", qvilib "I Schema", qvinam "Index", qvplib "T Schema", qvptbl "Table" FROM –monitor table name-- WHERE qqrid = 3001 GROUP BY qvplib, qvptbl, qvilib, qvinam ORDER BY qvplib, qvptbl, qvilib, qvinam;

•*NONE / MTI(Node_nnnn) is a SQE temporary index

38 IBM System i

3002 Row - Temporary Index Creation

• Index & Table Information – QVPTBL and QVPLIB: Table name for which index is built – QVINAM and QVILIB: Temporary index name – QQRCOD - Reason the index build was done • I2 - ordering or grouping • I3 - selection and ordering/grouping • I4 - nested loop join – QQTOTR: Number of rows in table – QQRIDX: Number of entries in temporary index – QQSTIM: Timestamp for start of index build – QQETIM: Timestamp for end of index build – QQ1000L: Name of columns used for index keys • Column names are the "short" column names • QQ1000 can also be used 77 i want an i. © 2006 IBM Corporation

IBM System i 3002 Row - Temporary Index Analysis

• Which temporary index creations are done? SELECT qqucnt, qqc16 "Index Reused", qvptbl "Table Name", qvplib "Schema", qqtotr "Rows in Table", qqridx "Entries in Index", qq1000 "Key columns" FROM –monitor table name-- WHERE qqrid = 3002 AND qqjnum = --job number-- ORDER BY 6 desc;

39 IBM System i

3007 Row - Index Evaluation

• Shows all indexes evaluated for a given table, including which one (if any) was selected for use in this query, and which were not (and why) – Documentation classifies this as the "Optimizer Timed Out“, a 3007 row will be generated even when the optimizer does not timeout

• Commonly used columns – QVPTBL: Table name – QVPLIB: Schema name – QQC11: Optimizer timed out (Y or N) – QQ1000: Contains library qualified index names, each with a reason code • Reason code of 0 indicates index was selected • Other codes in second level text of CPI432C and CPI432D messages

IBM System i

3010 Row - Host Variables

• QQ1000 column contains substitution values for host variables or parameter marker – Values (separated by commas) correspond left to right with host variables/parameter markers • All values show up as character, no special indication of type • Floating point values show up with *F • QQUCNT & QQI5 columns must be used to determine which exact query that the substitution values belong to

• 3010 row type not generated for INSERT with VALUES clause or UPDATE with SET values

40 IBM System i 3020 Row - Index Advised Analysis (V5R4) • What indexes are advised, and how often? SELECT count(*) "No. Times Advised", qvplib "Schema", qvptbl "Table Name", cast(substr(qq1000l, 1, 100) as CHAR(100)) as Keys_Advised FROM –monitor table name-- WHERE qqrid = 3020 AND qqjnum = ‘--job number—’ GROUP BY qvplib, qvptbl, cast(substr(qq1000l, 1, 100) as CHAR(100)) ORDER BY 1 desc, 2, 3, 4;

Remember, system wide index advice is available in QSYS2/SYSIXADV

IBM System i 3020 Row - Index Advised Analysis (V5R4)

• What indexes are advised for a given table and query? SELECT qqjnum, qqucnt, qvplib "Schema", qvptbl "Table Name", cast(substr(qq1000l, 1, 100) as CHAR(100)) as Keys_Advised FROM –monitor table name-- WHERE qqrid = 3020 AND qqjnum = ‘--job number—’ ORDER BY 2, 3, 4;

Remember, SQE can advise more than one index for a given table

41 IBM System i 3014 Row – SQE or CQE, and Why?

• QQC16 column in 3014 row will contain a 'Y' when SQE is used to process the SQL statement or ‘N’ when CQE is used • QVC43 column will contain the reason why SQE was not used SELECT qqc16 as “N = CQE”, COUNT(*) FROM –monitor table name-- WHERE qqrid = 3014 GROUP BY qqc16;

SELECT x.qqjnum, x.qqucnt, y.QVC43 as "Reason for CQE", x.qq1000l FROM –monitor table name--x, –monitor table name--y WHERE x.qqjfld = y.qqjfld AND x.qqrid = 1000 AND x.qqc21 IN ('SI','OP','IN','UP','DL') AND x.qqucnt <> 0 AND y.qqrid = 3014 AND y.qqc16 = 'N' ORDER BY qqjnum, qqucnt;

IBM System i

3015 Row - SQE Statistics Advised

• Generated by SQE when it determines that a column statistic needs to be collected or refreshed • Commonly used columns – QVPTBL: Table name – QVPLIB: Table library name – QQC11: Statistics request type • 'N' - No Statistic Existed for Column • 'S' - Column Statistic was Stale – QQ1000: Name of column that statistic is advised for • Can be used to recognize possible indexing opportunities • By default SQE will collect these column statistics – Usually no action is required 84 i want an i. © 2006 IBM Corporation

42 IBM System i

3015 Row - SQE Stats Advised Analysis • Which column statistics have been advised by SQE? SELECT qqucnt, qvptbl "Table", qvplib "Schema", qqc11 "Reason Stat Advised", CAST(SUBSTR(qq1000,1,100) as CHAR(100)) as Column_Name FROM –monitor table name-- WHERE qqrid = 3015 ORDER BY 2, 5;

ƒRemember column statistics are created in the background automatically by default for all Stats Advised ƒQQUCNT should be used to analyze SQL request to determine if an index would be beneficial 85 i want an i. © 2006 IBM Corporation

IBM System i Collection of Optimization Row Types

• Optimization row types (30xx) are generated first, so monitor must be running before an SQL statement is run in order to get the access plan details – To ensure capture of this data for a batch job, start monitor (STRDBMON JOB(*ALL) )before the batch job starts and collect over the entire length of the job (or as much as needed) – For remote clients, start the monitor (STRDBMON JOB(*ALL) ) before establishing the connection - this will ensure optimization data is collected • If optimization data was not collected for a given query, use query text collected by monitor and rerun the query – Be careful to rerun the query in the same environment • If monitor is ended while query is in progress, optimization data is collected but other data for that query (SQL text, etc.) is not collected • A job can be monitored by a maximum of two collections • New SQE Plan Cache support can help – no SQL monitor required

43 IBM System i

ODP Creation/Full Open Analysis

• QQUCNT - Unique number for each unique statement within a job – QQUCNT value assigned at full open time, stays constant for all subsequent reusable instances of that query – SQL operations (Prepare, Describe, Commit) that don't need an ODP have QQUCNT = 0 and thus cannot be linked to the SQL query • QQ1000 column in the Prepare or Describe 1000 row will contain the prepared SQL text • QQI5 - Refresh counter – Used in conjunction with QQUCNT value to look at specific instance of a query within a job – Only valid on 3010 and 1000 SQL summary rows • Non-ODP 1000 rows (Commit, Prepare, etc.) have QQI5 = 0 • Not set for optimization rows – QQI5 = 0 indicates Full Open when the SQL operation is – Update, Insert, Delete or Open/Select

IBM System i

• Same statement executed 4 times within the job (table scan). Monitor data for the non-reusable and reusable ODP cases... Non-Reusable ODP QQRID... QQUCNT... QQI5 3000 1 - Reusable ODP 3014 1 - QQRID... QQUCNT... QQI5 1000 1 0 3000 1 - 3000 2 - 3014 1 - 3014 2 - 1000 1 0 1000 2 0 3000 2 - 3000 3 - 3014 2 - 3014 3 - Start of 1000 2 0 1000 3 0 Reuse 1000 2 1 3000 4 - 1000 2 2 3014 4 - 1000 4 0

44 IBM System i

Analyzing Full Opens

• Which SQL requests are significantly affected by Full Opens: SELECT SUM(qqi6) "Total Time in mics", COUNT(*) "Nbr Full Opens", qq1000 FROM –monitor table name-- WHERE qqjnum = ‘--job number--' AND qqrid = 1000 AND qqi5 = 0 AND qqc21 IN ('OP', 'DL', 'IN', 'UP') GROUP BY qq1000 ORDER BY 1 DESC;

IBM System i Analyzing Full Opens • Next step is determining why an SQL statement has an excessive number of full opens (detailed analysis on each execution): SELECT a.qqjnum, a.qqrid, a.qqucnt, a.qqi5, a.qqc21, a.qqc15 "HC Reason Code", a.qqc22 "Rebuild Reason Code", a.qqc181 "Cursor Name", a.qqc182 "Stmt Name", a.qq1000 FROM –monitor table name--a WHERE a.qqjnum = ‘—job number—’ AND a.qqrid = 1000 AND ((a.qqucnt IN (SELECT b.qqucnt FROM –monitor table name--b WHERE b.qqjnum = ‘—job number—’ AND b.qqrid = 1000 AND b.qqc21 = 'OP' AND b.qq1000 LIKE '–specific query text%')) OR (qqc21 IN ('DI', 'ST', 'CM', 'RO'))) ORDER BY a.qqtime; • Interesting QQC21 statement type values for ODP analysis – HC: Hard Close – DI: Disconnect – ST: Set Transaction – CM: Commit – RO: Rollback

45 IBM System i Analyzing Full Opens/ODP Creation • Hard Close Reason Codes(QQC15): – 1: 'Internal Error' – E: 'Repeatable Read' – 2: 'Exclusive Lock' – F: 'Lock Conflict or QSQPRCED Threshold-Library' – 3: 'Interactive SQL Reuse Restriction' – G: 'Lock Conflict or QSQPRCED Threshold-File' – 4: 'Host Variable Reuse Restriction' – H: 'Execute Immediate Access Plan Space' – 5: 'Temporary Result Restriction' – I: 'Dummy Cursor Threshold' – 6: 'Cursor Restriction' (After First Execution) – 7: 'Cursor Hard Close Requested' (Proprietary – J: 'File Override Change' attribute) – K: 'Program Invocation Change' – 8: 'Internal Error' – L: 'File Open Options Change' – 9: 'Cursor Threshold' – M: 'Stmt Reuse Restriction' – A: 'Refresh Error' – B: 'Reuse Cursor Error' – N: 'Internal Error' – C: 'DRDA AS Cursor Closed' – O: 'Library List Change' – D: 'DRDA AR Not WITH HOLD' – P: 'Exit Processing' (End Connection) – Q: ‘Set Session User’

IBM System i Identifying and Tuning Problem Areas

• Identify the dominate, most time-consuming queries, and work on them individually – Determine which jobs are consuming the most time – Within a job, determine what type of SQL operations are being performed and how much time is consumed by each type – Determine which query (unique count) is the most time consuming and executed most often of the SQL operations – Now, you have a single SQL statement to analyze • Review higher-level indicators of DB2 performance problems: – Find all temporary index creations – Determine which queries are using table scans – Determine which queries are using hash tables – Determine which queries are performing full opens – … 92 i want an i. © 2006 IBM Corporation

46 IBM System i

Identifying and Tuning Problem Areas

• First concentrate on repetitious non-reusable ODPs, table scans, long index creations, hash table usage and no index available – Also look for repetitious short-running queries that are not optimized well – Joins and sorts can be more difficult to analyze, but if they are accounting for a significant portion of run time, they need to be addressed as well – Fine tuning smaller problems should be done after large problems addressed • Do not forget the “optimization goal”, and the applications intended behavior – *ALL I/O plans for *FIRST I/O applications can look like performance problems – Set the appropriate optimization goal based on the application behavior – Create indexes to support *FIRST I/O plans • The base set of reports and queries provided will help produce useful results in most situations – Use the custom queries to help you understand the data and SQL flow

IBM System i For Smaller Monitor Collections

• Large monitor tables can slow analysis – Try collecting only a specific job or connection (if possible) – Only collect Monitor Data for long running SQL statements - based on Optimizer's Estimated Runtime – New filtering options available • Expected run time threshold • Eliminate SQL statements generated by DB2 • IP address, job and object filters – New subset options available

• Consider starting with the Summarized Database Monitor

47 IBM System i For Faster Analysis

• If you are seeing slow response times during your analysis... – Tune the queries, used to tune your queries! – Create smaller monitor file with only the rows you're interested in • Use the GUI or a query to pick out only those jobs of interest, then copy the rows for those job(s) into a separate monitor table – CRTDUPOBJ and INSERT w/subselect specifying the QQJNUM value(s) OR – CREATE TABLE small AS (SELECT ... WHERE QQJNUM = xyz) WITH DATA

– Create indexes on the monitor file over common selection and grouping/order by columns • Common key columns to get you started: – QQJNUM – QQRID – QQUCNT, QQI5 – QQRID, QQ1000 (QQ1000L is a CLOB column and indexing is not allowed) – QQRID, QQC21

IBM System i Graphical DB Monitoring Tools - Review

• iSeries Navigator – SQL Performance Monitors – Summary->Memory-based Monitor collection – Detailed -> STRDBMON collection • Includes the ability Start, Pause & End collections • Analysis reports to assist investigation and provide awareness • Visual Explain on queries in detailed monitor collections – SQE Plan Cache Snapshots – SQE Plan Cache – URL: http://www.iseries.ibm.com/access

• insure/SQL Toolset by Centerfield Technology – Index Advisor based on Monitor Data – Monitor Data Analysis services – URL: http://insureSQL.com

48 IBM System i Additional Information

• DB2 UDB for iSeries home page – ibm.com/iseries/db2 • Education Resources - Classroom & Online (ibm.com/iseries/db2/db2educ_m.htm) – iSeries SQL Performance workshop (ibm.com/servers/eserver/iseries/service/igs/db2performance.html) – Online White papers on DB2 Performance (ibm.com/servers/enable/site/education/ibo/view.html?wp#db2) • Indexing & Statistics Strategy • Using iSeries Database Monitor to Identify and Tune SQL Queries – Online Labs on DB2 Performance (ibm.com/servers/enable/site/education/ibo/view.html?oc#db2) • LAB: Performance Tuning DB2 UDB with DB Monitor & Visual Explain • DB2 UDB for iSeries Publications – Online Manuals: ibm.com/iseries/db2/books.htm – Porting Help: ibm.com/servers/enable/site/db2/porting.html – DB2 UDB for iSeries Redbooks (ibm.com/redbooks) • Stored Procedures, Triggers, & User-Defined Functions on DB2 UDB for iSeries (SG24- 6503) • Advanced Functions and Administration on DB2 UDB for iSeries (SG24-4249) • SQL Query Engine Redbook (www.iseries.ibm.com/db2/sqe.html ) – SQL/400 Developer's Guide by Paul Conte & Mike Cravitz • http://www.iseriesnetwork.com/str/books/Uniquebook2.cfm?NextBook=183

IBM System i Trademarks and Disclaimers 8 IBM Corporation 1994-2006. All rights reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country.

The following terms are trademarks of International Business Machines Corporation in the United States, other countries, or both:

AS/400 e-business on demand i5/OS AS/400e IBM OS/400 eServer IBM (logo) System i5 iSeries Rational is a trademark of International Business Machines Corporation and Rational Software Corporation in the United States, other countries, or both. Intel, Intel Logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Other company, product or service names may be trademarks or service marks of others.

Information is provided "AS IS" without warranty of any kind.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products.

All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction.

Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. 98 i want an i. © 2006 IBM Corporation