SQL in V9 of DB2 for z/OS IDUG 2012 - DENVER
Bonnie K. Baker
Bonnie Baker Corporation PO Box 18123 Tampa, FL 33679-8123 USA
1-813-477-4885
[email protected] http://www.bonniebaker.com
1 AGENDA
PERFORMANCE ENHANCEMENTS
SQL CHANGES
OBJECT CHANGES
2 PERFORMANCE ENHANCEMENTS
Buffer Pool Management
• Efficient large BP (>5GB) support
• Bigger prefetch and deferred-write quantity for bigger buffer pools – Doubling the prefetch quantity for large buffer pools • Large buffer pool defined as VPSEQT x VPSIZE > 160MB • Max of 128 in V8; now 256KB in V9 for SQL tablespace scan – Was 32 4K pages; now 64 4K pages • 256 in V8; now 512KB in V9 for utilities – Was 64 4K pages; now 128 4K pages
3 PERFORMANCE ENHANCEMENTS
Buffer Pool Management continued
• Sequential Prefetch has changed in other ways - Now uses Sequential Detection for first 8 Get Pages
More use of dynamic prefetch over sequential prefetch
• Except for tablespace scans
Considered more robust
• Data is usually NOT all sequential NOR all random • Allows for data status in between REORGs • Plan_table PREFETCH column will be ‘D’ or ‘S’
4 PERFORMANCE ENHANCEMENTS
Consolidated Temporary Work Space
• Single source for all temporary space in DB2 for
– SORTWORK – SORTOUT – DECLARED GLOBAL TEMPORARY TABLES – CREATED GLOBAL TEMPORARY TABLES – MATERIALIZED VIEWS – STATIC SCROLL CURSORS – FINAL_TABLEs
– For Developers, this means more sharing, more possible -904s – AND 32K Work Space is used much more – should be same as 4K?
5
PERFORMANCE ENHANCEMENTS
Consolidated TEMP Space continued
• Supports 4K & 32K page sizes
. Automatic selection of the appropriate page size (for efficiency) . Expect an increased use of the 32K temp space . Consider sizing your 32K space @ 50% to 100% of your 4K buffers • Monitor statistics and adjust for actual usage
• Access is virtualized for small amounts of data (approximately 32K), eliminating cost of work file creation (reduced CPU and I/O)
• NEW ZPARM (MAXTEMPS) prevents WORKFILE monopolization – Similar to MaxDegree
6 NEW SQL • MERGE • SELECT FROM . UPDATE . DELETE . MERGE • FETCH FIRST / ORDER BY in Subselect • INTERSECT • EXCEPT • SKIP LOCKED DATA • TRUNCATE • NEW . Built In Functions . Scalar Functions . Session Variables • NATIVE SQL STORED PROCEDURES
7 NEW SQL
Select from Update or Delete
SINGLETON OR CURSOR SELECT
SELECT data from an UPDATE
DECLARE CAND CURSOR FOR SELECT ATHLETE, SCORE FROM FINAL_TABLE (UPDATE EVENTS SET SCORE = SCORE + 10 WHERE EVENT = ‘BOXING’)
8 NEW SQL
Select from Merge
SELECT data from a MERGE
with an INCLUDE column (returns the compare column from the ON clause + an indicator of whether the row was Updated or Inserted)
DECLARE CURSOR C1 FOR SELECT MEMBER_ID, UPSERT_INDICATOR FROM FINAL_TABLE ( MERGE INTO MEMBER_PROFILE AS T INCLUDE (UPSERT_INDICATOR CHAR(1)) USING (VALUES (:col1, :col2, :col3) ) AS A (MEMBER_ID, MEMBER_NAME, MEAL_PREFERENCE) ON (T.MEMBER_ID = A.MEMBER_ID) ------THE COMPARE COLUMN WHEN MATCHED THEN UPDATE SET T.MEMBER_NAME = A.MEMBER_NAME, T.MEAL_PREFERENCE = A.MEAL_PREFERENCE, UPSERT_INDICATOR = 'U' WHEN NOT MATCHED THEN INSERT (MEMBER_ID, MEMBER_NAME, MEAL_PREFERENCE ,UPSERT_INDICATOR) VALUES (A.MEMBER_ID, A.MEMBER_NAME, A.MEAL_PREFERENCE, 'I') NOT ATOMIC CONTINUE ON SQLEXCEPTION ) 9
NEW SQL
FETCH FIRST/ ORDER BY in SUBSELECT
INSERT INTO T2 (SELECT C1, C2, C3 FROM T1 FETCH FIRST 10 ROWS ONLY ORDER BY C2, C1);
10 NEW SQL INTERSECT AND EXCEPT
– Intersect returns the records that two result sets have in common Frequently used in ‘find the duplicates query’ Rows from either result set that do not match are excluded Also INTERSECT DISTINCT to eliminate duplicates – Except returns the rows from RS1 that are not in RS2 Similar to symmetric difference join
JOIN TO GET COLUMNS FROM TWO OR MORE TABLES UNION/EXCEPT/INTERSECT TO GET ROWS
RS1
RS1 RS2
RS2
11 NEW SQL data data dataX data dataX dataX data data
SKIP LOCKED DATA
• Rows with incompatible locks held by others are skipped
• SQL Clause available – On SELECT INTO, PREPARE, searched UPDATE, searched DELETE, and UNLOAD, so far – Effective only whenISOLATION = CS or RS – Effective only when data is locked at the row or page level
• QW0018SK – Rows skipped due to incompatible lock held – Reported in IFCID 018, not in MSTR or user address space
• Logic / Scenario – When a transaction needs to find work to do, regardless of order. – Messaging applications without strict ordering requirements expect to be able to skip over records that are locked by other transactions
12 NEW SQL TRUNCATE part 1
• Allows fast delete of all rows in base tables or declared global temporary tables – Simple, segmented, partitioned, or universal tablespaces – Use IMMEDIATE option – operation cannot be rolled back Allows reusing the allocated space immediately for subsequent insert operations in the same unit of work without committing. Not allowed if the table has in-flight modifications • Will Delete rows without firing DELETE triggers • Option to REUSE or DROP STORAGE – Useful for nightly refresh of summary tables, warehouses, etc.
13 NEW SQL
TRUNCATE part 2 • Table cannot be TRUNCATEd if it is a parent in a defined referential constraint
• Acts just like a mass DELETE (a DELETE with no WHERE clause) for tables with Change DataCapture, Multilevel Security VALIDPROC.
• ROLLBACK: TRUNCATE IMMEDIATE: Other tables are rolled back. The truncated table will be empty. TRUNCATE: The truncate is rolled back, along with changes to other tables.
• DELETE / ALTER, SYSADM, DBADM or table ownership privileges required. The ALTER privilege is needed for IGNORE DELETE TRIGGERS
14 NEW SQL Built-in Functions
• RANK Generates a row number defining the rank as 1 plus the number of rows that precede the row. If two or more rows are not distinct with respect to the ordering, there will be one or more GAPS in the sequential rank numbering. This is also referred to as OLYMPIC RANKING.
• DENSE_RANK Similar to RANK but ensures there are NO GAPS. Useful for statistical analysis
Even though these ranking functions are ordered, cursors remain updatable. These functions do not make the cursor unambiguously read-only! 15 NEW SQL Built-in Functions
• ROW_NUMBER DEFAULT DATATYPE = BIGINT – Creates a sequential row number that is computed for each row starting with 1. – No number skipping.
SELECT SMALLINT(ROW_NUMBER() OVER (ORDER BY score DESC)) AS ROW_NUMBER, athlete, score FROM scores ORDER BY score DESC
16 SOUNDEX Scalar Function • Powerful scalar functions for name identification – The SOUNDEX function returns a 4 character code that represents the sound of the words in the argument. – The result can be used to compare with the sound of other strings. – The SOUNDEX function is useful for finding strings for which the sound is known but the precise spelling is not.
SELECT athlete,SOUNDEX(athlete) – the 4 character code for the name - as snd_value FROM scores;
SELECT empno,lastname FROM dsn8910.emp WHERE SOUNDEX(lastname) = SOUNDEX(‘Loucesy’)
17 18 Unicode SBCS (single-byte character set):
An international character code for information processing that is designed to encode all characters that are used for written communication in a simple and consistent manner. The Unicode character encoding was originally established as a fixed-width, 16-bit encoding to provide enough code points for all the scripts and technical symbols in common usage around the world, plus some ancient scripts.
19 NEW SQL
DIFFERENCE Scalar Function • A function that returns a value from 0 to 4 • Represents the difference between the 4-character sounds of two strings based on applying the SOUNDEX function to the strings. – A value of 4 is the best possible sound match. – A value of 0 is the worst possible sound match. SELECT DIFFERENCE(‘CONSTRAINT’,’CONSTANT’) AS DIFFERENCE1, DIFFERENCE(‘CONSTANT’, ‘CONSTRICT’) AS DIFFERENCE 2, SOUNDEX(‘CONSTRAINT’) AS CONSTRAINT, SOUNDEX(‘CONSTANT’) AS CONSTANT, SOUNDEX(‘CONSTRICT’) AS CONSTRICT FROM SYSIBM.SYSDUMMY1;
DIFFERENCE1 DIFFERENCE2 CONSTRAINT CONSTANT CONSTRICT 4 2 C523 C523 C534
20 NEW SQL
. Session Variables contain server and application process information
. Builds on the 11 Session Variables introduced in V8
. Use GETVARIABLE to obtain the DB2 V9 zparm DSNHDECP value
SYSIBM.APPLICATION_ENCODING_SCHEME SYSIBM.DSNHDECP_NAME SYSIBM.COBOL_STRING_DELIMITER SYSIBM.DYNAMIC_RULES SYSIBM.DATE_FORMAT SYSIBM.ENCODING_SCHEME SYSIBM.DATE_LENGTH SYSIBM.MIXED_DATA SYSIBM.DECIMAL_ARITHMETIC SYSIBM.NEWFUN SYSIBM.DECIMAL_POINT SYSIBM.PAD_NUL_TERMINATED SYSIBM.DEFAULT_DECFLOAT_ROUND_MODE SYSIBM.SQL_STRING_DELIMITER SYSIBM.DEFAULT_DEFAULT_SSID SYSIBM.SSID SYSIBM.DEFAULT_LANGUAGE SYSIBM.STANDARD_SQL SYSIBM.DEFAULT_LOCALE_LC_CTYPE SYSIBM.TIME_FORMAT SYSIBM.DISTRIBUTED_SQL_STRING_DELIMITER SYSIBM.TIME_LENGTH
SET :HV1 = GETVARIABLE(SYSIBM.DECIMAL_POINT)
21 Native SQL Procedural Language
• Eliminates generated C code and compilation
• Fully integrated into the DB2 engine – Any SQL procedure created without the FENCED or EXTERNAL keywords are native SQL procedures
• zIIP enabled for DRDA clients
• Extensive support for versioning: – VERSION keyword on CREATE PROCEDURE – CURRENT ROUTINE VERSION special register – ALTER ADD VERSION – ALTER REPLACE VERSION – ALTER ACTIVATE VERSION
• BIND PACKAGE with new DEPLOY keyword 22 NEW SQL REOPT BIND parameter – see next page for details
V9 – REOPT(NONE) – REOPT(ALWAYS), AKA REOPT(VARS) • Dynamic & Static SQL – REOPT(ONCE) • Dynamic SQL
23 INDEX LOOKASIDE
V8 the Clustering Index could use lookaside for SELECTs, INSERTs, and UPDATEs But No lookaside for DELETEs
V9 can use Index Lookaside for DELETEs
24 EXPRESSIONS & AUTOMATIC CREATE
• EXPRESSIONS CREATE INDEX NEW_INDEX ON BIGTABLE ( COLA, MONTH(TRAN_TIMESTAMP), COLX )
OTHER EXAMPLES: UPPER(LASTNAME) SUBSTR(PONBR,1,5)
CAUTION: IMPACT ON REORG AND/OR INDEX REBUILD
25
CLONE Tables
• Allows fast replacing production data without renames and rebinds – A capability to support online load replace
• ALTER TABLE to create a Clone Table – All indexes are also cloned – Table and Index DATA are NOT copied – Base and Clone tables share the same table space and index names – Underlying data sets are differentiated by a data set instance number
26 CLONE Tables… • On single-table table spaces (partitioned or non-partitioned) • Use insert or load to populate clone tables • Utilities (except RUNSTATS) can operate on clone tables with a new CLONE keyword • Use EXCHANGE to switch logical names with underlying data – FLIGHTS now references the data that was in FLIGHTS_CLONE • Indicated in SYSTABLESPACE .CLONE
DDL ALTER TABLE FLIGHTS ADD CLONE FLIGHTS_CLONE
EXCHANGE DATA BETWEEN TABLE FLIGHTS AND FLIGHTS_CLONE
27
TABLESPACE CHANGES
WHAT ARE UNIVERSAL TABLESPACES? WHY DO WE NEED THEM?
WHAT IS A “NOT LOGGED TABLESPACE”?
WHAT IS INSERT WITH “APPEND”?
28 UNIVERSAL TABLESPACES
V8 SIMPLE, SEGMENTED, PARTITIONED
PARTITIONED TABLESPACES USE SIMPLE PAGE FORMATS, ETC.
SEGMENTED TABLESPACES HAVE MANY ADVANTAGES
BETTER SPACE MANAGEMENT MASS DELETES: INSTANTANEOUS VIA SPACE MAP PAGE; MINOR LOGGING IMAGE COPIES: COPY ONLY FORMATTED PAGES INSERT LOGIC: MORE INFO ON SPACE MAP PAGE
29 UNIVERSAL TABLESPACES cont. • V9 TABLESPACE TYPES
SEGMENTED (SPECIFY SEGSIZE) PARTITIONED (SPECIFY NUMPARTS)
BUT ALSO:
SEGMENTED (PARTITIONED BY GROWTH) = TYPE G SPECIFY SEGSIZE AND MAXPARTITIONS DSSIZE DETERMINES PARTITION SIZE GROW AS NEEDED WITH NO PARTITIONING COLUMNS NEW PARTITION USES PRIOR COMPRESSION DICTIONARY
SEGMENTED PARTITIONED BY RANGE = TYPE R SPECIFY NUMPARTS AND SEGSIZE LIKE OUR OLD PARTITIONED TABLESPACE BUT WITH ADVANTAGES OF SEGMENTED TABLESPACES
30 NOT LOGGED TABLESPACES
• UNRECOVERABLE DATA
• CAN IMPLEMENT WITH ALTER
• GOOD FOR HUGE PARALLEL INSERT JOBS
• GOOD FOR ALL OR NONE JOBS
• USE RECOVER TO GET BACK TO BEGINNING
• ESTABLISH POINT OF RECOVERY AFTER 31 NOT LOGGED Tables
• Actually NOT LOGGED tables, indexes, LOBs, XML • ALTER / CREATE a TABLESPACE as NOT LOGGED – ALTER not allowed if in same UOW with an update to the SAME tablespace • Indexes, LOB, and XML inherit the logging attribute of the base – This are considered “Linked” objects • Affects the UNDO / REDO records – Control information is still logged • LOB continues to log system pages & auxiliary indexes • A Unit of Recovery (UoR) is still created --- why?
LOG YES is a synonym for LOGGED LOG NO is a synonym for NOT LOGGED
32 DSN 32 INSERT WITH APPEND
• V8’s TABLE INSERT ALGORITHM?
CLUSTER NO ROOM FOR INSERT ON OPTIMAL PAGE? OPTIMAL PAGE LOCKED?
EVER-INCREASING KEY INSERTS AT END WHAT IF PAGE IS LOCKED OR FULL NO PCTFREE NEEDED, RIGHT? WRONG!
• V9’s INSERT WITH APPEND?
MOD IGNORE CLUSTER FOR INSERTS GREAT IF REORGing ANYWAY AFTER PROGRAM RUNS TRADEOFF: Storing the data quickly at the cost of space & certain queries More effective use of space at the cost of searching for space 33
DDL BUILT -IN UPDATE TIMESTAMP = ROW CHANGE TIMESTAMP
• INCORPORATES INSTRUCTIONS TO DB2 TO MAINTAIN THE TIMESTAMP & TRACK THE ROWID AT EVERY UPDATE PER ROW/PAGE
• ANYONE CAN USE THIS TIMESTAMP COLUMN FOR OPTIMISTIC LOCKING AND REFERENCE IT BY COLUMN NAME OR BY “ROW CHANGE TIMESTAMP”
CREATE TABLE PROD.BIGTABLE
(COLA INTEGER NOT NULL, COLB INTEGER NOT NULL, ROW_LAST_UPD NOT NULL GENERATED ALWAYS --- or “BY DEFAULT” FOR EACH ROW ON UPDATE AS ROW CHANGE TIMESTAMP)
34 BUILT-IN UPDATE TIMESTAMP
SELECT Col1, Col2, Col3, UNIQUE_ID, ROW CHANGE TIMESTAMP FOR TableA FROM TableA WHERE ...
. . .
UPDATE TableA SET ... WHERE UNIQUE_ID = BIGINT(?) AND ROW CHANGE TIMESTAMP FOR TableA = :VALUE-SELECTED
35 INSTEAD OF TRIGGERS
ON SELECT/DELETE/UPDATE BY USER FIRE TRIGGER TO DO THIS INSTEAD OF THAT
MAY BE ABLE TO REPLACE SOME VIEWS
MAY USE TO UPDATE TABLEA AND TABLEB (FOR UPDATING A VIEW THAT IS A JOIN OF TWO TABLES)
36 MORE PERFORMANCE IMPROVEMENTS • Optimizer Exploit FETCH N ROW(S) ONLY
EXAMPLE
SELECT PONBR FROM BIGTABLE WHERE CUSTNO = :HVCUSTNO ORDER BY PONBR DESC FOR FETCH OF 1 ROW ONLY
• Explain change – INSERT, UPDATE, and DELETE have the same support for subquery to join transformation • V8 was limited to SELECT – Support for Optimization Hints (OPTHINT) FOR SUBSELECTS OF INSERTS. ETC.
38