Abstract: Native SQL procedures were introduced with DB2 9 for z/OS, and SQL Procedure Language (SQL PL) was extended to UDFs with DB2 10. DB2 11 has delivered additional SQL PL-related enhancements. This session will provide information that attendees can use in the development, management, and monitoring of native SQL procedures and UDFs. It will also cover exploitation of native SQL procedures and UDFs to enhance DB2 for z/OS performance, scalability, and security - particularly for client-server applications. Robert Catterall IBM [email protected] 1 For years, COBOL ruled the roost with respect to the coding of DB2 for z/OS stored procedures. SQL PL, more recently on the scene, will, I think, come to be the dominant language for DB2 stored procedure development (and for development of user-defined functions). I believe that DB2 for z/OS people – systems programmers, DBAs, and application developers – should have at least some knowledge of SQL PL, and DB2-using organizations should engage in at least some degree of SQL PL routine development. 2 I’ll start by providing some information about the history of DB2 for z/OS native SQL procedures (and UDFs), and some recently delivered enhancements in this area. Then I’ll talk about the benefits of native SQL procedures and UDFs with respect to application performance, scalability, and security. After that I’ll discuss topics related to native SQL procedure and UDF development and management. I’ll conclude with some thoughts on shifting to SQL PL from other languages for DB2 stored procedure and UDF development. 3 A brief SQL PL history lesson, and a look at what’s new. 4 It was DB2 for z/OS Version 7 that introduced the ability to write stored procedure programs using only SQL statements (in other words, the stored procedure’s source is contained entirely within the associated CREATE PROCEDURE statement). The enriched SQL that enabled creation of SQL-only stored procedure source is called SQL Procedure Language, or SQL PL for short. To the existing SQL DML (e.g., SELECT, INSERT), DCL (GRANT and REVOKE), and DDL statements (CREATE, ALTER), SQL PL added what are called control statements. These statements are about variable declaration, assignment of values to variables, and logic flow control (via statements such as IF, WHILE, ITERATE, LOOP, and GOTO). 5 What did organizations like about SQL PL as introduced with DB2 for z/OS V7? Mostly they liked that the language expanded the pool of people who could develop DB2 stored procedures – people who didn’t know a programming language such as COBOL or C or Java could write DB2 stored procedures using SQL PL. What did organizations not like about first-generation SQL PL routines? Some organizations didn’t like the fact that the routines were turned into C language programs with embedded SQL DML statements as part of preparation for execution. These programs ran – like other external-to-DB2 stored procedure programs – in WLM-managed stored procedure address spaces. The concern in some people’s mind had to do with the CPU consumption of a C language stored procedure program, which often was a good bit greater than the CPU cost of executing a COBOL stored procedure program of equivalent functionality. Certain organizations opted not to use DB2 V7-style SQL procedures because the negative of added CPU cost (versus COBOL stored procedures) outweighed the benefit of an expanded pool of people who could develop DB2 stored procedures. 6 A very important enhancement was introduced with DB2 9 for z/OS running in new-function mode: native SQL procedures (organizations that migrated to DB2 10 from DB2 V8 can utilize native SQL procedure functionality in a DB2 10 NFM environment)). Unlike DB2 V7 or V8 SQL procedures (still supported in DB2 9, 10, and 11 systems, and now called external SQL procedures), DB2 9 (and beyond) native SQL procedures are internal procedures – they run in the DB2 DBM1 address space, as opposed to a WLM-managed stored procedure address space. Other differences versus external SQL procedures: • A native SQL procedure’s sole executable is its DB2 package – there’s nothing else to it. • A native SQL procedure executes under the task of the invoking application process – not under its own TCB in a stored procedure address space. • A native SQL procedure provides some capabilities not available for external SQL procedures, such as nested compound SQL statements (very handy for implementation of multi-statement condition handlers). • That functionality gap versus external SQL procedures continues to grow. DB2 11 delivered some important enhancements that are only applicable to native SQL PL routines (I’ll cover these enhancements momentarily). 7 DB2 10 for z/OS (in new-function mode) expanded the usefulness of native SQL PL routines to user-defined functions (UDFs). I like to call these routines “native” SQL UDFs, but the official term is “compiled SQL scalar functions.” As is true of SQL procedures, “native” SQL UDFs expand the pool of people who can develop user-defined functions in a DB2 environment. Could UDFs be written in SQL before DB2 10 NFM? Yes, but they were very limited in functionality (more on this to come). 8 Prior to DB2 10 (NFM), if you wanted to create a user-defined function that had some degree of sophistication, including declaration of variables and assignment of values to same, and logic such as “do loops,” you pretty much had to go with an external UDF that would be associated with a program written in COBOL (for example). With DB2 10 NFM (and beyond), that type of sophistication can be achieved with a compiled SQL scalar function written in SQL PL. As with a native SQL procedure, the “body” of a “native” SQL UDF is contained within what’s called a compound statement. Other things that “native” SQL UDFs have in common with native SQL procedures: • They run in the DB2 DBM1 address space. • They run under the task of the invoking application process. • The one and only executable is the “native” SQL UDF’s package – nothing about a “native” SQL UDF is external to DB2. 9 The ability to have a SQL PL routine as the body of a “native” SQL UDF is not the only DB2 10 enhancement related to SQL UDFs. Other goodies: • The RETURN statement in a compiled SQL scalar UDF (also called a non- inline SQL scalar UDF) can contain a scalar fullselect. Prior to DB2 10 NFM, not only could a SQL scalar UDF not contain a scalar fullselect – it couldn’t even reference a column of a table. • A table UDF, which returns a set of values, can be written in SQL in a DB2 10 NFM (and beyond) system (previously, a table UDF had to be external, i.e., associated with an external-to-DB2 program written in a language such as COBOL). Note that DB2 10, in addition to having new CREATE (and ALTER) statements for compiled SQL scalar UDFs and SQL table UDFs, still supports the pre-DB2 10 type of SQL scalar UDF (now called an inline SQL scalar UDF). 10 I had stated previously that one benefit of native SQL procedures versus external SQL procedures is the fact that new capabilities are, by and large, delivered for the former and not for the latter. An example of such a new capability is the XML support provided for native SQL procedures (and not external SQL procedures) starting with DB2 10 in new-function mode: • Input and output parameters for a native SQL procedure can be of the XML data type (prior to DB2 10 NFM, XML data had to be passed to or received from a native SQL procedure in character string form). Native SQL procedures can also declare variables of the XML data type. • The same holds true for SQL UDFs (whether scalar – inline or compiled – or table UDFs). 11 DB2 11 provides a couple of very cool native SQL procedure-only enhancements. One of those is support for array parameters: they can be passed to or received from a native SQL procedure, or declared as variables in a native SQL procedure (and the same is true for a compiled SQL scalar UDF). An important caveat: when array parameters are used for a native SQL procedure, the procedure has to be called from another SQL PL routine or from a Java program that connects to DB2 using the IBM Data Server Driver for JDBC and SQLJ type 4 driver (I underlined that Java bit because Java client-side programmers have long wanted DB2 stored procedure support for array parameters). Note that one first creates an array (it’s a category of user-defined data type) and then uses it with a native SQL procedure (or UDF). You can create ordinary arrays, in which a value in the array is identified by its ordinal position within the array (e.g., element 1, or 12, or 27), and associative arrays, in which values are ordered by, and can be referenced by, user-provided index values. The example on the slide shows a CREATE statement for an ordinary array. DB2 11 also provides several new built-in functions that facilitate working with arrays. 12 Another new and cool DB2 11 feature that is exclusive to native SQL procedures is the autonomous transaction. Here’s what this is about: suppose you want an application transaction to make a data change (e.g., insert a row into a DB2 table) that will persist even if the transaction itself is subsequently rolled back (perhaps due to a SQL error encountered by the transaction)? An autonomous transaction, which is a special form of a native SQL procedure, will do that for you, because it has a unit of work that is completely independent of the unit of work associated with the transaction that called the procedure.