Beyond Purexml: DB2 in a JSON World
Total Page:16
File Type:pdf, Size:1020Kb
#IDUG Beyond pureXML: DB2 in a JSON World Philip Nelson Lloyds Banking Group / ScotDB Limited Session Code: E15 May 16th 2014, 10:30 – 11:30 | Platform: DB2 for LUW #IDUG 2 Agenda • Introduction to JSON • DB2 JSON Support • Beyond DB2 JSON Support • Real World Examples • Performance Considerations #IDUG 3 Introduction to JSON • What is JSON ? • Simple JSON Examples • Where is JSON Used • Why is JSON Used #IDUG 4 What is JSON ? • JSON = JavaScript Object Notation • A lightweight data-interchange format • Part of the JavaScript Programming Language standard • ECMA-262 (3rd edition, December 1999) • Built around two common data structures – • Name-value pairs (associative arrays) • Ordered lists (arrays) • Extensively used in AJAX development • Standards (e.g. JSON Schema) still developing #IDUG 5 Simple JSON Example {“type”: “Airbus A.380-841”, “cn”: “095”, “reghistory”: [ {“reg”: “F-WWSK”, “regdate”: “2013-04-01”,“operator”: “Airbus”}, {“reg”: “G-XLEA”,”regdate”: “2013-07-03”,”operator”:”British Airways”} ], “notes”: [ {“notenum”: 1, “note”: “Shown at Paris Airshow 2013”}, “{notenum”: 2, “note”: “First A380 for BA”} ] } #IDUG 6 Where Is JSON Used ? • AJAX Applications • Mobile Applications (a special case of AJAX applications) • Cloud services APIs • Most new services provide JSON interface • Existing services adding JSON interface • Some services dropping XML in favour of JSON interfaces #IDUG 7 JSON API Usage (from programmableweb.com) #IDUG 8 XML API Usage (from programmableweb.com) #IDUG 9 Relative Interest: XML v JSON (2004 to current) #IDUG 10 Why Is JSON Used (Or Why Not XML) ? • XML is a markup language / document interchange language • JSON is a DATA interchange language • JSON is less verbose (more lightweight) • JSON is more “machine friendly” • Now the preferred format for AJAX applications • The “X” used to stand for XML • Most AJAX libraries now use JSON as the default • Widely used in mobile applications • Both for data storage and interchange with back-end services #IDUG 11 DB2 JSON Releases • DB2 for LUW • JSON Technology Preview • Shipped with DB2 for LUW 10.5 GA • Largely unannounced • First GA Release • DB2 for LUW Fixpack 1 • Ongoing improvements with each Fixpack • DB2 for z/OS • DB2 10 : via no charge “IBM DB2 Accessories Suite for z/OS V3.1” • DB2 11 : still to come #IDUG 12 DB2 JSON Capabilities • JSON storage facilities • Java API • Documentation in javadoc format • Command line (uses Java API) • NoSQL wire listener • Supports common MongoDB operations • Adding additional functionality will be an ongoing process #IDUG 13 Functionality Not (Yet) Provided • Integration with standard SQL engine or command line • Only access is through JSON only interfaces • No “joins” between JSON document store and other data • Direct (native) support for languages other than Java • All non-Java access through wire listener • JSON construction from relational data • Would help integrating existing DB2 data into common frameworks • Example : jQuery (especially DataTables) uses JSON extensively • Tooling #IDUG 14 DB2 JSON Architecture #IDUG 15 Enabling JSON Support • Support is found under ~<instance>/sqllib/json • Requires a UTF-8 database • Default pagesize of 32k is recommended • Command line at bin/db2nosql.sh • Must be in the bin directory to run successfully (internal references) • Will prompt for required parameters if not supplied • Once in command line run enable(true) to enable database • Running enable(false) prints out SQL to view (or modify) #IDUG 16 JSON Command Line Sample db2inst1@philip-Lenovo-U310:~/sqllib/json/bin$ ./db2nosql.sh -db DPJSN001 JSON Command Shell Setup and Launcher. This batch script assumes your JRE is 1.5 and higher. 1.6 will mask your password. Type db2nosql.sh -help to see options IBM DB2 NoSQL API 1.1.0.0 build 1.0.169 Licensed Materials - Property of IBM (c) Copyright IBM Corp. 2013 All Rights Reserved. Debug mode is off. nosql>Type your JSON query and end it with ;<ENTER> nosql>Type help() or help for usage information nosql> nosql>Setup Tables and Functions seems to have not been created or have been created incorrectly. Please type enable(true) and enter to setup them up. You must have the correct admin privileges. If you do not, type enable(false) and enter to see the SQL that will be used. #IDUG 17 What Enablement Provides (as at Fixpack 3) • All objects (tables, UDFs) under schema SYSTOOLS • Table SYSJSON_INDEX • UDFs (language C : backed by library sqllib/function/db2json) • JSON_VAL2 • JSON_BINARY and JSON_BINARY2 (new in Fixpack 3) • JSON_TABLE and JSON_TABLE_BINARY • JSON_LEN • JSON2BSON, BSON2JON and BSON_VALIDATE (new in Fixpack 3) • JSON_GET_POS_ARR_INDEX • JSON_UPDATE (2 signatures) • UDF (language SQL) : SYSTOOLS.REG_MATCHES (2 signatures) • Buffer pool and system temporary tablespace (32k pagesize) #IDUG 18 Creating a JSON Data Store • Explicitly – nosql>db.createCollection(“mycoll01”); • By inserting the first document into a collection - nosql>db.idugjson.insert({"event":"EMEA","month":"October"}); • Creates DB2 objects (tables etc) when collection created … #IDUG 19 Underlying A Collection • Table named after the collection • Case sensitive name (e.g. “DB2INST3”.”idugjson”) • DDL (with help from db2look) – CREATE TABLE "DB2INST3"."idugjson" ( “ID” CHAR(12 OCTETS) FOR BIT DATA NOT NULL , “DATA” BLOB(16777216) INLINE LENGTH 3000 LOGGED NOT COMPACT ) IN USERSPACE1; ALTER TABLE "DB2INST3"."idugjson“ ADD PRIMARY KEY ("ID"); #IDUG 20 Important Change Between GA and Fixpack 1 GA nosql>db.createCollection(“idugtest”) Collection: DB2INST1.”idugtest” created. Fixpack 1 nosql>db.createCollection(“idugtest”) Collection: TEST.”idugtest” created. Use db.idugtest. • Reason : compatibility with MongoDB • Hint : To switch to another schema type “use <schema>” • This also allows you to browse collections in other schemas #IDUG 21 Notes on JSON Collection Table • ID contains object ID • Can be explicitly defined as _id in JSON • If not explicitly defined will be added anyway • Index created on ID (definition stored in SYSTOOLS.SYSJSON_INDEX) • DATA contains JSON document in BSON format • BSON = binary encoded JSON format (as used by MongoDB) • Further indexes can be added as needed for performance #IDUG 22 Simple Command Line Retrieval (like “SELECT * FROM table”) nosql>db.idugjson.find() nosql>Row 1: nosql> { nosql> "_id":{"$oid":"5204df8f1f9850b22f6aca89"}, nosql> "event":"EMEA", nosql> "month":"October" nosql> } nosql>1 row returned in 101 milliseconds. #IDUG 23 Retrieval of selected elements with predicates (“SELECT month FROM table where event=‘EMEA’”) nosql>db.idugjson.find({"event":"EMEA"},{"month":1}) nosql>Row 1: nosql> { nosql> "_id":{"$oid":"5204df8f1f9850b22f6aca89"}, nosql> "month":"October" nosql> } #IDUG 24 Define elements to exclude from selection (no direct equivalent in SQL) nosql>db.idugjson.find({"event":"EMEA"},{"_id":0}) nosql>Row 1: nosql> { nosql> "event":"EMEA", nosql> "month":"October" nosql> } • Elements not to display are listed with a value of zero #IDUG 25 The DB2 JSON Java API • Only native interface to DB2 JSON support • Used by both CLP and Wire Listener • Calls JSON enablement UDFs to do much of the “heavy lifting” • Other languages currently must use Wire Listener • Including most common web dev languages e.g. PHP and Ruby • Connection to DB2 through standard JDBC connection • Means you can combine JSON API and standard SQL in same application • (but no “joins” possible between JSON store and relational tables) • Full API documentation shipped as zip file (in Javadoc format) #IDUG 26 The DB2 JSON Wire Listener • Allows access to DB2 JSON support from variety of languages • Follows mongoDB API (not complete) • Brings DB2 “strengths” to NoSQL applications • Particularly ACID properties and scalability • Relies on the Java API behind the scenes • Needs JAVA_HOME environment variable set • TIP : ensure you are at DB2 10.5 Fixpack 1 or higher !!! • Appears to start on GA (on Linux) but is not responding to clients #IDUG 27 Starting the DB2 JSON Wire Listener export JAVA_HOME= <path-to-java-home-directory> cd ~db2inst1/json/bin ./wplistener.sh -start -host localhost:50000 -mongoPort 27017 -userid db2inst1 -password mypasswd -debug -dbName DPJSN001 -logPath /tmp & • Needs default shell to be bash or ksh • Fails on Ubuntu (default shell is dash) : modify shebang (#! line at top) • Debug information in <logPath>/trace.log • Note parameters are case sensitive (e.g. logPath not logpath) #IDUG 28 Stopping the DB2 JSON Wire Listener cd ~db2inst1/json/bin ./wplistener.sh -shutdown -host localhost:50000 -mongoPort 27017 –noSQLHost localhost -userid db2inst1 -password mypasswd -dbName DPJSN001 -logPath /tmp & • Needs virtually same parameters as start for shutdown to work #IDUG 29 Still Needed in DB2’s JSON Support • JSON generation (construction from relational sources) • Like SQL/XML construction functions • JSON / relational integration (in one statement) • Like SQL/XML or XQuery support • Full JSON (mongoDB) API • More being added with every Fixpack #IDUG 30 Real World Examples of JSON Use • jQuery Datatables • Various cloud computing APIs • Mobile computing #IDUG 31 jQuery DataTables (https://datatables.net) • Example of JSON used for browser / app server communication • jQuery : extensively used JavaScript library (open source) • Features include cross-browser support and CSS3 compatibility • Enables easier delivery of AJAX features within websites • jQuery DataTables : plugin (extension) to jQuery • Facilitates tabular representation of data • Provides