April 5, 2018 Db2 on Cloud: Accelerate

April 5, 2018

Db2 on Cloud: Accelerate Development and Testing on the Cloud Db2

Simon Lightstone Offering Manager, Db2 on Cloud simonlig@ca.ibm.com Db2 Tech Talk series Host and Today’s Speaker

Lynn Chou Simon Lightstone Noah Kuttler Db2 Tech Talk Host Speaker Speaker Community Marketing Manager, IBM Offering Manager, Db2 on Cloud IBM Product Marketing Manager, Core Databases and IBM Analytics Db2 on Cloud Champions Safe Harbor Statement

U.S. Government Users Restricted Rights - Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON CURRENT THINKING REGARDING TRENDS AND DIRECTIONS, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. FUNCTION DESCRIBED HEREIN MY NEVER BE DELIVERED BY I BM. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.

IBM, the IBM logo, ibm.com and DB2 are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

3 Db2 Tech Talk: Accelerate Development and Testing on the Cloud Contents

1. Introduction 4. APIs

2. Db2 Cloud Family 5. Tutorial Samples Very quick overview Flex Architecture 6. Closing and Feedback

3. Noteable Addons: DSX and more DSX with DSM based Db2 data

6 Deployment Options On Premises On Cloud

APPLIANCE SOFTWARE CONTAINER HOSTED MANAGED DBaaS DBaaS

Db2 Hosted Db2 on Cloud

CONTROL SIMPLICITY “Nearly half of large enterprises will have hybrid cloud deployments by the end of 2017” Gartner – Special Report “Private Cloud Matures, Hybrid Cloud Is Next.”

Built on a common SQL engine enabling true hybrid integration, choice, and portability

Managed or Managed SW defined Custom Open source Dedicated hosted public public warehouse deployable Hadoop with analytics Cloud DBaaS Cloud DWaaS on-premises database SW Hortonworks appliance or in cloud Db2 Integrated Db2 on Cloud Warehouse Db2 Db2 Analytics Db2 Hosted on Cloud Warehouse Db2 Big SQL System

• Application Agility • Licensing • Standardized analytics Write once, run anywhere Flexible entitlements for Common programming One ISV product certification business agility and cost- model for all platforms optimization for in-DB analytics • Operational compatibility • Integration • Common Skills Reuse operational and Common Data Virtualization One skill set for all housekeeping procedures capabilities for query federation deployments and data movement Drive higher efficiencies and portfolio rationalization SQL guidance: http://ibm.biz/CommonSQL 8 8 A deeper dive into the components of the Common SQL Engine

Built on a common SQL engine enabling true hybrid integration, choice, and portability

Managed or Managed public SW defined Custom Open source Dedicated Open Source hosted public Cloud DWaaS warehouse deployable Hadoop with analytics and other Cloud DBaaS on-premises database SW Hortonworks appliance Commercial or in cloud databases Db2 Warehouse Integrated Db2 on Cloud Db2 Analytics My SQL, Pos tgreSQL on Cloud Db2 Warehouse Par quet ( D b2 Ev ent Stor e) Db2 Hosted Db2 Big SQL System Data services for SQL/NoSQL and REST APIS with secure data transfer Accept, parse/validate SQL (including Oracle and PostgreSQL), and process/return data

Statement optimization Leverage statistics on schema, data, and distribution to make better query decisions and improve over time

Identify and manage workloads meeting complex business goals Identification of work, management, monitoring and continuous improvement

Security and accessibility of data Encryption of data in transit and at rest, with authorized access to data only to those who are entitled

Virtualization via query federation Common SQL via Connect to 3rd party and Open Source data sources and objects - expanding your reach Query Federation 9 §Management with a Single Pane of Glass Monitor, Administer, Manage, and Tune your Environment § § A single pane of glass § Simple to install and setup § Scales to 100s of databases § Historical analysis: - minutes, hours, days § Smart alerts with recommendations IBM Supports Your Data Access Patterns Multiple access Styles and Database Languages

§ SQL – Structure Query Language • Rich support of the SQL language § JSON – Storage and manipulation of JSON records within a relational database § Rest – OData standards based access to data § pureXML – Storage and manipulation of XML documents • Supports XQuery and SQL/XML functions § SQL Procedural Language (SQL PL) • It can be used to implement procedural logic in SQL statements § PL/SQL – Procedural Language/Structured Query Language • Reduces the complexity of enabling existing PL/SQL solutions so that they will work with the Db2 data server § RDF Graph Store – Native support for graph triples New: Db2 on Cloud Free Plan (“Lite”)

§ Deploys instantly in 1 click. § No tools to download. SQL editor is built in. § No time limits.

§ Great for prototyping.

§ Get started using the IBM Cloud. https://ibm.biz/freedb2 A Simple “Download and Go” experience! Up, Running, and Productive in Minutes

§ Marketplace or Discovery Portal – single point of download § Download platform-specific executable (Mac, Windows, or Linux) § Small download, less than 15 minutes from “I want” to “I got” § 3 clicks and userID/pwd input only Db2 Family Cloud Flex Plans Precise Performance Free Tiers Offerings Plans

§ Db2 on Cloud Flex • Scale your app without downtime • Scaling typical takes < 20 minutes • Fully managed, 99.99% uptime SLA § Db2 Warehouse on Cloud Flex • Scale analytics workloads easily • MPP cluster with BLU in-memory acceleration • Fully managed, 45 min scale time. Up to 48 TB § Db2 Hosted Flex • Unmanaged Db2: IBM hosts it; you manage it • Full root access

§ Lower development costs § Scale down over weekends § Scale up before running tests § Deploy & scale via API Db2 on Cloud – Plans

Fully managed, secure, dedicated single tenant environments with built-in Oracle compatibility

Precise Performance Plans Flex Plan (fixed price, balanced configurations) (elastic scaling of compute & storage)

PP 500 PP 1400 PP 10000 Flex

Server Type: Virtual Bare Metal Bare Metal Server Type: Virtual

# Cores: 2 (virtual) 12 48 # Cores: 1 - 32 (virtual)

Memory: 8 GB 128 GB 1 TB Memory: 4 - 128 GB

Storage: 500 GB HDD 1.4 TB SSD 11 TB SSD Storage: 2 GB - 4 TB

Monthly Monthly $189 for: Scale beyond that for: Price (USD) Price (USD) • 1 core • $52 per core & • 4 GB RAM 4 GB RAM Non-HA: $250 $4,000 $18,000 Non-HA: • 2 GB storage • $1 per GB storage HA: $500 $8,000 $36,000 • 2 million IOs • $0.20 per 1 million IOs HA: 2x non-HA price (except IOs – IOs only charged for on primary server)

High Availability (HA) configurations include: • Primary/standby pair with synchronous replication • Automatic failover • Single IP address for simplified client access to cluster • Reduced maintenance time using rolling updates Flex Plan – Independently Scale Compute & Storage

§ Compute resource (cores/memory) can be scaled both up and down -Outage typically less than 15 minutes (reduced outage with HA plan)

§ Compute scaling levels – # virtual cores (RAM):

1 (4 GB) 2 (8 GB) 4 (16 GB) 8 (32 GB) 16 (64 GB) 32 (128 GB)

§ Storage can scale up only – can’t be scaled down § Storage scaling levels: -Starts with 100 GB local SSD storage, followed by addition of up to two 2 TB portable storage volumes

2 GB 10 GB 25 GB 50 GB 100 GB 250 GB 500 GB 1 TB 2 TB 4 TB

PSV PSV added added -Growth within a volume does not require an outage; adding PSV does (reduced outage with HA Plan) Flex Plan – Architecture

Cores and Memory 1 Core 32 Core (IBM Cloud VM) 4 GB 128 GB

Data Storage Local SSD 2TB 2TB (Note 100 GB for OS and 1 TB Portable Portable PSV in base for ( 100 GB – data OS/Metadata(LUKS Encrypted) 200 GB Active Logs) Storage Storage not shown) (added after 100GB) (added after 2000 GB) 2 - 100 GB 250 GB - 2 TB 4 TB Storage controlled at more granular levels by using Journaled disk quotas (2 GB – 4000 GB)

Recovery Object Storage Backups and for HA archive logs and load copy data are (Swift uploaded to object storage currently) Scaling an Instance of the Flex Plan Scaling an Instance of the Flex Plan (cont.) Db2 on Cloud – High Availability Configuration

§ Reduced outage times for unplanned outages and maintenance – 99.99% uptime SLA § Configuration includes both a primary server plus a warm standby server § Log shipping keeps standby up-to-date for fast failovers § Simplified client configuration – floating IP address always points to current primary § Not logged operations that compromise data recoverability are blocked § Reduced maintenance times due to rolling updates

Automatic Client Reroute Integrated Cluster Management • Clients transparently connect • Health of servers monitored to standby when primary fails • Automated takeover

Primary Standby Db2 on Cloud Server Server

High Transaction Log Record Shipping Availability • Keeps standby Configuration current Loading Data into Db2 on Cloud is Easy!

§ Cloud-base data services include - Lift - Data Connect, Data Refinery - Data Refinery (part of WDP)

§ Console of Db2 managed services include - Load from remote client files - Load from cloud object storage (IBM Cloud Swift and Amazon S3)

§ ETL tools such as DataStage

§ Remote Db2 load/import/ingest utilities

§ Data Conversion Workbench for data migration

§ Large data transfer services such as - Data Transfer Service - Mass Data Movement Service

See load section of Knowledge Center for further details on load options for Db2 cloud services: http://www-01.ibm.com/support/knowledgecenter/SS6NHC/com.ibm.swg.im.dashdb.doc/learn_how/loaddata_overview.html Db2 on Cloud Service Offerings – Accelerating Success

Jumpstart Accelerator

• 50 hours to be used • 50 hours to be used within within 90 days of the first year purchase • Use against scoped • Services for hand holding, projects; clients can choose not doing between • Pre-Scoped Project • Guidance on proven Custom Scope Project practices for Db2 on Cloud, “how to” focus • Projects where IBM does the work • Advice, guidance, assistance Db2 on Cloud – Service Level Agreements (SLA)

§ SLA is an agreement to compensate a customer if High Availability Plans: documented levels of uptime are not met Compensation (% of Availability during a - Credited against a future invoice for the service monthly fee) towards contracted month future invoice § Downtime is measured from the time the >= 99.99% 0% customer reports the event until the time the >= 99% and < 99.99% 10% service is restored < 99% 25%

§ As described in the Service Description document, Non-High Availability Plans: downtime does not include time related to: Compensation (% of Availability during a - A scheduled or announced maintenance outage monthly fee) towards contracted month (i.e. planned outages do not count) future invoice - Causes beyond IBM's control >= 99.5% 0% - Problems with Client or third party content or >= 99% and < 99.5% 10% technology, < 99% 25% designs or instructions

§ Details can be found in the Service Description (SD) documents (always use the most recent version of the SD available): § Pay-as-you-go: http://www-03.ibm.com/software/sla/sla db.nsf /pdf /7519- 07/$file/i126-7519- 07_ 09- 2017_en_US.pdf § IBM/BP sales: http://www-03.ibm.com/software /sla/sladb .n sf/p df/7579-04/$file/i126-7579-04_06- 2017_en_US.pdf IBM Data Server Manager Natively Supports Db2 in the Cloud

IBM Data Server Manager is a critical integrated part of the hybrid IBM Data Store value proposition. It's the up and running, management, and the problem determination console for Db2, Db2 on Cloud, Db2 Warehouse on Cloud, Db2 Warehouse and Big SQL.

IBM Data Server Manager is more than a tool for monitoring Db2. It is a set of common services and capabilities for Enterprise and Application DBAs, Data Engineers and Analysts and Developers.

It's the center of the Hybrid Enterprise.

By establishing a connection to a Db2 Warehouse on Cloud or a Db2 on Cloud database, Data Server Manager automatically reconfigures its feature set to complement the selected database type

§ The Data Science Experience (DSX) enables data scientists to master the art of data science in a number of environments - On-prem, Private cloud, Public Cloud - Simplified single user application § DSX provides data scientists access to - Commonly used tools for data scientists such as - Jupyter notebooks - RStudio § Numerous open source and commercial analytics libraries § Constantly updated resources § An environment that facilitates team work and collaboration RStudio and R Integration Making Analytics Easy

§ RStudio makes R easier to use - It includes a code editor, debugging & visualization tools - Syntax highlighting, code completion, and smart indentation - Quickly jump to function definitions § Execute R code directly from the source editor § Easily manage multiple working directories using projects § Integrated R help and documentation § Interactive debugger to diagnose and fix errors quickly § Extensive package development tools Data Server Manager: Core Capabilities § Monitoring and Alerts § Proactive alerts § Overview Dashboards and drilldown § Real-time, and historical performance data § Client Monitoring § HADR and pureScale § Query Tuning § Advisors to identify tuning opportunities § Capture SQL and workloads § Explain query access paths § Improve statistics quality, improve database and query design § Database Administration and Use § Build, format, explain and run SQL statements and download as excel § Perform or schedule routine admin § Manage privileges, maintenance and availability § Connect to remote data sources § Create, and mange database objects § Configuration Management and History § Keep an inventory of database objects history of change. § Registry variables, configuration settings, custom key/ value settings to help with diagnosis APIs and Rest Remote Mobile and Open access to IBM Data Stores

§ What is the problem? Mobile (Android, Windows, iOS) and other open application models need access to IBM DB environments § What is our solution? Robust APIs provide a broad set of capabilities. In addition, Data Server Gateway for OData supports open standards based connectivity to IBM Data Server environments REST API Exceptional power to automate your apps. Power to fully automate Exceptional power to automate your apps.

§ REST API for SQO § API for Deployment § API for Destroy • Note: Applies to free and paid § API for complex authentication and adding/deleting users. • Great for multi-tenant applications. § API for scaling • cf update-service flextest -c '{"ram_gb":8,"email":”[email protected]”,”scale_time":0}' Data Server Gateway for OData Remote Mobile and Open access to IBM Data Stores

§ What is the problem? Mobile (Android, Windows, iOS) and other open application models need DBA access to IBM DB environments Application including Mobile § What is our solution? or other Open Model Data Server Gateway for OData supports open standards based connectivity to IBM Data Server environments § Why OData? OData (Open Data Protocol) is an ISO/IEC approved, Oasis standard that defines a set of best practices for building and consuming Restful API’s OData § What’s included? Gateway* • Publish OData V4 Rest API endpoints/services for selected database tables • Support for most data types • Support for Crud operations • Get, Post, Put (update a full row), Patch (update selected columns), Delete * DS Driver supports OData Gateway access to IBM • JSON and XML payloads Data Stores • API persistence • Authentication and Authorization • SSL Support IBM Data Studio An integrated, modular environment for DB development and administration § Supports IBM Db2, Db2 Warehouse Db2 for z/OS, Db2 for i, Informix and Big SQL ibm.biz/IBMDataStudio § Enables developers and administrators to create Application and manage heterogeneous database environments Developer for increased productivity. § Streamline database development with advanced query validation, object management, procedure development, deploy and debug. § Improved collaboration through the open source integrated Eclipse environment § Support for popular source code management and team collaboration tools like Git § pureQuery integration for application acceleration § Data Studio available at no charge § Support entitlement provided through Db2 Deep dive tutorials

Think 2018 / DOC ID 8290 / March 19, 2018 / © 2018 IBM Corporation 35 Making Magic with Jupyter • Db2 Magic Commands • Secure and persistent database connections • One command from SQL to DataFrame • SQL Formatting in Cell • Graphing

github.com/DB2-Samples/db2jupyter Jupyter Notebook Tutorial

github.com/DB2-Samples/db2jupyter Db2 and JSON

Think 2018 / DOC ID 8290 / March 19, 2018 / © 2018 IBM Corporation 38 Db2 JSON Support § Initial JSON support in Db2 10.5 FP1 consisted of: • IBM NoSQL Wire Listener, JSON Java APIs, JSON command line § Db2 11.1.2.2 added full support for direct SQL access to underlying JSON interfaces used by externals above • Integrated installation/configuration during database creation or migration update so no need to catalog JSON functions manually • High-level documentation on their existence similar to what Db2 for z/OS already provides • JSON2BSON, BSON2JSON are catalogued as part of SYSIBM schema • JSON_VAL is function built into the SQL syntax • All other JSON functions are part of the SYSTOOLS schema • Overview of all functions and how to use them can be found in Db2 11 ebook at: ibm.box.com/v/DB2v11eBook • Manually catalog tables with prior releases using the DB2-V11-JSON-Examples.sql script in the eBook directory JSON SQL Combine your SQL with JSON records

Name Comments BSON2JSON Convert BSON formatted document into JSON strings BSON_VALIDATE Checks to make sure that a BSON field in a BLOB object is in a correct format JSON2BSON Convert JSON strings into a BSON document format JSON_GET_POS_ARR_INDEX Find a value within an array JSON_LEN Returns the count of elements in an array type inside a document JSON_TABLE Returns a table of values for an array field JSON_TYPE Returns the data type of a specific field within a JSON document JSON_UPDATE Update a field or document using set syntax JSON_VAL Extracts data from a JSON document into SQL data types Open Data Access with OData

Think 2018 / DOC ID 8290 / March 19, 2018 / © 2018 IBM Corporation 41 Data Without Borders § Customers want to access Data from non traditional platforms like Mobile platforms, from mobile apps (Android, Windows, iOS, etc.) • And in some cases want to publish access to data as well to 3rd parties § Mobile and Cloud developers expect direct HTTP and JSON access to data without requiring Database Drivers § OData is standards based way for our customers to leverage Db2 modern Cloud and Mobile Application Models

OData (Open Data Protocol) is an ISO/IEC approved, OASIS standard that defines a set of best practices for building and consuming RESTful APIs.

OData helps you focus on your business logic while building RESTful APIs without having to worry about the various approaches to define request and response headers, status codes, HTTP methods, URL conventions, media types, payload formats, query options, etc.

OData also provides guidance for tracking changes, defining functions/actions for reusable procedures, and sending asynchronous/batch requests. http://www.odata.org/ Open Data Protocol (OData) Gateway Removing the Need for Database Drivers and APIs

Java EE Application Server

Db2 Client Request HTTP Request

IBM Data Server Mobile Client Gateway for OData

Option to Co-locate with Db2 Server RESTful Support Simplifying Access to Data Sources from Mobile Devices § The IBM OData Gateway provides a RESTful service that provides for the following operations: - GET (SELECT) - POST (INSERT) - PUT (UPDATE a full row) - PATCH (UPDATE selected columns in a row) - DELETE (Delete a row) url = http://localhost:9080/ODataOne/ODataService/SAMPLE-db2ab6eeb40d4a75a525333ee610fb50

header = { 'Content-Type':'application/json', 'Accept':'application/json' }

OData = "/EMPLOYEES?$select=LASTNAME,SALARY&$filter=SALARY gt 50000"

RESTful.get(url + OData, headers=header) OData Sample

SELECT LASTNAME, SALARY FROM EMPLOYEE Service Address WHERE SALARY > 50000 url = http://localhost:9080/ODataOne/ODataService/SAMPLE- db2ab6eeb40d4a75a525333ee610fb50

header = { Handshake 'Content-Type':'application/json', 'Accept':'application/json' }

Odata = "/EMPLOYEES?$select=LASTNAME,SALARY&$filter=SALARY gt 50000" OData Request RESTful.get(url + OData, headers=header) Sample OData Application

Jupyter Notebook OData Gateway

iOS App Db2

Db2 V11.1.2.2 Summary & Close

We introduced the New tutorials and “Developer-C” edition on developerWorks improvements, June 22nd 2017, providing free to providing education to a new non-production use of generation of builders. Advanced Enterprise on http://ibm.biz/CalebTalksDb2 systems with 4-cores or less.

Launched Db2 on Cloud free We’re creating a small tier, allowing anyone to build handheld server running Db2 using Db2 on Cloud without to inspire a new generation of cost in a few clicks. makers, coders, and DIYers with IoT applications. Stay tuned! The Db2 Developer Community Db2 samples, expertise and community access for deep collaboration https://developer.ibm.com/data/db Launched in June 2017 2/ Enhancements ongoing § Add Db2 Tech Day recordings § Integrate and aggregate forums and dW Answers § Dynamic content posted from blogs forums social media. § Links to training, download and sample code included

Db2 community links from dW communities, product pages Resources § IBM DSX Desktop datascience.ibm.com/docs/content/desktop/welcome.html

§ Odata Gateway www.ibm.com/developerworks/community/blogs/96960515-2ea1-4391-8170- b0515d08e4da/entry/IBM_Data_Server_Gateway_for_OData_Version_1_0_0?lang=en • The code can be obtained through the following link: ibm.biz/Db2OData • Videos www.youtube.com/watch?v=z06bl_K6Ckc www.youtube.com/watch?v=dHVBiTphoCM

§ IBM Data Server Manager ibm.com/data-server-manager Sample Db2 OData Notebooks § Jupyter notebooks are available to try OData functions against Db2 11 github.com/DB2-Samples/db2odata.git § Includes a Jupyter notebook extension (%odata) to generate a valid service URL for requests to a Db2 Database § Translates SQL syntax for INSERT, UPDATE, DELETE, and SELECT commands into equivalent OData commands § JSON • Documentation ibm.box.com/v/DB2v11eBook • Jupyter Notebook github.com/DB2-Samples/db2jupyter.git § Db2 11 eBook contains syntax and examples of using JSON natively in Db2 ibm.box.com/v/DB2v11eBook Db2 for Linux, Unix, and Windows eBook

Db2 for Linux, Unix, and Windows Version 11 Highlights

Actionable Insights Continuous Availability Massive Scalability Outthink the possible ibm.box.com/v/DB2v11eBook

George Baklarz and Enzo Cialini Experience Data With IDUG The Worldwide Db2 User Community We Are… A Community Focused on and Your Go-To Resource for Online Driven by Users Education About Data • Network & learn from nearly 7,000 members • 100s of blog posts covering the latest topics • Top product developers, industry experts, such as Db2 on all platforms, analytics and renowned consultants, fellow users & vendors SQL • Find your IDUG community close to home – we have • On-demand webinars available 24/7 on the regional user groups across 40 countries Db2 Tech Channel • ¿hablas español? Tenemos contenido para ti All by and for DBAs, IT architects, developers and data scientists Upcoming IDUG Db2 Tech Conferences

India Philadelphia, Pennsylvania, USA Bengalore, March 27, 2018 April 29 – May 03, 2018 Chennai, March 29, 2018 @IDUGDB2 Malta Sydney, Australia www.IDUG.org 04-08 November, 2018 Fall 2018 Our next Tech Talk and other resources

§ Noon EST on June 7, 2018 -Title: What's New in Db2 V11.1.3.3 for SAP Solutions -Register at ibm.biz/Db2TT_Jun2018 § For our live audience, you can submit questions using the Q&A widget.

§ For audience listening this in replay, you can email your questions to Lynn Chou at [email protected]

§ Schedule a demo or learn more about our offerings in general: • 1 800 966-9875 (U.S) • 1-888-746-7426 (Canada) • 1800-425-3333 (India) • Or visit http://www.ibm.com/planetwide/ for contact information worldwide Q & A