DOCSLIB.ORG

Enabling Real-Time Analytics on IBM Z Systems Platform

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Enabling Real-Time Analytics on IBM Z Systems Platform Front cover Enabling Real-time Analytics on IBM z Systems Platform Lydia Parziale Oliver Benke Willie Favero Ravi Kumar Steven LaFalce Cedrine Madera Sebastian Muszytowski Redbooks International Technical Support Organization Enabling Real-time Analytics on IBM z Systems Platform August 2016 SG24-8272-00 Note: Before using this information and the product it supports, read the information in “Notices” on page vii. First Edition (August 2016) This edition applies to IBM DB2 Analytics Accelerator for z/OS v5.1. © Copyright International Business Machines Corporation 2016. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . vii Trademarks . viii IBM Redbooks promotions . ix Preface . xi Authors. xii Now you can become a published author, too! . xiii Comments welcome. xiii Stay connected to IBM Redbooks . xiv Chapter 1. Executive overview. 1 1.1 Introduction . 2 1.2 Real-time analytics . 4 1.2.1 Business advantages . 5 1.2.2 IT advantages . 5 1.3 In-database analytics . 5 1.3.1 Accelerated in-database transformation . 6 1.3.2 Accelerated in-database predictive modeling . 6 1.4 Enabling applications with machine learning capability . 6 1.5 Value propositions. 7 1.6 Related products . 7 1.7 Use cases . 8 1.7.1 Countering payment fraud and financial crimes . 8 1.7.2 Insurance claims in-process payment analytics . 9 1.7.3 Predictive customer intelligence . 10 Chapter 2. Analytics implementation on z Systems platform. 11 2.1 Adding analytics to a mainframe data sharing environment . 12 2.1.1 SPSS Modeler . 15 2.1.2 DB2 Analytics Accelerator wrapper stored procedure . 16 2.1.3 Using accelerator-only tables . 18 2.2 Installation and customization . 19 2.2.1 DB2 and DB2 Analytics Accelerator setup and installation. 20 2.2.2 Required DB2 privileges for SPSS users . 24 2.2.3 SPSS Modeler client . 25 2.2.4 SPSS Modeler server . 32 2.2.5 Data sources in SPSS . 33 2.2.6 AQT_ANALYTICS_DATABASE variable . 33 2.2.7 User management in the SPSS Modeler server . 33 2.2.8 Installing SPSS Modeler scoring adapter for DB2 z/OS . 34 2.3 Real-time analytics lifecycle . 34 2.3.1 Swim lane diagram of in-database analytics lifecycle. 37 2.3.2 Interaction between a DB2 DBA and a data scientist . 37 2.3.3 Key strengths of various components. 38 Chapter 3. Data integration using IBM DB2 Analytics Accelerator Loader for z/OS . 41 3.1 Functional overview . 42 3.1.1 Loader v2.1 enhancements . 43 © Copyright IBM Corp. 2016. All rights reserved. iii 3.1.2 Loader methods to move data . 44 3.1.3 Components and interfaces . 44 3.2 Getting started. 46 3.2.1 Installation. 46 3.2.2 Customization . 46 3.2.3 Workload Management (WLM) performance goals . 47 3.2.4 IBM z Systems Integrated Information Processor (zIIP) . 53 3.2.5 z Systems advantage . 56 3.2.6 Parallelism . 57 3.3 Scenarios . 57 3.3.1 ACCEL_LOAD_TASKS . 58 3.3.2 Sequential Input IDAA_ONLY and IDAA_DUAL. 58 3.3.3 Load RESUME . 64 3.3.4 IBM DB2 Analytics Accelerator Loader image copy input. 67 3.3.5 VSAM . 68 3.4 System Management Facility (SMF) . 77 Chapter 4. Data transformation . 83 4.1 Introduction . 84 4.1.1 Accelerator-only table (AOT). 85 4.1.2 Enabling in-database processing on SPSS Modeler client. 86 4.2 SQL pushback in SPSS Modeler . 86 4.2.1 How SQL generation works . 86 4.2.2 Where improvements can occur with IDT using Accelerator . 87 4.3 Nodes supporting SQL generation for DB2 Accelerator . 88 4.3.1 Source palette tab. 88 4.3.2 Record Ops palette tab . 89 4.3.3 Field Ops palette tab. 90 4.3.4 Graphs palette tab . 92 4.3.5 Database Modeling (Nuggets) palette tab. 93 4.3.6 Output palette tab . 94 4.3.7 Export palette tab . 94 4.4 In-database analytics Processing effort by components. ..
Load more

Recommended publications
  • SPSS to Orthosim File Conversion Utility Helpfile V.1.4
    SPSS to Orthosim File Conversion Utility Helpfile v.1.4 Paul Barrett Advanced Projects R&D Ltd. Auckland New Zealand email: [email protected] Web: www.pbarrett.net 30th December, 2019 Contents 3 Table of Contents Part I Introduction 5 1 Installation Details ................................................................................................................................... 7 2 Extracting Matrices from SPSS - Cut and Paste ................................................................................................................................... 8 3 Extracting Matrices from SPSS: Orthogonal Factors - E.x..c..e..l. .E..x..p..o..r.t................................................................................................................. 17 4 Extracting Matrices from SPSS: Oblique Factors - Exce.l. .E..x..p..o..r..t...................................................................................................................... 24 5 Creating Orthogonal Factor Orthosim Files ................................................................................................................................... 32 6 Creating Oblique Factor Orthosim Files ................................................................................................................................... 41 3 Paul Barrett Part I 6 SPSS to Orthosim File Conversion Utility Helpfile v.1.4 1 Introduction SPSS-to-Orthosim converts SPSS 11/12/13/14 factor loading and factor correlation matrices into the fixed-format .vf (simple ASCII text) files
    [Show full text]
  • Integrating Excel, SQL, and SPSS Within an Introductory Finance Intrinsic Value Assignment
    Journal of Finance and Accountancy Volume 20, September, 2015 Integrating Excel, SQL, and SPSS within an Introductory Finance Intrinsic Value Assignment Richard Walstra Dominican University Anne Drougas Dominican University Steve Harrington Dominican University ABSTRACT In 2013, the Association to Advance Collegiate Schools of Business (AACSB) issued revised accreditation standards to address a “new era” of business education. The standards recognize that students will be entering a data-driven world and need skills in information technology to analyze and manage data. Employer surveys also emphasize the importance of technological skills and knowledge in a rapidly changing environment. To address these challenges, business faculty in all disciplines must adapt course content to incorporate software tools and statistical applications and integrate active learning tools that provide students with hands-on experience. This paper describes a technology-based, business valuation assignment instructors can employ within an undergraduate managerial finance course. The assignment draws upon historical financial data from the restaurant industry as students predict the intrinsic value of a firm through the application of various software products. Students receive data in an Access database and create SQL queries to determine free cash flows before transferring information into Excel for further analysis. Students continue by uploading key data elements into SPSS and performing statistical analysis on three growth models in order to identify a desired predictor of intrinsic value. The assignment develops students’ abilities to navigate software tools, understand statistical concepts, and apply quantitative decision making. Key Words: Business valuation, growth rates, software tools Copyright statement: Authors retain the copyright to the manuscripts published in AABRI journals.
    [Show full text]
  • IBM DB2 for Z/OS: the Database for Gaining a Competitive Advantage!
    Why You Should Read This Book Tom Ramey, Director, DB2 for z/OS IBM Silicon Valley Laboratory “This book is a ‘must read’ for Enterprise customers and contains a wealth of valuable information! “It is clear that there is a technology paradigm shift underway, and this is opening enormous opportunities for companies in all industries. Adoption of Cloud, Mobile, and Analytics promises to revolutionize the way we do business and will add value to a company’s business processes across all functions from sales, marketing, procurement, manufacturing and finance. “IT will play a significant role enabling this shift. Read this book and find out how to integrate the heart of your infrastructure, DB2 for z/OS, with new technologies in order to maximize your investment and drive new value for your customers.” Located at IBM’s Silicon Valley Laboratory, Tom is the director of IBM’s premiere relational database management system. His responsibilities include Architecture, Development, Service, and Customer Support for DB2. He leads development labs in the United States, Germany, and China. Tom works closely with IBM’s largest clients to ensure that DB2 for z/OS continues as the leading solution for modern applications, encompassing OLTP to mobile to analytics. At the same time he continues an uncompromising focus on meeting the needs of the most demanding operational environments on Earth, through DB2’s industry- leading performance, availability, scaling, and security capabilities. IBM DB2 for z/OS: The Database for Gaining a Competitive Advantage! Shantan Kethireddy Jane Man Surekha Parekh Pallavi Priyadarshini Maryela Weihrauch MC Press Online, LLC Boise, ID 83703 USA IBM DB2 for z/OS: The Database for Gaining a Competitive Advantage! Shantan Kethireddy, Jane Man, Surekha Parekh, Pallavi Priyadarshini, and Maryela Weihrauch First Edition First Printing—October 2015 © Copyright 2015 IBM.
    [Show full text]
  • A FORTRAN 77 Program for a Nonparametric Item Response Model: the Mokken Scale Analysis
    BehaviorResearch Methods, Instruments, & Computers 1988, 20 (5), 471-480 A FORTRAN 77 program for a nonparametric item response model: The Mokken scale analysis JOHANNES KINGMA University of Utah, Salt Lake City, Utah and TERRY TAERUM University ofAlberta, Edmonton, Alberta, Canada A nonparametric item response theory model-the Mokken scale analysis (a stochastic elabo­ ration of the deterministic Guttman scale}-and a computer program that performs this analysis are described. Three procedures of scaling are distinguished: a search procedure, an evaluation of the whole set of items, and an extension of an existing scale. All procedures provide a coeffi­ cient of scalability for all items that meet the criteria of the Mokken model and an item coeffi­ cient of scalability for every item. Four different types of reliability coefficient are computed both for the entire set of items and for the scalable items. A test of robustness of the found scale can be performed to analyze whether the scale is invariant across different subgroups or samples. This robustness test serves as a goodness offit test for the established scale. The program is writ­ ten in FORTRAN 77. Two versions are available, an SPSS-X procedure program (which can be used with the SPSS-X mainframe package) and a stand-alone program suitable for both main­ frame and microcomputers. The Mokken scale model is a stochastic elaboration of which both mainframe and MS-DOS versions are avail­ the well-known deterministic Guttman scale (Mokken, able. These programs, both named Mokscal, perform the 1971; Mokken & Lewis, 1982; Mokken, Lewis, & Mokken scale analysis. Before presenting a review of the Sytsma, 1986).
    [Show full text]
  • Presto: the Definitive Guide
    Presto The Definitive Guide SQL at Any Scale, on Any Storage, in Any Environment Compliments of Matt Fuller, Manfred Moser & Martin Traverso Virtual Book Tour Starburst presents Presto: The Definitive Guide Register Now! Starburst is hosting a virtual book tour series where attendees will: Meet the authors: • Meet the authors from the comfort of your own home Matt Fuller • Meet the Presto creators and participate in an Ask Me Anything (AMA) session with the book Manfred Moser authors + Presto creators • Meet special guest speakers from Martin your favorite podcasts who will Traverso moderate the AMA Register here to save your spot. Praise for Presto: The Definitive Guide This book provides a great introduction to Presto and teaches you everything you need to know to start your successful usage of Presto. —Dain Sundstrom and David Phillips, Creators of the Presto Projects and Founders of the Presto Software Foundation Presto plays a key role in enabling analysis at Pinterest. This book covers the Presto essentials, from use cases through how to run Presto at massive scale. —Ashish Kumar Singh, Tech Lead, Bigdata Query Processing Platform, Pinterest Presto has set the bar in both community-building and technical excellence for lightning- fast analytical processing on stored data in modern cloud architectures. This book is a must-read for companies looking to modernize their analytics stack. —Jay Kreps, Cocreator of Apache Kafka, Cofounder and CEO of Confluent Presto has saved us all—both in academia and industry—countless hours of work, allowing us all to avoid having to write code to manage distributed query processing.
    [Show full text]
  • Nexus User Guide (Pdf)
    The Best Query Tool Works on all Systems When you possess a tool like Nexus, you have access to every system in your enterprise! The Nexus Query Chameleon is the only tool that works on all systems. Its Super Join Builder allows for the ERwin Logical Model to be loaded, and then Nexus shows tables and views visually. It then guides users to show what joins to what. As users choose the tables and columns they want in their report, Nexus builds the SQL for them with each click of the mouse. Nexus was designed for Teradata and Hadoop, but works on all platforms. Nexus even converts table structures between vendors, so querying and managing multi-vendor platforms is transparent. Even if you only work with one system, you will find that the Nexus is the best query tool you have ever used. If you work with multiple systems, you will be even more amazed. Download a free trial at www.CoffingDW.com. The Tera-Tom Video Series Lessons with Tera-Tom Teradata Architecture and SQL Video Series These exciting videos make learning and certification much easier Four ways to view them: 1. Safari (look up Coffing Studios) 2. CoffingDW.com (sign-up on our website) 3. Your company can buy them all for everyone to see (contact [email protected]) 4. YouTube – Search for CoffingDW or Tera-Tom. The Tera-Tom Genius Series The Tera-Tom Genius Series consists of ten books. Each book is designed for a specific audience, and Teradata is explained to the level best suited for that audience.
    [Show full text]
  • 2018 Corporate Responsibility Report
    2018 Corporate Responsibility Report Trust and responsibility. Earned and practiced daily. #GoodTechIBM IBM 2018 Corporate Responsibility Report | 1 Trust and responsibility. Earned and practiced daily. We have seen, for more than a century, that when to the boardroom. They are core to every — We invested hundreds of millions of dollars in we apply science to real-world problems, we can relationship — with our employees, our clients, programs to help train and prepare the global create a tomorrow that is better than today. More our shareholders, and the communities in which workforce for this new era. These initiatives sustainable. More equitable. More secure. we live and work. include 21st century apprenticeship programs, returnships for women reentering In fact, we have never known a time when In this report, you will read about the many the workforce, veterans programs and science and technology had more potential to achievements we made to further this foundation volunteer skills-building sessions for more benefit society than right now. of trust and responsibility throughout 2018. than 3.2 million students worldwide. And we For example: helped scale the P-TECH™ school model — a In the last 10 years alone, the world has achieved six-year program that offers a high school Ginni Rometty at P-TECH in Brooklyn, N.Y., May 2019 stunning advancements, from breaking the — After reaching our aggressive goals to increase diploma and an associate’s degree, along AI winter to the dawn of quantum computing. our use of renewable energy and reduce CO2 with real-world working experience and These and other advanced technologies have emissions 4 years ahead of schedule, we set mentorship — at no cost to students.
    [Show full text]
  • IBM SPSS Orientation: Version 23
    IBM SPSS Orientation: Version 23 Installation & Introduction to Operations [6 and 12-Month Licence version] This document provides an introduction to IBM SPSS, explaining how to install and run the program, as well as a basic overview of its features. Your Real Estate Division course workbook will provide software instructions for running necessary statistical procedures. However, you are expected to have installed the software and have a working knowledge of how to operate the software program and interpret its results. In general, IBM SPSS is very easy to learn and use. The user interface is similar to that of Microsoft Word and Microsoft Excel and its output is easily transferred to these programs. There is comprehensive help available within the program as well as customer support. Note that the Real Estate Division’s courses do not cover all of the capabilities of the IBM SPSS program. For a complete explanation of statistical procedures not covered in the lessons, you should refer to the IBM SPSS Help menu found in the program. (Note: from this point on SPSS means IBM SPSS) Computer Requirements It is essential that students have at least a very basic knowledge of how a computer operates. If you are unfamiliar with the operation of personal computers, you may wish to visit your local bookstore for a “how to” manual or investigate schools, colleges, or libraries in your area which may offer a course designed for computer beginners. SPSS operates with a Windows or Mac operating system. To use SPSS, you will need to have basic computer skills in order to do the following: • download a program from a website; • start an application; • use a mouse; • use menus and submenus; • use click, select, and drop and drag actions; and • save files to a hard disk drive and open them again.
    [Show full text]
  • IBM Db2 on Cloud Solution Brief
    Hybrid Data Management IBM Db2 on Cloud A fully-managed, relational database on IBM Cloud and Amazon Web Services with elastic scaling and autonomous failover 1 IBM® Db2® on Cloud is a fully-managed, SQL cloud database that can be provisioned on IBM Cloud™ and Amazon Web Services, eliminating the time and expense of hardware set up, software installation, and general maintenance. Db2 on Cloud provides seamless compatibility, integration, and licensing with the greater Db2 family, making your data highly portable and extremely flexible. Through the Db2 offering ecosystem, businesses are able to desegregate systems of record and gain true analytical insight regardless of data source or type. Db2 on Cloud and the greater Db2 family support hybrid, multicloud architectures, providing access to intelligent analytics at the data source, insights across the business, and flexibility to support changing workloads and consumptions cases. Whether you’re looking to build cloud-native applications, transition to a fully-managed instance of Db2, or offload certain workloads for disaster recovery, Db2 on Cloud provides the flexibility and agility needed to run fast queries and support enterprise-grade applications. Features and benefits of Db2 on Cloud Security and disaster recovery Cloud databases must provide technology to secure applications and run on a platform that provides functional, infrastructure, operational, network, and physical security. IBM Db2 on Cloud accomplishes this by encrypting data both at rest and in flight, so that data is better protected across its lifecycle. IBM Db2 on Cloud helps restrict data use to only approved parties with user authentication for platform services and resource access control.
    [Show full text]
  • Programming and Data Management for IBM SPSS Statistics 19
    Programming and Data Management for IBM SPSS Statistics 19 A Guide for IBM SPSS Statistics and SAS Users Raynald Levesque and SPSS Inc. Note: Before using this information and the product it supports, read the general information under “Notices” on p. 435. This document contains proprietary information of SPSS Inc, an IBM Company. It is provided under a license agreement and is protected by copyright law. The information contained in this publication does not include any product warranties, and any statements provided in this manual should not be interpreted as such. When you send information to IBM or SPSS, you grant IBM and SPSS a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligationtoyou. © Copyright SPSS Inc. 1989, 2010. Preface Experienced data analysts know that a successful analysis or meaningful report often requires more work in acquiring, merging, and transforming data than in specifying the analysis or report itself. IBM® SPSS® Statistics contains powerful tools for accomplishing and automating these tasks. While much of this capability is available through the graphical user interface, many of the most powerful features are available only through command syntax—and you can make the programming features of its command syntax significantly more powerful by adding the ability to combine it with a full-featured programming language. This book offers many examples of the kinds of things that you can accomplish using command syntax by itself and in combination with other programming language. For SAS Users If you have more experience with SAS for data management, see Chapter 32 for comparisons of the different approaches to handling various types of data management tasks.
    [Show full text]
  • SPSS Statistics 23.0.0.0 SPSS Statistics 23.0.0.0 Supported Operating Systems
    Software Product Compatibility Reports Supported Operating Systems Product SPSS Statistics 23.0.0.0 SPSS Statistics 23.0.0.0 Supported Operating Systems Contents Included in this report Operating systems Glossary Disclaimers Report data as of 2016-12-27 01:20:00 EST 1 SPSS Statistics 23.0.0.0 Supported Operating Systems Included in this report This report can be generated with filters applied to operating system platforms, components, and/or software capabilities. This section reflects how the report was filtered when it was generated. Legend • The information about this item is included in this report. • The information about this item is not included in the report filter. Platforms Components • AIX Desktop • Linux • IBM SPSS Statistics Client Mac OS • Server • Solaris • IBM SPSS Statistics Server • Windows Report data as of 2016-12-27 01:20:00 EST 2 SPSS Statistics 23.0.0.0 Supported Operating Systems Operating Systems The operating sysytem section specifies the operating systems that SPSS Component support Statistics 23.0.0.0 supports, organized by operating system familiy. Full Operating system families Partial AIX Linux Mac OS Solaris Windows None AIX Summary Operating system OperatingHardware Bitness Product Components Notes? system minimum minimum AIX 6.1 Base POWER 64-Exploit 23.0.0.0 Yes System - Big Endian AIX 7.1 Base POWER 64-Exploit 23.0.0.0 Yes System - Big Endian Report data as of 2016-12-27 01:20:00 EST 3 SPSS Statistics 23.0.0.0 Supported Operating Systems AIX 6.1 POWER System - Big Endian Legend: Supported Not supported
    [Show full text]
  • IBM SPSS Statistics Version 25: Mac OS Installation Instructions (Authorized User License) Installation Instructions
    IBM SPSS Statistics Version 25 Mac OS Installation Instructions (Authorized User License) IBM Contents Installation instructions ........ 1 Notes for installation .......... 1 System requirements............ 1 Licensing your product ........... 2 Authorization code ........... 1 Using the license authorization wizard .... 2 Installing ............... 1 Viewing your license .......... 2 Running multiple versions and upgrading from a Applying fix packs ............ 3 previous release ............ 1 Uninstalling .............. 3 Note for IBM SPSS Statistics Developer .... 1 Updating, modifying, and renewing IBM SPSS Installing from a downloaded file ...... 1 Statistics ................ 3 Installing from the DVD/CD ........ 1 iii iv IBM SPSS Statistics Version 25: Mac OS Installation Instructions (Authorized User License) Installation instructions The following instructions are for installing IBM® SPSS® Statistics version 25 using the license type authorized user license. This document is for users who are installing on their desktop computers. System requirements To view system requirements, go to http://publib.boulder.ibm.com/infocenter/prodguid/v1r0/clarity/ index.jsp. Authorization code You will also need your authorization code(s). In some cases, you might have multiple codes. You will need all of them. You should have received separate instructions for obtaining your authorization code. If you cannot find your authorization code, contact Customer Service by visiting http://www.ibm.com/software/analytics/ spss/support/clientcare.html. Installing Running multiple versions and upgrading from a previous release You do not need to uninstall an old version of IBM SPSS Statistics before installing the new version. Multiple versions can be installed and run on the same machine. However, do not install the new version in the same directory in which a previous version is installed.
    [Show full text]