Oracle Machine Learning

Oracle Machine Learning Platform Overview Move the algorithms, not the data! Mark Hornick, Senior Director, Product Management Data Science and Machine Learning Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

• Oracle Machine Learning family of products • Supporting multiple personas • OML component details • Enabling Oracle Applications use of ML

OML4SQL OML Notebooks Oracle Advanced Analytics with Apache Zeppelin on SQL API Autonomous Database OML4R Oracle Data Miner Oracle R Enterprise Oracle SQL Developer extension R API OML4Py* OML4Spark Python API Oracle R Advanced Analytics for Hadoop OML Microservices** Supporting Oracle Applications Image, Text, Scoring, Deployment, Model Management

* Coming soon ** For use through Oracle Application only

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 4 New Branding Corresponding Products and Components Oracle Advanced Analytics (OAA) and Oracle Machine Learning (OML) Oracle R Advanced Analytics for Hadoop (ORAAH) Oracle Machine Learning for SQL (OML4SQL) Oracle Advanced Analytics (OAA) / Oracle Data Mining (ODM) Oracle Machine Learning for R (OML4R) Oracle R Enterprise (ORE) Oracle Machine Learning for Python (OML4Py) Oracle Machine Learning for Python (OML4Py) Oracle Machine Learning for Spark (OML4Spark) Oracle R Advanced Analytics for Hadoop (ORAAH) Oracle Machine Learning Notebooks previously known as Oracle Machine Learning Oracle Data Miner (ODMr)

Note: The official price list product names have not changed.

Data Scientists

Business and Executives Oracle Data Analysts Machine Learning

Application and DBA and IT Dashboard Professionals Developers

Data Scientists

•Popular data science languages: Business and Executives R, Python, SQL Oracle Data Analysts •Augment with 3rd party packages •Scalability and performance Machine •Automation-enhanced productivity Learning •Greater enterprise collaboration •Integrate and analyze data across the enterprise Application and DBA and IT Dashboard Professionals Developers

•Expand analytical tool set with ML Data Scientists •Enable non-ML experts with AutoML •Leverage domain knowledge for better results •Collaborate with Data Scientists and IT Business and Executives Oracle Data Analysts Machine Learning

Application and DBA and IT Dashboard Professionals Developers

Data Scientists

Business and Executives Oracle Data Analysts Machine Learning • Even greater value from Oracle investment • Support scalability and performance • Simpler, streamlined infrastructure Application and DBA and IT • Maintain data security, backup, recovery Dashboard Professionals Developers • Use SQL, expand to Python and R • Leverage Database and Big Data sources

Data Scientists

Business and Executives Oracle Data Analysts Machine • Realize intelligent solutions faster through Oracle stack integration Learning • Easily uptake data scientists’ R, Python, and SQL scripts and rapidly deploy solutions Application and DBA and IT Dashboard • Embed ML in applications and dashboards Professionals using SQL, REST, and SODA APIs Developers

• Benefit from world-class data Data Scientists management technology and support • Democratize ML across the enterprise to enable better data-driven decisions Business and • Deploy solutions faster to realize ROI Executives Oracle Data Analysts Machine Learning

Application and DBA and IT Dashboard Professionals Developers

Data Ingest

Data Big Analytics Data

Machine Application Learning Development From essentially a database cloud service…

…to a tightly integrated data platform

Select API Cloud or Reach broader Oracle Object User Interface, e.g. Options On-premises Data Sources Storage Amazon OML Notebooks REST S3

Popular R Azure Blob OML4R Oracle Cloud SQL Storage IDEs Popular Python NoSQL OML4Python Oracle Big Data SQL Databases IDEs Kafka SQL Developer OML4SQL Streams Big Data OML4Spark Service (HDFS)

CLASSIFICATION REGRESSION FEATURE EXTRACTION – Naïve Bayes – Linear Model – Principal Comp Analysis (PCA) – Logistic Regression (GLM) – Generalized Linear Model – Non-negative Matrix Factorization – Decision Tree – Support Vector Machine (SVM) – Singular Value Decomposition (SVD) – Random Forest – Stepwise Linear regression – Explicit Semantic Analysis (ESA) – Neural Network – Neural Network – Support Vector Machine – LASSO TEXT MINING SUPPORT – Explicit Semantic Analysis – Algorithms support text columns ATTRIBUTE IMPORTANCE – Tokenization and theme extraction CLUSTERING – Minimum Description Length – Explicit Semantic Analysis (ESA) for – Hierarchical K-Means – Principal Component Analysis (PCA) document similarity – Hierarchical O-Cluster – Unsupervised Pair-wise KL Div – Expectation Maximization (EM) – CUR decomposition for row & AI STATISTICAL FUNCTIONS – Basic statistics: min, max, ANOMALY DETECTION ASSOCIATION RULES median, stdev, t-test, F-test, Pearson’s, – One-Class SVM – A priori/ market basket Chi-Sq, ANOVA, etc. TIME SERIES PREDICTIVE QUERIES R AND PYTHON PACKAGES – Forecasting - Exponential Smoothing – Predict, cluster, detect, features – Third-party R and Python Packages – Includes popular models through Embedded Execution e.g. Holt-Winters with trends, SQL ANALYTICS – Spark MLlib algorithm integration seasonality, irregularity, missing data – SQL Windows – SQL Patterns – SQL Aggregates

Feature In-Database Spark No data movement to separate analytical engines

Wide range of ML techniques supported Native Native and Spark MLlib

High performance from parallel, distributed execution Spark 2-based. Use all nodes from Hadoop cluster Greater scalability from improved memory utilization

Handle narrow, wide, and sparse data Plus star schema, nested data

Automation Data preparation, text mining, partitioned models, AutoML

Multiple language interfaces SQL, R, Python R, Java High performance R Formula for R Language

• Collaborative UI – Based on Apache Zeppelin – Supports data scientists, data analysts, application developers, DBAs – Easy sharing of notebooks and templates with permissions, versioning, and execution scheduling • Included with Autonomous Database – Automatically provisioned, managed, backed up – In-database machine learning algorithms and analytics functions via OML4SQL, and soon to be augmented with Python and R

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Oracle Machine Learning for SQL (OML4SQL) for Python (OML4Py) for R (OML4R) Empower SQL users with immediate access to ML in Oracle Database and Oracle Autonomous Database Empower data scientists with open source environments

Read/Write files using built-in tool capabilities Data Source ? read Flat Files extract / export export load

Data source connectivity packages

• Access latency Deployment • Paradigm shift: e.g., R/Python  Data Access Language  R/Python Ad hoc • Memory limitation – data size, in-memory processing cron job • Single threaded • Issues for backup, recovery, security • Ad hoc production deployment

• Use in-database parallel and distributed OML Notebooks SQL Interfaces machine learning algorithms from SQL SQL*Plus SQLDeveloper and PL/SQL … • ML models as first class database objects • Export / import models across databases • Batch and real-time scoring with explanatory predictive details • Leverage machine learning across SQL-enabled Oracle stack Oracle Database Autonomous with OAA option Database

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | OML4SQL: Model Build and Real-time Prediction Simple SQL Syntax—Classification Model Model build (PL/SQL) BEGIN DBMS_DATA_MINING.CREATE_MODEL( model_name => 'BUY_INSUR1', mining_function => dbms_data_mining.classification, data_table_name => 'CUST_INSUR_LTV', case_id_column_name => 'CUST_ID', target_column_name => 'BUY_INSURANCE', settings_table_name => 'CUST_INSUR_LTV_SET'); END;

Real-time scoring (SQL query) SELECT prediction_probability(BUY_INSUR1, 'Yes' USING 3500 as bank_funds, 825 as checking_amount, 400 as credit_balance, 22 as age, 'Married' as marital_status, 93 as MONEY_MONTLY_OVERDRAWN, 1 as house_ownership) FROM dual;

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Oracle Machine Learning for R and Python Components of Oracle Advanced Analytics option to Oracle Database • Use Oracle Database as HPC environment Client SQL Interfaces • Use in-database parallel and distributed SQL*Plus OML4Py OML4R SQLDeveloper machine learning algorithms • Manage R and Python scripts and objects in Oracle Database • Integrate open source results into Database Server applications and dashboards via SQL Machine • In OML4Py, automated machine learning – AutoML

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Oracle Machine Learning for R and Python Oracle Advanced Analytics option to Oracle Database • Transparency layer – Leverage proxy objects so data remain in database – Overload native functions translating functionality to SQL Client SQL Interfaces – Use familiar R / Python syntax to manipulate database data SQL*Plus OML4Py OML4R SQLDeveloper • Parallel, distributed algorithms – Scalability and performance – Exposes in-database algorithms available from OML4SQL • Embedded execution Database – Manage and invoke R or Python scripts in Oracle Database Server – Data-parallel, task-parallel, and non-parallel execution Machine – Use open source packages to augment functionality • In OML4Py, Automated Machine Learning - AutoML – Feature selection, model selection, hyper-parameter tuning

data.frame

Inherits from

Proxy data.frame

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Transparency Layer In-database performance from indexes, query optimization, parallelism, partitioning • Leverages proxy objects for database data: oml.DataFrame DATA.shape DATA.head() # Create table from Pandas DataFrame data DATA.describe() DATA = oml.create(data, table = 'BOSTON') DATA.std() # Get proxy object to DB table boston DATA.skew() DATA = oml.sync(table = 'BOSTON')

TRAIN, TEST = • Uses familiar Python syntax to manipulate database data DATA.split() • Overloads Python functions translating functionality to SQL TRAIN.shape TEST.shape

from oml import svm

# create proxy object Python Client ONTIME_S = oml.sync(table='ONTIME_S') OML4Py # define model object settings = {'svms_outlier_rate' : 0.01} svm_mod = svm('anomaly_detection', svms_kernel_function = Oracle Database 'dbms_data_mining.svms_linear', **settings) User tables # build anomaly detection model svm_mod = svm_mod.fit(x=ONTIME_S, y=None) # view model object svm_mod

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Embedded Python Execution Example of parallel execution for partitioned data flow using third party package # user-defined function using sklearn def build_lm(dat): from sklearn import linear_model Python Client lm = linear_model.LinearRegression() OML4Py X = dat[['PETAL_WIDTH']] y = dat[['PETAL_LENGTH']] Oracle Database lm.fit(X, y) pyq*eval () return lm interface User tables # select column(s) for partitioning data index = oml.DataFrame(IRIS['SPECIES']) extproc extproc # invoke function in parallel on IRIS table Python Engine mods = oml.group_apply(IRIS, index, func=build_lm, OML4Py parallel=2) Python Engine mods.pull().items() OML4Py

AutoMS AutoFS AutoTune Much faster than >50% reduction in Significant score Data exhaustive search features improvement Model

• Auto Model Selection • Auto Feature Selection • Auto Tune Hyperparameters – Identify algorithm that – Reduce # of features by – Significantly improve achieves highest model quality identifying most predictive model accuracy – Find best model faster than – Improve performance and – Avoid manual or exhaustive with exhaustive search accuracy search techniques

ML training time Prediction Accuracy Prediction Accuracy 30 1 1 0.95 25 0.99 0.9 0.98 20 0.85 +4% 0.8 33x 0.97 +18% 15 reduction 0.75

0.96 0.7 Accuracy 10 Accuracy 0.95 0.65 0.6 Training (seconds) time Training 5 0.94 0.55 0 0.93 0.5 299 9 299 9 784 309 97% reduction 60% reduction 1.3X time reduction to build SVM Gaussian model OpenML dataset 312 with 1925 rows, 299 columns OpenML dataset 40996 56K rows, 784 columns

Feature OML4SQL OML4Py OML4R Transparency Layer n/a Parallel, Distributed Algorithms Embedded Execution n/a Automated Data Preparation Automated Text Processing Partitioned Models Automated Machine Learning (AutoML) PGX Integration for Graph Analytics implicit DML table package transparency n/a Extensible Algorithm Models n/a – not applicable

• Java API Leverage Spark 2 environment for powerful R Client data preparation and machine learning OML4Spark • Use data across range of Data Lake sources • Achieve scalability and performance using HDFS / Hive / Spark DF / full Hadoop cluster Impala / JDBC Sources • Parallel and distributed machine learning algorithms from native and Spark MLlib BDA implementations BDS DIY • Use expressive R Formula specification

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Oracle Machine Learning for Spark R Language API Component to Oracle Big Data Connectors • Transparency layer – Proxy objects reference data from file system, HDFS, Hive, Impala, Spark DataFrame and JDBC sources Java API – Overloaded R functions translate functionality to native language, R Client e.g., HiveQL for HIVE and Impala – Users manipulate data via standard R syntax OML4Spark • Parallel, distributed machine learning algorithms

– Scalability and performance leveraging full Hadoop cluster HDFS / Hive / Spark DF / – Spark-based custom LM, GLM, NN, K-Means plus Spark Impala / JDBC Sources MLlib algorithms – Use expressive R Formula specification

• Compute framework with custom R mappers/reducers BDA – Data-parallel and task-parallel execution BDS – Allows for open source CRAN packages run on the Cluster Nodes DIY

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | OML4Spark Performance • Logistic Regression (GLM) OML4Spark vs. Spark MLlib for GLM Logistic Regression • Data fits in memory 10,000 – Up to 7x faster than Spark MLlib 1,000 • Data cannot fit memory – Able to solve a 10B row model 100 • Benchmark environment 10

– ORAAH 2.8.0 (seconds) Time Execution – Big Data Appliance X7-2 1 100K 1M 10M 100M 1B 10B – 6 Nodes, 256GB of RAM per Node Dataset Size (# rows)

OML4Spark MLlib

Formula: cancelled ~ distance + origin + dest + as.factor(month) + as.factor(year) + as.factor(dayofmonth) + as.factor(dayofweek) + as.factor(flightnum)

HCM Cloud Industry Data Models Workforce Predictions Communications, SNA, Utilities, Airlines, Retail, … CRM Sales Cloud Sales Prediction Retail GBU EBS Spend Classification Customer Insights, Organize spend into logical categories Customer Segmentation EBS Depot Repair Adaptive Intelligent Applications for Manufacturing Optimize speed, cost, quality of product repair, reuse, recycling Configure, Price, Quote Cloud Oracle Identity Management Content and Experience Adaptive Access Management Unstructured Data Analytics FSGBU Oracle Integration Cloud Analytical Applications Digital Process Automation Infrastructure

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Application Integration Spotlight – Platforms using OML Platform Oracle Integration Cloud (OIC) Oracle Content and Experience (OCE) • Digital Process Automation • Improve Content Discoverability – Help business users make better decisions by using – Search, organize content, reduce duplication recommendations from ML models – Find relevant images/docs during content creation – Increase automation of human-centric approval – Automatic tagging and classification of workflows images, videos, text • Used by Oracle SaaS process-centric apps – Visual search • PaaS service that exposes OML features • Cloud-based content hub to drive – Build models in ADB, deploy via OML Microservices omni-channel content management and accelerate experience delivery • Leverages OML cognitive microservices

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Oracle Machine Learning Microservices Available now internally to Oracle Applications teams • Model Management Services – Building and deploying OML models • Cognitive Services – Feature Extraction, Image and Text • Model repository – Store, version, compare ML models • REST APIs for application integration • Docker Containers for portability • Kubernetes support for container management

Model Model Repository REST Deploy REST

Store Models and Metadata

REST Build / Batch Scoring / Export

Oracle REST Data Services

Cognitive Services

Feature Cognitive Cognitive REST Extraction Image REST Text REST

• Automation, scalability, and performance

• Machine learning model deployment for applications

• Integrated with Oracle Database, Big Data, and Cloud environments

• APIs for REST, SQL, R, and Python

https://www.oracle.com/database/technologies/datawarehouse-bigdata/machine-learning.html