Oracle R Enterprise Charlie Berger Sr
Total Page:16
File Type:pdf, Size:1020Kb
1 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. FPO In-Database Analytics: Statistics and Advanced Analytics with R—Oracle R Enterprise Charlie Berger Sr. Director Product Management, Data Mining and Advanced Analytics Oracle Corporation [email protected] R 2 Copyrightwww.twitter.com/CharlieDataMine © 2011, Oracle and/or its affiliates. Open Source All rights reserved. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remain at the sole discretion of Oracle. 3 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Agenda • Big Data & Big Data Analytics • Open Source Project – Challenges limiting enterprise adoption of R New• R Enterprise Open Source – Features, benefits and advantages • Big Data Appliance New – Open source distribution of R • Oracle R Enterprise Beta Program • Q & A 4 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. What Makes it Big Data? SOCIAL BLOG SMART METER VOLUME VELOCITY VARIETY VALUE 5 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Big Data in Action DECIDE ACQUIRE Make Better Decisions Using Big Data ANALYZE ORGANIZE 6 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Announcing Oracle R Enterprise New 7 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. R Statistical Programming Language Open source language and environment Used for statistical computing and graphics Strength in easily producing publication-quality plots Highly extensible with open source community R packages 8 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Open Source Driven in part by the rise of big data, business intelligence (BI) is a rapidly growing market that has seen increasingly strong enterprise adoption rates. The concurrent to the growth of BI has been increased investment in predictive analytics; R is not only the tool of choice but the ideal environment for advanced analysis. R is designed to be extensible and integrate within BI suites to incorporate advanced analytics into reports. http://www.gartner.com/technology/core/products/research/topics/businessIntelligence.jsp “Hype Cycle for Analytic Applications, 2011, 30 August 2011 The number of web site links that point to the main web site of each software package on March 19, 2011. http://www.r4stats.com/popularity 9 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Growing Popularity • R’s rapid adoption over several years has earned its reputation as a new statistical software standard – Rival to SAS and SPSS While it is difficult to calculate exactly how many people use R, those most familiar with the software estimate that close to 250,000 people work with it regularly. “Data Analysts Captivated by R’s Power”, New York Times, Jan 6, 2009 http://www.r-project.org/ 10 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Typical R Approach Statistical and advanced analyses are run and stored on the user’s laptop 11 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. What Are ’s Challenges? 1. R is memory constrained – R processing is single threaded - does not exploit available compute infrastructure – R lacks industrial strength for enterprise use cases 2. R has lacked mindshare in Enterprise market – R is still met with caution by the long established SAS and IBM/SPSS statistical community • However, major university (e.g. Yale ) Statistics courses now taught in R • The FDA has recently shown indications for approval of new drugs for which the submission’s data analysis was performed using R 12 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Oracle R Enterprise Approach Data and statistical analysis are stored and run in- database Same R user experience & R same R clients Open Source Embed in operational systems Complements Oracle Data Mining 13 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. R What is Open Source Enterprise? • Oracle R Enterprise brings R’s statistical R functionality closer to the Oracle Database Open Source 1. Eliminate R’s memory constraint by enabling R to work directly & transparently on database objects – Allows R to run on very large data sets 2. Architected for Enterprise production infrastructure – Automatically exploits database parallelism without require parallel R programming – Build and immediately deploy 3. Oracle R leverages the latest R algorithms and packages – R is an embedded component of the DBMS server 14 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Architecture and Performance • Transparently function-ships R constructs to database via R SQL translation –Data structures –Functions • Data manipulation functions (select, project, join) • Basic statistical functions (avg, sum, summary) • Advanced statistical functions(gamma, beta) Seconds • Performs data-heavy computations in database –R for summary analysis and graphics • Transparent implementation enables using wide range of R “packages” from open source community 15 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Oracle R Enterprise Architecture Worst Use Case: Using ONTIME airline data, of the 36 busiest airports, run a box-plot analysis of the best/worst airports for arrival delay? R workspace console Best Function push-down Oracle statistics engine OBIEE, Web data transformation & – Services statistics R Open Source Development Production Consumption 16 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Oracle R Enterprise for Statistical Development R Oracle R Oracle R OCI package makes all R commands, graphics, Oracle packages are identical to user in tables/views visible to R both R and Oracle R Enterprise Data can be in R data frames or Oracle Tables/Views 17 Copyright © 2011, Oracle and/or its affiliates. Oracle Confidential All rights reserved. Oracle R Enterprise for Statistical Development R Oracle R Oracle R OCI package makes all R commands, graphics, Oracle packages are identical to user in tables/views visible to R both R and Oracle R Data can be in R data frames or Oracle Tables/Views 18 Copyright © 2011, Oracle and/or its affiliates. Oracle Confidential All rights reserved. Benefits "R for the Enterprise" OpenR Source • Oracle R Enterprise enables you to: – Run R to interactively explore and analyze data inside the Database – Develop R scripts on big data stored as tables and views inside the Oracle database and then deploy them within the enterprise—without requiring code changes – Leverage R’s familiar R console and open source R GUIs and IDEs to explore and analyze data either in the database and stored as R data frames – Meet the statistical and advanced analytical requirements of the enterprise – Exploit an information technology platform designed to support analytically- driven applications. – Leverage 30+ years of experience of ever advancing Oracle Database technology. 19 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. How Oracle R Enterprise Works ORE Computation Engines OpenR Source • Oracle R Enterprise tightly integrates R with the database and fully manages the data operated upon by R code. – The database is always involved in serving up data to the R code. – Oracle R Enterprise runs in the Oracle Database. • Oracle R Enterprise eliminates data movement and duplication, maintains security and minimizes latency time from raw data to new information. • Three ORE Computation Engines – Oracle R Enterprise provides three different interfaces between the open-source R engine and the Oracle database: 1. Oracle R Enterprise (ORE) Transparency Layer 2. In-Database Statistics Engine 3. Embedded R 20 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. How Oracle R Enterprise Works ORE Computation Engines OpenR Source 1. Oracle R Enterprise (ORE) Transparency Layer – Traps all R commands and scripts prior to execution and looks for opportunities to function ship them to the database for native execution – ORE transparency layer converts R commands/scripts into SQL equivalents and thereby leverages the database as a compute engine. 21 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. How Oracle R Enterprise Works ORE Computation Engines OpenR Source 2. In-Database Statistics Engine – Significantly extends the Oracle Database’s library of All Base R functions statistical functions and advanced analytical computations R Multiple Regression – Provides support for the complete R language and …. Driven by customers statistical functions found in Base R and selected R packages based on customer usage Base SAS PROCS • Open source packages - written entirely in R language with only • PROC FREQ • PROC MEANS the functions for which we have implemented SQL counterparts - • PROC RANK can be translated to execute in database. • PROC STANDARD • PROC SUMMARY – Without anything visibly different to the R users, their R • PROC UNIVARIATE commands and scripts are oftentimes accelerated by a • PROC APPEND • PROC SORT factor of 10-100x • PROC TRANSPOSE • PROC SQL – Base SAS and most common SAS PROC "knock-offs" • PROC CORR 22 Copyright © 2011, Oracle and/or its affiliates. All