Snap Clone and Data Refresh Solutions Using Oracle Enterprise Manager CON6982

Deep Dive: Snap Clone and Data Refresh Solutions using Oracle Enterprise Manager CON6982

Martin Peña Senior Director, Product Management

Subhadeep Sengupta Consulting Product Manager

Janardhana Korapala,

Database Admin Consultant, Presented with Dell Inc

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

1 Snap Clone Overview

2 Features and Use Cases

3 Customer Use Case: Dell - Snap Clone with EM13

4 Q & A

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Hurdles to Rapid Development and Deployment Innovation is a key requirement for business growth • Increasing the speed of application development – Fast moving application/database requirements require systematic approach to dev/test – System/application patch/upgrades can introduce risk to dev/test – Custom application/development require deployment in less time • Efficient and up-to-date copies of mission-critical databases for agile application development – Multiple clones of production databases/systems often required to support dev/test activities – Dev/test and cloning of application and databases must be non-disruptive to production database

62% manage more than 5 DB copies, 34% manage more than 20

IOUG Database Storage Survey, 2014

QA Engineer Manager DBA Network Admin Storage Admin DBA QA Engineer Request HW & Setup OS Allocate Configure Request DB Approval Storage & Network Storage Cluster & DB Access DB

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Challenges with Cloning Today Impeding your efforts to innovate • Lack of automation – Scripts are not a viable solution for an agile development process • Traditional cloning techniques are slow – Existing storage vendor solutions are DBA unfriendly • Data explosion often outpaces storage capacity – Inhibits ITs ability to provide clones for development and testing • Archaic processes that require multiple users to share a single clone, or only supports fixed refresh cycles – Different priorities between DBAs and Storage administrators

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Enterprise Manager Snap Clone • Space Efficient • Data Refresh – Over 90% storage savings – Refresh from production • Rapid Cloning • Integrated DB Lifecycle Mgmt. – ~ 2 minutes to provision a 1 TB database – Masking, subsetting, lineage and association • Storage Agnostic • Complete On Premise & Cloud Solutions – Supports NAS and SAN storage

Functional Copies User can create Create Test Master Provision Thin Clones additional copies Test Master for functional testing without Production DB Masking & consuming space Sub-setting

Development Agility Powerful Storage Support Storage savings, rapid Supports ALL storage provisioning, Self Service vendors (NAS & SAN)

Database Cloning across Data Refresh Clouds Empower the user to make For Compute Services and adhoc clones and restores Database Cloud Services

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Data Reduction via Thin Clones Traditional Clones Thin Clones Full copy of the original database Only changed blocks in new clone space 3TB for original + clones 1.015TB for original + clones

Base Image Base Image 1TB Data Files 1TB Data Files

Pointers to Clone A Clone A existing blocks 1TB Data Files 5GB Data Files New data

Pointers to Clone B Clone B existing blocks 1TB Data Files 10GB Data Files New data

Efficiency Consolidated, shared platform maintained with automation Database Accessibility Dedicated Database Pluggable Database Database Schema Automated request channel via self-service Elasticity Agile scale up, scale down, relocation of services OR OR Data Accountability Meter usage, quota, compliance and governance Empty Full Clone Snap clones Integration API integration with 3rd party tools , Infrastructure change requests, dev workflows Physical, Virtual, or Engineered Systems (Hybrid Cloud) On Premise & On Oracle Cloud

RMAN Restore

RMAN Duplicate

Data Pump

Database Native Hardware Solutions Software Solutions

Vendor Specific Vendor Agnostic Storage Agnostic ACFS

• Fast, space-efficient snapshot copies of database • Available for Self-Service consumption. • Integration with pluggable database enables creation of Snap Clone with a “Single Click” • Initiated by using Database snapshot integration • All Exadata features work on features snapshots (smart scans, smart – SNAPSHOT COPY (for pluggable databases) flash cache, resource management – CLONEDB (for non-container databases) …)

10 x Snap Clones created concurrently from 60TB database in 15 minutes.

T1-R/O T2-R/O

DB6

T3-R/O DB1 DB2 DB3 DB4 DB5 ASM SPARSE DG ASM SPARSE DG DB7

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Snap Clone with Oracle ZSF Storage Appliance Engineered for optimizing database storage • Oracle ZS Storage Appliance – Optimized Oracle Database efficiency with HCC – Optimized RMAN backup and recovery – Optimized data reuse in development and test – DBAs manage snapshots and clones directly

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Snap Clone using Solaris File System (ZFS) • Uses a single stock Solaris 11.1+ image – physical or virtual [and NOT the Sun ZS3 Appliance ] • Supports any kind storage – NAS or SAN • For SAN, mount luns as raw disk and format with ZFS filesystem • Does NOT require the snapshot/clone licenses from the storage vendor, these features are available for free • Additional features include compression, deduplication, IO caching, etc • HA has to be handled externally either via Solaris Clusters, or by using HA features of the underlying hypervisor

SNAP SHOTS

PRODUCTION DBS TEST MASTER

SNAP CLONES

Data Guard P S Change Config. (SI/RAC)

RMAN Backup

Mask,Change Config. (SI/RAC)

RMAN Testincremental Master data backup Data RMAN backupSnapshot of archive logs Archive

Snap Clones using Standby

Private backups (snapshots) for SSA user Production Continuous or Discrete Replication Standby / Test Database Master Database

Replication Types: Continuous Discrete Technology Data Guard, Golden Gate RMAN, Snap Mirror, import/export …

Data Refresh Automatic and instantaneous Manual and at scheduled intervals

Masking and Not possible At source (in production), or in place at test master Subsetting

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Agile ‘Data Refresh’ Time Travel – Refresh, Rewind across Snapshots Scheduled or Manual Storage Snapshots or RMAN Backups of the test master database, called Profiles

Test Master t0 t1 t2 …. tN

• Clones can be created from any profile Test master is • Each user gets a personal read-write database regularly clone refreshed with current data from Clones production Data Refresh

Private backups

• Non Self Service (Admin) flows • Scheduled Snapshot/RMAN backup creation • Clone from a library – snapshots / backups / Image copies • Clone from a snapshot to a point-in-time or SCN • 1-click Refresh of clones from source • REST APIs & Emcli verbs

49 clones/month 60X faster with 180TB savings over EMC

Production Masking 60,000 refreshes per year supporting DevOps for Test Master ClonesSnap 2000+ Developers

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Snap Clone • Scalable – Supports 1 to 1,000s of clones • Protects your existing Storage Investments – Choice between hardware and software solution – Use of trusted technologies like data guard for test master refresh • Integrated DB Management with Enterprise Manager – Oracle’s flagship management product for all your database needs – In sync with DB releases (e.g. support for PDBs on day one) – Secure and role based access control; used by Fortune 1,000 customers – Protection from unnecessary point tools; reduce TCO

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Customer Overview Key Pains & Challenges • 70,000+ full DB refreshes/year • 10-20 cloning issues/day • Majority refreshes scheduled; 10% on demand

Customer Thumbnail Solution • One of the largest European financial services • Enterprise Manager Snap Clone on NetApp groups, with group net income of €23 billion 2012. • Refresh takes couple of minutes • 54,000 employees, 76 countries, and 32 million • Cloning from Standby databases; no impact on clients worldwide production • Using REST APIs; zero manual intervention

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Customer Overview Key Pains & Challenges • Average 5 days to provision UAT Database(s) for Dev Team • Sprawling Computing Resources • Limited Space in Data center

Customer Thumbnail Solution • Storage Reduction on Exadata, ZFS Storage Appliance • Largest mobile phone company in Egypt in terms of active subscribers • DB provisioning time reduced from 4 Days to <1 hour for 1TB DB • 36 million customer, 22k employees • Integrated Data Masking • Resource Pooling, Rapid Elasticity, Metering and Chargeback

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Customer Overview Key Pains & Challenges • Long provisioning cycle • Quality concerns due to limited data set • Not possible to create training environments on demand

Customer Thumbnail Solution • International container transportation, logistics • Delivered comprehensive service catalog and terminal companies • Enable self services for application owners to request new • 320 offices in 70 countries or modify environments. • Fleet of more than 300 ships • Implement DBaaS Snap Clone. • 90% Storage Savings demonstrated on both EMC and ZFS storage. • Data Refresh with Data Guard /Standby DB.

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Customer Overview Key Pains & Challenges • Dozens of concurrent development projects • Synchronization of application and DB deployments • Cloning frequency • API for automation and integration Customer Thumbnail Solution • Founded 1856 • Full Clones using RM Backup Provisioning for • 47,180 employees from over 50 countries Performance, Stress, UAT testing • Recognized as one of the world's most • Snap Clones using ZFS SA and Oracle Database important banks; one of Fortune Magazine's Appliances most admired companies. • Exadata Sparse Disk Snapshots provide Exadata performance on thin provisioned clones

Janardhana Korapala, Dell

Global Marketing Agenda

Oracle Enterprise Manager at Dell

• Overview Oracle Enterprise Manager at Dell • Enterprise Manager in action at Dell • New Enterprise Manager Architecture

Snap Clone at Dell

• Business Drivers • How Snap Clone Works? • Snap Clone Pilot Architecture • Snap Clones Interface • Snap Clones (Self-Service) • Proof of Concept Summary & Next Steps……

Q & A

Global Marketing Oracle Enterprise Manager at Dell

Global Marketing Overview Oracle Enterprise Manager at Dell

App/DB Servers Monitored in App/DB Servers Monitored in Non- Production Enterprise Manager Production Enterprise Manager 29K 57K 7.4K 2.6K Database Application Targets URL Monitoring Agents Targets (Middleware/EBS/Siebel/PS) 5 1.4K 350 15 3K 1.5K Downstream UDM Reports Jobs / Grouping User Application DB/Host Automation

Global Marketing Enterprise Manager in action at Dell

Monitored Targets Oracle Enterprise Dell In-house Manager Applications

Oracle DB Monitoring Deployed in 2007 Deployed in 2007 10g, 11g, 12c RMAN Backup Agent Targets Installation Discovery Oracle SOA Monitoring DMMR Deployed in 2010 Pilot Test 2011 Admin Group 10g, 11g Groups Database Metadata Repository Deployed in 2012 Targets Compliance DBCM Oracle EBS Monitoring Properties Policies Deployed in 2010 Database Compliance Mgmt 11g, 12c Baseline Pilot Test 2013 UDM Metric Deployed in 2015 KPI Deployed in 2013 Oracle SIEBEL Monitoring Incident Rule Reports Key Performance Indicator 10g, 11g, 12c, 13c Pilot Test 2013 Notify Method User Access DMP Deployed in 2012 Deployed in 2016 URL Monitoring Database Maintenance Planner N/A

Pilot Test & Convert to Deployed in 2015 ServiceNow Dec/2016

Incident (ServiceNow) Jobs Incident (Remedy) OMR Ticketing Incident Json Automation Service EMCLI (Database) Improvement Automation Email Notification Since 2010-2016 SMTP Server

NT Authentication PAC Group

Global Marketing New Enterprise Manager Architecture From 12c Level 3… …to 13c Level 4

Oracle Management Agent 13 c

Agent Agent Agent Agent Agent Agent Agent Agent Agent

Production Application Server Cluster Production Application Server Cluster (U.S., Austin, PC1) DR Application Server Cluster (U.S., Austin, PC1) Dell 13g M630's Oracle (U.S., Austin, PS3) Dell 13g M630's Management Service (13 c)

Primary Server Additional OMS Additional OMS Additional OMS Primary Server Additional OMS Additional OMS Additional OMS Additional OMS Primary Server Additional OMS Additional OMS Additional OMS Additional OMS RAM 128GB RAM 128GB RAM 128GB RAM 128GB RAM 128GB Rsync RAM 128GB RAM 128GB RAM 128GB RAM 128GB RAM 128GB Shared Storage CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core 400GB 400GB CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core /u01 250GB /u01 250GB /u01 250GB /u01 250GB /u01 250GB /u01 250GB /u01 250GB /u01 250GB /u01 250GB /u01 250GB

em12p.prd.amer.dell.com (U.S., Austin, PC1) Primary Cluster DB DR Cluster DB (U.S., Austin, PC1) Dell 13g M630's (U.S., Austin, PS3) Dell 13g M630's Oracle Management Repository (13 c)

Oracle Database 12 c

EM 12 P1 EM 12 P2 EM 12 P3 EM 12 P4 EM 12 P1.DR EM 12 P2.DR EM 12 P3.DR EM 12 P4.DR EM 12 P1.Prd EM 12 P2.Prd EM 12 P4.Prd EM 12 P3.Prd RAM 128GB RAM 128GB RAM 128GB RAM 128GB RAM 128GB RAM 128GB RAM 128GB RAM 128GB CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core CPU 36 Core DataGuard CPU 36 Core /u01 100GB /u01 100GB /u01 100GB /u01 100GB /u01 100GB /u01 100GB /u01 100GB /u01 100GB

Archive: 512GB, FRA: 512GB, Data: 3TB Archive: 512GB, FRA: 512GB, Data: 3TB  Support Disaster Recovery

 Zero downtime for OEM rolling patching Global Marketing Snap Clone at Dell

Global Marketing Business Drivers

“Data” issues for testing

Data refresh is manual

Existing solution (Optim) unsupportable, expensive

Too many database copies

Data masking via custom scripts – cumbersome

Virtual Database cloning solution

Testing possible with Real production data

Global Marketing How Snap Clone Works?

Continuous or Test master is Discrete regularly refreshed with current data from Replication production

Source Database Standby / Test Master Database

Scheduled or Manual Storage Snapshots or RMAN Backups of the test master database, called Profiles

T0 T1 … TN

Source A … Source A

Source A Source B … Source B

Source B Clones can User A be created Each user gets a from a library Source C of profiles personal read-write User B database clone Global Marketing SnapClone Pilot Architecture

Oracle Enterprise Manager Snap Clone Appliance 13c

Test Master DB Pool

Scheduled or Manual Storage Snapshots or RMAN Backups

San Storage Pool OEM OEM OEM Agent Agent Agent 50 TB 50 TB 50 TB Source Database Source Database Source Database

Global Marketing Snap Clones Interface

• One Test Master

• 4 Single Instance Snap Clones running from storage snapshots of Test Master DB mounted via NFS

Global Marketing Snap Clones (Self-Service)

One Test Master • All Datafiles are thin clones of parent files from Test Master • Five (5) Snap Clones and many more can be created as system resources allow (CPU & Memory)

Snap Clone Analyzer: >95% Storage Savings using Snap Clone vs. Full Clones

Global Marketing Proof of Concept Summary Database Cloud Capabilities

Private Cloud DBaaS

• Self Service Enablement: Service Portal, 3rd Party Automation, Self Service Performance Monitoring, Service Instance Snap Shots/Roll- back, Resource Collaboration • User Governance: Role Based Service Catalog and Quota Control, Show-back Value

Platform as a Service (PaaS)

• PaaS: Platform Virtualization, Automated Resource Allocation, Resource Governance, Resource Optimization, Cloud Resource Monitoring, Service Refresh/Control, Service Consumption Monitoring

PaaS Enterprise Integration

• RESTful Web Service API: All PaaS Functionality Exposed via Web Service API for Dell Enterprise Automation, Custom Service Creation

Global Marketing Next Steps Enterprise Automation – “Push Button” Application Deployment

“Push Button” Application Deployment – Full Function Application Deployment, Dell Internal SaaS

MWaaS – Continued Focus on Platform Virtualization

Orchestration Integration Templates

Application Blueprints – SDDC Support

PaaS IT Service Management

Global Marketing . Enterprise Manager Page on O.com

. Database as a Service Page on OTN Additional . Cloud Administration Guide (Documentation) Resources MOS Note 1549855.1 : Enterprise Manager Cloud Control Recommended Plug-Ins and Patches for Database as a Service (DBaaS) Join the Conversation Related Videos Twitter.com/oracle_em Youtube.com/OracleEnterpriseMgr Facebook.com/oracleenterprisemanager Blogs.Oracle.com/OEM Oracle.com/newsletters