Oracle DB High Availability on IBM Power Systems Alain Cyr Share Best Practices for Demanding IT IT Architect Environments [email protected]

IBM Oracle Center | November 2011

Oracle DB High Availability on IBM Power Systems Alain Cyr Share Best Practices for demanding IT IT Architect environments cyralain@fr.ibm.com

Frédéric Dubois IT Specialist [email protected] Agenda

IBM Power Technologies for Oracle

High -Availability and Disaster Recovery

IBM Live Partition Mobility with Oracle Database Demo

Oracle Certifies for the O/S & All Power servers run the same AIX = Power 795 Complete flexibility for Oracle workload deployment Power 780

Power 770 Consistency ° Binary compatibility ° Mainframe-inspired reliability ° Support for virtualization Power 750 ° AIX, Linux and IBM i OS Power 740 2S/4U Corporate Enterprise Downtime Power 720 1S/4U (Hours per Year) IBM AIX POWER Sun Solaris / SPARC HP UX 11/ PA RISC Power Systems Apple MAC Power 730 2S/2U HP UX 11/ HP Integrity with AIX deliver Power 710 1S/2U Red Hat Enterprise PS Blades Windows Server 2008 99.997% uptime Windows Server 2003 Open Source Linux

UNIX Server Rolling Four Quarter Average Revenue Share 45% POWER6 POWER6 Active Memory POWER6 PowerVM Lx86 Sharing 40% Live Partition POWER5 Mobility Micro -Partitioning 35% POWER7 Shared Storage Pools

30% POWER4 Dynamic LPARs 25% POWER6 Shared Processor Pools

20% HP Sun/Oracle IBM

15%

300 201 301 102 302 402 203 303 104 204 404 205 305 206 307 208 109 210 310 Q Q400 Q101 Q Q Q401 Q Q202 Q Q Q103 Q Q Q403 Q Q Q304 Q Q105 Q Q Q405 Q106 Q Q306 Q406 Q107 Q207 Q Q407 Q108 Q Q308 Q408 Q Q209 Q309 Q409 Q110 Q Q Q410 Source: IDC Server Tracker, Feb 2011

° A Collaborative Continuous Process between Oracle and IBM to ensure the Oracle Certification of IBM SWG and STG products at its most Current Releases with Oracle Product Releases *

° Applications Unlimited (PSFT, JDE, Siebel CRM, E-Business Suite)

° Fusion Applications

° Business Intelligence and EPM (BI Apps, OBI EE, Hyperion EPM)

° Retail GBU (Retek, 360Commerce, ProfitLogic)

° Communications GBU (Portal Software, MetaSolv)

° Insurance GBU (AdminServer, Skywire) ° Edge Applications: G-Log OTM, Agile PLM, Demantra

° Oracle Technology (DB and RAC, Fusion Middleware, Enterprise Mgr)

° Focus on Currency and Parity

° IBM Cross-Brand Technology Focus (IBM SWG and STG Products): extended technical advocates from Dev Labs

* Continuous evaluation as new companies are acquired

For 23+ years, IBM has delivered the best infrastructure components available in the market to support customers who have selected Oracle as their SW provider

•HUGE NEWS: Fusion Apps is available on AIX 6.1 concurrent with Oracle’s Base Development Platform!!!

Another Proof point that Oracle is committed to “red on blue ”

Oracle Virtualization Matrix

° Reach 70 to 80% CPU average usage Shared Oracle CPU ° Consolidate different workload profiles 18 and better optimize the resources Applications Micro-partitionning ° Development environment could be

Oracle 12 consolidated into the production Middleware infrastructure, workloads can be isolated and resources can be shared Oracle 6 Database – CPU resources for Dvt can be given a lower priority than 0 production as needed by business Dev & Test • Shared Processor Pool • Memory dynamic LPAR • Different VIO Server to isolate Storage I/Os for Dvt environment

Virtual Ethernet ° Partitions can be interconnected with Virtual Oracle Ethernet network. ° Virtual Ethernet is easy, flexible, performance and Applications it’s integrated at the low level of the system (microcode) Oracle ° Does not require any hardware for internal interconnection, can be bridged to the outside Middleware network infrastructure thought a VIO Server Host Ethernet Adapter (also called IVE) Oracle VLANs ° HEA switch can be configured and logical ports can Database be assigned to partitions for network interconnection

Virtual Ethernet Devices Ethernet Virtual ° HEA (also called IVE) is a hardware switch adapter connected to the internal Bus of the processor(s) Dev & Test (GX bus) and can connect to the external network VIO Server infrastructure ° Oracle supports both Virtual Ethernet and Host Ethernet Adapter technologies Enterprise Storage Network

You can build heterogeneous network configuration and mix virtual and physical infrastructure at the client LPAR

Virtual Fiber Chanel (NPIV) ° Partitions can be interconnected with Virtual Oracle Fiber Chanel adapters to the SAN throught an VIO Server and an NPIV adapter. Applications ° Virtual Fiber Chanel is is easy, flexible, and performance and it’s integrated at the low Oracle level of the system (microcode/Porwer Hypervisor). VIO Server maps the Virtual to Middleware Virtual SCSI or Fiber Chanel Devices Physical fiber chanels Oracle ° NPIV protocol does not require any hardware for internal interconnection, SAN disks are Database directly assigned from the SAN Virtual SCSI Dev & Test ° Physical Storage is assigned to the VIO VIO Server Server, and virtual mapping of the disks is done at the VIO Server level. ° VIOS provides Storage management techniques. (Multipathing, Logical Volume Storage Manager) ° Oracle supports both NPIV and Virtual SCSI protocols

Consolidate and Virtualize the Oracle Workloads to optimizes the IT resources

– Create Logical Partitions instead of using a server for each workload

• Mix different Workloads to screw CPU peaks (Production and Oracle Test/Dev) Applications • Reduce global number of CPUs

– Virtualize Server resources Micro-partitionning Oracle • Optimizes CPU usage, using Shared Processor Pool Middleware • Create a Virtual I/O Server partition to share physical Oracle

resources and simplify I/O infrastructure 18 Database

6 Shared CPU Test & Dev

0 + many more PowerVM features to reduce costs, improve flexibility, scalability, reliability VIO Servers

– SMT – LPM – MVSPP – AMS Power System – Dedicated Shared Processor – AME Network – NPIV – CuOD Storage

° During the day Database partition has workload peaks – Node1 database is defined as uncapped mode, and is able to get free capacity from the Shared Processor Pool (SPP). • Uncapped mode gets additional CPU up to the number of Virtual Processors • You can change the number of VPs using Dynamic LPAR • Weight parameter defines the priority for the uncapped partitions

– Test and Development partitions will be capped and will not pick up CPU cycles from the Shared Processor Pool

° Minimize license cost on core usage and define a Virtual Shared Processor Pool with a CPU capacity Entitlement. – Host DB partitions in a Virtual Shared Processor Pool – Core licensing is based on the VSPP Capacity

Technology Building Block is based on Oracle Apps IBM Power system and PowerVM AIX LPAR 70-80% • Shared Processor pool • Physical or Multitple virtual Pools Oracle DB • Uncapped partitions for PROD AIX LPAR Shared • Capped partitions for DEV • VIO Server virtualization Processor Oracle DB Pool(s) AIX LPAR • VIO Server 2.1 • 8Gbps NPIV FC adapters for Storage access • Ethernet adapters shared in VIO servers for virtual network access Oracle DB and /or Host Ethernet Adapters (IVE) AIX LPAR • AIX 7.1, 6.1, 5.3 or 5.2 (WPARs) • 2 VIO servers for Storage and network redundancy Shared Ethernet Shared Ethernet • Oracle single instance or Real Application Cluster Adapter(s) VIO VIO Adapter(s) Server 1 Server 2 • CRS/ASM/RAC 11g (Support for virtualized network NPIV FC NPIV FC and NPIV virtual disks Adapter(s) Adapter(s) • Consolidate DB Server Lpars up to 70 % to 80% of the server CPU capacity

SAN LAN

15 DOAG Conference 2011 (November 15th-17th) © 15 11/3/2011 2011 IBM Corporation Agenda

IBM Power Technologies for Oracle

High -Availability and Disaster Recovery

IBM Live Partition Mobility with Oracle Database Demo

99,999% = 5 minutes outage per year High 99,99% = 1 hour outage per year Availability Availability levelsAvailability 99,95% = 4 hours outage per year Superior Where are you ? 99,90% = 43 hours outage per year Availability 99% = 3 days outage per year

98% = 7 days outage per year Standard Availability 97% = 11 days outage per year

• Flood • Cpu • Electrical problem • Memory • Air-conditioning • Disk Live Partition • …. • Electrical outage Mobility improves it • …

Live Partition Mobility improves it

• Backup • Upgrade • diagnostic • …

Single Active / Passive Active / Active Extended Solution OAS Load Balancing (MAA) Router Application Weblogic Load Balancing Load Balancing + Tier OracleAS Guard Failover mode IBM Websphere Router IBM PowerHA Oracle Database 11g Oracle Database 10g Backup -RMAN RAC + Data Guard -User backups Oracle Data Guard Real Application Extended RAC Database -TSM Clusters (RAC) Recovery IBM PowerHA Oracle Streams Tier - RMAN IBM Infosphere - Oracle Flashback RAC + PowerHA -Media Recovery -TSM Clustering CRS PowerHA CRS PowerHA CRS PowerHA Server CFS ASM JFS2 ASM GPFS JFS2 ASM GPFS ASM GPFS Infra Virtualization SAN Volume Controller PowerVM: micro-p, VIOs, NPIV, AME, AMS, ... FlashCopy Remote replication Remote replication Storage RAID - Metro & Global - Metro & Global Mirroring (PPRC) Mirroring (PPRC) VolumeCopy - Stretch & Fabric - Stretch & Fabric SnapMirror Metro Cluster Metro Cluster

Cold Fail-Over Infrastructure

Protect all components • Third Party Applications

Oracle Oracle • Oracle Applications Micro-partitionning Micro-partitionning Applications Applications • Oracle Middleware

Oracle Oracle • Oracle Database Middleware Middleware

Database Database • Cold Failover with downtime

PowerHA or Oracle Grid Infrastructure • PowerHA for any product

VIO Server VIO Server or Oracle clusterware for Oracle products or DataGuard

Production Server Passive Server Cold Fail-Over architecture is an HA solution with downtime

IBM PowerHA or Oracle DataGuard • Requires a minimum downtime of the DB • DataGuard replays the logs on a standby DB Production and uses a 2 nd copy of the original database

Standby • PowerHA can use the same Database storage DB Instance 1 LPAR or volumes. Instance 1 • CPU and Memory resources for the passive server are not wasted, they are reserved and can be automatically activated with Capacity on Demand Stanby LPAR or DB Instance 2 • Simple process for switch over operation to DR and Instance 2 recovery to production server • Optimize the overall infrastructure with consolidation of other workloads (i.e. Development, Test, …) and

15 30 Capacity on Demand

22,5 • Workload balancing across the servers with PowerVM

7,5

15 features, LPM, micro-partitioning, .

CPU % 7,5 0

Use PowerVM and Oracle Grid Infrastructure

Provide High Availability and Scalable Solutions for the entire stack

Oracle Applications Micro-partitionning Micro-partitionning (Cluster Implementation) • No downtime on node failure • Rolling Upgrade Patching Oracle Middleware (Cluster Implementation) • Increase Workload treatment by adding nodes with no downtime Real Application Cluster Database

Oracle Grid Infrastructure • Combine RAC and PowerVM to Scale Right, in and out of the box ! VIO Server VIO Server

Active/active mode with Real Application Cluster for High Availability

15 ° Oracle Real Application Cluster (RAC) is Oracle flexible architecture 10 Applications – Workload balancing across the

Micro-partitionning Micro-partitionning nodes (partitions) of the servers Oracle 5 – Easy maintenance as 1 node can be Middleware stopped without Application 0 disruption Oracle Automated workload balancing Real Application Cluster ° Combine RAC and PowerVM Virtualization features Test Dev – Define RAC nodes with Micro partitioning and uncapped mode and high priority VIO Server VIO Server – Define Test and Development with 15 15 Micro partitioning and low priority or Server A Server B 10 10 capped mode 5 5 ° More resources per application 0 0 ‰increase average usage

Production Site DR Site Reduce downtime and delay the fail-over process • Easy maintenance as cluster nodes can be stopped with minimum disruption • Define RAC nodes with Micro partitioning and uncapped mode and high priority Production Cluster 1 But it is still is poor CPU usage Standby DB Instance 1 DB Instance 1 • Workload peaks at the same time can’t take full Clusterware Clusterware DB Instance 1 benefit of CPU virtualization • Idle CPU is wasted and multiplied by number of Production Cluster 2 servers in the cluster Stanby DB Instance 2 DB Instance 2 • Could be more flexible infrastructure for DB Clusterware Clusterware provisioning, maintenance and failover Instance 2 operations

Oracle RAC Cluster • Free resources are reserved in DR server to get Automated workload balancing 30 the additional workload in case of a node

22,5 15 15 failure/maintenance

7,5 7,5

7,5

0 0 CPU %0 Do not host only RAC(s) DB in the server …

Production Site DR Site Test / Dev Cluster 1 Test/Dev

DB Instance 1 DB Instance 1 Another single • Test and Development are different DB Clusterware Clusterware workloads profiles than Production • Mix Production/DR and Test Production Cluster 1 Production environment to optimize resources Standby • Define Test and Development workloads DB Instance 1 DB Instance 1 DB Clusterware Clusterware Instance 1 as less priority without impact on activities Standby Production Cluster 2 • Less hardware resources Stanby DB Instance 2 DB Instance 2 DB Clusterware Clusterware • Simplified and Flexible IT infrastructure Instance 2 • Less administration and maintenance VIOS1 VIOS2 VIOS1 VIOS2 VIOS1 VIOS2

15 15

22,5

7,5 7,5

7,5

0 0

SANSAN LANLAN LANLAN SANSAN Storage services (i.e. Flash copy/PPRC/MetroMirror)

Combine PowerVM Virtualization and Oracle Grid Infrastructure

Provide :

Oracle Applications Oracle Oracle • Cold Failover Micro-partitionning Micro-partitionning Micro-partitionning Micro-partitionning (Cluster Implementation) Applications Applications • Workload Live Oracle Middleware Oracle Oracle migration (Cluster Implementation) Middleware Middleware • High Availability Oracle Real Application Cluster Database Database • Scalability Oracle Grid Infrastructure

VIO Server VIO Server VIO Server VIO Server

Technology that makes one set of resources look and feel like another set of resources, preferably with more Logical desirable characteristics… Representation A logical representation of resources not constrained by physical limitations – Hides some of the complexity

Storage Virtualization – Adds or integrates new function with existing services – Can be nested or applied to multiple layers of a system

Physical Resources

Use IBM SAN Volume Controler

rootvg Oracle ASM IBM GPFS

Make changes to the storage Virtual Virtual Virtual Virtual Virtual without disrupting host Disk Disk Disk Disk Disk applications SAN Combine the capacity from SAN Volume Controller multiple arrays into several Advanced Copy Services Storage Pools

Apply common copy services DS4000 Storage Pools across the storage pool HP

HDS EMC Manage the storage pool from DS5000 Sym XiV DS8000 a central point

Load Balancers Use Oracle & IBM

Oracle Applications Oracle Oracle Technologies (Cluster Implementation) Applications Applications

Oracle Middleware Oracle Oracle (Cluster Implementation) Middleware Middleware Micro-partitionning Micro-partitionning Micro-partitionning Micro-partitionning

Oracle Single Access Name (my.cluster.com) Optimize Server Oracle Real Application Cluster RAC One DB Database

Oracle Grid Infrastructure ressources usage !

VIO Server VIO Server VIO Server VIO Server Add/Remove server ressources on the fly … Oracle ASM IBM GPFS Add/Remove server(s) on

Virtual Virtual Virtual Virtual Virtual the fly … Disk Disk Disk Disk Disk SAN Add/Remove Storage on the SAN Volume Controller fly …

Advanced Copy Services HP Reallocate ressources on Storage Pools the fly HDS DS4000 EMC DS5000 XiV Sym Ï No disruptions ! DS8000

IBM Power Technologies for Oracle

High -Availability and Disaster Recovery

IBM Live Partition Mobility with Oracle Database Demo

Use IBM Live Partition Mobility LPM migrates a partition from a source server to target server without shutdown

LPM is easy maintenance Oracle Other Oracle LPM is Live Workload Management Applications Application Micro-partitionning Micro-partitionning Oracle Oracle Middleware Middleware Examples of LPM operations: – Database partition runs a batch Oracle and Server A is overloaded, CPU Database is 100% busy, while Server B has free CPU capacity Test & Dev Test & Dev • Migrate Test & Dev partition to server B and free VIO Server VIO Server corresponding resource for Database partition on server A

24 24 Server A Server B – You need to maintain Server A, 18 18 Migrate Oracle database partition 12 12

6 6 to Server B without disruption

0 0

LPAR-1 LPAR-2 LPAR-3 LPAR-4 VIOS VIOS LPAR-1 LPAR-2 LPAR-3 LPAR-4

Migration Migration Controller Controller

Oracle Def 1 Oracle P P P P Def 2 P P PP PP P P P P P P P P P P P Def 3 P P P P PP PP PP P P P Def 4 AIX Kernel AIX Kernel AIX Kernel AIX Kernel AIX Kernel AIX Kernel AIX Kernel AIX Kernel Hypervisor Hypervisor Ethernet SAN Partition Mobility Requires: Partition Mobility Steps • POWER6 °Validation • AIX 5.3 / 6.1 or Linux Boot °Copy memory pages • All resources must be “Virtualized” ∑Host to target systems •No real resources Data °Transfer • SAN storage environment ∑Turn off Host resources •SAN Boot, temp space, same network ∑Activate Target resources The number of Oracle cores needed does not change before and after the migration

Power6 Power7 ° Application Server (swingbench) simulates users and generates workload to the Oracle Other Another Apps 1 DB DB ° The Oracle DB is run as a Real Application Swingbench Any other Cluster (RAC) with 3 nodes in the Power6 Apps DNS server.

DB Instance Web server ° You need to: Clusterware ° perform a maintenance on the left Other server/location Node A Apps 1 ° or run additional workload on the Power6 server Node B ° or migrate the production to a new Power7 LPM server Node C

VIOS1 VIOS2 VIOS1 VIOS2 This is just a few (11) clicks on the HMC

HMC 15

15 GUI and a few minutes process

7,5

0 SANSAN LANLAN ‹LPM the RAC node and make it fly !!

° Just try to migrate one RAC node without stopping the cluster on it. ° In case of any disruption of the node during the migration: ° the node will get out of the cluster and will reboot ;-( ° the workload will run on the remaining node(s). This is regular behavior of the RAC cluster

° This scenario is not yet supported , and certification tests are under progress, so you must not use it for production purpose. (only for test). This demo process without cluster stop on the migrated RAC node is a non official disclosure

° The infrastructure is based on PowerVM virtualization using VIO Servers on both source and target servers.

° It is basic installation without any customization and tuning ° SAN disks of RAC nodes are mapped and shared to all VIO servers. It’s done using VSCSI, but it’s easier to setup as NPIV and Virtual Fiber Adapters ° ASM is setup to share the disks across the nodes of the Oracle cluster. It could have been GPFS, it is also supported.

° The RAC interconnect network is trunked with other networks to the physical network infrastructure through a single physical adapter of the VIOS.

LPM is certified for Oracle single instance, certification tests for RAC are under Power6 Power7 progress Other Another Apps 1 DB ° Stop the Oracle cluster on the node, DNS Any other keep AIX alive. App server Apps crsctl stop crs Swingbench

DB Instance Web server ° Click on the HMC GUI and LPM the node, Clusterware remaining RAC nodes run the workload Other Node A Apps 1 and get CPU cycles (uncapped micro- partitions). You can also add RAM using Node B DLPAR

Node C ° Restart the Oracle cluster on the VIOS1 VIOS2 VIOS1 VIOS2 migrated node

HMC 15 15 crsctl start crs 7,5

7,5

0 SANSAN LANLAN http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101965