Oracle Database Service High Availability with Data Guard? Robert Bialek Senior Principal Consultant @RobertPBialek doag2017 Who Am I Senior Principal Consultant and Trainer at Trivadis GmbH in Munich. – Master of Science in Computer Engineering. – At Trivadis since 2004. – Trivadis Partner since 2012. Focus: – Data and Service High Availability, Disaster Recovery. – Architecture Design, Optimization, Automation. – New Technologies (Trivadis Technology Center). – Open Source. – Technical Project Leadership. – Trainer: O-GRINF, O-RAC, O-DG. 2 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Our company. Trivadis is a market leader in IT consulting, system integration, solution engineering and the provision of IT services focusing on and technologies in Switzerland, Germany, Austria and Denmark. We offer our services in the following strategic business fields: O P E R A T I O N Trivadis Services takes over the interacting operation of your IT systems. 3 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? With over 600 specialists and IT experts in your region. COPENHAGEN 14 Trivadis branches and more than 600 employees. HAMBURG 200 Service Level Agreements. Over 4,000 training participants. Research and development budget: DÜSSELDORF CHF 5.0 million. FRANKFURT Financially self-supporting and sustainably profitable. STUTTGART Experience from more than 1,900 FREIBURG VIENNA MUNICH projects per year at over 800 BRUGG customers. BASEL ZURICH BERN GENEVA LAUSANNE 4 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Technology on its own won't help you. You need to know how to use it properly. 5 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Database Service High Availability – Goal Increase database service uptime, by: – eliminating any single point of failure to avoid unplanned outages. – minimizing the effect of an unplanned outage on the end user (automatic failover). – reducing downtimes during planned outages. Consider the whole SW/HW stack. Find the best cost/risk ratio. Effort Downtime Costs Database Application Complexity Storage Clients Server(s) Server(s) Best cost/risk ratio Availability 6 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Database Service High Availability – Options HA Cluster – primarily used option for service high availability: – Real Application Clusters. SPOF – RAC One Node. – Cold Failover Cluster. HA Data Replication – used mostly for data, rather than service high availability: – Data Guard (Fast-Start Failover/Global Data Services). – GoldenGate (Global Data Services). – Other replication technologies. HA 7 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Agenda 1. Introduction 2. Configuration 3. Special Cases 4. Conclusions 8 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Introduction 9 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Database Service HA with Data Guard? – Introduction Data Guard Yes, it can also be used for service high availability: – Planned downtimes – manual switchover. – Unplanned downtimes – fast-start failover or manual failover. Primary Standby FSFO Configuration Why might we consider Data Guard for service high availability: – Less complex than a cluster installation. – Infrastructure requirements not that high (even local storage is sufficient). – Not subject to additional license fees (EE license assumed). – Additionally, many other advantages: data high availability, snapshot standby, potentially rolling upgrade capability, ... But, with some restrictions we need to consider... 10 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Database Service HA with Data Guard? – Big Picture Backup Observers (optional, 12.2) Database Clients Master Observer Required Ping Ping Ping Primary RW Service Transparency required (failover/ switchover) ... Primary Target Candidate Target Failover Standbys Failover Standby (optional, 12.2) 11 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Database Service HA with Data Guard? – Monitoring Observer Failover condition detected Reconnect interval expired Logoff Timeout ObserverReconnect DGMGRL – Threads property set and reached Connect Failover re-tries W000 B001 P001 S001 SLEEP SLEEP ~ 3sec. Connect ~ 3sec. Enter PING State PING PING PRIMARY TARGET STANDBY 12 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Configuration 13 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Data Guard Protection Modes with FSFO – Prerequisites FSFO: Guaranted zero data FSFO: Data loss possible. FSFO: Guaranted zero data loss. loss. MaxAvailability (10.2+) MaxPerformance (11.1+) MaxProtection (12.2) ▪ LogXptMode=SYNC or ▪ LogXptMode=ASYNC ▪ LogXptMode=SYNC FASTSYNC (12.1+) ▪ FastStartFailoverTarget(*) ▪ FastStartFailoverTarget(*) ▪ FastStartFailoverTarget(*) ▪ FastStartFailoverLagLimit ▪ Flashback Database ▪ Flashback Database ▪ Flashback Database ▪ Recommended: at least 2 STDBY DBs (protection Mostly used mode downgrade!) protection mode All Protection Modes DGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold = <xy>; DGMGRL> ENABLE FAST_START FAILOVER; Value in seconds 14 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Observer Fast-Start Failover – Observer (1) Ping Monitoring component, initiates a failover procedure. In 12.2, up to 3 observers (in background) can be started: W000 B001 P001 S001 – One master and up to two backup (standby) observers. PRIMARY TARGET DGMGRL> START OBSERVER OBS1.TRIVADIS.COM IN BACKGROUND Failover Standby FILE IS '$ADMIN_SID/fsfo_$ORACLE_SID.dat' LOGFILE IS '$ADMIN_SID/fsfo_$ORACLE_SID.log' CONNECT IDENTIFIER IS <Alias1>.TRIVADIS.COM; Oracle wallet required In older releases: – Only one running observer (HA needs to be adressed). nohup dgmgrl -logfile $ADMIN_SID/fsfo_$ORACLE_SID.log <<EOD & CONNECT $CONNECT_DATA START OBSERVER FILE='$ADMIN_SID/fsfo_$ORACLE_SID.dat'; EOD 15 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Observer Fast-Start Failover – Observer (2) Ping Fast start-failover is initiated by the master observer to the target standby database, if one of the following W000 B001 conditions is detected: P001 S001 – observer and the target standby database cannot reach PRIMARY TARGET the primary database (default: ObserverOverride=‘FALSE‘). Failover Standby – user-configurable condition is met. – DBMS_DG.INITIATE_FS_FAILOVER function has been executed. Additonally, other pre-condidtions enforced by a protection mode need to be fulfilled: – MaxProtection/MaxAvailability: target failover standby is in SYNC. – MaxPerformance: FastStartFailoverLagLimit not reached for the target failover standby. 16 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Data Guard: Role-Based Services For a Data Guard system, we need a role-based service, Database Clients that is running only if database has a specific role: – Read-write service on a primary database. – Optionally, a service on standby databases for reporting. ? – Optionally, a service on snapshot standby databases. To accomplish this task: Service Service – Use Oracle Grid Infrastructure role-based services. R/W R/O [SNAP] – Create your own AFTER STARTUP ON DATABASE trigger. 17 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Data Guard: Example Role-Based Services Example role-based services with Grid Infrastructure. srvctl add service -db DB_SITE1 –service SRV_RW.trivadis.com -role PRIMARY srvctl add service -db DB_SITE1 –service SRV_RO.trivadis.com -role PHYSICAL_STANDBY srvctl add service -db DB_SITE1 -service SRV_SP.trivadis.com -role SNAPSHOT_STANDBY Services are started only if database and service role match. SvcAgent::start 680 query_db_role SvcAgent::start 710 not starting service srv_rw Role mismatch - Service role:PRIMARY, current DB role:PHYSICAL_STANDBY Depending on used client HA features (TAF, FAN/FCF, AC) additional service properties need to be specified. 18 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? Client-Side Configuration – Main Problems To Address CASE 1 CASE 2 New network session (connect) Already established network session Database Clients Database Clients 2 Connect attempts 4 Client failover 1 Connected 5 Client failover 3 Re-connect attempts 3 Wait for Problem Problem connect timeout 4 Wait for Problem re-connect timeout Problem 1 IP not reachable (server/network/… issue) 2 IP not reachable (server/network/… issue) 19 23.11.2017 Trivadis DOAG17: Oracle Database Service High Availability with Data Guard? New Oracle Net Session – Connect Timeout (1) 1 sqlnet.ora parameters (OCI, ODP.net) – Applies to each IP that a host name resolves to! – All Oracle client versions supported. LSNR LSNR TCP.CONNECT_TIMEOUT=3 #default 60 sec. 2 Oracle Net SQLNET.OUTBOUND_CONNECT_TIMEOUT=5 #no default Three-way handshake 3 For clients >=11.2: OLTP.trivadis.com = (DESCRIPTION = (FAILOVER=ON) (LOAD_BALANCE=OFF) Introduced in 12.1.0.2 (CONNECT_TIMEOUT=5)(RETRY_COUNT=3)(RETRY_DELAY=1)(TRANSPORT_CONNECT_TIMEOUT=3) (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP )(HOST = italy )(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP