Session F12
Federated Two-Phase Commit
Amy Chang IBM
Day 11th, May, 2006 • 08:00 a.m. – 09:40 a.m.
Platform: DB2 UDB for Linux, Unix, Windows Agenda • Federated Two-Phase Commit Overview • Federated Two-Phase Commit Configuration • Federated Two-Phase Commit Data Sources Supported • Federated Two-Phase Commit Usage Scenarios • Summary
2 Preface • This is a preview of a feature in an upcoming release. Some of the information can still get updated before it's made available in the final delivery. • The term “Websphere Federation Server” in this presentation refers to the Information Integrator.
3 Data Federation Basic Concepts I
Wrapper: a library WS II Federation Server federated views allowing access to a (optional) particular class of Nickname
Nickname Nickname Nickname data sources or DB2 UDB Catalog protocols (Net8...). Contains information Nicknames can have: about data source columns characteristics Table statistics Server: represents a Indexes Server1 Server2 Server3 specific data source Informational constraints WrapperA WrapperB Nickname: a local alias to data on a remote server (mapped to rows and Data at WS II instance: columns) Global catalog Option: an additional User data (Optional) piece of information Persistent query to customize the cache (Optional) (Optional (remote data (remote data (remote data object, e.g. collating Local Data) source 1) source 2) source 3) sequence of a server
4 Data Federation Basic Concepts II • Based on DB2 technology • Nicknames look just like tables to the application WS II Federation Server federated views (optional) • Information about DB2 cost-based optimizer Nickname Nickname federated objects are Table stored in DB2 system Local + Remote catalog Execution Plans • Execution plans are chosen by DB2 cost- Rel. Wrapper NR. Wrapper based optimizer, using Client library Client library knowledge about the data sources and Non-SQL objects Data at fed server: • Query fragments •Global catalog executed remotely are •User data •MQTs sent in the source’s Remote sources own dialect 5 System Supporting Single Site Update
2)Withdraw 3)COMMIT DB2 UDB for z/OS
MY_ACCOUNT Websphere Client Federation Server 1)Transaction invoke
4)Deposit Informix 5)COMMIT
YOUR_ACCOUNT 6 System Supporting Multi Site Update
2)Withdraw 5)PREPARE DB2 UDB for z/OS
7)COMMIT MY_ACCOUNT Websphere Client Federation Server 3)Deposit 1)Transaction invoke 4)PREPARE
6)COMMIT Informix
YOUR_ACCOUNT
7 Overview: Federated Two-Phase Commit
• Provides multi-site update functionality and location transparency for federated transactions via the industry standard X/Open XA protocol.
• Multiple data sources as well as local tables can be read and updated in a single transaction.
• Provides error recovery protection including automatic resynchronization and heuristic processing.
8 Terms • Federated One-Phase Commit Data Source (F1PC Data Source)
• Federated Two-Phase Commit Data Source (F2PC Data Source)
• Inbound Connection
• Outbound Connection
• Sub-TM
• TM
• RM
9 Terms Outbound Connection Federated Data Source 1 Inbound 1PC Connection F1PC Data Source Client Websphere Federation Server
2PC Federated Data Source 2
F2PC Data Source
10 Federated Two-Phase Commit: Connection Supported • Applications, Clients connect to Websphere Federation Server using: • DB2 Type 1 Connection – Application connects to a single DB2 database, Websphere Federation Server • DB2 Type 2 Connection – Application connects to multiple DB2 databases including Websphere Federation Server • XA DTP Environment – Application connects to Websphere Federation Server via XA API. • Websphere Federation Server connects to Federated data sources using: • 1PC - Data source native API (access F1PC Data Source) • 2PC - XA 2PC API (access F2PC Data Source)
11 DB2 Type 1 Inbound Connection
Type 1 Connect Client Websphere Federated Federation Data Server Source
The CONNECT (Type 1) statement connects an application process to the identified application server according to the rules for remote unit of work
An application process can only be connected to one application server at a time.
12 DB2 Type 2 Inbound Connection
Type 2 Connect Client Websphere Federated Federation Data Server Source
With the CONNECT (Type 2) statement, multiple connections can be maintained in a unit of work. Both TM database and RM databases are of DB2 family.
An application process can be connected to multiple application servers. Each unit of work can establish connection to multiple application servers.
Interface via DB2 DRDA 2PC Protocol.
13 XA Inbound Connection
XA API Websphere Federated XA TM Federation Data Server Source
In the XA DTP environment, external TM interacts with Websphere Federation Server via XA API. Websphere Federation Server acts as a RM to the external TM.
Interface via XA Protocol: xa_open, xa_start, xa_end, xa_prepare, xa_commit, xa_rollback, xa_close, etc.
14 Commit with F1PC Data Sources: Single Site Update is Supported
1PC DB2 for UPDATE nick_390; z/OS COMMIT; ------Websphere UPDATE nick_ora; Federation COMMIT; Server ------UPDATE local_tbl; COMMIT; 1PC ------Oracle
15 Commit with F2PC Data Sources: Multi Site Update is Supported
UPDATE nick_390; 2PC DB2 for z/OS UPDate nick_ora; UPDATE local_tbl; COMMIT; Websphere ------Federation Server
2PC
Oracle
16 Enabling Federated Two-Phase Commit • DB2_TWO_PHASE_COMMIT server option • Enable or disable federated two-phase commit • XA_OPEN_STRING_OPTIONS server option • There is default xa_open string for each data source • This option appends additional string on the xa open string • Making permanent changes across connections via • CREATE SERVER • ALTER SERVER • Making temporary change in a single connection via • SET SERVER OPTION
17 DB2_TWO_PHASE_COMMIT
• DB2_TWO_PHASE_COMMIT • ‘Y’: enable federated two-phase commit • ‘N’: default; federated one-phase commit • When a data source has the DB2_TWO_PHASE_COMMIT server option set to ‘Y’, it is an indication that transactions accessing that data source require two-phase commit semantics. • Not possible to switch transaction semantics on transaction boundary. • Even if all updates of a transaction occur on a single data source, if the data source is defined as a federated two-phase commit data source, then it goes through federated two-phase commit processing.
18 XA_OPEN_STRING_OPTIONS
• Necessary for MS SQL Server data source • Optional for Oracle, Sybase data sources • MS SQL Server Example • alter server mssql options (add xa_open_string_options 'RMRecoveryGuid=C53209C-D47C-4621-A1C8- FA288FA383FB') • Oracle Example • alter server orcl options (add xa_open_string_options '+LogDir=/home/michaelo/sqllib/db2dump+DbgFl=0x7') • Sybase Example • alter server sybs options (set XA_OPEN_STRING_OPTIONS '-LD:\Temp\sybase_xa.log -V11')
19 Federated Transaction Processing Rules
• Read operations are always permitted for both federated one-phase commit and federated two-phase commit data sources. • At most single federated one-phase commit data source can be updated in a single transaction. • Multi-site update is permitted on federated two-phase commit data sources. • Update both local site and remote site is considered as multi-site update. • If DB2 Type 1 connection is made to the WebSphere Federation Server, WebSphere Federation Server internally permits and coordinates multi-site update. • If DB2 Type 2 connection or XA inbound is made to the WebSphere Federation Server, because other RM(s) could be updated , WebSphere Federation Server will restrict all F1PC updates.
20 New Viper Feature
• Description: Update F1PC Data Source in a Type 2 Connect Transaction • Set up requirement: None • Usage Examples: • Update Teradata (which does not support two-phase commit); …or … • Update Oracle (defined as federated one-phase commit data source) • Implication: • In a multi-level global transaction where external TM exists, in the event of external TM sends in Rollback on phase 2 after updates against federated one-phase commit has been committed, then heuristic damage message will be issued. 21 Background V82FP5: DB2 UDB LUW refines the connection decision based on connection type info sent by DB2 z/OS client
z/OS Client ApApplplyy 1PC 1PC R/W
WebSphere Federation Server
Updating one F1PC data source is allowed for Federated 1PC inbound Data Source
If BATCH applications such as Apply in Replication, it’s classified as 1PC inbound connection.
22 Background V8FP5: DB2 UDB LUW refines the connection decision based on connection type info sent by DB2 z/OS client
CICS z/z/OOSS ClientClient CICS 2PC AApplppliicatiions 2PC R/W
WebSphere UUppdatingdating localocall tatablesbles Federation araree allall aallowlloweedd Server
RReeadad opeoperraationtion onon F1F1PCPC datdataa ssoouurrcecess iiss allalloowweedd Federated Data Source
If other applications such as CICS applications, it’s classified as 2PC inbound connection.
23 New Viper Feature: Open up update against one F1PC data source from z/OS client in an upcoming release
newnew z/OSz/OS cliclieentnt
2PC R/W
Websphere Federation Server
1PC R/W
Federated Update operations on one Date F1PC data source are Source allowed for 2PC inbound
24 Roles of Websphere Federation Server
• As an Internal TM • As a RM and subTM in a DB2-Coordinated Transaction • As a TM and subTM in a DB2-Coordinated Transaction • As a RM and subTM to External TM
25 Role: as an Internal TM
Client
1PC
WebSphere Federation Server
2PC 2PC 2PC
Federated Federated Federated Data Data Data Source 1 Source 2 Source N
26 Role: as a RM and a SubTM in a DB2-Coordinated Transaction
Client Inbound Connection: 2PC 2PC I am the 2PC DRDA 2PC TM_DATABASE
WebSphere DB2 DB2 Federation z/OS UDB Server
2PC 2PC 2PC
Federated Federated Federated Data Data Data Source 1 Source 2 Source N
27 Role: as a TM and a SubTM in a DB2-Coordinated Transaction
Inbound ClientClient Connection: 2PC DRDA 2PC 2PC 2PC I am the TM_DATABASE
WebSphere DB2 DB2 Federation z/OS UDB Server
2PC 2PC 2PC
Federated Federated Federated Data Data Data Source 1 Source 2 Source N
28 Role: as a RM and a subTM to External TM Inbound Connection: XA 2PC
XA 2PC External WebSphere TM Federation Server 2PC 2PC 2PC
Federated Federated Federated Data Data Data Source 1 Source 2 Source N
29 Usage Scenario 1
TM
Oracle DB2 z/OS WebSphere Federation Server
F2PCTEST UPDATE F2PCTEST ORCL_T SET C1=C1+1; CONNECT TO WFS; UPDATE ORCL_T SET C1=C1+1;
Websphere Federation Server as a RM to DB2 z/OS TM in a DB2-Coordinated Transaction
30 Usage Scenario 2
STPLEX4A (DSN7)
TM Oracle DB2 z/OS WS Federation Server CICS 2PC 2PC
UPDATE T1 T1 SET C1=C1+1;
Websphere Federation Server as a RM to DB2 z/OS, with CICS as External TM
31 Usage Scenario 3
STPLEX4A (DSN7)
TM MQ(TM) WebSphere Federation Server DB2 Connect XA<->DRDA MQBANKT
SPM
SQL UPDATE NK SET… Concentrator = ON Websphere Federation Server as a RM to MQ TM
Updating DB2 z/OS database with an XA-compliant TM requirement: To support MQ transaction manager with commit sequences from different processes, the DB2 Connect(TM) connection concentrator must be enabled.
32 Sync Point Manager (SPM) • When does SPM come into the picture • When the F2PC data sources are either DB2 for z/OS or DB2 for iSeries Version V5.3. • Reason: DB2 for z/OS and DB2 for iSeries Version V5.3 database servers do not support the XA interface. • What can SPM do • The SPM is a component of DB2 Connect which maps the XA two-phase commit protocol into the two-phase commit protocol used by DB2 for z/OS and DB2 for iSeries Version V5.3 database servers. • How to set up SPM • By default, the DB2 instance has pre-defined values for the SPM configuration parameters. No set up is required. • Trouble Shooting: Ensure the SPM_NAME and SVCENAME parameters in the database manager configuration are properly set, as they should be by default.
33 Transaction Error Recovery – part 1
• Errors during the federated two-phase commit process: • First Phase Error If a database communicates that it has failed to prepare to commit the unit of work, the Websphere Federation Server will roll back the unit of work during the second phase of the commit process and a rollback message is sent to all participating data sources that awaits transaction outcome. • Second Phase Error Error handling at this stage is dependent upon whether the second phase will commit or rollback. The second phase will only roll back the transaction if the first phase encountered an error.
34 Transaction Error Recovery – part 2
• Indoubt Transactions could happen on one of the following • Transaction is prepared, first phase is complete • Commit on second phase is not complete • Rollback on second phase is not complete • Impact to Application on Second Phase Failure • The application will be informed that the operation was successful through the SQLCA if the application connects to the Websphere Federation Server via a Type 2 connection. • The application will be disconnected from the Websphere Federation Server if the application connects to the Websphere Federation Server via a Type 1 connection.
35 Transaction Error Recovery – part 3
• Resynchronization • The resynchronization processing is initiated and managed by the Websphere Federation Server automatically. • Resynchronization attempts to sync up the incomplete Commit or Rollback to the federated two-phase commit data sources. • RESYNC_INTERVAL • Database manager configuration parameter • This parameter specifies the time interval in seconds for which resync should retry the recovery of any outstanding indoubt transactions • The default is 180 seconds
36 Heuristic Processing – part 1
• Manually Recovering Indoubt Transactions • If you cannot wait for the resynchronization process to automatically resolve indoubt transactions, you can resolve the indoubt transactions manually. • Heuristic commit or rollback decisions made at Websphere Federation Server are propagated to F2PC data sources. • In most situations, it is best to let the automated Websphere Federation Server resynchronization processes recover transactions.
37 Heuristic Processing – part 2
• List Indoubt Transactions command • List Indoubt Transactions With Prompting command • Only transactions whose status is indoubt (i), missing commit acknowledgment (m), or missing federated commit acknowledgment (d) can be committed. • Only transactions whose status is indoubt (i), missing federated rollback acknowledgment (b), or ended (e) can be rolled back. • Only transactions whose status is committed (c), rolled back (r), missing federated commit acknowledgment (d), or missing federated rollback acknowledgment (b) can be forgotten.
38 Federated Two-Phase Commit Performance – Overview 1 • Federated two-phase commit coordination is achieved by additional logging and communication overhead: • Additional database log writes to the Websphere Federation Server (for all transactions) • Additional database log writes to the remote data source (for update transactions only) • Additional database communication between the Websphere Federation Server and the data source (for all transactions) • Even if all updates of a transaction occur on a single data source, if the data source is defined as a federated two-phase commit data source, then it goes through federated two-phase commit processing.
39 Federated Two-Phase Commit Performance – Overview 2 • Generally speaking, a federated transaction accessing a data source which is enabled for two-phase commit naturally requires additional processing than one which accesses a data source enabled for one- phase commit. • A data source should only be enabled for two-phase commit when the federated transactions require two-phase commit semantics.
40 Federated Two-Phase Commit Performance - Impact • Impact of enabling federated two-phase commit • Measure the difference in elapsed time for operations involves a data source enabled for F2PC compared to the same operations against a data source that is not enabled for F2PC. (See Figure 1) • For a particular system configuration, the increase in elapsed time as a result of enabling federated two-phase commit is approximately constant; that is, it is approximately the same for a short transaction as it is for a long running transaction. • Applications that execute federated two-phase commit transactions concurrently are likely to incur less overhead than applications that execute the same transactions serially. (See Figure 2)
41 Figure 1: Single Stream Overhead of F2PC over F1PC
1PC -> F2PC for single site transactions. Percentage overhead by average transaction time in a 2-tier environment. Compare WriteCache and SCSI
300%
head 250% Read only - WriteCache er v Read+Update - WriteCache o 200%
% ReadOnly - SCSI 150% Read+Update - SCSI 100%
50% 0% 0.01 0.03 0.05 0.07 0.09 0.11 Average total transaction time (sec)
The elapsed time overhead of F2PC becomes small as transaction time increases – especially if the log is on a fast device with write caching. 42 Figure 2: Multi-Stream Overhead of F2PC over F1PC
WS II TPCC tpmC over time. Single source, 1PC vs F2PC. 5000
4000
3000 C
m Note: This tp 2000 was not a 1PC formal 1000 F2PC - WriteCache F2PC - SCSI TPC-C 0 benchmark Time (5m ramp-up + 60m run)
The overhead of F2PC is not as high in a workload with multiple transaction streams as in a single stream. Physical log writes can be shared by multiple agents that have flushed the logs, so the agents benefit from "piggybacking" their log writes with writes for other agents. 43 Federated Two-Phase Commit Performance – Recommendation 1 • Improving federated two-phase commit performance • The Websphere Federation Server log files should be placed on a device capable of fast writes – preferably a device which has a write cache. This is particularly true for readonly transactions. • For update transactions, placing the remote data source log files on a media capable of fast writes (if they are not already) will also help to reduce the elapsed time of federated two-phase commit transactions.
44 Federated Two-Phase Commit Performance – Recommendation 2 • Improving federated two-phase commit performance • For improving communication overhead • Locating the Websphere Federation Server on the same machine as at least one of the data sources. • Reducing the number of network hops between the Websphere Federation Server and data sources. • Increasing the network speed between the Websphere Federation Server and data sources.
45 Summary
• Federated Two-Phase Commit provides multi-site update functionality for federated transactions via the industry standard X/Open XA protocol in an upcoming release. • In the Viper delivery of Websphere Federation Server, Federated Two-Phase Commit for DRDA, MS SQL Server, Informix, Oracle, and Sybase CTLIB data sources in Trusted Serial environment will be supported. • Federated Two-Phase Commit performance is manageable and reasonable with a fast log device. • The resynchronization processing is initiated and managed by the Websphere Federation Server automatically.
46 Session F12 Session Title: Federated Two Phase Commit
Amy Chang IBM [email protected]
47