Session: A10 Let’s Take a Walk With WebSphere Federated Server Tom Glaser AT&T May 21, 2008 • 01:30 P.M. – 02:30 P.M. Platform: Cross Platform Abstract: Ever had a need to join data sources and you had to do this across platforms? I bet you thought it would be difficult, so that's as far as the project got. We'll tackle the issue of joining data sources across DBMSs and flat files by using WebSphere Federated Server. That's right, we'll join DB2, Oracle, Teradata, SQL Server and flat files within the same query. I'll show you how easy it is to query a flat file on z/OS using Classic Federation. We’ll take a look at one of our applications. * AT&T application overview * Current architecture and the problems associated with it: Requires access to: - DB2 z/OS - flat files on z/OS - SQL Server - Teradata - Oracle * Overview of WebSphere Federated Server and how it can make life a little easier * Installation/user experiences of WebSphere Federated Server * Installation/user experiences of WebSphere Classic Federation * New architecture of application using WebSphere Federated Server and WebSphere Classic Federation 1 Agenda • Who is AT&T • Application Overview / Complexity • IBM Tools Required to Reduce Complexity • WebSphere Federated Server • WebSphere Classic Data Architect • WebSphere Classic Federation for z/OS • Querying a sequential file on z/OS • Setup; step by step example • Setting up access to: • DB2 • SQL Server • Oracle • Teradata • Benefits obtained with IBM’s tool set 2 Agenda 2 Who is AT&T? 3 Who is AT&T? It’s now made up of the following companies: SBC + Classic AT&T + BellSouth + Cingular 3 System Environments 175+ IMS, 380+ CICS 185 1,000,000,000+ DB2 Production IMS Sub- Classic SBC only Transactions/Day Systems 62,000+ MIPS (15 IGS + 2 AT&T z/OS Support vs LUW Support(5 AT&T) 4 Today’s environment at SBC. This does not include Classic AT&T, BellSouth or Cingular. 4 Application Overview FinanceWeb Web Site and SQL Database Server • Acronym: FinanceWeb • Purpose: The FinanceWeb Servers are used to distribute monthly Budget Variance Reports, Headcount Reports, and various operational metrics reports for the Midwest Region Network Services Organization as well as similar Reports for the Credit & Collections Organization in all regions • Geographic Footprint: Illinois, Indiana, Michigan, Ohio, Wisconsin, Arkansas, Kansas, Missouri, Oklahoma, Texas, California, Nevada • #Users: 1500 5 FinanceWeb is an in-house developed application used to distribute monthly Budget Variance Reports, Headcount Reports, and various operational metrics reports for the Midwest Region Network Services Organization as well as similar Reports for the Credit & Collections Organization in all regions. 5 Application Overview 6 Home page for the FinanceWeb application. From here, the application users can drill down to specific reports. 6 Application Overview Statements from FinanceWeb application support team: • We want to join tables in a DB2 environment like AIRS (BIAS) in the Midwest to EDW (Enterprise Data Warehouse – Teradata) tables • We want to join SQL Server tables to datasets on the MVS platforms and to EDW tables • Right now we spend a large amount of time, pushing and pulling datasets between mainframes, SQL Server, QMF for Windows queries, SAS results, Access databases, and EDW • I'm afraid you would laugh at some of it. We FTP, import to SQL, pull from EDW, link tables and on and on just to get data together • We can't afford an IT project so we have learned to mold what we can get the best we can • I'm quite proud of what we have been able to accomplish but again, there has to be a better way without having to build huge complex IT projects • Soon….we will need access to Oracle • “Often I feel like I am hand cranking ice cream when I know there is an electric machine on the market” 7 Exact quotes from our “Associate Director of Credit Collections.” As you can see, data is coming from many sources. DB2, Teradata, SQL Server, Access, flat files,,,and soon, Oracle. 7 Application Overview “To get a current balance as of today on all business accounts in Illinois, I would run this QMF for Windows from the BIAS Table and export to SQL/or ACCESS or to .txt:” “…I would run it for the 5 states. Push it into Access.” 8 QMF for Windows is used to extract data from multiple tables, from multiple subsystems. The joining of the 5 states can be done just using the DB2 Federated Database that we currently use to join DB2 z/OS tables across subsystems. 8 Application Overview “Then I would have to go to Teradata, set up a temp table and run SQL Assistant to get more of the Business Unit Identifier and Collection Control Indicator (Collection Treatment performed on account) since that is not on the BIAS Tables:” 9 Teradata is used 9 Application Overview “Then I need to run a SAS query on the Mainframe to get some point in time info:” 10 SAS is used to extract data and creates multiple sequential files on z/OS. 10 Application Overview “Then we would use SQL Server Views or Query Analyzer to lump it all together…” 11 SQL Server is used…..so many pieces to this puzzle 11 To Sum it up… What can we do to make life easier for this application? They need access to the following: • Flat file, located on z/OS • DB2 on z/OS • Oracle • Teradata • SQL Server 12 There must be a way to bring all this diversity into one meaningful location. If data replication is not an option, maybe the data can be accessed in real time? To the programmer, all this data might appear as one data source….and who cares where the data is physically located, or what data sources it comes from. 12 What Tools Do We Need? WebSphere Federated Server(WFS) + Classic Data Architect(CDA) + Classic Federation Server 13 Three tools will help us to achieve one data source… WebSphere Federated Server + Classic Data Architect + Classic Federation. Federated Server is what pulls all the remote locations together. Data Architect is the GUI tool that connects to Classic Federation. Classic Federation is what allows the user to query a flat file on z/OS. 13 WebSphere Federated Server Where is WebSphere Federated Server going to run? • INTEL LINUX RHEL 4.0 (X86) • Current processes running on this server: • Federated Database for DB2 z/OS access • QMF for WebSphere • DB2 Web Query 14 Our existing Federated Database was running on Linux. The database allowed our DB2 z/OS users to join tables across subsystems. We had a test and a production server. We used the test server to install WebSphere Federated Server. We also had two other tools running on this server. DB2 Web Query and QMF for WebSphere. Both tools used WebSphere to access DB2 on z/OS. Web Query stored its repository on the local DB2 instance. 14 WebSphere Federated Server The problems (opportunities) begin…. DB2 UDB V8.2 was originally installed. But we wanted to upgrade to DB2 LUW 9.1. Why? Because WebSphere Federated Server was at V9 • DB2 LUW V9 requires Red Hat v4 U4 and kernel 2.6.9. We were on Red Hat v3 U6 and Kernel 2.4. So, we had to upgrade to Red Hat v4 U4 and kernel 2.6.9 • IBM Web Site showing Platform Requirements: http://www-306.ibm.com/software/data/db2/linux/validate/rhel4_viper.html 15 We migrated DB2 UDB V8 to DB2 LUW V9. 15 WebSphere Federated Server The problems...Continued • We attempted to upgrade the Linux server from 32 bit to 64 bit. 64 bit will exploit the additional memory that's available, providing better performance when joining the data sources within DB2. This was rejected due to the fact everything had to be reinstalled 16 To exploit additional memory for joining large result sets coming from the various data sources, we wanted to upgrade our server from 32 bit to 64 bit. This, however, did not happen. Today, out test server is still 32 bit. For production usage, we will upgrade to 64 bit. 16 WebSphere Federated Server Installation: • Do we need to discuss internal issues large companies might have to acquiring the software…..even thought there’s no cost for the software? 17 Like many large companies, simply downloading and installing the software is not as easy as it sounds. Even though our company had already purchased the software, the software had to purchased somehow. Acquiring the funding was the biggest issue. Then, to install, we have a “Change Management Tool” in place called Vantive. Everything must be done via Vantive, and requires a 3 day notice….unless to correct a production problem. 17 WebSphere Federated Server DB2 Data Joiner: DB2 Relational Connect DB2 Information Integrator WebSphere Information Integrator WebSphere Federated Server A closer look: • Federated Server and Database • Wrappers and other Federated Objects • Query Optimizer • Two-phase Commit for Federated Transactions • Rational Data Architect 18 The federated server and database Central components of a federated system include the federated server and the federated database. Wrappers and other federated objects Within a federated server and federated database, you use connectors (referred to as wrappers in the federated system), server definitions, user mappings, and nicknames to configure connections to a data source and to reference objects within the data source. Query optimization The federated database optimizes the performance of SQL queries against heterogeneous data sources by leveraging the DB2 query optimizer and by determining when it is faster to process a query on the data source or on the federated database.