P RODUCT B RIEF DataDirect Technologies DataDirect XQuery™ Technical Overview Introduction Many Internet applications need to integrate information from multiple sources, including data found in web messages, relational data, and various XML sources. However, using XML with relational databases has its challenges. Each major database vendor provides XML extensions, but these extensions are different for each vendor and do not allow applications to be portable among databases. Some developers use XML standards (such as DOM, SAX, or StAX) in combination with database standards (such as ODBC or JDBC), but this approach requires developers to write and maintain large amounts of code. DataDirect XQuery™ allows applications to query both XML and relational sources, and can return XML results as text, DOM, SAX, or StAX. It runs in any Java environment, on any operating system, using any major database, with or without application servers or other servers. DataDirect XQuery, a Java™ implementation of XQuery that uses the XQuery API for Java (XQJ), allows developers to work with standards instead of proprietary query language extensions and APIs. In addition, DataDirect XQuery provides the best performance possible, using sophisticated techniques to optimize XML queries that access data from relational sources. This paper introduces XQuery, XQJ, and DataDirect XQuery, providing example queries and Java code. What Is XQuery? XQuery is a query language for XML. In the same way that SQL is used to query relational tables, XQuery is used to query XML or anything for which a virtual XML view can be provided, such as relational data. Typically, SQL queries create tables to represent the result of a query and XQuery queries create XML to represent the result of a query. This resulting XML can be simple or complex. For example, the result of an XQuery query may be as simple as a single integer; for example, the query might count the number of items that satisfy a condition. The result of a query can also be a complex document, such as an inventory report that has dynamic content or a SOAP message. In this paper, we use the term XML result to refer to the results of an XQuery query.1 1 In XQuery terminology, the result of an XQuery query is an instance of the XQuery data model. We use the term "XML result" for simplicity. TM D ATAD IRECT XQUERY T ECHNICAL O VERVIEW XQuery goes beyond the functionality of relational query languages and includes many features traditionally found in functional programming languages. Just as SQL is a relational query language and Java is an object- oriented language, XQuery often is thought of as a native XML programming language. In XQuery, the only complex data structure is XML, and the operations that are regularly needed for processing XML are directly supported in a convenient manner. XQuery can easily search any XML structure with path expressions, create any XML structure using constructors, and transform XML structures using FLWOR (for, let, where, order by, return) expressions. In addition, XQuery simplifies the tasks encountered when working with XML namespaces or data types. Because XML is used to represent and transfer data from a wide variety of sources, XQuery is also widely used for data integration. Even when data is not physically stored as XML, XQuery can be used with middleware that provides an XML view of the data. For example, SOAP may be used to acquire data from a variety of sources, and XQuery may be used to query the resulting SOAP messages (in XML) and data found in a relational database (using an XML view), and integrate the results. XQuery is currently under development in the W3C (World Wide Web Consortium), which is a standards body for the World Wide Web. The W3C maintains a Web page for XQuery, including pointers to the XQuery specifications, tutorials, and a variety of products, at http://www.w3.org/XML/Query.html. What Is XQJ? The XQuery API for Java (XQJ) is an API designed to support the XQuery language, just as the JDBC API supports the SQL query language. The XQJ standard (JSR 225) is being developed under the Java Community Process. For more information, refer to http://www.jcp.org/en/jsr/detail?id=225. What Is DataDirect XQuery™? DataDirect XQuery is the first embeddable component for XQuery that implements XQuery for Java API (XQJ). It supports all major relational databases on any Java platform. DataDirect XQuery allows you to query XML, relational databases, or a combination of the two, integrating the results for XML-based data exchange, XML-driven Web sites, and other applications that require or leverage the power of XML. DataDirect XQuery is designed for software developers and independent software vendors (ISVs) who need to manage heterogeneous data sources in XML applications. 2 OF 9 DATAD IRECT T ECHNOLOGIES J UNE 2006 TM D ATAD IRECT XQUERY T ECHNICAL O VERVIEW DataDirect XQuery supports both relational and XML sources, such as: • Databases through a JDBC™ connection • XML files through http:, ftp:, and file: URI schemes and Stylus Studio XML Deployment Adapter URI schemes, providing access to legacy data (for example, EDI and flat files) as XML • XML represented through DOM • XML stored in database columns using an XML data type • XML stored in character columns DataDirect XQuery™ Architecture The following diagram provides a high-level architectural overview of DataDirect XQuery. D ATAD IRECT T ECHNOLOGIES J UNE 2006 3 OF 9 TM D ATAD IRECT XQUERY T ECHNICAL O VERVIEW When you execute an XQuery query using DataDirect XQuery, DataDirect XQuery processes the query as described in the following flow: • A Java application passes a query to DataDirect XQuery’s implementation of XQJ. • The XQuery Engine analyzes the query and divides it into one or multiple XQuery expressions to be processed by the adaptors. • The XQuery Engine sends the query to the SQL adaptor or the Streaming XML adaptor based on its analysis: - If a relational source is queried, the XQuery Engine sends the query to the SQL adaptor. The SQL adaptor translates the query into SQL, which is used to query the database. The SQL adaptor receives the results, maps them into XML, and loads them in memory as required. - If an XML source is queried, the XQuery Engine sends the query to the Streaming XML adaptor, which executes the query and returns XML results. - If a flat or EDI file is queried, the XQuery Engine sends the query to the Streaming XML adaptor, which relies on the Flat File/EDI adaptors to retrieve an XML representation of the flat or EDI file. • The adaptors send the XML results to the XQuery Engine. If the XML results are obtained from more than one source, the XQuery Engine combines the results. • The Java application receives results as XML, using XQJ. Examples In this section, we'll examine some XQuery examples and a Java code example that uses XQJ to execute an XQuery query. NOTE: DataDirect XQuery is shipped with multiple examples showing how to use XQJ in your Java applications. The XQuery examples presented in this paper use the example database tables and XML files provided with DataDirect XQuery. XQuery Examples Example 1: Simple XQuery Query Using a FLWOR Expression The following simple XQuery query uses a FLWOR expression to return only the rows of the holdings database table that contain a value of AMZN in the stockticker column. for $h in collection('holdings')/holdings where $h/stockticker='AMZN' return $h 4 OF 9 DATAD IRECT T ECHNOLOGIES J UNE 2006 TM D ATAD IRECT XQUERY T ECHNICAL O VERVIEW Result The query returns the XML representation of each row. <holdings> <userid>Jonathan</userid> <stockticker>AMZN</stockticker> <shares>3000</shares> </holdings> <holdings> <userid>Minollo</userid> <stockticker>AMZN</stockticker> <shares>3000</shares> </holdings> Example 2: Creating a Specific XML Structure In this example, the XQuery query returns the same data as Example 1, but it uses an element constructor to create a different XML structure than that created in the previous example. for $h in collection('holdings')/holdings where $h/stockticker='AMZN' return <Amazon Client="{$h/userid}" Shares="{$h/shares}"/> Result The return clause creates an element named Amazon. It creates two attributes, Client and Shares, which contain the values of the userid and shares columns from the relational table. <Amazon Client="Jonathan" Shares="3000"/> <Amazon Client="Minollo" Shares="3000"/> Example 3: Combining Data from XML and Relational Sources Web messages, such as SOAP requests, are XML documents, and they can parameterize or provide data for a query. The following example joins an XML document named request.xml to two relational database tables named holdings and statistical. The request.xml file is joined to the holdings table by the UserId element in the XML file and the userid column of the holdings table. The two tables are joined by the ticker column of the statistical table and the stockticker column of the holdings table. let $request := doc('request.xml')/request for $user in $request/performance/UserId return <portfolio UserId="{$user}"> {$request } { for $st in collection('holdings')/holdings, $stats in collection('statistical')/statistical where $st/userid = $user and $stats/ticker = $st/stockticker return <stock> {$stats/companyname} {$st/stockticker} D ATAD IRECT T ECHNOLOGIES J UNE 2006 5 OF 9 TM D ATAD IRECT XQUERY T ECHNICAL O VERVIEW {$st/shares} {$stats/annualrevenues} </stock> } </portfolio> Result The result of this query is an element named portfolio. The first child