Deep Set Operators for XQuery
Bo Luo Dongwon Lee Wang-Chien Lee Peng Liu Penn State / IST Penn State / IST Penn State / CSE Penn State / IST [email protected] [email protected] [email protected] [email protected]
ABSTRACT There are three set operators defined in XQuery, namely union, intersect and except. They take node sequences as operands, in which each node is identified by its node-ID and treated as an atomic entity. However, according to XML semantics, each node is “a set of set(s)”, which have de- scendants in a tree-structured hierarchy. Unfortunately, the regular set operators as described above ignored this struc- tural feature of XML data. On the other hand, some XML applications can be benefited from set operators with differ- Figure 1: Tree structure of an XML document. ent semantics considering ancestor-descendant relationships between nodes. In this extended semantics, the comparison during query processing are conducted not only on nodes of that occur in both operands.” both operands, but also on their descendants, in a “deep” manner. In this paper, we identify the needs of such “deep” • “The except operator takes two node sequences as operands set operators and propose the deep-union, deep-intersect and returns a sequence containing all the nodes that oc- and deep-except operators. We further explore their prop- cur in the first operand but not in the second operand.” erties as well as relationships to regular set operators, and Second, according to [5], the set operators of XQuery are present a preliminary experience on implementing them as defined using the notion of node-IDs – unique (conceptual) user-defined functions of XQuery. ID per XML node. Therefore, for instance, the “union” of two node sequences are the union of node-IDs from both 1. INTRODUCTION sequences. On the other hand, XQuery uses XPath [1] to lo- In recent years, the eXtensible Markup Language (XML) [3] cate nodes. As defined in XPath, once the query is processed has emerged as the de facto standard for storing and ex- and the final node-IDs, say {5, 7}, are found, the answer to changing information. As such, the needs arise to query be returned to the user is the entire subtree rooted at the and tailor information in XML documents for various re- node with IDs 5 and 7. For instance, Figure 1, an XPath quirements in a more flexible and powerful manner. Towards expression “//person” represents all the subtree rooted at this goal, XQuery [2] was developed by two W3C working
creditcard
(b): a partial “person” node (//person deep-except //person/credicard) Figure 2: An example of deep-except semantics
Example 1 (XML Access Controls). In [8], we proposed information of Northern America to be sent to B. That is, an XML access control enforcement method that re-writes what B will receive is the “merged snapshot” of //item/name users’ incoming query Q to a safe query Q0 such that all frag- and //namerica/item. Like Example 1, this cannot be han- ments in Q that are asking for illegal data access are pruned dled by the regular union operator, and can only be coped out. In this context, conceptually, the safe query is either by introducing the novel deep-union operator as follows: (1) “intersect” of Q and what users are granted to access “//item/name deep-union //namerica/item”. 2 (i.e., positive access control rules), or (2) Q “except” what users are prohibited to access (i.e., negative access control 2. RELATED WORK rules). That is, if Q is //person, but a positive access control In [4], “deep union” and “deep update” operators are rule grants only the data matching //person/name[age>18], proposed to process semi-structured data. This operator then users must have an access to the data that are the takes two edge-labeled tree-structural documents as input intersect of //person and //person/name[age>18]. However, and merges/updates them based on their structural similar- the expression using the regular intersect operator, “//per- ities. son intersect //person/name[age>18]” would return NULL In [9], a deep-equal function is introduced to check the since no node-IDs from //person and //person/name[age>18] equality of two sequences. It checks if the arguments contain match. What is desirable here is, thus, a novel “intersect” items that are equal in values and positions. Despite the operator that compares operands to check their structural same name, their deep operators (functions) are different overlap in a deep manner, and returns the overlapped region. from ours in semantics or underlying operation objects. That is, “//person deep-intersect //person/name[age>18]” In [7], TAX algebra is proposed for tree-structured data, should return the nodes matching //person/name[age>18] in accordance with the relational algebra for relational data. since it is completely nested in //person. TIMBER [10] is developed based on this algebra. Symmetrically, for negative access control rules, we need As we illustrate above, there are needs for novel set opera- novel deep-except operator. For instance, if a user issues a tors with enhanced semantics than regular set operators. In query //person but a negative access control rule prevents this paper, we explore this issue of deep set operators. The her from accessing the data matching //person/creditcard, remainder of the paper is organized as follows. In section then what she can really access is the data matching the ex- 3, we define deep set operators and illustrate them with ex- pression “//person deep-except //person/creditcard”. Again, amples. Section 4 proposes their properties and comparison the regular except operator, if used, would have resulted with regular set operators. Section 5 describes the algo- wrong semantics. Figure 2 illustrates the above semantics rithm of our implementation of three deep set operators, of deep-except: (a) shows two
(r :: parent() 6∈ Pd ∧ r :: parent() 6∈ Qd)} 2 Table 1: Comparison between set operators and In the above definition, condition (1) represents the founda- deep set operators mental semantics of deep union operator: compare not only nodes in operand node sequences but also their descendants; (2) serves as a supplement: when a node satisfies condition Finally, deep-except operator is defined as follows. (1), all its descendants also satisfy condition (1), thus we D wanted to eliminate the descendants and keep the “greatest Definition 3 (deep-except) deep-except operator (−) takes 1 common node” only . Condition (2) is directly expressed two node sequences as inputs, and conducts deep-except-nodeseq as: operation between each node in the first operands vs. the second operand, and combine the outputs. Formally, !(r :: parent() ∈ Pd ∨ r :: parent() ∈ Qd) D According to De Morgan’s Law, it is equal to: P − Q = {r|r ∈ (pi deep − except − nodeseq Q)} 2 (r :: parent() 6∈ P ∧ r :: parent() 6∈ Q ) d d Here we can see that the definition of deep-except oper- Similarly, we have ator appears to be different from the other two deep set op- erators. The differences are further discussed and explained Definition 2 (deep-intersect) deep-intersect operator D in Section 4. (∩) takes two node sequences P and Q as operands, returns a sequence of nodes (1) who exist as a node or as a descendant 3.2 Examples of the nodes in “both” operand sequences, and (2) whose Consider the following XML fragment: parent does not satisfy (1). Formally,
peoplesite regions D D peopleperson regions P ∪ Q = Q ∪ P D D person P ∩ Q = Q ∩ P name 2 1 D D name 2 1 P − Q 6= Q − P, unless P = Q Associativity
D D D D (P ∪ Q) ∪ R = P ∪ (Q ∪ R) D D D D Figure 3: set operators defined in XQuery (P ∩ Q) ∩ R = P ∩ (Q ∩ R) Distributivity: D D D D D D D (P ∪ Q) ∩ R = (P ∩ R) ∪ (Q ∩ R) In addition, we would like to point out one essential differ- ence between deep-except operator and the other two. As we can see, both deep-union and deep-intersect operators D return a sequence of nodes that are originated from the given D XML tree, i.e. they do not create any new nodes. In this way, the production of these two operators are available for any other operations defined in XQuery that accepts nodes as operands, e.g. union, intersect, etc. On the other hand, whenever deep-except-nodeseq (case 2 of Remark 2) op- D eration is conducted for deep-except operator, new nodes D are constructed. Therefore, the production of deep-except operator may not be existing node in the XML tree. In this way, we should identify that although three operators are named “deep set operators” together, they are actually Figure 4: Deep set operators operators of different properties. deep-except operator is constructing new nodes while the other two operators re- turn nodes of the original XML tree, which is similar to the and except operator returns the first operand (grandparent regular union and intersect operators. node). As an example, “//person deep-except //person/name” On the other hand, if we take the above nodes and con- returns “person” nodes. However, the returned “person” duct deep-set operations, different results are generated, as nodes are not the same as “person” nodes that reside in shown in Figure 4. Deep set operators detects that the node original XML document: the “person” nodes produced by of second operand is a descendant of the first operand, thus: deep-except operation do not have “name” child. As a re- (1) deep-union operation returns the ancestor (e.g //per- sult, the newly constructed “person” nodes have new nodeIDs, son[@id=’1’]) only; (2) deep-intersect operation returns the which are different from original “person” nodes. Therefore, descendant only (e.g //person[@id=’1’]/address/zip); and with the production of deep-except operator, we have to (3) deep-intersect operation returns a newly constructed node, be careful when conducting further operations with exist- which contains partial content of the ancestor node, with one ing “person” nodes in the XML document, such as union, of the descendant node (the second operand) been removed. intersect etc. Moreover, we cannot use
The above example is case 2 in Table 1. Comparing Fig- D D ure 3 with Figure 4, we can observe the essential differ- //person − //person/name − //person/age ence between regular set operators and deep set operators: although it looks fine in semantics. Instead, we have to use: the regular set operators only compare and process node(s) in operands, while deep set operators compare and process D node(s) as well as their descendants. //person − (//person/name ∪ //person/age) 4.2 Comparison with Set Operators 4. PROPERTIES As we described above, deep-union and deep-intersect operators return nodes of the original document (probably 4.1 Arithmetic Properties node-IDs in actual applications). Here we provide two the- 2For case “A is a descendant of B” (//B//A), we can simply orems to further describe the output of these two operators, swap token A and B, thus it is still categorized as case 2 especially their relationships with regular set operators. Lemma 1. deep-union of two node sequences is subset of 5.1 deep-union operator their union production: According to theorem 1, product of deep-union operator D is a subset of regular union operation, i.e. each output node (P ∪ Q) ⊆ (P ∪ Q) must originally resides in at least one operand. Following this theorem, deep-union operator, as shown in Algorithm Proof. Lemma 1 is equivalent to: 1, enumerates the nodes in each operands, referring to re-
D quirements of deep-union and return the satisfied ones. if r ∈ P ∪ Q, then r ∈ P ∪ Q. (1) D Algorithm 1: deep-union Suppose we have an r that r ∈ P ∪Q, according to Definition 1: Input: input node sequences P and Q foreach node Pi of P do r ∈ P or r ∈ Q . d d if empty(Pi ∩ Q//*) & empty(Pi ∩ P //*) then Consider the equivalency of P and Q, we can assume Pi r ∈ P/descendant − or − self :: ∗ (2) foreach node Qi of Q do if empty(Qi ∩ (P ∪ P//∗)) & empty(Qi ∩ Q//∗) According to Definition 1, we also have then Qi r :: parent() 6∈ P/descendant − or − self :: ∗ which means r 6∈ P/descendant :: ∗ (3) 5.2 deep-intersect operator Comparing (2) and (3), we have deep-intersect is implemented in a similar way as deep-union. r ∈ P, thus r ∈ P ∪ Q, According to theorem 2, each output node of deep-intersect operator must originally resides in at least one operand. which proves Equation 1. (q.e.d) To enhance the readability of the algorithm, we divide it into three steps: first extract the regular “intersect” of two Lemma 2. of two node sequences is sub- deep-intersect operands, remove the possible ancestor-descendant relation- set of their union production: ship that may exists, and output the remaining. Then, for D each of the operand, enumerate the node items, output it (P ∩ Q) ⊆ (P ∪ Q), if it is a descendant of the other operand (some exceptions are eliminated). Algorithm 2 shows how deep-intersect On the other hand, it is not possibly subset of their intersect works. production: Algorithm 2: D deep-intersect ∃P,Q that (P ∩ Q) 6⊆ (P ∩ Q) Input: input node sequences P and Q
D foreach node r in (P ∩ Q) do This theorem is also described as : if r ∈ P ∩ Q, then if empty(r ∩ (P ∩ Q)//∗) then r ∈ P ∪ Q, but not always r ∈ P ∩ Q. r Lemma 3. Unless both operands contain ancestor-descendant foreach node Pi of P do if empty(P ∩ P //*) & !empty(P ∩ Q//∗)& nodes, deep-intersect of two node sequences is superset of i i empty(P ∩ (P ∩ Q)) then their intersect production: i Pi D (P ∩ Q) ⊆ (P ∩ Q) foreach node Qi of Q do if empty(Qi ∩ Q//*) & !empty(Qi ∩ P//∗)& The proofs of the above two theorems are similar to that empty(Qi ∩ (P ∩ Q)) then of Lemma 1, and thus omitted. Qi 5. PRELIMINARY IMPLEMENTATIONS We have implemented the deep set operators through user- defined functions of XQuery. With this implementation, 5.3 deep-except operator these operators are executable in any XML engine that sup- For deep-except operator, as we have described in Section ports XQuery. On the other hand, as a drawback, this 3.1, it is different from the other two deep set operators. engine-independent implementation may not be as efficient We implement it in a recursive manner: for each node in as implementations at lower level (say, engine level). the first operand, (1) if it has no overlap with any node in As XQuery’s user-defined functions, our implementations the second operand, output it; (2) if it is ancestor of any take node sequences as inputs, including XPath expressions node(s) in the second operand, construct a new node with and other forms of node sequences (e.g. products of set op- the same name, attributes and text, and enumerate all the erators). In addition, as described above, the results of both children to conduct deep-except with the second operand; deep-union and deep-intersect operations are XML node (3) otherwise, the node is “covered” by nodes in the second sequences and are available to further XQuery operations. operand, eliminate it. The general algorithm is shown in Our implementation also supports this property. Algorithm 3. ∩ =
site
people regions ∩ person =
name 2 1
_ =
Algorithm 3: deep-except 500
Input: input node sequences P and Q 400 D foreach node Pi in P do = if empty(Pi//* ∩ Q) and empty(Pi ∩ (Q ∪ Q//*)) 300 then Pi 200 if ! empty(Pi//* ∩Q) then construct element{ 100 Query evaluation time (ms) time Query evaluation
element name=name(Pi); 0 ∩ D D _ element attributes=Pi/@*; ∩ _D D element text()=Pi/text(); Operators deep-except(Pi/*,Q); Figure 5: Experiment results of deep-except = } else are slower because of two main reasons: 1. Deep set operators need extra overhead to support 5.4 Complexity more intensive semantics. It is not fair to directly com- pare these two groups of set operators, since they are Although the above preliminary implementations may not in fact different things. In this way, the preliminary _D be fully optimized, we can still estimate the computation experiments are just used as a reference of the perfor- = of deep set operators. The computation of deep-union mance of deep set operators. and deep-intersect operators are both O(nc ∗ ns), where nc denotes the total number of nodes in the sequences of 2. In our experiments, deep set operators are implemented operands, and ns denotes the total size of the subtrees rooted through user-defined functions (UDF) of XQuery. Thus at these nodes. On the other hand, the computation of they could not compete with regular set operators, deep-except operator (P deep-except Q) is denoted as which are well optimized in the engine (core). For O(ncq ∗nsp ∗i), where ncq denotes number of nodes in P , nsp instance, the main part of deep intersect operator is denotes the size of subtrees rooted at nodes in P , i denotes comparing two node sequences and picking the com- the maximum depth of these subtrees. This appears to be mon elements, for which algorithms of O(log n) ex- more expensive than the other two, since recursive function ists. However, we cannot employ these optimized al- call is employed in the implementation. It could be greatly gorithms in XQuery UDF. optimized if implemented at XML engine level. In addition, deep-except is slower than other two because of the recursive implementation of deep except semantics. 6. PRELIMINARY EXPERIMENTS However, this could not be avoided as long as we are using UDF of XQuery. 6.1 Experimental Results In the future, we plan to improve the performance of deep In the experiments, we use the well-known XMark schema set operators by implementing them in the XML database and its XML document generator [11] to generate the test engine. In this way, we can design and implement more document. We generate a 570KB XML document and test efficient (and more complicated) algorithms. Together with the performance of deep set operators vs. regular set oper- the advantage of engine level implementation, we are sure ators on it. For the underlying XML engine, as we have to gain performance improvement. described, any engine that implements XQuery is usable since our implementation relies on XQuery only; here we 7. CONCLUSION pick Galax 0.5.0 (and its Java API) [12]. In this paper, we propose deep set operators for XQuery. For each operator, we generate pairs of XPath expressions Regular set operators defined in [2, 5] only rely on node- (P and Q) as operands, and test all three deep set opera- IDs to identify and compare nodes. Therefore they ignore tors. For comparison, we also test regular set operators with the comparability issue of operands and neglect the tree- the same operands. However, please note that the straight structured hierarchy of XML data. The newly defined deep comparison of query evaluation time is unfair since regular set operators take the ancestor-descendant relationships within set operators are supported within the core of XML engine, input XML nodes into consideration. Items in input node but our deep set operators are only implemented through sequences are not regarded as atomic entities, instead, they user defined functions (UDF) of XQuery. We present the are treated as hierarchically nested elements, which accords comparison only for reference, not for competition. with the designed semantics of XML. Deep set operators Experiment results are shown in figure 5, note we only traverse into descendants of nodes to conduct comparison D measure the pairs where P and Q overlap (i.e. P ∩ Q 6= ϕ), and processing, in a “deep” manner. We propose the formal since only in these cases the products of regular set operators definition of deep set operators, then explore their compu- and deep set operators differ. tational properties as well as relationships with regular set operators. Finally we provide an implementation of deep 6.2 Discussions set operators, which purely rely on XQuery and thus in- From the figure, we can see that deep set operators are dependent from XML engine. Experiment show that some slower than regular set operators, but query processing time reasonable overhead is required to support deep set opera- is still within 0.5 ms on this document. Deep set operators tions. 8. REFERENCES APPENDIX [1] A. Berglund, S. Boag, D. Chamberlin, M. F. A. SOURCE CODE Fernndez, M. Kay, J. Robie, and J. Simeon. “XML Path Language (XPath) 2.0”. W3C Working Draft, A.1 Deep Union Nov. 2003. http://www.w3.org/TR/xpath20. [2] S. Boag, D. Chamberlin, M. F. Fern´andez, declare function local:d_u($p,$q){ D. Florescu, J. Robie, and J. Sim´eon.“XQuery 1.0: let $pqr:=$q//* union $p//* An XML Query Language”. W3C Working Draft, Feb. for $pn in $p 2005. return [3] T. Bray, J. Paoli, and C. M. Sperberg-McQueen if (empty($pn intersect $pqr)) then (Eds). “Extensible Markup Language (XML) 1.0 (2nd $pn Ed.)”. W3C Recommendation, Oct. 2000. else(), http://www.w3.org/TR/2000/REC-xml-20001006. let $pqr:=$p//* union $q//* [4] P. Buneman, A. Deutsch, and W.-C. Tan. “A for $qn in $q Deterministic Model for Semistructured Data”. In return Workshop on Query Processing for Semistructured if (empty($qn intersect $p) and Data and Non-Standard Data Formats, 1998. empty($qn intersect $pqr)) then [5] D. Draper, P. Fankhauser, M. Fernandez, $qn A. Malhotra, K. Ross, M. rys, J. Sim´eon,and else() P. Wadler (Eds). “XQuery 1.0 and XPath 2.0 Formal }; Semantics”. W3C Working Draft, Apr. 2004. http://www.w3.org/TR/xquery-semantics/. A.2 Deep Intersect [6] H. Hacigumus, B. R. Iyer, C. Li, and S. Mehrotra. “Executing SQL over encrypted data in the declare function local:d_i($p,$q){ database-service-provider model”. In ACM SIGMOD, let $pr:=$p//* 2002. let $qr:=$q//* [7] H. V. Jagadish, L. V. S. Lakshmanan, D. Srivastava, for $pn in $p and K. Thompson. “TAX: A Tree Algebra for XML”. return In Int’l Workshop on Data Bases and Programming if (empty($pn intersect $pr) and Languages (DBPL), Frascati, Rome, Sep. 2001. (not(empty($pn intersect $qr or [8] B. Luo, D. Lee, W.-C. Lee, and P. Liu. “QFilter: not(empty($pn intersect $q)))))) then Fine-Grained Run-Time XML Access Control via $pn NFA-based Query Rewriting”. In ACM CIKM’ 2004, else(), Washington D.C., USA, Nov. 2004. let $pr:=$p//* [9] Ashok Malhotra, Jim Melton, and Norman Walsh. let $qr:=$q//* “XQuery 1.0 and XPath 2.0 Functions and for $qn in $q Operators”. W3C Working Draft, Feb. 2005. return http://www.w3.org/TR/2005/WD-xpath-functions- if (empty($qn intersect $qr) and 20050211/. not(empty($qn intersect $pr))) then [10] Stelios Paparizos, Shurug Al-Khalifa, Adriane $qn Chapman, H. V. Jagadish, Laks V. S. Lakshmanan, else() Andrew Nierman, Jignesh M. Patel, Divesh }; Srivastava, Nuwee Wiwatwattana, Yuqing Wu, and Cong Yu. Timber: a native system for querying xml. A.3 Deep Except In SIGMOD ’03: Proceedings of the 2003 ACM SIGMOD international conference on Management of declare function local:d_e($p,$q){ data, pages 672–672, New York, NY, USA, 2003. ACM let $qr:=$q//* Press. for $pn in $p [11] A. R. Schmidt, F. Waas, M. L. Kersten, D. Florescu, return I. Manolescu, M. J. Carey, and R. Busse. “The XML if (empty($pn//* intersect $q) and Benchmark Project”. Technical Report INS-R0103, empty($pn intersect $q) and CWI, April 2001. empty($pn intersect $qr)) then $pn [12] J. Simeon and M. Fernandez. “Galax V 0.3.5”, Jan. else if (not(empty($pn//* intersect $q))) then 2004. http://db.bell-labs.com/galax/. element{name($pn)}{ $pn/@*, $pn/text(), local:d_e($pn/*,$q) } else() };