4/12/17 2 Announcements (Wed., Apr. 12) • HomeworK #4 due Monday, April 24, 11:55 pm • 4.1, 4.2, 4.3, X1 is posted XML- • Please start early XPath and XQuery • There may be another extra credit problem Introduction to Databases • Projects CompSci 316 Spring 2017 • keep working on them and write your final report • Demo in the weeK of April 24 • Google Cloud use? • If anyone is planning to use or using google cloud for HW/project, please send me an email 3 4 Approaches to XML processing • Text files/messages • Specialized XML DBMS • Tamino (Software AG), BaseX, eXist, Sedna, … Relational Mapping • Not as mature as relational DBMS • Relational (and object-relational) DBMS • Middleware and/or extensions • IBM DB2’s pureXML, PostgreSQL’s XML type/functions… 5 6 Mapping XML to relational 1. Node/edge-based: schema • Store XML in a column • Element(eid, tag) • Simple, compact • Attribute(eid, attrName, attrValue) Key: (eid, attrName) • CLOB (Character Large OBject) type + full-text indexing, or better, special XML type + functions • Attribute order does not matter • Poor integration with relational query processing • ElementChild(eid, pos, child) Keys: (eid, pos), (child) • Updates are expensive • pos specifies the ordering of children • Alternatives? • child references either Element(eid) or Text(tid) • Schema-oblivious mapping: ← Focus of this lecture well-formed XML → generic relational schema • Text(tid, value) 1. Node/edge-based mapping for graphs • tid cannot be the same as any eid 2. Interval-based mapping for trees F 3. Path-based mapping for trees Need to “invent” lots of id’s • Schema-aware mapping: FNeed indexes for efficiency, e.g., Element(tag), valid XML → special relational schema based on DTD Text(value) 1 4/12/17 7 8 Node/edge-based: example Node/edge-based: simple paths <bibliography> ElementChild <book ISBN="ISBN-10" price="80.00"> eid pos child <title>Foundations of Databases</title> Element • //title <author>Abiteboul</author> eid tag e0 1 e1 <author>Hull</author> • SELECT eid FROM Element WHERE tag = 'title'; e0 bibliography e1 1 e2 <author>Vianu</author> <publisher>Addison Wesley</publisher> e1 book e1 2 e3 • //section/title <year>1995</year> e1 3 e4 </book>… e2 title • SELECT e2.eid </bibliography> e3 author e1 4 e5 e1 5 e6 FROM Element e1, ElementChild c, Element e2 e4 author WHERE e1.tag = 'section' e5 author e1 6 e7 eid attrName attrValue AND e2.tag = 'title' Attribute e6 publisher e2 1 t0 e1 ISBN ISBN-10 e7 year e3 1 t1 AND e1.eid = c.eid e1 price 80 e4 1 t2 AND c.child = e2.eid; e5 1 t3 tid value Text e6 1 t4 FPath expression becomes joins! t0 Foundations of Databases e7 1 t5 t1 Abiteboul • Number of joins is proportional to the length of the path t2 Hull expression t3 Vianu t4 Addison Wesley t5 1995 9 10 Node/edge-based: complex paths Node/edge-based: descendent-or-self • //bibliography/book[author="Abiteboul"]/@price • //book//title • SELECT a.attrValue FROM Element e1, ElementChild c1, • Requires SQL3 recursion Element e2, Attribute a • WITH RECURSIVE ReachableFromBook(id) AS WHERE e1.tag = 'bibliography' ((SELECT eid FROM Element WHERE tag = 'book') AND e1.eid = c1.eid AND c1.child = e2.eid UNION AND e2.tag = 'book' (SELECT c.child AND EXISTS (SELECT * FROM ElementChild c2, some author of e2 FROM ReachableFromBook r, ElementChild c Element e3, ElementChild c3, Text t WHERE r.eid = c.eid)) WHERE e2.eid = c2.eid AND c2.child = e3.eid is ’Abiteboul’ AND e3.tag = 'author' SELECT eid AND e3.eid = c3.eid AND c3.child = t.tid FROM Element AND t.value = 'Abiteboul') WHERE eid IN (SELECT * FROM ReachableFromBook) AND tag = 'title'; AND e2.eid = a.eid AND a.attrName = 'price'; 11 12 2. Interval-based: schema Interval-based: example 1<bibliography> 2<book ISBN="ISBN-10" price="80.00"> bibliography 1,999,1 • Element(left, right, level, tag) 3<title>4Foundations of Databases</title>5 6<author>7Abiteboul</author>8 • left is the start position of the element 9<author>10Hull</author>11 book 2,21,2 12<author>13Vianu</author>14 15<publisher>16Addison Wesley</publisher>17 • right is the end position of the element 18<year>191995</year>20 </book>21… • level is the nesting depth of the element </bibliography>999 • Key is left title author author author publisher year • Text(left, right, level, value) 3,5,3 6,8,3 9,11,3 12,14,3 15,17,3 18,20,3 • Key is left FWhere did ElementChild go? • Attribute(left, attrName, attrValue) • � is the parent of � iff: • Key is (left, attrName) $ % [�$.left, �$.right] ⊃ [�%.left, �%.right], and �$.level = �%.level − 1 2 4/12/17 13 14 Interval-based: queries Summary so far • //section/title Node/edge-based vs. interval-based mapping • SELECT e2.left FROM Element e1, Element e2 WHERE e1.tag = 'section' AND e2.tag = 'title' AND e1.left < e2.left AND e2.right < e1.right • Path expression steps AND e1.level = e2.level-1; • Equality vs. containment join FPath expression becomes “containment” joins! • Descendent-or-self • Number of joins is proportional to path expression length • Recursion required vs. not required • //book//title • SELECT e2.left FROM Element e1, Element e2 WHERE e1.tag = 'book' AND e2.tag = 'title' AND e1.left < e2.left AND e2.right < e1.right; FNo recursion! 15 16 3. Path-based mapping Label-path encoding: queries Label-path encoding: paths as strings of labels • Simple path expressions with no conditions • Element(pathid, left, right, …), Path(pathid, path), //book//title … • Perform string matching on Path • Join qualified pathid’s with Element • path is a string containing the sequence of labels on a path starting from the root • //book[publisher='Prentice Hall']/title • Why are left and right still needed? • Evaluate //book/title • Evaluate //book/publisher[text()='Prentice Hall'] Element Path • Must then ensure title and publisher belong to the same pathid left right … pathid path 1 1 999 … 1 /bibliography book (how?) 2 2 21 … 2 /bibliography/book FPath expression with attached conditions needs to be 3 3 5 … 3 /bibliography/book/title broKen down, processed separately, and joined bacK 4 6 8 … 4 /bibliography/book/author 4 9 11 … … … 4 12 14 … … … … … 17 18 Another Path-based mapping Dewey-order encoding: queries Dewey-order encoding • Examples: • //title Each component of the id represents the order of //section/title the child within its parent //book//title //book[publisher='Prentice Hall']/title bibliography 1 • WorKs similarly as interval-based mapping • Except parent/child and ancestor/descendant relationship are book 1.1 1.2 1.3 1.4 checked by prefix matching 1.4.1 1.4.2 Element(dewey_pid, tag) title author author author publisher year Text(dewey_pid, value) 1.1.1 1.1.2 1.1.3 1.1.4 1.1.5 1.1.6 Attribute(dewey_pid, attrName, attrValue) 3 4/12/17 19 20 XSLT • XML-to-XML rule-based transformation language • Used most frequently as a stylesheet language • An XSLT program is an XML document itself An overview of XSLT, SAX, and DOM XSLT program XSLT processor Input XML Output XML Actually, output does not need to be in XML in general 21 22 XSLT program XSLT elements • An XSLT program is an XML document containing • Element describing transformation rules • Elements in the <xsl:> namespace • <xsl:template> • Elements in user namespace • Elements describing rule execution control • Roughly, result of evaluating an XSLT program on an input XML document = the XSLT document • <xsl:apply-templates> where each <xsl:> element is replaced with the • <xsl:call-template> result of its evaluation • Elements describing instructions • Basic ideas • <xsl:if>, <xsl:for-each>, <xsl:sort>, etc. • Templates specify how to transform matching • Elements generating output input nodes • <xsl:value-of> <xsl:copy-of> <xsl:element> • Structural recursion applies templates to , , , input trees recursively <xsl:attribute>, <xsl:text>, etc. • Uses XPath as a sub-language 23 24 XSLT example <xsl:template> • <xsl:template match="book[author='Abiteboul']"> Find titles of booKs authored by “Abiteboul” <booktitle> <xsl:value-of select="title"/> <?xml version="1.0"?> Standard header of an XSLT document </booktitle> <xsl:stylesheet </xsl:template> xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> <xsl:template match="book[author='Abiteboul']"> • <xsl:template match="match_expr"> is the basic XSLT <booktitle> construct describing a transformation rule <xsl:value-of select="title"/> </booktitle> • match_expr is an XPath-liKe expression specifying which </xsl:template> nodes this rule applies to </xsl:stylesheet> • <xsl:value-of select="xpath_expr"/> evaluates xpath_expr • Not quite; we will see why later within the context of the node matching the template, and converts the result sequence to a string • <booktitle> and </booktitle> simply get copied to the output for each node matched 4 4/12/17 25 26 Template in action Removing the extra output <xsl:template match="book[author='Abiteboul']"> <booktitle> • Add the following template: <xsl:value-of select="title"/> </booktitle> <xsl:template match="text()|@*"/> </xsl:template> • This template matches all text and attributes • Example XML fragment Template applies • XPath features <book ISBN="ISBN-10" price="80.00"> <title>Foundations of Databases</title> <booktitle> • text() is a node test that matches any text node <author>Abiteboul</author> Foundations of Databases <author>Hull</author> </booktitle> • @* <author>Vianu</author> matches any attribute <publisher>Addison Wesley</publisher> Template does not apply; • | <year>1995</year> means “or” in XPath <section>…</section>… default behavior is to process </book> the node recursively and