DOCSLIB.ORG

UNSW >013758187

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
UNSW >013758187 UNSW >013758187 PLEASE TYPE THE UNIVERSITY OF NEW SOUTH WALES Thesis/Dissertation Sheet Surname or Family name: Lam First name: Franky Shung Lai Qtlier name/s: Abbreviation for degree as given in the University calendar: PhD School: CSE Faculty: Engineering Title: Optimization Techniques for XML Databases Abstract 350 words maximum: (PLEASE TYPE) In this thesis, we address several fundamental concerns of maintaining and querying huge ordered label trees. We focus on practical implementation issues of storing, updating and query optimisation of XIVIL database management system. Specifically, we address the XML order maintenance problem, efficient evaluation of structural join, intrinsic skew handling of join, succinct storage of XML data and update synchronization of mobile XML data. Declaration relating to disposition of project thesis/dissertation I hereby grant to the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or in part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968.1 retain all property rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstragt^lnternational (this is applicable to doctoral theses only). A / •/• Sighature ^itn^ Date The University recognises that there may be exceptional circumstances requiring^estrictions on copying or conditions on use. Requests for restriction for a period of up to 2 years must be made in writing. Requests for^longer period of restriction may be considered in exceptional circumstances and require the approval of the Dean of Graduate Research. FOR OFFICE USE ONLY Date of completion of requirements for Award: COPYRIGHT STATEMENT 'I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International (this is applicable to doctoral theses only). I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restriction of the digital copy of my thesis or dissertation.' C Signed 2 7/3 Date AUTHENTICITY STATEMENT 'I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the conversion to digital format.' Signed Date THE UNIVERSITY OF NEW SOUTH WALES SCHOOL OF COMPUTER SCIENCE & ENGINEERING OPTIMIZATION TECHNIQUES FOR XML DATABASES Pranky Shung Lai LAM (2288414) PhD in Computer Science and Engineering Supervisor: Dr. Raymond K. Wong 5 5 ? A C li^f 11 AUG 20G- Originality Statement 1 isBRARY I hereby declare that this submission is own work and to the best of my knowl- edge it contains no materials previously pubhshed or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extend that assistance from others in the project's design and conception or in style, presentation and linguistic expression is acknowledged. TT li/ c 7 Abstract In this thesis, we address several fundamental concerns of maintaining and querying huge ordered label trees. We focus on practical implementation issues of storing, up- dating and query optimization of XML database management system. Specifically, we address the XML order maintenance problem, efficient evaluation of structural join, intrinsic skew handling of join, succinct storage of XML data and update syn- chronization of mobile XML data. Acknowledgments Gratitude is not only the greatest of virtues, but the parent of all others. — Cicero (106-43 BC) First of all, I would like to express my utmost respect and deepest gratitude toward Dr. Raymond Wong, my supervisor, for his persistent guidance, excellent advice and the inspirations I received that have made this thesis possible. Furthermore, the aspects of having an entrepreneurial mindset, working as part of a research team and as part of the research community that I learned from him are invaluable. I am extremely grateful and appreciative of the co-authors of all my publications, especially Damien Fisher and Wilham Shui, for their numerous constructive dis- cussions and collaborations during these years. Their positive contributions to this thesis are immeasurable. They contributed on the the Order Maintenance chapter. Efficient Structural Join chapter and the Maintaining Succinct XML Data Chap- ter. William's critical contribution included implementation of the related works in the experiment sections; whilst Damien's most important input included significant correction of lingustic expressions throughout the above three chapters, as well as the probablistic formulas on the Order Maintenance chapter. I would also like to pay my sincere gratitude to the reviewers that have spent their precious time to review this thesis and their insightful comments that I received. Last, but definitely not least, I would love to thank my family, Jenny, Sebastian and Chantel, for their unlimited support, patience, understanding and encouragement. n Related Publications 1. Raymond K. Wong, Pranky Lam, William M. Shui. Querying and Main- taining a Compact XML Storage. In Proceedings of International World Wide Web Conference (WWW), Banff, Alberta, Canada, May 08-May 12, 2007. (Acceptance rate: 14%) 2. Damien K. Fisher, Franky Lam, William M. Shui, Raymond K. Wong. Dy- namic Labeling Schemes for Ordered XML Based on Type Information. In Proceedings of l?^'^ Australasian Database Conference (ADC), Hobart, Tas- mania, Australia, Jan 16-Jan 19, 2006. p69~78. 3. WilHam M. Shui, Franky Lam, Damien K. Fisher, Raymond K. Wong. Querying and Maintenance Ordered XML Data Using Relational Databases. In Proceedings of IG^'^ Australasian Database Conference (ADC), Newcastle, Austraha, Jan 31-Feb 03, 2005. p85-94. 4. Wilham M. Shui, Damien K. Fisher, Franky Lam, Raymond K. Wong. Ef- fective Clustering Schemes for XML Databases. In Proceedings of Inter- national Conference of Database and Expert Systems Apphcations (DEXA), Zaragoza, Spain, Aug 30~Sep 03, 2004. p569-579. 5. Damien K. Fisher, Pranky Lam, Raymond K. Wong. Algebraic Transforma- tion and Optimization for XQuery. In Proceedings of the Asian Pacific Web Conference (APWeb), Hangzhou, China, Apr 14-17, 2004. p201-210. 6. Pranky Lam, Wilham M. Shui, Damien K. Fisher, Raymond K. Wong. Skip- ping Strategies for Efficient Structural Joins. In Proceedings of the 9^^ Inter- national Conference on Database Systems for Advanced Applications (DAS- FAA), Jeju Island, Korea, Mar 17-19, 2004. pl96-207. (Acceptance rate: 60/272 = 22%) 7. Michael Barg, Raymond K. Wong, Franky Lam. An Efficient Path Index for Querying Semi-structured Data. In Proceedings of the Asian Pacific Web Conference (APWeb), Xi'an, China, Sep 27-29, 2003. p89-94. (Acceptance rate: 39/136 = 28%) 8. Damien K. Fisher, Franky Lam, Wilham M. Shui, Raymond K. Wong. Effi- cient ordering for XML data. In Proceedings of 12^^ ACM International Con- ference on Information and Knowledge Management (CIKM), New Orleans, Louisiana, USA, Nov 2-8, 2003. p350-357. (Acceptance rate: 59/400 = 15%) 9. Franky Lam, Nicole Lam, Raymond K. Wong. Efficient synchronization for Mobile XML data. In Proceedings of ACM International Conference on Information and Knowledge Management (CIKM), McLean, Virginia, USA, Nov 4-9, 2002. pl53-160 (Acceptance rate: 74/300 = 25%) 10. Franky Lam, Nicole Lam, Raymond K. Wong. Performance Evaluation of XSync: An Efficient Synchronizer for Mobile XML Data. In Proceedings of The IEEE International Conference on Communications Systems (ICCS), Singapore, Nov 25-28, 2002. pl08 (2P-02-07). 11. Franky Lam, Nicole Lam, Raymond K. Wong. Efficient Update Propagations for Semistructured Data in Mobile Environment. In Proceedings of Inter- national Conference on Information Technology and Apphcations (ICITA), Bathurst, NSW, Australia, Nov 25-28 2002. p98-l. 12. Franky Lam, Raymond K. Wong, Mehmet A. Orgun. Modeling and Manip- ulating Multidimensional Data in Semistructured Databases. In Proceedings of the International Conference on Database Systems for Advanced Appli- cations (DASFAA), Hong Kong, China, Apr 18-20, 2001. pl4-21. 13. Raymond K. Wong, Franky Lam, Mehmet A. Orgun. Modeling and Manipu- lating Multidimensional Data in Semistructured Databases. World Wide Web 4(1^2): 79-99 (2001) 14. Raymond K. Wong, Franky Lam, Stephen Graham, Wilham Shui. An XML Repository for Molecular Sequence Data. In Proceedings of IEEE Inter- national Symposium on Bioinformatics and Biomedical Engineering (BIBE), Arlington, Virginia, USA, November 8 10, 2000. IEEE CS. p35 -42. Related Technical Reports 1. Franky Lam, Raymond K. Wong. Rotated Library Sort. Technical Report. UNSW-CSE-TR-0506. University of New South Wales. Mar 2005 2. Franky Lam, Wilham M. Shui, Damien K. Fisher, Raymond K. Wong. Querying and Maintaining Succinct XML Data. Technical Report. UNSW- CSE-TR-0424.
Load more

Recommended publications
  • Space-Efficient Data Structures for String Searching and Retrieval
    Louisiana State University LSU Digital Commons LSU Doctoral Dissertations Graduate School 2014 Space-efficient data structures for string searching and retrieval Sharma Valliyil Thankachan Louisiana State University and Agricultural and Mechanical College, [email protected] Follow this and additional works at: https://digitalcommons.lsu.edu/gradschool_dissertations Part of the Computer Sciences Commons Recommended Citation Valliyil Thankachan, Sharma, "Space-efficient data structures for string searching and retrieval" (2014). LSU Doctoral Dissertations. 2848. https://digitalcommons.lsu.edu/gradschool_dissertations/2848 This Dissertation is brought to you for free and open access by the Graduate School at LSU Digital Commons. It has been accepted for inclusion in LSU Doctoral Dissertations by an authorized graduate school editor of LSU Digital Commons. For more information, please [email protected]. SPACE-EFFICIENT DATA STRUCTURES FOR STRING SEARCHING AND RETRIEVAL A Dissertation Submitted to the Graduate Faculty of the Louisiana State University and Agricultural and Mechanical College in partial fulfillment of the requirements for the degree of Doctor of Philosophy in The Department of Computer Science by Sharma Valliyil Thankachan B.Tech., National Institute of Technology Calicut, 2006 May 2014 Dedicated to the memory of my Grandfather. ii Acknowledgments This thesis would not have been possible without the time and effort of few people, who believed in me, instilled in me the courage to move forward and lent a supportive shoulder in times of uncertainty. I would like to express here how much it all meant to me. First and foremost, I would like to extend my sincerest gratitude to my advisor Dr. Rahul Shah. I would have hardly imagined that a fortuitous encounter in an online puzzle forum with Rahul, would set the stage for a career in theoretical computer science.
    [Show full text]
  • Succinct Permutation Graphs and Graph Isomorphism [5]
    Succinct Permutation Graphs Konstantinos Tsakalidis∗ Sebastian Wild† Viktor Zamaraev‡ October 9, 2020 Abstract We present a succinct, i.e., asymptotically space-optimal, data structure for permuta- tion graphs that supports distance, adjacency, neighborhood and shortest-path queries in optimal time; a variant of our data structure also supports degree queries in time indepen- dent of the neighborhood’s size at the expense of an O(log n/ log log n)-factor overhead in all running times. We show how to generalize our data structure to the class of circular permutation graphs with asymptotically no extra space, while supporting the same queries in optimal time. Furthermore, we develop a similar compact data structure for the special case of bipartite permutation graphs and conjecture that it is succinct for this class. We demonstrate how to execute algorithms directly over our succinct representations for sev- eral combinatorial problems on permutation graphs: Clique, Coloring, Independent Set, Hamiltonian Cycle, All-Pair Shortest Paths, and others. Moreover, we initiate the study of semi-local graph representations; a concept that “in- terpolates” between local labeling schemes and standard “centralized” data structures. We show how to turn some of our data structures into semi-local representations by storing only O(n) bits of additional global information, beating the lower bound on distance labeling schemes for permutation graphs. 1 Introduction As a result of the rapid growth of electronic data sets, memory requirements become a bottle- neck in many applications, as performance usually drops dramatically as soon as data structures do no longer fit into faster levels of the memory hierarchy in computer systems.
    [Show full text]
  • Advanced Rank/Select Data Structures: Succinctness, Bounds, and Applications
    Universita` degli Studi di Pisa Dipartimento di Informatica Dottorato di Ricerca in Informatica Settore Scientifico Disciplinare: INF/01 Ph.D. Thesis Advanced rank/select data structures: succinctness, bounds, and applications Alessio Orlandi Supervisor Roberto Grossi Referee Referee Referee Veli M¨akinen Gonzalo Navarro Kunihiko Sadakane December 19, 2012 Acknowledgements This thesis is the end of a long, 4 year journey full of wonders, obstacles, aspirations and setbacks. From the person who cautiously set foot in Pisa to the guy who left his own country, none of these steps would have been possible without the right people at the right moment, supporting me through the dark moments of my life and being with me enjoying the bright ones. First of all, I need to thank Prof. Roberto Grossi, who decided that he could accept me as a Ph.D. Student. Little he knew of what was ahead :). Still, he bailed me out of trouble multiple times, provided invaluable support and insight, helping me grow both as a person and as a scientist. I also owe a lot to Prof. Rajeev Raman, who has been a valuable, patient and welcoming co-author. I also want to thank Proff. Sebastiano Vigna and Paolo Boldi, who nurtured my passion for computer science and introduced me to their branch of research. Also Prof. Degano, as head of the Ph.D. course, provided me with non-trivial help during these years. In Pisa, I met a lot of interesting people who were crucial for my happiness, was it when studying on a Sunday, on a boring the-proof-does-not-work-again day, in moments of real need, or on a Saturday night.
    [Show full text]
  • MTAT.03.238 Succinct Trees
    Algorithmics (6EAP) MTAT.03.238 Succinct Trees Jaak Vilo Thanks to S. Srinivasa Rao 2020 Jaak Vilo 1 succinct Suppose that is the information-theoretical optimal number of bits needed to store some data. A representation of this data is called • implicit if it takes bits of space, • succinct if it takes bits of space, and • compact if it takes bits of space. ! http://en.wikipedia.org/wiki/Succinct_data_structure • In computer science, a succinct data structure is a data structure which uses an amount of space that is "close" to the information-theoretic lower bound, but (unlike other compressed representations) still allows for efficient query operations. The concept was originally introduced by Jacobson[1] to encode bit vectors, (unlabeled) trees, and planar graphs. Unlike general lossless data compression algorithms, succinct data structures retain the ability to use them in-place, without decompressing them first. A related notion is that of a compressed data structure, in which the size of the data structure depends upon the particular data being represented. Succinct Representations of Trees S. Srinivasa Rao Seoul National University Outline p Succinct data structures n Introduction n Examples p Tree representations n Motivation n Heap-like representation n Jacobson’s representation n Parenthesis representation n Partitioning method n Comparison and Applications p Rank and Select on bit vectors Succinct data structures p Goal: represent the data in close to optimal space, while supporting the operations efficiently. (optimal –– information-theoretic lower bound) Introduced by [Jacobson, FOCS ‘89] p An “extension” of data compression. (Data compression: n Achieve close to optimal space n Queries need not be supported efficiently ) Applications p Potential applications where n memory is limited: small memory devices like PDAs, mobile phones etc.
    [Show full text]
  • Hypersuccinct Trees – New Universal Tree Source Codes for Optimal Compressed Tree Data Structures and Range Minima
    Hypersuccinct Trees – New universal tree source codes for optimal compressed tree data structures and range minima J. Ian Munro∗ Patrick K. Nicholson† Louisa Seelbach Benkner‡ Sebastian Wild§ September 6, 2021 Abstract We present a new universal source code for distributions of unlabeled binary and ordinal trees that achieves optimal compression to within lower order terms for all tree sources covered by existing universal codes. At the same time, it supports answering many navigational queries on the compressed representation in constant time on the word-RAM; this is not known to be possible for any existing tree compression method. The resulting data structures, “hypersuccinct trees”, hence combine the compression achieved by the best known universal codes with the operation support of the best succinct tree data structures. We apply hypersuccinct trees to obtain a universal compressed data structure for range- minimum queries. It has constant query time and the optimal worst-case space usage of 2n + o(n) bits, but the space drops to 1.736n + o(n) bits on average for random permutations of n n elements, and 2 lg r + o(n) for arrays with r increasing runs, respectively. Both results are optimal; the former answers an open problem of Davoodi et al. (2014) and Golin et al. (2016). Compared to prior work on succinct data structures, we do not have to tailor our data structure to specific applications; hypersuccinct trees automatically adapt to the trees at hand. We show that they simultaneously achieve the optimal space usage to within lower order terms for a wide range of distributions over tree shapes, including: binary search trees (BSTs) generated by insertions in random order / Cartesian trees of random arrays, random fringe-balanced BSTs, binary trees with a given number of binary/unary/leaf nodes, random binary tries generated from memoryless sources, full binary trees, unary paths, as well as uniformly chosen weight-balanced BSTs, AVL trees, and left-leaning red-black trees.
    [Show full text]
  • Statistical Encoding of Succinct Data Structures
    Statistical Encoding of Succinct Data Structures Rodrigo Gonz´alez2 ⋆ and Gonzalo Navarro1,2 ⋆⋆ 1 Center for Web Research 2 Department of Computer Science, University of Chile. {rgonzale,gnavarro}@dcc.uchile.cl Abstract. In recent work, Sadakane and Grossi [SODA 2006] introduced a scheme to represent any n sequence S = s1s2 ...sn, over an alphabet of size σ, using nHk(S)+ O( (k log σ + log log n)) bits logσ n of space, where Hk(S) is the k-th order empirical entropy of S. The representation permits extracting any substring of size Θ(logσ n) in constant time, and thus it completely replaces S under the RAM model. This is extremely important because it permits converting any succinct data structure requiring o(|S|) = o(n log σ) bits in addition to S, into another requiring nHk(S)+ o(n log σ) (overall) for any k = o(logσ n). They achieve this result by using Ziv-Lempel compression, and conjecture that the result can in particular be useful to implement compressed full-text indexes. In this paper we extend their result, by obtaining the same space and time complexities using a simpler scheme based on statistical encoding. We show that the scheme supports appending symbols in constant amortized time. In addition, we prove some results on the applicability of the scheme for full-text self- indexing. 1 Introduction Recent years have witnessed an increasing interest on succinct data structures, motivated mainly by the growth over time on the size of textual information. This has triggered a search for less space-demanding data structures bounded by the entropy of the original text.
    [Show full text]
  • Lower Bound for Succinct Range Minimum Query
    Lower Bound for Succinct Range Minimum Query Mingmou Liu∗ Huacheng Yuy April 14, 2020 Abstract Given an integer array A[1::n], the Range Minimum Query problem (RMQ) asks to pre- process A into a data structure, supporting RMQ queries: given a; b 2 [1; n], return the index i 2 [a; b] that minimizes A[i], i.e., arg mini2[a;b] A[i]. This problem has a classic solution using O(n) space and O(1) query time by Gabow, Bentley, Tarjan [GBT84] and Harel, Tarjan [HT84]. The best known data structure by Fischer, Heun [FH11] and Navarro, Sadakane [NS14] uses log n t ~ 3=4 2n + n=( t ) + O(n ) bits and answers queries in O(t) time, assuming the word-size is w = Θ(log n). In particular, it uses 2n + n=poly log n bits of space when the query time is a constant. In this paper, we prove the first lower bound for this problem, showing that 2n+n=poly log n space is necessary for constant query time. In general, we show that if the data structure has 2 query time O(t), then it must use at least 2n + n=(log n)O~(t ) space, in the cell-probe model with word-size w = Θ(log n). arXiv:2004.05738v1 [cs.DS] 13 Apr 2020 ∗Nanjing University. [email protected]. Supported by National Key R&D Program of China 2018YFB1003202 and the National Science Foundation of China under Grant Nos. 61722207 and 61672275. Part of the research was done when Mingmou Liu was visiting the Harvard University.
    [Show full text]
  • A Review on Succinct Dynamic Data Structure
    International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 02 Issue: 08 | Nov-2015 www.irjet.net p-ISSN: 2395-0072 A Review on Succinct Dynamic Data Structure Pradhumn Soni1, Mani Butwall2 1 Assistant Professor, Computer Science and Engineering, Mandsaur Institute of Technology, Madhya Pradesh, India 2 Assistant Professor, Computer Science and Engineering, Mandsaur Institute of Technology ,Madhya Pradesh, India ---------------------------------------------------------------------***--------------------------------------------------------------------- 1. Abstract:- Over the years, computer scientists have studied many different data structures for representing and manipulating data within computer main and secondary memory. Ways of representing and efficiently manipulating data are central to many computer programs. These investigations have shown, both in practice and in theory, that the choice of data structures often has a considerable effect on algorithmic performance. This review paper addresses this issue by analyzing space efficient geometric data structures. The study also represent the detail description of succinct data structure which can be useful for space and time efficient applications. 2.Keywords:- Rank,Select,bit vectors. 3.Introduction:- A succinct data structure is a data structure which uses an amount of space that is "close" to the information-theoretic lower bound, but (unlike other compressed representations) still allows for efficient query operations. The concept was originally introduced by Jacobson to encode bit vectors, (unlabeled) trees, and planar graphs. Suppose that is the information-theoretical optimal number of bits needed to store some data. A representation of this data is called: succinct if it takes bits of space For example, a data structure that uses bits of storage is compact, bits is succinct.[8].
    [Show full text]
  • Arxiv:0812.2775V3
    Optimal Succinctness for Range Minimum Queries Johannes Fischer Universit¨at T¨ubingen, Center for Bioinformatics (ZBIT), Sand 14, 72076 T¨ubingen [email protected] Abstract. For a static array A of n totally ordered objects, a range minimum query asks for the position of the minimum between two specified array indices. We show how to preprocess A into a scheme of size 2n + o(n) bits that allows to answer range minimum queries on A in constant time. This space is asymptotically optimal in the important setting where access to A is not permitted after the preprocessing step. Our scheme can be computed in linear time, using only n + o(n) additional bits for construction. We also improve on LCA-computation in BPS- or DFUDS-encoded trees. 1 Introduction For an array A[1,n] of n natural numbers or other objects from a totally ordered universe, a range minimum query rmq (i, j) for i j returns the position of a minimum element in the sub-array A ≤ A[i, j]; i.e., rmq (i, j) = argmin A[k] . This fundamental algorithmic problem has numerous A i≤k≤j{ } applications, e.g., in text indexing [1, 15, 36], text compression [7], document retrieval [31, 37, 42], flowgraphs [19], range queries [40], position-restricted pattern matching [8], just to mention a few. In all of these applications, the array A in which the range minimum queries (RMQs) are performed is static and known in advance, which is also the scenario considered in this article. In this case it makes sense to preprocess A into a (preprocessing-) scheme such that future RMQs can be answered quickly.
    [Show full text]
  • Space-Efficient Data Structures for Collectionsof Textual Data
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Electronic Thesis and Dissertation Archive - Università di Pisa UNIVERSITÀ DEGLI STUDI DI PISA DIPARTIMENTO DI INFORMATICA DOTTORATO DI RICERCA IN INFORMATICA SETTORE SCIENTIFICO DISCIPLINARE INF/01 PH.D. THESIS SPACE-EFFICIENT DATA STRUCTURES FOR COLLECTIONS OF TEXTUAL DATA Giuseppe Ottaviano SUPERVISOR Prof. Roberto Grossi REFEREE Prof. Rajeev Raman REFEREE Prof. Jeffrey Scott Vitter 7 JUNE 2013 ABSTRACT This thesis focuses on the design of succinct and compressed data structures for collections of string-based data, specifically sequences of semi-structured documents in textual format, sets of strings, and sequences of strings. The study of such collections is motivated by a large number of applications both in theory and practice. For textual semi-structured data, we introduce the concept of semi-index, a succinct construction that speeds up the access to documents encoded with textual semi-structured formats, such as JSON and XML, by storing separately a compact description of their parse trees, hence avoiding the need to re-parse the documents every time they are read. For string dictionaries, we describe a data structure based on a path decomposition of the compacted trie built on the string set. The tree topology is encoded using succinct data structures, while the node labels are compressed using a simple dictionary-based scheme. We also describe a variant of the path-decomposed trie for scored string sets, where each string has a score. This data structure can support efficiently top-k completion queries, that is, given a string p and an integer k, return the k highest scored strings among those prefixed by p.
    [Show full text]
  • Storage and Retrieval of Individual Genomes
    Storage and Retrieval of Individual Genomes Veli M¨akinen1⋆, Gonzalo Navarro2⋆⋆, Jouni Sir´en1⋆ ⋆ ⋆, and Niko V¨alim¨aki1† 1 Department of Computer Science, University of Helsinki, Finland. {vmakinen,jltsiren,nvalimak}@cs.helsinki.fi 2 Department of Computer Science, University of Chile, Chile. [email protected] Abstract. A repetitive sequence collection is one where portions of a base sequence of length n are repeated many times with small variations, forming a collection of total length N. Examples of such collections are version control data and genome sequences of individuals, where the differences can be expressed by lists of basic edit operations. Flexible and efficient data analysis on a such typically huge collection is plausible using suffix trees. However, suffix tree occupies O(N log N) bits, which very soon inhibits in-memory analyses. Recent advances in full-text self- indexing reduce the space of suffix tree to O(N log σ) bits, where σ is the alphabet size. In practice, the space reduction is more than 10-fold, for example on suffix tree of Human Genome. However, this reduction factor remains constant when more sequences are added to the collection. We develop a new family of self-indexes suited for the repetitive sequence collection setting. Their expected space requirement depends only on the length n of the base sequence and the number s of variations in its repeated copies. That is, the space reduction factor is no longer constant, but depends on N/n. We believe the structures developed in this work will provide a fun- damental basis for storage and retrieval of individual genomes as they become available due to rapid progress in the sequencing technologies.
    [Show full text]
  • Succinct Data Structures and Big Data
    International Journal of Applied Engineering Research ISSN 0973-4562 Volume 13, Number 15 (2018) pp. 12277-12282 © Research India Publications. http://www.ripublication.com Succinct Data Structures and Big Data Vinesh kumar1, Dr Udai Shankar 2, Dr. Rakesh Kumar3 1Research Scholar, S.V. S University, Meerut, Assistant Professor,GLAU Mathura(UP), India. 2 Professor, S.V. S University, Meerut(UP), India. 3 Professor and Head,CSE Department MMM University of Technology, Gorakhpur(UP), India. Abstract spatial data. The message is demonstrated equally a chain of source symbols푥 푥 … . , 푥 . Coding procedure of message The growth of cloud data stores and cloud computing has 1 2 푛 contains making use of cipher to every sign in message and been expediter and predecessor to appearance of big data. A concatenating all of codeword resultant. Output of encoding is Cloud computing is co-modification of data storage and series of target symbols퐶(푥 )퐶(푥 ) … . , 퐶(푥 ) [2]. calculating time by means of identical technologies. It has 1 2 푛 Decrypting technique is opposite method that acquires source important benefits over conventional physical deployments. symbol consistent to every code word to reconstruct real note. Nevertheless, cloud platforms originated in numerous forms Compressed data is a universal trouble in computer science. and from time to time it is combined with traditional Solidity method is utilized approximately universally to architectures. In proposed work, the main data structure used permit efficient storage and management of big datasets [3]. in big data is tree. Quad tree is used Graphics and Spatial data Large quantity of data (in kind of text, image, video, and so in main memory.
    [Show full text]