DOCSLIB.ORG

Stringology Ewels of Stringology This Page Is Intentionally Left Blank .^ X

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Stringology Ewels of Stringology This Page Is Intentionally Left Blank .^ X ; Maxime Crochemore Wojciech Rytter ewels of. Stringology ewels of Stringology This page is intentionally left blank .^ X ^,,.-^""'''"•'"' •• •>•» \ ( 1 :i y „.,.,-<•-•• Maxime Crochemore -• r: Universite Marne-la- Vallee, France ^ Wojciech Rytter Warsaw University, Poland & University of Liverpool, UK ewels °f. Stringoh World Scientific New Jersey * London • Singapore • Hong Kong Published by World Scientific Publishing Co. Pte. Ltd. P O Box 128, Fairer Road, Singapore 912805 USA office: Suite IB, 1060 Main Street, River Edge, NJ 07661 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. JEWELS OF STRINGOLOGY Copyright © 2002 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher. For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. ISBN 981-02-4782-6 This book is printed on acid-free paper. Printed in Singapore by Mainland Press Preface The term stringology is a popular nickname for string algorithms as well as for text algorithms. Usually text and string have the same meaning. More formally, a text is a sequence of symbols. Text is one of the basic data types to carry information. This book is a collection of the most beautiful and at the same time very classical algorithms on strings. The selection has been done by the authors, and is rather personal, among so many famous algorithms that were natural candidates to be included and that belong to a field that has become now fairly popular. One can partition algorithmic problems discussed in this book into practical and theoretical problems. Certainly string matching and data compression are in the first class, while most problems related to symmetries and repetitions are in the second. However, we believe that all the problems are interesting from an algorithmic point of view and enable the reader to appreciate the importance of combinatorics on words. In most textbooks on algorithms and data structures the presentation of ef­ ficient algorithms on words is quite short as compared to issues in graph theory, sorting, searching, and some other areas. At the same time, there are many presentations of interesting algorithms on words accessible only in journals and in a form directed mainly at specialists. There are still not many books on text algorithms, especially the books which are oriented toward undergraduate and graduate students. In the book the difficult parts are indicated by a star, so the basic text becomes painless for undergraduate students. We hope that this book will cover a gap on algorithms on words in book literature for the broader audience, and bring together the many results presently dispersed in the masses of journal articles. March 2002 M. Crochemore, W. Rytter v This page is intentionally left blank Contents Preface v 1 Stringology 1 1.1 Text file facilities 2 1.2 Dictionaries 5 1.3 Data compression 6 1.4 Applications of text algorithms in genetics 7 1.5 Efficiency of algorithms 8 1.6 Some notation and formal definitions 10 1.7 Some simple combinatorics of strings 11 1.8 Some other interesting strings 14 1.9 Cyclic shifts and primitive words 16 Bibliographic notes 17 2 Basic string searching algorithms 19 2.1 Knuth-Morris-Pratt algorithm 20 2.2 Boyer-Moore algorithm and its variations 26 Bibliographic notes 31 3 Preprocessing for basic searchings 33 3.1 Preprocessing patterns for MP and KMP algorithms 33 3.2 Table of prefixes 36 3.3 Preprocessing for Boyer-Moore algorithm 39 3.4 * Analysis of Boyer-Moore algorithm 41 Bibliographic notes 44 4 On-line construction of suffix trees 45 vn viii CONTENTS 4.1 Tries and their compact versions 45 4.2 Prelude to Ukkonen algorithm 49 4.3 Ukkonen algorithm 51 Bibliographic notes 53 5 More on suffix trees 59 5.1 Several applications of suffix trees 59 5.2 McCreight algorithm 63 Bibliographic notes 68 6 Subword graphs 69 6.1 Directed acyclic graph 69 6.2 On-line construction of subword graphs 73 6.3 The reverse perspective 79 6.4 Compact subword graphs 82 Bibliographic notes 84 7 Text algorithms related to sorting 85 7.1 The naming technique: KMR algorithm 85 7.2 Two-dimensional KMR algorithm 90 7.3 Suffix arrays 91 7.4 Constructing suffix trees by sorting 95 7.5 The Lowest-Common-Ancestor dictionary 101 7.6 Suffix-Merge-Sort 103 Bibliographic notes 107 8 Symmetries and repetitions in texts 111 8.1 Searching for symmetric words Ill 8.2 Compositions of symmetric words 114 8.3 Searching for square factors 119 Bibliographic notes 123 9 Constant-space searchings 125 9.1 Constant-space matching for easy patterns 125 9.2 MaxSuffix-Matching 127 9.3 Computation of maximal suffixes 129 CONTENTS ix 9.4 Matching patterns with short maximal suffixes 131 9.5 Two-way matching and magic decomposition 133 9.6 Sequential sampling for unordered alphabets 136 9.7 Galil-Seiferas algorithm 138 9.8 Cyclic equality of words 139 Bibliographic notes 140 10 Text compression techniques 141 10.1 Substitutions 142 10.2 Static Huffman coding 145 10.3 Dynamic Huffman coding 151 10.4 Factor encoding 154 Bibliographic notes 161 11 Automata-theoretic approach 163 11.1 Aho-Corasick automaton 164 11.2 Determinizing automata 173 11.3 Two-way pushdown automata 176 Bibliographic notes 181 12 Approximate pattern matching 183 12.1 Edit distance 183 12.2 Longest common subsequence problem 186 12.3 String matching with errors 191 12.4 String matching with don't care symbols 194 Bibliographic notes 196 13 Matching by dueling and sampling 199 13.1 String matching by duels 199 13.2 String matching by sampling 204 Bibliographic notes 207 14 Two-dimensional pattern matching 209 14.1 Multi-pattern approach 211 14.2 Don't cares and non-rectangular patterns 212 14.3 2D-Pattern matching with mismatches 214 x CONTENTS 14.4 Multi-pattern matching 215 14.5 Matching by sampling 218 14.6 An algorithm fast on the average 221 Bibliographic notes 222 15 Two-dimensional periodicities 225 15.1 Amir-Benson-Farach algorithm 225 15.2 Geometry of two-dimensional periodicities 235 15.3 * Patterns with large monochromatic centers 242 15.4 * A version of the Galil-Park algorithm 244 Bibliographic notes 247 16 Parallel text algorithms 249 16.1 The abstract model of parallel computing 249 16.2 Parallel string-matching algorithms 252 16.3 * Splitting technique 255 16.4 Parallel KMR algorithm and application 258 16.5 Parallel Huffman coding 263 16.6 Edit distance — efficient parallel computation 268 Bibliographic notes 269 17 Miscellaneous 271 17.1 Karp-Rabin string matching by hashing 271 17.2 Shortest common superstrings 274 17.3 Unique-decipherability problem 276 17.4 Parameterized pattern matching 278 17.5 Breaking paragraphs into lines 281 Bibliographic notes 284 Bibliography 285 Index 305 Chapter 1 Stringology One of the simplest and natural types of information representation is by means of written texts. This type of data is characterized by the fact that it can be written down as a long sequence of characters. Such linear a sequence is called a text. The texts are central in "word processing" systems, which provide facilities for the manipulation of texts. Such systems usually process objects that are quite large. For example, this book probably contains more than a million characters. Text algorithms occur in many areas of science and information processing. Many text editors and programming languages have facilities for processing texts. In biology, text algorithms arise in the study of molecular sequences. The complexity of text algorithms is also one of the central and most studied problems in theoretical computer science. It could be said that it is the domain in which practice and theory are very close to each other. The basic textual problem in stringology is called pattern matching. It is used to access information and, no doubt, at this moment many computers are solving this problem as a frequently used operation in some application system. Pattern matching is comparable in this sense to sorting, or to basic arithmetic operations. Consider the problem of a reader of the French dictionary "Grand Larousse," who wants all entries related to the name "Marie-Curie-Sklodowska." This is an example of a pattern matching problem, or string matching. In this case, the name "Marie-Curie-Sklodowska" is the pattern. Generally we may want to find a string called a pattern of length m inside a text of length n, where n is greater than m. The pattern can be described in a more complex way to denote a set of strings and not just a single word. In many cases n is very large. In genetics the pattern can correspond to a gene that can be very long; in image 1 2 CHAPTER 1. STRINGOLOGY processing, digitized images sent serially contain millions of characters each. The string-matching problem is the basic question considered in this book, together with its variations. String matching is also the basic subproblem in other algorithmic problems on texts. Following is a (not exclusive) list of basic groups of problems discussed in this book: • variations on the string-matching problem • problem related to the structures of the segments of a text • data compression • approximation problems • finding regularities • extensions to two-dimensional images • extensions to trees • optimal time-space implementations • optimal parallel implementations.
Load more

Recommended publications
  • The Book Review Column1 by Frederic Green Department of Mathematics and Computer Science Clark University Worcester, MA 02465 Email: [email protected]
    The Book Review Column1 by Frederic Green Department of Mathematics and Computer Science Clark University Worcester, MA 02465 email: [email protected] In this column, the following books are reviewed: 1. Games and Mathematics: Subtle Connections, by David Wells. Reviewed by S. C. Coutinho. This is a (mostly non-technical) book that explores the mathematical nature of games. In addition, and more extensively, it discusses the game-like nature of mathematics through a variety of mathematical examples. 2. Jewels of Stringology, by Maxime Crochemore and Wojciech Rytter. Reviewed by Shoshana Marcus. This is a book about string algorithms, with an emphasis on those that are fundamental and elegant: i.e., “algorithmic jewels.” Written at the advanced undergraduate/beginning graduate level. 3. Algorithms on Strings, by Maxime Crochemore, Christophe Hancart and Thierry Lecroq. Reviewed by Matthias Galle.´ Another book on stringology, aimed more at the graduate level. Although there is some overlap with “Jewels” (e.g., algorithms and data structures for pattern matching), the two books have a substantial symmetric difference. 4. Polyhedral and Algebraic Methods in Computational Geometry, by Michael Joswig and Thorsten Theobald. Reviewed by Brittany Terese Fasy and David L. Millman. This book treats a wide array of topics, beginning from the basics (e.g., convex hulls) and proceeding through the fundamental tools of computational algebraic geometry (e.g., Grobner¨ bases), and containing a section on applications as well. 5. A Mathematical Orchard - Problems and Solutions, by Mark Krusemeyer, George Gilbert, and Loren Larson. Reviewed by S.V. Nagaraj. This is a collection of challenging mathematical problems, published by the MAA.
    [Show full text]
  • Text Searching
    Text Searching Thierry Lecroq [email protected] Laboratoire d’Informatique, du Traitement de l’Information et des Syst`emes. International PhD School in Formal Languages and Applications Tarragona, November 20th and 21st, 2006 Introduction Left-to-right scan — shift Right-to-left scan — shift Plan 1 Introduction 2 Left-to-right scan — shift 3 Right-to-left scan — shift Thierry Lecroq Text Searching 2/77 Introduction Left-to-right scan — shift Right-to-left scan — shift Plan 1 Introduction 2 Left-to-right scan — shift 3 Right-to-left scan — shift Thierry Lecroq Text Searching 3/77 Introduction Left-to-right scan — shift Right-to-left scan — shift Pattern matching text pattern - position of occurrence Problem Find all the occurrences of pattern x of length m inside the text y of length n Two types of solutions Fixed pattern preprocessing time O(m) Use of combinatorial properties searching time O(n) Static text preprocessing time O(n) Solutions based on indexes searching time O(m) Thierry Lecroq Text Searching 4/77 Introduction Left-to-right scan — shift Right-to-left scan — shift Searching for a fixed string String matching given a pattern x, find all its locations in any text y Pattern a string x of symbols, of length m t a t a Text a string y of symbols, of length n c acgtatatatgcgttataat Thierry Lecroq Text Searching 5/77 Introduction Left-to-right scan — shift Right-to-left scan — shift String matching Occurrences c acgtatatatgcgttataat t a t a t a t a t a t a Basic operation symbol comparison (=, 6=) Thierry Lecroq Text Searching 6/77 Introduction Left-to-right scan — shift Right-to-left scan — shift Interest Practical basic problem for search for various patterns lexical analysis approximate string matching comparisons of strings—alignments ..
    [Show full text]
  • {PDF} Jewels of Stringology: Text Algorithms
    JEWELS OF STRINGOLOGY: TEXT ALGORITHMS PDF, EPUB, EBOOK Maxime Crochemore, Wojciech Rytter | 320 pages | 01 Jan 2003 | World Scientific Publishing Co Pte Ltd | 9789810248970 | English | Singapore, Singapore Jewels Of Stringology: Text Algorithms PDF Book DPReview Digital Photography. Tell the Publisher! Matching by dueling and sampling. Top reviews from India. If you like books and love to build cool products, we may be looking for you. If you expect that there will be ready to use code, look for O'Reilly catalog. Be the first to ask a question about Jewels of Stringology. Introduction to equations on words. Read more Read less. We are all precious Jewels of God and we are to be light and salt in a darkened world. Audible Download Audio Books. Wojciech Rytter. Certainly, string matching and data compression are in the former class, while most problems related to symmetries and repetitions in texts are in the latter. Warning: this is not to "copy and paste" book. See all reviews. Gilbert Strang. This book deals with the most basic algorithms in the area. Myers' algorithm for edit distance algorithm used in 'diff' program from UNIX. Namespaces Article Talk. Presents algorithms that are very hard to find elsewhere. Cambridge: Cambridge University Press, To ask other readers questions about Jewels of Stringology , please sign up. This book fills the gap in the book literature on algorithms on words, and brings together the many results presently dispersed in the masses of journal articles. From Wikipedia, the free encyclopedia. Thanks for telling us about the problem. Verified Purchase. View 14 excerpts, cites methods.
    [Show full text]
  • IMPROVED ALGORITHMS for STRING SEARCHING PROBLEMS Doctoral Dissertation
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Aaltodoc TKK Research Reports in Computer Science and Engineering A TKK-CSE-A1/09 Espoo 2009 IMPROVED ALGORITHMS FOR STRING SEARCHING PROBLEMS Doctoral Dissertation Leena Salmela Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Faculty of Information and Natural Sciences for public examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo, Finland) on the 1st of June, 2009, at 12 noon. Helsinki University of Technology Faculty of Information and Natural Sciences Department of Computer Science and Engineering Teknillinen korkeakoulu Informaatio- ja luonnontieteiden tiedekunta Tietotekniikan laitos Distribution: Helsinki University of Technology Faculty of Information and Natural Sciences Department of Computer Science and Engineering P.O. Box 5400 FI-02015 TKK FINLAND URL: http://www.cse.tkk.fi/ Tel. +358 9 451 3228 Fax +358 9 451 3293 e-mail: [email protected].fi c Leena Salmela c Cover photo: Teemu J. Takanen ISBN 978-951-22-9887-7 ISBN 978-951-22-9888-4 (PDF) ISSN 1797-6928 ISSN 1797-6936 (PDF) URL: http://lib.tkk.fi/Diss/2009/isbn9789512298884/ Multiprint Oy Espoo 2009 AB ABSTRACT OF DOCTORAL DISSERTATION HELSINKI UNIVERSITY OF TECHNOLOGY P. O. BOX 1000, FI-02015 TKK http://www.tkk.fi/ Author Leena Salmela Name of the dissertation Improved Algorithms for String Searching Problems Manuscript submitted 09.02.2009 Manuscript revised 11.05.2009 Date of the defence 01.06.2009 X Monograph Article dissertation (summary + original articles) Faculty Faculty of Information and Natural Sciences Department Department of Computer Science and Engineering Field of research Software Systems Opponent(s) Prof.
    [Show full text]
  • Faster Recovery of Approximate Periods Over Edit Distance
    Faster Recovery of Approximate Periods over Edit Distance Tomasz Kociumaka, Jakub Radoszewski,∗ Wojciech Rytter,y Juliusz Straszy´nski,∗ Tomasz Wale´n,and Wiktor Zuba Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Warsaw, Poland [kociumaka,jrad,rytter,jks,walen,w.zuba]@mimuw.edu.pl Abstract The approximate period recovery problem asks to compute all approximate word-periods of a given word S of length n: all primitive words P (jP j = p) which have a periodic extension at edit n distance smaller than τp from S, where τp = b (3:75+)·p c for some > 0. Here, the set of periodic extensions of P consists of all finite prefixes of P 1. We improve the time complexity of the fastest known algorithm for this problem of Amir et al. [Theor. Comput. Sci., 2018] from O(n4=3) to O(n log n). Our tool is a fast algorithm for Approximate Pattern Matching in Periodic Text. We consider only verification for the period recovery problem when the candidate approximate word-period P is explicitly given up to cyclic rotation; the algorithm of Amir et al. reduces the general problem in O(n) time to a logarithmic number of such more specific instances. 1 Introduction The aim of this work is computing periods of words in the approximate pattern matching model (see e.g. [8, 10]). This task can be stated as the approximate period recovery (APR) problem that was defined by Amir et al. [2]. In this problem, we are given a word; we suspect that it was initially periodic, but then errors might have been introduced in it.
    [Show full text]
  • Efficient Data Structures for Internal Queries in Texts
    University of Warsaw Faculty of Mathematics, Informatics and Mechanics Tomasz Kociumaka Efficient Data Structures for Internal Queries in Texts PhD dissertation Supervisor prof. dr hab. Wojciech Rytter Institute of Informatics University of Warsaw October 2018 Author’s declaration: I hereby declare that this dissertation is my own work. October 26, 2018 . Tomasz Kociumaka Supervisor’s declaration: The dissertation is ready to be reviewed October 26, 2018 . prof. dr hab. Wojciech Rytter i Abstract This thesis is devoted to internal queries in texts, which ask to solve classic text-processing problems for substrings of a given text. More precisely, the task is to preprocess a static string T of length n (called the text) and construct a data structure answering certain questions about the substrings of T . The substrings involved in each query are specified in constant space by their occurrences in T , called fragments of T , identified by the start and the end positions. Components for internal queries often become parts of more complex data structures, and they are used in many algorithms for text processing. Longest Common Extension Queries, asking for the length of the longest common prefix of two substrings of the text T , are by far the most popular internal queries. They are used for checking if two fragments match (represent the same string) and for lexicographic comparison of substrings. Due to an optimal solution in the standard setting of texts over polynomially-bounded integer alphabets, with O(1)-time queries, O(n) size, and O(n) construction time, they have found numerous applications across stringology. In this dissertation, we provide the first optimal data structure for smaller alphabets of size σ n: it handles queries in O(1) time, takes O(n/ logσ n) space, and admits an O(n/ logσ n)-time construction (from the packed representation of T with Θ(logσ n) characters in each machine word).
    [Show full text]
  • Proceedings of the Prague Stringology Conference 2014
    Proceedings of the Prague Stringology Conference 2014 Edited by Jan Holub and Jan Zdˇ '´arek September 2014 PSC Prague Stringology Club http://www.stringology.org/ Conference Organisation Program Committee Amihood Amir (Bar-Ilan University, Israel) Gabriela Andrejkov´a (P. J. Saf´arikUniversity,ˇ Slovakia) Maxime Crochemore (King's College London, United Kingdom) Simone Faro (Universit`adi Catania, Italy) FrantiˇsekFranˇek (McMaster University, Canada) Jan Holub, Co-chair (Czech Technical University in Prague, Czech Republic) Costas S. Iliopoulos (King's College London, United Kingdom) Shunsuke Inenaga (Kyushu University, Japan) Shmuel T. Klein (Bar-Ilan University, Israel) Thierry Lecroq, Co-chair (Universit´ede Rouen, France) Boˇrivoj Melichar, Honorary chair (Czech Technical University in Prague, Czech Republic) Yoan J. Pinz´on (Universidad Nacional de Colombia, Colombia) Marie-France Sagot (INRIA Rh^one-Alpes, France) William F. Smyth (McMaster University, Canada) Bruce W. Watson (FASTAR Group (Stellenbosch University and University of Pretoria, South Africa)) Jan Zdˇ '´arek (Czech Technical University in Prague, Czech Republic) Organising Committee Miroslav Bal´ık, Co-chair Jan Janouˇsek Boˇrivoj Melichar Jan Holub, Co-chair Jan Zdˇ '´arek External Referees J´er´emy Barbay Juan Mendivelso Elise Prieur-Gaston Loek Cleophas Yuto Nakashima Ayumi Shinohara Arnaud Lefebvre iii Preface The proceedings in your hands contains the papers presented in the Prague Stringol- ogy Conference 2014 (PSC 2014) at the Czech Technical University in Prague, which organizes the event. The conference was held on September 1{3, 2014 and it focused on stringology and related topics. Stringology is a discipline concerned with algorith- mic processing of strings and sequences. The papers submitted were reviewed by the program committee.
    [Show full text]