Efficient Multiple Pattern Matching Algorithm Based on BMH: MP-BMH
Total Page:16
File Type:pdf, Size:1020Kb
The International Arab Journal of Information Technology, Vol. 16, No. 6, November 2019 1121 Efficient Multiple Pattern Matching Algorithm Based on BMH: MP-BMH Akhtar Rasool1, Gulfishan Firdose Ahmed2, Raju Barskar3, and Nilay Khare1 1Department of Computer Science and Engineering, Maulana Azad National Institute of Technology, India 2Department of Computer Science, College of Agriculture, India 3Department of Computer Science and Engineering, University Institute of Technology RGPV, India Abstract: String matching is playing a key role in a wide range of applications in the information computing. Due to its importance, large numbers of different algorithms have been developed over the last 50 years. There are various standards of single and multiple pattern string matching algorithms like Boyer-Moore (BM), Boyer-Moore-Horspool (BMH), Aho-Corasick etc. Multiple pattern string matching is very useful in real world applications. Standard benchmark multiple pattern algorithm Aho-Corasick and many other multiple pattern string matching algorithms are not memory and time efficient for wider length and large number of patterns set on large text data set because they are using the concept like DFA and require full scan of text data. Many string matching tools which are currently popular for string matching are based on these algorithms. Using the bad character principle of BMH, a method for multiple pattern string matching is being developed. Using this method a time and memory efficient exact multiple pattern string matching algorithm Multiple Pattern BMH (MP-BMH) is proposed. Unlike Aho-Corasick, the proffered MP-BMH algorithm provides skipping of unnecessary matching of characters in text while searching like BMH Algorithm. It also provides the skipping of characters in patterns using the similarity between patterns. Along with the shifts while searching, this algorithm also provides shrewd switches among the patterns for efficacious matching.In this paper, the aforesaid method, MP-BMH algorithm with its time, memory and experimental analysis are described. Keywords: String matching; multiple pattern string matching, Boyer-Moore BM,BMH, MP-BMH. Received January 8, 2017; accepted January 16, 2018 1. Introduction 2. Related Work String matching is one of the important concepts for There are various popular single pattern string solving problems in computer science. In the multi- matching algorithms such as Boyer-Moore (BM) [45, pattern string matching algorithm [9, 35, 39] all 47], Knutt Morris Pratt (KMP) [49, 50], Rabin occurrences of the desired patterns in the text string are Karp[51], Boyer-Moore-Horspool (BMH) [27, 42, 44, searched. The taxonomy and relative performance 50], BMHS [30, 34], BMHS2 [47], BMI [53], measures of multi-pattern matching algorithms are Improved BMHS [44] and Backward Non- mentioned in [28, 51, 53]. String Matching has many Deterministic Dawg Matching (BNDM) [39] and Two real world applications like intrusion detection system way Non-Deterministic Dawg Matching (TNDM) [49]. [28, 51, 53], plagiarism detection [3], text mining These algorithm are simple but time consuming so techniques [35], digital forensics technique [31] and move to multi-pattern string matching[16, 49] Some of many more. An effective multiple string matching the multiple pattern string matching algorithms algorithm can boost the performance of these subsume Multi-pattern string matching with q- applications as these applications are widely used in grams[29, 50], a fast algorithm for multi-pattern real world. An effective selection of multiple string searching [46], Wu Manber [31, 36, 48],commentz- matching algorithms [29] can improve the efficiency of walter [22] and standard benchmark algorithm Aho- these real world applications. The most important Corasick [11, 12, 13, 15, 17, 18, 19, 20, 21, 22, 23, 24, benefit of multiple string matching algorithms is that 30, 32, 33, 34, 38, 40, 41, 43]. Boyer-Moore-Horspool no additional search structure is needed. Only one pass (BMH) [50] algorithm uses the concept of Bad of searching is sufficient to search multiple patterns or Character [14] of BM algorithm [47] for shifting keywords from a given text. Knowledge of single through text in matching process. In BMH algorithm, pattern string matching could help in better no matter the location of mismatching, the distance of understanding of the multiple pattern string matching shift to right is determined by the character in the text algorithms. string which is aligned to the last character of pattern string. It means here always the Bad Character value 1122 The International Arab Journal of Information Technology, Vol. 16, No. 6, November 2019 [14] of a character of text corresponding to last version is better than original Wu-Manber algorithm character of pattern is calculated. And then the window [46]. Wu-Manber and Quadrupled Wu Manber is shifted right by that value. BMH always provides the (QWM) algorithm is relatively not better than Aho- jump at least one and maximum of pattern length [50]. Corasick and Commentz-Walter algorithm [8] when In [50] they applied the BMH algorithm for multiple binary text is employed rather than increased pattern patterns in a different way using q-grams. They called length and alphabet size. Among all four, QWM their proposed filtering algorithm as Horspool with q- algorithm has better performance. That’s why QWM Grams (HG) algorithm. The HG algorithm performs algorithm is applied in various fields, such as network faster for less than 2000 patterns and works well for content analysis, intrusion detection, and text retrieval less than 10000 patterns. Here full text is scanned systems. when the pattern set contains more than 50000 The grep tools contributed a lot to the field of multi patterns. This algorithm is a filtering method and so is pattern string matching. First implementation of grep not designed for searching texts with large number of was given by Thompson using Nondeterministic Finite matches. Automaton (NFA) for regular expression [37]. Then, In [46] extended the BM algorithm for handling Alfred Aho made egrep (enhanced grep and later multiple patterns. They proposed two new algorithms extended grep) using Deterministic Finite Automaton whose operations involves remembrance of previous (DFA) [3]. Also fgrep (fixed-string grep) tool is based matches. With N as text string length and maximum on Aho-Corasick algorithm [43]. The UNIX grep pattern length D, the first proposed algorithm utility searches the input and outputs the lines which remembers maximum 1+ log4 D previous matches and contain any of the input patterns. The UNIX egrep consults O(N log D) text characters. The second supports additional operators for regular expressions algorithm, being designed for distinct time-space trade- [1]. UNIX fgrep performs like grep but does not off, consults O (kN log D) text characters and recognize any regular expression meta-characters as remembers maximum t/k non periodic matches for any special. So, fgrep is basically a fixed pattern matching given k and t number of patterns. tool, hence performs faster [1]. The original egrep and Aho-Corasick [42] is standard multi-pattern string fgrep could not perform well for more than few matching algorithm [9] which is based on finite hundred patterns [43]. Agrep [44] advances from egrep automata. The complexity of Aho-Corasick (AC) [2] and fgrep as it searches the patterns in given text algorithm is in linear time, which scans on character by approximately, which is based on bitap algorithm. In character basis in the text by creating finite state 1964 Balint Domolki proposed the bitap algorithm for machine from the patterns given at the beginning. By exact string searching [9, 26] and extended by taking basis of that work, Commentz-Walter [48] have Shyamsundar in 1977 [32], this algorithm is reinvented provided almost linear solution by combining Boyer- by Manber and Wu [44] from exact string matching to Moore’s [47] matching with the AC machine. In approximate string matching also known as fuzzy addition to the approaches of automata, researchers string matching based on work done by Baeza- have also considered bit parallelism approach (e.g., Yates and Gonnet [4]. Further improvement was done shift-or algorithm [4], which proved to deliver by Sunday [34] and then for long patterns Gene considerable speed up in case of single pattern string Myers proposed an improved algorithm based on bitap matching but it is also observed that applying bit- algorithm in 1998 [25]. parallelism for multiple pattern string matching is In this paper, multiple pattern string matching tedious task [27]. By utilizing arithmetic and logical method is designed on the basis of BMH algorithm. By operations and assigning numbers to different states, using this method, a multiple pattern string matching Baeza-Yates and Gonnet [4] proposed searching algorithm is proposed. This algorithm provides shifting method, which includes all keywords of the pattern set mechanism for unnecessary matched character. Here when computer word size is not large enough. Kim & MP-BMH algorithm, its time and space complexity Kim [35] proposed an algorithm which performs analysis and comparative analysis with other standard bitwise hashing operations while scanning by encoding tools and algorithm are described. the patterns and text at the beginning of the process. Timo Raita [29] improves the BMH algorithm by 3. Multiple Pattern String Matching Using introducing the dependencies between the characters