Needleman–Wunsch Algorithm
Total Page:16
File Type:pdf, Size:1020Kb

Load more
Recommended publications
-
Logmine: Fast Pattern Recognition for Log Analytics
LogMine: Fast Pattern Recognition for Log Analytics Hossein Hamooni Biplob Debnath Jianwu Xu University of New Mexico NEC Laboratories America NEC Laboratories America 1 University Blvd 4 Independence Way 4 Independence Way Albuquerque, NM 87131 Princeton, NJ 08540 Princeton, NJ 08540 [email protected] [email protected] [email protected] Hui Zhang Guofei Jiang Abdullah Mueen NEC Laboratories America NEC Laboratories America University of New Mexico 4 Independence Way 4 Independence Way 1 University Blvd Princeton, NJ 08540 Princeton, NJ 08540 Albuquerque, NM 87131 [email protected] [email protected] [email protected] ABSTRACT Keywords Modern engineering incorporates smart technologies in all Log analysis; Pattern recognition; Map-reduce aspects of our lives. Smart technologies are generating ter- abytes of log messages every day to report their status. It 1. INTRODUCTION is crucial to analyze these log messages and present usable The Internet of Things (IoT) enables advanced connec- information (e.g. patterns) to administrators, so that they tivity of computing and embedded devices through internet can manage and monitor these technologies. Patterns mini- infrastructure. Although computers and smartphones are mally represent large groups of log messages and enable the the most common devices in IoT, the number of \things" administrators to do further analysis, such as anomaly de- is expected to grow to 50 billion by 2020 [5]. IoT involves tection and event prediction. Although patterns exist com- machine-to-machine communications (M2M), where it is im- monly in automated log messages, recognizing them in mas- portant to continuously monitor connected machines to de- sive set of log messages from heterogeneous sources without tect any anomaly or bug, and resolve them quickly to mini- any prior information is a significant undertaking. -
The Problem of Fuzzy Duplicate Detection of Large Texts
The problem of fuzzy duplicate detection of large texts E V Sharapova1 and R V Sharapov1 1Vladimir State University, Orlovskaya street 23, Murom, Russia, 602264 Abstract. In the paper, considered the problem of fuzzy duplicate detection. There are given the basic approaches to detection of text duplicates–distance between strings, fuzzy search algorithms without indexing data, fuzzy search algorithms with indexing data. Thereview of existing methods for the fuzzy duplicate detection is given. The algorithm of fuzzy duplicate detection is present. The algorithm of fuzzy duplicate texts detection was implemented in the system AVTOR.NET. The paper presents the results of testing of the system.The use of filtering text, stemming and character replacement, allow the algorithm to found duplicates even in minor modified texts. 1. Introduction Today computer technology and the World Wide Web have become part of our lives. Billions of users use the Internet every day. The Internet contains a huge number of documents. Many books are transformed in a digital form. Users can read books, articles, newspapers directly from computers and electronic gadgets. Today, in the Internet there are many studying materials: lectures, textbooks, methodical literature, etc. There are big collections of essays, course and diploma projects and scientific dissertations. It leads to problem un-controlling coping of information. The text of document may be a part of another document, formed from several documents, may be modified part of another document (with a change of words to synonyms, endings, times, etc.). In this way, the document is fuzzy duplicate of other documents. There is the problem of text checking to presence of fussy duplicates. -
Headprint: Detecting Anomalous Communications Through Header-Based Application Fingerprinting
HeadPrint: Detecting Anomalous Communications through Header-based Application Fingerprinting Riccardo Bortolameotti Thijs van Ede Andrea Continella University of Twente University of Twente UC Santa Barbara [email protected] [email protected] [email protected] Thomas Hupperich Maarten H. Everts Reza Rafati University of Muenster University of Twente Bitdefender [email protected] [email protected] [email protected] Willem Jonker Pieter Hartel Andreas Peter University of Twente Delft University of Technology University of Twente [email protected] [email protected] [email protected] ABSTRACT 1 INTRODUCTION Passive application fingerprinting is a technique to detect anoma- Data breaches are a major concern for enterprises across all indus- lous outgoing connections. By monitoring the network traffic, a tries due to the severe financial damage they cause [25]. Although security monitor passively learns the network characteristics of the many enterprises have full visibility of their network traffic (e.g., applications installed on each machine, and uses them to detect the using TLS-proxies) and they stop many cyber attacks by inspect- presence of new applications (e.g., malware infection). ing traffic with different security solutions (e.g., IDS, IPS, Next- In this work, we propose HeadPrint, a novel passive fingerprint- Generation Firewalls), attackers are still capable of successfully ing approach that relies only on two orthogonal network header exfiltrating data. Data exfiltration often occurs over network covert characteristics to distinguish applications, namely the order of the channels, which are communication channels established by attack- headers and their associated values. Our approach automatically ers to hide the transfer of data from a security monitor. -
Author Guidelines for 8
I.J. Information Technology and Computer Science, 2014, 01, 68-75 Published Online December 2013 in MECS (http://www.mecs-press.org/) DOI: 10.5815/ijitcs.2014.01.08 FSM Circuits Design for Approximate String Matching in Hardware Based Network Intrusion Detection Systems Dejan Georgiev Faculty of Electrical Engineering and Information Technologies , Skopje, Macedonia E-mail: [email protected] Aristotel Tentov Faculty of Electrical Engineering and Information Technologies, Skopje, Macedonia E-mail: [email protected] Abstract— In this paper we present a logical circuits to perform a check to a possible variations of the design for approximate content matching implemented content because of so named "evasion" techniques. as finite state machines (FSM). As network speed Software systems have processing limitations in that increases the software based network intrusion regards. On the other hand hardware platforms, like detection and prevention systems (NIDPS) are lagging FPGA offer efficient implementation for high speed behind requirements in throughput of so called deep network traffic. package inspection - the most exhaustive process of Excluding the regular expressions as special case ,the finding a pattern in package payloads. Therefore, there problem of approximate content matching basically is a is a demand for hardware implementation. k-differentiate problem of finding a content Approximate content matching is a special case of content finding and variations detection used by C c1c2.....cm in a string T t1t2.......tn such as the "evasion" techniques. In this research we will enhance distance D(C,X) of content C and all the occurrences of the k-differentiate problem with "ability" to detect a a substring X is less or equal to k , where k m n . -
Comparison of Levenshtein Distance Algorithm and Needleman-Wunsch Distance Algorithm for String Matching
COMPARISON OF LEVENSHTEIN DISTANCE ALGORITHM AND NEEDLEMAN-WUNSCH DISTANCE ALGORITHM FOR STRING MATCHING KHIN MOE MYINT AUNG M.C.Sc. JANUARY2019 COMPARISON OF LEVENSHTEIN DISTANCE ALGORITHM AND NEEDLEMAN-WUNSCH DISTANCE ALGORITHM FOR STRING MATCHING BY KHIN MOE MYINT AUNG B.C.Sc. (Hons:) A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Computer Science (M.C.Sc.) University of Computer Studies, Yangon JANUARY2019 ii ACKNOWLEDGEMENTS I would like to take this opportunity to express my sincere thanks to those who helped me with various aspects of conducting research and writing this thesis. Firstly, I would like to express my gratitude and my sincere thanks to Prof. Dr. Mie Mie Thet Thwin, Rector of the University of Computer Studies, Yangon, for allowing me to develop this thesis. Secondly, my heartfelt thanks and respect go to Dr. Thi Thi Soe Nyunt, Professor and Head of Faculty of Computer Science, University of Computer Studies, Yangon, for her invaluable guidance and administrative support, throughout the development of the thesis. I am deeply thankful to my supervisor, Dr. Ah Nge Htwe, Associate Professor, Faculty of Computer Science, University of Computer Studies, Yangon, for her invaluable guidance, encouragement, superior suggestion and supervision on the accomplishment of this thesis. I would like to thank Daw Ni Ni San, Lecturer, Language Department of the University of Computer Studies, Yangon, for editing my thesis from the language point of view. I thank all my teachers who taught and helped me during the period of studies in the University of Computer Studies, Yangon. Finally, I am grateful for all my teachers who have kindly taught me and my friends from the University of Computer Studies, Yangon, for their advice, their co- operation and help for the completion of thesis. -
Algorithms for Approximate String Matching Petro Protsyk (28 October 2016) Agenda
Algorithms for approximate string matching Petro Protsyk (28 October 2016) Agenda • About ZyLAB • Overview • Algorithms • Brute-force recursive Algorithm • Wagner and Fischer Algorithm • Algorithms based on Automaton • Bitap • Q&A About ZyLAB ZyLAB is a software company headquartered in Amsterdam. In 1983 ZyLAB was the first company providing a full-text search program for MS-DOS called ZyINDEX In 1991 ZyLAB released ZyIMAGE software bundle that included a fuzzy string search algorithm to overcome scanning and OCR errors. Currently ZyLAB develops eDiscovery and Information Governance solutions for On Premises, SaaS and Cloud installations. ZyLAB continues to develop new versions of its proprietary full-text search engine. 3 About ZyLAB • We develop for Windows environments only, including Azure • Software developed in C++, .Net, SQL • Typical clients are from law enforcement, intelligence, fraud investigators, law firms, regulators etc. • FTC (Federal Trade Commission), est. >50 TB (peak 2TB per day) • US White House (all email of presidential administration), est. >20 TB • Dutch National Police, est. >20 TB 4 ZyLAB Search Engine Full-text search engine • Boolean and Proximity search operators • Support for Wildcards, Regular Expressions and Fuzzy search • Support for numeric and date searches • Search in document fields • Near duplicate search 5 ZyLAB Search Engine • Highly parallel indexing and searching • Can index Terabytes of data and millions of documents • Can be distributed • Supports 100+ languages, Unicode characters 6 Overview Approximate string matching or fuzzy search is the technique of finding strings in text or dictionary that match given pattern approximately with respect to chosen edit distance. The edit distance is the number of primitive operations necessary to convert the string into an exact match. -
Coding Theory, Information Theory and Cryptology : Proceedings of the EIDMA Winter Meeting, Veldhoven, December 19-21, 1994
Coding theory, information theory and cryptology : proceedings of the EIDMA winter meeting, Veldhoven, December 19-21, 1994 Citation for published version (APA): Tilborg, van, H. C. A., & Willems, F. M. J. (Eds.) (1994). Coding theory, information theory and cryptology : proceedings of the EIDMA winter meeting, Veldhoven, December 19-21, 1994. Euler Institute of Discrete Mathematics and its Applications. Document status and date: Published: 01/01/1994 Document Version: Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication: • A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal. -
Automating Error Frequency Analysis Via the Phonemic Edit Distance Ratio
JSLHR Research Note Automating Error Frequency Analysis via the Phonemic Edit Distance Ratio Michael Smith,a Kevin T. Cunningham,a and Katarina L. Haleya Purpose: Many communication disorders result in speech corresponds to percent segments with these phonemic sound errors that listeners perceive as phonemic errors. errors. Unfortunately, manual methods for calculating phonemic Results: Convergent validity between the manually calculated error frequency are prohibitively time consuming to use in error frequency and the automated edit distance ratio was large-scale research and busy clinical settings. The purpose excellent, as demonstrated by nearly perfect correlations of this study was to validate an automated analysis based and negligible mean differences. The results were replicated on a string metric—the unweighted Levenshtein edit across 2 speech samples and 2 computation applications. distance—to express phonemic error frequency after left Conclusions: The phonemic edit distance ratio is well hemisphere stroke. suited to estimate phonemic error rate and proves itself Method: Audio-recorded speech samples from 65 speakers for immediate application to research and clinical practice. who repeated single words after a clinician were transcribed It can be calculated from any paired strings of transcription phonetically. By comparing these transcriptions to the symbols and requires no additional steps, algorithms, or target, we calculated the percent segments with a combination judgment concerning alignment between target and production. of phonemic substitutions, additions, and omissions and We recommend it as a valid, simple, and efficient substitute derived the phonemic edit distance ratio, which theoretically for manual calculation of error frequency. n the context of communication disorders that affect difference between a series of targets and a series of pro- speech production, sound error frequency is often ductions. -
Role of Similarity Measures in Time Series Analysis
UNIVERSITY OF NOVI SAD FACULTY OF SCIENCE DEPARTMENT OF MATHEMATICS AND INFORMATICS Zoltan Geler Role of Similarity Measures in Time Series Analysis - Doctoral Dissertation - Mentor: Prof. Dr. Mirjana Ivanović Novi Sad, 2015 Preface The main subject of this dissertation is a comprehensive examination of the role of similarity measures in time-series analysis through studying the influence of the Sakoe-Chiba band on accuracy of classification. A time series represents a series of numerical data points in successive order, usually with uniform intervals between them. Time series are used for storage, analysis and visualization of data collected in many different domains, including science, medicine, economics and others. The increasing demand to work with large amounts of data has led to a growing interest in researching different tasks of time-series data mining. In recent years, there is an increasing interest in studying various aspects of time-series classification. One of the most important issues of time-series classification is a thoughtful and adequate choice of the similarity measure. They enable comparison of time series by describing their similarity (or dissimilarity) with numerical values. In the field of time-series data mining a great number of different similarity measures has been proposed and used. Many of these measures are implemented using dynamic programming. Unfortunately, the time requirements of this technique may adversely affect the possibilities of its application in practice. One way of dealing with this issue is to limit the search area by applying global constraints. However, these restrictions may affect the accuracy of classification, and therefore understanding their impact is of great importance. -
APPROXIMATE STRING MATCHING METHODS for DUPLICATE DETECTION and CLUSTERING TASKS by Oleksandr Rudniy
Copyright Warning & Restrictions The copyright law of the United States (Title 17, United States Code) governs the making of photocopies or other reproductions of copyrighted material. Under certain conditions specified in the law, libraries and archives are authorized to furnish a photocopy or other reproduction. One of these specified conditions is that the photocopy or reproduction is not to be “used for any purpose other than private study, scholarship, or research.” If a, user makes a request for, or later uses, a photocopy or reproduction for purposes in excess of “fair use” that user may be liable for copyright infringement, This institution reserves the right to refuse to accept a copying order if, in its judgment, fulfillment of the order would involve violation of copyright law. Please Note: The author retains the copyright while the New Jersey Institute of Technology reserves the right to distribute this thesis or dissertation Printing note: If you do not wish to print this page, then select “Pages from: first page # to: last page #” on the print dialog screen The Van Houten library has removed some of the personal information and all signatures from the approval page and biographical sketches of theses and dissertations in order to protect the identity of NJIT graduates and faculty. ABSTRACT APPROXIMATE STRING MATCHING METHODS FOR DUPLICATE DETECTION AND CLUSTERING TASKS by Oleksandr Rudniy Approximate string matching methods are utilized by a vast number of duplicate detection and clustering applications in various knowledge domains. The application area is expected to grow due to the recent significant increase in the amount of digital data and knowledge sources. -
Open Thesis-Cklee-Jul21.Pdf
The Pennsylvania State University The Graduate School College of Engineering EFFICIENT INFORMATION ACCESS FOR LOCATION-BASED SERVICES IN MOBILE ENVIRONMENTS A Dissertation in Computer Science and Engineering by Chi Keung Lee c 2009 Chi Keung Lee Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy August 2009 The dissertation of Chi Keung Lee was reviewed and approved∗ by the following: Wang-Chien Lee Associate Professor of Computer Science and Engineering Dissertation Advisor Chair of Committee Donna Peuquet Professor of Geography Piotr Berman Associate Professor of Computer Science and Engineering Daniel Kifer Assistant Professor of Computer Science and Engineering Raj Acharya Professor of Computer Science and Engineering Head of Computer Science and Engineering ∗Signatures are on file in the Graduate School. Abstract The demand for pervasive access of location-related information (e.g., local traffic, restau- rant locations, navigation maps, weather conditions, pollution index, etc.) fosters a tremen- dous application base of Location Based Services (LBSs). Without loss of generality, we model location-related information as spatial objects and the accesses of such information as Location-Dependent Spatial Queries (LDSQs) in LBS systems. In this thesis, we address the efficiency issue of LDSQ processing through several innovative techniques. First, we develop a new client-side data caching model, called Complementary Space Caching (CSC), to effectively shorten data access latency over wireless channels. This is mo- tivated by an observation that mobile clients, with only a subset of objects cached, do not have sufficient knowledge to assert whether or not certain object access from the remote server are necessary for given LDSQs. -
Checking Whether a Word Is Hamming-Isometric in Linear Time
Checking whether a word is Hamming-isometric in linear time Marie-Pierre B´eal ID ,∗ Maxime Crochemore ID † July 26, 2021 Abstract duced Fibonacci cubes in which nodes are on a binary alphabet and avoid the factor 11. The notion of d- A finite word f is Hamming-isometric if for any two ary n-cubes has subsequently be extended to define word u and v of the same length avoiding f, u can the generalized Fibonacci cube [10, 11, 19] ; it is the 2 be transformed into v by changing one by one all the subgraph Qn(f) of a 2-ary n-cube whose nodes avoid letters on which u differs from v, in such a way that some factor f. In this framework, a binary word f is 2 all of the new words obtained in this process also said to be Lee-isometric when, for any n 1, Qn(f) 2 ≥ avoid f. Words which are not Hamming-isometric can be isometrically embedded into Qn, that is, the 2 have been characterized as words having a border distance between two words u and v vertices of Qn(f) 2 2 with two mismatches. We derive from this charac- is the same in Qn(f) and in Qn. terization a linear-time algorithm to check whether On a binary alphabet, the definition of a Lee- a word is Hamming-isometric. It is based on pat- isometric word can be equivalently given by ignor- tern matching algorithms with k mismatches. Lee- ing hypercubes and adopting a point of view closer isometric words over a four-letter alphabet have been to combinatorics on words.