DOCSLIB.ORG

Abstractive Text Summarization Using Rich Semantic Graph for Marathi Sentence

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Abstractive Text Summarization Using Rich Semantic Graph for Marathi Sentence JASC: Journal of Applied Science and Computations ISSN NO: 1076-5131 Abstractive Text Summarization using Rich Semantic Graph for Marathi Sentence Sheetal Shimpikar, Sharvari Govilkar Computer Technology Department, Mumbai University [email protected] [email protected] Abstract — Text summarization helps to find, take out important sentences from the original document and links together to construct a short and clear summary. Large text documents are difficult to summarize manually. Using the computer program, a text can be reduced to get important points; the summary obtained from it is termed as Text summarization. The objective of the work is the representation and summarization of Indian language for “Marathi” text documents using abstractive text summarization techniques. The proposed approach takes Marathi documents as input text. The first step is pre-processing of the input text. Rich semantic graph method. The challenge in doing abstractive text summarization in Marathi documents is due to the complexity because it cannot be formulated mathematically or logically. And no work has been done on Marathi language. The Rich Semantic Graph based method gives the correct, bug free result. Keywords — Text Summarization, Rich Semantic Graph, Part of Speech Tagging, Name Entity Recognition, Ontology I. INTRODUCTION In current era of digital cultures and technologies, to understand the huge amount of information, text summarization is very important method for all. The purpose of text summarization is to minimize the text from the documents into meaningful form and save important contents. Abstractive text summarization methods use language understanding tools to generate a summary. They extract phrases and lexical chains from the documents. Important steps in abstractive text summarization are taking out the basic features, pick out the applicable information, clarifying and minimizing information. Abstractive text summarization is divided into two parts. In structured based, important information is converted into coded form in the document through patterns like frames, scripts and templates. In semantic based, natural language generation system (NLG) is used. The input will be the semantic representation of document. There are some limitations in abstractive text summarization techniques, for example, combination of sentences is not much developed, so the result obtained from the machine generated automatic summaries will be unclear. Abstractive text summarization systems are tough to reproduce as they depend on the internal tools. These are required to extract the information and language generation. Abstraction needs meaningful text understanding. II. LITERATURE SURVEY Here different techniques for abstractive text summarization in Indian languages are cited. Most of the researchers concentrate on abstract data to be extracted from source document. Extractive summarizations are weaker, whereas Abstractive summarization is more impressive. Abstractive text summarization techniques have been applied on various languages to detect whether the valid sentences are chosen. The following data is arranged as per the various regional languages on which abstractive text summarization techniques are applied. Jagadish S Kallimani [1] suggests a solution for abstractive summarization by making use of extractive methodology in Kannada language. In this method, the abstract data is extracted from source document. This information dense data is then post processed to gather key or most important concepts from original text. The main idea is to generate abstractive summary by gathering key concepts from source document using extractive summary technique. Rajina Kabeer [2] used semantic Graph based method which concentrates on summarizing documents in Malayalam. In this method the input document undergoes series of linguistics processing to get triples from each sentence of source document. With help of semantic triples, the semantic graph is generated and graph needs to be reduced in order to remove redundancy and to attain concise abstract summary. It gives opportunities and confidence to move forward with abstractive summarization techniques using various methods like domain based ontology, semantic graphic representation, word net etc. Manjula Subramanian and Vipul Dalal [3] discussed about Hindi language by using semantic graph reducing technique to generate abstractive summary. This approach is divided into three phases. Developing Rich Semantic Graph from source document, reducing Rich Semantic Graph to form the abstractive Rich Semantic Graph and final step includes the generation of abstractive summary from abstracted Rich Semantic Graph. Volume V, Issue XII, December/2018 Page No:2381 JASC: Journal of Applied Science and Computations ISSN NO: 1076-5131 Sabina, Priyanka, Adiba, Palash [4] had done a survey on various Indian regional languages that are based on abstractive type. After doing the study they analysed and discussed the articles and came to a conclusion that semantic graph based method is perfect for Bengali language. They also compared rule based and ontology based method. Before working on Bengali summarizer, domain ontology and word net should be build. Comparison between single and multi-document summarization techniques has done. Ibrahim and Mostafa [5] have worked on abstractive summary using rich semantic graph reducing technique for single document. In this type, the original document is taken in the form of rich semantic graph, then the graph is reduced and then abstractive summary is obtained from the reduced graph. A case study is shown that how the original text was reduced to fifty percent. Nikita Munot and Dr.Sharvari Govilkar [6] have worked on English language. They have proposed a system that takes a single document as input and by using Rich semantic graph technique, output summary is obtained. An example is shown which obtains an abstractive text summary using Rich semantic graph method. Khan and N. Salim [7] had a survey on different methods proposed by different researchers on abstractive summarization. The common thing between them is to find key sentences from the document. J. Mohan, Sunitha, A. Ganesha, Jaya [8] have worked on ontology based abstractive summarization. Ontology is a approved and clear definition of a shared abstract idea. Various frequently used ontology based methods are been discussed which are related to abstractive type. N. Moratanch [9] worked on the various abstractive type summarization methods. Here the paper consists of the important types improved, bugs obtained, fact findings and upcoming guidance in text summarization. I. Fathy, D. Fadl and M.Aref [10] have worked on rich semantic graph based type. In this paper RSG technique is used to obtain the English text. Here the Word Net ontology comes in picture to generate multiple texts. Used in five parts: Text planning, Sentence planning, Surface Realization, Writing Styles Selected Essay Generation and Text Evaluation. D.Bartakke, S. D.Sawarkar, and A. Gulati [11] have worked on multi document text summarization by using abstractive text summarization. So there is use of more than one document to create abstractive summary. Semantic graphs are generated for each sentence, generated graphs are reduced, some rules are created and used to obtain abstractive summary. It can be inferred from the literature review that very negligible amount of work has been done for Marathi Abstractive Text Summarization and we propose the following system to do Abstractive Text Summarization in Marathi language. III. METHODOLOGY A set of Marathi text document is given as input to the system. The proposed approach consists of following phases: Marathi text document as input, Pre-processing, Rich Semantic Graph phase (graph creation, reduction, generate summary). Fig. 1 The proposed system architecture Volume V, Issue XII, December/2018 Page No:2382 JASC: Journal of Applied Science and Computations ISSN NO: 1076-5131 A. Pre-processing First step of pre-processing is used to present the text documents into clear word format. Example- !”@£}[]$ मुरलीधर देिवदास आमटे उफ बाबा आमटे हे एक थोर मराठी समाजसेवक होते.बाबा आमट$9ा जU िडस$बर २३ १९१४ रोजी महारा ातील वधा िज,ात \हाला .बाबा आमट$ना भारत सरकार कडून १९७१ म;े प3Iी पुरार Bा] \हाला आहे. तसेच बाबा आमट$ना भारत सरकार कडून १९८६ म;े प3िवभूषण पुरार Bा] \हाला आहे.. Baba Amte was a very good human being. 3.1 Input Validation To find out either the given document is valid in Devanagari script or not. The invalid words are removed before using them. Algorithm: Input: Marathi single document as input. Output: Marathi (Devanagari) script Steps- 1. Use the character set as UTF-8. 2. Scan the input document. 3. Compare each character from scanned input document with UTF-8. 4. If character is present in UTF-8, then it is valid to Devanagari script otherwise not. 5. Ignore all invalid Devanagari script characters. 6. Repeat step 3 till all characters from input script document get verified. 7. Store all valid Devanagari character, words in file to process further. 3.2 Tokenization Separating tokens from the input document is referred as Tokenization. The separate word/token from the sentences is called Lexicons. With the help of lexical analyzer one can tokenize the input document as one token per line. Marathi is a grouped language where word limits are firm. By searching spaces between the words, tokenization can be done. The importance of this phase is that, we can deal with each word separately. Algorithm:- Input- Marathi (Devanagari) document. Output- List of tokens. Steps- 1. Start 2. Initialize all pointers (input (for character), output (for token), initially assign tokens = NULL) 3. Scan the input document. 4. Check for not end of file. i Read a character from input file. ii If character is not special character then do following while space character is not found. • Treat all character as a token. • Add token into token list file. • Increment respective pointers. 5. Repeat step 4 until end of file. 6. Stop. 3.3 Stemming Stemming is used to remove suffixes from words. Stem is not necessarily the linguistic root of the word.
Load more

Recommended publications
  • Hindi Named Entities Recognition (Ner) Using Natural Language Processing and Machine Learning
    International Journal of Advances in Electronics and Computer Science, ISSN(p): 2394-2835 Volume-6, Issue-9, Sep.-2019 http://iraj.in HINDI NAMED ENTITIES RECOGNITION (NER) USING NATURAL LANGUAGE PROCESSING AND MACHINE LEARNING 1RIA MEHTA, 2DWEEP PANDYA, 3PRATIK CHAUDHARI, 4DEVIKA VERMA, 5KRISHNANJAN BHATTACHARJEE, 6SHIVA KARTHIK S, 7SWATI MEHTA, 8AJAI KUMAR 1,2,3,4Vishwakarma Institute of Information Technology, Pune, India 5,6,7,8Centre for Development of Advanced Computing, Pune, India E-mail: 1ria.mehta [email protected], 2dweep.pandya [email protected], 3pratik.chaudhari [email protected], [email protected], [email protected], [email protected], [email protected], [email protected] Abstract - Named Entity Recognition (NER) is an important task in Natural Language Processing (NLP) that aims to auto identify and annotate Named Entities in the text, such as Person, Location, Organization etc. NER has been an essential component in various applications such as Information Extraction and Retrieval, Machine Translation, Question Answering (Q-A), Text Summarization etc. For NER in Hindi, while there have been a number of studies carried out, no high accuracy tool has yet been developed as per the Literature Survey. In this research, a methodology for Hindi Named Entities Recognition using NLP algorithms with RDF and Conditional Random Fields has been proposed. The results derived shows that the hybrid approach for NER achieves the recognition accuracy to 90.7% on Hindi texts. Keywords - Named Entity Recognition, Machine Learning, Natural Language Processing, CRF I. INTRODUCTION The accuracy of NER systems vary as per texts. The process of identifying Named Entities (NEs) Rule based systems have higher accuracy from a textual document and classifying them into but are not customizable different conceptual categories (Name, Place, Party, Limited tagset is considered in existing Designation) is an important step in the task of systems.
    [Show full text]
  • Cs224n-2021-Lecture11-Qa.Pdf
    Lecture 11: Question Answering Danqi Chen Princeton University Lecture plan 1. What is question answering? (10 mins) Your default final project! 2. Reading comprehension (50 mins) ✓ How to answer questions over a single passage of text 3. Open-domain (textual) question answering (20 mins) ✓ How to answer questions over a large collection of documents 2 1. What is question answering? Question (Q) Answer (A) The goal of question answering is to build systems that automatically answer questions posed by humans in a natural language The earliest QA systems dated back to 1960s! (Simmons et al., 1964) 3 Question answering: a taxonomy Question (Q) Answer (A) • What information source does a system build on? • A text passage, all Web documents, knowledge bases, tables, images.. • Question type • Factoid vs non-factoid, open-domain vs closed-domain, simple vs compositional, .. • Answer type • A short segment of text, a paragraph, a list, yes/no, … 4 Lots of practical applications 5 Lots of practical applications 6 Lots of practical applications 7 IBM Watson beated Jeopardy champions 8 IBM Watson beated Jeopardy champions Image credit: J & M, edition 3 (1) Question processing, (2) Candidate answer generation, (3) Candidate answer scoring, and (4) Confidence merging and ranking. 9 Question answering in deep learning era Image credit: (Lee et al., 2019) Almost all the state-of-the-art question answering systems are built on top of end- to-end training and pre-trained language models (e.g., BERT)! 10 Beyond textual QA problems Today, we will mostly focus on how to answer questions based on unstructured text. Knowledge based QA Image credit: Percy Liang 11 Beyond textual QA problems Today, we will mostly focus on how to answer questions based on unstructured text.
    [Show full text]
  • UNIVERSITY of CALIFORNIA Los Angeles
    UNIVERSITY OF CALIFORNIA Los Angeles Constructing Diasporic Identity Through Kathak Dance: Flexibility, Fixity, and Nationality in London and Los Angeles A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Culture and Performance by Shweta Saraswat 2019 © Copyright by Shweta Saraswat 2019 ABSTRACT OF THE DISSERTATION Constructing Diasporic Identity Through Kathak Dance: Flexibility, Fixity, and Nationality in London and Los Angeles by Shweta Saraswat Doctor of Philosophy in Culture and Performance University of California, Los Angeles, 2019 Professor Anurima Banerji, Chair This dissertation focuses on the role of the classical Indian dance form Kathak in negotiating questions of cultural identity and national affiliation among members of the Indian diaspora residing in London, UK, and Los Angeles, USA. This study considers how institutional actions and discourses related to the practice of Kathak dance in these two cities and the personal experiences of dancers themselves reflect certain political, aesthetic, and social values that impact the formation of diasporic identity. The dissertation argues that Indian diasporic subjects negotiate a fundamental tension through their practice of Kathak dance: the tension between Kathak’s inherent flexibility and contextual conditions of fixity. ii As described in Chapter 1, Kathak’s inherent flexibility refers to certain foundational elements of the dance that center around creative interpretation, improvisation, and immersive practice (riyaaz), as well as the expression of multiple identities that these foundational elements enable. A discourse of Kathak’s flexibility frames the dancer’s transcendence and/or transformation of socially assigned identifications as an act of aesthetic virtuosity with cross- cultural significance.
    [Show full text]
  • Oil and Gas News 05 Feb 2021
    ONGC News, 05.02.2021 Print Page 1 of 61 Sanmarg Satat vikas ke liya gahan sahyog ki zarurat ONGC CMD Feb 5, 2021 | Kolkata | Pg No.: 13 | | Sq Cm:385 | AVE: 1107459 | PR Value: 5537294 Page 2 of 61 The Hindu Business Line HPCL Q3 net zooms Rs215% to Rs2,354.6 cr on inventory gains Feb 5, 2021 | Delhi | Pg No.: 2 | | Sq Cm:156 | AVE: 288260 | PR Value: 1441302 Page 3 of 61 The Asian Age HPCL REPORTS RECORD PROFIT Feb 5, 2021 | Delhi | Pg No.: 7 | | Sq Cm:42 | AVE: 147965 | PR Value: 739823 Page 4 of 61 The Times of India HPCL net trebles to Rs 2,354cr in Q3 Feb 5, 2021 | Delhi | Pg No.: 19 | | Sq Cm:15 | AVE: 314541 | PR Value: 1572707 Image: 2 / 2 Page 5 of 61 Mint HPCL's Q3 net profit sees three-fold rise Feb 5, 2021 | Delhi | Pg No.: 6 | | Sq Cm:118 | AVE: 473690 | PR Value: 2368450 Page 6 of 61 Business Standard HPCL REPORTS RECORD PROFIT ON INVENTORY... Feb 5, 2021 | Delhi | Pg No.: 1,4 | | Sq Cm:141 | AVE: 349778 | PR Value: 1748890 Page 7 of 61 Business Standard HPCL REPORTS RECORD PROFIT ON INVENTORY... Feb 5, 2021 | Delhi | Pg No.: 1,4 | | Sq Cm:141 | AVE: 349778 | PR Value: 1748890 Page 8 of 61 The Economic Times HPCL Q3 Profit Triples to Rs 2,355 Crore Feb 5, 2021 | Delhi | Pg No.: 1,10 | | Sq Cm:103 | AVE: 1055098 | PR Value: 5275488 Page 9 of 61 The Economic Times HPCL Q3 Profit Triples to Rs 2,355 Crore Feb 5, 2021 | Delhi | Pg No.: 1,10 | | Sq Cm:103 | AVE: 1055098 | PR Value: 5275488 Page 10 of 61 The Financial Express HPCL profit increases 215% to Rs2,355 crore Feb 5, 2021 | Delhi | Pg No.: 4 | | Sq Cm:157 | AVE: 514707
    [Show full text]
  • Word Sense Disambiguation Using Semantic Web for Tamil to English Statistical Machine Translation Santosh Kumar T.S
    IRA-International Journal of Technology & Engineering ISSN 2455-4480; Vol.05, Issue 02 (2016) Pg. no. 22-31 Institute of Research Advances http://research-advances.org/index.php/IRAJTE Word Sense Disambiguation Using Semantic Web for Tamil to English Statistical Machine Translation Santosh Kumar T.S. Bharathiar University, India. Type of Reviewed: Peer Reviewed. DOI: http://dx.doi.org/10.21013/jte.v5.n2.p1 How to cite this paper: T.S., Santosh Kumar (2016). Word Sense Disambiguation Using Semantic Web for Tamil to English Statistical Machine Translation. IRA-International Journal of Technology & Engineering (ISSN 2455-4480), 5(2), 22-31. doi:http://dx.doi.org/10.21013/jte.v5.n2.p1 © Institute of Research Advances This work is licensed under a Creative Commons Attribution-Non Commercial 4.0 International License subject to proper citation to the publication source of the work. Disclaimer: The scholarly papers as reviewed and published by the Institute of Research Advances (IRA) are the views and opinions of their respective authors and are not the views or opinions of the IRA. The IRA disclaims of any harm or loss caused due to the published content to any party. 22 IRA-International Journal of Technology & Engineering ABSTRACT Machine Translation has been an area of linguistic research for almost more than two decades now. But it still remains a very challenging task for devising an automated system which will deliver accurate translations of the natural languages. However, great strides have been made in this field with more success owing to the development of technologies of the web and off late there is a renewed interest in this area of research.
    [Show full text]
  • Hotstar Us App Download Hotstar Us App Download
    hotstar us app download Hotstar us app download. Completing the CAPTCHA proves you are a human and gives you temporary access to the web property. What can I do to prevent this in the future? If you are on a personal connection, like at home, you can run an anti-virus scan on your device to make sure it is not infected with malware. If you are at an office or shared network, you can ask the network administrator to run a scan across the network looking for misconfigured or infected devices. Another way to prevent getting this page in the future is to use Privacy Pass. You may need to download version 2.0 now from the Chrome Web Store. Cloudflare Ray ID: 669b589c5b8ac433 • Your IP : 188.246.226.140 • Performance & security by Cloudflare. Hotstar Android. Hotstar for Android smartphones is the application that offers us the best movies, TV series and programs, and sports broadcasts on Indian television. 1 2 3 4 5 6 7 8 9 10. Watching TV on our phone is getting more and more usual thanks to the speed modern-day connections can offer us whether through 4G data plans or WiFi networks. That's why so many apps similar to You TV Player are popping up all over the place. Free online TV from India. One of those applications is Hotstar , the audiovisual contents of which are focused on India, offering its users series, movies, television programs, and live or pre-recorded sports broadcasts . Once you download its APK, you'll be able to appreciate that it comes along with an eye-catching and intuitive design that allows us to play certain contents via streaming and also download them to be able to watch them offline.
    [Show full text]
  • Question Answering System Using Ontology in Marathi Language
    International Journal of Artificial Intelligence and Applications (IJAIA), Vol.8, No.4, July 2017 QUESTION ANSWERING SYSTEM USING ONTOLOGY IN MARATHI LANGUAGE Sharvari S. Govilkar 1 and J. W. Bakal 2 1Department of Computer Engineering, PCE, Mumbai, India 2Department of Computer Engineering, SJCOE, Mumbai, India ABSTRACT Humans are always in a quest to extract information related to some topic or entity. Question answering system helps user to find the precise answer of the question articulated in natural language. Question answering system provides explicit, concise and accurate answer to user questions rather than providing set of relevant documents or web pages as answers as most of the information retrieval system does. The paper proposes question answering system for Marathi natural language by using concept of ontology as a formal representation of knowledge base for extracting answers. Ontology is used to express domain specific knowledge about semantic relations and restrictions in the given domains. The ontologies are developed with the help of domain experts and the query is analyzed both syntactically and semantically. The results obtained here are accurate enough to satisfy the query raised by the user. The level of accuracy is enhanced since the query is analyzed semantically. KEYWORDS Question answering system (QAS), Ontology, Marathi Natural language QA system (NLQA), Natural language processing (NLP) 1. INTRODUCTION With the rapid growth of the amount of online and electronic documents in Indian regional language, the keyword based approaches lack many important elements to enable QA driven process. So a system is required which can provide user with accurate answers for their queries .Question answering system provides user with functionality where they can ask questions in natural language and the system returns answer which is most accurate and precise of all the possible answers for the given input question.
    [Show full text]
  • Danqi Chen Princeton University Lecture Plan
    Lecture 11: Question Answering Danqi Chen Princeton University Lecture plan 1. What is question answering? (10 mins) Your default final project! 2. Reading comprehension (50 mins) ✓ How to answer questions over a single passage of text 3. Open-domain (textual) question answering (20 mins) ✓ How to answer questions over a large collection of documents 2 1. What is question answering? Question (Q) Answer (A) The goal of question answering is to build systems that automatically answer questions posed by humans in a natural language The earliest QA systems dated back to 1960s! (Simmons et al., 1964) 3 Question answering: a taxonomy Question (Q) Answer (A) • What information source does a system build on? • A text passage, all Web documents, knowledge bases, tables, images.. • Question type • Factoid vs non-factoid, open-domain vs closed-domain, simple vs compositional, .. • Answer type • A short segment of text, a paragraph, a list, yes/no, … 4 Lots of practical applications 5 Lots of practical applications 6 Lots of practical applications 7 IBM Watson beated Jeopardy champions 8 IBM Watson beated Jeopardy champions Image credit: J & M, edition 3 (1) Question processing, (2) Candidate answer generation, (3) Candidate answer scoring, and (4) Confidence merging and ranking. 9 Question answering in deep learning era Image credit: (Lee et al., 2019) Almost all the state-of-the-art question answering systems are built on top of end- to-end training and pre-trained language models (e.g., BERT)! 10 Beyond textual QA problems Today, we will mostly focus on how to answer questions based on unstructured text. Knowledge based QA Image credit: Percy Liang 11 Beyond textual QA problems Today, we will mostly focus on how to answer questions based on unstructured text.
    [Show full text]