DOCSLIB.ORG

Text Sentiment Analysis of Marathi Language in English and Devanagari Script

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Text Sentiment Analysis of Marathi Language in English and Devanagari Script Text Sentiment Analysis of Marathi Language in English And Devanagari Script by Harry Pradeep Gavali Dissertation Submitted in partial fulfilment of the requirements for the degree of MSc in Data Analytics At Dublin Business School Supervisor: Pierpaolo Dondio January 2020 2 DECLARATION I declare that this dissertation that I have submitted to Dublin Business School for the award of MSc in Data Analytics is my own research and original work; except where otherwise stated, where it is clearly acknowledged by references. Also, this work has not been submitted for any other degree and is entirely in compliance with Dublin Business School’s academic policy. Signed: Harry Pradeep Gavali Dated: 6th January 2020. 3 ACKNOWLEDGEMENT I would like to express my deepest gratitude to my mentor and research supervisor Mr Pierpaolo Dondio, for guiding me on the right track from the beginning to the end of the research. I appreciate his constructive feedbacks and his willingness to always dedicate his precious time generously. He was always available to help, and his remarks have helped me a lot in order to make the research successful and professional. I would also like to thank my parents, my friends and their parents who invested their precious time and helped me in compiling the data for the research. Few of them had to act as judges in order to rate the data, which was a crucial part of the research. Their help is deeply appreciated. Furthermore, I would like to thank Google for allowing me to incorporate their Cloud Translation API which translated the data smoothly and swiftly. Finally, I am indebted to my family for they have been a continuous support and have always motivated me throughout the completion of the thesis. 4 ABSTRACT Marathi is a language spoken by a very large number of people in India and about 10% of the Indian population uses ‘Marathi + English’ while texting one another. This study focuses on text sentiment analysis of the mixed language text of Marathi (written in English) first, and then compares the accuracy again after the same sentences (written in Devanagari script) have been translated using Google’s cloud translation services. Same machine learning techniques were applied on both in order to maintain equality. A new and accurate dataset which comprised of day to day sentences was compiled manually in order to reduce error. The outputs were later compared and the need to develop such researches further is highlighted. The results of the research show that the algorithms like Random Forest and SVM give us the highest accuracies of 65.41% and 64.16% respectively. Keywords: machine learning, text classification, Marathi sentiment analysis, NLP Marathi. 5 Table of Contents DECLARATION ............................................................................................................................ 2 ACKNOWLEDGEMENT ................................................................................................................ 3 ABSTRACT ................................................................................................................................... 4 LIST OF FIGURES ......................................................................................................................... 6 Chapter 1: INTRODUCTION ........................................................................................................ 7 Chapter 2: LITERATURE REVIEW .............................................................................................. 13 Chapter 3: METHODOLOGY ..................................................................................................... 19 3.1 Datasets: ................................................................................................................................. 20 3.2 Sentiment Valuation: .............................................................................................................. 21 3.3 Data Preprocessing: ................................................................................................................ 21 3.4 Machine Learning:................................................................................................................... 23 3.5 Statistical Testing: ................................................................................................................... 27 Chapter 4: IMPLEMENTATION ................................................................................................. 30 Chapter 5: RESULT ................................................................................................................... 37 5.1 Random Forest algorithm. ...................................................................................................... 39 5.2. K Nearest Neighbour Algorithm. ............................................................................................ 39 5.3. Naïve Bayes Algorithm. .......................................................................................................... 40 5.4. Decision Tree Algorithm. ........................................................................................................ 41 5.5. Support Vector Machine Algorithm. ..................................................................................... 41 5.6. Logistic Regression Algorithm. ............................................................................................... 42 5.7. Statistical testing: ................................................................................................................... 42 Chapter 6: CONCLUSION .......................................................................................................... 44 6.1 Future Scope ..................................................................................................................... 46 PLAGIARISM AND REFERENCES ............................................................................................... 48 APPENDIX ................................................................................................................................. 51 6 LIST OF FIGURES Figure 1: Dissertation Roadmap .............................................................................................. 12 Figure 2: Conceptual diagram of the research process ........................................................... 29 Figure 3: Initial dataset ............................................................................................................ 30 Figure 4: Final Rating ................................................................................................................ 31 Figure 5: Final rating for Marathi dataset ................................................................................ 31 Figure 6: Graph showing different classes and their count ..................................................... 32 Figure 7: Number of stop words in every sentence ................................................................ 32 Figure 8: Sentences with stop words removed ....................................................................... 33 Figure 9: Tokenized sentences ................................................................................................. 33 Figure 10: o/p table for the algorithms ................................................................................... 34 Figure 11: Final Marathi data ................................................................................................... 34 Figure 12: Translated text obtained after using google services ............................................. 35 Figure 13: Final translated text from Marathi to English ......................................................... 35 Figure 14: Marathi data after translation and removing stop words ...................................... 36 Figure 15: o/p table for translated Marathi dataset ............................................................... 36 Figure 16: output table comparing accuracies of both methods of text analysis ................... 37 Figure 17: RF output or mixed language text .......................................................................... 39 Figure 19: KNN o/p for mixed language text ........................................................................... 39 Figure 21: Naïve Bayes o/p for mixed language text ............................................................... 40 Figure 23: DT o/p for mixed language text .............................................................................. 41 Figure 25: SVM o/p for mixed language text ........................................................................... 41 Figure 27: LR o/p for mixed language text ............................................................................... 42 Figure 29: t test output for the algorithms .............................................................................. 43 Figure 30: output for the comparison of the two text classification methods ....................... 44 7 Chapter 1: INTRODUCTION Languages have existed for millions of years and since then have been going through a lot of advancements and changes. Over the course of history, we have seen language and culture being transferred and exchanged due to travellers visiting other shores and warriors capturing different lands. This led people of the land to learn the language of the other people. Those were the earliest instances of people learning different languages for convenience purposes. Fast-forward to today, with the invention of
Load more

Recommended publications
  • Pakistani Nationalism and the State Marginalisation of the Ahmadiyya Community in Pakistan
    Sadia Saeed: Pakistani Nationalism and the State Marginalisation of the Ahmadiyya Community in Pakistan Pakistani Nationalism and the State Marginalisation of the Ahmadiyya Community in Pakistan Sadia Saeed University of Michigan, Ann Arbor, USA Abstract This paper examines the relationship between nationalism, state formation, and the marginalisation of national minorities through an historical focus on Pakistani state’s relationship with the Ahmadiyya community, a self-defined minority sect of Islam. In 1974, a constitutional amendment was enacted that effectively rendered the Ahmadiyya community a non-Muslim minority, in spite of claims by the community that it was Muslim and hence not a minority. This paper attempts to account for this anti-Ahmadiyya state legislation by arguing that the genealogy of the idea of a Pakistani state is key for understanding the politics of exclusion of the Ahmadiyya community from ‘Muslim citizenship’ – that is, who is and isn’t a Muslim. Introduction This paper explores the relationship between nationalism, state formation, and the marginalisation of national minorities. In the literature on the nation and nationalism, the relationship between state formation and nationalist exclusions is often under-theorised, with the implicit assumption made that processes of nation building occur independently of the construction of state institutions. Many works have shown the pitfalls of this neglect by demonstrating that the articulation of nationalist ideologies, and the processes which take place in the national community as a result of such nationalisms, are either aided by, or are articulated in response to, the formation of state institutions and practices of social closure (Brubaker 1992; Balibar 2004; Omi and Winant 1994).
    [Show full text]
  • Indian Politics and Society Since Independence: Events, Processes
    Indian Politics and Society since Independence This book focuses on politics and society in India. It explores new areas enmeshed in the complex social, economic and political processes in the country. Linking structural characteristics with broader sociological context, the book emphasizes the strong influence of sociological issues on politics, such as the shaping of the social milieu and the articulation of the political in day-to-day events. Political events are connected with the ever-changing social, economic and political proc- esses in order to provide an analytical framework to explain ‘peculiarities’ of Indian politics. The main argument of the book is that three major ideological influences have provided the foundational values of Indian politics: colonialism, nationalism and democracy. The colonial, nationalist and democratic articulation of the political have shaped Indian politics in a complex way. Structured themati- cally, with a multitude of pedagogical features, this work is a useful text for stu- dents of political science, sociology and South Asian Studies. Bidyut Chakrabarty is Professor of the Department of Political Science at the University of Delhi. His main research areas are public administration and mod- ern Indian political thought. He has published extensively on the subjects, and is the author of The Partition of Bengal and Assam, 1932–47 (Routledge, 2004), Social and Political Thought of Mahatma Gandhi, 1933–1943 (Routledge, 2006), Forging Power: Coalition Politics in India (Oxford University Press, 2006),
    [Show full text]
  • Puneet Kaushik
    PUNEET KAUSHIK Born 1972, New Delhi Lives and works in New Delhi EDUCATION 1992-1996 BFA, Jamia Millia Islamia 1998-2000 Studio Art, University of California, Berkeley (Extension), San Francisco, US SOLO EXHIBITIONS 2016 Barren Red, Gallery Espace, New Delhi 2011 The Skin and the Surface, Gallery Beyond, Mumbai Evolution, Aman Hotel, New Delhi 2010 Germination, Apparao Gallery, Bangalore Germination, Apparao Gallery, Chennai Symbiosis, ApparaoGallery, Bangalore 2009 Germination, Kerkar Art Complex, Goa Germination, Gallery Beyond, Mumbai Symbiosis, Art Konsult, New Delhi 2008 Alter Ego, Alliance Françoise de Delhi, New Delhi 2007 Takashimaya, Tokyo, Japan 2004 The Embryo, Jehangir Art Gallery, Mumbai 2002 Indelible Imprints, Art Inc., New Delhi 2001 Paradox, Apparao Gallery, Chennai Paradox, Art Inc., New Delhi 2000 Markings on the Wall, Kala Art Gallery, San Francisco, California, USA 1999 Modern Primitive Gallery, Atlanta, Georgia, USA 1997 Roy G Biv Gallery, Columbus, Ohio, USA ANZ Grindlays Bank, New Delhi Selected GROUP EXHIBITIONS 2021 Fibre of our Lives, Gallery Art Positive, New Delhi 2017 Contrabanned-Provocations of Our Times, Art Konsult, New Delhi 2014 The Shape Things Will Take, Gallery Espace, New Delhi W.A.R, Engendered Gallery, American Centre, New Delhi Combine, Gallery Beyond, Mumbai Drawing 2014: 7 Decades of Indian Drawing, Gallery Espace, IGNCA, New Delhi 2013 The Embedded Landscape, Religare Art Gallery, Delhi “Resist”, An Engendered Art Protest, Gallery Beyond, Mumbai 7 Artists & 7 Mediums, Gallery 7, Mumbai Small
    [Show full text]
  • The Islamization of Pakistan, 1979-2009
    Viewpoints Special Edition The Islamization of Pakistan, 1979-2009 The Middle East Institute Washington, DC Middle East Institute The mission of the Middle East Institute is to promote knowledge of the Middle East in Amer- ica and strengthen understanding of the United States by the people and governments of the region. For more than 60 years, MEI has dealt with the momentous events in the Middle East — from the birth of the state of Israel to the invasion of Iraq. Today, MEI is a foremost authority on contemporary Middle East issues. It pro- vides a vital forum for honest and open debate that attracts politicians, scholars, government officials, and policy experts from the US, Asia, Europe, and the Middle East. MEI enjoys wide access to political and business leaders in countries throughout the region. Along with information exchanges, facilities for research, objective analysis, and thoughtful commentary, MEI’s programs and publications help counter simplistic notions about the Middle East and America. We are at the forefront of private sector public diplomacy. Viewpoints are another MEI service to audiences interested in learning more about the complexities of issues affecting the Middle East and US rela- tions with the region. To learn more about the Middle East Institute, visit our website at http://www.mei.edu Cover photos, clockwise from the top left hand corner: Government of Pakistan; Flickr user Kash_if; Flickr user Kash_if; Depart- ment of Defense; European Parliament; Flickr user Al Jazeera English; Flickr user groundreporter; Flickr user groundreporter. 2 The Middle East Institute Viewpoints: The Islamization of Pakistan, 1979-2009 • www.mei.edu Viewpoints Special Edition The Islamization of Pakistan, 1979-2009 The Middle East Institute Viewpoints: The Islamization of Pakistan, 1979-2009 • www.mei.edu 3 Viewpoints: 1979 The year 1979 was among the most tumultuous, and important, in the history of the modern Middle East.
    [Show full text]
  • 24 04 2016 :: Mahinename: 110114 :: Prooftime: 4/22/2016 10:58:1110:59:01 PM
    User: ASINGH2402 :: Product: HD :: Page: 2 :: PubDate: 24 04 2016 :: MahineName: 110114 :: ProofTime: 4/22/2016 10:58:1110:59:01 PM 2 नवभारत टाइस, रिववार, 24 अैल 2016 KKB 37/5'8" B.Tech IITK SRIV MCA Boy UP/ MP MBA ISB Hyd Veg TT Diretor Based Dr. Family 30/ 5'7"/ 12.5 Top MNC NCR seek b'ful cult- LPA wkg in S/W MNC. Cont: ed h'gly qlfd girlM:9935317848 09971233195 , 0124 - 4015706 Em:[email protected] Email: [email protected] SM4 Rjpt Bank Officer 30/ 5'10" wkng Dehradun.Father very Sr PSU Officer at Delhi. MATHUR Boy Nov'87, 6'1" Bank Officer pref. Contact ans mglk B.Tech wkg in Delhi with Biodata Mob:7500723929 Ast. Mgr 9 LPA seeks Tall wkg girl M: 09627501370 E: muke [email protected] PQM4 Haldwani based well stld Kumaoni rajput 36+ boy own business Kumaoni preferred call:7830120089 ·é¤×檤Ùè·é¤×檤Ùè ÚæÁÂêÌ ×ÔãÚæ BA Âæâ PQM4 Sriv fair H'sum 15-02- DU,26/5Ò7",¥ÂÙæ ȤôÅô SÅêçÇØô, 88/5'9" B.Tech MBA wkg Pvt. 100 »Á ·¤æ ×·¤æÙ, ¥æØ 25000/- DELHI Based affluent Gaur Bank (CRPT Fin) Mum 14 LPA ÂýçÌ×æã ãÔÌé Âɸè-çÜ¹è ·¤‹Øæ Brahmin fmly L'kng For B'ful seeks b'ful girl. Mob: 9161968463, 9415820306. edu. girl for their Son wkg US SM4 Amritsar based Garhw- ‘ææçã°! #9311714417, 9313714417 in MNC He is BE,MBA from ali Rajput boy 5'6"/1983, fair, US.smart tall 5'8"Nov87.Send only son.
    [Show full text]
  • Original Research Paper Dr. Girish Joshi* Pharmacology
    VOLUME-9, ISSUE-2, FEBRUARY-2020 • PRINT ISSN No. 2277 - 8160 • DOI : 10.36106/gjra Original Research Paper Pharmacology THE ETHNIC ORIGIN OF KONKANASTHA BRAHMINS: FACTS, MYTHS AND CONTROVERSIES Dr. Girish Joshi* Pharmacologist, Thane (W) *Corresponding Author KEYWORDS : The “Konkanastha Brahmins” also called the “Chitpavan father Nrusimha, a well known astrologer, was conferred the Brahmins” are a Brahmin community of the Konkan region of title of “Jyotirvitsaras” by the Moghul Emperor Akbar in 1599 the Indian State of Maharasthra. The community had A.D. The Brahmin Raghunāth must have learnt his caste name historically been very important to local Maharashtrian Chitpvan from his grandfather Nrusimha who had migrated politics, and Indian politics as a whole, as it provided the to Kashi from Konkan. Nrusimha also must have learnt it from Peshwas, i.e. the Prime ministers of the Maratha Empire that his father. Therefore the name “Chitpavan” must have been was the most dominant state in the subcontinent in the pre- used as back as 1570. (3) colonial period and had come to rule most of the region. (1) The reference in a letter dated 19th Feb 1677 written by a clerk Unheard of before 1700, the Chitpavan Brahmins of in service of King Shivāji Maharaj. The letter mentioned that Maharashtra came to dominate the elds of social reform, “Deshstha, Karhāde and Chitpāvan brāhmins lived in Kokan law, scholarship, government service and the arts by the area and were honored on for their merit”. After 1700 A.D. nineteenth century. Their two names, Konkanasth and many documents one can nd were the word Chitpāvan is Chitpavan, suggest their origins.
    [Show full text]
  • Does the Indian Electorate Benchmark Parliamentary Performance?
    DOES THE INDIAN ELECTORATE BENCHMARK PARLIAMENTARY PERFORMANCE? A Thesis submitted to the Faculty of the Graduate School of Arts and Sciences of Georgetown University in partial fulfillment of the requirements for the degree of Master of Public Policy in Public Policy By Shrihari Ramachandra, B.E Washington, D.C. April 14, 2021 Copyright 2021 by Shrihari Ramachandra All Rights Reserved ii DOES THE INDIAN ELECTORATE BENCHMARK PARLIAMENTARY PERFORMANCE? Shrihari Ramachandra, B.E Thesis Advisor: Dr. Andreas Kern, Ph.D. ABSTRACT The Indian electorate elects the Members of Parliament (MP) to represent them in the biggest forum of Indian democracy – the Parliament. The MPs represent the people's hopes and aspirations and play a key role in furthering the voters' social and economic welfare. However, does the electorate consider their representative's performance when casting their ballot? I hypothesize that parliamentary performance does not play any role in an MP's chance of re-election. Using data on the 2019 general election results, I show that parliamentary performance in the 16th Lok Sabha (2014-19) - measured by the number of questions, the private member legislations, attendance, gender, age, the term of office, debates participated did not enhance the likelihood of a representative parliamentarian to get re-elected. From a policy perspective, my findings underscore the importance of decline in the spirit of parliamentary democracy. This also hints at the significance of other mechanisms such as social biases, cash-based vote-buying, and campaign financing, which might play a far more important role in determining a policymaker's destiny in Indian politics.
    [Show full text]
  • The Journal of Parliamentary Information ______VOLUME LXIV NO.2 JUNE 2018 ______
    The Journal of Parliamentary Information ________________________________________________________ VOLUME LXIV NO.2 JUNE 2018 ________________________________________________________ LOK SABHA SECRETARIAT NEW DELHI ___________________________________ The Journal of Parliamentary Information __________________________________________________________________ VOLUME LXIV NO.2 JUNE 2018 CONTENTS Page ADDRESSES - Address by the President to Parliament … - Inaugural and Concluding Addresses of the Speaker, Lok Sabha, Smt. Sumitra Mahajan at the Sixth Commonwealth Parliamentary Association, India Region Conference, held in Patna, Bihar. PARLIAMENTARY EVENTS AND ACTIVITIES … PARLIAMENTARY AND CONSTITUTIONAL … DEVELOPMENTS PROCEDURAL MATTERS … DOCUMENTS OF CONSTITUTIONAL AND … PARLIAMENTARY INTEREST SESSIONAL REVIEW Lok Sabha … Rajya Sabha … State Legislatures … RECENT LITERATURE OF PARLIAMENTARY INTEREST … APPENDICES I. Statement showing the work transacted during the … Fourteenth Session of the Sixteenth Lok Sabha II. Statement showing the work transacted during the … 245th Session of the Rajya Sabha III. Statement showing the activities of the Legislatures of … the States and Union Territories during the period 1 January to 31 March 2018 IV. List of Bills passed by the Houses of Parliament … and assented to by the President during the period 1 January to 31 March 2018 V. List of Bills passed by the Legislatures of the States … and the Union Territories during the period 1 January to 31 March 2018 VI. Ordinances promulgated by the Union … and State Governments during the period 1 January to 31 March 2018 VII. Party Position in the Lok Sabha, the Rajya Sabha … and the Legislatures of the States and the Union Territories ADDRESS BY THE PRESIDENT TO PARLIAMENT ____________________________________________________________ The provision for an Address by the Head of State to Parliament goes back to the year 1921 when the Central Legislature was set up for the first time under the Government of India Act, 1919.
    [Show full text]
  • Resume of Work Done by Lok Sabha
    RESUME OF WORK DONE BY LOK SABHA SIXTEENTH LOK SABHA — THIRTEENTH SESSION (15 December, 2017 to 05 January, 2018) LOK SABHA SECRETARIAT NEW DELHI February, 2018/Phalguna, 1939 (Saka) T.O. No. 4/16 LS Vol. XIII © 2018 BY LOK SABHA SECRETARIAT Published under Rule 382 of the Rules of Procedure and Conduct of Business in Lok Sabha (Fifteenth Edition) and Printed by the Manager, Government of India Press, Minto Road, New Delhi. PREFACE This Publication contains a brief resume of work done in the Sixteenth Lok Sabha during the Thirteenth Session i.e. from 15 December, 2017 to 05 January, 2018. NEW DELHI; SNEHLATA SHRIVASTAVA February, 2018 Secretary General Phalguna, 1939 (Saka) CONTENTS PAGE(S) 1. DURATION OF SESSION .............................................................. 1 2. BILLS (i) Government Bills ............................................................ 2 (ii) Bills referred to Standing Committees ............................ 7 (iii) Private Members’ Bills .................................................... 9 3. CANCELLATION OF SITTINGS OF LOK SABHA .................................. 30 4. COMMITTEES (i) Financial Committees ..................................................... 31 (ii) Standing Committees ..................................................... 32 (iii) Committees other than Financial and Standing Committees .................................................................... 34 5. DIVISIONS ................................................................................ 38 6. FINANCIAL BUSINESS ................................................................
    [Show full text]