DOCSLIB.ORG

Aidarabic a Named-Entity Disambiguation Framework for Arabic Text

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Aidarabic a Named-Entity Disambiguation Framework for Arabic Text AIDArabic A Named-Entity Disambiguation Framework for Arabic Text Mohamed Amir Yosef, Marc Spaniol, Gerhard Weikum Max-Planck-Institut fur¨ Informatik, Saarbrucken,¨ Germany mamir|mspaniol|weikum @mpi-inf.mpg.de { } Abstract tion (NED) is essential for many application in the domain of Information Retrieval (such as informa- There has been recently a great progress in tion extraction). It also enables producing more the field of automatically generated knowl- useful and accurate analytics. The problem has edge bases and corresponding disambigua- been exhaustively studied in the literature. The tion systems that are capable of mapping essence of all NED techniques is using background text mentions onto canonical entities. Ef- information extracted from various sources (e.g. forts like the before mentioned have en- Wikipedia), and use such information to know the abled researchers and analysts from vari- correct/intended meaning of the mention. ous disciplines to semantically “understand” The Arabic content is enormously growing on contents. However, most of the approaches the Internet, nevertheless, background ground in- have been specifically designed for the En- formation is clearly lacking behind other languages glish language and - in particular - sup- such as English. Consider Wikipedia for example, port for Arabic is still in its infancy. Since while the English Wikipedia contains more than 4.5 the amount of Arabic Web contents (e.g. million articles, the Arabic version contains less in social media) has been increasing dra- than 0.3 million ones 1. As a result, and up to our matically over the last years, we see a knowledge, there is no serious work that has been great potential for endeavors that support done in the area of performing NED for Arabic an entity-level analytics of these data. To input text. this end, we have developed a framework called AIDArabic that extends the existing AIDA system by additional components 1.2 Problem statement that allow the disambiguation of Arabic texts based on an automatically generated NED is the problem of mapping ambiguous names knowledge base distilled from Wikipedia. of entities (mentions) to canonical entities regis- Even further, we overcome the still exist- tered in an entity catalog (knowledgebase) such as ing sparsity of the Arabic Wikipedia by ex- Freebase (www.freebase.com), DBpedia (Auer et ploiting the interwiki links between Arabic al., 2007), or Yago (Hoffart et al., 2013). For ex- and English contents in Wikipedia, thus, ample, given the text “I like to visit Sheikh Zayed. Despite being close to Cairo, it is known to be a enriching the entity catalog as well as dis- ambiguation context. quiet district”, or in Arabic,“ qJ Ë@ èPA KPI . k@ á ÓAîE.Q¯ á ÓÑ«QËAK. ZðYêËAK. Q ÒJK úæê¯ .Y K@P 1 Introduction èQ ëA®Ë@”. When processing this text automatically, we need to be able to tell that Sheikh Zayed de- 1.1 Motivation notes the the city in Egypt2, not the mosque in Internet data including news articles and web pages, Abu Dhabi3 or the President of the United Arab contain mentions of named-entities such as people, places, organizations, etc. While in many cases 1as of July 2014 the intended meanings of the mentions is obvi- 2http://en.wikipedia.org/wiki/Sheikh Zayed City ous (and unique), in many others, the mentions http://ar.wikipedia.org/wiki/Y K@P _ qJ .Ë@_ éJ KYÓ are ambiguous and have many different possible 3http://en.wikipedia.org/wiki/Sheikh Zayed Mosque meanings. Therefore, Named-Entity Disambigua- http://ar.wikipedia.org/wiki/Y K@P _ qJ .Ë@_ ©ÓAg. 187 Proceedings of the EMNLP 2014 Workshop on Arabic Natural Langauge Processing (ANLP), pages 187–195, October 25, 2014, Doha, Qatar. c 2014 Association for Computational Linguistics Emirates4. In order to automatically establish such order of magnitude smaller than the English mappings, the machine needs to be aware of the one. In addition, many entities in the Arabic characteristic description of each entity, and try to Wikipedia are specific to the Arabic culture find the most suitable one given the input context. with no corresponding English counterpart. In our example, knowing that the input text men- As a consequence, even many prominent enti- tioned the city of Cairo favors the Egyptian city ties are missing from the Arabic Wikipedia. over the mosque in Abu Dhabi, for example. In Name-Entity Dictionary: Most of the name- principle, state-of-the-art NED frameworks require • main four ingredients to solve this problem: entity dictionary entries originate from man- ual input (e.g. anchor links). Like outlined Entity Repository: A predefined universal before, Arabic Wikipedia has fewer resources • catalog of all entities known to the NED to extract name-entity mappings, caused by framework. In other words, each mention in the lack of entities and lack of manual input. the input text must be mapped to an entity in Entity-Descriptions: As already mentioned, the repository, or to null indicating the correct • there is a scarcity of anchor links in the Arabic entity is not included in the repository. Wikipedia. Further, the categorization system Name-Entity Dictionary: It is a many-to- of entities is insufficient, Both are essential • many relation between possible mentions and sources of building the entities descriptions. the entities in the repository. It connects an Hence, it is more challenging to produce com- entity with different possible mentions that prehensive description of each entity. might be used to refer to this entity, as well as Entity-Entity Relatedness Model: Related- connecting a mention with all potential candi- • ness estimation among entities is usually com- date entity it might denote. puted using the overlap in the entities descrip- Entity-Descriptions: It keeps per entity a tion and/or link structure of Wikipedia. Due to • bag of characteristic keywords or keyphrases the previously mentioned scarcity of contents that distinguishes an entity from another. In in the Arabic Wikipedia, it is also difficult to addition, they come with scoring scheme that accurately estimate the entity-entity related- signify the specificity of such keyword to that ness. entity. As a consequence, the main challenge in per- Entity-Entity Relatedness Model: For co- forming NED on Arabic text is the lack of a com- • herent text, the entities that are used for map- prehensive entity catalog together with rich descrip- ping all the mentions in the input text, should tions of each entity. We considered our open source 5 be semantically related. For that reason, an AIDA system (Hoffart et al., 2011)- mentioned as entity-entity relatedness model is required to state-of-the-art NED System by (Ferrucci, 2012) - asses the coherence. as a starting point and modified its data acquisition pipeline in order to generate a schema suitable for For the English language, all of the ingredi- performing NED on Arabic text. ents mentioned above are richly available. For instance, the English Wikipedia is a comprehen- 1.3 Contribution sive up-to-date resource. Many NED systems We developed an approach to exploit and fuse cross- use Wikipedia as their entity repository. Further- lingual evidences to enrich the background informa- more, many knowledge bases are extracted from tion we have about entities in Arabic to build a com- Wikipedia as well. When trying to apply the exist- prehensive entity catalog together with their con- ing NED approaches on the Arabic text, we face text that is not restricted to the Arabic Wikipedia. the following challenges: Our contributions can be summarized in the follow- ing points: Entity Repository: There is no such compre- • hensive entity catalog. Arabic Wikipedia is an Entity Repository: We switched to • YAGO3(Mahdisoltani et al., 2014), the 4http://en.wikipedia.org/wiki/Zayed bin Sultan Al Nahyan 5 http://ar.wikipedia.org/wiki/àA JîE _ È@_ àA ¢Ê_ áK ._ Y K@P https://www.github.com/yago-naga/aida 188 multilingual version of YAGO2s. YAGO3 mentions and the entities are the contextual similar- comes with a more comprehensive catalog ity between mention’s context and entity’s context. that covers entities from different languages The weights on the edges between the entities are (extracted from different Wikipedia dumps). the semantic relatedness among those entities. In a While we selected YAGO3 to be our back- subsequent process, the graph is iteratively reduced ground knowledge base, any multi-lingual to achieve a dense sub-graph where each mention knowledge base such as Freebase could be is connected to exactly one entity. used as well. The CSAW system uses local scores computed from 12 features extracted from the context sur- Name-Entity Dictionary: We compiled a • rounding the mention, and the candidate entities dictionary from YAGO3 and Freebase to pro- (Kulkarni et al., 2009). In addition, it computes vide the potential candidate entities for each global scores that captures relatedness among anno- mention string. While the mention is in Ara- tations. The NED is then formulated as a quadratic bic, the entity can belong to either the English programming optimization problem, which nega- or the Arabic Wikipedia. tively affects the performance. The software, how- ever, is not available. Entity-Descriptions: We harnessed different • ingredients in YAGO3, and Wikipedia to pro- DBpedia Spotlight uses Wikipedia anchors, ti- duce a rich entity context schema. For the tles and redirects to search for mentions in the input sake of precision, we did not employ any au- text (Mendes et al., 2011). It casts the context of the tomated translation. mention and the entity into a vector-space model. Cosine similarity is then applied to identify the Entity-Entity Relatedness Model: We candidate with the highest similarity. Nevertheless, • fused the link structure of both the English their model did not incorporate any semantic relat- and Arabic Wikipedia’s to compute a com- edness among entities.
Load more

Recommended publications
  • A Survey of Orthographic Information in Machine Translation 3
    Machine Translation Journal manuscript No. (will be inserted by the editor) A Survey of Orthographic Information in Machine Translation Bharathi Raja Chakravarthi1 ⋅ Priya Rani 1 ⋅ Mihael Arcan2 ⋅ John P. McCrae 1 the date of receipt and acceptance should be inserted later Abstract Machine translation is one of the applications of natural language process- ing which has been explored in different languages. Recently researchers started pay- ing attention towards machine translation for resource-poor languages and closely related languages. A widespread and underlying problem for these machine transla- tion systems is the variation in orthographic conventions which causes many issues to traditional approaches. Two languages written in two different orthographies are not easily comparable, but orthographic information can also be used to improve the machine translation system. This article offers a survey of research regarding orthog- raphy’s influence on machine translation of under-resourced languages. It introduces under-resourced languages in terms of machine translation and how orthographic in- formation can be utilised to improve machine translation. We describe previous work in this area, discussing what underlying assumptions were made, and showing how orthographic knowledge improves the performance of machine translation of under- resourced languages. We discuss different types of machine translation and demon- strate a recent trend that seeks to link orthographic information with well-established machine translation methods. Considerable attention is given to current efforts of cog- nates information at different levels of machine translation and the lessons that can Bharathi Raja Chakravarthi [email protected] Priya Rani [email protected] Mihael Arcan [email protected] John P.
    [Show full text]
  • Modeling Popularity and Reliability of Sources in Multilingual Wikipedia
    information Article Modeling Popularity and Reliability of Sources in Multilingual Wikipedia Włodzimierz Lewoniewski * , Krzysztof W˛ecel and Witold Abramowicz Department of Information Systems, Pozna´nUniversity of Economics and Business, 61-875 Pozna´n,Poland; [email protected] (K.W.); [email protected] (W.A.) * Correspondence: [email protected] Received: 31 March 2020; Accepted: 7 May 2020; Published: 13 May 2020 Abstract: One of the most important factors impacting quality of content in Wikipedia is presence of reliable sources. By following references, readers can verify facts or find more details about described topic. A Wikipedia article can be edited independently in any of over 300 languages, even by anonymous users, therefore information about the same topic may be inconsistent. This also applies to use of references in different language versions of a particular article, so the same statement can have different sources. In this paper we analyzed over 40 million articles from the 55 most developed language versions of Wikipedia to extract information about over 200 million references and find the most popular and reliable sources. We presented 10 models for the assessment of the popularity and reliability of the sources based on analysis of meta information about the references in Wikipedia articles, page views and authors of the articles. Using DBpedia and Wikidata we automatically identified the alignment of the sources to a specific domain. Additionally, we analyzed the changes of popularity and reliability in time and identified growth leaders in each of the considered months. The results can be used for quality improvements of the content in different languages versions of Wikipedia.
    [Show full text]
  • Multilingual Ranking of Wikipedia Articles with Quality and Popularity Assessment in Different Topics
    computers Article Multilingual Ranking of Wikipedia Articles with Quality and Popularity Assessment in Different Topics Włodzimierz Lewoniewski * , Krzysztof W˛ecel and Witold Abramowicz Department of Information Systems, Pozna´nUniversity of Economics and Business, 61-875 Pozna´n,Poland * Correspondence: [email protected]; Tel.: +48-(61)-639-27-93 Received: 10 May 2019; Accepted: 13 August 2019; Published: 14 August 2019 Abstract: On Wikipedia, articles about various topics can be created and edited independently in each language version. Therefore, the quality of information about the same topic depends on the language. Any interested user can improve an article and that improvement may depend on the popularity of the article. The goal of this study is to show what topics are best represented in different language versions of Wikipedia using results of quality assessment for over 39 million articles in 55 languages. In this paper, we also analyze how popular selected topics are among readers and authors in various languages. We used two approaches to assign articles to various topics. First, we selected 27 main multilingual categories and analyzed all their connections with sub-categories based on information extracted from over 10 million categories in 55 language versions. To classify the articles to one of the 27 main categories, we took into account over 400 million links from articles to over 10 million categories and over 26 million links between categories. In the second approach, we used data from DBpedia and Wikidata. We also showed how the results of the study can be used to build local and global rankings of the Wikipedia content.
    [Show full text]
  • © 2020 Xiaoman Pan CROSS-LINGUAL ENTITY EXTRACTION and LINKING for 300 LANGUAGES
    © 2020 Xiaoman Pan CROSS-LINGUAL ENTITY EXTRACTION AND LINKING FOR 300 LANGUAGES BY XIAOMAN PAN DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the University of Illinois at Urbana-Champaign, 2020 Urbana, Illinois Doctoral Committee: Dr. Heng Ji, Chair Dr. Jiawei Han Dr. Hanghang Tong Dr. Kevin Knight ABSTRACT Information provided in languages which people can understand saves lives in crises. For example, language barrier was one of the main difficulties faced by humanitarian workers responding to the Ebola crisis in 2014. We propose to break language barriers by extracting information (e.g., entities) from a massive variety of languages and ground the information into an existing Knowledge Base (KB) which is accessible to a user in their own language (e.g., a reporter from the World Health Organization who speaks English only). The ambitious goal of this thesis is to develop a Cross-lingual Entity Extraction and Linking framework for 1,000 fine-grained entity types and 300 languages that exist in Wikipedia. Given a document in any of these languages, our framework is able to identify entity name mentions, assign a fine-grained type to each mention, and link it to an English KB if it is linkable. Traditional entity linking methods rely on costly human annotated data to train supervised learning-to-rank models to select the best candidate entity for each mention. In contrast, we propose a novel unsupervised represent-and-compare approach that can accurately capture the semantic meaning representation of each mention, and directly compare its representation with the representation of each candidate entity in the target KB.
    [Show full text]
  • Mapping Arabic Wikipedia Into the Named Entities Taxonomy
    Mapping Arabic Wikipedia into the Named Entities Taxonomy Fahd Alotaibi and Mark Lee School of Computer Science, University of Birmingham, UK {fsa081|m.g.lee}@cs.bham.ac.uk ABSTRACT This paper describes a comprehensive set of experiments conducted in order to classify Arabic Wikipedia articles into predefined sets of Named Entity classes. We tackle using four different classifiers, namely: Naïve Bayes, Multinomial Naïve Bayes, Support Vector Machines, and Stochastic Gradient Descent. We report on several aspects related to classification models in the sense of feature representation, feature set and statistical modelling. The results reported show that, we are able to correctly classify the articles with scores of 90% on Precision, Recall and balanced F-measure. KEYWORDS: Arabic Named Entity, Wikipedia, Arabic Document Classification, Supervised Machine Learning. Proceedings of COLING 2012: Posters, pages 43–52, COLING 2012, Mumbai, December 2012. 43 1 Introduction Relying on supervised machine learning technologies to recognize Named Entities (NE) in the text requires the development of a reasonable volume of data for the training phase. Manually developing a training dataset that goes beyond the news-wire domain is a non-trivial task. Examination of online and freely available resources, such as the Arabic Wikipedia (AW) offers promise because the underlying scheme of AW can be exploited in order to automatically identify NEs in context. To utilize this resource and develop a NEs corpus from AW means two different tasks should be addressed: 1) Identifying the NEs in the context regardless of assigning those into NEs semantic classes. 2) Classifying AW articles into predefined NEs taxonomy.
    [Show full text]
  • Culture Contacts and the Making of Cultures
    CULTURE CONTACTS AND THE MAKING OF CULTURES Papers in homage to Itamar Even-Zohar Edited by Rakefet Sela-Sheffy Gideon Toury Tel Aviv Tel Aviv University: Unit of Culture Research 2011 Copyright © 2011 Unit of Culture Research, Tel Aviv University & Authors. All rights reserved. No part of this book may be reproduced in any form, by photostat, microform, retrieval system, or any other means, without prior written permission of the publisher or author. Unit of Culture Research Faculty of Humanities Tel Aviv University Tel Aviv, 69978 Israel www.tau.ac.il/tarbut [email protected] Culture Contacts and the Making of Cultures: Papers in Homage to Itamar Even-Zohar / Rakefet Sela-Sheffy & Gideon Toury, editors. Includes bibliographical references. ISBN 978-965-555-496-0 (electronic) The publication of this book was supported by the Bernstein Chair of Translation Theory, Tel Aviv University (Gideon Toury, Incumbent) Culture Contacts and the Making of Cultures: Papers in Homage to Itamar Even-Zohar Sela-Sheffy, Rakefet 1954- ; Toury, Gideon 1942- © 2011 – Unit of Culture Research & Authors Printed in Israel Table of Contents To The Memory of Robert Paine V Acknowledgements VII Introduction Rakefet Sela-Sheffy 1 Part One Identities in Contacts: Conflicts and Negotiations of Collective Entities Manfred Bietak The Aftermath of the Hyksos in Avaris 19 Robert Paine† Identity Puzzlement: Saami in Norway, Past and Present 67 Rakefet Sela-Sheffy High-Status Immigration Group and Culture Retention: 79 German Jewish Immigrants in British-Ruled Palestine
    [Show full text]
  • Automatically Extending Named Entities Coverage of Arabic Wordnet Using Wikipedia
    International Journal on Information and Communication Technologies, Vol. 1, No. 1, June 2008 1 Automatically Extending Named Entities coverage of Arabic WordNet using Wikipedia Musa Alkhalifa and Horacio Rodríguez performed automatically using English WordNet as source Abstract— This paper focuses on the automatic extraction of ontology and bilingual resources for proposing alignments. Arabic Named Entities (NEs) from the Arabic Wikipedia, their automatic attachment to Arabic WordNet, and their automatic Okumura and Hovy [30] proposed a set of heuristics for link to Princeton's English WordNet. We briefly report on the current status of Arabic WordNet, focusing on its rather limited associating Japanese words to English WordNet by means of NE coverage. Our proposal of automatic extension is then an intermediate bilingual dictionary and taking advantage of presented, applied, and evaluated. the usual genus/differentiae structure of dictionary definitions. Index Terms—Arabic NLP, Arabic WordNet, Named Entities Extraction, Wikipedia. I. INTRODUCTION ntologies have become recently a core resource for many O knowledge-based applications such as knowledge management, natural language processing, e-commerce, intelligent information integration, database design and integration, information retrieval, bio-informatics, etc. The Semantic Web, [7], is a prominent example on the extensive Table 1. Content of different versions of Princeton’s English WordNet use of ontologies. The main goal of the Semantic Web is the semantic annotation of the data on the Web with the use of ontologies in order to have machine-readable and machine- Later, Khan and Hovy [20] proposed a way to reduce the understandable Web that will enable computers, autonomous hyper-production of spurious links by this method by software agents, and humans to work and cooperate better searching common hyperonyms that could collapse several through sharing knowledge and resources.
    [Show full text]
  • Critical Point of View: a Wikipedia Reader
    w ikipedia pedai p edia p Wiki CRITICAL POINT OF VIEW A Wikipedia Reader 2 CRITICAL POINT OF VIEW A Wikipedia Reader CRITICAL POINT OF VIEW 3 Critical Point of View: A Wikipedia Reader Editors: Geert Lovink and Nathaniel Tkacz Editorial Assistance: Ivy Roberts, Morgan Currie Copy-Editing: Cielo Lutino CRITICAL Design: Katja van Stiphout Cover Image: Ayumi Higuchi POINT OF VIEW Printer: Ten Klei Groep, Amsterdam Publisher: Institute of Network Cultures, Amsterdam 2011 A Wikipedia ISBN: 978-90-78146-13-1 Reader EDITED BY Contact GEERT LOVINK AND Institute of Network Cultures NATHANIEL TKACZ phone: +3120 5951866 INC READER #7 fax: +3120 5951840 email: [email protected] web: http://www.networkcultures.org Order a copy of this book by sending an email to: [email protected] A pdf of this publication can be downloaded freely at: http://www.networkcultures.org/publications Join the Critical Point of View mailing list at: http://www.listcultures.org Supported by: The School for Communication and Design at the Amsterdam University of Applied Sciences (Hogeschool van Amsterdam DMCI), the Centre for Internet and Society (CIS) in Bangalore and the Kusuma Trust. Thanks to Johanna Niesyto (University of Siegen), Nishant Shah and Sunil Abraham (CIS Bangalore) Sabine Niederer and Margreet Riphagen (INC Amsterdam) for their valuable input and editorial support. Thanks to Foundation Democracy and Media, Mondriaan Foundation and the Public Library Amsterdam (Openbare Bibliotheek Amsterdam) for supporting the CPOV events in Bangalore, Amsterdam and Leipzig. (http://networkcultures.org/wpmu/cpov/) Special thanks to all the authors for their contributions and to Cielo Lutino, Morgan Currie and Ivy Roberts for their careful copy-editing.
    [Show full text]
  • Automatic Linking of Short Arabic Texts to Wikipedia Articles
    Journal of Software Automatic Linking of Short Arabic Texts to Wikipedia Articles Fatoom Fayad1*, Iyad AlAgha2 1 Computer Center, Palestine Technical College-Deir El-Balah, Gaza Strip, Palestine. 2 Faculty of Information Technology, The Islamic University of Gaza, Gaza Strip, Palestine. * Corresponding author. Tel.: 00972592542727; email: [email protected] Manuscript submitted January 10, 2016; accepted March 8, 2016. doi: 10.17706/jsw.11.12.1207-1223 Abstract: Given the enormous amount of unstructured texts available on the Web, there has been an emerging need to increase discoverability of and accessibility to these texts. One of the proposed solutions is to annotate texts with information extracted from background knowledge. Wikipedia, the free encyclopedia, has been recently exploited as a background knowledge to annotate text with complementary information. Given any piece of text, the main challenge is how to determine the most relevant information from Wikipedia with the least effort and time. While Wikipedia-based annotation has mainly targeted the English and Latin versions of Wikipedia, little effort has been devoted to annotate Arabic text using the Arabic version of Wikipedia. In addition, the annotation of short text presents further challenges due to the inability to apply statistical or machine learning techniques that are commonly used with long text. This work proposes an approach for automatic linking of Arabic short texts to articles drawn from Wikipedia. It reports on the several challenges associated with the design and implementation of the linking approach including the processing of the Wikipedia's enormous content, the mapping of texts to Wikipedia articles, the problem of article disambiguation, and the time efficiency.
    [Show full text]
  • A Wikipedia Reader
    UvA-DARE (Digital Academic Repository) Critical point of view: a Wikipedia reader Lovink, G.; Tkacz, N. Publication date 2011 Document Version Final published version Link to publication Citation for published version (APA): Lovink, G., & Tkacz, N. (2011). Critical point of view: a Wikipedia reader. (INC reader; No. 7). Institute of Network Cultures. http://www.networkcultures.org/_uploads/%237reader_Wikipedia.pdf General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl) Download date:05 Oct 2021 w ikipedia pedai p edia p Wiki CRITICAL POINT OF VIEW A Wikipedia Reader 2 CRITICAL POINT OF VIEW A Wikipedia Reader CRITICAL POINT OF VIEW 3 Critical Point of View: A Wikipedia Reader Editors: Geert Lovink
    [Show full text]
  • Analyzing Wikidata Transclusion on English Wikipedia
    Analyzing Wikidata Transclusion on English Wikipedia Isaac Johnson Wikimedia Foundation [email protected] Abstract. Wikidata is steadily becoming more central to Wikipedia, not just in maintaining interlanguage links, but in automated popula- tion of content within the articles themselves. It is not well understood, however, how widespread this transclusion of Wikidata content is within Wikipedia. This work presents a taxonomy of Wikidata transclusion from the perspective of its potential impact on readers and an associated in- depth analysis of Wikidata transclusion within English Wikipedia. It finds that Wikidata transclusion that impacts the content of Wikipedia articles happens at a much lower rate (5%) than previous statistics had suggested (61%). Recommendations are made for how to adjust current tracking mechanisms of Wikidata transclusion to better support metrics and patrollers in their evaluation of Wikidata transclusion. Keywords: Wikidata · Wikipedia · Patrolling 1 Introduction Wikidata is steadily becoming more central to Wikipedia, not just in maintaining interlanguage links, but in automated population of content within the articles themselves. This transclusion of Wikidata content within Wikipedia can help to reduce maintenance of certain facts and links by shifting the burden to main- tain up-to-date, referenced material from each individual Wikipedia to a single repository, Wikidata. Current best estimates suggest that, as of August 2020, 62% of Wikipedia ar- ticles across all languages transclude Wikidata content. This statistic ranges from Arabic Wikipedia (arwiki) and Basque Wikipedia (euwiki), where nearly 100% of articles transclude Wikidata content in some form, to Japanese Wikipedia (jawiki) at 38% of articles and many small wikis that lack any Wikidata tran- sclusion.
    [Show full text]
  • Is Wikipedia Really Neutral? a Sentiment Perspective Study of War-Related Wikipedia Articles Since 1945
    PACLIC 29 Is Wikipedia Really Neutral? A Sentiment Perspective Study of War-related Wikipedia Articles since 1945 Yiwei Zhou, Alexandra I. Cristea and Zachary Roberts Department of Computer Science University of Warwick Coventry, United Kingdom fYiwei.Zhou, A.I.Cristea, [email protected] Abstract include books, journal articles, newspapers, web- pages, sound recordings2, etc. Although a “Neutral Wikipedia is supposed to be supporting the 3 “Neutral Point of View”. Instead of accept- point of view” (NPOV) is Wikipedia’s core content ing this statement as a fact, the current paper policy, we believe sentiment expression is inevitable analyses its veracity by specifically analysing in this user-generated content. Already in (Green- a typically controversial (negative) topic, such stein and Zhu, 2012), researchers have raised doubt as war, and answering questions such as “Are about Wikipedia’s neutrality, as they pointed out there sentiment differences in how Wikipedia that “Wikipedia achieves something akin to a NPOV articles in different languages describe the across articles, but not necessarily within them”. same war?”. This paper tackles this chal- Moreover, people of different language backgrounds lenge by proposing an automatic methodology based on article level and concept level senti- share different cultures and sources of information. ment analysis on multilingual Wikipedia arti- These differences have reflected on the style of con- cles. The results obtained so far show that rea- tributions (Pfeil et al., 2006) and the type of informa- sons such as people’s feelings of involvement tion covered (Callahan and Herring, 2011). Further- and empathy can lead to sentiment expression more, Wikipedia webpages actually allow to con- differences across multilingual Wikipedia on tain opinions, as long as they come from reliable au- war-related topics; the more people contribute thors4.
    [Show full text]