Master Thesis
Total Page:16
File Type:pdf, Size:1020Kb
MASTER THESIS TITLE : Cultural configuration of Wikipedia: measuring Autoreferentiality in different languages MASTER DEGREE: Master of Science in Telecommunication Engineering & Management AUTHOR: Marc Miquel Ribe´ DIRECTOR: Horacio Rodr´ıguez Hontoria TUTOR: Sebastia` Sallent Ribes DATE: March 31, 2011 T´ıtol : Cultural configuration of Wikipedia: measuring Autoreferentiality in different languages Autor: Marc Miquel Ribe´ Director: Horacio Rodr´ıguez Hontoria Tutor: Sebastia` Sallent Ribes Data: 31 de marc¸de 2011 Resum ”Wikipedia es´ un projecte enciclopedic` multiling¨ue, col·laboratiu, basat en web i sense anim` de lucre impulsat per la Fundacio´ Wikimedia”, aix´ı es´ com s’autodescriu Wikipedia en la definicio´ de l’article que du el seu nom. Aixo` significa que l’enciclopedia` pot ser modificada en qualsevol moment, per qualsevol persona i des de qualsevol lloc. Aquestes premisses i la seva gran participacio´ fan que es tracti d’un excel·lent objecte social d’estudi, que a la vegada, per tractar-se d’un artefacte tecnologic,` permeti tambe´ l’´us de tecniques` de processament llenguatge natural, obtencio´ i mineria de dades. Tanmateix, en la recerca actual hi ha una clara mancanc¸a en software que pugui aproximar-s’hi d’una manera integral. Tenint en compte aquest buit realitzem una caracteritzacio´ de Wikipedia amb l’objectiu de coneixer` a fons quins son´ els elements i estructures d’informacio´ que conte´ i com despres´ poden obtenir-se mitjanc¸ant una eina anal´ıtica. Partim de l’API existent anomenada wikAPIdia, que desenvolupem fins a incloure-hi noves funcionalitats i posar-la apunt per a encarar m´ultiples escenaris i problematiques` de les ciencies` socials. Com a cas practic,` revisem l’estat de l’art pel que` fa a la motivacio´ dels editors i la cobertura tematica` del repositori. Aixo` ens permet considerar l’objectiu de coneixer` Wikipedia des del punt de vista de tenir una configuracio´ cultural ´unica per a cada llengua. Formulat com a pregunta: ”existeix una motivacio´ nacional o autorepresentativa que es reflecteixi en els continguts i els disposi diferenciadament?” Autoreferencialitat es´ el concepte que presentem per tal d’analitzar aquest hipotetic` interes` superior reflectit en el contingut local. Realitzem una identificacio´ i recol·leccio´ dels articles que poden referir-se tant a la historia,` als equips esportius o la cultura pop, entre d’altres, pero` que mantenen un lligam semantic` clar entorn l’ambit` dels editors. Posteriorment, en plantegem un analisi` multidimensional fins a arribar a indicadors significatius, que permetin arribar a conclusions comunes i a la vegada compondre un ´ındex per a la comparacio.´ En darrer lloc, avaluem quin es´ l’impacte d’aquest contingut i la necessitat de que la seva existencia` sigui tinguda en compte en el disseny d’aplicacions basades en repositoris de coneixement que donin servei a usuaris. Paraules clau: Viquipedia,` Desenvolupament del Software, Web Col·laborative, Cobertura Tematica,` Mine- ria de Dades. Title : La configuracio´ cultural de Wikipedia:` mesurant l’Autoreferencialitaten diferents versions ling¨u´ıstiques Author: Marc Miquel Ribe´ Director: Horacio Rodr´ıguez Hontoria Tutor: Sebastia` Sallent Ribes Date: March 31, 2011 Abstract ”Wikipedia is a free web-based, collaborative, multilingual encyclopedia project supported by the non-profit Wikimedia Foundation” this is the way the definition of Wikipedia in the article of the English language edition starts. This means it can be modified at any time, by anyone and at any place. These bases and their participation success make of Wikipedia an excellent social object of study which, at the same time, for being a technological construct, can be approached by techniques of natural language processing, information retrieval or data mining. However, in the current research there is a clear lack of software which can make an integral approach. Taking this into account, we make an in depth characterization of Wikipedia with the end goal of under- standing which elements and structures compound its data and how they can be obtained with an analytical tool. We start with the existing API called wikAPIdia, which we develope until include new functionalities and have it ready to use in multiple scenarios and problematics of social sciences. Looking for a practical case to test it, we review the current state of art in motivation of editors and the topical coverage in the repository. This allows us to consider the aim of understanding Wikipedia from the perspective of having a different cultural configuration for each language. Phrasing it as a question: ”is there a national or self-representative motivation which is reflected in the content and thus disposes them differenciately?” Autoreferentiality is the concept we present in order to analyse this hypothetical higher interest in local content. An identification and recollection is made on articles from heterogenous topics which can refer to the local history, sport teams or pop culture, but still maintain a semantic relation to the context of editors. Later, we propose a multidimensional analysis of them on features which can be significant indicators, to reach common conclusions and evaluate the language editions through an index of Autoreferentiality. Last, we point out which is the impact of this content and the risk of not considering its existance in the design of applications based on user generated content. Keywords: Wikipedia, Software Development, Collaborative Web, Topical Coverage, Data Mining. To my beloved family. To Diana. To my friends. ”The goal is to give people a free encyclopedia to every person in the world, in their own lan- guage. Not just in a ’free beer’ kind of way, but also in the free speech kind of way.” Jimmy Wales, Wikipedia co-founder ”But this emphasis would be lavished in vain, if it served, in your opinion, only to abstract a general type from phenomena whose particularity in our work would remain the essential thing for you, and whose original arrangement could be broken up only artificially.” Jacques Lacan, Psychoanalist Seminar on The Purloined Letter CONTENTS PREFACE ................................................ 1 GOALS ................................................. 3 CHAPTER 1. Introduction ..................................... 5 1.1. Wikipedia .............................................. 5 1.1.1. Free, web-based, collaborative, multilingual encyclopedia ................ 5 1.1.2. Governance ................................... ..... 5 1.1.3. Hyperlingua and Viquipedia................................` 6 1.1.4. Technology ................................... ..... 7 1.2. Related research .......................................... 9 1.2.1. Technicalapproach . ........ 10 1.2.2. Culturalconfiguration . .......... 11 CHAPTER 2. Approach ...................................... 15 2.1. Technical characterization and model ............................... 15 2.1.1. Structuresandcharacteristics . .............. 15 2.1.2. Otherindicatorsandmethodology . ............ 17 2.1.3. Workset ...................................... .... 19 2.2. Autoreferentiality case ....................................... 20 2.2.1. Analysisdimensions. ......... 20 2.2.2. Languageeditions. ........ 22 2.2.3. Setofarticles ................................ ....... 22 2.2.4. Hypothesis................................... ...... 25 CHAPTER 3. Design and implementation .......................... 29 3.1. Requirements ............................................ 29 3.2. Framework ............................................. 29 3.2.1. Mainclasses.................................. ...... 29 3.2.2. Database ..................................... .... 31 3.3. Implementations .......................................... 32 3.3.1. Newfeatures .................................. ..... 32 3.3.2. Worksetpackage . ... .... ... .... ... .... ... .... .. ...... 33 3.3.3. Storingandretrievingscripts . ............. 34 3.3.4. SetProcessingtools . ......... 35 3.4. Infrastructure ............................................ 35 3.4.1. Initialscenarios . ......... 36 3.4.2. Cluster...................................... ..... 36 3.4.3. Cost ......................................... .. 37 CHAPTER 4. Results ........................................ 41 4.1. Hyperlingua ............................................. 41 4.1.1. Hypothesistesting. ......... 42 4.1.2. Indicatorevaluation . .......... 49 4.1.3. Autoreferencialityindex . ............ 51 4.2. Viquipedia` .............................................. 52 4.2.1. Top20mostratedarticles. .......... 52 4.2.2. Typology..................................... ..... 52 CHAPTER 5. Conclusions ..................................... 55 5.1. Discussion and achieved goals .................................. 55 5.1.1. Technical.................................... ...... 55 5.1.2. Scientific.................................... ...... 56 5.1.3. Social ....................................... .... 57 5.2. Conclusions and future lines ................................... 58 ACKNOWLEDGEMENTS ...................................... 61 TERMINOLOGY AND ACRONYMS ................................ 63 BIBLIOGRAPHY ........................................... 65 APPENDIX A. Complementary results ............................. 1 A.1. Levels analysis ........................................... 1 A.1.1. H1........................................... ... 1 A.1.2. H2........................................... ... 2 A.1.3. H3........................................... ..