Human Computation and Crowdsourcing Meet the Semantic Web: a Survey
Total Page:16
File Type:pdf, Size:1020Kb
Undefined 1 (2009) 1–5 1 IOS Press Human Computation and Crowdsourcing meet the Semantic Web: A Survey Editor(s): Name Surname, University, Country Solicited review(s): Name Surname, University, Country Open review(s): Name Surname, University, Country Amna Basharat a;∗, I. Budak Arpinar a and Khaled Rasheed a a Department of Computer Science, University of Georgia, Athens, GA, USA E-mail: {amnabash,budak, khaled}@uga.edu Abstract. Challenges associated with large-scale adoption of semantic web technologies continue to confront the researchers in the field. Researchers have recognized the need for human intelligence in the process of semantic content creation and analytics, which forms the backbone of any semantic application. Realizing the potential that human computation, collective intelligence and the fields of the like such as crowdsourcing and social computation have offered, semantic web researchers have effec- tively taken up the synergy to solve the bottlenecks of human experts and the needed human contribution in the semantic web development processes. In this paper, we present a comprehensive survey of the intersection of semantic web and the human computation paradigm. We adopt a two fold approach towards understanding this intersection. As the primary focus, we analyze how the semantic web domain has adopted the dimensions of human computation to solve the inherent problems. We present an in-depth analysis of the need for human computation in semantic web tasks such as ontology engineering and linked data management. We provide a ’collective intelligence genome’ adapted for the semantic web as means to analyze the threads of composing semantic web applications using human computation methods. As a secondary contribution we also analyze existing research efforts through which the human computation domain has been better served with the use of semantic technologies. We present a comprehensive view of the promises and challenges offered by the successful synergy of semantic web and human computation. In conclusion, we discuss several key outstanding challenges and propose some open research directions. Keywords: semantic web, ontologies, crowdsourcing, human computation 1. Introduction tion capabilities in an effort to solve the inherent prob- lems. Semantic web visionists have put forth a number 1.1. Research Context of visionary ideas for the road ahead for the success of the semantic web. The notion of ’The Global Brain After more than a decade of semantic web research, Semantic Web’ [10] - a semantic Web interleaving a researchers remain challenged by the large scale adop- large number of human and machine computation - has tion of the semantic technologies. Semantic technolo- come to be seen as a vision with great potential to over- gies have been deployed in the context of a wide range come some of the issues of the current semantic web of information management tasks, for which machine- . This idea of interleaved human-machine computa- driven algorithmic techniques aiming at full automa- tion has already resulted in successful systems that are tion do not reach a level of accuracy and reliability to able to solve problems in manner and ways unthink- ensure usable systems. Researchers have started aug- able for either computers or machines to be able to menting automatic techniques with human computa- solve alone. The domain of human computation, col- lective intelligence, social computing and crowdsourc- *Corresponding author. E-mail: [email protected]. ing have all contributed to this successful synergy of 0000-0000/09/$00.00 © 2009 – IOS Press and the authors. All rights reserved 2 Basharat et al. / Human Computation and Crowdsourcing meet the Semantic Web: A Survey humans and machines and contribute to the constantly mantic web and human compution will bear its own set evolving metaphor of the ’Global Brain’ [36]. Much of challenges. Both semantic web and human compu- like the concept of the programming needed for the tation seem to have a long way to go before being fully ’Global Brain’ [11], the ’Global Brain Semantic Web’ able to reap the benefits promised by the intersection will need new strands of programming, workflows and with the other [23]. challenges to be accomplished. The challenge for the semantic web community, is 1.2. Contributions to rethink the original semantic web vision, which was largely built on the vision of computers populating the The first contribution of this paper is to provide a web of machines [10]. Researchers have recognized review of the challenges that the semantic web domain the need for human intelligence in the process of se- has faced especially in terms of the need for human in- mantic content creation [98] which forms the back- tervention. Secondly, we analyze the intersection of se- bone of any semantic application. The entrance bar- mantic web and the human computation paradigm. We rier for many semantic applications is said to be high, adopt a two fold approach towards understanding this given the dependence on expertise in knowledge engi- intersection. As the primary focus, we analyze how neering, logics and more. In short, semantic web lacks the semantic web domain has adopted the dimensions the sufficient user involvement in various aspects. Hu- of human computation to solve the inherent problems. mans are simply considered indispensable[23] for the We present a review of studies and applications span- semantic web to realize its full potential. ning the two most common genres in human compu- Realizing the potential that human computation, tation namely Games With A Purpose (GWAP) and collective intelligence and the fields of the like such Micro-Task Crowdsourcing. We also adopt a classifi- as crowdsourcing and social computation have offered, cation scheme in the form of ’collective intelligence semantic web researchers have attempted to effectively genome’ and apply it to some of the key studies and taken up the synergy to solve the bottlenecks of hu- approaches that combine semantic web, human com- man experts and the needed human contribution in se- putation and crowdsourcing. This genome is adapted mantic web development processes. Semantic web re- from the original collective intelligence genome pro- search can be seen as experiencing a shift from increas- posed by [60]. We apply it specifically to the context ingly expert driven to one embracing the larger com- munity and the users involved in the semantic content of the semantic web with the aim to provide useful in- creation process. Some early efforts that led to the evo- sights in analyzing the various threads that constitute lution of this approach includes myOntology [99] and the design of studies for creating semantic web driven inPho [75]. by human computation. These threads are considered Two major genres of research may be seen emerg- useful for possible further investigation to be taken up ing in the last few years, in an attempt to bring human be researchers. At the same time, it is also meant to computation methods to the semantic web: 1) Mech- serve as means of analyzing the strengths and weak- anized Labour and 2) Games with a Purpose for the nesses of existing researches. Semantic Web. In this paper, we not only take a de- Recent research in crowdsourcing and semantic web tailed look at the challenges leading to the adoption of has also seen the emergence of some workflow sys- the human computation methods for the semantic web, tems designed to meet the need of providing a generic we also provide a comprehensive coverage of the ap- framework for automating human-machine computa- proaches in these mentioned genres. On a parallel note, tion workflows. We undertake a comparative analysis human computation systems can also potentially bene- of few of the most prominent studies to this end, and fit from the promises offered by the semantic web. The highlight the essential dimensions constituting these next generation of human computation systems are be- workflow systems. We give special mention to these ing envisioned that go beyond the platform they were systems for they seem closer to the directions that we built on, offering data reusability in ways unintended expect to see in the future that awaits this emerging by their creators [23]. Semantic web may be seen as research domain. means of providing better user continuity and platform As a secondary contribution, we also analyze exist- consistency across human computation systems. ing research efforts through which the human compu- While the potential is clearly evident in going about tation domain has been better served with the use of such a synergy, effectively realizing the synergy of se- semantic technologies. Basharat et al. / Human Computation and Crowdsourcing meet the Semantic Web: A Survey 3 We also present a comprehensive view of the promises highlighting the distinctions and similarities based on and challenges offered by the successful synergy of several factors such as motivation, quality control, ag- semantic web and human computation. gregation, process order and task request cardinality We hope that this review will serve as basis for ex- [81]. ploring newer threads of synergy between the seman- What is Collective Intelligence? Collective intelli- tic web and human computation research, resulting in gence is an encompassing term to broadly refer to the creation of better applications and approaches that groups of individuals doing things collectively that advance both domains. seem intelligent [60,61]. This idea of loosely orga- nized group of people accomplishing more than what individuals can alone has been garnering a lot of inter- 2. Background est within the research community. What is Social Computing? Technologies such as In this section, we provide a short introduction on blogs, wikis, and online communities are examples of some aspects of the theoretical foundations and basics social computing [81]. The scope is broad, but always of human computation and crowdsourcing, semantic includes humans in a social role where communication web, and define concepts that will be used throughout is mediated by technology.