• Abstract and Figures
• Public Full-text

Requirement evaluation for expert-based ranking of web interface environments: the ZEEF.com case study Submitted in partial fulfillment of the degree of Master of Science Yana Ledeneva 10837418 Master Information Sciences Business Information Systems Faculty of Science University of Amsterdam June 16, 2015 Thesis Supervisor: Second Reader: Dr. Frank Nack Dr. Peter Weijland Table of Contents Abstract ...................................................................................................................................... 2 1. Introduction ......................................................................................................................... 2 2. Related work and definitions ..................................................................................................... 3 2.1 Overview of content curation .................................................................................................. 3 2.2 Ranking of web-pages ............................................................................................................ 3 2.2.1 Definition of ranking ....................................................................................................... 3 2.2.2 Search engines ranking .................................................................................................... 4 2.2.3 Human-based ranking ...................................................................................................... 4 2.2.4 Factors for evaluating web pages ....................................................................................... 4 2.3 Concepts of trust and agreement .............................................................................................. 6 2.4 Conclusion and research gap ................................................................................................... 7 3. Conceptual Framework and research question ............................................................................ 7 4. The ZEEF use case ................................................................................................................... 9 4.1 Why ZEEF .........................................................................................................................10 4.2 Survey for the curators ..........................................................................................................10 4.2.1 Design .........................................................................................................................10 4.2.2 Participants ...................................................................................................................11 4.3 Survey for the users ..............................................................................................................11 4.3.1 Design .........................................................................................................................11 4.3.2 Participants ..................................................................................................................13 4.4. Results of the surveys ..........................................................................................................14 4.4.1 The process of creating a curator page ...............................................................................14 4.4.2 Commission and fraud ..................................................................................................15 4.4.3 Ranking factors ............................................................................................................16 4.4.4 Visitors’ general perception ...........................................................................................16 4.4.5 Conclusions .................................................................................................................20 5. Interface experiment ................................................................................................................20 5.1 Design ...............................................................................................................................20 5.2 Results ...............................................................................................................................21 5.3 Discussion ..........................................................................................................................22 6. Overall discussion ....................................................................................................................22 6.1. General discussion and limitations ..........................................................................................22 6.2. Guidelines for the curators ....................................................................................................24 7. Conclusions and future work ....................................................................................................26 Appendix 1. Preliminary list of questions for surveys ......................................................................29 Appendix 2. Table with ranking factors .........................................................................................34 Requirement evaluation for expert-based ranking of web interface environments: the ZEEF.com case study Yana Ledeneva 10837418 Abstract This research is aimed to analyze the process of creation of the expert-based rankings and their perception by the users of a human-based search environment. The first part of the research is focused on the investigation of the main criteria that influence the ranking of websites in human-based search engines, as well as the main factors that influence trust, agreement and general perception of these rankings by the visitors. The second part of the research covers the analysis of the main use case of this study - a human-based search engine ZEEF.com, in which the curators create their pages and establish the rankings of web sites for one or several topics. The empirical part of this study was conducted with the help of two surveys. The first survey addressed how curators of ZEEF pages generate their pages to check the reasoning behind, the effort invested, and the time needed to establish a ranking. The second survey addressed existing and potential visitors of ZEEF pages to check how, in their opinion, the rankings were established, and whether the results are satisfactory and trustworthy. Based on this evaluation, the list of the main ranking factors that were taken into account by both parties was developed. Finally, an interface test was performed to check which aspects of web pages influence the perception of the quality of the page by the visitors. As a result, recommendations for the curators of ZEEF pages regarding the process of creation of ZEEF pages were designed. 1. Introduction In the era of information overload and significant use of search engines, the problem of filtering the world’s information is getting more and more important. However, the algorithmic search that is implemented in the main search engines, social networks and other websites sometimes does not provide a 100% retrieval quality. Taking into account that people usually tend to trust other people more than machines, especially if other people are experts in their field, it is possible to come up with a better alternative to algorithmic search: a human-based search environment. Nowadays such an environment is available in form of a content curation or human-based search engine. One example of such a search engine is ZEEF.com. However, the main algorithm of ZEEF - a human-based ranking of the links to different websites - has its risks. For instance, the curator may be too subjective or may use ranking factors that do not align with those applied by visitors of the page, or the visitor may distrust the curator, his page and/or the platform in general. Therefore, in order to analyze the process of ranking websites in a human-based search environment in general, and to reduce the business risks of ZEEF company in particular, the main factors and parameters that are taken into account by the curators for making the lists of links, and their methods of making rankings need to be analyzed. The aim of this research is to understand what influences the perception of the quality of a human- based search environment by the visitors, and to check the difference between the ranking criteria curators take into account, and the criteria visitors think should be taken. Significant difference between these criteria may lead to possible misunderstandings, which in turn may lead to decrease of quality of the content from visitors’ point of view. 2 2. Related work and definitions 2.1 Overview of content curation Due to information overload these days it becomes much more challenging to find the information one is interested in (Stephen Dale, 200). One of the solutions to this problem is content curation, which becomes more popular as a way to filter a massive torrent of information (Nancy K. Herther, 2012a). Content curation is “the process of sifting through information on the web, organizing, filtering and making sense of it and sharing the best and most relevant content on a specific issue within your network”. (Nancy K. Herther, 2012b, p. 41) A content curator is a person who continually finds, groups, organizes, and shares the best and most relevant content on a specific issue online (Nancy K. Herther, 30). It is worth mentioning that the key component of this definition

Load more

  35

Copy Link