DOCSLIB.ORG

Deliverable 3.5: Report on User and Usability Testing (1)

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Deliverable 3.5: Report on User and Usability Testing (1) ECP-2007-DILI-517009 Europeana Local Deliverable 3.5: Report on user and usability testing (1) Deliverable number D 3.5 PP; Restricted to other programme Dissemination level participants (incl. Commission services and project reviewers) Delivery date 13 November 2011 Status Final Author(s) Olga McHenry eContentplus This project is funded under the eContentplus programme, a multiannual Community programme to make digital content in Europe more accessible, usable and exploitable. Contents EuropeanaLocal Advanced User Test Group ...................................................................................................... 3 1 Introduction .............................................................................................................................................. 4 2 Search and results ..................................................................................................................................... 5 Search APIs ................................................................................................................................................... 5 Query context ............................................................................................................................................... 5 The query mechanism ................................................................................................................................... 5 Result specification ....................................................................................................................................... 6 Advanced search ........................................................................................................................................... 6 Word autocompletion ................................................................................................................................... 7 Alternative spellings ..................................................................................................................................... 7 Query expression specification for advanced search ...................................................................................... 7 Tips and suggestions ..................................................................................................................................... 7 3 Browse ...................................................................................................................................................... 7 Spatial, person, index browsing ..................................................................................................................... 7 SubJect cloud ................................................................................................................................................ 8 Related items ................................................................................................................................................ 8 People are currently thinking about .............................................................................................................. 8 Visibility of individual data providers ............................................................................................................ 8 4 Distribution – APIs .................................................................................................................................... 9 Extended search ........................................................................................................................................... 9 Query expansion ........................................................................................................................................... 9 OpenSearch API ............................................................................................................................................ 9 Zoom (present image or a part of it with a specified scale) ............................................................................. 9 5 Facilitation: Europeana Labs ................................................................................................................... 10 Technical requirements .............................................................................................................................. 10 6 Content Ingestion .................................................................................................................................... 10 Content checker .......................................................................................................................................... 11 OAI-PMH .................................................................................................................................................... 11 Continuity of ingestion process and information about it ............................................................................. 12 Editorial Virtual Exhibitions ........................................................................................................................ 12 7 Editorial Virtual Exhibitions .................................................................................................................... 12 8 Future developments .............................................................................................................................. 12 Tagging/annotation .................................................................................................................................... 13 Infrastructure for automatic upload mechanism .......................................................................................... 13 Quality of content ....................................................................................................................................... 13 Reports for content providers on attracted traffic and sources of this traffic ................................................ 13 Annex 1 Questions and issues raised by the content providers ....................................................................... 14 Ontologies .................................................................................................................................................. 14 Query expansion ......................................................................................................................................... 14 Clustering on the fly .................................................................................................................................... 14 Target group of Rhine/Danube Specifications ............................................................................................. 14 Collection, normalization and data enrichment ............................................................................................ 14 Query mechanism ....................................................................................................................................... 14 Annex 2 EuropeanaLocal Advance User Testing Specifications ........................................................................ 15 Page 2 of 31 EuropeanaLocal Advanced User Test Group The following 8 EuropeanaLocal content providers took part in the advanced user testing: v ABM-Utvikling, NO v Angewandte Informationstechnik Forschungsgesellschaft mbH, AT v Ministry of Culture, Spain, ES v Narodna in univerzitetna knJižnica, SI v PAN - Poznańskie Centrum Superkomputerowo-Sieciowe, PL v Provincie Limburg, BE v Public Library 'Pencho Slaveykov' (Public Library of Varna), BG v Sogn og FJordane County Municipality, NO Page 3 of 31 1 Introduction This report is a part of EuropeanaLocal D3.5 ‘Report on user and usability testing’ reflecting the results of Advanced User Testing (Test Phase 1). EuropeanaLocal user testing is a part of work conducted under the EuropeanaLocal Evaluation Work Group (EWG). The EuropeanaLocal user testing consists of 2 phases: TP1 – Advanced User Testing, and TP2 – End-User Testing. This report covers solemnly TP1 – Advanced User Testing. Thus, EuropeanaLocal D3.5 consists of 2 individual reports corresponding to described above test phases. EuropeanaLocal user testing work plan was initially introduces to all EuropeanaLocal partners at the proJect mid-term meetings in Poznan in December 2009. In March 2010 it was presented at Europeana meeting in Paris and was successfully validated by Europeana V1.0 WG ‘Users’. Following this, the Advanced User Testing Specifications were prepared by EuropeanaLocal to assist the test group. Those were validated at EWG meeting in Lisbon in July 2010 (Annex 2). The testing took place between July and October 2010. As additional task EuropeanaLocal also provided feedback to the first draft of Danube specifications released in September 2010. The feedbacK was included in the report produced under Europeana V 1.0 ‘Validation’ WG. Test Phase 1 (TP1) was focusing on assessment of the functionalities that were coming up in the new Europeana portal (Rhine release). The objective of TP1 was to identify which functionalities were missing and could be added to Europeana portal in order to maKe it more attractive specifically to the local and regional content providers. Thus, the report consists of feedbacKs and remarKs Jointly produced by the test group in relation to main functionalities of Europeana portal (after Rhine). The test group for the TP1 was a small focused group where the test users were selected from the advanced EuropeanaLocal
Load more

Recommended publications
  • NEXT GENERATION CATALOGUES: an ANALYSIS of USER SEARCH STRATEGIES and BEHAVIOR by FREDRICK KIWUWA LUGYA DISSERTATION Submitted I
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Illinois Digital Environment for Access to Learning and Scholarship Repository NEXT GENERATION CATALOGUES: AN ANALYSIS OF USER SEARCH STRATEGIES AND BEHAVIOR BY FREDRICK KIWUWA LUGYA DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Library and Information Science in the Graduate College of the University of Illinois at Urbana-Champaign, 2017 Urbana, Illinois Doctoral Committee: Associate Professor Kathryn La Barre, Chair and Director Assistant Professor Nicole A. Cooke Dr. Jennifer Emanuel Taylor, University of Illinois Chicago Associate Professor Carol Tilley ABSTRACT The movement from online catalogues to search and discovery systems has not addressed the goals of true resource discoverability. While catalogue user studies have focused on user search and discovery processes and experiences, and construction and manipulation of search queries, little insight is given to how searchers interact with search features of next generation catalogues. Better understanding of user experiences can help guide informed decisions when selecting and implementing new systems. In this study, fourteen graduate students completed a set of information seeking tasks using UIUC's VuFind installation. Observations of these interactions elicited insight into both search feature use and user understanding of the function of features. Participants used the basic search option for most searches. This is because users understand that basic search draws from a deep index that always gives results regardless of search terms; and because it is convenient, appearing at every level of the search, thus reducing effort and shortening search time.
    [Show full text]
  • A Framework for Evaluating the Retrieval Effectiveness of Search Engines Dirk Lewandowski Hamburg University of Applied Sciences, Germany
    1 A Framework for Evaluating the Retrieval Effectiveness of Search Engines Dirk Lewandowski Hamburg University of Applied Sciences, Germany This is a preprint of a book chapter to be published in: Jouis, Christophe: Next Generation Search Engine: Advanced Models for Information Retrieval. Hershey, PA: IGI Global, 2012 http://www.igi-global.com/book/next-generation-search-engines/59723 ABSTRACT This chapter presents a theoretical framework for evaluating next generation search engines. We focus on search engines whose results presentation is enriched with additional information and does not merely present the usual list of “10 blue links”, that is, of ten links to results, accompanied by a short description. While Web search is used as an example here, the framework can easily be applied to search engines in any other area. The framework not only addresses the results presentation, but also takes into account an extension of the general design of retrieval effectiveness tests. The chapter examines the ways in which this design might influence the results of such studies and how a reliable test is best designed. INTRODUCTION Information retrieval systems in general and specific search engines need to be evaluated during the development process, as well as when the system is running. A main objective of the evaluations is to improve the quality of the search results, although other reasons for evaluating search engines do exist (see Lewandowski & Höchstötter, 2008). A variety of quality factors can be applied to search engines. These can be grouped into four major areas (Lewandowski & Höchstötter, 2008): • Index Quality: This area of quality measurement indicates the important role that search engines’ databases play in retrieving relevant and comprehensive results.
    [Show full text]
  • E-Book: an Introduction to Custom Search Engines for Websites
    E-book: An Introduction to Custom Search Engines for Websites What are you looking for? Search www.sitesearch360.com 1 Contents An introduction to search engines........................................ 4 Your visitor’s search journey: the key elements................ 23 What is site search?................................................................ 6 Presentation — how does it look?................................................ 24 Why is site search important?............................................... 6 Processing — how does it work?.................................................. 28 Navigation vs. search................................................................. 7 Results — how does it deliver?.................................................... 32 How are people using search?............................................... 9 Implementation challenges.................................................... 36 Why are custom search engines the perfect choice?........ 12 JavaScript................................................................................ 36 Semantic search........................................................................ 12 Security.................................................................................. 37 Natural search language............................................................ 13 WordPress............................................................................... 37 Actionable analytics.................................................................. 14 Cloudflare and
    [Show full text]
  • Natural Language Processing for Online Applications Text Retrieval, Extraction and Categorization
    Natural Language Processing for Online Applications Natural Language Processing Editor Prof. Ruslan Mitkov School of Humanities, Languages and Social Sciences University of Wolverhampton Stafford St. Wolverhampton WV1 1SB, United Kingdom Email: [email protected] Advisory Board Christian Boitet (University of Grenoble) Jn Carroll (University of Sussex, Brighton) Eugene Charniak (Brown University, Providence) Eduard Hovy (Information Sciences Institute, USC) Richard Kittredge (University of Montreal) Geoffrey Leech (Lancaster University) Carlos Martin-Vide (Rovira i Virgili Un., Tarragona) Andrei Mikheev (University of Edinburgh) Jn Nerbonne (University of Groningen) Nicolas Nicolov (IBM, T.J. Watson Research Center) Kemal Oflazer (Sabanci University) Allan Ramsey (UMIST, Manchester) Monique Rolbert (Université de Marseille) Richard Sproat (AT&T Labs Research, Florham Park) K-Y Su (Baviour Design Corp.) Isabelle Trancoso (INESC, Lisbon) Benjamin Tsou (City University of Hong Kong) Jun-ichi Tsujii (University of Tokyo) Evene Tzoukermann (Bell Laboratories, Murray Hill) Yorick Wilks (University of Sheffield) Volume 5 Natural Language Processing for Online Applications: Text Retrieval, Extraction and Categorization by Peter Jackson and Isabelle Moulinier Natural Language Processing for Online Applications Text Retrieval, Extraction and Categorization Peter Jackson Isabelle Moulinier omson Legal & Regulatory John Benjamins Publishing Company Amsterdam / iladelia TM The paper used in this publication meets the minimum requirements of American 8 National Standard for Information Sciences – Permanence of Paper for Printed Library Materials, ansi z39.48-1984. Library of Congress Cataloging-in-Publication Data Jackson, Peter, 1948- Natural language processing for online applications : text retrieval, extraction, and categorization / Peter Jackson, Isabelle Moulinier. p.cm.(Natural Language Processing, issn 1567–8202 ; v.5) Includes bibliographical references and index.
    [Show full text]
  • AIIP Connections Is the Official Newsletter for Members of the Association of Independent Information Professionals
    Connections Volume 22 No . 6 November/December 2008 President’s Message Ed Vawter, QD Information IN THIS ISSUE I want to take this opportunity to wish everyone Happy Holidays and hope that President’s Message . 1 you and your family are doing well. Footnotes . 2 ow, time has flown by and winter new look. We are also still working on is nearly upon us. I can’t believe the website and adding some Web 2, Opportunities in a W over half my term as President of AIIP including the podcasts I have done in Changing World . 4 is past. Given that our conference is the past, and including the new blog The Perils of Blogging earlier than usually next year, I actually called i to i. the News . 6 only get to serve 11 months instead of a I mentioned the conference earlier and full year. I wish I had more time to get What is Fair Use? . 9 just wants to reminder everyone that the many things done that myself and it is from March 26-29 at the Marriott the board have planned but I will have WordWise . 10 in Albuquerque, New Mexico, USA. I the pleasure of continuing on the board understand that the economy right now The Four Phases for one more year under the leadership does not look promising but I would of a Project . .13 of Marcy Phelps. I’ve been very happy encourage you to start putting away with what we have accomplished and I Coach’s Corner . .15 some money each week to enable you to think the whole board would agree that attend.
    [Show full text]
  • A Search Engine Features Comparison. PUB DATE 1997-08-00 NOTE 48P.; Master's Research Paper, Kent State University
    DOCUMENT RESUME ED 413 900 IR 056 730 AUTHOR Vorndran, Gerald TITLE A Search Engine Features Comparison. PUB DATE 1997-08-00 NOTE 48p.; Master's Research Paper, Kent State University. PUB TYPE Dissertations/Theses (040) -- Reports Evaluative (142) EDRS PRICE MF01/PCO2 Plus Postage. DESCRIPTORS Access to Information; Computer Interfaces; Databases; *Online Searching; *Online Systems; Online Vendors; Reference Services; Relevance (Information Retrieval); Search Strategies; *User Needs (Information); *World Wide Web IDENTIFIERS *DIALOG; *Search Engines ABSTRACT Until recently, the World Wide Web (WWW) public access search engines have not included many of the advanced commands, options, and features commonly available with the for-profit online database user interfaces, such as DIALOG. This study evaluates the features and characteristics common to both types of search interfaces, examines the Web search interfaces to define lingering deficiencies as compared to the online interfaces, and presents suggestions for improvement to those areas of the Web interfaces found lacking. The most advanced interface features of the AltaVista, Excite, HotBot, and Infoseek Web search interfaces were compared to the DIALOG interface features. The Web search interfaces, as a whole, still trail the DIALOG search interface in terms of the quality, quantity, depth (robustness), and usability of the search system. Appendices include background information, search parameters, and output for the Web engines, and for DIALOG. (Contains 46 references.) (Author/SWC) ******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ******************************************************************************** A Search Engine Features Comparison A Master's research Paper submitted to the Kent State University School of Library Science and Information Science in partial fulfillment of the requirements for the degree Master of Library Science by Gerald Vorndran August, 1997 OF EDUCATION U.S.
    [Show full text]
  • An Agent-Based Semantic Search Engine for Scalable Enterprise Applications*
    An Agent-based Semantic Search Engine for Scalable Enterprise Applications* Andrea Passadore1, Alberto Grosso1, Antonio Boccalatte1 1 University of Genova, DIST, Via Opera Pia 13, 16145 Genova, Italy {passa, agrosso, nino}dist.unige.it Abstract. In this paper we present AgentSeeker: a multi-agent platform aimed to index local or online documents, with the support of ontologies which de- scribe the application domain and the competences the user is referring to, dur- ing his query session. AgentSeeker is a flexible and scalable solution mainly devoted to enterprise applications where electronic knowledge bases are par- ticularly important for their business activity. An Ontology Agent is devoted to manage semantic representations of the enterprise domain, organizing the re- sults of a user’s query, according to the concepts which represent the relevant entities in the company business. Keywords: ontology, search engine, multi-agent system, web site, document. 1 Introduction The Information Age leads to us various benefits and comforts, encouraging our thirst for knowledge, helping us at work, and gladdening us during our leisure time. On the other hand, the bits which encode the information are so much that we are lost in a sea of electronic data. The orientation is so hard that oftentimes we lose documents both in the little pond of our personal hard disks and in the boundless ocean of Internet. Search engines represent a saving compass which enables us to find an Internet page winnowing the whole network or to find a personal document through a desktop application which parses private files. Usual search engines denote an intuitive behaviour: they store the textual content of the parsed documents in a database and they return an ordered list of files contain- ing the keywords suggested through a user’s query.
    [Show full text]
  • Comparison of Search Engines
    International Journal of Information Technology and Library Science. Volume 1, Number 1 (2012), pp. 9-25 © Research India Publications http://www.ripublication.com Comparison of Search Engines 1Vikas Malviya, 2Devendra Kumar Mishra, 3Manisha Gawde and 4Madhu Singh Solnki 1Librarian, Rau Dental College, Indore, Madhya Pradesh, India 2Librarian, Mathuradevi Institute of Technology & Management Indore-452018 Madhya Pradesh, India 3Assistant Librarian, Mathuradevi Institute of Technology & Management Indore -452018, Madhya Pradesh, India 4Librarian, Kasturba Gram Mahavidyala, Indore, Madhya Pradesh, India E-mail: [email protected], [email protected], [email protected]., gawdemanisha881@ gmail.com, [email protected], [email protected] Abstract To achieve the best possible results and fulfill the aim of this evaluation exercise, by studying the capabilities that search engines provide to end users. After a general examination of the search engines, a classification took place according to the types of features supported by each search engine. This classification of features was essential in order to determine the areas, in which the evaluation should focus. Goals, non-goals and limitations were extracted from this initial part of the research. The scope of the evaluation of search engine based on accuracy of the most popular search engines, along with their database coverage and other issues such as response time, user- friendliness of the interface, ease in query syntax and submission, are evaluated. Keywords: Search Engine; World Wide Web; Search Strategies; GOOGLE. Introduction The explosive growth of the Internet has rendered the World Wide Web as the primary tool for information retrieval today. However, the amount of information published is increasing constantly and this makes it impossible for anyone to monitor changes.
    [Show full text]
  • Online Evaluation for Information Retrieval
    Foundations and Trends R in Information Retrieval Vol. 10, No. 1 (2016) 1–117 c 2016 K. Hofmann, L. Li, and F. Radlinski DOI: 10.1561/1500000051 Online Evaluation for Information Retrieval Katja Hofmann Lihong Li Microsoft Microsoft [email protected] [email protected] Filip Radlinski Microsoft fi[email protected] Contents 1 Introduction 2 1.1 Terminology . 3 1.2 Motivation and Uses . 4 1.3 This Survey . 5 1.4 Organization . 6 2 Controlled Experiments 7 2.1 Online Controlled Experiments in Information Retrieval . 7 2.2 Planning Controlled Experiments . 10 2.3 Data Analysis . 16 2.4 Between-subject Experiments . 20 2.5 Extensions to AB testing . 22 2.6 Within-subject Experiments . 26 2.7 Extensions to Interleaving . 29 3 Metrics for Online Evaluation 31 3.1 Introduction . 31 3.2 Absolute Document-level Metrics . 33 3.3 Relative Document-level Metrics . 36 3.4 Absolute Ranking-level Metrics . 37 3.5 Relative Ranking-level Metrics . 39 ii iii 3.6 Absolute Session-level and Longer-term Metrics . 44 3.7 Relative Session-level Metrics . 48 3.8 Beyond Search on the Web . 48 3.9 Practical Issues . 48 4 Estimation from Historical Data 51 4.1 Motivation and Challenges . 51 4.2 Problem Setup . 54 4.3 Direct Outcome Models . 56 4.4 Inverse Propensity Score Methods . 58 4.5 Practical Issues . 67 4.6 Concluding Remarks . 68 5 The Pros and Cons of Online Evaluation 70 5.1 Relevance . 71 5.2 Biases . 72 5.3 Experiment Effects . 73 5.4 Reusability . 74 6 Online Evaluation in Practice 76 6.1 Case Studies Approach .
    [Show full text]
  • AWARENESS and USED of SEARCH ENGINES by UNDERGRADUATE STUDENT in DELTA STATE UNIVERSITY, ABRAKA NIGERIA Judith Imoniwe [email protected]
    University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Library Philosophy and Practice (e-journal) Libraries at University of Nebraska-Lincoln Summer 8-18-2018 AWARENESS AND USED OF SEARCH ENGINES BY UNDERGRADUATE STUDENT IN DELTA STATE UNIVERSITY, ABRAKA NIGERIA Judith Imoniwe [email protected] Follow this and additional works at: http://digitalcommons.unl.edu/libphilprac Part of the Library and Information Science Commons Imoniwe, Judith, "AWARENESS AND USED OF SEARCH ENGINES BY UNDERGRADUATE STUDENT IN DELTA STATE UNIVERSITY, ABRAKA NIGERIA" (2018). Library Philosophy and Practice (e-journal). 1982. http://digitalcommons.unl.edu/libphilprac/1982 1 AWARENESS AND USE OF SEARCH ENGINES BY UNDERGRADUATE STUDENTS IN DELTA STATE UNIVERSITY, NIGERIA. BY IMONIWE .O. JUDITH DEPARTMENT OF LIBRARY AND INFORMATION SCIENCE, FACULTY OF EDUCATION, DELTA STATE UNIVERSITY, ABRAKA ABSTRACT The study investigated the awareness and use of search engines by undergraduate students in Delta State University, Abraka. Case study design was used for the study. The population comprised one thousand five hundred and thirty nine (1,539) students in the Faculty of Social Science in Delta State University. The sample of the study was 154. Five research questions guided the study and two hypothesis tested at 0.05 level of significant. One instrument titled search engine use questionnaire (SEUQ) was used for the study. Percentages and mean rating were used to answer research questions. Chi Square and Anova were used to test hypotheses. The major findings of the study include: Undergraduate Students level of awareness of search engines was low in Delta State University. The test for hypothesis shows that there was a significant difference from the respondents awareness of search engines according to their departments.
    [Show full text]
  • A Strategic Perspective on Search Engines: Thought Candies for Practitioners and Researchers
    Available online at www.sciencedirect.com Journal of Interactive Marketing 23 (2009) 49–60 www.elsevier.com/locate/intmar A Strategic Perspective on Search Engines: Thought Candies for Practitioners and Researchers Arvind Rangaswamy,a,⁎ C. Lee Gilesa & Silvija Seresb a Penn State University, USA b FAST, ASA, Norway Abstract Search engines, such as Google and Yahoo! Search, are more than just portals or information tools. In fact, they are agents of a transformation that is making the business environment more transparent, and thus, potentially more competitive. This new environment is creating opportunities and challenges for businesses of every stripe. In this paper, we explore the following topics: (1) what are search engines exactly, (2) what businesses can do with search engines, (3) how are, and how should, senior executives be viewing the strategic impact of search engines, and (4) what are some important research issues for academics and practitioners that would help us gain a better understanding of the strategic impact of search engines. © 2009 Direct Marketing Educational Foundation, Inc. Published by Elsevier B.V. All rights reserved. Keywords: Search engines; Business impact Introduction tion insights for managers and researchers. Search engines are no longer just convenient information tools. In fact, Just fifteen years ago, most people could not have imagined they are powerful agents of a transformation that is making the crucial role that search engines would play in facilitating the business environment more transparent, and thus, poten- today's commerce. Now search engines can be used to find tially more competitive. This environment is creating new every conceivable kind of information about people, places, opportunities and challenges as highlighted by the following things, and more.
    [Show full text]
  • Real Time Search on the Web: Queries, Topics, and Economic Value ⇑ Bernard J
    Information Processing and Management xxx (2011) xxx–xxx Contents lists available at ScienceDirect Information Processing and Management journal homepage: www.elsevier.com/locate/infoproman Real time search on the web: Queries, topics, and economic value ⇑ Bernard J. Jansen a, , Zhe Liu a, Courtney Weaver a, Gerry Campbell b, Matthew Gregg b a College of Information Sciences and Technology, The Pennsylvania State University, University Park, PA 16802, United States b Collecta, Santa Monica, CA 90401, United States article info abstract Article history: Real time search is an increasingly important area of information seeking on the Web. In Received 21 June 2010 this research, we analyze 1,005,296 user interactions with a real time search engine over Received in revised form 5 January 2011 a 190 day period. Using query log analysis, we investigate searching behavior, categorize Accepted 17 January 2011 search topics, and measure the economic value of this real time search stream. We examine Available online xxxx aggregate usage of the search engine, including number of users, queries, and terms. We then classify queries into subject categories using the Google Directory topical hierarchy. Keywords: We next estimate the economic value of the real time search traffic using the Google Real time search AdWords keyword advertising platform. Results shows that 30% of the queries were unique Real time content Collecta (used only once in the entire dataset), which is low compared to traditional Web searching. Twitter Also, 60% of the search traffic comes from the search engine’s application program inter- Economic value of search face, indicating that real time search is heavily leveraged by other applications.
    [Show full text]