DOCSLIB.ORG

An Intelligent Metasearch Engine for the World Wide

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Intelligent Metasearch Engine for the World Wide AN INTELLIGENTMETASEARCH ENGINE FOR THE WORLDWIDE WEB Andrew Agno A thesis submitted in conformity with the requirements for the degree of Master of Science Graduate Department of Cornputer Science University of Toronto Copyright @ 2000 by Andrew Agno National Library Bibliothèque nationale ($1 of Canada du Canada Acquisitions and Acquisitions et Bibfiographic Services services bibliographiques 395 Wellington Street 395. rue Wellington OttawaON K1AON4 Ottawa ON K1A ON4 Canada Canada The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Lïbrary of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sel1 reproduire, prêter, distribuer ou copies of this thesis in rnicrofonn, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/film, de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts ffom it Ni la thèse ni des extraits substantiels may be p~tedor otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. tract An Intelligent Met asearch Engine for the n'orld nïde Uéb .Anchen- Agno Mas ter of Science Graduate Department of Cornputer Science Cniversi ty of Toronto 2000 Uachine learning and informat ion retried techniques are appliecl t o met asearch on the Vorld n'ide ?\éb as a means of providing user specific relennr documents in respome to user queries. .A rnerasearch agent works in conjunction n-ith a user to provide daiiy sers of relel-ant documents. Csers provide relennce feedback which is incorporarcd into future resdts b. a choice of machine I~arningalgorithms. Csing a fisecl ranking niethoci. the algorithms incorporating relelance feetlback per- forni rriuch bet ter than t hose t hat do not. Furthemore. using heterogeneoits information sources on the Lorld Wide \\éb is shown ro be effective in short and long term usage. Acknowledgement s 1 n-ould he much less proticl of m\- work if it tvere not for the help of a nurnber of people. 1 woiild like to firsr thank Grigoris Iiarakoulas and John 11~-lopotilos.my super~isors. for their guidance and support of mj- work. nïthoiit theni. 1 woiild still be fishing for a perfect ropic. Thank oualso for making me look ar quesrions insteacl of answrs. I ,..,,uuU - .. I I nLXe tu iZnriP iio DG ;LLLLC~gruup for rkir quesrions ro my presenration oi this thesis. which helped me focus on \\-kat questions other people woiild be inreresred in. upon seeing my work. Some of the n-ork in the implementation of my project used other peoples software. In particular. 1 n-ould like to thank S teyen Brandt for the package com.stevesoft.pat. Doug Lea. for util.concurrenr . and Brian Chambers for his pret-ioiis work in word sterrirning and document vectorization. Last. but certainly not least. 1 woiiltl like ro thank mu deJobie. for coming to Toronto ancl staying wirh me thrse lasr two yars. 3 Architecture 24 3.1 O\-erall -1rchitect ure ............................. 24 3.2 Global Data Structures ............................ 2s 3.1 Zipf's Law ............................... 29 3 .2 Stopword List ............................. 29 3.2.3 Stemniing ............................... 30 3.3 Scalahility ................................... 30 3.3.1 Caching ................................ 31 3.4 Lem Weighting ................................ 33 3.5 Topics ..................................... 33 4 Experimental Results and Evaluation 35 4.1 Description of Data Gathering Procedure .................. 35 4. Eduation Framework ............................ 37 4.3 TREC ~lcasures............................... 41 4 .3. TheF3lIeasure ........................ 41 4.32 The T9L- Neasure .......................... 46 4.3.3 The T9P Measure ........................... 50 4.4 Precision of Learning Algorithms ...................... 5.L 4.4.1 Continuous Learning vs Train/Test ................. 60 4.5 Daily Recall .................................. 63 4.6 Spikes ..................................... 65 4.6.1 Data Gathering Gaps ......................... 66 4.6.2 Flushing ................................ 66 4 Individual Search Engine Recall ....................... 69 5 Conclusions and Future Directions 77 .. 5 .1 Conclusions and Discussion ..................... r , 5. Future Directions . 79 5.2-1 Implicit Ranking . 79 5.2 Ontologies ........ ....... ... ............ SO 5.2.3 Collaboration . S 1 3.4 Alternate Document or Featiire Space . S 1 5.2. Thresholds . S? 5.2.6 Alternative Methods of Learning . S.' C - a.?. 1 !discellaneou Iniprovements and Direct ions . S9 Bibliograp hy List of Figures 3.1 Architecture Diagam ............................. 26 4.1 Daily Document Counts ........................... 36 4 Precision for Plain Algorithm ........................ 39 4.3 Precision for Random aigorithm ....................... 40 4 .4 F3 Measure .................................. 42 4 .5 F3 Sleasiire. Top 5 .............................. 43 4.6 F3 blessure. Top 10 .............................. 44 4 .7 T9C Ueasure ................................. 47 4.S T9C Sleasure . Top 5 ............................. 4s 4.9 T9C Ueasure . Top 10 ............................. 49 4.10 T9P 'lleasure ................................. 51 4.11 T9P Skasure ................................. 52 -4.12 T9P 'Ikasrire ................................. 53 - - 4.13 Running Average of Precision . AU Topics .................. XI 4.14 Precision Running Average . Various Topics . Rocchio tk Grigoris ..... 56 4.15 Running .A \.erage Precision . Student . Grigoris ............... 5s 4.16 Running Average Precision . .AU Topics . Continuous Training vs Train / Test 5s -4.17 Running avg precision . nrious topics . continuous training vs Train/Test 59 4-18 Dail- recall for Rocchio algorithm ...................... 61 4.19 Daily recd for Grigoris algonthm ...................... 62 vii 4.20 Daily precision . -411 ropics . Top 10 ...................... 66 4.21 Dail>-precision . I'arious topics . Top 10 ................... 67 4.22 Daily precision on srudenrs topic . Top 30 .................. (jS 4.23 Search Engine Red. -411 Topics . Riuining Average ............ 70 4.24 Search Engine Recd. Al1 Topics . Running -Iverage ............ 71 4.23 Runriing Average of Recall of làhoo on Palni Pilot ............ 72 4.26 Riinning .l verage of Recall of Lycos on Student .............. 73 4.27 Running average recall . lIS/DO*I ...................... 74 Chapter 1 Introduction Problem and Motivation Findilig informat ion on the World Wide Web (\\'WU')can be difficult withottt sonir form of assistance. As estimated by Lawrence and Giles [LG99]in 1999. there rvere SOO million pages. an increaoe of 250% from their previous estimate in their 1998 study [LC;9Sc]. C-illance [MMOO] claims there are 2.1 billion unique and ptihlicly amilable pages on the Internet. Given the size and the gowth of the WWY. one can see that we neeci tools to help us find information. One would typicdy turn to a search engine. like Yahoo![Ya.h] or Coogle [Goo]. Cnfortunately. the mos t frequently used search engines [StaOO. SulOOa] do not always do an adequate job. due to their Iack of coverage [LG99.LGSSc] and their lack of ability t-Ofind the relevant documents in those that are covered. One potential ierned-- is to enable searches with more -intelligence". Given a search engine. ir mai- be imbued with -intelligencen in at least tn-O ways: through the use of specialized. larger. or sirnply different informat ion sources: or t hrough the implement at ion of various machine learning or information filtering algorithms. The purpose of this R-ork is to create an intelligent search engine by combining both of these approaches into a single search engine. dra~ingfrom machine learning and information retriemi techniques. The remainder of this chapter id1 deal with an 01-er~ien-of a particiilar technique for searching t hrough her erogeneoiis infornia t ion sources. called met asearch. The chapter also inclutles an esplanation of various information retrieval and machine learning techniques rhar have been used in other work. the contributions of this work as iveU as a layout of the remainder of this thesis. 1.2 Metasearch on the World Wide Web One commonly tised method for adding intelligence. even mong search engines not norrnally thought of as eiichjSulOOc]. is to use metasearch. For the purposes of this research. metasearch will refer to nietasearch on the 1V1171'. 1.2.1 A simple anatomy of a search engine In the following disciission. it helps to have nn idea of hou- search engines typically ivork. The portion of a search engine that the user sees is only one aspect of the entire system that makes up an engine. .U the results given by a search engine corne from some forrn of a database underlying the engine. which may list documents and information about those documents. This database is populated by certain software agents that visit WWV' servers and index the documents that those semers contain. In the Hantest system[BDW95]. these agents are cded Gatherers. whereas in Google[BPSSb]. they are called c~awlers. The purpose of these crawlers is to dow the collection of information about documents. also known as the indexiag of documents. to proceed independently of an? search that may be using the information. These cran-lers must revisit
Load more

Recommended publications
  • A Meta Search Engine Based on a New Result Merging Strategy
    Usearch: A Meta Search Engine based on a New Result Merging Strategy Tarek Alloui1, Imane Boussebough2, Allaoua Chaoui1, Ahmed Zakaria Nouar3 and Mohamed Chaouki Chettah3 1MISC Laboratory, Department of Computer Science and its Applications, Faculty of NTIC, University Constantine 2 Abdelhamid Mehri, Constantine, Algeria 2LIRE Laboratory, Department of Software Technology and Information Systems, Faculty of NTIC, University Constantine 2 Abdelhamid Mehri, Constantine, Algeria 3Department of Computer Science and its Applications, Faculty of NTIC, University Constantine 2, Abdelhamid Mehri, Constantine, Algeria Keywords: Meta Search Engine, Ranking, Merging, Score Function, Web Information Retrieval. Abstract: Meta Search Engines are finding tools developed for improving the search performance by submitting user queries to multiple search engines and combining the different search results in a unified ranked list. The effectiveness of a Meta search engine is closely related to the result merging strategy it employs. But nowadays, the main issue in the conception of such systems is the merging strategy of the returned results. With only the user query as relevant information about his information needs, it’s hard to use it to find the best ranking of the merged results. We present in this paper a new strategy of merging multiple search engine results using only the user query as a relevance criterion. We propose a new score function combining the similarity between user query and retrieved results and the users’ satisfaction toward used search engines. The proposed Meta search engine can be used for merging search results of any set of search engines. 1 INTRODUCTION advantages of MSEs are their abilities to combine the coverage of multiple search engines and to reach Nowadays, the World Wide Web is considered as the deep Web.
    [Show full text]
  • Meta Search Engine Examples
    Meta Search Engine Examples mottlesMarlon istemerariously unresolvable or and unhitches rice ichnographically left. Salted Verney while crowedanticipated no gawk Horst succors underfeeding whitherward and naphthalising. after Jeremy Chappedredetermines and acaudalfestively, Niels quite often sincipital. globed some Schema conflict can be taken the meta descriptions appear after which result, it later one or can support. Would result for updating systematic reviews from different business view all fields need to our generated usually negotiate the roi. What is hacking or hacked content? This meta engines! Search Engines allow us to filter the tons of information available put the internet and get the bid accurate results And got most people don't. Best Meta Search array List The Windows Club. Search engines have any category, google a great for a suggestion selection has been shown in executive search input from health. Search engine name of their booking on either class, the sites can select a search and generally, meaning they have past the systematisation of. Search Engines Corner Meta-search Engines Ariadne. Obsession of search engines such as expedia, it combines the example, like the answer about search engines out there were looking for. Test Embedded Software IC Design Intellectual Property. Using Research Tools Web Searching OCLS. The meta description for each browser settings to bing, boolean logic always prevent them the hierarchy does it displays the search engine examples osubject directories. Online travel agent Bookingcom has admitted that playing has trouble to compensate customers whose personal details have been stolen Guests booking hotel rooms have unwittingly handed over business to criminals Bookingcom is go of the biggest online travel agents.
    [Show full text]
  • Google Overview Created by Phil Wane
    Google Overview Created by Phil Wane PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Tue, 30 Nov 2010 15:03:55 UTC Contents Articles Google 1 Criticism of Google 20 AdWords 33 AdSense 39 List of Google products 44 Blogger (service) 60 Google Earth 64 YouTube 85 Web search engine 99 User:Moonglum/ITEC30011 105 References Article Sources and Contributors 106 Image Sources, Licenses and Contributors 112 Article Licenses License 114 Google 1 Google [1] [2] Type Public (NASDAQ: GOOG , FWB: GGQ1 ) Industry Internet, Computer software [3] [4] Founded Menlo Park, California (September 4, 1998) Founder(s) Sergey M. Brin Lawrence E. Page Headquarters 1600 Amphitheatre Parkway, Mountain View, California, United States Area served Worldwide Key people Eric E. Schmidt (Chairman & CEO) Sergey M. Brin (Technology President) Lawrence E. Page (Products President) Products See list of Google products. [5] [6] Revenue US$23.651 billion (2009) [5] [6] Operating income US$8.312 billion (2009) [5] [6] Profit US$6.520 billion (2009) [5] [6] Total assets US$40.497 billion (2009) [6] Total equity US$36.004 billion (2009) [7] Employees 23,331 (2010) Subsidiaries YouTube, DoubleClick, On2 Technologies, GrandCentral, Picnik, Aardvark, AdMob [8] Website Google.com Google Inc. is a multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products,[9] and generates profit primarily from advertising through its AdWords program.[5] [10] The company was founded by Larry Page and Sergey Brin, often dubbed the "Google Guys",[11] [12] [13] while the two were attending Stanford University as Ph.D.
    [Show full text]
  • List of Search Engines
    A blog network is a group of blogs that are connected to each other in a network. A blog network can either be a group of loosely connected blogs, or a group of blogs that are owned by the same company. The purpose of such a network is usually to promote the other blogs in the same network and therefore increase the advertising revenue generated from online advertising on the blogs.[1] List of search engines From Wikipedia, the free encyclopedia For knowing popular web search engines see, see Most popular Internet search engines. This is a list of search engines, including web search engines, selection-based search engines, metasearch engines, desktop search tools, and web portals and vertical market websites that have a search facility for online databases. Contents 1 By content/topic o 1.1 General o 1.2 P2P search engines o 1.3 Metasearch engines o 1.4 Geographically limited scope o 1.5 Semantic o 1.6 Accountancy o 1.7 Business o 1.8 Computers o 1.9 Enterprise o 1.10 Fashion o 1.11 Food/Recipes o 1.12 Genealogy o 1.13 Mobile/Handheld o 1.14 Job o 1.15 Legal o 1.16 Medical o 1.17 News o 1.18 People o 1.19 Real estate / property o 1.20 Television o 1.21 Video Games 2 By information type o 2.1 Forum o 2.2 Blog o 2.3 Multimedia o 2.4 Source code o 2.5 BitTorrent o 2.6 Email o 2.7 Maps o 2.8 Price o 2.9 Question and answer .
    [Show full text]
  • Architecture Based Study of Search Engines and Meta Search Engines for Information Retrieval A
    International Journal of Engineering Research & Technology (IJERT) ISSN: 2278-0181 Vol. 2 Issue 5, May - 2013 Architecture Based Study Of Search Engines And Meta Search Engines For Information Retrieval A. Madhavi1,K. Harisha Chari2 1Asst.Professor, Matrusri Institute of PG Studies, Hyderabad, India. 2Asst.Professor, K.V Ranga Reddy College, Hyderabad, India. Abstract not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to WWW is a huge repository of information. republish to post on servers or to redistribute to The complexity of accessing the web data has lists, requires prior specific permission and/or a fee. increased tremendously over the years. There is Single search engine are increased coverage and a a need for efficient searching techniques to consistent interface. A recent study by Lawrence extract appropriate information from the web, as and Giles estimated the size of the web at about 800 million indexable pages. This same study the users require correct and complex concluded that no single search engine covered information from the web. more than about sixteen percent of the total. By This paper analyses the architectures and searching multiple search engines simultaneously features of metasearch engines for searching and via a metasearch engine, coverage increases receiving documents on single and multiples dramatically over searching only one engine. domains on the web. A web search engine searches Lawrence and Giles found that combining the for information in WWW. results of 11 major search engines increased the coverage to about 42% of the estimated size of the 1.
    [Show full text]
  • CS 101 - Intro to Computer Science Such Software Has Capabilities Such As: Search Engines Organizing an Index to the Information in a Database
    What is a Search Engine? A search engine is a computer program that allows a user to submit a query and retrieve information from a database. CS 101 - Intro to Computer Science Such software has capabilities such as: Search Engines organizing an index to the information in a database, Dr. Stephen P. Carl enabling the formulation of a query by a user, and searching the index in response to a query. Web Crawlers - Mining Web Information Types of web search engines A search engine for the WWW uses a program called a robot or spider to index the information on Web pages. Topical search engines organize their catalogs of sites by topic or subject. Examples are Yahoo! and Lycos a2z. Spiders work by following all the links on a page according to a specific search strategy. The simplest strategy is to collect all links from a page and then follow them, one by one, collecting new links as new pages are visited. The content of each page is added to a database. The database is indexed and this index is searched upon receipt of a query from the user; the search engine then presents a sorted list of matching results to the user. 1-4 Types of web search engines Types of web search engines Keyword or Key Phrase search engines let the user specify a set of Metasearch Engines send queries to several other search engines keywords or phrases related to the desired content. and consolidate the results. Some, such as ProFusion, filter the results to remove duplicates and check the validity of the links.
    [Show full text]
  • SAVVYSEARCH: a Metasearch Engine That Learns Which Search
    AI Magazine Volume 18 Number 2 (1997) (© AAAI) Articles SAVVYSEARCH A Metasearch Engine That Learns Which Search Engines to Query Adele E. Howe and Daniel Dreilinger ■ Search engines are among the most successful single query to multiple search engines. applications on the web today. So many search The simplest metasearch engines are forms engines have been created that it is difficult for that allow the user to indicate which search users to know where they are, how to use them, engines should be contacted (for example, and what topics they best address. Metasearch ALL-IN-ONE [www.albany.net/allinone] and engines reduce the user burden by dispatching METASEARCH [members.gnn.com/infinet/ queries to multiple search engines in parallel. The meta.htm]). PROFUSION (Gauch, Wang, and SAVVYSEARCH metasearch engine is designed to efficiently query other search engines by carefully Gomez 1996; www.designlab.ukans.edu/pro- selecting those search engines likely to return fusion) gives the user the choice of selecting useful results and responding to fluctuating load search engines themselves or letting PROFUSION demands on the web. SAVVYSEARCH learns to iden- select three of six robot-based search engines tify which search engines are most appropriate using hand-built rules. METACRAWLER (Selberg for particular queries, reasons about resource and Etzioni 1995; metacrawler.cs.washing- demands, and represents an iterative parallel ton.edu:8080/home.html) significantly search strategy as a simple plan. enhances the output by downloading and analyzing the links returned by the search engines to prune out unavailable and irrele- ompanies, institutions, and individuals vant links.
    [Show full text]
  • Download Download
    International Journal of Management & Information Systems – Fourth Quarter 2011 Volume 15, Number 4 History Of Search Engines Tom Seymour, Minot State University, USA Dean Frantsvog, Minot State University, USA Satheesh Kumar, Minot State University, USA ABSTRACT As the number of sites on the Web increased in the mid-to-late 90s, search engines started appearing to help people find information quickly. Search engines developed business models to finance their services, such as pay per click programs offered by Open Text in 1996 and then Goto.com in 1998. Goto.com later changed its name to Overture in 2001, and was purchased by Yahoo! in 2003, and now offers paid search opportunities for advertisers through Yahoo! Search Marketing. Google also began to offer advertisements on search results pages in 2000 through the Google Ad Words program. By 2007, pay-per-click programs proved to be primary money-makers for search engines. In a market dominated by Google, in 2009 Yahoo! and Microsoft announced the intention to forge an alliance. The Yahoo! & Microsoft Search Alliance eventually received approval from regulators in the US and Europe in February 2010. Search engine optimization consultants expanded their offerings to help businesses learn about and use the advertising opportunities offered by search engines, and new agencies focusing primarily upon marketing and advertising through search engines emerged. The term "Search Engine Marketing" was proposed by Danny Sullivan in 2001 to cover the spectrum of activities involved in performing SEO, managing paid listings at the search engines, submitting sites to directories, and developing online marketing strategies for businesses, organizations, and individuals.
    [Show full text]
  • Role of Meta Search Engines in Web- Based Information System: Fundamentals and Challenges
    445 ROLE OF META SEARCH ENGINES IN WEB- BASED INFORMATION SYSTEM: FUNDAMENTALS AND CHALLENGES Subarna Kumar Das Abstract Meta search engines are powerful tools that search multiple search engines simulta- neously. Unlike crawler-based search engines such as Google, AllTheWeb, AltaVista and others, meta search engines generally do not build and maintain their own web indexes. Instead, they use the indexes built by others, aggregating and often post- processing results in unique ways. Meta search engines accept your query, and send it out to multiple search engines in parallel. They are often quite fast, using private “backdoor” servers made available by the search engines they query. They get this privileged status thanks to revenue sharing agreements. There are several advantages to using meta search engines, though they’re not always the most appropriate tool. The most obvious advantage is that you can get results from multiple search engines with- out having to visit each in turn. Apart from the time savings, there is some evidence that this gives your search a broader scope, since each individual search engine’s index differs from all others. Present paper attempts to highlights the role of meta search engine in web- based information retrieval Keywords: Search Engine, World Wide Web, Meta Search Engine, Metacrawler In- formation Storage, 1. Introduction A search engine is a program designed to help find information stored on a computer system such as the World Wide Web, inside a corporate or proprietary network or a personal computer. The search engine allows one to ask for content meeting specific criteria (typically those containing a given word or phrase) and retrieves a list of references that match those criteria.
    [Show full text]
  • Retrieval Efficiency of Search Engines on Medical Tourism in Kerala: a Webometric Analysis
    Retrieval Efficiency of Search Engines on Medical Tourism in Kerala: A Webometric Analysis Thanuskodi S, Naseehath S Alagappa University Karaikudi India ABSTRACT: World Wide Web is like an ocean of information; it is not easy to find specific information from it because the web is very huge and growing all the time. So there are certain tools used to search information from the web. They are called search engines. They search through all the web sites and create an index of the information of the web sites and act as a single point to find relevant information. E.g. Google, Yahoo, Bing, Lycos, Info seek etc. Analysis on 8 representative variables form all the subsectors of on Medical Tourism in Kerala, the search engine Lycos retrieves 59% of the total results and stands first followed by Bing in the second position with 37% and Google in third position with only 2% of results . The remaining less than 1% is shared by all Metasearch engines, of these Ixquick stands first with a slight difference from others followed by Dogpile and WebCrawler Analysis of 10 variables on Ayurveda tourism Lycos occupies first place with a high result of 85% and Google in the second position followed by Bing in the third position with a less Percentage of 7 and 6 respectively. Among Metasearch engines Ixquick stands first, followed by WebCrawler and Dogpile. Keywords: Search Engines, Retrieval Efficiency, Medical Tourism, Webometrics, Internet Received: 19 February 2019, Revised 4 April 2019, Accepted 8 May 2019 DOI: 10.6025/jism/2019/9/3/91-99 © 2019 DLINE.
    [Show full text]
  • A Study of Results Overlap and Uniqueness Among Major Web Search Engines
    COVER SHEET Spink, Amanda and Jansen, Bernard J. and Blakely, Chris and Koshman, Sherry (2006) A study of results overlap and uniqueness among major web search engines. Information Processing and Management 42(5):pp. 1379-1391. Copyright 2006 Elsevier. Accessed from: http://eprints.qut.edu.au/archive/00004755 1 A STUDY OF RESULTS OVERLAP AND UNIQUENESS AMONG MAJOR WEB SEARCH ENGINES Amanda Spink * Faculty of Information Technology Queensland University of Technology Gardens Point Campus, 2 George St, GPO Box 2434 Brisbane QLD 4001 Australia Tel: 61-7-3864-2782 Fax: 61-7-3864-2703 Email: [email protected] Bernard J. Jansen School of Information Sciences and Technology The Pennsylvania State University University Park PA 16802 Tel: (814) 865-6459 Fax: (814) 865-6426 Email: [email protected] Chris Blakely Market Strategy Manager Infospace, Inc. – Search & Directory 601 108th Ave NE, Ste 1200 Bellevue, WA 98004 USA Tel: (425) 709-8101 Fax: (425) 201-6162 Email: [email protected] Sherry Koshman School of Information Sciences University of Pittsburgh 611 IS Building, 135 N. Bellefield Avenue Pittsburgh PA 15260 Tel: (412) 624-9441 Fax: (412) 648-7001 Email: [email protected] * To whom all correspondence should be sent. 2 .ABSTRACT The performance and capabilities of Web search engines is an important and significant area of research. Millions of people world wide use Web search engines very day. This paper reports the results of a major study examining the overlap among results retrieved by multiple Web search engines for a large set of more than 10,000 queries.
    [Show full text]
  • Improving Search Engines Via Classification
    Improving Search Engines via Classi¯cation Zheng Zhu May 2011 A Dissertation Submitted to Birkbeck College, University of London in Partial Ful¯llment of the Requirements for the Degree of Doctor of Philosophy Department of Computer Science & Information Systems Birkbeck College University of London Declaration This thesis is the result of my own work, except where explicitly acknowledge in the text. Zheng Zhu i Abstract In this dissertation, we study the problem of how search engines can be improved by making use of classi¯cation. Given a user query, traditional search engines output a list of results that are ranked according to their relevance to the query. However, the ranking is independent of the topic of the document. So the results of di®erent topics are not grouped together within the result output from a search engine. This can be problematic as the user must scroll though many irrelevant results until his/her desired information need is found. This might arise when the user is a novice or has super¯cial knowledge about the domain of interest, but more typically it is due to the query being short and ambiguous. One solution is to organise search results via categorization, in particular, the classi¯cation. We designed a target testing experiment on a controlled data set, which showed that classi¯cation-based search could improve the user's search experience in terms of the numbers of results the user would have to inspect before satisfying his/her query. In our investigation of classi¯cation to organise search results, we not only consider the classi¯cation of search results, but also query classi¯cation.
    [Show full text]