First 20 Precision among World Wide Web Search Services (Search Engines) H. Vernon Leighton* Winona State University Library, Winona, MN 55987-5838. E-mail:
[email protected] Jaideep Srivastava Computer Science, University of Minnesota, 4-192 EECSci, Minneapolis, MN 55455. E-mail:
[email protected] Five search engines, Alta Vista, Excite, HotBot, Infoseek, customize each search’s ranking algorithm. Lycos has and Lycos, are compared for precision on the first 20 bought Wired Digital and with it, HotBot. There has been a results returned for 15 queries, adding weight for rank- ing effectiveness. All searching was done from January trend toward raising the rank of more popular or more 31 to March 12, 1997. In the study, steps are taken to heavily cited pages. One- and two-word queries are now ensure that bias has not unduly influenced the evalua- often rerouted to Web directories so that users can enjoy the tion. Friedmann’s randomized block design is used to topical coverage that comes with a classification scheme. perform multiple comparisons for significance. Analysis Clustered pages are now grouped under a link that allows shows that Alta Vista, Excite and Infoseek are the top three services, with their relative rank changing depend- users to view all pages retrieved from the same site in ing on how one operationally defines the concept of Infoseek, Lycos, and HotBot. Infoseek and Excite have relevance. Correspondence analysis shows that Lycos raised the ranking of reviewed sites, and Alta Vista has performed better on short, unstructured queries, incorporated the Ask Jeeves service into their results.