DOCSLIB.ORG

CS 101 - Intro to Computer Science Such Software Has Capabilities Such As: Search Engines Organizing an Index to the Information in a Database

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
CS 101 - Intro to Computer Science Such Software Has Capabilities Such As: Search Engines Organizing an Index to the Information in a Database What is a Search Engine? A search engine is a computer program that allows a user to submit a query and retrieve information from a database. CS 101 - Intro to Computer Science Such software has capabilities such as: Search Engines organizing an index to the information in a database, Dr. Stephen P. Carl enabling the formulation of a query by a user, and searching the index in response to a query. Web Crawlers - Mining Web Information Types of web search engines A search engine for the WWW uses a program called a robot or spider to index the information on Web pages. Topical search engines organize their catalogs of sites by topic or subject. Examples are Yahoo! and Lycos a2z. Spiders work by following all the links on a page according to a specific search strategy. The simplest strategy is to collect all links from a page and then follow them, one by one, collecting new links as new pages are visited. The content of each page is added to a database. The database is indexed and this index is searched upon receipt of a query from the user; the search engine then presents a sorted list of matching results to the user. 1-4 Types of web search engines Types of web search engines Keyword or Key Phrase search engines let the user specify a set of Metasearch Engines send queries to several other search engines keywords or phrases related to the desired content. and consolidate the results. Some, such as ProFusion, filter the results to remove duplicates and check the validity of the links. An The engine returns a list of documents containing the keywords or example of a Metasearch Engine is MetaCrawler. phrases. Most engines allow the using booleans in queries; some support more advanced searching such as inexact match Examples of keyword or key phrase search engines are AltaVista, InfoSeek, Lycos, and WebCrawler. Web Robots AKA Spiders Web Robots AKA Spiders Advantages of using robots: Web Robots are programs that automatically visit web pages in order Tasks can be performed automatically to perform specific tasks such as collecting information from the Tasks can be executed in parallel by creating several robots to pages for indexing. work concurrently Common Uses: Users can easily access up-to-date information retrieving web pages whose contents will be indexed by the Search Disadvantages of using robots: Engine Users can easily access up-to-date information administrative tasks, such as collecting statistics, checking links, and verifying HTML Robots often visit irrelevent URLs and sometimes invoke programs (like CGI) with side effects distributing information Malicious or poorly written robots can severely degrade the quality of service for legitimate users 5-8 Types of Robots Managing Robots Access to Sites Robots generally fall into two categories: server-invoked or client-invoked. The Robot Exclusion Standard at http://info.webcrawler.com/mak/projects/robots/norobots.html outlines Server robots, such as WebCrawler, are created and controlled by a a method to reduce robot problems. Robots that adhere to the server process responding to a request, while client robots are standard can be excluded from a particular web page. The settings invoked directly by the user. for the standard are stored in a file named robots.txt. Client robots can cause network traffic problems. Should several By using the field User-agent in the file, the user can specify which thousand users search the web using personal search engines at the robots are to be excluded. Most search engine services publish the same time, the traffic generated by these clients will have a negative names of their robots. effect on network performance. Displaying Results Displaying Results, Google Style Google uses an algorithm called PageRank to determine not only After receiving a query and working through the indexed database to how well a page matches a query, but also how important and find matches, how should the search engine display the results to the relevant it might be. user? PageRank is was developed at Stanford University by the founders of Alphabetic Order, by Title for example? Google, Larry Page and Sergey Brin, while they were graduate students there. In the order results were found? Curiously, Stanford, not Google, holds the PageRank patent Using some criterion specified by the user? The criterion used is simple on the surface...the more links to a page, Let's see what Google does... the more relevant it is considered; the more important the pages linking to this page are, the higher the page's rank in the results. 9-12 References Lager, Mark. Spinning a Web Search http://www.library.ucsb.edu/untangle/lager.html http://en.wikipedia.org/wiki/Search_engine http://www.google.com/corporate/tech.html Much of this information came from Dr. Lankewicz's course pages 13-14.
Load more

Recommended publications
  • A Meta Search Engine Based on a New Result Merging Strategy
    Usearch: A Meta Search Engine based on a New Result Merging Strategy Tarek Alloui1, Imane Boussebough2, Allaoua Chaoui1, Ahmed Zakaria Nouar3 and Mohamed Chaouki Chettah3 1MISC Laboratory, Department of Computer Science and its Applications, Faculty of NTIC, University Constantine 2 Abdelhamid Mehri, Constantine, Algeria 2LIRE Laboratory, Department of Software Technology and Information Systems, Faculty of NTIC, University Constantine 2 Abdelhamid Mehri, Constantine, Algeria 3Department of Computer Science and its Applications, Faculty of NTIC, University Constantine 2, Abdelhamid Mehri, Constantine, Algeria Keywords: Meta Search Engine, Ranking, Merging, Score Function, Web Information Retrieval. Abstract: Meta Search Engines are finding tools developed for improving the search performance by submitting user queries to multiple search engines and combining the different search results in a unified ranked list. The effectiveness of a Meta search engine is closely related to the result merging strategy it employs. But nowadays, the main issue in the conception of such systems is the merging strategy of the returned results. With only the user query as relevant information about his information needs, it’s hard to use it to find the best ranking of the merged results. We present in this paper a new strategy of merging multiple search engine results using only the user query as a relevance criterion. We propose a new score function combining the similarity between user query and retrieved results and the users’ satisfaction toward used search engines. The proposed Meta search engine can be used for merging search results of any set of search engines. 1 INTRODUCTION advantages of MSEs are their abilities to combine the coverage of multiple search engines and to reach Nowadays, the World Wide Web is considered as the deep Web.
    [Show full text]
  • Meta Search Engine Examples
    Meta Search Engine Examples mottlesMarlon istemerariously unresolvable or and unhitches rice ichnographically left. Salted Verney while crowedanticipated no gawk Horst succors underfeeding whitherward and naphthalising. after Jeremy Chappedredetermines and acaudalfestively, Niels quite often sincipital. globed some Schema conflict can be taken the meta descriptions appear after which result, it later one or can support. Would result for updating systematic reviews from different business view all fields need to our generated usually negotiate the roi. What is hacking or hacked content? This meta engines! Search Engines allow us to filter the tons of information available put the internet and get the bid accurate results And got most people don't. Best Meta Search array List The Windows Club. Search engines have any category, google a great for a suggestion selection has been shown in executive search input from health. Search engine name of their booking on either class, the sites can select a search and generally, meaning they have past the systematisation of. Search Engines Corner Meta-search Engines Ariadne. Obsession of search engines such as expedia, it combines the example, like the answer about search engines out there were looking for. Test Embedded Software IC Design Intellectual Property. Using Research Tools Web Searching OCLS. The meta description for each browser settings to bing, boolean logic always prevent them the hierarchy does it displays the search engine examples osubject directories. Online travel agent Bookingcom has admitted that playing has trouble to compensate customers whose personal details have been stolen Guests booking hotel rooms have unwittingly handed over business to criminals Bookingcom is go of the biggest online travel agents.
    [Show full text]
  • Google Overview Created by Phil Wane
    Google Overview Created by Phil Wane PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Tue, 30 Nov 2010 15:03:55 UTC Contents Articles Google 1 Criticism of Google 20 AdWords 33 AdSense 39 List of Google products 44 Blogger (service) 60 Google Earth 64 YouTube 85 Web search engine 99 User:Moonglum/ITEC30011 105 References Article Sources and Contributors 106 Image Sources, Licenses and Contributors 112 Article Licenses License 114 Google 1 Google [1] [2] Type Public (NASDAQ: GOOG , FWB: GGQ1 ) Industry Internet, Computer software [3] [4] Founded Menlo Park, California (September 4, 1998) Founder(s) Sergey M. Brin Lawrence E. Page Headquarters 1600 Amphitheatre Parkway, Mountain View, California, United States Area served Worldwide Key people Eric E. Schmidt (Chairman & CEO) Sergey M. Brin (Technology President) Lawrence E. Page (Products President) Products See list of Google products. [5] [6] Revenue US$23.651 billion (2009) [5] [6] Operating income US$8.312 billion (2009) [5] [6] Profit US$6.520 billion (2009) [5] [6] Total assets US$40.497 billion (2009) [6] Total equity US$36.004 billion (2009) [7] Employees 23,331 (2010) Subsidiaries YouTube, DoubleClick, On2 Technologies, GrandCentral, Picnik, Aardvark, AdMob [8] Website Google.com Google Inc. is a multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products,[9] and generates profit primarily from advertising through its AdWords program.[5] [10] The company was founded by Larry Page and Sergey Brin, often dubbed the "Google Guys",[11] [12] [13] while the two were attending Stanford University as Ph.D.
    [Show full text]
  • List of Search Engines
    A blog network is a group of blogs that are connected to each other in a network. A blog network can either be a group of loosely connected blogs, or a group of blogs that are owned by the same company. The purpose of such a network is usually to promote the other blogs in the same network and therefore increase the advertising revenue generated from online advertising on the blogs.[1] List of search engines From Wikipedia, the free encyclopedia For knowing popular web search engines see, see Most popular Internet search engines. This is a list of search engines, including web search engines, selection-based search engines, metasearch engines, desktop search tools, and web portals and vertical market websites that have a search facility for online databases. Contents 1 By content/topic o 1.1 General o 1.2 P2P search engines o 1.3 Metasearch engines o 1.4 Geographically limited scope o 1.5 Semantic o 1.6 Accountancy o 1.7 Business o 1.8 Computers o 1.9 Enterprise o 1.10 Fashion o 1.11 Food/Recipes o 1.12 Genealogy o 1.13 Mobile/Handheld o 1.14 Job o 1.15 Legal o 1.16 Medical o 1.17 News o 1.18 People o 1.19 Real estate / property o 1.20 Television o 1.21 Video Games 2 By information type o 2.1 Forum o 2.2 Blog o 2.3 Multimedia o 2.4 Source code o 2.5 BitTorrent o 2.6 Email o 2.7 Maps o 2.8 Price o 2.9 Question and answer .
    [Show full text]
  • Architecture Based Study of Search Engines and Meta Search Engines for Information Retrieval A
    International Journal of Engineering Research & Technology (IJERT) ISSN: 2278-0181 Vol. 2 Issue 5, May - 2013 Architecture Based Study Of Search Engines And Meta Search Engines For Information Retrieval A. Madhavi1,K. Harisha Chari2 1Asst.Professor, Matrusri Institute of PG Studies, Hyderabad, India. 2Asst.Professor, K.V Ranga Reddy College, Hyderabad, India. Abstract not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to WWW is a huge repository of information. republish to post on servers or to redistribute to The complexity of accessing the web data has lists, requires prior specific permission and/or a fee. increased tremendously over the years. There is Single search engine are increased coverage and a a need for efficient searching techniques to consistent interface. A recent study by Lawrence extract appropriate information from the web, as and Giles estimated the size of the web at about 800 million indexable pages. This same study the users require correct and complex concluded that no single search engine covered information from the web. more than about sixteen percent of the total. By This paper analyses the architectures and searching multiple search engines simultaneously features of metasearch engines for searching and via a metasearch engine, coverage increases receiving documents on single and multiples dramatically over searching only one engine. domains on the web. A web search engine searches Lawrence and Giles found that combining the for information in WWW. results of 11 major search engines increased the coverage to about 42% of the estimated size of the 1.
    [Show full text]
  • An Intelligent Metasearch Engine for the World Wide
    AN INTELLIGENTMETASEARCH ENGINE FOR THE WORLDWIDE WEB Andrew Agno A thesis submitted in conformity with the requirements for the degree of Master of Science Graduate Department of Cornputer Science University of Toronto Copyright @ 2000 by Andrew Agno National Library Bibliothèque nationale ($1 of Canada du Canada Acquisitions and Acquisitions et Bibfiographic Services services bibliographiques 395 Wellington Street 395. rue Wellington OttawaON K1AON4 Ottawa ON K1A ON4 Canada Canada The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Lïbrary of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sel1 reproduire, prêter, distribuer ou copies of this thesis in rnicrofonn, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/film, de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts ffom it Ni la thèse ni des extraits substantiels may be p~tedor otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. tract An Intelligent Met asearch Engine for the n'orld nïde Uéb .Anchen- Agno Mas ter of Science Graduate Department of Cornputer Science Cniversi ty of Toronto 2000 Uachine learning and informat ion retried techniques are appliecl t o met asearch on the Vorld n'ide ?\éb as a means of providing user specific relennr documents in respome to user queries.
    [Show full text]
  • SAVVYSEARCH: a Metasearch Engine That Learns Which Search
    AI Magazine Volume 18 Number 2 (1997) (© AAAI) Articles SAVVYSEARCH A Metasearch Engine That Learns Which Search Engines to Query Adele E. Howe and Daniel Dreilinger ■ Search engines are among the most successful single query to multiple search engines. applications on the web today. So many search The simplest metasearch engines are forms engines have been created that it is difficult for that allow the user to indicate which search users to know where they are, how to use them, engines should be contacted (for example, and what topics they best address. Metasearch ALL-IN-ONE [www.albany.net/allinone] and engines reduce the user burden by dispatching METASEARCH [members.gnn.com/infinet/ queries to multiple search engines in parallel. The meta.htm]). PROFUSION (Gauch, Wang, and SAVVYSEARCH metasearch engine is designed to efficiently query other search engines by carefully Gomez 1996; www.designlab.ukans.edu/pro- selecting those search engines likely to return fusion) gives the user the choice of selecting useful results and responding to fluctuating load search engines themselves or letting PROFUSION demands on the web. SAVVYSEARCH learns to iden- select three of six robot-based search engines tify which search engines are most appropriate using hand-built rules. METACRAWLER (Selberg for particular queries, reasons about resource and Etzioni 1995; metacrawler.cs.washing- demands, and represents an iterative parallel ton.edu:8080/home.html) significantly search strategy as a simple plan. enhances the output by downloading and analyzing the links returned by the search engines to prune out unavailable and irrele- ompanies, institutions, and individuals vant links.
    [Show full text]
  • Download Download
    International Journal of Management & Information Systems – Fourth Quarter 2011 Volume 15, Number 4 History Of Search Engines Tom Seymour, Minot State University, USA Dean Frantsvog, Minot State University, USA Satheesh Kumar, Minot State University, USA ABSTRACT As the number of sites on the Web increased in the mid-to-late 90s, search engines started appearing to help people find information quickly. Search engines developed business models to finance their services, such as pay per click programs offered by Open Text in 1996 and then Goto.com in 1998. Goto.com later changed its name to Overture in 2001, and was purchased by Yahoo! in 2003, and now offers paid search opportunities for advertisers through Yahoo! Search Marketing. Google also began to offer advertisements on search results pages in 2000 through the Google Ad Words program. By 2007, pay-per-click programs proved to be primary money-makers for search engines. In a market dominated by Google, in 2009 Yahoo! and Microsoft announced the intention to forge an alliance. The Yahoo! & Microsoft Search Alliance eventually received approval from regulators in the US and Europe in February 2010. Search engine optimization consultants expanded their offerings to help businesses learn about and use the advertising opportunities offered by search engines, and new agencies focusing primarily upon marketing and advertising through search engines emerged. The term "Search Engine Marketing" was proposed by Danny Sullivan in 2001 to cover the spectrum of activities involved in performing SEO, managing paid listings at the search engines, submitting sites to directories, and developing online marketing strategies for businesses, organizations, and individuals.
    [Show full text]
  • Role of Meta Search Engines in Web- Based Information System: Fundamentals and Challenges
    445 ROLE OF META SEARCH ENGINES IN WEB- BASED INFORMATION SYSTEM: FUNDAMENTALS AND CHALLENGES Subarna Kumar Das Abstract Meta search engines are powerful tools that search multiple search engines simulta- neously. Unlike crawler-based search engines such as Google, AllTheWeb, AltaVista and others, meta search engines generally do not build and maintain their own web indexes. Instead, they use the indexes built by others, aggregating and often post- processing results in unique ways. Meta search engines accept your query, and send it out to multiple search engines in parallel. They are often quite fast, using private “backdoor” servers made available by the search engines they query. They get this privileged status thanks to revenue sharing agreements. There are several advantages to using meta search engines, though they’re not always the most appropriate tool. The most obvious advantage is that you can get results from multiple search engines with- out having to visit each in turn. Apart from the time savings, there is some evidence that this gives your search a broader scope, since each individual search engine’s index differs from all others. Present paper attempts to highlights the role of meta search engine in web- based information retrieval Keywords: Search Engine, World Wide Web, Meta Search Engine, Metacrawler In- formation Storage, 1. Introduction A search engine is a program designed to help find information stored on a computer system such as the World Wide Web, inside a corporate or proprietary network or a personal computer. The search engine allows one to ask for content meeting specific criteria (typically those containing a given word or phrase) and retrieves a list of references that match those criteria.
    [Show full text]
  • Retrieval Efficiency of Search Engines on Medical Tourism in Kerala: a Webometric Analysis
    Retrieval Efficiency of Search Engines on Medical Tourism in Kerala: A Webometric Analysis Thanuskodi S, Naseehath S Alagappa University Karaikudi India ABSTRACT: World Wide Web is like an ocean of information; it is not easy to find specific information from it because the web is very huge and growing all the time. So there are certain tools used to search information from the web. They are called search engines. They search through all the web sites and create an index of the information of the web sites and act as a single point to find relevant information. E.g. Google, Yahoo, Bing, Lycos, Info seek etc. Analysis on 8 representative variables form all the subsectors of on Medical Tourism in Kerala, the search engine Lycos retrieves 59% of the total results and stands first followed by Bing in the second position with 37% and Google in third position with only 2% of results . The remaining less than 1% is shared by all Metasearch engines, of these Ixquick stands first with a slight difference from others followed by Dogpile and WebCrawler Analysis of 10 variables on Ayurveda tourism Lycos occupies first place with a high result of 85% and Google in the second position followed by Bing in the third position with a less Percentage of 7 and 6 respectively. Among Metasearch engines Ixquick stands first, followed by WebCrawler and Dogpile. Keywords: Search Engines, Retrieval Efficiency, Medical Tourism, Webometrics, Internet Received: 19 February 2019, Revised 4 April 2019, Accepted 8 May 2019 DOI: 10.6025/jism/2019/9/3/91-99 © 2019 DLINE.
    [Show full text]
  • A Study of Results Overlap and Uniqueness Among Major Web Search Engines
    COVER SHEET Spink, Amanda and Jansen, Bernard J. and Blakely, Chris and Koshman, Sherry (2006) A study of results overlap and uniqueness among major web search engines. Information Processing and Management 42(5):pp. 1379-1391. Copyright 2006 Elsevier. Accessed from: http://eprints.qut.edu.au/archive/00004755 1 A STUDY OF RESULTS OVERLAP AND UNIQUENESS AMONG MAJOR WEB SEARCH ENGINES Amanda Spink * Faculty of Information Technology Queensland University of Technology Gardens Point Campus, 2 George St, GPO Box 2434 Brisbane QLD 4001 Australia Tel: 61-7-3864-2782 Fax: 61-7-3864-2703 Email: [email protected] Bernard J. Jansen School of Information Sciences and Technology The Pennsylvania State University University Park PA 16802 Tel: (814) 865-6459 Fax: (814) 865-6426 Email: [email protected] Chris Blakely Market Strategy Manager Infospace, Inc. – Search & Directory 601 108th Ave NE, Ste 1200 Bellevue, WA 98004 USA Tel: (425) 709-8101 Fax: (425) 201-6162 Email: [email protected] Sherry Koshman School of Information Sciences University of Pittsburgh 611 IS Building, 135 N. Bellefield Avenue Pittsburgh PA 15260 Tel: (412) 624-9441 Fax: (412) 648-7001 Email: [email protected] * To whom all correspondence should be sent. 2 .ABSTRACT The performance and capabilities of Web search engines is an important and significant area of research. Millions of people world wide use Web search engines very day. This paper reports the results of a major study examining the overlap among results retrieved by multiple Web search engines for a large set of more than 10,000 queries.
    [Show full text]
  • Improving Search Engines Via Classification
    Improving Search Engines via Classi¯cation Zheng Zhu May 2011 A Dissertation Submitted to Birkbeck College, University of London in Partial Ful¯llment of the Requirements for the Degree of Doctor of Philosophy Department of Computer Science & Information Systems Birkbeck College University of London Declaration This thesis is the result of my own work, except where explicitly acknowledge in the text. Zheng Zhu i Abstract In this dissertation, we study the problem of how search engines can be improved by making use of classi¯cation. Given a user query, traditional search engines output a list of results that are ranked according to their relevance to the query. However, the ranking is independent of the topic of the document. So the results of di®erent topics are not grouped together within the result output from a search engine. This can be problematic as the user must scroll though many irrelevant results until his/her desired information need is found. This might arise when the user is a novice or has super¯cial knowledge about the domain of interest, but more typically it is due to the query being short and ambiguous. One solution is to organise search results via categorization, in particular, the classi¯cation. We designed a target testing experiment on a controlled data set, which showed that classi¯cation-based search could improve the user's search experience in terms of the numbers of results the user would have to inspect before satisfying his/her query. In our investigation of classi¯cation to organise search results, we not only consider the classi¯cation of search results, but also query classi¯cation.
    [Show full text]