Information Rereival, Part 1
Total Page:16
File Type:pdf, Size:1020Kb
11/4/2019 Information Retrieval Deepak Kumar Information Retrieval Searching within a document collection for a particular needed information. 1 11/4/2019 Query Search Engines… Altavista Entireweb Leapfish Spezify Ask Excite Lycos Stinky Teddy Baidu Faroo Maktoob Stumpdedia Bing Info.com Miner.hu Swisscows Blekko Fireball Monster Crawler Teoma ChaCha Gigablast Naver Walla Dogpile Google Omgili WebCrawler Daum Go Rediff Yahoo! Dmoz Goo Scrub The Web Yandex Du Hakia Seznam Yippy Egerin HotBot Sogou Youdao ckDuckGo Soso 2 11/4/2019 Search Engine Marketshare 2019 3 11/4/2019 Search Engine Marketshare 2017 Matching & Ranking matched pages ranked pages 1. 2. query 3. muddy waters matching ranking “hits” 4 11/4/2019 Index Inverted Index • A mapping from content (words) to location. • Example: the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat 5 11/4/2019 Inverted Index the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3 cat 1 3 dog 2 3 mat 1 2 on 1 2 sat 1 3 stood 2 3 the 1 2 3 while 3 Inverted Index the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3 cat 1 3 dog 2 3 mat 1 2 Every word in every on 1 2 web page is indexed! sat 1 3 stood 2 3 the 1 2 3 while 3 6 11/4/2019 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3 cat 1 3 query dog 2 3 mat 1 2 cat on 1 2 sat 1 3 stood 2 3 the 1 2 3 while 3 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3 cat 1 3 query dog 2 3 mat 1 2 cat on 1 2 sat 1 3 stood 2 3 the 1 2 3 while 3 7 11/4/2019 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3 cat 1 3 the cat sat on dog 2 3 1 query the mat mat 1 2 cat on 1 2 the cat stood 3 sat 1 3 while a dog sat stood 2 3 the 1 2 3 while 3 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3 cat 1 3 the dog stood on 2 query dog 2 3 the mat mat 1 2 dog on 1 2 the cat stood 3 sat 1 3 while a dog sat stood 2 3 the 1 2 3 while 3 8 11/4/2019 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3 cat 1 3 query dog 2 3 mat 1 2 cat dog on 1 2 sat 1 3 stood 2 3 the 1 2 3 while 3 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3 cat 1 3 query dog 2 3 mat 1 2 cat dog on 1 2 sat 1 3 stood 2 3 the 1 2 3 while 3 9 11/4/2019 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3 cat 1 3 the cat stood 3 query dog 2 3 while a dog sat mat 1 2 cat dog on 1 2 sat 1 3 stood 2 3 the 1 2 3 while 3 Searching the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3 cat 1 3 query dog 2 3 mat 1 2 cat the sat on 1 2 ??? sat 1 3 stood 2 3 the 1 2 3 while 3 10 11/4/2019 Phrase Queries the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3 cat 1 3 the cat sat on dog 2 3 1 query the mat mat 1 2 “cat sat” on 1 2 the cat stood 3 sat 1 3 while a dog sat stood 2 3 the 1 2 3 while 3 Phrase Queries the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3 cat 1 3 the cat sat on dog 2 3 1 query the mat mat 1 2 “cat sat” on 1 2 the cat stood 3 sat 1 3 while a dog sat stood 2 3 the 1 2 3 while 3 How to tell if two words occur next to each other? 11 11/4/2019 Phrase Queries the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3 cat 1 3 the cat sat on dog 2 3 1 query the mat mat 1 2 “cat sat” on 1 2 the cat stood 3 sat 1 3 while a dog sat stood 2 3 the 1 2 3 while 3 How to tell if two words occur next to each other? EFFICIENTLY??? Inverted Index with Location the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3-5 cat 1-2 3-2 dog 2-2 3-6 mat 1-6 2-6 on 1-4 2-4 sat 1-3 3-7 stood 2-3 3-3 the 1-1 1-5 2-1 2-5 3-1 while 3-4 12 11/4/2019 Inverted Index with Location the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3-5 cat 1-2 3-2 query dog 2-2 3-6 mat 1-6 2-6 “cat sat” on 1-4 2-4 sat 1-3 3-7 stood 2-3 3-3 the 1-1 1-5 2-1 2-5 3-1 while 3-4 Inverted Index with Location the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3-5 cat 1-2 3-2 1-2, 3-2 query dog 2-2 3-6 mat 1-6 2-6 “cat sat” on 1-4 2-4 sat 1-3 3-7 1-3, 3-7 stood 2-3 3-3 the 1-1 1-5 2-1 2-5 3-1 while 3-4 13 11/4/2019 Inverted Index with Location the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat a 3-5 cat 1-2 3-2 1-2, 3-2 query dog 2-2 3-6 mat 1-6 2-6 “cat sat” on 1-4 2-4 sat 1-3 3-7 1-3, 3-7 stood 2-3 3-3 the 1-1 1-5 2-1 2-5 3-1 while 3-4 Inverted Index with Location the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3-5 cat 1-2 3-2 the cat sat on dog 2-2 3-6 1-2 1 query the mat mat 1-6 2-6 “cat sat” on 1-4 2-4 sat 1-3 3-7 1-3 stood 2-3 3-3 the 1-1 1-5 2-1 2-5 3-1 while 3-4 14 11/4/2019 NEAR* Queries the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3-5 cat 1-2 3-2 3-2 the cat stood dog 2-2 3-6 3 query 3-6 while a dog sat mat 1-6 2-6 cat NEAR dog on 1-4 2-4 sat 1-3 3-7 stood 2-3 3-3 the 1-1 1-5 2-1 2-5 3-1 while 3-4 *NEAR: distance <= 5 NEAR* Queries the cat sat on the dog stood on the cat stood 1 2 3 the mat the mat while a dog sat hits a 3-5 cat 1-2 3-2 3-2 the cat stood dog 2-2 3-6 3 query 3-6 while a dog sat mat 1-6 2-6 cat NEAR dog on 1-4 2-4 sat 1-3 3-7 stood 2-3 3-3 the 1-1 1-5 2-1 2-5 3-1 while 3-4 Useful in ranking! *NEAR: distance <= 5 15 11/4/2019 Matching & Ranking matched pages ranked pages 1. 2. query 3. muddy waters matching ranking “hits” Ranking & Relevance 1 By far the most common 2 Our cause was not cause of malaria is helped by the poor being bitten by an health of the troops, infected mosquito, but many of whom were there are also other suffering from malaria ways to contract the and other tropical disease.