How to get the most from free news search

Lars Våge Mid Sweden University, Sweden

Online Information Conference, Wednesday 1st December 2010 Free news search at a glance

• Free news search on the web has been available since 1996 • News search engines came into focus and developed significantly after 9/11 • Excellent near real-time indexing of thousands of news sites, news agencies, broadcast news and press release services • Several employ news story clustering or topical clustering • Some offer advanced search and keyword-based search feeds and email news alerts that can be used for free monitoring • Some with paid content offer great free search capabilities The best free news search engines

• Most offer a back archive of only one month or so

• Google News (2002-) with news story clustering, 60 regional editions and GN Archive Search (2006-)

• Yahoo! News (1996-) can search in 40 languages at once • Topix (2004-) employs topical clustering and has a huge community commenting stories • NewsNow (1998-) the original UK news SE, great topical clustering but only search in headlines for free • Bing News offers news story clustering, fewer search options In Yahoo! News you can choose any set of languages

In Google News you can’t mix languages Newsnow offers smart navigating of news topics

Comments on articles in Topix

Bing News uses story clustering News search engines w. paid content

• They provide free search but you have to pay to view articles • Use their search engines, try to find the articles on the free web or in the Internet Archive • Use phrase searches of titles in general web search engines

• Highbeam Research has both free and paid content and tremendous archives of stories and PR wires • Newslibrary.com has free search and is great for U.S. content • ProQuest Archiver has free search and exclusive archives of majors like Washington Post, LA Times and USA Today Newslibrary offers great search capabilities for U.S. news articles

Highbeam has a good interface for finding news publications Add blog search to find news stories

• Search the blogosphere for comments/linking to news stories

• Find blog posts that link to and comment on an already known news story • Use trending tools to find unknown stories that bloggers link to • Composite trend graphs showing frequencies of several keywords at a time in Blogscope, Blogpulse and Icerocket • Google Blog Search, Twingly and Technorati are good choices

• Combined news and blog search tools also available but some offer only limited indexing a.k.a. feed search engines Use Blogpulse to create composite trend graphs for keywords used in the blogosphere

Click on peaks to find out what happened that day News agency stories

• AP, , AFP, UPI, DPA, TASS, Interfax, Xinhua, Al- Jazeera • These are indexed by many NSEs but again only a month back • Monitor with NSEs like Google News and Yahoo! News with the option to restrict to a specific source

• Some have great web sites, sometimes with archived articles - Reuters, TASS, Xinhua (2001-), Al-Jazeera (2003-) • Highbeam Research has incredible coverage of Asian news agencies and wires - Korea, India, Indonesia, Pakistan

• Don't forget the alternatives - , IndyMedia Use Google News to set up monitoring of individual news sources such as news agencies.

Don’t forget to sort by date! Broadcast news stories and videos

• Try the internal search engines of broadcast news web sites • They sometimes have better search capabilities than the NSEs, sometimes not • They may have several years worth of free archives • Their search often include video, audio and photos as well • ABC News, CBS News, BBC News and CBC News are particularly rich in content and have good search features • Check out the BBC Motion Gallery with thousands of high quality video clips, some royalty-free ABC News and CBS News has very good post-search refining tools

Hidden treasures in the CBC archives - early interviews with singer-songwriter Joni Mitchell Finding press releases

• Very often indexed by the NSEs but with shallow archives and it is similar at their own sites

• Try using the fee-based NSEs with free search • Highbeam Research has outstanding archives of PRs (M2 PW 1988-, PR NW 1989-), with broad international coverage • Full text of older press releases are often in the Internet Archive

• PR aggregators with press rooms - PressePortal, Categorynet

• Science news press releases have their own SEs and aggregators - EurekAlert (1996-), ScienceDaily (1995-) Higheam Research has amazing coverage of the most important press release wires Evaluating news search tools

• What types of sources are indexed and how deep is the indexing? Is it the full text or only headlines being indexed?

• How quickly is the index updated? Perform your own tests! • What languages are covered by the search engine?

• If they use any form of clustering, how well does it work? • Do they offer any advanced search features? • Do they have spelling correction and suggestions as you type? • Do they offer monitoring with RSS and/or email alerts? News monitoring with RSS feeds

• News SEs such as Google News and Yahoo! News offer great opportunities for free business intelligence using search feeds

• In these SEs limitations to single news sources can be applied

• Always date sort the results list before grabbing the feed link

• Except your news search feeds to get broken from time to time

• Backup your news search subscriptions as an OPML file • Use Google News regional editions or Yahoo! News advanced search for monitoring non-English sources Reading headlines in the Netvibes hosted feed reader

Tools for importing and exporting OPML files in Google Reader Choosing feed readers

• Feed readers embedded in browsers and email programs are easy to start with but can be limiting • Web hosted feed readers offer much more flexibility, can be accessed from everywhere • Good choices are for instance Google Reader and Netvibes • Dedicated news aggregator software can have excellent extra features and be free to use too like FeedDemon • Try several feed readers by importing your OPML file • Don't forget news alerts delivered by email are available too Finding really old news stories

• More and more good news for those searching old news stories • Many historical newspaper digitizing projects are active in the world on national, regional and local scales • Most projects are led by larger libraries and the content is free to search, read and even download • Gallica, Die Digitale Bibliothek, Gazettes Online, Google

• Free full text can also be offered by newspapers themselves, outstanding examples are New York Times and Die Zeit • Search in OCR-generated transcripts of news articles can be difficult and error-prone but ultimately very rewarding Using the Gazettes Online to find official coverage of historic events

The official account of The Great Fire of London in 1666 Finding news stories in social media

• Microbloggers using services such as Twitter link extensively to news stories and can be followed for early warning • Links are obscured by the use of URL shortening services • Real blogs offer much more in-depth comments on stories • Tweepz by Exalead is a great alternative to the Twitter search engine for finding tweets • Many specialized directories of tweeters are now available • Muckrack.com is a good place to start for finding twittering journalists (especially in the U.S. and UK) to follow Using Muckrack to find journalists who twitter In conclusion

• Free NSEs are great for finding the very latest stories but their archives normally stop at one month • Fee-based NSEs can offer free searching which you should take advantage of to find older news and press releases

• With patience you may be able to find the full text in the IA

• Google News and Yahoo! News have advanced searching that can be used to monitor keywords in specific sources via feeds

• Try several feed readers to find the best for you with OPML • Monitor blogs and follow Twitter streams to get the latest buzz