Duckduckgo Respects Our Privacy Fast, Simple Search with an Open Customer-Friendly Ecosystem

Home , DuckDuckGo

Patricia Seybold Group / Product Review DuckDuckGo Respects Our Privacy Fast, Simple Search with an Open Customer-Friendly Ecosystem

By Patricia B. Seybold, CEO & Sr. Consultant, Patricia Seybold Group June 28, 2013

NETTING IT OUT DuckDuckGo is a search engine that is worth switching to as your default. It works on the Web and as an app on mobile devices. It is probably the most customer-centric search engine you’ll find; not because it personalizes results, but because it doesn’t. DuckDuckGo has a number of advantages over Google and other commonly used search engines.

First, it does not track your searches, nor does it capture what you click on. So your privacy is safe and marketers can’t follow you around with their annoying ads based on things they think you care about. And, if a government or law enforcement agency asks to see any information about what you’ve been looking up, DuckDuckGo can’t provide that information because they don’t have it.

Second, it provides a clean, uncluttered set of results, often with an “Instant Answer” at the top of the list. This is NOT a sponsored search. It is, in fact, the most likely answer you’re seeking.

Third, it quickly gives you the authoritative site you may be seeking, e.g., a quick link to a company’s official web site. (These links may be sponsored, but they aren’t misleading, e.g., if you are looking for American Airlines, you won’t see a competitor featured in the search results.)

Fourth, it is an open platform for customer innovation. People can suggest and request vertical search (e.g., topic-specific) search engines they’d like to have included. Developers can offer their own integrations for specific Web sites, vertical search engines, and/or databases.

Fifth, it is built on top of Bing and Yahoo Search, but also prioritizes crowdsourced answers from sites like Wikipedia, Quora, StackExchange, and other Q&A sites where experts hang out and answer one another’s questions.

Gabriel Weinberg, CEO and founder of DuckDuckGo

Direct link: http://dx.doi.org/10.1571/pp06-28-13cc Customer Scenario and Customers.com are registered trademarks and Customer Flight Deck and Quality of Customer Experience (QCE) are service marks of the Patricia Seybold Group Inc. • P.O. Box 783, Needham, MA 02494 USA • www.customers.com • Unauthorized redistribution of this report is a violation of copyright law. 2 • Product Review

DuckDuckGo gained quite a bit of attention when it advertised on this billboard in early 2011 in Google’s back yard on Highway 101 near San Francisco.

WHY USE DUCKDUCKGO INSTEAD OF GOOGLE OR BING?

None of Your Searches Are Tracked, Connected Together, Nor Connected to You With the recent disclosures that the National Security Agency is tracking everything we do online as well as our phone calls, many people have become much more concerned about the lack of privacy for their online or phone communication. Although we might have been somewhat paranoid before, we now realize that everything we do online—every search we do, every site we navigate, every click we make—is being vacuumed up into huge big data repositories that are mined by marketers, but can also be analyzed by police or investigators, typically without our knowledge or con- sent. One of the easiest ways to avoid contributing to the stash of data being gathered about you is to change the search engine you use from Google, or Bing, or something else, to DuckDuckGo. How does this work? How does DuckDuckGo avoid collecting information about you? Particu- larly since their business model relies on being paid for clickthroughs?

Customers.com® Research Service Unauthorized redistribution of this report is a violation of copyright law. © 2013 Patricia Seybold Group For Reprints/Redistribution rights, contact: [email protected] DuckDuckGo Respects Our Privacy • 3

Duck Duck Go’s Privacy Policy is very clear. The site does not collect or share any personal information. That means that even if a government agency requests information about you, DuckDuckGo does not have anything to give them.

Here’s their explanation: “When you search at DuckDuckGo, we don't know who you are and there is no way to tie your searches together.

When you access DuckDuckGo (or any Web site), your Web browser automatically sends information about your computer, e.g. your User agent and IP address.

Because this information could be used to link you to your searches, we do not log (store) it at all. This is a very unusual practice, but we feel it is an important step to protect your privacy.

It is unusual for a few reasons. First, most server software auto-stores this information, so you have to go out of your way not to store it. Second, most businesses want to keep as much information as possible because they don't know when it will be useful. Third, many search engines actively use this information, for example to show you more targeted advertising.

Another way that your searches are often tied together at other search engines are through browser cookies, which are pieces of information that sit on your computer and get sent to the search engine on each request. What search engines often do is store a unique identifier in your browser and then associate that identifier with your searches. At DuckDuckGo, no cookies are used by default.

In response to efforts by the EFF and others, the major search engines have begun ‘anonymizing’ their search log data after periods of time. Sure, this is better than not doing so, but you should note that this does not make your search history anonymous in the same way that it is at DuckDuckGo.

What search engines generally do when they anonymize data is get rid of part of your IP address or turn it into something that doesn't look exactly like an IP address. And they do the same thing for uniquely identifying cookies.

However, in many cases, this so-called anonymous data can still tie your searches together, which can be used to reconstruct who you are and what you searched for. Additionally, search engines usually are silent on what they do with the User agent, which has been shown to also have enough information to often be personally identifiable, especially if isolated to a particular search session (day)…..

At DuckDuckGo, no cookies are used by default. If you have changed any settings, then cookies are used to store those changes. However, in that case, they are not stored in a

Unauthorized redistribution of this report is a violation of copyright law. Customers.com® Research Service For Reprints/Redistribution rights, contact: [email protected] © 2013 Patricia Seybold Group 4 • Product Review

personally identifiable way. For example, the large size setting is stored as 's=l'; no unique identifier is in there….

We also save searches, but again, not in a personally identifiable way, as we do not store IP addresses or unique User agent strings. We use aggregate, non-personal search data to improve things like misspellings.

Similarly, we may add an affiliate code to some eCommerce sites (e.g. Amazon & eBay) that results in small commissions being paid back to DuckDuckGo when you make purchases at those sites. We do not use any third parties to do the code insertion, and we do not work with any sites that share personally identifiable information (e.g. name, address, etc.) via their affiliate programs. This means that no information is shared from DuckDuckGo to the sites, and the only information that is collected from this process is product information, which is not tied to any particular user and which we do not save or store on our end. … This whole affiliate process is an attempt to keep advertising to a minimal level on DuckDuckGo…..

Like anyone else, we will comply with court ordered legal requests. However, in our case, we don't expect any because there is nothing useful to give them since we don't collect any personal information.”

Increase in Use of DuckDuckGo Since NSA Spying Disclosures

You can’t miss the spike in the use of DuckDuckGo for direct search queries since the NSA story broke on June 6th, 2013. As of June 20th, there were 3.5 million searches per day using DuckDuckGo– up from about 1.9 million.

I don’t know about you, but this explanation makes me feel pretty comfortable. There are more explanations on the site and in the user group discussions about how to make your searches end-to- end encrypted. There is good, open debate about what privacy really is and means. But, bottom line, I feel that DuckDuckGo is responsive to what most users are concerned about: government invasion of privacy, and marketers’ spamming us based on things we searched for in the past.

Everyone Sees the Same Search Results No Personalization. Your searches aren’t filtered based on what you’ve clicked on before. There’s no personalization engine telling DuckDuckGo that you prefer leftwing or rightwing news sources, or that you like certain kinds of music more than others. DuckDuckGo refers to this as avoiding the “Filter Bubble.” You won’t be shown search results that are tailored based on your search history, nor based on any inferences made about you from your search history.

Other Search Engines Filter What You See Based on Who They Think You Are

Gabriel Weinberg uses this example to hammer home the point that most search engines will filter the search results they show you based on your past history clicking on search results. If the engine “notices” that you usually click on stories from Fox News and other similar media sources, results from those sources will appear much higher in the search results you’re shown. So each of us actually “sees” different things. Weinberg refers to this phenomenon as the “Filter Bubble.”

If you’re interested in learning more about the ways in which other search engines use your search history to skew the results you see, I recommend DuckDuckGo’s Filter Bubble tutorial, called “Escape Your Search Engine’s Filter Bubble” at dontbubble.us.

Is Non-Personalized Search Better or Worse? Having spent many years “teaching” Google search what I care about, I’ve developed an appre- ciation for the fact that Google almost always guesses correctly about what information I want to see and what information I find useful when I do a query. So I was concerned when I switched to Duck- DuckGo that I wouldn’t be able to research and find the information I need to do my job quickly enough. I have been pleasantly surprised. I find that I am getting good, useful, informative answers on the first page every time. I rarely feel the need to try the next pages (which is not always true with Google) because with DuckDuckGo, the top three or four search results are meaty and have the information I was hoping to find.

More Useful Search Results at DuckDuckGo

Note that both DuckDuckGo and Google show the same ABCNews article dated back in 2010. But it’s the top hit on Google and the 5th in DuckDuckGo. However, note that the sponsored link is more relevant on Google than on DuckDuckGo.

You Can Avoid Marketing Profiling Marketers use the information they glean from your searches and clicks to offer you things they think you might want (e.g., a better airfare for a trip to Paris after you searched for a hotel in Paris). But often merchants use that information to change the pricing they offer you.

Your Search History Can Cost You Money

DuckDuckGo’s CEO, Gabriel Weinberg, used this example in a recent speech he gave at the GEL 2013 conference to point out that many marketers change the prices they charge based on your search profile and activity. Watch video of Gabriel Weinberg at Gel 2013.

If you want to learn more about how other search engines are tracking you and how that information can be detrimental to the quality of your life and to how much you pay for things, take a look at DuckDuckGo’s tutorial on the subject: Don’t Track Us. In short, DDG doesn’t know or care whether you’re a lawyer or a doctor, or whether you are interested in cures for AIDS or Herpes.

HOW DOES DUCKDUCKGO WORK?

Search Engine that Aggregates & Improves on Other Search Engines DuckDuckGo’s CEO and founder, Gabriel Weinberg, describes DuckDuckGo as a “search engine,” but I believe that it’s actually a combination of a search aggregator, a hybrid search engine, and an open search ecosystem. Here’s how DuckDuckGo describes the sources of its information: “DuckDuckGo gets its results from over one hundred sources, including DuckDuckBot (our own crawler), crowd-sourced sites (like Wikipedia, which are stored in our own index), Yahoo! (through BOSS), Yandex, WolframAlpha, and Bing. For any given search, there is usually a vertical search engine out there that does a better job at answering it than a general search engine. Our long-term goal is to get you information from that best source, ideally in instant answer form.

You will notice that the results of our sources and our results are often very different. That's because we have an intelligence layer on top that attempts to improve upon results provided by external sources. Not only do we try to pick the best source and use it in the best manner, but we also do other things like remove spam, re-rank, provide instant answers and add official sites on top, among other things.

While our indexes are getting bigger, we do not expect to be wholly independent from third-parties. Bing and Google each spend hundreds of millions of dollars a year crawling and indexing the deep Web. It costs so much that even big companies like Yahoo and Ask are giving up general crawling and indexing. Therefore, it seems silly to compete on crawling and, besides, we do not have the money to do so. Instead, we've focused on building a better search engine by concentrating on what we think are long-term value- adds — having way more instant answers, way less spam, real privacy and a better overall search experience.”

Crawling the Web: YahooSearch. Gabriel Weinberg did his own Web crawling using DuckDuckBot to come up with the DuckDuckGo search database starting in 2007. (DuckDuckGo was incorporated in September, 2008.) In the meantime (July 10, 2008), Yahoo had launched a service called Build Your Own Search Service, or BOSS, and invited developers to use its open interface to build their own customized search engines on top of Yahoo Search. Here’s what a developer like Gabriel Weinberg could do with Yahoo’s BOSS when it was ini- tially launched, according to Programmable Web’s John Musser: “Ability to re-rank and blend results — BOSS partners can re-rank search results as they see fit and blend Yahoo!’s results with proprietary and other web content in a single search experience Total flexibility on presentation — Freedom to present search results using any user interface paradigm, without Yahoo! branding or attribution requirements BOSS Mashup Framework — We’re releasing a Python library and UI templates that allow developers to easily mashup BOSS search results with other public data sources Web, news and image search — At launch, developers will have access to web, news and image search and we’ll be adding more verticals soon Unlimited queries — There are no rate limits on the number of queries per day The RESTful API (our profile) lets developers query up to 50 items at a time with results returned in XML or JSON. Developers can choose to search from the web, news, image or spelling indexes.” ~ John Musser, Yahoo’s BOSS Advances Search APIs

So, over time, Weinberg switched from focusing on his own Web Crawler to relying more heav- ily on Yahoo’s. On March 25, 2010, Gabriel Weinberg explained, “I started with my own crawler before BOSS existed and then switched that piece to it (highly modified of course) to concentrate on where the marginal benefit lies.” Yahoo Search Replaced by Microsoft Bing. But what is Yahoo Search under the covers? It’s Mi- crosoft Bing. On July 29, 2009, Yahoo and Microsoft signed a ten-year agreement in which Bing would power YahooSearch. So, under the covers, DuckDuckGo uses Microsoft’s Bing among other search engines—both directly and indirectly, by accessing searches from Yahoo (which is powered by Bing) thru BOSS. Prioritize Crowdsourced Sources. DuckDuckGo’s search results also give a high priority to crowdsourced sources of information, like Wikipedia and Wookieepedia (Star Wars Encyclopedia). They have found StackExchange to also be great resource for a variety of questions. Quora is another good crowdsourced Q&A site. Often the best answer from a crowdsourced site shows up as an “In- stant Answer” in a DuckDuckGo Search. You’ll see other likely sites that may be relevant for.

StackExchange is one source of crowdsourced Q&A that is often used by DuckDuckGo to provide the best answer to a query.

Include Social Media Search. Among DuckDuckGo’s quoted sources is Yandex (a Russian search engine that integrates public social media including Twitter into its search results). DuckDuckGo searches also include Twitter Lists.

Include Quick Site-Specific Look-Ups. Often as searchers, we know which site might have the answer we’re seeking, but we’re loathe to click off to another website. DuckDuckGo has sup- ported !bang searches since the very early days of DuckDuckGo. They were introduced in 2008, and anyone can submit one, according to Gabriel Weinberg. You use the !bang syntax which can be found here to integrate a site-specific search into DuckDuckGo. “Generally they are approved as they usually work with the site name,” he said.

Quick Search on a Linked Site

!Search: Referred to as “Bang” Search (slang for exclamation point) has been offered by DuckDuckGo since 2008. You type “!” followed by the website name, and if someone has used the DuckDuckGo API to link the two sites, your search will be carried out on THAT site’s search engine. Here, I searched for “IMF” on the Financial Times site and got good results. If the sites haven’t yet been linked, you just get a list of normal search results. The web site owner doesn’t have to make the linkage. Anyone can do it, using the published API.

Search Engine that Focuses on “Instant Answers” As often as possible, DuckDuckGo provides a single “Instant Answer” to a search query. This is never a sponsored link. It is simply the best shot at the most informative result. “We only show an instant answer or one instant answer over another based on relevancy,” Gabriel explains.

Examples of Instant Answers

Many search queries on DuckDuckGo show a reasonably complete answer at the top of the search results. These are called “Instant Answers.” Here are three examples. A search for Christine Lagarde shows a useful snippet about her from Wikipedia. A query about the current population of Uganda delivers the answer from WolframAlpha. A query for “Khan geometry” takes you to the library of free online courses on Khan Academy’s site.

This focus on Instant Answers is likely to be really valuable as DuckDuckGo rolls out its mobile apps. “Our autocomplete is a unique and great way to quickly search on the go. We blend our instant-answers with other suggestions so you autocomplete to answers, instead of more searches. Less searching, more finding.”

DuckDuckGo Keeps Clutter to a Minimum Gabriel Weinberg isn’t a fan of eye candy and distractions. He believes that his users want a clean, uncluttered search experience, more akin to the “original” Google experience. He points out that Google’s typical results pages are now predominantly ads, including ads for other Google sites. DuckDuckGo search results never have more than one “sponsored” link which is typically right be- low the Instant Answer and is clearly marked. DDG uses the real estate in the right-hand column to list other search sites that are most likely to have more similar answers—these are often crowdsourced sites. The list of recommended links changes depending on the contents of your search, not based on who you are.

Search and Stories: Example of New DuckDuckGo Stories Home Screen

You can replace the “Stories” home page view with your most recent saved searches. You can also see what sources DuckDuckGo is using to present the most “popular” stories at any time. Presumably you’ll be able to delete categories and sources you don’t care to see.

DUCKDUCK GO INTRODUCES MOBILE APPS

Focus on Quick Search On June 27, 2013 DuckDuckGo launched its mobile apps for iPhone and Android. There is also a Windows phone app. What’s better about the way DuckDuckGo works than many mobile searches is that, as you type in your search string, it presents the Instant Answers for the first characters as you type. Other search apps make you type the search string and THEN get the result. The chances are good that you’ll find what you’re seeking faster with DuckDuckGo on your Mobile device. With DuckDuckGo, you are highly likely to get the most relevant result before you’ve completed your search string.

Searching with DuckDuckGo’s Mobile App

DuckDuckGo’s Mobile Search App. On the left: As you type, you see the Instant Answers for the most likely auto-completed words or phrases. On the right: Top search results from a variety of sources based on what you’ve selected—not based on who or where you are.

Popular Stories Many people like to use their mobile phones to track what’s going on in the world. The mobile app’s home page features the most recent and most interesting crowdsourced “stories”—replete with photos. (You can replace that screen to show your most recent searches if you prefer.) These top stories are selected so they’re not depressing or sensationalized, and you won’t see eight versions of the same breaking story (unless you search on it to learn more). Here’s how DDG describes its “stories” feature: “Stories you'll love. We only feature the most shared stories from hand-crafted sources: • Just #1 stories like most emailed or upvoted • No depressing news or graphic content • No repeat topics per day (even across sources) • Readability view with search suggestions that tie Search & Stories together.”

You can view the sources being used. Hopefully, you’ll be able to prioritize the sources you prefer and drop those you don’t like. While this feature may be a nod to consumer taste, my own hunch is that most of us will eschew the stories feature and stick with the search interface—so that what we see when we open the app are the last searches we did and the instant results for the new search we’re doing now.

DUCKDUCKGO’S OPEN ECOSYSTEM FOR CUSTOMER-LED INNOVATION

Community Forum for Customer Service and Ideas There’s a vibrant online community forum called Duck.co. It currently is hosted on the Zoho community platform which means you have to agree to the Zoho privacy policy in order to use it as a registered user. (This seems a bit counter to the DuckDuckGo privacy stance, but it’s clearly spelled out.) You can post and see the answers to questions and problems, see and participate in discussions on a variety of topics, and offer ideas for improvement.

DuckDuckGo hosts a very active Community Forum online community. People answer each others’ questions, solve each others’ problems, and propose ideas for new functionality. Here we see the Ideas section in which several ideas are either “under review” or “implemented” or “will not implement.”

Community Development Platform DuckDuckGo’s Community Platform is similar to the one Mozilla pioneered. It includes places for non-technical folks to help evolve the functionality and platform—for example, by helping to translate entries. It includes an area to help market and evangelize DuckDuckGo. And, it includes a link to the open source development environment, DuckDuckHack, where you can help extend the code.

Easy to Contribute Search Sources & Instant Answers DuckDuckHack was launched in the spring of 2012. “Since then,” Gabriel Weinberg says, “we've added over 200 various types of instant-answers. DuckDuckGo is evolving rapidly as a result of customer-contributed ideas and sources.” Gabriel commented, “We getting more and more plugin contributions/updates every day. Our ideas page has grown, as well. There are over 300 ideas being contributed to at this moment.” One of the most interesting developments, in my opinion, is the addition of additional sources for “longtail” highly specialized searches. For example, you can now search for LEGO parts directly from DuckDuckGo.

LEGO Parts Search Implemented in DuckDuckGo

Here’s an example of a custom vertical search developed as an open source add-on. If you type in a LEGO part number, you’ll get a result directly out of the database of LEGO’s customer-created parts and designs.

As the whole platform is open-source, the community plays a large part in nominating sources for various ideas and coming up with the ideas in the first place.” I asked Gabriel how they vet these Instant Answer suggestions to ensure that they aren't just promoting a product or service? Here’s what he said: “When an Instant Answer plugin hits our doorstep for integration, we work with the plugin authors to arrive at the best solution for the users. The main criteria is that the plugin output has to be more useful than the organic links because we show it above them. We're happy to promote a source as an instant answer, so long as it's the best source for that data, like IMDB for movies. There are guidelines in the documentation.

“We're not currently showing any ads or sponsored links in the instant answers, nor do we have any plans to. We only show an instant answer or one instant answer over another based on relevancy. We do make some anonymous affiliate revenue from Amazon and eBay links if people end up buying from them, but these links are not sponsored and these programs do not influence ranking.” ~ Gabriel Weinberg, CEO and Founder, DuckDuckGo

DuckDuckHack is another place where customers post ideas of functionality they’d like to develop or have developed by others. Note that there are four different kinds of plugins you can develop here: API calls, Perl functions, keyword data, or full-text data.

I asked Gabriel how much effort it takes on DDG’s part to vet developers’ submissions. Here’s his reply: It depends on the developer's abilities and the type of plugin that it is. For something that's easily triggered but may not always give a great instant answer, it takes time to find the right balance of when/when not to show it. For obvious things, like movie ratings or recipes, they're easy to vet and, if the developer did a great job with their code, it makes it that much easier.

We've gone through iterations with our documentation and coding interface, and it is getting easier to use every day. The easier it is, the better submissions we get, and the less vetting is necessary. We are also trying to encourage more and more people in the community to help us with this process.” ~ Gabriel Weinberg, CEO and Founder, DuckDuckGo

WHAT’S MISSING AND WHAT NEEDS TO BE ADDED NEXT

Image Search & Filter by Date I’ve only been using DuckDuckGo as my main search engine for a month or so. The two areas of missing functionality that often send me back to Google are image search and the ability to filter search results so that I only see answers that are current (within the last month, week, or year). The good news is that both of these are under development by the open source community in conjunction with DuckDuckGo.

COME FOR THE PRIVACY; STAY FOR THE UTILITY

Try It! You’ll Like It! I have to admit that I’ve been trying to switch to DuckDuckGo for over a year. But habits are hard to break, and I found myself heading back to Google search with great regularity. It wasn’t until Edward Snowden’s disclosure about how much my online behavior may be subjected to government scrutiny that I began to get unhappy about being the target of government surveillance. Now that I’ve made DuckDuckGo the default search on the browsers I most commonly use, I rarely need to stray. I know that there are lots of features and shortcuts I have yet to discover. I am also really happy to be part of a customer community that is constantly evolving and suggesting new search sources and features. I hope you’ll join me.

ABOUT THE AUTHOR

With 30 years of experience consulting to customer-centric executives in technology-aggressive businesses across many industries, PATRICIA B. SEYBOLD is a visionary thought leader with the unique ability to spot the impact that technology enablement and customer behavior will have on business trends very early. She assesses and predicts how new and evolving technologies will impact customers. She forecasts the ways in which both business and consumer customers will make new demands on companies in many different industries.

Seybold provides customer-centric executives within Fortune 1000 companies with strategic insights, technology guidance, and best practices. Her hands-on experience, her discovery and chronicling of best practices, her deep understanding of information technology, her large, loyal client base, and her ongoing case study research enhances the thought leadership she provides.

Seybold uses a coaching, mentoring, and learn-by-doing consultative approach to help clients achieve their goals as they transform their corporate cultures to be more customer-centric. She helps her clients’ teams redesign their businesses from the outside in by inviting their customers to invent new streamlined ways of accomplishing their desired outcomes, using their own real- world scenarios.

Patricia Seybold Group Trusted Advisors to Customer-Centric Executives

If you're a visionary customer-focused executive, the Patricia Seybold Group should be your first choice for ongoing strategic advice, business and technology guidance, customer experience best practices, and help with customer-centric initiatives.

Founded in 1978 and based in Boston, we provide consulting, research and advisory services, peer groups, and interactive workshops. We help clients to design and continuously improve their customer-focused business strategies and processes using our proven consulting methodology, Customer Scenario® Design.

The CEO and founder, Patricia Seybold, is the New York Times best-selling author of Customers.com and The Customer Revolution. Patty's latest book, Outside Innovation, is now available.

Patricia Seybold Group P.O. Box 783 Needham, MA 02494 Phone: (617) 742-5200 Fax: (617) 742-1028 Email: [email protected] Web: http://www.customers.com/