How Browsers' Explanations Impact Misconceptions About Private

Home , Brave (web browser), In Private, MDN Web Docs, Opera (web browser), Surf (web browser)

Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

Yuxi Wu, Panya Gupta, Miranda Wei, Yasemin Acar†, Sascha Fahl†, Blase Ur University of Chicago, †Leibniz University Hannover {yuxiwu,panyagupta,weim,blase}@uchicago.edu, {acar,fahl}@sec.uni-hannover.de

ABSTRACT the history bar, auto-filled forms, or search suggestions. Private All major web browsers include a private browsing mode that does modes thus help users hide browsing sessions from people sharing not store browsing history, cookies, or temporary files across brows- a device with them. ing sessions. Unfortunately, users have misconceptions about what While these private modes aim not to save user data locally, this mode does. Many factors likely contribute to these misconcep- many threats to privacy are outside their scope. For example, pri- tions. In this paper, we focus on browsers’ disclosures, or their in- vate mode does not aim to prevent tracking over the network. While browser explanations of private browsing mode. In a 460-participant browser cookies do not persist across private browsing sessions, online study, each participant saw one of 13 different disclosures this is a minor barrier to tracking by third-party advertising and (the desktop and mobile disclosures of six popular browsers, plus a analytics companies, who can employ more sophisticated finger- control). Based on the disclosure they saw, participants answered printing techniques [14, 17]. Furthermore, implementations are questions about what would happen in twenty browsing scenarios imperfect, often leaving some traces of local activity [1, 36, 47]. capturing previously documented misconceptions. We found that Despite the limited protections private modes provide, some browsers’ disclosures fail to correct the majority of the misconcep- users overestimate these protections [13, 20]. This overestimation tions we tested. These misconceptions included beliefs that private reaches far; Eric Schmidt, former CEO of Google, once stated, “If browsing mode would prevent geolocation, advertisements, viruses, you’re concerned, for whatever reason, you do not wish to be and tracking by both the websites visited and the network provider. tracked by federal and state authorities, my strong recommendation Furthermore, participants who saw certain disclosures were more is to use incognito mode, and that’s what people do” [38]. likely to have misconceptions about private browsing’s impact on In this paper, we take initial steps toward unpacking these mis- targeted advertising, the persistence of lists of downloaded files, conceptions by conducting a user study of how browsers’ own and tracking by ISPs, employers, and governments. explanations of private mode impact users’ understanding of what these modes do. Browsers differ in how they explain private mode; CCS CONCEPTS even the mode’s name differs (e.g., “Incognito Mode” in Chrome, “InPrivate” in Edge, and “Private Browsing Mode” in Firefox). We • Security and privacy → Usability in security and privacy; focus on the disclosure, or full-page explanation browsers present Browser security; when users open a new window in private mode (the catch-all term KEYWORDS we use across browsers). Notably, disclosures often differ between a given browser’s desktop and mobile versions. Private browsing, Web browser privacy, Usable privacy, User study, We conducted a between-subjects online study in which 460 par- Incognito mode, Private browsing mode ticipants each saw one of 13 different disclosures (the desktop and ACM Reference Format: mobile disclosures of six popular browsers, plus a control disclosure) Yuxi Wu, Panya Gupta, Miranda Wei, Yasemin Acar†, Sascha Fahl†, Blase rebranded for a hypothetical new browser. Participants answered Ur. 2018. Your Secrets Are Safe: How Browsers’ Explanations Impact Mis- questions about what would happen in twenty different browsing conceptions About Private Browsing Mode. In WWW 2018: The 2018 Web scenarios. In some of these scenarios, private mode protects the Conference, April 23–27, 2018, Lyon, France. ACM, New York, NY, USA, user’s privacy, while the other scenarios encapsulate previously 10 pages. https://doi.org/10.1145/3178876.3186088 documented misconceptions [13, 20] about private mode. 1 INTRODUCTION We found that participants had many misconceptions, including beliefs that private mode would prevent geolocation, protect from Each of the five web browsers widely used today — Chrome, Edge, malware, eliminate advertisements, and prevent tracking by the Firefox, Safari, and Opera — offers a private mode that stops stor- websites visited and network providers. The particular disclosure ing browsing history and caching data across sessions. Unwanted participants saw impacted their misconceptions. Compared to a information from the user’s browsing history will not appear in meaninglessly vague control condition, the current and previous This paper is published under the Creative Commons Attribution 4.0 International Chrome desktop disclosure led participants to answer more sce- (CC BY 4.0) license. Authors reserve their rights to disseminate the work on their narios correctly; no other disclosure we tested had a significant personal and corporate Web sites with the appropriate attribution. effect. Participants who saw certain disclosures were more likely to WWW 2018, April 23–27, 2018, Lyon, France © 2018 IW3C2 (International World Wide Web Conference Committee), published have misconceptions about private mode’s impact on specific sce- under Creative Commons CC BY 4.0 License. narios: targeted advertising, the persistence of lists of downloaded ACM ISBN 978-1-4503-5639-8/18/04. files, and tracking by Internet service providers (ISPs), employers, https://doi.org/10.1145/3178876.3186088 WWW 2018, April 23–27, 2018, Lyon, France Yuxi Wu, Panya Gupta, Miranda Wei, Yasemin Acar†, Sascha Fahl†, Blase Ur and governments. Phrases used in some disclosures (e.g., “tracking save the list of files downloaded in private mode. All browsers can protection”) appear to contribute to misconceptions. access existing bookmarks in private mode, and new bookmarks While prior work has documented misconceptions about private saved in private mode are available in future sessions [22, 31, 34]. mode, our study is the first to focus on the role browsers’ disclosures Browsers differ somewhat in their private modes. Chrome and play. These disclosures, which users see each time they open a Edge do not save preferences (e.g., pop-up blocking, download loca- new private browsing window, are the main vector for disabusing tions) from private sessions, while Firefox and Opera do. Browsers users of misconceptions that might derive from loaded names like similarly differ in how they handle browser extensions. Edge com- “private browsing mode” and “incognito mode” [24, 39, 48]. While pletely disables extensions, while Firefox and Safari leave them we found that some browsers’ disclosures are better than others, all enabled by default. Chrome and Opera disable extensions by de- disclosures we tested failed to correct important misconceptions. fault, but let users manually enable them.

2 BACKGROUND AND RELATED WORK 2.2 Previously Documented Misconceptions We first summarize features of current browsers’ private modes. We then discuss prior work on misconceptions about private mode To create the browsing scenarios for our study, we gathered mis- and work on privacy communication more broadly. conceptions about private mode previously documented in the academic literature and news media. Two prior user studies have 2.1 Features of Private Browsing focused on misconceptions about private mode. Gao et al. surveyed 200 Mechanical Turk workers, examining qualitatively why partic- All major web browsers include a private mode designed to protect ipants used private browsing modes and what they believed the against local privacy threats with physical access to a machine [1]. modes do [20]. More recently, the search engine DuckDuckGo con- These modes aim to remove local copies of browsing history, tem- ducted a demographically representative sample of 5,710 Americans, porary files, and cookies from a machine. That said, the implemen- documenting seven misconceptions about private browsing [13]. tation of these features is imperfect [1]. Xu et al. demonstrated that Prior work has found that some users incorrectly believe private sophisticated attackers can still gain access to information from pri- mode shields them from tracking by visited websites, online adver- vate sessions [47]. Similarly, Satvat et al. found vulnerabilities that tising companies, the government, and ISPs [13, 20, 24, 27, 39], and enable sophisticated attackers and malware to steal information some are also under the misconception that websites cannot view from private mode [36]. While such attacks are important to overall IP addresses in private mode, thereby preventing geolocation [13]. browser security, they are not the focus of our work. Instead, we Furthermore, some users believe that private mode can control what study how users’ expectations compare to private mode’s intended third parties save [24]. For instance, users may not consider that protections. To determine these intended protections, we examined companies can record browsing histories when users are logged both browser-specific documentation and prior research on private into their accounts, regardless of browsing mode [13, 16]. Users modes. For potential behaviors that were not formally documented, were also not aware that ad networks can track browsing across we conducted informal experiments. the Internet [11, 17, 25]. Finally, some users believe that private No browser intentionally stores browsing history from private mode protects them from viruses/malware and blocks all ads [20]. mode after a session is closed [6, 21, 22, 28, 31, 34]. This guarantee While we aimed to test all misconceptions documented in prior does not at all apply to remote services; if a user is logged into an work, some were too vague to test in a controlled manner. For in- account, the provider of that account can certainly record the user’s stance, the misconception “websites that I visit do not know my browsing activity [1]. Browsers aim to delete temporary files (e.g., identity” [13, 20] could refer to websites associating pseudonymous cached files) when private sessions are closed. All browsers except activity from different browsing sessions, or it could refer to deter- Edge launch private browsing with fresh cookie storage; Edge uses mining a user’s real-world identity. Similarly, the misconception the store from standard mode. All browsers clear cookies after that “private browsing keeps my computer clean” [20], could refer the private session is closed. Across browsers, autofill information to any range of files being deleted upon exiting a private session. from standard mode is available when using private mode, but no browser offers to save autofill details automatically [6, 30,34]. Private mode does not, however, protect against remote threats. 2.3 Communicating Privacy and Security With one exception, browsers’ private modes do not alter the visi- While our study is the first to focus on privacy disclosures re- bility of a user’s IP address or otherwise change how traffic appears lated to browsers’ private mode, there is a large literature on user- on the network. Only Opera’s VPN feature [33] provides some centered communication about online privacy and security. Privacy limited protection. While it hides browsing activities from users’ disclosures can take many forms and be delivered through differ- ISPs, Opera’s VPN servers have the ability to snoop on their users’ ent modalities [37]. Within web browsers, numerous studies have browsing behavior. Between this limitation and the opt-in nature investigated SSL warnings [2, 18, 41, 42], as well as visual icons for of Opera’s VPN, this tool’s ability to prevent tracking is limited indicating connection security [19], ad personalization [26], site 1 for most users. Also, Firefox and Opera VPN provide protection trustworthiness [35], and privacy policies [43, 46]. against common tracking networks. Private mode also does not aim Despite the web’s privacy and security risks [25], users often click to stop all local activity. Files downloaded in private sessions remain through browsers’ warnings and disclosures [2]. Recent research afterwards, while all browsers we tested other than Brave do not has designed disclosures that are harder to ignore [10] by using 1https://support.mozilla.org/en-US/kb/tracking-protection polymorphic warnings [4, 9, 45] or strategic timing [15]. Recent Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions... WWW 2018, April 23–27, 2018, Lyon, France work has focused on presenting users with personal examples [23] addition, because Chrome’s disclosure had been redesigned shortly or automatically generated, data-rich disclosures [3, 7, 12]. before the beginning of the study, we included Chrome’s previous disclosure. We also tested the mobile versions of these browsers, 3 METHODOLOGY excluding Edge, whose mobile version was in beta at the time We conducted an online survey of participants recruited through of research. In cases where mobile disclosures differed between Amazon’s Mechanical Turk service for “a research study on web Android and iOS, we tested the one that differed most from the browsing behavior.” Participants were paid $5 to complete the half- desktop version. Finally, to understand what actionable information hour survey. We required that participants be 18 years or older, disclosures provided participants, we created a control condition live in the United States, and have completed 100+ HITs with a that made meaninglessly vague assertions about protecting privacy. 95%+ approval rating. Below, we present our overall survey structure followed by detailed descriptions of the thirteen conditions 3.3 Survey Scenarios and twenty browsing scenarios we tested. Finally, we describe our We created twenty browsing scenarios to capture actual benefits and quantitative and qualitative analysis methods. previously documented misconceptions of private mode [13, 20]. We tested nine where private browsing does not have any effect 3.1 Survey Structure and six where it does, even if the protection is imperfect. We also We began by asking participants to imagine a new web browser included five scenarios where the effect of private browsing is called Onyx. We chose to use a hypothetical browser to minimize browser- or context-dependent as perceptions of these scenarios biases associated with perceptions of current browser vendors or color overall expectations of private mode. Scenarios took two main prior experiences with a particular browser [44]. We explained, forms: distinguishing and comparative. “Onyx has a mode called private browsing mode. When you open a Sixteen distinguishing scenarios measured differences partici- new private browsing mode window in Onyx, you see the screen pants perceived between standard mode and private mode. To do below.” At this point, we presented a simulated browser window this, we constructed parallel yes-or-no questions about the browser containing the disclosure specified by the participant’s condition feature in each mode and asked participants to explain why they (detailed in Section 3.2) and specified, “Please make sure to read the chose the same or different responses across modes. We asked about information presented below before continuing on to the rest of the standard and private mode separately to disambiguate misconcep- survey.” We clarified that we used Standard Mode to mean “Onyx’s tions specific to private mode from broader misunderstandings. default mode, which is modeled after the default browsing mode The following is an example set of questions for one scenario: of current web browsers (e.g., Chrome, Edge, Firefox, Opera, and (1) Imagine that, using Onyx in standard mode, Brian goes Safari).” Similarly we defined Private Browsing Mode as the “mode online shopping on FunFashion.com and adds two items to described on the previous page” so that participants would base his shopping cart. Brian then closes Onyx. Brian then opens their answers on the disclosure text. Onyx in standard mode. Will his items still be available in The main survey section presented twenty browsing scenarios his shopping cart?(Yes, No) (Section 3.3) in randomized order. These represented a mixture of (2) How confident are you in your answer to the previous ques- situations in which private mode would have an effect and where tion? (Very confident, Confident, Somewhat confident, Notat it would not. We asked participants to assume an individual named all confident) Brian had opened a brand new computer and installed Onyx in (3) Imagine that, using Onyx in private mode, Brian goes online its default configuration immediately prior to each scenario. We shopping on FunFashion.com and adds two items to his collected responses to both multiple-choice questions and a free- shopping cart. Brian then closes Onyx. Brian then opens response question for each scenario. We provided a link to “reopen Onyx in standard mode. Will his items still be available in the disclosure in a new tab” on top of each page, instrumenting this his shopping cart?(Yes, No) link to record when participants reopened the disclosure. Finally, we (4) How confident are you in your answer to the previous ques- asked participants demographic questions about age, gender, and tion? (Very confident, Confident, Somewhat confident, Notat technical background. We also asked about their usage of current all confident) browsers and their private modes. (5) If you chose the same answers for standard and private mode, why? If you chose different answers, why? 3.2 Survey Conditions In addition to the persistence of shopping cart items, we pre- Each participant was randomly assigned to one of thirteen condi- sented distinguishing scenarios related to the persistence of brows- tions corresponding to a private browsing disclosure. All disclosures ing history, site pop-up preferences, bookmarks, the list of down- had a uniform design to prevent bias and differed only in the text loaded files, downloaded files themselves, and cached photos across displayed. Table 1 details the text of all thirteen disclosures. sessions. To gauge perceptions related to tracking by the web- We took twelve of these disclosures verbatim from the respective sites visited, we presented separate distinguishing scenarios about landing pages of six major browsers. We chose the five most popular whether Google searches across browsing sessions can be asso- browsers–Chrome, Edge, Firefox, Safari, and Opera– and Brave ciated with the same user if the user is, or is not, logged into a because it is a relatively new browser explicitly designed for privacy. Google account. We also presented separate distinguishing scenar- We tested the disclosure used on the Mac OS desktop version for ios regarding tracking from the following three entities: an Internet all browsers other than Edge (Windows 10) as of August 2017. In Service Provider (ISP), an employer (while using the employer’s WWW 2018, April 23–27, 2018, Lyon, France Yuxi Wu, Panya Gupta, Miranda Wei, Yasemin Acar†, Sascha Fahl†, Blase Ur

Table 1: The 13 conditions we tested, along with the browser network), and government agencies. Other scenarios gauged per- version from which we took that disclosure’s text. All dis- ceptions about whether browser extensions would be active, IP closures were rebranded for the hypothetical Onyx browser. addresses would be visible on the network, estimating geoloca- Text in italics was displayed in larger font as a header. tion was possible, and browsing history could be recovered by a forensics expert with physical access to the machine. Name Disclosure The remaining four scenarios were more naturally expressed as Control private browsing mode. Onyx’s private browsing mode a comparison between standard and private modes. In these four protects your privacy and keeps you safe as you browse the comparative scenarios, we asked participants to compare the relative Internet. It was carefully designed by Onyx’s engineers to let you stay incognito as you browse. incidence of some property between modes. For example, participants saw the following questions and multiple-choice responses Brave (v0.19) this is a private tab. Private tabs are not logged in page history. Private tabs and their cookies vanish when the for a scenario about the loading speed of webpages: browser is closed. File downloads, new bookmarks, and passwords can still be saved while using a private tab and (1) How will the speed at which webpages load compare in will not be removed when the private tab is closed. Please standard versus private browsing mode? note: Even though sites you visit in private tabs are not • They will load much more quickly in private mode than in saved locally, they do not make you anonymous or invisible to your ISP, your employer, or to the sites you are visiting. standard mode. • They will load more quickly in private... Brave-Mobile private browsing. Onyx won’t remember any of your history (Android, v1.0) or cookies, but new bookmarks will be saved. • They will load a little more quickly in private... Chrome (v60) you’ve gone incognito. Now you can browse privately, and • They will load at the same speed. other people who use this device won’t see your activity. • They will load a little more quickly in standard... However, downloads and bookmarks will be saved. Onyx • They will load more quickly in standard... won’t save the following information: Your browsing history, Cookies and site data, Information entered in forms. • They will load much more quickly in standard... Your activity might still be visible to: Websites that you visit, (2) How confident are you in your answer... Your employer or school, Your Internet service provider. (3) Why? Chrome-Mobile incognito mode. Your recent activity and search history (iOS, v60) aren’t available when you’re in incognito. In addition to loading speed, we also presented comparative sce- Chrome-Old you’ve gone incognito. Pages that you view in incognito tabs narios about the total number of ads, the proportion of targeted ads (v59) won’t stick around in your browser’s history, cookie store based on prior browsing, and virus and other malware protection. or search history after you’ve closed all of your incognito tabs. Any files you download or bookmarks you create will be kept. However, you aren’t invisible. Going incognito 3.4 Quantitative Analysis doesn’t hide your browsing from your employer, your Internet service provider or the websites that you visit. We aimed to understand to what degree the disclosures or demographic factors influenced the correctness of participants’ responses Edge (v40) browsing inprivate. When you use InPrivate tabs, your browsing data (like cookies, history, or temporary files) for all twenty scenarios in aggregate, as well as participants’ overall isn’t saved on your device after you’re done. Onyx deletes confidence in these responses. For each participant, we computed temporary data from your device after all of your InPrivate a correctness score by summing the number of scenarios for which tabs are closed. their response was correct (comparative scenarios) or response Firefox (v54) private browsing with tracking protection. When you browse in a Private Window, Onyx does not save: visited pages, about private mode was correct (distinguishing scenarios). The cor- cookies, searches, temporary files. Onyx will save your: rect response differed based on the browser or the browsing context bookmarks, downloads. Private Browsing doesn’t make you for six scenarios relating to the page loading speed, the persistence anonymous on the Internet. Your employer or Internet service provider can still know what page you visit. of domain-specific preferences, whether browser extensions are tracking. Some web sites use trackers that can monitor your active, information extractable by a forensics expert, shopping carts, activity across the Internet. With Tracking Protection Onyx will block many trackers that can collect information about and the total number of ads. We excluded these six and calculated your browsing behaviour. the correctness score from the fourteen remaining scenarios. Firefox-Mobile private browsing + tracking protection. Onyx blocks parts of We similarly created an overall confidence score per participant (Android, v54) the pages that may track your browsing activity. We won’t by summing that participants’ self-reported confidence (0 through remember any history, but downloaded files and new 3, from “not at all confident” through “very confident”) across the bookmarks will still be saved to your device. same scenarios as the correctness score. We created linear regres- Opera (v45) private browsing. As soon as you close all private windows, all the information connected with them will be erased. If sion models with each of the scores as the dependent variable and you want even more privacy - turn on the VPN. the following independent variables: the condition; the number of Opera-Mobile your secrets are safe. Onyx won’t save the browsing history times the participant reopened the disclosure; and the participants’ (Android, v42) of your private tabs. gender, age, technical expertise, and use of private mode. Safari (v10) private browsing enabled. Onyx will keep your browsing Our second type of quantitative analysis aimed to understand, history private for all tabs in this window. After you close at a finer-grained level, how the disclosure participants saw im- this window, Onyx won’t remember the pages you visited, your search history, or your AutoFill information. pacted each individual scenario (and thus the particular private Safari-Mobile private browsing mode. Onyx won’t remember the pages browsing feature or misconception that scenario encodes). For each (iOS, v10) you visited, your search history, or your AutoFill of the twenty scenarios, we conducted an omnibus test to deter- information after you close a tab in Private Browsing Mode. mine whether responses about what would happen in private mode differed significantly depending on which of the 13 disclosures Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions... WWW 2018, April 23–27, 2018, Lyon, France the participant saw. We analyzed the yes/no responses for private recurring themes, we only report codes that occurred for at least mode in the sixteen distinguishing scenarios using Pearson’s χ2 10% of incorrect responses. test. When a contingency table contained an expected value below 5, we used Fisher’s exact test (FET). In the four comparative sce- 4 RESULTS narios, our response variable was ordinal. Therefore, we used the We first provide an overview of our 460 participants (Section 4.1) nonparametric Kruskall-Wallis (KW) H test for our omnibus test and, for those who used private mode in their daily browsing, across groups. We use α = .05 throughout. We correct for multiple the reasons they gave for doing so (Section 4.2). Echoing prior testing using the Benjamini-Hochberg method. work, we found that participants use private mode to hide browsing We chose 12 planned comparisons (equal to the degrees of free- activity, prevent the saving of log-in information, and avoid cookies. dom in our experiment) to investigate three targeted research ques- Augmenting this prior work, we found that some participants used tions. First, we investigated whether the widely-deployed disclo- private mode to avoid ad targeting and web personalization. sures led to significantly more correct conceptions of private mode The remaining sections focus on how the disclosures impacted compared to the control disclosure we created. participants’ expectations for the twenty browsing scenarios. In RQ 1: How do each of the six desktop disclosures compare to the Section 4.3, we present the results of participants’ correctness scores control? To answer this, we performed the following six planned and confidence scores aggregated across scenarios. Compared toour comparisons: Brave vs. Control; Chrome vs. Control; Edge vs. Con- control disclosure, participants who saw the Chrome or Chrome- trol; Firefox vs. Control; Opera vs. Control; Safari vs. Control. Old disclosure gave significantly more correct responses. Surpris- We further noticed that the disclosure in the desktop versions ingly, no other disclosures we tested differed significantly from the tended to differ substantially from the same browsers’ mobile dis- meaningless control disclosure, and our results suggest that some closures, which were often much shorter. disclosures may have led to additional misconceptions. RQ 2: How do mobile versions of disclosures compare to the dis- To more fully understand the reason for incorrect responses, closure in the same browser’s desktop version? To answer this, we we delved into the twenty scenarios one at a time. For three sce- performed the following five planned comparisons: Brave vs. Brave- narios, participants’ responses were overwhelmingly correct (Sec- Mobile; Chrome vs. Chrome-Mobile; Firefox vs. Firefox-Mobile; tion 4.4). Unfortunately, participants overestimated the protections Opera vs. Opera-Mobile; Safari vs. Safari-Mobile. afforded by private mode for eight scenarios, whereas they under- Finally, because Chrome’s disclosure changed noticeably shortly estimated the protections in three scenarios (Section 4.5). These before we performed our study, we investigated how this redesign misconceptions included expectations that private browsing mode impacted misconceptions. would prevent geolocation, ads, and viruses, as well as tracking RQ 3: How does the recently redesigned Chrome disclosure com- by both the websites visited and the network provider. For five of pare to the previous Chrome disclosure? To answer this, we per- the twenty scenarios, participants’ responses differed significantly formed the following planned comparison: Chrome vs. Chrome-Old. across disclosures. These five scenarios related to targeted adver- We only performed planned comparisons when the omnibus test tising, the persistence of lists of downloaded files, and tracking by was significant. For planned comparisons in the sixteen distinguish- ISPs, employers, and governments. ing scenarios, we again used Pearson’s χ2 test (or Fisher’s exact test, as appropriate). For planned comparisons of ordinal responses 4.1 Participants and Their Browser Usage in comparative scenarios, we used the Mann-Whitney U (MW) test. Of our 460 participants, 54.6% identified as male, 44.6% as female, and 0.8% as another gender. 70.9% of participants were between 3.5 Qualitative Analysis 25 and 54 years old, 8.0% were younger, and 20.8% older. Only 11.5% of participants were classified as having technical expertise To gauge why participants had particular misconceptions, we per- due to having held degrees or jobs in computer science, IT, or formed qualitative coding for the seventeen scenarios at least 20% related fields. Nearly all participants used Chrome for desktop of participants answered incorrectly. For each of these scenarios, browsing on a regular basis (96.7%), although many additionally a member of the research team read the free-text explanations used Firefox (64.3%), and to a lesser extent, Edge/Internet Explorer accompanying incorrect responses. The researcher subsequently (21.1%) and Safari (14.3%). Chrome was also the most popular for performed open coding, iteratively updating the codebook as nec- mobile browsing (73.7%), although Safari (33.0%) was used more essary. We followed a similar process to identify themes in the than Firefox (12.6%) and Edge/Internet Explorer (2.2%). reasons participants gave for using private browsing. First, a mem- While 80.4% of participants use private browsing at least some ber of the research team performed open coding to identify eighteen of the time for desktop browsers, participants more frequently and twenty codes for desktop and mobile users, respectively. The used standard mode. Across browsers, participants reported using researcher then used axial coding for consolidation and clarifica- private mode for 15% (median) of their browsing. In stark contrast, tion, resulting in fifteen themes for desktop and sixteen for mobile. only 19.1% of participants used private browsing on mobile devices. Codes were not mutually exclusive; participants could give multiple reasons for using private browsing. 4.2 Reasons for Using Private Browsing Mode A second member of the research term independently coded For participants who reported using private mode in their own all of the data following the codebook. Cohen’s κ ranged from browsing, we asked a free-response question about why they did so. 0.651 to 0.860 per scenario, with a median of 0.716. To focus on The top six reasons participants provided were to: 1) hide browsing WWW 2018, April 23–27, 2018, Lyon, France Yuxi Wu, Panya Gupta, Miranda Wei, Yasemin Acar†, Sascha Fahl†, Blase Ur

Table 2: Linear regression with the number of scenarios the as detailed in Section 3.4. Higher values indicate a larger participant answered correctly as the dependent variable. number of scenarios answered correctly. Higher numbers correspond to more correct answers. • Confidence score: The sum of the confidence ratings (each Factor β SE t p on a 0–3 scale) participants gave for the same fourteen scenarios. Higher values indicate greater confidence. (Intercept) 9.53 0.40 24.0 <.001 We then created linear regression models for these two indices. Condition: Brave 0.79 0.49 1.61 .108 For both models, we used the following six independent variables, Condition: Brave-Mobile 0.49 0.49 0.08 .940 representing the study-specific and demographic features we hy- Condition: Chrome 1.07 0.49 2.16 .032 Condition: Chrome-Mobile 0.62 0.50 1.25 .211 pothesized might be correlated with correctness or confidence: Condition: Chrome-Old 1.09 0.50 2.20 .028 • Condition; categorical with control as the baseline Condition: Edge 0.05 0.50 0.10 .923 • Participant’s gender; categorical with female as the baseline Condition: Firefox 0.88 0.59 1.80 .073 • Whether the participant had technical expertise; categorical Condition: Firefox-Mobile -0.30 0.50 -0.60 .550 with “no” as the baseline Condition: Opera 0.57 0.50 1.15 .252 • Participant’s age range; ordinal with 18-24 as the baseline Condition: Opera-Mobile -0.34 0.49 -0.70 .484 • The percent of time, averaged across browsers used, that the Condition: Safari 0.78 0.51 1.53 .127 participant browses in private mode; continuous (0–100) Condition: Safari-Mobile 0.95 0.49 1.93 .055 • The number of times the participant reopened the disclosure Gender: Male 0.50 0.20 2.51 .013 during the study; continuous (0–5, the maximum observed) Technical: Yes 0.49 0.31 1.60 .111 Age Range -0.65 0.52 -1.24 .216 Of the thirteen disclosures we tested, twelve are currently de- Browsing in Private Mode (%) -0.00 0.01 -0.26 .792 ployed in browsers. Surprisingly, we found that most of these dis- Reopened Disclosure (#) 0.19 0.16 1.19 .236 closures did not significantly impact participants’ understanding of what private browsing mode does and does not do compared to our meaninglessly vague control condition. For only two of these twelve disclosures did the correctness score differ significantly history, especially visits to adult websites; 2) prevent targeted ads from the control disclosure. As shown in Table 2, participants who and search suggestions; 3) achieve "safer" browsing; 4) prevent saw the disclosures Google Chrome either currently uses (Chrome browsers from saving login-related information; 5) avoid cookies; condition) or previously used (Chrome-Old) for desktop browsers 6) accommodate intentional or unintentional use by others. Our answered more scenarios correctly than those who saw the control observations of points 1, 4, and 5 echo Gao et al.’s findings20 [ ]. disclosure (p = .032 and p = .028, respectively). Of the fourteen sce- In addition, we observed a new reason — preventing targeted ads narios whose answers did not depend on the browser or browsing and search suggestions — that 15.1% of participants mentioned for context, participants in the control condition answered 10.4 cor- desktop browsing and 15.6% mentioned for mobile. rectly on average. In contrast, Chrome and Chrome-Old participants Although all six reasons largely align with the intended char- answered 11.1 and 11.3 questions correctly on average. Two other acteristics of private browsing, careful reading of the third point’s disclosures (Firefox and Safari-Mobile) seemed to positively impact free text revealed a plurality of definitions of "safer" browsing. For correctness scores. While the p-values for these disclosures were example, P221 simply noted, “I feel a little more secure,” while P448 not below our α = .05 threshold, they were marginally significant reported, “If I’m going to a site that may be questionable, I use the (.05 < p < .10) and therefore may warrant further investigation. private mode to protect myself.” Some participants also mentioned Unfortunately, three of the mobile disclosures appeared less tracking in a vague sense, such as “I like not being able to be stalked successful than the meaninglessly vague control at informing par- by people when I’m surfing the web” (P148). We further analyze ticipants about private browsing mode. Compared to the control related misconceptions in Section 4.5. (10.4 average correctness score), participants who saw Brave-Mobile Finally, some reasons for using private mode were tied to a sense (10.1), Firefox-Mobile (9.9), or Opera-Mobile (9.9) had lower cor- of device ownership. Overall, 27 participants considered a desktop rectness scores on average, though these differences were not sta- or mobile device’s shared nature in choosing to use private mode, tistically significant. Among demographic factors, we found that for example, “[s]o that others in my household don’t see the sites only gender was significantly correlated with the correctness score. I visited” (P123). Another 47 participants considered unforeseen Participants who identified as male had higher correctness scores “users,”such as thieves, finders of a lost device, or friendly borrowers. than those who identified as femalep ( = .013) As shown in Table 3, which disclosure participants saw mostly 4.3 Aggregate Correctness and Confidence did not impact their self-reported confidence in their answers. Our first goal was to understand at a high level the degree towhich Relative to the control, only participants who saw Chrome-Old the disclosure the participant saw impacted their understanding of (p = .031) or Firefox (p = .043) were significantly more confident private mode. To do so, we created two indices per participant: in their answers. Participants who identified as male were more • Correctness score: How many scenarios the participants confident in their answers than those who identified asfemale answered correctly, summed across the comparative scenar- (p < .001), while younger participants were more confident in their ios and responses for private mode in distinguishing sce- answers than older participants (p < .001). Surprisingly, partici- narios. This index excludes six scenarios where the correct pants who spent more time browsing in private mode in their own answer depended on the browser used or browsing context, browsing were less confident in their answersp ( = .010), which Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions... WWW 2018, April 23–27, 2018, Lyon, France

Table 3: Linear regression with the sum of participants’ con- 4.5 Misconceptions fidence ratings across scenarios as the dependent variable. Despite focusing participants’ attention on the disclosure, we still Higher numbers correspond to high confidence. observed misconceptions that cover the spectrum of previously doc- Factor β SE t p umented misconceptions. We differentiate between misconceptions that overestimate and underestimate private mode. (Intercept) 26.0 1.19 22.0 <.001 Condition: Brave 1.72 1.46 1.18 .240 4.5.1 Overestimating Private Mode’s Protections. Participants Condition: Brave-Mobile -0.22 1.47 -0.15 .881 had a number of misconceptions in which they overestimated what Condition: Chrome 1.24 1.47 0.84 .402 private mode would do. A surprising 56.3% of participants believed Condition: Chrome-Mobile -0.31 1.48 -0.21 .836 that even while a user was logged into a Google account, their Condition: Chrome-Old 3.20 1.48 1.2.16 .031 search queries would not be saved while in private mode. The Condition: Edge -0.48 1.50 0.32 .748 Condition: Firefox 2.98 1.47 2.03 .043 large majority of participants (144) believed this to be the case Condition: Firefox-Mobile 1.47 1.48 0.99 .322 because private mode does not save search histories, conflating Condition: Opera 0.08 1.48 0.05 .958 the browser’s local history with Google’s. Further, 77 participants Condition: Opera-Mobile 0.50 1.46 0.34 .734 believed that private mode would remove logged info retroactively. Condition: Safari 0.12 1.52 0.08 .938 Some participants (46.5%) believed bookmarks saved in private Condition: Safari-Mobile 0.91 1.48 0.62 .538 mode would not persist in later sessions. For example, 54 partici- Gender: Male 3.46 0.60 5.76 <.001 pants believed that private mode deletes all local, temporary data, Technical: Yes 1.65 0.92 1.79 .074 including bookmarks. Due to a typo in the survey instrument, the <.001 Age Range -5.41 1.56 -3.46 data we report includes a fraction of users (around 25%, according Browsing in Private Mode (%) -0.05 0.02 -2.60 .010 to qualitative responses) who understood the typo to imply that we Reopened Disclosure (#) -0.16 0.48 -0.32 .746 were asking about a different browser. Nonetheless, we still label this scenario a misconception because of participants who believed bookmarks would be deleted when closing private mode. Table 4: Non-misconceptions. The first pair of Std. and Priv. In addition, 40.2% of participants thought websites would not be columns give the correct answer in standard and private able to estimate a user’s location. Most (125 participants) believed mode, respectively. The second pair of columns give the per- location hiding was simply a feature of private browsing, but a centage of participants who answered incorrectly. further 21 participants specified geolocation would be impossible because IP addresses, used to estimate location, were hidden. Answer % Incorrect Nearly all participants correctly realized that their browsing Scenario Std. Priv. Std. Priv. could be tracked by their ISP, employer (when on their network), Downloaded files remain Yes Yes 0.7 2.8 and the government while in standard mode. However, 22.0%, 37.0%, History saved in history menu Yes No 1.1 5.2 and 22.6% of participants mistakenly believed that ISPs, employers, Photos cached across sessions Yes No 9.6 10.2 and the government would be unable to track them when they used private mode.2 Most commonly, participants simply believed that no search history would be saved, so outside entities would not have access. Other reasons included mistaken expectations that we hypothesize may have been due to experiencing events that IP addresses would be hidden, cookies could no longer be used, challenged their assumptions about private mode. and that private mode automatically included VPN functionality. Interestingly, 10 participants believed the government would need a warrant to access browsing activity from private mode. 4.4 Non-Misconceptions Overall, 27.1% of participants believed private mode offered more We denoted scenarios where over 20% of participants (across dis- protection against viruses and malware than standard, primarily closures) answered the question about private mode incorrectly as attributing this to private mode not saving information, especially exhibiting misconceptions. As shown in Table 4, there were three cookies and ads, locally. For example, P146 declared, “it’s a no scenarios for which nearly all participants answered the question brainer really, when in private mode nothing is stored,” while P67 about private mode correctly (i.e., were non-misconceptions). Over- added, “when you are in private mode there is no direct link to you all, 97.2% of participants correctly realized that files downloaded or your computer so malware etc. would not be saved onto your in private mode would remain on the computer after the browsing computer or in your cookies.” Other participants simply believed, session was closed, as in standard mode. All but 1.1% of partici- without specifics, that private mode was more secure, “because pants knew that their browsing history was saved in the history private mode sounds and usually is more secure when visiting menu in standard mode, and all but 5.2% of participants knew that websites, I would think it would be much safer to visit them” (P197). it would not be saved in the history menu in private mode. Simi- Additionally, 25.2% believed sites a user visited in private mode larly, whereas all but 9.6% of participants knew that, in standard would be unable to see the user’s IP address. Most (89 participants) mode, photos viewed in the browser would be cached for quicker believed that this was a feature of private browsing, similar to loading in future sessions, all but 10.2% of participants knew that 2Although Opera’s VPN functionality would provide some of these protections, it is this caching would not occur in private mode. not enabled by default. WWW 2018, April 23–27, 2018, Lyon, France Yuxi Wu, Panya Gupta, Miranda Wei, Yasemin Acar†, Sascha Fahl†, Blase Ur

Table 5: Scenarios where participants held misconceptions, Table 6: Distinguishing scenarios where private mode’s im- shown with the correct answers and percentage of partici- pact depends on the browser or context. pants who gave incorrect answers. For comparative scenarios, (in)equality symbols denote the correct answer, and we % Yes Scenario Std. Priv. give the sum of all participants answering otherwise. Items in shopping cart saved across sessions 97.8 78.8 Answer % Incorrect Browser extensions active across sessions 98.3 69.1 Scenario Std. Priv. Std. Priv. Forensic expert can reconstruct browsing history 98.7 52.8 Overestimating private mode’s privacy protections Site-specific preferences (e.g., for pop-ups) saved 98.3 31.3 Search queries associated (logged in) Yes Yes 1.5 56.3 Bookmarks saved across sessions Yes Yes 25.4 46.5 Table 7: Distribution of responses for comparative scenarios Geolocation can be estimated Yes Yes 5.2 40.2 where the impact depends on the browser or context. Employer can track browsing Yes Yes 1.1 37.0 Better protected from viruses/malware Std. = Priv. 27.1 % Responses IP address can be collected Yes Yes 0.7 25.2 Scenario Std. > Priv. Std. = Priv. Std. < Priv. Government can track browsing Yes Yes 4.1 22.6 Amount of ads 32.2 64.9 2.9 ISP can track browsing Yes Yes 3.0 22.0 Page loading speed 24.8 53.6 21.6 Underestimating private mode’s privacy protections Downloaded file in browser’s list Yes No* 1.3 51.7 Proportion of targeted ads Std. > Priv. 30.9 2 Search queries associated (not logged in) Yes No 20.2 30.0 (χ (12) = 38.1, p = .001). In the control condition, 32.4% of participants mistakenly believed downloaded files would still be listed in *Except in Brave’s private mode, which does retain download history the browser. A higher proportion of participants in Brave (62.2%, χ2(1) = 5.4, p = .020) and Firefox (77.1%, χ2(1) = 12.7, p < .001) geolocation. P151 noted, “private modes...block your outgoing in- held this misconception. Notably, both the Brave and Firefox dis- formation, this includes IP." Some (10) confused private mode with closures mention downloaded files. VPNs: “Private mode usually masks the IP address by using the As private mode deletes cookies across sessions, one should ex- VPN style option that would generate a different IP address” (P72). pect to see fewer targeted ads in private mode over time. Many The disclosure significantly impacted participants’ responses participants were unaware of this; 27.2% believed the proportion about tracking by ISPs, employers, and governments (FET, p = .005, of advertisements that would be targeted based on a user’s search p < .001, p = .014, respectively). Supporting RQ2, Firefox-Mobile history would be the same across modes. Most of these partici- performed significantly worse than Firefox in all three scenarios pants (70) believed ads were irrelevant to the “privacy” of private (FET, p = .009, p = .020, p = .002). Whereas only 5.9%, 17.1%, mode. They did not fully appreciate that private mode would make and 8.6% of Firefox participants mistakenly believed private mode third-party tracking more difficult, though not impossible [17]. Ex- would prevent tracking, 37.1%, 45.7%, and 42.9% of Firefox-Mobile pectations about targeting differed by condition (KW, H (12) = 30.5, participants had such a misconception, respectively. In Section 5, p = .009). Surprisingly, a higher proportion of participants who saw we discuss how Firefox’s “tracking protection” feature appears to the control disclosure were correct than those who saw Brave (MW, impact this misconception. For tracking by ISPs and employers, we U = 399, p < .001), Chrome (MW, U = 465, p = 0.010), or Edge similarly found a higher rate of misconceptions for Brave-Mobile (MW, U = 387, p = 0.004). Furthermore, Brave-Mobile participants compared to Brave (FET, p = .003 and p = .002). While 5.4% and were more correct than Brave (MW, U = 464.5, p = 0.019). 13.5% of Brave participants expected private mode to prevent track- In contrast to search queries when logged into a Google account, ing by ISPs and employers, 33.3% and 50.0% of Brave-Mobile par- which can be associated trivially, associating search queries across ticipants expected the same. We also observed some evidence in sessions in private mode is more difficult, albeit not impossible [17]. support for RQ1. Whereas 13.5% of Brave participants expected pri- While debatable, we chose to consider the 30.0% of participants vate mode would prevent tracking by employers, 40.5% of control who believed such searches could be associated across sessions participants had this misconception (FET, p = .018). as misconceptions based on their explanations. For example, 47 participants simply expected that Google would be able to know. 4.5.2 Underestimating Private Mode’s Protections. In three sce- In P326’s words, “Google.com is the worst for snooping on every narios, however, participants underestimated private mode’s pro- single individual in the world. Nothing escapes their tentacles." tections. As shown in Table 5, over half of participants believed the names of files downloaded in private mode would appear inthe list of downloaded files, when the opposite is true for all browsers 4.6 Scenarios With Context-Dependent Impact except for Brave. The common explanations for this misconception In the final six scenarios, the impact of private mode depends on were closely related: private mode does not prevent files from be- either the browser used or the browsing context. While there is not ing downloaded (93 participants) and, if downloads are saved, the a clear correct answer for these scenarios, we nonetheless included file would still be available (86 participants). This misconception them to document participants’ perceptions and expectations. seems to stem in part from participants’ confusion between the Browsers differ in handling browser extensions, which can asso- browser’s list of recently downloaded files and the files themselves. ciate sessions and respawn identifiers if active in private mode[48]. Nonetheless, we observed significant differences across conditions Extensions are always enabled in Firefox [29] and Safari [5], yet Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions... WWW 2018, April 23–27, 2018, Lyon, France always disabled in Edge. In both Chrome and Opera, extensions Of the thirteen disclosures we tested, only the current and old are disabled by default in private mode, but can be enabled. Simi- versions of Chrome’s desktop disclosure led to significantly more larly, site-specific preferences (e.g., pop-ups) are not saved in private correct answers than our meaninglessly vague control condition. mode by Chrome [21] and Edge, but are saved by Firefox and Opera. Chrome’s successes may be a result of their disclosures anticipating While nearly all participants expected browser extensions to misconceptions that users have about private mode, and explaining be active and site-specific preferences to persist in standard mode, the contrary: for example, that it does not hide browsing activity expectations for private mode were mixed (Table 6). This is notable from employers or ISPs. Other disclosures only explain capabil- because these behaviors are subtle vectors for privacy leaks. In ities, such as deleting local history, or are longer in text length. private mode, 69.1% of participants expected extensions to be active The difference between Chrome’s disclosures and others, however, and 31.3% expected preferences to be saved. Although browsers amounted to only one additional scenario answered correctly. For differ in actual behavior, responses did not differ by disclosure. all disclosures, participants maintained important misconceptions While private modes eliminate much of the browsing data stored about private mode even though the study focused on disclosures, locally, implementations are imperfect [1, 36, 47]. Nevertheless, in many ways a best-case scenario for privacy notices. 47.2% of participants thought a forensics expert could not determine The term “private” is heavily overloaded [40], and our results a user’s private browsing history even with physical access. suggest the name “private mode” implies unintended meanings. Whereas Brave blocks ads [8] and Firefox blocks only ads that When disclosures claim users can “browse privately” (Chrome), track users [32], other browsers do not block ads in private mode. users may refer back to their broader conceptualization of privacy. Despite these actual differences, responses did not differ by con- For example, P300 explained that “[collecting] the IP address would dition; 32.2% of participants expected more ads in standard mode defeat the purpose of privacy if the site could see that even in than private mode, while 64.9% expected no difference (Table 7). private mode,” while P133 deduced that because tracking “would be 100 participants believed standard mode’s saving of history would a violation of privacy,” ISPs could not collect browsing history. As facilitate ads, while 37 participants believed standard mode would such, browsers’ disclosures have the herculean task of correcting have more ads because they thought there would be no cookies misconceptions users derive from the name “private mode.” in private mode: “This has to do with cookies...If they can’t see The disclosures fail at this task. We found that participants anything then they have no idea what ads to show” (P82). were able to answer correctly about three scenarios where pri- Because private browsing mode creates a new cookie store per vate mode’s effects would be apparent locally. In contrast, many session, less data may be transferred in web requests, leading to over-estimations of private mode’s protections reflect more opaque faster page loads. In contrast, because less information is cached, behaviors. In a scenario about searches being tracking while logged page loads might also be slower. Among participants, 21.6% ex- into a Google account, P232 wrote about how she actually tried pected pages to load faster in private mode as fewer “extras” would to test her theory that private mode would prevent Google from weigh browsing down. In contrast, 24.8% expected pages to load remembering her search. When the search did not subsequently faster in standard mode, mostly explaining that private mode would autocomplete, she validated her theory. However, Google likely lack cached data. However, a few expected private mode to be still recorded that search. Even without being logged into an ac- slowed by security features. P241 wrote, “Private mode has to put count, users can be tracked through browser fingerprinting, and the websites into more secure protocols prior to displaying them.” experiments to detect how private mode actually impacts tracking Finally, for cookie-based shopping carts, items placed in the cart over the network and by websites visited [17] are far beyond the in private mode would not remain in the cart if the site were opened capabilities of end-users. This is the role of privacy disclosures. in standard mode. For carts associated with a user who has logged Some elements of disclosures were especially problematic. Firefox- into an account, private mode likely would not make a difference. Mobile highlights its “tracking protection” feature (for third-party Based on our qualitative analysis, this difference in mental models tracking) without further explaining what “tracking protection” appears to explain why 21.2% of participants expected items added is. Notably, many Firefox-Mobile participants mistakenly believed to a cart in private mode would persist in standard mode. they would be protected from network-level tracking. More gen- erally, disclosures should avoid vague phrases implying private 5 DISCUSSION mode will have a broad or comprehensive impact. Poor-performing disclosures all contained such phrases. Opera-Mobile states “your While prior work has documented general misconceptions about secrets are safe,” while Brave notes “tabs and their cookies vanish.” browsers’ private modes [13, 20], our study is the first to exam- Disclosures appear to be getting worse, not better. Firefox Fo- ine how browsers’ disclosures impact these misconceptions. We cus recently introduced a disclosure that ill-advisedly encourages conducted a user study in which 460 participants saw one of 13 users to “browse like no one’s watching.” The poor performance of disclosures and answered questions about 20 different browsing disclosures in our study suggests future work consider personal ex- scenarios. In most browsers, disclosures about private mode take amples [23], different modalities37 [ ], and even changing the name up the full page each time a new private window is opened. The of the mode to better convey private mode’s actual protections. content of these disclosures differs substantially across browsers, as well as between a given browser’s desktop and mobile versions. Users might also learn about private browsing from friends, on- 6 ACKNOWLEDGMENTS line articles, or blog posts. Nonetheless, effective disclosures are We gratefully acknowledge support from a Mozilla Research Award vendors’ most reliable channel to communicate features to users. and assistance from Adam Freymiller. WWW 2018, April 23–27, 2018, Lyon, France Yuxi Wu, Panya Gupta, Miranda Wei, Yasemin Acar†, Sascha Fahl†, Blase Ur

REFERENCES [24] Chris Hoffman. 2012. How private browsing works, and whyit [1] Gaurav Aggarwal, Elie Bursztein, Collin Jackson, and Dan Boneh. 2010. An doesn’t offer complete privacy. How-To Geek. (June 30, 2012). analysis of private browsing modes in modern browsers. In Proc. USENIX Security https://www.howtogeek.com/117776/htg-explains-how-private-browsing- Symposium. works-and-why-it-doesnt-offer-complete-privacy/. [2] Devdatta Akhawe and Adrienne Porter Felt. 2013. Alice in Warningland: A [25] Balachander Krishnamurthy and Craig Wills. 2009. Privacy diffusion on the web: large-scale field study of browser security warning effectiveness. In Proc. USENIX A longitudinal perspective. In Proc. WWW. Security Symposium. [26] Pedro Giovanni Leon, Justin Cranshaw, Lorrie Faith Cranor, Jim Graves, Manoj [3] Hazim Almuhimedi, Florian Schaub, Norman Sadeh, Idris Adjerid, Alessandro Hastak, Blase Ur, and Guzi Xu. 2012. What do online behavioral advertising Acquisti, Joshua Gluck, Lorrie Faith Cranor, and Yuvraj Agarwal. 2015. Your privacy disclosures communicate to users?. In Proc. WPES. location has been shared 5,398 times!: A field study on mobile app privacy [27] David Meyer. 2016. Opera makes major push for private browsing. Fortune. nudging. In Proc. CHI. (September 19, 2016). http://fortune.com/2016/09/20/opera-vpn-privacy/. [28] Microsoft Edge. Accessed July 7, 2017. In-Private mode (landing page). (Accessed [4] Bonnie Brinton Anderson, Anthony Vance, Jeffrey L Jenkins, C Brock Kirwan, July 7, 2017). and Daniel Bjornn. 2017. It all blurs together: How the effects of habituation [29] Mozilla. Accessed 2017. Add-ons: private-browsing. MDN web docs. (Accessed generalize across system notifications and security warnings. In Information 2017). https://developer.mozilla.org/en-US/Add-ons/SDK/High-Level_APIs/ Systems and Neuroscience. private-browsing. [5] Apple. 2017. Safari for Mac: Use private browsing windows in Safari. (March 30, [30] Mozilla. Accessed July 7, 2017. Private browsing - Use Firefox without sav- 2017). https://support.apple.com/kb/ph21413?locale=en_GB. ing history. (Accessed July 7, 2017). https://support.mozilla.org/en-US/kb/ [6] Apple Safari. Accessed July 7, 2017. Private browsing mode (landing page). private-browsing-use-firefox-without-history. (Accessed July 7, 2017). [31] Mozilla Firefox. Accessed July 7, 2017. Private browsing mode (landing page). [7] Rebecca Balebako, Jaeyeon Jung, Wei Lu, Lorrie Faith Cranor, and Carolyn (Accessed July 7, 2017). Nguyen. 2013. “Little brothers watching you”: Raising awareness of data leaks [32] Nick Nguyen. 2015. The Mozilla Blog: Firefox now offers a more private brows- on smartphones. In Proc. SOUPS. ing experience. (November 3, 2015). https://blog.mozilla.org/blog/2015/11/03/ [8] Brave. Accessed 2017. Brave browser. (Accessed 2017). https://brave.com/. firefox-now-offers-a-more-private-browsing-experience/. [9] Cristian Bravo-Lillo, Lorrie Cranor, Saranga Komanduri, Stuart Schechter, and [33] Opera. [n. d.]. Free VPN in Opera browser. Surf the web with enhanced privacy. Manya Sleeper. 2014. Harder to ignore: Revisiting pop-up fatigue and approaches ([n. d.]). http://www.opera.com/computer/features/free-vpn. to prevent it. In Proc. SOUPS. [34] Opera. Accessed July 7, 2017. Private browsing. (Accessed July 7, 2017). http: [10] Cristian Bravo-Lillo, Saranga Komanduri, Lorrie Faith Cranor, Robert W. Reeder, //help.opera.com/Mac/12.10/en/private.html. Manya Sleeper, Julie Downs, and Stuart Schechter. 2013. Your attention please: [35] Yohko Orito, Kiyoshi Murata, and Yasunori Fukuta. 2013. Do online privacy Designing security-decision UIs to make genuine risks harder to ignore. In Proc. policies and seals affect corporate trustworthiness and reputation. International SOUPS. Review of Information Ethics 19, 7 (2013), 52–65. [11] Aaron Brown. 2015. Watching porn on your internet browser’s [36] Kiavash Satvat, Matthew Forshaw, Feng Hao, and Ehsan Toreini. 2013. On the private mode isn’t very private at all. Express. (November 20, privacy of private browsing: A forensic approach. Computing Science, Newcastle 2015). http://www.express.co.uk/life-style/science-technology/620887/ University. Porn-Online-Private-Incognito-Browser-Mode. [37] Florian Schaub, Rebecca Balebako, Adam L Durity, and Lorrie Faith Cranor. 2015. [12] Lorrie Faith Cranor, Pedro Giovanni Leon, and Blase Ur. 2016. A large-scale A design space for effective privacy notices. In Proc. SOUPS. evaluation of U.S. financial institutions’ standardized privacy notices. ACM Trans. [38] Chris Smith. 2014. Even Eric Schmidt has no idea how privacy works on one Web 10, 3 (2016), 17:1–17:33. of GoogleâĂŹs top products. (December 17, 2014). http://bgr.com/2014/12/17/ [13] DuckDuckGo. January 2017. A study on private browsing: Consumer usage, eric-schmidt-on-google-chrome-incognito-mode/. knowledge, and thoughts. (January 2017). https://duckduckgo.com/download/ [39] Christopher Soghoian. 2011. Why private browsing modes do not deliver real Private_Browsing.pdf. privacy. Center for Applied Cyber security Research, Bloomington (2011). [14] EFF. Accessed 2017. Panopticlick: Is it possible to defend against browser finger- [40] Daniel J Solove. 2005. A taxonomy of privacy. U. Pa. L. Rev. 154 (2005), 477. printing? https://panopticlick.eff.org/self-defense. (Accessed 2017). [41] Andreas Sotirakopoulos, Kirstie Hawkey, and Konstantin Beznosov. 2012. On [15] Serge Egelman, Janice Tsai, Lorrie Faith Cranor, and Alessandro Acquisti. 2009. the challenges in usable security lab studies: Lessons learned from replicating a Timing is everything?: The effects of timing and placement of online privacy study on SSL warnings. In Proc. SOUPS. indicators. In Proc. CHI. [42] Joshua Sunshine, Serge Egelman, Hazim Almuhimedi, Neha Atri, and Lorrie Faith [16] Alicia Eler. 2012. 5 ways to keep Your Google browsing private. ReadWrite. Cranor. 2009. Crying wolf: An empirical study of SSL warning effectiveness. In (April 3, 2012). https://readwrite.com/2012/04/03/5_ways_to_keep_your_google_ Proc. USENIX Security Symposium. browsing_private/. [43] Janice Y Tsai, Serge Egelman, Lorrie Cranor, and Alessandro Acquisti. 2011. The [17] Steven Englehardt and Arvind Narayanan. 2016. Online tracking: A 1-million-site effect of online privacy information on purchasing behavior: An experimental measurement and analysis. In Proc. ACM CCS. study. Information Systems Research 22, 2 (2011), 254–268. [18] Adrienne Porter Felt, Alex Ainslie, Robert W Reeder, Sunny Consolvo, Somas [44] Blase Ur, Pedro Giovanni Leon, Lorrie Faith Cranor, Richard Shay, and Yang Thyagaraja, Alan Bettes, Helen Harris, and Jeff Grimes. 2015. Improving SSL Wang. 2011. Smart, Useful, Scary, Creepy: Perceptions of Online Behavioral warnings: Comprehension and adherence. In Proc. CHI. Advertising. In Proc. SOUPS. [19] Adrienne Porter Felt, Robert W Reeder, Alex Ainslie, Helen Harris, Max Walker, [45] Anthony Vance, Brock Kirwan, Daniel Bjorn, Jeffrey Jenkins, and Bonnie Brinton Christopher Thompson, Mustafa Embre Acer, Elisabeth Morant, and Sunny Con- Anderson. 2017. What do we really know about how habituation to warnings solvo. 2016. Rethinking connection security indicators. In Proc. SOUPS. occurs over time?: A longitudinal fMRI study of habituation and polymorphic [20] Xianyi Gao, Yulong Yang, Huiqing Fu, Janne Lindqvist, and Yang Wang. 2014. warnings. In Proc. CHI. Private browsing: An inquiry on usability and privacy protection. In Proc. WPES. [46] Shomir Wilson, Florian Schaub, Rohan Ramanath, Norman Sadeh, Fei Liu, Noah A. [21] Google Chrome. 2017. Google Chrome privacy notice. (April 25, 2017). https: Smith, and Frederick Liu. 2016. Crowdsourcing annotations for websites’ privacy //www.google.com/chrome/browser/privacy/. policies: Can it really work?. In Proc. WWW. [22] Google Chrome. Accessed July 7, 2017. Incognito mode (landing page). (Accessed [47] Meng Xu, Yeongjin Jang, Xinyu Xing, Taesoo Kim, and Wenke Lee. 2015. Ucognito: July 7, 2017). Private browsing without tears. In Proc. CCS. [23] Marian Harbach, Markus Hettig, Susanne Weber, and Matthew Smith. 2014. [48] Bin Zhao and Peng Liu. 2015. Private browsing mode not really that private: Using personal examples to improve risk communication for security & privacy Dealing with privacy breach caused by browser extensions. In Proc. DSN. decisions. In Proc. CHI.