That Was Then, This Is Now: A Security Evaluation of Password Generation, Storage, and Autofill in Browser-Based Password Managers∗ Sean Oesch Scott Ruoti University of Tennessee, Knoxville University of Tennessee, Knoxville [email protected] [email protected] Abstract websites [11, 15, 25, 33]. Herley points out that this rejection Password managers have the potential to help users more of security advice by users is rational when the low effectively manage their passwords and address many of the percentage of users affected by breaches is contrasted with concerns surrounding password-based authentication. the effort required [18]. However, the number of data However, prior research has identified significant breaches is on the rise [28], and this situation leaves many vulnerabilities in existing password managers; especially in users vulnerable to exploitation. browser-based password managers, which are the focus of Password managers can help users more effectively manage this paper. Since that time, five years has passed, leaving it their passwords. They reduce the cognitive burden placed unclear whether password managers remain vulnerable or upon the user by generating strong passwords, storing those whether they have addressed known security concerns. To passwords, and then filling in the appropriate password when answer this question, we evaluate thirteen popular password a site is visited. The user is now able to follow the latest managers and consider all three stages of the password security advice regarding passwords without placing a high manager lifecycle—password generation, storage, and cognitive burden on themselves. But password managers autofill. Our evaluation is the first analysis of password are not impervious to attack. Li et al. [19] previously found generation in password managers, finding several significant vulnerabilities in major password managers like non-random character distributions and identifying instances LastPass and RoboForm. Both Silver et al. [29] and Stock where generated passwords were vulnerable to online and and Johns [31] demonstrated that browser-based password offline guessing attacks. For password storage and autofill, managers, including LastPass and 1Password, are vulnerable we replicate past evaluations, demonstrating that while to cross-site scripting attacks (XSS) and network injection password managers have improved in the half-decade since attacks as a result of their password autofill features. those prior evaluations, there are still significant issues; these Since these studies five or more years have passed, leaving problems include unencrypted metadata, insecure defaults, it unclear whether password managers remain vulnerable or and vulnerabilities to clickjacking attacks. Based on our whether they are now ready for broad adoption. To answer this results, we identify password managers to avoid, provide question, we update and expand on these previous results and recommendations on how to improve existing password present a thorough, up-to-date security evaluation of thirteen managers, and identify areas of future research. popular password managers. We provide a comprehensive evaluation of browser-based password managers, including five browser extensions and six password managers integrated 1 Introduction directly into the browser. We also include two desktop clients for comparison. Despite the well-established problems facing password-based In our evaluation, we consider the full password manager authentication, it continues to be the dominant form of lifecycle [8]—password generation (Section4), storage authentication used on the web [4]. Because passwords that (Section5), and autofill (Section6). For password generation, are difficult for an attacker to guess are also hard for users to we evaluate a corpus of 147 million passwords generated by remember, users often create weaker passwords to avoid the the studied password managers to determine whether they cognitive burden of recalling them [12,26]. In fact, with the exhibit any non-randomness that an attacker could leverage. increase in the number of passwords users are required to Our results find several issues with the generated passwords, store, they often reuse passwords across the most severe being that a small percentage of shorter ∗This paper will appear at USENIX Security 2020. generated passwords are weak against online and offline 1 attacks (shorter than 10 characters and 18 characters, 2 Background respectively). We also replicate earlier work examining the security of password storage [17] and autofill [19, 29, 31]. In this section, we describe the responsibilities of a password manager. We also describe prior work that has analyzed password managers. Our results find that while password managers have improved in the past five years, there are still significant security concerns. We conclude the paper with several 2.1 Password Managers recommendations on how to improve existing password managers as well as identifying future work that could In the most basic sense, a password manager is a tool that significantly increase the security and usability of password stores a user’s credentials (i.e., username and password) to managers generally (Section7). alleviate the cognitive burden associated with a user remembering many unique login credentials [19]. This store of passwords is commonly referred to as a password vault. Our contributions include: The vault itself is ideally stored in encrypted form, with the encryption key most commonly derived from a user-chosen 1. Our research finds that app-based and extension-based password known as the master password. Optionally, the password managers have improved security compared password vault can be stored online, allowing it to be to five years ago. However, there are still residual synchronized across multiple devices. vulnerabilities that need to be addressed—for example, In addition to storing user-selected passwords, most several tools will automatically fill passwords into modern password managers can help users generate compromised domains without user interaction and passwords. Password generation takes as input the length of others that do require user interaction allow users to the desired password, the desired character set, and any disable it. As such, it is important to both carefully special attribute the password should exhibit (e.g., at least select a password manager and to configure it properly, one digit and one symbol, no hard to recognize characters). something that may be difficult for many users. The password generator outputs a randomly generated 2. To our knowledge, this paper is the first evaluation of password that meets the input criterion. password generation in password managers. As part of Many password managers also help users authenticate to this evaluation, we generated 147 million passwords websites by automatically selecting and filling in (i.e., representing a range of different password managers, autofill) the appropriate username and password. If users character composition policies, and length. We have multiple accounts on the website, the password manager evaluated this corpus using various methods (Shannon will allow users to select which account they wish to use for entropy, c2 test, zxcvbn, and a recurrent neural net) to autofill. find abnormalities and patterns in the generated If properly implemented and used, a password manager has passwords. We found several minor issues with several tangible benefits to the user: generated passwords, as well as a more serious problem 1. It reduces the cognitive burden of remembering where some generated passwords are vulnerable to usernames and passwords. online and offline attacks. 3. Our work is the most comprehensive evaluation of 2. It is easy to assign a different password to every website, password manager security to date. It studies the largest addressing the problem of password reuse. number of password managers (tied with Gasti and Rasmussen [17]) and is the only study that 3. It is easy to generate passwords that are resilient to online simultaneously considers all three stages of the and offline guessing attacks. password manager lifecycle [8]—password generation, storage, and autofill (prior studies considered either storage or autofill, but not both simultaneously). 2.2 Related Work 4. Prior security evaluations of password managers in the Several studies have looked at various aspects of password literature are now five or more years old. In this time, manager security. there have been significant improvements to password Web Security Li et al. [19] analyzed the security of five managers. In our work, we partially or fully replicate extension-based password managers, finding significant these past studies [17, 19, 29, 31] and demonstrate that vulnerabilities in the tools as well as the websites that hosted while many of the issues identified in these studies have the user’s password vault. These vulnerabilities included been addressed, there are still problems such as logic and authorization errors, misunderstandings about the unencrypted metadata, unsafe defaults, and web security model, and CSRF/XSS attacks. They also found vulnerabilities to clickjacking attacks. that password managers that were deployed using 2 bookmarklets did not use iframes properly, leaving the tools likely to reuse password than users of app-based or vulnerable to malicious websites. extensions-based password managers. Google’s Project Zero found a bug in LastPass where Relation to This Work To our knowledge, our work is the