That Was Then, This Is Now: A Security Evaluation of Password Generation, Storage, and Autofill in Browser-Based Password Managers Sean Oesch and Scott Ruoti, University of Tennessee https://www.usenix.org/conference/usenixsecurity20/presentation/oesch This paper is included in the Proceedings of the 29th USENIX Security Symposium. August 12–14, 2020 978-1-939133-17-5 Open access to the Proceedings of the 29th USENIX Security Symposium is sponsored by USENIX. That Was Then, This Is Now: A Security Evaluation of Password Generation, Storage, and Autofill in Browser-Based Password Managers Sean Oesch Scott Ruoti University of Tennessee, Knoxville University of Tennessee, Knoxville [email protected] [email protected] Abstract of security advice by users is rational when the low percentage of users affected by breaches is contrasted with Password managers have the potential to help users more the effort required [18]. However, the number of data effectively manage their passwords and address many of the breaches is on the rise [28], and this situation leaves many concerns surrounding password-based authentication. users vulnerable to exploitation. However, prior research has identified significant Password managers can help users more effectively manage vulnerabilities in existing password managers; especially in their passwords. They reduce the cognitive burden placed browser-based password managers, which are the focus of upon the user by generating strong passwords, storing those this paper. Since that time, five years has passed, leaving it passwords, and then filling in the appropriate password when unclear whether password managers remain vulnerable or a site is visited. The user is now able to follow the latest whether they have addressed known security concerns. To security advice regarding passwords without placing a high answer this question, we evaluate thirteen popular password cognitive burden on themselves. But password managers managers and consider all three stages of the password are not impervious to attack. Li et al. [19] previously found manager lifecycle—password generation, storage, and significant vulnerabilities in major password managers like autofill. Our evaluation is the first analysis of password LastPass and RoboForm. Both Silver et al. [29] and Stock generation in password managers, finding several and Johns [31] demonstrated that browser-based password non-random character distributions and identifying instances managers, including LastPass and 1Password, are vulnerable where generated passwords were vulnerable to online and to cross-site scripting attacks (XSS) and network injection offline guessing attacks. For password storage and autofill, attacks as a result of their password autofill features. we replicate past evaluations, demonstrating that while password managers have improved in the half-decade since Since these studies five or more years have passed, leaving those prior evaluations, there are still significant issues; these it unclear whether password managers remain vulnerable or problems include unencrypted metadata, insecure defaults, whether they are now ready for broad adoption. To answer this and vulnerabilities to clickjacking attacks. Based on our question, we update and expand on these previous results and results, we identify password managers to avoid, provide present a thorough, up-to-date security evaluation of thirteen recommendations on how to improve existing password popular password managers. We provide a comprehensive managers, and identify areas of future research. evaluation of browser-based password managers, including five browser extensions and six password managers integrated directly into the browser. We also include two desktop clients 1 Introduction for comparison. In our evaluation, we consider the full password manager Despite the well-established problems facing password-based lifecycle [8]—password generation (Section4), storage authentication, it continues to be the dominant form of (Section5), and autofill (Section6). For password generation, authentication used on the web [4]. Because passwords that we evaluate a corpus of 147 million passwords generated by are difficult for an attacker to guess are also hard for users to the studied password managers to determine whether they remember, users often create weaker passwords to avoid the exhibit any non-randomness that an attacker could leverage. cognitive burden of recalling them [12,26]. In fact, with the Our results find several issues with the generated passwords, increase in the number of passwords users are required to the most severe being that a small percentage of shorter store, they often reuse passwords across generated passwords are weak against online and offline websites [11, 15, 25, 33]. Herley points out that this rejection attacks (shorter than 10 characters and 18 characters, USENIX Association 29th USENIX Security Symposium 2165 respectively). We also replicate earlier work examining the 2 Background security of password storage [17] and autofill [19, 29, 31]. In this section, we describe the responsibilities of a password manager. We also describe prior work that has analyzed Our results find that while password managers have password managers. improved in the past five years, there are still significant security concerns. We conclude the paper with several recommendations on how to improve existing password 2.1 Password Managers managers as well as identifying future work that could In the most basic sense, a password manager is a tool that significantly increase the security and usability of password stores a user’s credentials (i.e., username and password) to managers generally (Section7). alleviate the cognitive burden associated with a user remembering many unique login credentials [19]. This store of passwords is commonly referred to as a password vault. Our contributions include: The vault itself is ideally stored in encrypted form, with the encryption key most commonly derived from a user-chosen 1. Our research finds that app-based and extension-based password known as the master password. Optionally, the password managers have improved security compared password vault can be stored online, allowing it to be to five years ago. However, there are still residual synchronized across multiple devices. vulnerabilities that need to be addressed—for example, In addition to storing user-selected passwords, most several tools will automatically fill passwords into modern password managers can help users generate compromised domains without user interaction and passwords. Password generation takes as input the length of others that do require user interaction allow users to the desired password, the desired character set, and any disable it. As such, it is important to both carefully special attribute the password should exhibit (e.g., at least select a password manager and to configure it properly, one digit and one symbol, no hard to recognize characters). something that may be difficult for many users. The password generator outputs a randomly generated 2. To our knowledge, this paper is the first evaluation of password that meets the input criterion. password generation in password managers. As part of Many password managers also help users authenticate to this evaluation, we generated 147 million passwords websites by automatically selecting and filling in (i.e., representing a range of different password managers, autofill) the appropriate username and password. If users character composition policies, and length. We have multiple accounts on the website, the password manager evaluated this corpus using various methods (Shannon will allow users to select which account they wish to use for entropy, c2 test, zxcvbn, and a recurrent neural net) to autofill. find abnormalities and patterns in the generated If properly implemented and used, a password manager has passwords. We found several minor issues with several tangible benefits to the user: generated passwords, as well as a more serious problem 1. It reduces the cognitive burden of remembering where some generated passwords are vulnerable to usernames and passwords. online and offline attacks. 3. Our work is the most comprehensive evaluation of 2. It is easy to assign a different password to every website, password manager security to date. It studies the largest addressing the problem of password reuse. number of password managers (tied with Gasti and Rasmussen [17]) and is the only study that 3. It is easy to generate passwords that are resilient to online simultaneously considers all three stages of the and offline guessing attacks. password manager lifecycle [8]—password generation, storage, and autofill (prior studies considered either storage or autofill, but not both simultaneously). 2.2 Related Work 4. Prior security evaluations of password managers in the Several studies have looked at various aspects of password literature are now five or more years old. In this time, manager security. there have been significant improvements to password Web Security Li et al. [19] analyzed the security of five managers. In our work, we partially or fully replicate extension-based password managers, finding significant these past studies [17, 19, 29, 31] and demonstrate that vulnerabilities in the tools as well as the websites that hosted while many of the issues identified in these studies have the user’s password vault. These vulnerabilities included been addressed, there are still problems such as logic and authorization errors, misunderstandings about the unencrypted metadata,