End-to-End Measurements of Email Spoofing Attacks Hang Hu Gang Wang Virginia Tech Virginia Tech [email protected] [email protected] Abstract email phishing has involved in nearly half of the 2000+ reported security breaches in recent two years, causing a Spear phishing has been a persistent threat to users and leakage of billions of user records [4]. organizations, and yet email providers still face key chal- Email spoofing is a critical step in phishing, where lenges to authenticate incoming emails. As a result, at- the attacker impersonates a trusted entity to gain the tackers can apply spoofing techniques to impersonate a victim’s trust. According to the recent report from the trusted entity to conduct highly deceptive phishing at- Anti-Phishing Working Group (APWG), email spoof- tacks. In this work, we study email spoofing to answer ing is widely in spear phishing attacks to target em- three key questions: (1) How do email providers detect ployees of various businesses [2]. Unfortunately, to- and handle forged emails? (2) Under what conditions can day’s email transmission protocol (SMTP) has no built- forged emails penetrate the defense to reach user inbox? in mechanism to prevent spoofing [56]. It relies on (3) Once the forged email gets in, how email providers email providers to implement SMTP extensions such as warn users? Is the warning truly effective? SPF [40], DKIM [19] and DMARC [50] to authenticate We answer these questions by conducting an end-to- the sender. Since implementing these extensions is vol- end measurement on 35 popular email providers and ex- untary, their adoption rate is far from satisfying. Real- amining user reactions to spoofing through a real-world world measurements conducted in 2015 have shown that spoofing/phishing test. Our key findings are three folds. among Alexa top 1 million domains, 40% have SPF, 1% First, we observe that most email providers have the nec- have DMARC, and even fewer are correctly/strictly con- essary protocols to detect spoofing, but still allow forged figured [23, 27]. emails to reach the user inbox (e.g., Yahoo Mail, iCloud, The limited server-side protection is likely to put users Gmail). Second, once a forged email gets in, most email in a vulnerable position. Since not every sender domain providers have no warning for users, particularly for mo- has adopted SPF/DKIM/DMARC, email providers still bile email apps. Some providers (e.g., Gmail Inbox) even face key challenges to reliably authenticate all the incom- have misleading UIs that make the forged email look au- ing emails. When an email failed the authentication, it is thentic. Third, a few email providers (9/35) have imple- a “blackbox” process in terms of how email providers mented visual security indicators on unverified emails. handle this email. Would forged emails still be deliv- Our phishing experiment shows that security indicators ered to users? If so, how could users know the email is have a positive impact on reducing risky user actions, questionable? Take Gmail for example, Gmail delivers but cannot eliminate the risk. Our study reveals a ma- certain forged emails to the inbox and places a security jor miscommunication between email providers and end- indicator on the sender icon (a red question mark, Fig- users. Improvements at both ends (server-side protocols ure 6(a)). We are curious about how a broader range of and UIs) are needed to bridge the gap. email providers handle forged emails, and how much the security indicators actually help to protect users. 1 Introduction In this paper, we describe our efforts and experience in evaluating the real-world defenses against email spoof- Despite the recent development of the system and net- ing1. We answer the above questions through empiri- work security, human factors still remain a weak link. cal end-to-end spoofing measurements, and a user study. As a result, attackers increasingly rely on phishing tac- tics to breach various target networks [62]. For example, 1Our study has been approved by our local IRB (IRB-17-397). First, we conduct measurements on how popular email ing test where deception is carefully applied such that providers detect and handle forged emails. The key idea users examine the spoofed emails without knowing that is to treat each email provider as a blackbox and vary the email is part of an experiment (with IRB approval). the input (forged emails) to monitor the output (the re- We debrief the users and obtain their consent after the ceiver’s inbox). Our goal is to understand under what experiment. conditions the forged/phishing emails are able to reach Our result shows that security indicators have a pos- the user inbox and what security indicators (if any) are itive impact on reducing risky user actions but cannot used to warn users. Second, to examine how users react eliminate the risk. When a security indicator is not pre- to spoofing emails and the impact of security indicators, sented (the controlled group), out of all the users that we conduct a real-world phishing test in a user study. opened the spoofed email, 48.9% of them eventually We have carefully applied “deception” to examine users’ clicked on the phishing URL in the email. For the other natural reactions to the spoofing emails. group of users to whom we present the security indica- tor, the corresponding click-through rate is slightly lower Measurements. We start by scanning Alexa top 1 (37.2%). The impact is consistently positive for users million hosts from February 2017 to January 2018. We of different demographics (age, gender, education level). confirm that the overall adoption rates of SMTP secu- On the other hand, given the 37.2% click-through rate, rity extensions are still low (SPF 44.9%, DMARC 5.1%). we argue that the security indicator cannot eliminate the This motivates us to examine how email providers handle phishing risk. The server-side security protocols and the incoming emails that failed the authentication. user-end security indicators should be both improved to We conduct end-to-end spoofing experiments on 35 maximize the impact. popular email providers used by billions of users. We find that forged emails can penetrate the majority of Contributions. We have 3 key contributions: email providers (34/35) including Gmail, Yahoo Mail • First, our end-to-end measurement provides new in- and Apple iCloud under proper conditions. Even if sights into how email providers handle forged emails. the receiver performs all the authentication checks (SPF, We reveal the trade-offs between email availability DKIM, DMARC), spoofing an unprotected domain or a and security made by different email providers domain with “relaxed” DMARC policies can help the • Second, we are the first to empirically analyze the forged email to reach the inbox. In addition, spoofing usage of security indicators on spoofed emails. We an “existing contact” of the victim also helps the attacker show that most email providers not only lack the to penetrate email providers (e.g., Hotmail). necessary security indicators (particularly on mobile More surprisingly, while most providers allow forged apps), but also have misleading UIs that help the at- emails to get in, rarely do they warn users of the unver- tackers. ified sender. Only 9 of 35 providers have implemented some security indicators: 8 providers have security in- • Third, we conduct a real-world phishing test to eval- dicators on their web interface (e.g., Gmail) and only 4 uate the effectiveness of the security indicator. We providers (e.g., Naver) have the security indicators con- demonstrate the positive impact (and potential prob- sistently for the mobile apps. There is no security warn- lems) of the security indicator and provide the initial ing if a user uses a third-party email client such as Mi- guidelines for improvement. crosoft Outlook. Even worse, certain email providers The quantitative result in this paper provides an end- have misleading UI elements which help the attacker to to-end view on how spoofed emails could penetrate ma- make forged emails look authentic. For example, when jor email providers and all the way affect the end users. attackers spoof an existing contact (or a user from the We hope the results can draw more attention from the same provider), 25 out of 35 providers will automatically community to promoting the adoption of SMTP security load the spoofed sender’s photo, a name card or the email extensions. In addition, we also seek to raise the atten- history along with the forged email. These UI designs are tion of email providers to designing and deploying more supposed to improve the email usability, but in turn, help effective UI security indicators, particularly for the less the attacker to carry out the deception when the sender protected mobile email apps. We have communicated address is actually spoofed. the results with the Gmail team and offered suggestions Phishing Experiment. While a handful of email to improve the security indicators. providers have implemented security indicators, the real question is how effective they are. We answer this ques- 2 Background and Methodology tion using a user study (N = 488) where participants ex- amine spoofed phishing emails with or without security Today’s email system is built upon the SMTP protocol, indicators on the interface. This is a real-world phish- which was initially designed without security in mind. Sender Service Receiver Service HTTP authorized to send emails for its domain (RFC7208 [40]). MUA HTTP IMAP MUA SMTP MSA MTA SMTP MTA MDA POP For example, if a domain “a.com” published its SPF 1 2 3 record in the DNS, then the receiving email services can Alex a.com b.com Bob check this record to match the sender IP with the sender email address.