MEASURING THE IMPACT OF DMARC’S PART IN PREVENTING BUSINESS EMAIL COMPROMISE Adam Shostack, Jay Jacobs and Wade Baker
WWW.GLOBALCYBERALLIANCE.ORG Executive Summary
The deployment of DMARC is an effective way to limit the spoofing of email from protected domains, however, there has been little research on the In short, for 2018, the estimated value to economic value of deploying DMARC. This paper the 1,046 organizations that have deployed shows that there is in fact significant value from DMARC at a policy level of “reject” or deployment of DMARC, even looking at only one “quarantine,” after using GCA’s tool, is specific benefit of many (limited to Business Email likely $19M (USD). That assumes: Compromise, or BEC) and only from deploying DMARC at one of the higher policy levels. ɬɬ Only 1% of BEC emails that a person receives result in a successful BEC attack.
ɬɬ DMARC stops email impersonation, not other attacks, such as Since June 2016, the Global Cyber Alliance (GCA) phishing/theft of passwords, has been working to accelerate adoption of DMARC, identity theft, or other compromises an email security standard, by providing a set of of the computer. Note: these other easy-to-use tools and campaigns to drive deployment. attacks may be impacted by DMARC, This paper investigates and measures the economic but the pathways are diverse, and benefit from that work. Having reviewed the available measurement is impeded by data data, we have chosen to focus on Business Email quality issues. Compromise (BEC) because it is a rapidly growing issue, with high direct losses, and relevant data is ɬɬ There are other benefits from available for analysis from multiple sources1. We DMARC, including benefits for email derive a conservative minimum bar estimate for the deliverability and by deploying loss avoidance tied to GCA’s initiatives and discuss the DMARC at a monitoring level. Again, potential scale of other benefits gained from DMARC. there are benefits from both, but these costs are not quantified in this study.
ɬɬ GCA’s public advocacy and training had no other benefits in encouraging the deployment of DMARC.
1 According to the 2017 FBI/IC3 Internet Crime Report, more losses were attributed to BEC scams than any other type of threat.
www.globalcyberalliance.org | 2 Executive Summary
This number scales up rapidly as factors are adjusted. For example, if 5% of BEC attacks result in a user taking action, the prevented loss rises to $66M. These amounts grow over time, so that if:
1. 1,046 organizations maintain DMARC, but GCA’s efforts result in no additional deployments (that unlikely assumption makes this a low-water mark); and
2. The economic impact of BEC grows at 5% per year2, then the benefit over 5 years would be approximately $110M. Projecting over a longer timeframe involves increasing uncertainty.
2 The FBI released new numbers as this report went to press, showing a 136% increase in BEC from December 2016 to May 2018. Such rates seem unlikely to sustain consistently over a 5 year period.
Moreover, the $19M estimate is merely from GCA’s activity. GCA’s other partners, including nonprofit organizations, commercial vendors, governments, and enterprises are also deploying and supporting the deployment of DMARC. For these organizations, the most-affected 1% of them will save $302,000 per year solely from reducing BEC through DMARC, even assuming only 1% of BEC emails lead to action. If that number grows to 5%, the loss reduction for that 1% is $1.3M per year. Adding that up across the Internet is a big number, and that is from BEC losses alone. In conclusion, if the people in your company will not be fooled by a targeted email that appears to be from your CEO asking for a transfer of funds to a supplier as an emergency matter, you may not worry about BEC. If, however, there is only a 25% chance your CFO might make a mistake, and you are an average size company, it is more likely than not you will lose somewhere between $8,800 and $4,700,000 depending on how lucky you are.
If there is only a 25% chance your CFO might make a mistake, and you are an average size company, it is more likely than not you will lose somewhere between $8,800 and $4,700,000.
www.globalcyberalliance.org | 3 Contents
Executive Summary...... 2 1. Introduction...... 5 1.1 Context and Background ...... 5 1.2 Methodology...... 6 2. Models...... 8 2.1 Models of Email Attack ...... 8 2.1.1 Business Email Compromise ...... 11 2.2 Data ...... 13 2.2.1 Data for Item 1, Number of BEC Attacks Per Firm Per Year...... 13 2.2.2 Data for Item 2, Emails Spoofing Domain...... 15 2.2.3 Data for Item 3, Human Action Rate...... 17 2.3 Other Email Attacks...... 17 2.4 Loss Model...... 17 2.4.1 Loss Per Incident...... 18 2.4.2 Estimating Losses Over Many Organizations...... 20 3. Benefits From GCA’s DMARC Project...... 22 4. Other Benefits of DMARC...... 24 5. Discussion & Methodology...... 26 5.1 Economics...... 26 5.2 Phishing...... 29 5.3 Caveats and Limitations...... 30 5.4 Assumptions...... 30 Conclusion...... 32 DMARC Primer...... 33 Glossary...... 34 References...... 35 About the Global Cyber Alliance...... 38 1. Introduction
In August of 2017, 1.1 Context and Background GCA asked for help to: DMARC stands for Domain-based Message Authentication, Reporting and Conformance. It is the ɬɬ Measure return on investment simple, trusted, solution that brings together email from GCA projects, determining authentication protocols, and adds reporting the level of risk reduction and compliance. provided by the resources expended by GCA (and others). There is growing support for DMARC. In June 2016, the ɬ U.K. government mandated that all U.K. government ɬ Develop a mechanism for departments adopt DMARC, and the EU-CERT has determining systemic cyber also made a recommendation for the use of DMARC. risk that can be used for future In October 2017, the U.S. Department of Homeland projects both to measure risk Security issued Binding Operational Directive 18-01, and determine what data needs which requires the adoption of DMARC by federal to be recorded. civilian domains. Shostack and Associates and the Cyentia Institute have been helping GCA discovered, however, that despite the GCA work towards these goals, with the tremendous benefits of DMARC, it was not being first project focused on their DMARC widely deployed in the public or private sectors. work. As we conducted the analysis, They wanted to make it easier and affordable for it became apparent that the impact any organization or user to implement DMARC. of DMARC has both loss-avoidance So GCA created the step-by-step DMARC Setup impacts and brand and deliverability Guide, currently available in 18 languages, to help impacts. We have focused this analysis organizations of all sizes implement DMARC. on the BEC subset of loss avoidance, and have developed new mechanisms for understanding that set of losses.
www.globalcyberalliance.org | 5 Introduction
1.2 Methodology We began with a review of the GCA DMARC Setup Guide tool, and implemented DMARC across several of our domains to gain familiarity with the technology. We reviewed available reports on DMARC drawing heavily but not exclusively on the Cyentia Library, a public and searchable collection which at the time comprised 862 industry research reports curated by the Cyentia Institute. In a first pass, we reviewed 44 matches for the term “BEC”3 and also looked at references to “phish” and “phishing”. In a final review pass, we checked for the same search terms in 300 new documents added to the library to see if additional data had been published. We reached out to GCA partners repeatedly to ensure that we had gathered all the data available. We crafted preliminary models and presented them at the Anti-Phishing Working Group’s E-Crime Conference in San Diego in May, 2018, and then again in a webinar with the goals of getting feedback on the models and data to inform our analysis. Some of this preliminary analysis led us to the understanding of the breadth and diversity of definitions, methods, analyses and production processes associated with published data. We discuss some of these approaches in section 5 to address why we chose not to go down these avenues.
Estimates of economic benefit from DMARC can be grouped into 4 categories:
ɬɬ Costs from BEC attacks
ɬɬ Costs from other attacks (phishing, malware and breaches)
ɬɬ Benefits of deliverability
ɬɬ Ecosystem benefits
We focus on BEC benefits to the organizations deploying DMARC because it represents a major category of threats relevant to DMARC, the data is relatively closely grouped, with much of it coming from the FBI4. The FBI remains a credible source of crime data. This methodology provides a floor, rather than a complete estimate. The other types of benefit, with their higher variance, may be addressed in future work.
We consider these benefits to accrue to those who have defined a DMARC policy of ‘quarantine’ or ‘reject.’ (Organizations can also deploy DMARC with a policy of ‘none,’ see section 4 for a discussion of the benefits from the none setting, and section 7 for more on the effects of these settings.)
3 More precisely, filter(token0 == “bec” | token0 == “business” & token == “email” & (token2 %in% c(“compromise”, “spoof”, “spoofing”)))
4 As this report went to press, the FBI’s Internet Crime Complaint Center released new data, which we have not incorporated. (FBI18) They note that ‘between December 2016 and May 2018, there was a 136% increase in identified global exposed losses,” which means that our estimates are again, a low-water mark. (EDIT-SA-19)
www.globalcyberalliance.org | 6 Introduction
Reviewers pointed out that the per-organization loss avoidance numbers “don’t seem like they’re enough to drive activity.” We are reminded of a story about Miss Manners, who gets into a taxi at JFK and asks to be taken to Newark. “Newark?!?” the driver exclaims, “But that’s so far!” “Very well,” Miss Manners responds, “where would you like to go?” We have gone where the data has taken us, but would add that the cost of deploying DMARC does not have to be particularly high. One author of this report did it over the course of a few hours. Large organizations may take months to track down all of the various departments, contractors, and service bureaus sending email on their behalf around the globe. That time may be because finding email senders can be tricky, and because some emails may be sent monthly or even quarterly by different processes. The work to get all that “under control” is also a hard-to-measure benefit.
We also chose not to address estimates of criminal gain from these activities, as criminals are often untrustworthy. We chose to not address societal losses from these activities.
The rest of this paper is organized as follows. Section 2 presents the models of email attacks and BEC that we used, the data we considered, and the loss model on which we settled. Section 3 presents the results of a set of simulations based on the data and loss model. Section 4 presents an overview of the other benefits of DMARC. This is followed by a discussion of the methodology (section 5), some conclusions (section 6), a primer on DMARC (section 7) , a glossary (section 8), references (section 9) and finally, information about the authors.
Recommendations
ɬɬ Use the Global Cyber Alliance DMARC Setup Guide at https://www.globalcyberalliance.org/dmarc/ to check the state of your organization’s DMARC records today.
ɬɬ If you do not publish DMARC records, set them up and start gathering data. DMARC allows you to do this without impact to your email infrastructure or deliverability by setting a “none” policy.
ɬɬ Ensure that your inbound email is configured to respect other organizations’ DMARC policies.
www.globalcyberalliance.org | 7 2. Models
We use a variety of models, including models of email attacks and models of losses, to help us understand the impact of DMARC on inbound BEC. We present models and then the data which informs their use. We look at the email models first, then the economic models.
2.1 Models of Email Attack To assess the impact of DMARC, we consider the likelihood that DMARC prevents an inbound email from reaching an inbox. DMARC actually makes spoofing email from an organization harder, and thus applies to all mail, and we are looking at this subset for methodological reasons. We present a modified attack tree5 for understanding the flow of inbound email, which is important in understanding how DMARC helps. We then compare this attack tree to Agari’s email threats taxonomy.
Dmarc actually makes spoofing email from an organization harder.
Our phishing attack tree is scoped to email attacks on a target that the attacker has not yet compromised. It considers a message progressing through time and systems on the vertical axis. A message starts when an attacker sends it, goes to the target’s mail server where it is subject to DMARC testing, then the target’s inbox where it can be acted upon.
5 A threat tree is a structured technique to enumerate all the paths by which a system may be attacked. Our tree differs from a standard threat tree in that it shows time and systems, and not all nodes are threats. It is also not a complete enumeration of all threats. See (Amaroso) for a detailed description of threat trees.
www.globalcyberalliance.org | 8 Models
Attacker Sends ail nbo n