PhishTime: Continuous Longitudinal Measurement of the Effectiveness of Anti- Blacklists

Adam Oest, Yeganeh Safaei, Penghui Zhang, Yan Shoshitaishvili, Adam Doupé, Gail-Joon Ahn Arizona State University

Brad Wardman, Kevin Tyers PayPal Motivation

• Phishing attacks deceive users through malicious websites/messages

• May seem trivial on the surface…

• But phishing occurs at scale and works

2 3 Anti-phishing Blacklists

• Key ecosystem defense • Default in major desktop + mobile browsers • App and e-mail integration • Automated crawler backend • Goals • Timely, comprehensive detection • Low false positive rate

• Vulnerable to evasion techniques (“cloaking”) [1] [1] PhishFarm: A Scalable Framework for Measuring Evasion Techniques Against Browser Phishing Blacklists Adam Oest, Yeganeh Safaei, Adam Doupé, Gail-Joon Ahn, Brad Wardman, and Kevin Tyers. IEEE Symposium on Security & Privacy, May 2019. 4 Browser/Blacklist Selection • Safe Browsing (GSB), MS SmartScreen,

Desktop Blacklists Mobile Blacklists

SmartScreen Opera None/Other None/Other 10% 2% 2% 14% Opera 3%

GSB GSB 86% 83%

Estimated market share as of December 2019 5 Blacklist Evaluation Criteria

• Coverage: does blacklisting always occur?

• Speed: delay between attack deployment and blacklisting

• Consistency across platforms

Security implications of gaps?

6 Research Objectives

How vulnerable is the ecosystem, as a whole, to modern-day phishing?

• Continuous monitoring of blacklists • Long-term verification of baseline defenses • Identification of practical gaps

• Realistically evaluate blacklisting delays • Discover then test evasion used in the wild • Simulate ecosystem detection methods

7 PhishTime Framework: Discovering Evasive Phishing in the Wild

Monitor Blacklisting of Report non-backlisted (4,393) Live Phishing URLs Discard if blacklisted

Design & Deploy Analyze non- (183 - 4.2%) Experiments blacklisted sites w/ Artificial Websites*

*using an enhanced version of the empirical testbed proposed in [1] 8 Artificial Website Configurations

A. Allow all traffic (control group)

Baseline B. Basic cloaking

C. Combinations of cloaking (redirection + .htaccess)

Typical D. Combinations w/ infrastructure re-use

F. Innovative evasion techniques

Emerging G. New reporting protocols

9 10 Longitudinal Experiments

6 deployments + 1 preliminary A B C D F G

Simultaneously reported to anti-phishing entities:

2,862 sites / 4,158 URLs total (new, randomized .com domains)

Monitor blacklisting status for 1 week

11 Baseline Blacklisting

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% May-19 Jul-19 Sep-19 Oct-19 Nov-19 Dec-19 SmartScreen

12 Baseline Blacklisting

13 Blacklist Speed & Coverage

Desktop Chrome Mobile Mobile Chrome

Speed Coverage Coverage Speed Coverage (hh:mm)

Baseline (no evasion) 00:50 99% 99% 24:04 53%

Basic Evasion 00:59 94% 94% - 0%

Typical Evasion 02:48 88% 88% 21:05 2%

Infrastructure Re-use 02:10 96% 96% 23:27 4%

Emerging Evasion - 0% 0% - 0%

14 Current Reporting Channels

15 Reporting Protocol Shortcomings

• (re)submission of the URL alone is no good against advanced cloaking

16 URL Submission Metadata

17 Evidence-based Reporting

18 Enhanced Reporting vs. Evasive Phishing

19 Disclosures & Impact

20 Conclusions

• Longitudinal measurements are key to understanding ecosystem protections • Proactive anti-phishing approach • Discovering sophisticated attack variants • Not currently being done at the ecosystem level

• Sophisticated evasion remains a threat • Closing blacklisting gaps on mobile devices • Improving data sharing, reporting, detection

• Understanding the impact of blacklisting delays on victims [2]

[2] Sunrise to Sunset: Analyzing the End-to-end Life Cycle and Effectiveness of Phishing Attacks at Scale Adam Oest, Penghui Zhang, Brad Wardman, Eric Nunes, Jakub Burgis, Ali Zand, Kurt Thomas, Adam Doupé, Gail-Joon Ahn. USENIX Security Symposium, August 2020.

21 Thank you!

Adam Oest [email protected]

22