A Macroscopic Study of Network Security Threats at the Organizational Level by Jing Zhang A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Science and Engineering) in the University of Michigan 2015 Doctoral Committee: Associate Professor Michael Donald Bailey, UIUC, Co-Chair Professor Mingyan Liu, Co-Chair Assistant Professor J. Alex Halderman Professor Farnam Jahanian, Carnegie Mellon University Professor Brian D. Noble Professor Nicholas A. Valentino c Jing Zhang 2015 All Rights Reserved To my mother Min, my father Jianwei, and my husband Hao. ii ACKNOWLEDGEMENTS First and foremost, I’d like to thank my advisors Michael Bailey and Mingyan Liu. During my doctoral studies at the University of Michigan, they have been close advisors to me on personal, academic, and professional levels. I cannot thank Bailey enough for acting as a great mentor and friend and for providing invaluable direction for my research and life. Moreover, Mingyan inspired me with lots of valuable ideas and gave me great support especially after Bailey moved to UIUC. I want to thank the rest of my doctoral committee, including Alex Halderman, Farnam Jahanian, Brian Nobel, and Nicholas Valentino — all of whom have provided guidance and feedback for my dissertation. I’d like to especially thank Farnam, who guided me through my first steps as a graduate student and continued providing that guidance regarding the direction of my research even after he moved to other positions. I also want to express my gratitude to many people that made this dissertation possible. I’d like to thank the IT and security staff at the University, including: Paul, Will, Matt, who have been great resources in problem discussions, providing access to data, and operational advice. I am very fortunate to have worked with many experienced researchers outside of the University, who have provided data access and valuable comments throughout my doctoral study: Robin Berthier and William Sanders at University of Illinois at Urbana- Champaign, Manish Karir and Michael Kallitsis with Merit Network, Marc Eisenbarth at Arbor Networks, Paul Royal at Gatech, and Elie Bursztein at Google. I’d like to thank all of the software faculty that provide a friendly and helpful envi- ronment for the doctoral program. And I’d like to thank my friends and colleagues at the University that have not only provided support, but given me an enjoyable graduate school experience: Yunjing Xu, Jakub Czyz, Kee Shen Quah, Zakir Durumeric, Yang Liu, Shirley iii Zhe Chen, Zhen Qi, Feng Qian, Xiaoen Ju, and Huan Feng. Finally, and most importantly, I wish to thank my beloved family. This dissertation is dedicated to my mother, Min, my father, Jianwei, and my husband, Hao. I’d like to thank them for their constant support and encouragement throughout my life. iv TABLE OF CONTENTS DEDICATION ..................................... ii ACKNOWLEDGEMENTS .............................. iii LIST OF TABLES ................................... viii LIST OF FIGURES ..................................x ABSTRACT ...................................... xiii CHAPTER 1 Introduction . .1 1.1 Perspectives of Network Security Studies . .1 1.2 Overview of Thesis . .3 1.3 Main Contributions . .4 1.4 Structure of Thesis . .7 2 Characterization of IP-based Reputation Blacklists and Their Impact on An Organization’s Traffic . .8 2.1 Data Set . .9 2.2 Characterize IP-based Reputation Blacklists . 10 2.2.1 Timing . 10 2.2.2 Regional Characteristics . 11 2.2.3 Overlap . 11 2.3 Impact of IP-based Reputation . 12 2.4 Impact of Heavy Hitting IPs . 16 2.4.1 External IP Addresses . 16 2.4.2 Internal IP Addresses . 17 2.4.3 Heavy Hitter Distribution . 18 2.5 Summary . 21 3 Measuring the Longitudinal Evolution of Maliciousness at the Organization- Level ..................................... 22 3.1 Constructing Organization Maliciousness . 24 v 3.1.1 Abstraction for Organizations . 24 3.1.2 Construction . 24 3.1.3 Impact of Abstraction on Persistency and Predictability . 26 3.2 Evolution on the Magnitude of Maliciousness . 29 3.2.1 Characterizing Magnitude of Maliciousness . 29 3.2.2 Trend in Average Malicious Magnitude . 31 3.3 Dynamics in Maliciousness . 36 3.3.1 Characterizing Dynamics . 36 3.3.2 Change in Dynamics . 37 3.3.3 Dynamics v.s. Magnitude . 38 3.4 Summary . 43 4 Understanding the Relationship between Organization Maliciousness and Security Mismanagement . 44 4.1 Symptoms of Mismanagement . 45 4.1.1 Open Recursive Resolvers . 46 4.1.2 DNS Source Port Randomization . 47 4.1.3 Consistent A and PTR records . 48 4.1.4 BGP Misconfiguration . 49 4.1.5 Egress Filtering . 50 4.1.6 Untrusted HTTPS Certificates . 50 4.1.7 SMTP server relaying . 51 4.1.8 Publicly Available Out-of-Band Management Cards . 51 4.1.9 Summary and Limitations of Symptoms . 52 4.2 Mismanagement Symptoms at Autonomous System Level . 53 4.2.1 Abstracting Networks . 53 4.2.2 Distribution of Misconfigured Systems . 53 4.2.3 Correlations between Symptoms . 57 4.3 Unified Network Mismanagement Metric . 59 4.3.1 Combining Symptoms . 59 4.3.2 Geographical Distribution . 59 4.3.3 Topological Roles . 60 4.4 Mismanagement and Maliciousness . 61 4.4.1 Maliciousness of Autonomous Systems . 61 4.4.2 Are Mismanaged Networks more Malicious? . 62 4.4.3 Impact of Aggregation Type on Maliciousness Correlations 67 4.5 Limitations . 67 4.6 Summary . 68 5 Exploring the Tradeoffs of Network Takedowns . 69 5.1 Tradeoff Analysis Framework . 71 5.1.1 Network Abstraction for Takedowns . 72 5.1.2 Cost and Benefit Metrics . 73 5.1.3 Pareto Efficiency Analysis of Costs and Benefits . 75 5.2 Application . 76 vi 5.2.1 Identifying Disreputable Organizations . 77 5.2.2 Measuring the Costs and Benefits . 78 5.2.3 Pareto Efficiency Analysis . 87 5.3 Implications . 90 5.4 Summary . 92 6 Related Work . 93 6.1 Measuring security threats at the organizational level . 93 6.2 Detecting rogue organizations and Detecting threats at the organization- level ................................. 94 6.3 Mitigating solutions at the organization level . 95 6.4 Understanding security threats at the organizational level . 95 7 Conclusion and Future Work . 97 7.1 Summary of Contributions . 97 7.2 Insights and Future Work . 99 7.2.1 Granularity of organizations . 100 7.2.2 External and internal study of organizational security . 100 7.2.3 Proactive and reactive defense . 101 7.2.4 Incentive to secure networks . 102 BIBLIOGRAPHY ................................... 104 vii LIST OF TABLES Table 2.1 Blacklists data sources and attack categories. .9 2.2 Geographic distribution of IPs for each blacklist (%). Reputation black- lists in the same classes share affinity for specific geographic distributions: RIPE and APNIC dominate SPAM; ARIN and RIPE dominate phishing and malware. 11 2.3 The average % (of column) overlap between blacklists (row, column). Classes of blacklists show significant internal entry overlap, but little similarity is seen between classes. 12 2.4 Blacklist entries touched by our network traffic. 14 2.5 Distribution over TCP/UDP ports for top blacklisted external IPs. 17 2.6 Organization of blacklisted internal IP addresses. 18 2.7 Top TCP/UDP ports for traffic tainted by top 50 contributors per blacklist. 20 2.8 Service hosts in top 50 contributors for each blacklist. 20 4.1 Summary of mismanagement metrics and the third-party, public data sources used for validation . 46 4.2 Correlation coefficients and p-values between different mismanagement symptoms. There are significant correlations between different symptoms. (RED: Moderate correlation; BLUE: Weak correlation.) . 58 4.3 Correlation coefficients and p-values between mismanagement and mali- ciousness. There is a statistically significant correlation between our mis- management symptoms and maliciousness. 63 4.4 Multivariate regression with rank of mismanagement as the dependent vari- able and the four social and economic factors as independent variables. All the factors significantly influence the organization’s rank of mismanage- ment: the higher the GDP/GDP per capita, the better the management; the more the customers and peers, the worse the management. 64 4.5 Multivariate regression with rank of maliciousness as the dependent vari- able and the four social and economical factors as independent variables. The results are similar to those of mismanagement in Table 4.4. 65 viii 4.6 Multivariate regression with rank of maliciousness as the dependent vari- able, and mismanagement and the four social and economic factors as in- dependent variables. 66 4.7 Final multivariate regression model for maliciousness and mismanagement when controlled by social and economic factors. Mismanagement is a sig- nificant influencing factor for organization maliciousness when controlled by these latent variables. 66 4.8 Aggregation at BGP prefix level: Correlation coefficient and p-value be- tween mismanagement and maliciousness. 67 5.1 Summary of cost and benefit metrics. 72 5.2 Summary of datasets used in measuring costs and benefits. 73 5.3 T-tests on the difference in number of authoritative name servers between disreputable ASes and other ASes. Disreputable service providers host sig- nificantly fewer name servers than other ASes at a 95% confidence level. 81 5.4 T-tests on the difference in number of TLD queries between disreputable ASes and other ASes. Both disreputable ASes enterprise customers and service providers sent significantly more queries than other ASes at a 95% confidence level. 83 5.5 T-tests on the centrality (degree) in AS-graph between disreputable ASes and others. The disreputable service providers occupied significantly less central positions in routing topology than other providers at a 95% confi- dence level. 84 5.6 T-tests on the difference in number of update events between disreputable ASes and others. The disreputable ASes announced significantly more BGP update events than other ASes at a 95% confidence level.