A Botnet Needle in a Virtual Haystack
Total Page：16
File Type：pdf, Size：1020Kb
ANGLIA RUSKIN UNIVERSITY FACULTY OF SCIENCE AND TECHNOLOGY A BOTNET NEEDLE IN A VIRTUAL HAYSTACK MARK GRAHAM A thesis in partial fulfilment of the requirements of Anglia Ruskin University for the degree of Doctor of Philosophy Submitted: June 2017 Acknowledgements This dissertation was prepared in part fulfilment of the requirements of the degree of Doctor of Philosophy under the supervision of Adrian Winckles and Dr Erika Sanchez-Velazquez at Anglia Ruskin University. This Ph.D. journey would not have been possible without the support that I have received from many people. In particular, I express huge gratitude to my first supervisor Adrian Winckles for his inspiration and support. Adrian has been a mentor to me for many years. A huge thank you also to my second supervisor Dr. Erika Sanchez for her encouragement and motivation. I also extend my thanks to Chris Holmes for his friendship and companionship during my years spent at Anglia Ruskin University. I gratefully acknowledge the funding I received for my Ph.D. from Anglia Ruskin University. Sincere thanks goes to my head of department, Professor Marcian Cirstea for his guidance and advice. May I express my thanks to other members of the department, especially my fellow Ph.D. students; Mohamed Kettouch and Dr. Arooj Fatima. This work is dedicated to Samantha who is always there to listen. i ANGLIA RUSKIN UNIVERSITY ABSTRACT FACULTY OF SCIENCE AND TECHNOLOGY DOCTOR OF PHILOSOPHY Abstract A BOTNET NEEDLE IN A VIRTUAL HAYSTACK MARK GRAHAM JUNE 2017 The Cloud Security Alliance’s 2015 Cloud Adoption Practices and Priorities Survey reports that 73% of global IT professionals cite security as the top challenge holding back cloud services adoption. Malware with the capabilities to jump between the abstracted virtual infrastructures found within cloud service provider networks heightens the threat from botnet attack upon a cloud infrastructure. This research project aimed to provide a novel methodological approach for capturing communication traffic between botnets. The originality of this study comes from the application of standards-based IPFIX flow export protocol as a traffic capture mechanism. The first contribution to knowledge is a critical investigation into how IPFIX export overcomes the limitations of traditional NetFlow-based botnet communication traffic capture in cloud provider networks. The second contribution is the BotProbe IPFIX template, comprising eleven IANA IPFIX information elements. Field occupancy count and Spearman’s Rank correlation on 25 million botnet flows created an IPFIX template tailored specifically for botnet traffic capture. The third contribution is BotStack, a modular, non-intrusive IPFIX monitoring framework, created upon Xen hypervisor and virtual switched platforms, to incorporate IPFIX export into existing cloud stacks. The fourth contribution is compelling empirical evidence from weighted- factor observation across multiple network vantage points, that siting IPFIX exporters on the host hypervisor provides maximum traffic visibility. BotProbe performs on average 26.73%±0.03% quicker than traditional NetFlow v5, with 14.06%±0.01% less storage requirements. BotProbe can be extended with additional application layer attributes, for use in less privacy sensitive environments. Both novel IPFIX templates were tested on the BotStack framework, capturing four distinct traffic profiles in the life cycle of a Zeus botnet. The techniques developed in this research can be repurposed to create IPFIX traffic capture templates for most Cybersecurity threats, including DDoS and spam, turning behavioural-based traffic capture from a big data challenge into a manageable data solution. Keywords: botnet detection, cloud service provider, ipfix, netflow, traffic capture ii Contents Abstract ........................................................................................................................... ii List of Figures .................................................................................................................. v List of Tables ................................................................................................................... vi List of Acronyms ............................................................................................................ vii List of Publications ....................................................................................................... viii Copyright ........................................................................................................................ ix 1. Introduction ................................................................................................................ 1 1.1 Motivation ..................................................................................................................... 2 1.2 Research Aim ................................................................................................................. 4 1.3 Research Hypothesis, Objectives and Boundaries ......................................................... 4 1.4 Research Methodology .................................................................................................. 6 1.5 Ethical Considerations ................................................................................................... 9 1.6 A Summary of the Contributions to Knowledge .......................................................... 10 1.7 Thesis Structure and Organisation ............................................................................... 11 2. Technological Review ................................................................................................ 12 2.1 Introduction ................................................................................................................. 12 2.2 Botnet Fundamentals .................................................................................................. 13 2.3 A Review of the Limitations of Botnet Detection Techniques ..................................... 14 2.4 A Review of Botnet Mitigation Legislation and Responsibility .................................... 16 2.5 A Review of the Cloud as an Attack Platform .............................................................. 17 2.6 A Review of the Cloud as an Attack Surface ................................................................ 19 2.7 A Model for Attacking a Cloud Infrastructure ............................................................. 21 2.8 Summary ...................................................................................................................... 24 3. An Overview of Flow Export ...................................................................................... 25 3.1 Introduction ................................................................................................................. 25 3.2 A Brief History of Flow ................................................................................................. 26 3.3 Flow Export Architecture ............................................................................................. 27 3.4 IPFIX Compared with NetFlow ..................................................................................... 28 3.5 Flow Export Compared with Packet Capture ............................................................... 36 3.6 Flow-Based Botnet Detection ...................................................................................... 38 3.7 Summary ...................................................................................................................... 45 4. BotProbe: A Novel IPFIX Template for Botnet Traffic Capture ................................. 47 4.1 Introduction ................................................................................................................. 47 4.2 IPFIX Template Customisation ..................................................................................... 48 4.3 BotProbe IPFIX Template Creation Methodology........................................................ 49 4.4 Information Elements Results ...................................................................................... 58 4.5 Enterprise Elements Results ........................................................................................ 63 4.6 Presenting the BotProbe IPFIX Templates ................................................................... 72 4.7 Discussion of the BotProbe IPFIX Templates ............................................................... 74 4.8 BotProbe Performance Test Methodology .................................................................. 86 4.9 BotProbe Performance Results .................................................................................... 95 4.10 Discussion of BotProbe Performance ........................................................................ 97 iii 4.11 Summary .................................................................................................................... 99 5. BotStack: A Novel IPFIX Framework ........................................................................ 101 5.1 Introduction ............................................................................................................... 101 5.2 Design Considerations for IPFIX Export in a CSP Environment .................................. 101 5.3 Presenting BotStack: An IPFIX Framework for CSPs .................................................. 113 5.4 Probe Positioning Test Methodology