shankar karuppayah ADVANCED MONITORING IN P2P BOTNETS [ June 20, 2016 at 11:21 – classicthesis ] [ June 20, 2016 at 11:21 – classicthesis ] ADVANCED MONITORING IN P2P BOTNETS shankar karuppayah Vom Fachbereich Informatik der Technischen Universität Darmstadt genehmigte Dissertation zur Erlangung des akademischen Grades Doctor rerum naturalium (Dr. rer. nat.) Eingereicht von: M.Sc. Shankar Karuppayah geboren in Georgetown, Penang Erstreferent: Prof. Dr. Max Mühlhäuser (Technische Universität Darmstadt) Koreferent: Prof. Dr. Vern Paxson (University of California, Berkeley) Tag der Einreichung: 18. Mai 2016 Tag der Prüfung: 01. Juni 2016 Fachgebiet Telekooperation Fachbereich Informatik Technische Universität Darmstadt Hochschulkennziffer D-17 Darmstadt 2016 [ June 20, 2016 at 11:21 – classicthesis ] Shankar Karuppayah: Advanced Monitoring in P2P Botnets, © May 2016 [ June 20, 2016 at 11:21 – classicthesis ] ACKNOWLEDGMENTS This thesis would not have come into existence without the help and encouragement of colleagues, friends and family. My heartfelt grati- tude and thanks go out to all of them. First, I would like to express my sincere gratitude to my advisor Max Mühlhäuser for the continuous support of my PhD study and related research, for his patience, motivation, and immense knowledge. His guidance helped me in all the time of research and writing of this the- sis. I could not have imagined having a better advisor and mentor for my PhD study. Second, I am grateful to Vern Paxson for acting as a second referee. Third, I am indebted to the supervision of Mathias Fischer since the beginning of my PhD study.The various discussions, experiences and guidance I received from him are definitely unforget- table. I would like to thank all my colleagues and students at the Teleco- operation Group of TU Darmstadt, who have all shared wonderful memories with me, especially all of those at the areas of Smart Se- curity Infrastructures and Smart Security and Trust: Leon Böck, Jörg Daubert, Carlos Garcia, Tim Grube, Sheikh Mahbub Habib, Steffen Haas, Sascha Hauke, Stefan Schiffner, Emmanouil Vasilomanolakis, Florian Volk. They, and everybody else at TK, made this phase of my life a pleasure. Not forgetting, a great thank you to the great staff at the group: Elke Halla and Fabian Herrlich among others, for their pa- tience and cooperation in handling me throughout this three and a half year. Special thanks to my MIG family that has always been proud of my achievements. Particularly, Mr. Sivasuriyamoorthy Sundara Raja and Mrs. Dhamayanthy Singaram (Selvi Akka) for being both a strong emotional support and a mentor for me. Also not forgetting, Aroon Kumar Mathivanan whom has always been there whenever I am in need of a break from my stressful days. My biggest thanks definitely goes to my future wife, Prevathe Poniah. You have tolerated my physical and psychological absence through- out these years. Thank you for being my support when I needed it the most. I could not have done it without you. Last but not the least, I would like to thank my family: my parents Mr. C. Karuppayah and Mrs. K. Anjama, my brothers and sisters-in- law for supporting me emotionally and spiritually throughout writing this thesis and my life in general. Without your supports, I would not have been successful. This thesis would also not have been possible without the funding and support of Malaysian Ministry of Higher Education and Univer- siti Sains Malaysia. v [ June 20, 2016 at 11:21 – classicthesis ] [ June 20, 2016 at 11:21 – classicthesis ] ABSTRACT Botnets are increasingly being held responsible for most of the cyber- crimes that occur nowadays. They are used to carry out malicious ac- tivities like banking credential theft and Distributed Denial of Service (DDoS) attacks to generate profit for their owner, the botmaster. Tradi- tional botnets utilized centralized and decentralized Command-and- Control Servers (C2s). However, recent botnets have been observed to prefer P2P-based architectures to overcome some of the drawbacks of the earlier architectures. A P2P architecture allows botnets to become more resilient and robust against random node failures and targeted attacks. However, the distributed nature of such botnets requires the defenders, i.e., re- searchers and law enforcement agencies, to use specialized tools such as crawlers and sensor nodes to monitor them. In return to such mon- itoring, botmasters have introduced various countermeasures to im- pede botnet monitoring, e.g., automated blacklisting mechanisms. The presence of anti-monitoring mechanisms not only render any gathered monitoring data to be inaccurate or incomplete, it may also adversely affect the success rate of botnet takedown attempts that rely upon such data. Most of the existing monitoring mechanisms identified from the related works only attempt to tolerate anti-monitoring mechanisms as much as possible, e.g., crawling bots with lower fre- quency. However, this might also introduce noise into the gathered data, e.g., due to the longer delay for crawling the botnet. This in turn may also reduce the quality of the data. This dissertation addresses most of the major issues associated with monitoring in P2P botnets as described above. Specifically,it analyzes the anti-monitoring mechanisms of three existing P2P botnets: 1) GameOver Zeus, 2)Sality, and 3) ZeroAccess, and proposes counter- measures to circumvent some of them. In addition, this dissertation also proposes several advanced anti-monitoring mechanisms from the perspective of a botmaster to anticipate future advancement of the botnets. This includes a set of lightweight crawler detection mech- anisms as well as several novel mechanisms to detect sensor nodes deployed in P2P botnets. To ensure that the defenders do not loose this arms race, this dissertation also includes countermeasures to cir- cumvent the proposed anti-monitoring mechanisms. Finally, this dis- sertation also investigates if the presence of third party monitoring mechanisms, e.g., sensors, in botnets influences the overall churn measurements. In addition, churn models for Sality and ZeroAccess are also derived using fine-granularity churn measurements. The works proposed in this dissertation have been evaluated us- ing either real-world botnet datasets, i.e., that were gathered using crawlers and sensor nodes, or simulated datasets. Evaluation results indicate that most of the anti-monitoring mechanisms implemented by existing botnets can either be circumvented or tolerated to ob- vii [ June 20, 2016 at 11:21 – classicthesis ] tain monitoring data with a better quality. However, many crawlers and sensor nodes in existing botnets are found vulnerable to the anti- monitoring mechanisms that are proposed from the perspective of a botmaster in this dissertation. Analysis of the fine-grained churn measurements for Sality and ZeroAccess indicate that churn in these botnets are similar to that of regular P2P file-sharing networks like Gnutella and Bittorent. In addition, the presence of highly respon- sive sensor nodes in the botnets are found not influencing the over- all churn measurements. This is mainly due to low number of sen- sor nodes currently deployed in the botnets. Existing and future bot- net monitoring mechanisms should apply the findings of this disser- tation to ensure high quality monitoring data, and to remain unde- tected from the bots or the botmasters. viii [ June 20, 2016 at 11:21 – classicthesis ] ZUSAMMENFASSUNG Heute werden Botnetze zunehmen für die Mehrzahl der verübten Cyber- Straftaten verantwortlich gemacht. Die Besitzer der Botnetze, soge- nannte Botmaster, nutzen die Netze um bösartige Aktivitäten wie bei- spielsweise den Diebstahl von Bankzugangsdaten und Distributed Denial of Service (DDOS) Angriffe durchzuführen. Ein Botmaster kon- trolliert das eigene Botnetz mit einem Command-and-Control Ser- ver (C2) und verteilt über diesen Befehle und Aktualisierungen an die Bots. Traditionelle Botnetze verwendeten zentrale oder verteilte C2s. Allerdings zeigen Beobachtungen, dass aktuelle Botnetze mehr und mehr auf P2P-basierte Architekturen setzen und damit die Nachtei- le einer zentralen Architektur umgehen. P2P-basierte Botnetze sind widerstandsfähiger und robuster gegenüber zufälligen Ausfällen und gezielten Angriffen. Bots in einem P2P Botnetz sind über ein Overlay miteinander ver- bunden. Dieses Overlay wird kollaborativ und verteilt von den Bots verwaltet. Diese verteilte Verwaltung erschwert die Überwachung und macht spezialisierte Überwachungslösungen wie Crawler und Sen- sorknoten nötig. Diese Überwachungslösungen setzen Wissen über das jeweilige Botnetzprotokoll und dessen Nachrichtenformat vor- aus um teilnehmende Bots und deren Kommunikationsbeziehungen zu bestimmen. Darüber hinaus setzen Botmaster diverse Gegenmaß- nahmen ein, wie beispielsweise automatisches Blacklisting. Diese Ge- genmaßnahmen führen nicht nur dazu, dass die gewonnenen Erkennt- nisse unvollständig sind, sondern erschweren auch Angriffe auf das Botnetz selber (sogenannte Takedown-Angriffe). Der Großteil der ver- wandten Arbeiten im Feld der Botnetz-Überwachung versucht diese von Botnetzen initierten Gegenmaßnahmen beispielsweise über ein langsameres Crawling zu umgehen. Allerdings führen solche Ansätze aufgrund der höheren Verzögerungen im Crawling auch zu Rauschen in den gewonnenen Daten, was wiederum deren Qualität reduziert. Diese Dissertation adressiert die meisten der oben genannten Her- ausforderungen in der Überwachung von Botnetzen. Im Detail wer- den Gegenmaßnahmen in