Using Internal Sensors for Computer Intrusion Detection
Total Page:16
File Type:pdf, Size:1020Kb
USING INTERNAL SENSORS FOR COMPUTER INTRUSION DETECTION A Thesis Submitted to the Faculty of Purdue University by Diego Zamboni CERIAS TR 2001-42 Center for Education and Research in Information Assurance and Security, Purdue University August 2001 USING INTERNAL SENSORS FOR COMPUTER INTRUSION DETECTION A Thesis Submitted to the Faculty of Purdue University by Diego Zamboni In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy August 2001 ii To my parents for giving me life, and to Susana for sharing it with me. iii ACKNOWLEDGMENTS As usual, a large number of people were very important for the completion of this thesis work, and I would like to acknowledge at least some of them. First the official acknowledgments: Portions of the research contributing to this dis- sertation were supported by the various sponsors of CERIAS, and my stay at Purdue was partially funded by a Fulbright/Conacyt fellowship. Their support is gratefully acknowl- edged. I would like to thank my advisor, Eugene Spafford. He received me with open arms from my first day at Purdue, and provided continuous guidance and support throughout my stay here. For that I am very, very grateful. I would like to acknowledge the other members of my Ph.D. committee: Stephanie For- rest, Mikhail Atallah, Jens Palsberg and Carla Brodley. They provided invaluable feedback and advice for my work and for this dissertation. Many people contributed significantly to my work, and my deepest gratitude goes to all of them. Susana Soriano, Benjamin Kuperman, Thomas Daniels, Florian Kerschbaum, Rajeev Gopalakrishna and Jim Early provided me with many hours of discussion, continu- ously questioned my assumptions, and led to many new ideas. The original ideas for how internal sensors would be implemented evolved from discussions with Ben Kuperman, and he also came up with the name “ESP”. Florian Kerschbaum poured enormous amounts of work into implementing and testing detectors, and Jim Early implemented the file integrity detector. Angel Soriano guided me through the statistical analysis of the experimental results, and suggested multiple avenues for future research. The staff at the Statistical Con- sulting Service at Purdue also provided invaluable guidance in the analysis of data. Other iv students at CERIAS provided support for my work: Sofie Nystrom (who let me use her office), Kevin Du, Hoi Chang, Chapman Flack, Chris Telfer, and many others. Substantial administrative, technical and logistic support were needed for the comple- tion of my work. The system administrators at CERIAS, Susana Soriano and Vince Koser, always maintained our computers up and running and kept up with my continuous requests and questions, even at times when what they really wanted to do was remove my account and get rid of me. All the administrative personnel at CERIAS, including Mary Jo Maslin, Lori Floyd, Paula Cheatham, Steve Hare and Andra Boehning, always gave me their sup- port. In particular, I would like to thank Marlene Walls, who tirelessly stayed on top of things at CERIAS to make sure everything was going as it should. Without her, my life (and the scheduling for seeing my advisor) would have been infinitely more complicated. Before and throughout my stay at Purdue there were people who contributed to my ca- reer by inspiring, supporting, and challenging me. These include Gerardo Cisneros (who may not know it, but he was my main inspiration for pursuing a Ph.D.); my friends Rey- naldo Roel, Carlos Gonzalez,´ Claudia Fajardo, Agust´ın and Adriana Casimiro, Luis and Claudia Graf, Luis and Carmen Teresa Mart´ınez and Eduardo Asbun, all of whom gave me so much support and friendship; Ivan Krsul, Christoph Schuba, Tanya Mastin, Kathy Price, Keith Watson and Robin Sundaram, who gave me a great welcome to the COAST laboratory and helped me through my first years at Purdue. Thank you all. Who I am is a result of my formation, and for that I have to thank my family. My parents, Laura Zamboni and Gilberto De La Rosa, and my sisters, Ana, Daniela and Ines,´ have been an inexhaustible source of inspiration, support, advice, and happiness. And finally, but most importantly, I would like to thank my best friend and wife, Susana Soriano. She lifted me at times when I thought I could not keep going, and her support and care made it possible for me to start and finish this endeavor. She listened patiently to my ideas and tolerated me being locked up in my office 20 hours a day (although she always found ways of spending some time together!), and on top of that, she managed to expertly proofread my dissertation. Susana: you are the love of my life, and words cannot express how much I need to thank you. v TABLE OF CONTENTS Page LIST OF TABLES :::::::::::::::::::::::::::::::::: ix LIST OF FIGURES ::::::::::::::::::::::::::::::::: xi LIST OF DEFINITIONS :::::::::::::::::::::::::::::: xiii ABSTRACT ::::::::::::::::::::::::::::::::::::: xiv 1 INTRODUCTION :::::::::::::::::::::::::::::::: 1 1.1 Background and problem statement :::::::::::::::::::: 1 1.2 Basic concepts ::::::::::::::::::::::::::::::: 2 1.2.1 Intrusion detection ::::::::::::::::::::::::: 2 1.2.2 Desirable characteristics of an intrusion detection system ::::: 3 1.3 Problems with existing intrusion detection systems :::::::::::: 5 1.4 Thesis statement :::::::::::::::::::::::::::::: 5 1.5 Document organization :::::::::::::::::::::::::: 6 2 RELATED WORK AND ARCHITECTURES FOR INTRUSION DETECTION 7 2.1 Data collection architectures :::::::::::::::::::::::: 7 2.1.1 Data collection structure: centralized and distributed ::::::: 10 2.1.2 Data collection mechanisms: direct and indirect monitoring :::: 11 2.1.3 Data collection mechanisms: host-based and network-based ::: 14 2.1.4 Data collection mechanisms: external and internal sensors :::: 16 2.2 Data analysis architectures ::::::::::::::::::::::::: 21 2.3 Experiences in building a distributed intrusion detection system ::::: 22 2.4 Comments about intrusion detection architectures ::::::::::::: 25 2.5 Related work :::::::::::::::::::::::::::::::: 26 vi Page 3 AN ARCHITECTURE FOR INTRUSION DETECTION BASED ON INTER- NAL SENSORS ::::::::::::::::::::::::::::::::: 29 3.1 Embedded detectors :::::::::::::::::::::::::::: 29 3.1.1 How embedded detectors work :::::::::::::::::: 30 3.1.2 Relationship between internal sensors and embedded detectors :: 31 3.1.3 Stateless and stateful detectors ::::::::::::::::::: 33 3.1.4 Strengths and weaknesses of embedded detectors ::::::::: 34 3.2 The ESP architecture :::::::::::::::::::::::::::: 35 3.2.1 Internal sensors and embedded detectors ::::::::::::: 35 3.2.2 Per-host external sensors :::::::::::::::::::::: 35 3.2.3 Network-wide external sensors :::::::::::::::::: 35 3.3 Distinguishing characteristics of the ESP architecture ::::::::::: 36 3.3.1 Types of data observed ::::::::::::::::::::::: 36 3.3.2 Tighter coupling between event collection and event analysis ::: 37 3.3.3 Intrusion detection at the application and operating system level : 38 3.3.4 Size of the intrusion detection system ::::::::::::::: 38 3.3.5 Timeliness of detection :::::::::::::::::::::: 39 3.3.6 Impact on the host ::::::::::::::::::::::::: 39 3.3.7 Resistance to attack :::::::::::::::::::::::: 39 4 THE ESP IMPLEMENTATION ::::::::::::::::::::::::: 41 4.1 Purpose of the implementation ::::::::::::::::::::::: 41 4.2 Specific and generic detectors ::::::::::::::::::::::: 41 4.3 Sources of information ::::::::::::::::::::::::::: 42 4.4 Implementation platform :::::::::::::::::::::::::: 43 4.5 Reporting mechanism ::::::::::::::::::::::::::: 44 4.6 Methodology for implementation of detectors ::::::::::::::: 46 4.7 Applicability of CVE entries :::::::::::::::::::::::: 47 4.8 Design and implementation considerations for detectors :::::::::: 48 vii Page 4.9 Naming, testing and measuring detectors ::::::::::::::::: 49 4.10 Relationships between detectors :::::::::::::::::::::: 50 4.11 Recording information about sensors and detectors :::::::::::: 51 4.12 Case studies :::::::::::::::::::::::::::::::: 53 4.12.1 Embedded detectors for network-based attacks :::::::::: 53 4.12.2 Embedded detectors for sendmail attacks ::::::::::::: 60 4.13 Detectors implemented ::::::::::::::::::::::::::: 65 4.13.1 By vulnerable platform or program :::::::::::::::: 65 4.13.2 By implementation directory ::::::::::::::::::: 67 4.13.3 By size ::::::::::::::::::::::::::::::: 67 4.13.4 By type :::::::::::::::::::::::::::::: 70 4.13.5 By data sources used :::::::::::::::::::::::: 72 4.13.6 By vulnerability type ::::::::::::::::::::::: 74 4.13.7 By detection and implementation rates :::::::::::::: 77 4.14 Auxiliary components ::::::::::::::::::::::::::: 77 4.15 Comments about the ESP implementation ::::::::::::::::: 80 5 TESTING THE ESP IMPLEMENTATION ::::::::::::::::::: 82 5.1 Performance testing :::::::::::::::::::::::::::: 82 5.1.1 Test design and methodology ::::::::::::::::::: 82 5.1.2 Results of the NetPerf test ::::::::::::::::::::: 85 5.1.3 Results of the http load test :::::::::::::::::::: 88 5.1.4 Comparison and comments about the tests ::::::::::::: 88 5.2 Detection testing :::::::::::::::::::::::::::::: 93 5.2.1 Test design and methodology ::::::::::::::::::: 93 5.2.2 Results from the detection test ::::::::::::::::::: 97 5.2.3 Comments about the detection test ::::::::::::::::: 109 6 CONCLUSIONS, SUMMARY AND FUTURE WORK ::::::::::::: 112 6.1 Conclusions :::::::::::::::::::::::::::::::: 112 viii Page 6.2 Summary of main contributions :::::::::::::::::::::: 114 6.3 Future work :::::::::::::::::::::::::::::::: 115