Performance Testing an Inline Network Intrusion Detection System Using Snort
Total Page:16
File Type:pdf, Size:1020Kb
PERFORMANCE TESTING AN INLINE NETWORK INTRUSION DETECTION SYSTEM USING SNORT A Thesis Presented to the Faculty of the College of Science and Technology Morehead State University In Partial Fulfillment of the Requirements for the Degree Master of Science by Richard Wagoner November 16, 2007 Accepted by the faculty of the College of Science and Technology, Morehead State University, in partial fulfillment of the requirements for the Master of Science degree. Master's Committee: PERFORMANCE TESTING AN INLINE NETWORK INTRUSION DETECTION SYSTEM USING SNORT Rick Wagoner, M.S. Morehead State University, 2007 Network Security is fast becoming an absolute necessity to protect the information contained on computer systems worldwide. The ever-changing scope of network use and operation along with the public concern for protection of sensitive information makes implementing an efficient and effective security plan a must. Part of such a plan.includes an approach of security in depth. This includes many different pieces of software and hardware working together to provide the desired protection. One component of this approach is that of network intrusion detection systems and for the purposes of this research, the Snort intrusion detection system will be used. However, to ensure that such an implementation is likely to be successful the system must be tested prior to implementation to provide decision makers with assurance to reduce risks. Making a network intrusion detection system work effectively should not degrade overall network performance, so some type of performance measurement plan must be designed and implemented. Such a plan is the topic of this research. The effect of using an inline intrusion detection system, measured by end-to-end delay introduced by the engine, will be discussed as a method of performance testing intrusion detection systems at a basic level to ensure network traffic is unaffected. This will provide some information as to whether the intrusion detection system is capable of efficiently detecting intrusions while maintaining network performance. Accepted by: CHAPTER 1................................................................................................................. 1 INTRODUCTION .................................................................................................... 1 STATEMENT OF THE PROBLEM ................................................................. 1 Question 1 ....................................................................................................... 2 Question 2 ....................................................................................................... 2 Question 3 ....................................................................................................... 2 IMPORTANCE OF THE STUDY ..................................................................... 3 ASSUMPTIONS ................................................................................................. 4 Assumption 1 .................................................................................................. 5 Assumption 2 .................................................................................................. 5 Assumption 3 ........................................................................................... ,...... 5 LIMITATIONS ...................................................................................................... 5 Limitation 1...................................................................................................... 6 Limitation 2 ...................................................................................................... 6 Limitation 3 ...................................................................................................... 6 DEFINITION OF TERMS .................................................................................. 7 CHAPTER 11 .............................................................................................................. 1O REVIEW OF LITERATURE ................................................................................ 10 HISTORY OF NETWORKING ....................................................................... 10 Network Growth .•.............••........................................................................... 10 Network Security Concerns ........................................................................ 11 Security In-Depth. ......................................................................................... 12 NETWORK SECURITY INTRUSION SYSTEMS ....................................... 14 Intrusion Detection Systems....................................................................... 14 Intrusion Prevention Systems .................................................................... 18 SNORT INTRUSION DETECTION SYSTEM .............................................. 19 Snort Capabilities ......................................................................................... 20 Snort Pre-processors ................................................................................... 22 Advantages and Disadvantages of Snort................................................. 25 CURRENT RESEARCH PROJECTS ...........................................................' 26 Network Security .......................................................................................... 26 Intrusion Detection ....................................................................................... 27 Performance Measurement ........................................................................ 28 CHAPTER 111 ............................................................................................................. 30 METHODOLOGY ................................................................................................. 30 POPULATION AND SAMPLE SELECTION ............................................... 30 TESTING ENVIRONMENT............................................................................. 31 DESIGN OF EXPERIMENT ........................................................................... 34 METHOD OF MEASUREMENT .................................................................... 34 APPLICATION TO RESEARCH QUESTIONS ........................................... 37 CHAPTER IV ....................................................................................................... i••.. 38 FINDINGS AND ANALYSIS OF DATA ...........................................................' 38 I DATA COLLECTED ........................................................................................ 38 METHOD OF ANALYSIS ............................................................................... 38 CHAPTER V ............................................................................................................. 48 CONCLUSIONS ................................................................................................... 48 SUMMARY OF FINDINGS ............................................................................. 48 CONCLUSIONS MADE .................................................................................. 49 FUTURE EXTENSIONS TO RESEARCH ................................................... 53 REFERENCES ......................................................................................................... 55 APPENDIX A ............................................................................................................ 57 APPENDIX B ............................................................................................................ 61 APPENDIX C ............................................................................................................ 66 APPENDIX D ............................................................................................................ 86 Acknowledgement I would like to acknowledge and thank those individuals who provided assistance and support in completing this thesis. First, I would like to thank my thesis committee: Dr. Ahmed Zargari who provided general guidance and direction in preparing and completing my thesis, Dr. Gregory Russell wh9 assisted with detailed guidance on my statistical analysis methodologies! and ' Dr. Xiaolong Li who helped me clarify my thesis topic. I would also like to thank my wife, Callie, and son, Zack, who supported me when I had to spend long hours away from home working in the lab. Both were extremely understanding making my work much easier to complete. 1 CHAPTER I INTRODUCTION Network intrusion detection is becoming a key element in the total security approach to network protection. With the use of network intrusion detection systems on the increase and the growth in high volume, high-speed I networks, network administrators need to ensure that network traffic is not I being unduly delayed by overhead introduced from the inline network intrusion detection system. Network administrators do not want to jeopardize network security or add unneeded overhead to the already extremely busy networks by introducing an inline network detection system. STATEMENT OF THE PROBLEM There have been many studies done to test the performance of both network and host intrusion detection systems. However, in most of these instances the performance was based on the ability to detect intrusions and not on the ability of the system to ensure that continued network traffic performance levels are maintained when placed inline. lnline network intrusion detection systems obviously