Network Intrusion Detection: Monitoring, Simulation and Visualization
Total Page:16
File Type:pdf, Size:1020Kb
University of Central Florida STARS Electronic Theses and Dissertations, 2004-2019 2005 Network Intrusion Detection: Monitoring, Simulation And Visualization Mian Zhou University of Central Florida Part of the Computer Sciences Commons, and the Engineering Commons Find similar works at: https://stars.library.ucf.edu/etd University of Central Florida Libraries http://library.ucf.edu This Doctoral Dissertation (Open Access) is brought to you for free and open access by STARS. It has been accepted for inclusion in Electronic Theses and Dissertations, 2004-2019 by an authorized administrator of STARS. For more information, please contact [email protected]. STARS Citation Zhou, Mian, "Network Intrusion Detection: Monitoring, Simulation And Visualization" (2005). Electronic Theses and Dissertations, 2004-2019. 520. https://stars.library.ucf.edu/etd/520 Network Intrusion Detection: Monitoring, Simulation and Visualization by Mian Zhou B.E. Beijing University, 1998 M.S. University of Central Florida, 2001 A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the School of Computer Science in the College of Engineering and Computer Science at the University of Central Florida Orlando, Florida Summer Term 2005 Major Professor: Sheau-Dong Lang c 2005 by Mian Zhou Abstract This dissertation presents our work on network intrusion detection and intrusion sim- ulation. The work in intrusion detection consists of two different network anomaly-based approaches. The work in intrusion simulation introduces a model using explicit traffic gen- eration for the packet level traffic simulation. The process of anomaly detection is to first build profiles for the normal network activity and then mark any events or activities that deviate from the normal profiles as suspicious. Based on the different schemes of creating the normal activity profiles, we introduce two approaches for intrusion detection. The first one is a frequency-based approach which creates a normal frequency profile based on the periodical patterns existed in the time-series formed by the traffic. It aims at those attacks that are conducted by running pre-written scripts, which automate the process of attempting connections to various ports or sending packets with fabricated payloads, etc. The second approach builds the normal profile based on variations of connection-based behavior of each single computer. The deviations resulted from each individual computer are carried out by a weight assignment scheme and further used to build a weighted link graph representing the overall traffic abnormalities. The functionality of this system is of a distributed personal IDS system that also provides a iii centralized traffic analysis by graphical visualization. It provides a finer control over the internal network by focusing on connection-based behavior of each single computer. For network intrusion simulation, we explore an alternative method for network traffic simulation using explicit traffic generation. In particular, we build a model to replay the standard DARPA traffic data or the traffic data captured from a real environment. The replayed traffic data is mixed with the attacks, such as DOS and Probe attack, which can create apparent abnormal traffic flow patterns. With the explicit traffic generation, every packet that has ever been sent by the victim and attacker is formed in the simulation model and travels around strictly following the criteria of time and path that extracted from the real scenario. Thus, the model provides a promising aid in the study of intrusion detection techniques. iv Dedicated to my parents. v Acknowledgments My deepest thanks to my advisor Dr. Sheau-Dong Lang for his guidance, encouragement and support, I am very fortunate to have an opportunity to work with him during my long Ph.D. time. During my research, I have benefited from interactions with many people. I would like to thank Dr. Morgan Wang for many illuminating discussions. Most of all, I am deeply grateful to my parents for supporting my study for over twenty years. Also, I would like to thank my husband Jiangjian Xiao, who gives his endless support and encouragement and I would not have come this far if without him. vi Table of Contents LISTOFTABLES ................................... xi LISTOFFIGURES .................................. xii CHAPTER 1 INTRODUCTION ......................... 1 1.1 PROBLEMSTATEMENT............................ 3 1.2 OUR APPROACH FOR NETWORK INTRUSION DETECTION AND SIM- ULATION..................................... 4 1.3 DISSERTATIONOVERVIEW.......................... 6 CHAPTER 2 SECURITY OF THE INTERNET ............... 7 2.1 PENETRATION FROM THE PERSPECTIVE OF ATTACKERS . 7 2.2 ATTACKTAXONOMY ............................. 9 2.3 INTRUSION DETECTION BACKGROUND AND TAXONOMY . 17 2.3.1 DataCollection .............................. 19 CHAPTER 3 RELATED WORK ......................... 22 3.1 INTRUSIONDETECTION ........................... 22 vii 3.1.1 Host-based Intrusion Detection . 23 3.1.2 Attack Type Specific Intrusion Detection . 25 3.1.3 Network-based Intrusion Detection . 32 3.1.4 Data Mining Approach for Network Intrusion Detection . ..... 34 3.2 INTRUSION TRAFFIC SIMULATION AND MODELING . 37 3.2.1 Simulators................................. 40 CHAPTER 4 MINING NETWORK TRAFFIC FOR INTRUSION PAT- TERNS .......................................... 41 4.1 IMPLEMENTATIONOFNITA. 42 4.2 A FREQUENCY-BASED APPROACH FOR INTRUSION DETECTION . 45 4.2.1 MotivationandGoal ........................... 46 4.2.2 Frequency Analysis of Time-series . 48 4.2.3 ResultsonSyntheticData . 55 4.2.4 Summary ................................. 68 4.3 MORE EXPERIMENTAL RESULTS BY NITA . 69 4.3.1 PortSweep................................. 69 4.3.2 Unidentified Attack from Real Data . 72 4.3.3 GuessTelnetAttack............................ 73 viii CHAPTER 5 WEIGHTED LINK GRAPH: A DISTRIBUTED PERSONAL IDS FOR LAN ...................................... 76 5.1 MOTIVATION .................................. 77 5.2 RELATEDWORK................................ 79 5.3 CONNECTION-BASED BEHAVIOR ANALYSIS . 81 5.3.1 WeightAssignment............................ 83 5.4 WEIGHTEDLINKGRAPH........................... 89 5.4.1 SystemFunctionality ........................... 89 5.4.2 GraphConstruction ........................... 90 5.4.3 GraphDrawing .............................. 93 5.4.4 WeightVectors .............................. 95 5.4.5 IntrusionPatterns ............................ 101 5.4.6 Possible Responses for Overweight . 103 5.5 WHY WEIGHT CHANGES . 104 5.6 EXPERIMENTALRESULTS .......................... 106 5.6.1 TheSizeofWorkingSets. 106 5.6.2 SwappingRate .............................. 109 5.6.3 StealthyIntrusions ............................ 113 5.7 CONCLUSIONS ................................. 115 ix CHAPTER 6 SIMULATION OF NETWORK INTRUSION ........ 117 6.1 SIMULATIONOFINTRUSIONBYOPNET . 119 6.1.1 SimulationModelsUsingOPNET. 120 6.1.2 Analysis of OPNET Simulation Results . 125 6.2 CONCLUSIONS ................................. 130 CHAPTER 7 CONCLUSIONS ........................... 132 7.1 SUMMARIES................................... 132 7.2 LIMITATIONS AND FUTURE WORK . 134 REFERENCES ...................................... 136 x LIST OF TABLES 4.1 DOS and Probe Attacks in 1999 week 5 and week 4 DARPA evaluation data. 64 5.1 The connections with internal IP address collected in a week......... 85 5.2 The connection observation between a desktop and an internal server: long- wood.cs.ucf.edu .................................. 86 5.3 The algorithm for building G ........................... 92 5.4 The algorithm for determine the size of screen set S .............. 98 5.5 The components of a feature vector within a screen unit . ........ 98 5.6 The session collected for bootStrap service for internal server: *.*.107.109 . 100 5.7 The eight statistical features used for calculating W3 for UDP services. 116 5.8 Column 2 is the Mahalanobis distance for traffic to a DNS server, column 3 is the Mahalanobis distance for Netbios Name services traffic. The first 10 sessions are normal ones while the last two are from a port sweep. 116 xi LIST OF FIGURES 2.1 DDOSattackandRDDOSattack . 13 2.2 Intrusion detection system taxonomy . 17 4.1 Themenu-basedinterfaceoftheNITA . 42 4.2 The visualization results from the NITA . 43 4.3 The feature selection panel of the NITA . 44 4.4 Overall Structure of frequency-based detection . ......... 50 4.5 Va, Vb are the heights of valleys, W is width of peak, P is the height of peak, N is the area between Va and Vb covered by the peak. 55 4.6 The frequency pattern on inter-arrival time of each connection for Proc- cessTableattack. ................................. 58 4.7 The frequency pattern of Ping of death. (a) Frequency on inter-arrival time. (b) Frequency on packets0.005 seconds. 59 4.8 (a) The local frequency pattern on inter-arrival-time for the first 9 connections of portsweep attack. (b) The global frequency pattern on inter-arrival time of portsweep attack, which includes 18 connections. ..... 59 xii 4.9 The frequency pattern on inter-arrival time of each connection for Dictionary attack........................................ 60 4.10 Frequency analysis on packet rates for sshprocessTable attack ........ 62 4.11 The frequency pattern on inter-arrive time of each connection for sshProc- cessTable attack. Connection 2 is the attack traffic. ..... 63 4.12 The intrusion patterns analysis by NITA for portsweep attack, data source: Monday,L1999w5, the duration of attack is about