Table of Contents List of Figures ..................................................................................................................... 4 List of Tables ...................................................................................................................... 5 Chapter 1: Introduction....................................................................................................... 6 1.1 Introduction....................................................................................................................6 1.2 Problem Statement .......................................................................................................12 1.3 Thesis Objective...........................................................................................................12 1.4 Thesis Organization.....................................................................................................14 Chapter 2: Intrusion Detection.......................................................................................... 15 2.1 Introduction..................................................................................................................15 2.2 What is an IDS .............................................................................................................15 2.2.1 The Basic Concepts of Intrusion Detection......................................................16 2.2.2 A Generic Intrusion-Detection System.............................................................17 2.2.3 Characteristics of Intrusion Detection Systems ................................................18 2.2.4 Efficiency of intrusion detection systems .........................................................19 2.3 Classification of IDS ....................................................................................................20 2.3.1 Classification based on the detection method...................................................20 2.3.1.1 Misuse Detection or detection by appearance. .....................................20 2.3.1.2 Anomaly Detection or detection by behavior. ......................................21 2.3.2 Classification based on the type of the protected system .................................21 2.3.2.1 Host based systems ...............................................................................22 2.3.2.2 Network based systems .........................................................................22 2.4 Different Approaches to anomaly host-based IDS ......................................................22 2.4.1 Analysis of sequences of system calls ..............................................................22 2.4.2 Detection through the use of Neural Networks.................................................23 2.4.3 Detection through the use of Data Mining........................................................25 2.5 Conclusion...................................................................................................................26 Chapter 3: Related Work................................................................................................... 28 3.1 Introduction..................................................................................................................28 3.2 Detecting attacks through source code analysis...........................................................29 1 3.2.1 Attacks exploiting buffer-overflow systems vulnerability ...............................30 3.2.2 Attacks increasing system privileges ................................................................30 3.3 Statistical-based anomaly detection techniques...........................................................32 3.3.1 Statistical process control .................................................................................33 3.3.2 Multivariate statistical techniques in intrusion detection..................................36 3.3.3 Comparison between the different Statistical techniques .................................40 3.4 Conclusion.................................................................................................................. 41 Chapter 4: Development of an Anomaly Host Based IDS ............................................... 42 4.1 Introduction..................................................................................................................42 4.2 Malicious code study...................................................................................................43 4.2.1 Data Collection.................................................................................................44 4.2.2 Attack pattern analysis......................................................................................45 4.3 Pattern Identification (feature extractor)......................................................................50 4.3.1 Application of Principal Component Analysis to the feature extraction problem ....................................................................................................................................50 4.3.2 Definition of Principal Component Analysis....................................................51 4.3.3 Steps to perform PCA in feature extraction......................................................52 4.4 Attack Classifier...........................................................................................................57 4.5 Results Analysis & Validation.....................................................................................59 4.5.1 Efficiency and performance measures ..............................................................59 4.5.2 Experiments for training and testing.................................................................60 4.6 Conclusion...................................................................................................................61 Chapter 5: Results Analysis and Validation......................................................................63 5.1 Introduction..................................................................................................................63 5.2 Input Data Analysis......................................................................................................63 5.3 Parser design and operation.........................................................................................65 5.4 Multivariate Statistical Analysis: PCA........................................................................66 5.4.1 Data Normalization...........................................................................................66 5.4.2 The Computation of the Covariance Matrix (or the Correlation matrix)..........71 5.4.3 Eigenvectors and Eigenvalues ..........................................................................75 5.4.4 Selection of Principal Components...................................................................76 2 5.5 Testing & Results.........................................................................................................78 5.5.1 Experiment with known data ............................................................................80 5.5.2 Cross validation over unseen data.....................................................................83 Chapter 6: Conclusion and Future Work .......................................................................... 86 6.1 Research Summary......................................................................................................86 6.2 Contributions................................................................................................................88 6.3 Future work..................................................................................................................89 References......................................................................................................................... 91 Appendix A....................................................................................................................... 97 Appendix B..................................................................................................................... 111 Appendix C ..................................................................................................................... 119 Appendix D..................................................................................................................... 126 3 List of Figures Figure 1. 1: Attack sophistication versus intruder technical knowledge. ............................8 Figure 2. 1: A simple Intrusion Detection System…………………………….................17 Figure 2. 2: Characteristics of Intrusion Detection Systems..............................................18 Figure 3. 1: The System Architecture of Detecting Source Code of Attacks. ...................31 Figure 3. 2: Differences between the control limits of separate Univariate tests and the control limit of a Multivariate test. ....................................................................................35 Figure 4. 1: The Life Cycle of Attack Codes.....................................................................47 Figure 4. 2: Elements of Primary filter ..............................................................................54 Figure 4. 3: Elements sorted according to their highest Total Variance Ratio (R)............55 Figure 4. 4: The Computation of the Euclidean Distance from the center of both clean and malicious codes..................................................................................................................58