2020 International Conference on Computational Science and Computational Intelligence (CSCI) Network Intrusion Detection with XGBoost and Deep Learning Algorithms: An Evaluation Study Amr Attia Miad Faezipour Abdelshakour Abuzneid Computer Science & Engineering Computer Science & Engineering Computer Science & Engineering University of Bridgeport, CT 06604, USA University of Bridgeport, CT 06604, USA University of Bridgeport, CT 06604, USA [email protected] [email protected] [email protected] Abstract— This paper introduces an effective Network Intrusion In the KitNET model introduced in [2], an unsupervised Detection Systems (NIDS) framework that deploys incremental technique is introduced for anomaly-based intrusion statistical damping features of the packets along with state-of- detection. Incremental statistical feature extraction of the the-art machine/deep learning algorithms to detect malicious packets is passed through ensembles of autoencoders with a patterns. A comprehensive evaluation study is conducted predefined threshold. The model calculates the Root Mean between eXtreme Gradient Boosting (XGBoost) and Artificial Neural Networks (ANN) where feature selection and/or feature Square (RMS) error to detect anomaly behavior. The higher dimensionality reduction techniques such as Principal the calculated RMS at the output, the higher probability of Component Analysis (PCA) and Linear Discriminant Analysis suspicious activity. (LDA) are also integrated into the models to decrease the system Supervised learning has achieved very decent results with complexity for achieving fast responses. Several experimental algorithms such as Random Forest, ZeroR, J48, AdaBoost, runs confirm how powerful machine/deep learning algorithms Logit Boost, and Multilayer Perceptron [3]. Machine/deep are for intrusion detection on known attacks when combined learning-based algorithms for NIDS have been extensively with the appropriate features extracted. To investigate unknown attacks, the models were trained on a subset of the studied in the literature. Some models manage imbalanced attack datasets, while a different set (with a different attack datasets [4, 5], while others mainly focus on dimensionality type) was kept aside for testing. The decent results achieved reduction techniques implemented using Principal further support the belief that through supervised learning, the Component Analysis PCA [6], autoencoders [5], sparse model could additionally detect unknown attacks. autoencoders in conjunction with well-known classifiers such Keywords- NIDS; Machine Learning; ANN; XGBoost; LDA; as Random Forest [7]. The mentioned techniques have PCA. mostly been applied to the CICIDS2017 dataset [8]. On the other hand, very few evaluations have been carried out on the I. INTRODUCTION Kitsune family dataset [2] for NIDS. A. Background C. Contribution The criticality of intrusion detection has been increasing Reliable and effective NIDS are highly dependable on significantly, especially in the era of big data where a huge accurate and fast detection of the attacks. In this regard, amount of information is continuously transferred at high- creating less complex models while achieving 100% speed data rates. Moreover, the COVID-19 pandemic has detection is highly desirable [9]. This paper introduces a drastically increased the urgent need of transferring data framework which deploys popular machine learning digitally and providing almost all possible workflows online. algorithms such as eXtreme Gradient Boosting (XGBoost) Such surge also raises the pressing need for more secure and deep learning models such as Artificial Neural Networks Internet usage. (ANN) implemented solely, or along with feature selection It is very important to devise intelligent network intrusion techniques such as Principal Component Analysis (PCA) and detection systems (NIDS) using state-of-the-art technology. Linear Discriminant Analysis (LDA) to achieve the lowest There are two types of network intrusion detection systems: complexity possible while maintaining high-performance i) signature-based and ii) anomaly-based detection where intrusion detection results. What makes both algorithms machine learning is widely deployed [1]. extremely powerful and achieve very decent results is that B. Related Work they are applied to incremental statistical features. Accordingly, the framework proposed here could learn the Applying effective machine/deep learning in network patterns more efficiently and detect potentially known and intrusion detection systems has become increasingly popular unknown attacks. We managed to achieve great results by and significantly crucial due to the rising demand for Internet applying this concept. Initially, we applied XGBoost and deployment in every aspect of our lives. For this purpose, deep learning on Kitsune family datasets [2] one by one. We NIDS engineers need to come up with model(s) that could then, merged all 9 datasets in one very large dataset which efficiently protect and detect all the known attacks as well as had more than 21 million instances, to test the efficiency of unknown attacks that may not be previously known. detecting various attacks using the same trained model. Then, 978-1-7281-7624-6/20/$31.00 ©2020 IEEE 138 DOI 10.1109/CSCI51800.2020.00031 we considered the extreme case in supervised learning by building a model trained on 8 datasets and keeping one aside Input Data (Packet) (considered as the unknown attacks) for testing to study the effectiveness of the model for detecting unknown attacks via Incremental Statistical Features capturing and learning the common patterns of the attacks. This also gives an idea to some extent as to how different attacks share similar characteristics. This allowed us to Feature Selection Dimension Reduction develop a model that can detect known attacks and other unknown attacks using supervised learning techniques rather Classifiers (XGBoost / ANN) than unsupervised learning. Figure 1. Proposed Framework for NIDS II. METHODOLOGY AND PROCEDURE TABLE I. CHARACTERISTIC SUMMARY OF KITSUNE FAMILY DATASET Attack # of True Negative A. Proposed Idea Attack Name Dataset size Type Features Percentage In this paper, we introduce a framework with XGBoost as Botnet 84.08%: Mirai 764,136 115 a classifier applied to incremental statistical damping features Malware Imbalanced of the intrusion data and compare its performance with ANN. 4.19%: Denial of SSL 2,207,570 115 Heavily Different dimensionality reduction techniques are applied to Service Renegotiation reduce the complexity of the model and compared to the Imbalanced Denial of 35.31%: SSDP Flood 4,077,265 115 specific selection of the features by XGBoost. The proposed Service Semi-balanced hybrid use of machine learning algorithms with incremental 0.25% Denial of statistical features for NIDS has not been investigated earlier SYN DoS 2,771,275 115 Extremely Service in the literature on the Kitsune family dataset. Figure 1 Imbalanced 3.87%: presents the flow chart of our machine learning model for Recon OS_SCAN 1,697,850 115 Heavily NIDS. Imbalanced Man in the 45.73% The rationale behind using the selected machine learning, ARP MitM 2,504,266 115 dimensionality reduction, and/or classifier techniques is Middle Semi-balanced 4.145% noticeable from the details presented hereafter: Man in the Video Injection 2,472,400 115 Heavily Middle 1) Principal Component Analysis is used to reduce the Imbalanced dimensions of the features by searching for the orthogonal Man in the 40.5% Active Wiretap 2,278,688 115 vectors that carry the most important information of the Middle Semi-balanced original features [10, 11]. 19.285% Recon Fuzzing 2,244,138 115 2) Linear Discriminant Analysis is a supervised algorithm Imbalanced also used as a dimension reduction technique. It searches for 23.08% Total 21,017,588 115 the direction of maximum discriminability in the space [12, Imbalanced 13]. 3) XGBoost (eXtreme Gradient Boosting) is a machine learning system using tree boosting algorithms. We III. RESULTS AND DISCUSSION implemented XGBoost for both feature selection and also as A. Experimental Setup a classifier [14]. In the proposed NIDS framework, the applied hybrid 4) Artificial Neural Network is applied as a binary classifier algorithms are implemented using PCA and LDA as using supervised training. A neural network, in general, dimension reduction techniques, followed by ANN and consists of input, hidden, and output layers and is a very XGBoost as classifiers. Feature selection using XGBoost is powerful tool for pattern classification [15]. In this work, we alternatively used to lower the complexity of the model and applied different structures starting from deep learning neural achieve fast detection responses with high accuracies. networks with 3 layers and above and also implemented We have applied and tested more than 250 different simple neural networks using only one layer. models on the 9 different datasets. For illustration, we mostly B. Kitsune Dataset demonstrate the models that achieved accuracy higher than The dataset family employed in this paper is cited in [2] 99.9%. However, for the merged datasets and the trials for and could be accessed publicly. It was created via a real IP the unknown attacks, we demonstrate most of the trials due camera video surveillance network. Features were extracted to the challenges of experimenting and detecting unknown by applying incremental statistics to capture the behavior of attacks through supervised