An Interactive Web Based Platform for Modeling and Analysis of Large Scale Argus Network Flow Data
Total Page:16
File Type:pdf, Size:1020Kb
University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange Masters Theses Graduate School 8-2017 InSight2: An Interactive Web Based Platform for Modeling and Analysis of Large Scale Argus Network Flow Data Hansaka Angel Dias Edirisinghe Kodituwakku University of Tennessee, Knoxville, [email protected] Follow this and additional works at: https://trace.tennessee.edu/utk_gradthes Recommended Citation Kodituwakku, Hansaka Angel Dias Edirisinghe, "InSight2: An Interactive Web Based Platform for Modeling and Analysis of Large Scale Argus Network Flow Data. " Master's Thesis, University of Tennessee, 2017. https://trace.tennessee.edu/utk_gradthes/4885 This Thesis is brought to you for free and open access by the Graduate School at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in Masters Theses by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. To the Graduate Council: I am submitting herewith a thesis written by Hansaka Angel Dias Edirisinghe Kodituwakku entitled "InSight2: An Interactive Web Based Platform for Modeling and Analysis of Large Scale Argus Network Flow Data." I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the equirr ements for the degree of Master of Science, with a major in Computer Engineering. Jens Gregor, Major Professor We have read this thesis and recommend its acceptance: Mark E. Dean, Audris Mockus Accepted for the Council: Dixie L. Thompson Vice Provost and Dean of the Graduate School (Original signatures are on file with official studentecor r ds.) InSight2: An Interactive Web Based Platform for Modeling and Analysis of Large Scale Argus Network Flow Data A Thesis Presented for the Master of Science Degree The University of Tennessee, Knoxville Hansaka Angel Dias Edirisinghe Kodituwakku August 2017 Copyright © 2017 by Hansaka Angel Dias Edirisinghe Kodituwakku All rights reserved. ii In loving memory of Jovi iii ACKNOWLEDGEMENTS I would like to thank my advisor and mentor, Dr. Jens Gregor for his excellent guidance, infallible support and prospective wisdom throughout the course of my Master‟s program, without which I could not have completed this immense task within this time frame. I learned a great deal from his effectiveness in planning and optimizing. His timely decisions saved me a lot of time and enabled me to streamline my workflow to do more in less time. It was an honor and a joy to work with him and I am looking forward to continue working with him in the coming years. I am thankful for Dr. Mark E. Dean and Dr. Audris Mockus for their time in serving in my thesis committee and their guidance during class work. I would like to extend my sincere gratitude to Mr. Greg S. Cole for his unfailing faith in me and giving me the opportunity to be a part of GLORIAD family. He saw my potential more than me and encouraged me to achieve it even though at the time it seemed impossible to me. His visionary ideas inspired me to achieve more and made me realize that there is no limit for one‟s potential. I would like to thank my parents for their incredible love, sacrifice and support they gave for me throughout my life. I would like to thank my mom who stood up for me when the tides were rough. I learned to always do the right thing and that hard work always pays in the end, from her. All my virtues and good characteristics are inherited from her outstanding parenting and shrewdness. I would like to thank my dad who worked hard to support our family and being my hero. His fearless behavior observed over the years primed my ability to face any circumstance head-on. I would like to thank my sister for her never-ending love and being there all the time for me. She is the best sibling anyone would hope for. I would like to thank Fan Zhang for her support and never giving up on me. Her encouragement and appreciation enabled me to work harder and discover new potentials I never knew I had. I would like to thank Michele Norris for treating me as her own family and being there for me all the time. I would like to thank Dana Bryson for helping me out countless times with paperwork amidst her busy work schedule. I would like to thank Saeed, Seena and Mustafa for being my brothers which I never had in my childhood. iv I would like to thank Jovi and Sasha for giving me their unconditional love. Jovi, you left your paw-prints of joy on our hearts and we will never forget you. You are a warrior and you fought through battles life unfairly bestowed upon you. Sasha you are the sweetest and one of a kind. No wonder why you steal everyone‟s hearts! I would also like to thank The University of Tennessee for letting me be a part of the Big Orange to study the cutting edge technology from the best professors in the world via its Masters of Science program. I would like to thank Laurel Residence for being my home away from home half-way across the globe, for all the great memories and for all the amazing friends I made during my stay. The work presented in this thesis was supported by the National Science Foundation under Grant No. IRNC-1450959. Any opinions, findings, and conclusions or recommendations expressed are those of the author and do not necessarily reflect the views of the National Science Foundation. v ABSTRACT Monitoring systems are paramount to the proactive detection and mitigation of problems in computer networks related to performance and security. Degraded performance and compromised end-nodes can cost computer networks downtime, data loss and reputation. InSight2 is a platform that models, analyzes and visualizes large scale Argus network flow data using up-to-date geographical data, organizational information, and emerging threats. It is engineered to meet the needs of network administrators with flexibility and modularity in mind. Scalability is ensured by devising multi-core processing by implementing robust software architecture. Extendibility is achieved by enabling the end user to enrich flow records using additional user provided databases. Deployment is streamlined by providing an automated installation script. State-of-the-art visualizations are devised and presented in a secure, user friendly web interface giving greater insight about the network to the end user. vi TABLE OF CONTENTS 1. Introduction ...................................................................................................... 1 1.1 Motivation: Need for Network Monitoring and Analytics ............................. 1 1.1.1 GLORIAD: A Research and Education Network .................................. 1 1.1.2 Argus Network Flow Data .................................................................... 3 1.1.3 GLORIAD InSight (2013-2014) ............................................................ 4 1.1.4 InSight2 (2017) .................................................................................... 8 1.2 Thesis Outline .......................................................................................... 13 2. Overview of network Monitoring Software ..................................................... 14 2.1 Network Data Capture Techniques .......................................................... 14 2.1.1 Packet Level Data Capture ................................................................ 14 2.1.2 Flow Level Data Capture ................................................................... 16 2.2 Existing Network Monitoring Solutions ..................................................... 21 2.2.1 Performance Monitoring ..................................................................... 21 2.2.2 Security Monitoring ............................................................................ 23 3. Server Configuration ...................................................................................... 24 3.1 Hardware Configuration ........................................................................... 24 3.2 Argus Archives and Other Data Recovery ................................................ 29 3.2.1 Extracting Argus Archives .................................................................. 29 3.2.2 Data Forensics ................................................................................... 30 3.3 Preventive Measures Taken ..................................................................... 32 3.3.1 Contingency Backups ........................................................................ 32 3.3.2 Hardware and Software Setup ........................................................... 34 4. Description and Assessment of GLORIAD InSight ........................................ 39 4.1 Software Architecture ............................................................................... 39 4.2 Description of „Farm of Animals‟ ............................................................... 44 4.3 Global Science Registry ........................................................................... 53 4.4 Assessment .............................................................................................. 56 vii 4.4.1 Software Architecture......................................................................... 56 4.4.2 „Farm of Animals‟ ............................................................................... 59 4.4.3 Global Science Registry (GSR) ......................................................... 60 4.4.4 Hardware Capabilities .......................................................................