Part I CAPTURE

Security Analysis And Data Visualization Olu Akindeinde October 16, 2009 Copyright © 2009 Olu Akindeinde Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in Appendix B entitled "GNU Free Documentation License". 2 Security Analysis and Data Visualization i PREFACE This is not a book on information security assessment methodologies, neither is it on penetration testing techniques: it is however, about a subtle combination of the core elements of both, but this time employed in the process of interactive security analysis and statistical graphics. Why a book on security analysis and visualization? Well, over the last couple of years, the analysis and visualization concepts have slowly become important components of overall enterprise security assessment albeit devoid of proper acknowledgment. In reality though, both are relatively uncharted territories even in the field of computing. Notwithstanding this surrounding conspiracy of obscurity, this book explores the subject matter in great depth. Security Analysis and Data Visualization primarily concerns itself with the use of perception to recognize and amplify cognitive complexities. This book therefore, sets out to present an interactive and practical approach to these abstractions. In modern business, to effectively compete and excel, organizations must increasingly raise service delivery and operational levels whilst expanding reach. Today’s global economic climate creates considerable obstacles to managing the enterprise. The proliferation of bandwidth and telecommuting via remote access, as well as worldwide emerging markets, are in essence driving the development of a virtual workforce which is widely distributed across a rapidly rising number of remote sites. A network is therefore required to provide the personnel at these locations access to the applications, systems, and data that often reside within the head quarter’s data center. As a result of employee productivity’s dependence on network security and health, statistical network analysis and interactive visualization becomes a pivotal component of the entire organizational security process. The team responsible for maintaining the network security is constantly living with the day-to-day analysis of security data, a fact hitherto lost on most enterprises. Data leakages, log monitoring, fault analysis, packet captures, alerts and even binary files take time and effort to analyze using text-based and non-intuitive tools - and once the analysis is complete, the picture does not always reflect the root cause and the result isn’t necessarily clear, or even timely for that matter and in the analysis of enterprise security, time is always of the essence. Endeavours towards security data retrieval and meaningful exploration are usually a major concern for most security analysts. The amount and variety of the information contained in the raw data is sometimes an impediment to developing concise conclusions and obtaining a high level view of the real events hidden behind audit trails. However, the crystallization of iii security events in an analytical and visual representation will lead to a higher quality security analysis than the one obtained from the text-based reports and even from more traditional graphical approaches such as pie or bar charts. Perhaps, a subtle but often overlooked advantage of the statistical representation of security data is that while it is unnatural for human beings to remember patterns expressed in a static file, it is fairly straight forward to remember spatial objects such as pictures and maps or visual diagrams based on event data. To tackle today’s information security landscape therefore, there is a need for new methods of analysis. Pattern recognition and trending through detailed data exploration and visualization can be used to profile threats on the data link through to the application layer of the enterprise architecture. Security Analysis and Data Visualization offers perspectives into what is happening in and around the network of an enterprise. Information pertaining to traffic flows, protocols, and even individual data packets can indue the team responsible for the network security to keep it operating securely and at peak performance. As a result, the net benefit viz-a-viz information reduction compared with the unmanageable number of events on the audit trails provides us with an aggregate form of handling when the analyst intends to do a meaningful iteration on various security data elements. On a final note, this book is heavily tilted in favour of the use of free and open source tools (both on Microsoft Windows and Linux platforms). Part of the rationale for this is to bring the analyst up to speed with the concepts and techniques of security analysis and visualization without having a recourse to proprietary tools and applications. I think in my humble estimation, it bridges the knowledge gap quicker whilst bringing the core subject matter to the fore. HOW THIS BOOK IS ORGANIZED This book consists primarily of four distinct but complementary parts modeled on the concept of Capture -> Process -> Visualize -> Govern. Part 1: Capture Chapter 1 - Security Data Acquisition: chronicles the challenges facing security analysts in gaining access to information so they can conduct security analysis and respond accord- iv ingly. In view of this, the chapter begins with an appraisal of the techniques of collecting and managing network security data set. It further examines data source formats as well as methods of extracting and preserving security data for analysis. Chapter 2 - Security Data Integration: delves into the multivariate analysis and methodol- ogy for integrating enterprise security data by examining packet inspection and traffic flow. Furthermore, we introduce the concept of security data parsing and advance analysis of unstructured data, finishing off by taking an in-depth look at the tools and applications that will aid in completing the initial stage of security analysis. Part 2: Process Chapter 3 - Security Data Carving: examines active and passive security data carving and data mining viz-a-viz statistical protocol Identification (SPID) and port independent protocol identification (PIPI) algorithms - using advance techniques of packet regeneration and reassembly for statistical analysis. Chapter 4 - Security Data Exploration: forages into interactive off-line security data recon- struction and exploration techniques. This is done by way of free form concrete and contextual reassembly of raw network security data. Part 3: Visualize Chapter 5 - Security Visual Mapping: investigates the concept of visualization, the processes involved as well as the various elements of security data dissemination and mapping from generic free form graphs and charts to scatter plots, link graphs and parallel coor- dinates. The chapter further discusses when and how to choose the right graph for any specific data visualization scenario. Chapter 6 - Security Visualization Techniques: The main goal of this chapter is to aid the analyst in conducting decision support and communicating information clearly and effectively through graphical means. In order to convey ideas effectively, we need to marry both aesthetic form and functionality so as to provide insights into a rather sparse and complex data set by communicating its key aspects in a more intuitive way. With this in mind, the chapter starts by investigating the techniques of time-based visualization v of security data set. It then examines some of the components of interactive visual mapping of data variables and attributes already abstracted. Lastly, it examines the tools and applications available for interactive data visualization and statistical graphics. Part 4: Govern Chapter 7 - Security Data Management: We plumb the discipline that embodies a conver- gence of governance and data management surrounding the handling of security data in an organization. The concepts of deep packet inspection, privacy, data loss prevention and data protection will be highlighted Finally, The ultimate aim of this book is not only to get security analysts interested in this field of endeavour but be stimulated enough to go forthwith and infix security data analysis and statistical visualization in their daily itinerary. The book gives all the necessary information and tools at the same time illustrating test and use-case scenarios that should assist in applying the concepts to security analysis difficulties. AUDIENCE This book is primarily written for security administrators, network and security analysts as well as auditors and digital forensic investigators. If you want to learn security analysis and data visualization, then this book is for you. I bring to the fore new ways of analyzing security data: from the analysis of perimeter bottlenecks, insider threats, intrusion detection, compliance monitoring through to fraud detection and forensics. I assume that the reader has a basic understanding of networking concepts and is conversant with the TCP/IP model. I also assume that the reader is fairly familiar with the Linux environment especially basic CLI operations as well as basic installation and execution of Linux binary applications (rpm anyone!). However, you don’t have to be an expert in

Load more