Revista Informatica Economică nr.2(46)/2008 75 Intrusion Detection using Open Source Tools Jack TIMOFTE [email protected] We have witnessed in the recent years that open source tools have gained popularity among all types of users, from individuals or small businesses to large organizations and en- terprises. In this paper we will present three open source IDS tools: OSSEC, Prelude and SNORT. Keywords: Network security, IDS, IPS, intrusion detection, intrusion prevention, open source. ntroduction OSSEC (www.ossec.net) is an open source I The traditional form of securing the net- host-based intrusion detection system work, the firewall proved to be insufficient. (HIDS). Given the nature and the complexity of the According to their own website, "OSSEC is a attacks, new ways of protecting the network scalable, multi-platform, open source Host- had to be developed. Intrusion Detection Sys- based Intrusion Detection System (HIDS). It tems, or shortly IDS are tools to monitor the has a powerful correlation and analysis en- events occurring in a computer system or gine, integrating log analysis, file integrity network and detect signs of possible inci- checking, Windows registry monitoring, cen- dents, such as violation of computer security tralized policy enforcement, rootkit detection, policies, acceptable use policies or standard real-time alerting and active response. It security practices. An Intrusion Prevention runs on most operating systems, including System, or IPS in addition to detecting inci- Linux, OpenBSD, FreeBSD, MacOS, Solaris dents, can play an active role by attempting and Windows." to stop the possible incidents which are de- In the recent years, OSSEC has received sev- tected. The Intrusion Detection/Prevention eral awards. In 2007, it was placed on the Systems (IDPS) can be classified according first position in the Top 5 Open Security to different criteria. The NIST Guide to In- Tools in the enterprise by LinuxWorld. In trusion Detection and Prevention Systems 2006, OSSEC was voted as the second best (NIST 800-94), lists four main types: IDS tool by the survey conducted by the sec- - Network-Based, which monitor the net- tools.org website. Recently, OSSEC has work; been acquired by Third Brigade, but accord- - Wireless - monitor wireless network traffic; ing to the press release, OSSEC will remain - Network Behavior Analysis (NBA) - which open source. examine the network traffic to identify The OSSEC HIDS can be installed as a threats like denial of service attacks and stand-alone tool to monitor one host or can malware; be deployed in a multi-host scenario, one in- - Host-Based - monitor a single host and the stallation being the server and the others as events occurring within that host. agents. The server and agents communicate According to the way an IDPS detect the in- securely using encryption. cidents, they can be classified into three cat- OSSEC also has intrusion prevention fea- egories: signature-based, anomaly-based and tures, being able to react to specific events or stateful protocol analysis, but most IDPS sys- set of events by using commands and active tems use multiple detection methodologies, responses. The system allows the creation of either separately or integrated, to improve the new commands which can be bound to performance. events. The system comes with some prede- Below we will briefly present a selection of fined active response tools, but the adminis- three open source tools: OSSEC, Prelude and trator can add others. SNORT. OSSEC was designed initially for Linux, but 76 Revista Informatica Economică nr.2(46)/2008 it evolved and since version 0.8 it also fea- them into the database, having several fea- tures a Windows agent, which can monitor tures such as relaying or filtering the events. the event log and other files. The Prelude LML is a signature-based log PRELUDE. Prelude (www.prelude- analyzer monitoring log files and received ids.com) is more than an open source IDS syslog messages for suspicious activity. It system - it is a framework which enables can handle events generated by a large set of other security applications to report to a cen- components, such as: Cisco PIX, Clamav, tralized system. ipchains, Netfilter, ipfw, Nokia ipso, Ms- It makes use of the IDMEF (Intrusion Detec- SQL, Nagios, NTsyslog, Pam, Portsentry, tion Message Exchange Format) standard Postfix, Proftpd, ssh etc. proposed by IETF, which allows defining the The PreludeDB library allows the developers events recorded by different sensors using a to use the Prelude IDMEF database, provid- single language. Prelude is considered a hy- ing an abstraction layer based upon the type brid system, since it allows the coexistence and the format of the database used to store of the event data from host-based, network- IDMEF alerts. based or wireless IDS agents, or simply any The last component, Prewikka is a console other security application. Existing security providing advanced features like contextual applications can be modified to use the Pre- filtering, aggregation, etc. Below we can see lude system, using the provided C, Python a simplified architecture with three sensors. and Perl frameworks. An interesting feature of Prelude is the possi- The main components of a Prelude system bility to “relay” the events between manag- are: the Prelude library (framework), the ers: in the following example, Branch A is Prelude Manager, the Prelude-LML, Prelu- relaying all the events to the Prelude Manag- deDB library and Prewikka (the console). er located in the Network Operation Center The Prelude library, Libprelude, is used to of the organization. Using this setup, Branch access the Prelude system and provides an A can only access the events recorded by API (Application Programming Interface) to sensors D, E and F, while the NOC (Network create events in the IDMEF (Intrusion Detec- Operation Center) can access the events gen- tion Message Exchange Format) format. It erated from all branches. can be used for failover and allows the crea- An interesting thing related to the three open tion of sensors that read the events received source tools presented in this article, is that by one or a set of prelude managers. they can be integrated: Prelude IDS frame- The Prelude Manager is a high-availability work has native support for both Snort and server which collects and normalizes infor- OSSEC. A list with all the external sensors mation from distributed sensors and stores which are supported natively is presented in Table 1. Fig.1. Prelude Simple Architecture Using the IDMEF API, additional own sen- SNORT. Snort (www.snort.org) is, without sors can be defined and programmed. In the doubt, one of the most popular open source PRELUDE documentation there is such an security tools. With millions of downloads, it example written in C. is used by individuals as well as large corpo- Revista Informatica Economică nr.2(46)/2008 77 rations or government organizations. The be faster than for most commercial IDS tools. first version was written in 1998 by Martin According to Jennifer Albornoz Mulligan, Roesch, who later founded Sourcefire. Since security researcher at Forrester Research, then, the product evolved both as features “Once the community writes those signa- and as portability: currently Snort is available tures, Sourcefire takes a little extra time to for most major platforms including Win- test them before it puts them out there, but dows, BSD, Solaris or Mac OS X. It is worth the process is still generally more responsive mentioning that Snort has an excellent sup- than others". This community ruleset comes port from the user community. This can be in addition to the official ruleset, released by considered as a big advantage since the the Sourcefire Vulnerability Research Team. availability of signatures for new attacks can Fig.2. Prelude Relaying Architecture Security Tool Description AuditD audit system Nepenthes detection and collection of worms and malware NuFW authenticating firewall OSSEC HIDS PAM authentication tasks PFLogger reports alerts from OpenBSD firewall Sancp information collection on network activity Samhain file integrity checker Snort NIDS Table 1. Native Support in Prelude IDS Snort can run in different modes, ranging and rules. from a simple sniffer to a IPS system: Snort Preprocessors allow the functionality - Sniffer mode; of Snort to be extended by allowing users - Packet Logger mode - logs packets to disk; and programmers add modular plug-ins. Pre- - NIDS mode - allows Snort to analyze net- processor code is run before the detection work traffic for matches against a user- engine is called, but after the packet has been defined rule set and to perform several ac- decoded. Such preprocessors exist for IP de- tions based on these matches; fragmentation (Frag3), TCP stream reassem- - Inline (IPS) mode - allows Snort to drop or bly (Stream4), HTTP, FTP, SMTP, SSH etc. pass packets based on the specific Snort Snort rules are used by the system to detect rules. incidents. Snort rules are divided into two Working as an IDS, Snort uses preprocessors logical sections, the rule header and the rule 78 Revista Informatica Economică nr.2(46)/2008 options. The rule header contains the rule's notification and logging when the system action, protocol, source and destination IP fires events. The supported types are text addresses and netmasks, and the source and (console), syslog and Unified 2 (a serialized destination ports information. The rule op- binary stream format). tion section contains alert messages and in- - The TCP Stream Reassembler – provide formation on which parts of the packet target-based services for reassembling TCP should be inspected to determine if the rule segments into normalized streams. action should be taken. The Data Source API is an interface between At this time, the latest stable version is the Data Source component and the Dis- 2.8.2.2. Currently a new beta version is under patcher.