INF526: Secure Systems Administration

SIEM and Intrusion Detection

Prof. Clifford Neuman

Lecture 12 5 April 2017 OHE100C Class Presentation Schedule

4/5 Mohammed Alsubaie – SIEM and Intrusion Detection 4/12 Vishnu Vadlamani - Network Monitoring/Attack Forensics 4/19 Andrew Gronski - Accreditation and acceptance testing

1 Detecting Intrusions

• Visibility of artifacts from intrusions – Passive attacks • Difficult to detect because technically no changes made. – But in some cases – evidence of connecting to a network might be present. – Honeytokens – Intrusions are really all Active attacks • Identifying Subversions • Identifying traffic that is indicative of an intrusion – Best point of observation is outside affected system

2 Where Detection Occurs

• Events visible in the network – New network peer entities is evidence – Significant change in frequency or bandwidth is evidence • Events on the Compromised System – Changes to system binaries are evidence of subversion – Changes to accounts and privileges are evidence – Changes to the running processes and CPU share are evidence – Creation of new files are evidence • The above are anomalies and an administrators role is to sort though them.

3 Events Visible in the Network

• Network Monitoring uses a system to monitor a computer network for abnormalities and notifies an administrator or other system components when an issue is identified. – Enterprise health monitoring • Network Monitoring tools allow us to see the devices connected to our network and traffic between them. • Packet analyzers can capture network traffic for viewing through an event management system. • Such data is useful for intrusion detection.

4 Network Monitoring and Secure SA

• Secure administration of a network and systems requires visibility of current activities and status of devices, and the ability to examine past activities. In particular: – Ensures Availability is maintained • Ability to correct system issues, and to identify potential DoS attacks – Ensures Non-Repudiation • Gives administrator a history of who did what, and when • While not directly involved in enforcement, it also assists in ensuring other principles (Integrity, Confidentiality, Authentication, Authorization) have not been breached. – Can identify suspicious activity that may indicate a security compromise.

Slide by Josh McCamey INF526 Summer 2016 5 How Does it Work

• Enterprise Network Monitoring systems usually check the health of systems on the network by using a “ping” signal to test whether ports respond as expected. – Period between pings can be anywhere from minutes to hours – Checks response times and up-time – May also check for things like consistency and configuration of specific equipment or software on the network • Can check just about any network protocol (e.g. HTTP/S, SNMP, FTP, SMTP,POP3, IMAP, DNS, SSH, TELNET, SSL, TCP, etc…) – In the case of a protocol like SMTP/POP3, system may send a test message periodically through one protocol while receiving through another to check the health of servers the message passes through.

Slide by Josh McCamey INF526 Summer 2016 6 How Does it Work • A failure occurs when a test operation or “ping” times out – When a failure is recorded the Network Monitor produces an “action” – System can use any available means to notify an administrator (e.g. e-mail, text, phone call, pop-up notification on local machine, even an audible alarm) • Advanced networks may have automatic fail-safes that off-load a server’s duty to the rest of the network to make the failure transparent to the user – This is critical for big companies/sites that can’t afford down-time. Slide by Josh McCamey INF526 Summer 2016 7 How Does it Work

• Packet analyzers, on the other hand, capture packets going across an interface – Sometimes these are also called “packet sniffers” – Close relationship to forensics and intrusion detection – Can be used to determine the nature of connections and identify malicious activity • Wide variety of visualization, recording and analysis capabilities depending on thetool used • Packet analyzers and health monitors not necessarily mutually exclusive – Many different tools, some can “do-it-all”, depending on your definition of “all”

Slide by Josh McCamey INF526 Summer 2016 8 Common Tools (not a compressive list)

• ntop • NetworkMiner • Microsoft Message Analyzer • Pandora FMS • MRTG • Zenoss Core • PRTG • The Dude • Nagios • Splunk • OpenNMS • Angry IP Scanner • SolarWinds • Icinga • Spiceworks IT Management • Total Network Monitor • Advanced IP Scanner • NetXMS • Capsa • Xymon • Fiddler • And… WIRESHARK

Slide by Josh McCamey INF526 Summer 2016 9 Microsoft Message Analyzer

• Successor to Microsoft Network Monitor – Free, 32-bit and 64-bit: Windows 7, Windows 8, Windows 8.1, Windows Server 2008 R2, Windows Server 2012, Windows Server 2012 R2, and Windows 10 clients • Capture, display, and analyze protocol messaging traffic • Trace and assess system events and other messages from Windows components • Main Features: – Integrated “live” event and message capture at various system levels and endpoints – Parsing and validation of protocol messages and sequences – Automatic parsing of event messages described by ETW manifests – Summarized grid display – top level is “operations”, (requests matched with responses) – User controlled “on the fly” grouping by message attributes – Ability to browse for logs of different types (.cap, .etl, .txt) and import them together – Automatic re-assembly and ability to render payloads – Ability to import text logs, parsing them into key element/value pairs – Support for “Trace Scenarios” (one or more message providers, filters, and views)

Slide by Josh McCamey INF526 Summer 2016 10 Microsoft Message Analyzer

Slide by Josh McCamey INF526 Summer 2016 11 Nagios Network Analyzer

• Popular paid IT admin tool to monitor for configuration issues, high load, etc. • Focused on high-level goals rather than low level packet inspection – Netflow Analysis, Monitoring, and Bandwidth Utilization – Security and reliability monitoring and user notifications – Advanced visualization capabilities – Custom application monitoring – Bandwidth monitoring and threshold settings with alert capability – Advanced user management to allow for multi-tenancy • Linux focused • Specialized versions for machines running in virtual environments like VMWare

Slide by Josh McCamey INF526 Summer 2016 12 Nagios Network Analyzer

Slide by Josh McCamey INF526 Summer 2016 13 Open NMS

• Free & open-source enterprise grade network monitoring & management tool • Runs on Linux/Win/Mac • “Network Management Application Platform” – Made to be customized and have network monitoring solutions/applications built on top of it • Core feature set – Java-based framework – API-extendable notification mechanism including SMTP, Slack & Mattermost, Jabber and XMPP, Microblogs (i.e. Twitter) – Ticketing integration framework including Request Tracker, BMC Remedy, IBM Tivoli Service Request Manager, – Atlassian JIRA, extensible ticketing API – Alarm forwarding to external applications for handling or analysis (ElasticSearch), “Northbound” API and JMS, – AQMP integration – “Southbound integration” enables gathering of performance measurements and logging messages – from network applications and devices (test connections, check resource configurations, etc.)

Slide by Josh McCamey INF526 Summer 2016 14 Open NMS

Slide by Josh McCamey INF526 Summer 2016 15 NTopNG

• Network traffic probe that shows network usage similar to Linux “top” command – Full-featured network analysis tool, free & paid versions, Portable, made to run on any Unix platform (&Win x64) – Uses a web server to provide a web browser interface – Sort network traffic according to many criteria including IP address, port, Layer 7 protocol (app layer), throughput, and more – Show network traffic and IPv4/v6 active hosts. – Produce long-term reports about various network metrics such as throughput, application protocols – Top X talkers/listeners, top ASs, top Layer 7 applications – For each communication flow report network/application latency/round trip time (RTT), TCP stats (retransmissions, packets OOO, packet lost), bytes/packets – Store on disk persistent traffic statistics in RRD format. – Geolocate hosts and display reports according to host location. – Discover application protocols by leveraging on nDPI, ntop’s DPI framework. – Characterise HTTP traffic by leveraging on characterisation services provided by Google and HTTP Blacklist. – Show IP traffic distribution among the various protocols. – Analyse IP traffic and sort it according to the source/destination. – Display IP Traffic Subnet matrix (who’s talking to who?) – Report IP protocol usage sorted by protocol type. – Produce HTML5/AJAX network traffic statistics.

Slide by Josh McCamey INF526 Summer 2016 16 NTOP NG

Slide by Josh McCamey INF526 Summer 2016 17 Wireshark

• World's foremost network protocol analyzer (self-claimed) – - Deep inspection of hundreds of protocols, with more being added all the time – - Live capture and offline analysis – - Multi-platform: Runs on Windows, Linux, OS X, Solaris, FreeBSD, NetBSD, and many others – - Captured network data can be browsed via a GUI, or via the TTY-mode TShark utility – - Rich VoIP analysis – - Read/write many different capture file formats (too many to show here) – - Capture files compressed with gzip can be decompressed on the fly – - Live data can be read from Ethernet, IEEE 802.11, PPP/HDLC, ATM, Bluetooth, USB, Token Ring, Frame Relay, FDDI, and more – - Decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2 – - Coloring rules can be applied to the packet list for quick, intuitive analysis – - Output can be exported to XML, PostScript®, CSV, or plain text • - Standard for analyzing traffic for many industry and educational purposes – - Similar to tcpdump w/GUI • - Paid add-ons (SteelCentral) can make more effective for large-scale monitoring

Slide by Josh McCamey INF526 Summer 2016 18 WireShark

Slide by Josh McCamey INF526 Summer 2016 19 Deployment of Network Monitoring

Switch Spanning Port • Can miss traffic during burst periods Network Tap • Usually a temporary /emergency solution • Not commonly found, used when options A&C not possible Inline • Makes all network connectivity dependent on sensor health Other • Bottom line, your network monitoring device must be deployed in a way that enables it to see/monitor all relevant traffic.

Slide by Josh McCamey INF526 Summer 2016 20 Summary of Network Monitoring

There are multiple focuses of Network Monitoring – Focus on Health – Internally vs. Externally Geared – All play a role in Secure Systems Administration Works either via actively inquiring about network devices or passively observing – Ping and inquiries – Packet Inspection/Recording/Analysis Many different ways for admins to interact with systems and receive notifications – Advanced Visualization – Notifications, Automated Response A tool for every use case (including frameworks to build your own) Can be as high-level or as granular as you need Close relationship between monitoring and IDS/Forensics

Slide by Josh McCamey INF526 Summer 2016 21 Intrusion Detection

• Signature Based – Specific characteristics of known attacks and attack tools are maintained, and network based IDS’s scant traffic looking for these patterns. – Host based IDS’s scan system looking for these patterns. – Administrator installs IDS and keeps signatures up-to- date. • Anomaly Based – In the examples from the previous slide many anomalous events will be normal or innocuous. – Administrator must add to patters of normal events or biome overwhelmed with false positives.

22 Gathering Information

• Useful Reading: • SANS Institute InfoSec Reading Room - Logging and Monitoring to Detect Network Intrusions and Compliance Violations in the Environment by Sunil Gupta. • Data Sources – Syslog – SNARE agent – Dedicated monitor on host – Bro for network event data • Collected by – An event collection system • E.g. a SNARE Server

23 Application of NM to Intrusion Detection

• An intrusion detection system (IDS) monitors network activity for malicious behavior or policy violations – Clearly, network monitoring and intrusion detection go hand in hand • Multiple ways to detect intrusions – Can use a unified application to analyze traffic and identify anomalies – Have a separate application analyze logs generated by a network monitor • Many applications can provide this functionality – Later We’ll demonstrate using our “de facto standard” (self-proclaimed): Wireshark – Examples taken from “Intrusion Detection Using Wireshark” by S. Gupta and R. Mamtora – (International Journal of Advanced Research in Computer Science and Software Engineering, Volume 2, Issue 11,2012.)

Slide by Josh McCamey INF526 Summer 2016 24 Wireshark ID Example

Using Wireshark -> Expert Info - Each will contain the following: – Chat (grey): information about usual workflow, e.g. a TCP packet with the SYN flag set – Note (cyan): notable things, e.g. an application returned a "usual" error code like HTTP 404 – Warn (yellow): warning, e.g. application returned an "unusual" error code like a connection problem – Error (red): serious problem, e.g. Malformed Packet

Malformed packets can indicate malicious activity

Slide by Josh McCamey INF526 Summer 2016 25 Wireshark ID Example

• Using Wireshark -> Chats • TCP connections should each have SYN, SYN+ACK, ACK sequence • Sequence of SYNs could indicate DoS

Slide by Josh McCamey INF526 Summer 2016 26 Wireshark ID Example

• Same thing can be done with Wireshark -> Flow Graph • Continuous barrage of SYNs on various ports with no other messages (i.e. ACK) • indicates DoS

Slide by Josh McCamey INF526 Summer 2016 27 Wireshark ID Example

• Using Wireshark -> Conversations • Shows all interaction between two endpoints (e.g. two IP addresses) • Add a column to each row called “conclusion” which contains either “normal” or “abnormal”. Set all values to “abnormal” and update to “normal” when behavior is verified (e.g. packet from A- >B and B->A that is not equal to zero, which reflects a connection being established) • Can apply any data mining technique such as association rule, classification, clustering, etc. to find rules that reflect connection legitimacy. • Figure on bottom-right shows that if the port number is less than 1066 and packet sent is greater than 3,then all data sent is normal; otherwise, it’s abnormal.

Slide by Josh McCamey INF526 Summer 2016 28 Visualizing the Data

• Security Incident Event Management (SIEM) – Collects data from many sources • Host, network, and application – Converts data into a common database format – Enables search and analysis of the collected data – Provide visualization tools that allow one to push down on and query for statistics from its database of events. – Examples: Snort, many many commercial products • Qradar, OSSIM, others

29 Filtering the Data

• Signature Based – Looking at known attacks • Anomaly Based – How to define normal – Baseline for system – Baseline for network • Log Analysis and Correlation – Categorize events (as authorized) – Drops from view of things to be monitoried • Reduces false positives • Lets brainstorm on how to do this. – Examples from your exercises?

30 Brainstorming: How and What

• Unexpected Flows – What might this indicate, Where might we see these – Filtering the expected • Some events should only occur following specific administrative action. – Consider privileged operations on systems – What does root do • Forking / creation of new processes • Logins • DNS Lookups • Access to specific files • Installations of software – new executables

31 Monitoring from the Outside

• If sensors on affected system, send data to SIEM as soon as possible. – Actions leading to a breach are moved off system before breach compromises reporting. • Monitor from Host OS/Hypervisor – Breach of guest does not affect such reporting. • Monitor network activity through appliance or embedded device. – Requires compromise of separate device to subvert reporting.

32 Taking Action

• The purpose of intrusion detection is to provide the administrator with actionable intelligence. – Actions are taken based on that insight. • Actions – Shut down the affected systems – Restore to a known state – Block access from parts of the internet • Some actions can be taken automatically – Simply closing a vulnerability might be too late.

33 IDS and SIEM

Mohammad AlSubaie Outline

• Motivation • Intrusion Detection Systems – Types – Detection Methods • Security Information and Event Management – Features – Workflow Motivation

• No system is 100% secure • We can’t prevent all break-ins • There will always be new holes, new attacks, and new attackers • We need some way to cope Defense in Depth

• Single defenses can and will fail • Instead, we need multiple layers of defense of different designs and objectives Network Defense in Depth

1 1. Advance Threat Protection 2 e.g. FireEye, Cisco/IronPort 3 2. Intrusion Detection Systems 4 e.g. McAfee 5 3. Web Security 6 e.g. Fortinet 7 4. Email Security 8 e.g. Bluecoat, Cisco/IronPort 5. Forensics Analysis e.g. RSA/NetWitness 6. Data Loss Prevention (DLP) e.g. TrendMicro 7. Next-Gen Firewalls e.g. Palo Alto Networks 8. Security Event Monitoring e.g. HP/ArcSight Network Defense in Depth

• A device or software application that monitors a network or systems for malicious activity or policy violations. [CIA violation]

• Types of Intrusion Detection Systems – Network-based IDSs – Host-based IDSs Network-based IDS

• Deploying sensors at strategic locations – E.G., Packet sniffing via tcpdump at routers • Inspecting network traffic – Watch for violations of protocols and unusual connection patterns • Monitoring user activities – Look into the data portions of the packets for malicious command sequences • May be easily defeated by encryption – Data portions and some header information can be encrypted Network-based IDS

Countermeasure

Policy Alarms Event Event Analyzer stream Event Storage Filtered packet stream Configuration

Packets Collector

Packet stream

Sensor Sensor Sensor Sensor SYSTEM Firewall versus Network IDS

• What is the difference between firewalls and network-based Intrusion Detection Systems? Firewall vs. Network-based IDS

• What is the difference between firewalls and network-based Intrusion Detection Systems?

• Firewall – Active filtering – Fail-close

• Network-based IDS FW – Passive monitoring – Fail-open

IDS Host-based IDS

• Using OS auditing mechanisms – EventLog: Windows Event Logs, System, Security, Application. – Netstat: Read information about network usage. – Health: Read information about CPU, Memory, and Swap usage. – Ps: Read information about running processes • Monitoring user activities – E.g. analyze shell command • Monitoring execution of system programs Host-based IDS

Policy Configuration

Alarms Detector Countermeasure

Vulnerability Audit logs Actions Scanning

SYSTEM IDS detection Methods

• Knowledge-based

• Specification-based

• Behavior-based Evaluation of IDS

• Accuracy • Completeness • Performance • Fault tolerance • Timeliness Knowledge-based IDS

• High accuracy, but low in completeness

• Drawback: need regular update of knowledge (malware signatures)

• High performance with minimal processing power Specification-based IDS

• Manually develop specifications that capture the legitimate system behavior. Any deviation from it is an intrusion. • Pro: can avoid false-positives since the specification can capture all legitimate behavior. • Con: hard to develop a complete and detailed specification. Behavior-based IDS

• High in completeness, but low accuracy

• Detect intrusion by observing a deviation from the normal or expected behavior of the system or the users

• Can detect attempts to exploit new and unforeseen vulnerabilities (zero-day attacks) Typical Corporation Environment

Intrusion Prevention System Remote User Internet Router

Database Server Web Server App Server Email DC Server Server Switch Switch DNS EPO Server Server Typical Corporation Environment

Intrusion Prevention System Remote User Internet Router

Database Server Web Server App Server Email DC Server Server Switch Switch DNS EPO Server Server Typical Corporation Environment

Intrusion Prevention System Remote User Internet Router

Database Server Web Server App Server Email DC Server Server Switch Switch DNS EPO Server Server Security Information and Event Management (SIEM) • SIEM is a system combining Security Information Management (SIM) and Security Event Management (SEM). • SEM deals with – Real-time monitoring – Correlation of events and threat intelligence – Notifications – Console views • SIM deals with – Long-term storage – Analysis and reporting of log data SIEM Features

* Source “ManageEngine” https://www.manageengine.com/products/eventlog/manageengine-siem-whitepaper.html SIEM Workflow

Generate Collect Data Archive Logs for Alerts Security Security and from Log Correlate Events Forensic Incidents Compliance Sources Analysis Reports SIEM Workflow

Generate Collect Data Archive Logs for Alerts Security Security and from Log Correlate Events Forensic Incidents Compliance Sources Analysis Reports

• Methods of collecting data from sources

Data Source Collector Data Source

• Aggregation: to gather data together as a whole in singular repository • Normalization: to create consistent records by type and format Normalization

• Original log format from source 1 10:32, 12/3/2017, alsubaim, ad.corporate.com, error, failed login attempt • Original log format from source 2 12:45, 3/23/2017, malicious code detected, host1.corporate.com, alsubaim • Normalized logs 10:32, 12/3/2017, alsubaim, ad.corporate.com, failed login attempt 12:45, 23/3/2017, alsubaim, host1.corporate.com, malicious code detected SIEM Workflow

Generate Collect Data Archive Logs for Alerts Security Security and from Log Correlate Events Forensic Incidents Compliance Sources Analysis Reports

• Link events to identify attacks • Event based: – a single event identifies an attack • Rule based: – If X + Y + Z then do A – If X repeated 3 times within an hour, then do Y • Anomaly based: – If the traffic on port X exceeds the standard deviation of historic traffic patterns, then there may be a problem SIEM Workflow

Generate Collect Data Archive Logs for Alerts Security Security and from Log Correlate Events Forensic Incidents Compliance Sources Analysis Reports

• Severity Low Medium High Critical • Notification: upon identifying a threat, notifications are sent to the security administrators (SOC) • Automated Response: the majority of SIEM tools can execute external scripts to react on identified threats. (Change to FW rules, issue a Remedy ticket) Security Operations Center (SOC) SIEM Workflow

Generate Collect Data Archive Logs for Alerts Security Security and from Log Correlate Events Forensic Incidents Compliance Sources Analysis Reports SIEM Workflow

Generate Collect Data Archive Logs for Alerts Security Security and from Log Correlate Events Forensic Incidents Compliance Sources Analysis Reports

• Collected log data is stored for future forensic investigations.

• Not equivalent to Log Management Solutions SIEM vs. LM

Security Information and Event Functionality Log Management Management Log collection Security related logs All logs Parsing, normalization, Indexing, parsing, or Log pre-processing categorization, and enrichment none Log retention Retain parsed and normalized data Retain raw log data Reporting Security focused reporting Broad use reporting Correlation, threat scoring, event Full text analysis, Analysis prioritization tagging Alerting and Advanced security focused Simple alerting on all notification reporting logs Incident management, analyst High scalability of Other features workflow, context analysis, etc. collection and storage HP ArcSight (SIEM) Why SIEM Implementation Fails?

• Lack of planning – No defined scope • Faulty Deployment Strategies – Incoherent log management data collection – High volume of irrelevant data can overload the system • Operational – Lack of management oversight – Assume plug and play

Security is a process not a product Thank You! Question? Second Exercise - Criminal Enterprises

• Chosen because of differences in the high level principles. – Not because I expect you to implement these kinds of systems in your future endeavors. – But you may be called upon to break some of these systems if later employed by government organizations. • Your organization must: – Accept Bitcoin as payment (not really, but it must accept something that stands in for bitcoin) – Manage an inventory of stolen account identifiers with passwords – Control access to such information – Prevent collection of evidence or intelligence by third parties. – Note, do not deal in any illegal goods, but use dummy information to stand in for such goods. Also, do not use terms associated with such illegals goods or information in communications, make up new names for this dummy information.

69 Teams for Second Exercise

• Group A – Enrolled: Srivastav, Binkulaib, Dmytrisin, Alsubaie, Bubshait, Gronski – Advisors: haiboz, jaskaran, rajattan

• Group B – Enrolled: Guerrerobognoli, Li, Wright-Walker, Jackoski, Vadlamani, Zhang – Advisors: sajmani, slakkadi, tosirisu, wang907