Elastic Stack for Security Monitoring 1 in a Nutshell 2019 Pass the SALT Workshop Overview 2 Introduction to Elastic Stack Beats Logstash Elasticsearch Kibana Elastic Stack Alerting and Security Introductory Workshop! 3 • This is an introductory workshop • You probably won’t hear/see a lot of new things if you have: • Used Elastic Stack in the past; • Took the Elastic training…; • Followed SANS SEC455, SEC555, FOR572, etc.; • If you are stuck, please do not suffer in silence! Workshop VM 4 • ais_workshop_xubuntu-18.04.2-desktop-amd64 • VMware Workstation, Player, or Fusion • You can try VirtualBox too, but you are on your own with that… sorry! • 8 GB RAM • 30-50 GB disk space • Keyboard layout: EN-US !!! • Workshop VM (Ubuntu) user/pass: user / Workshop1234% • Normally, it should not require password for login and sudo About David 5 • Managing partner at Alzette Information Security (@AlzetteInfoSec) • Network penetration testing, security architectures, security monitoring, incident response • Instructor at SANS Institute: FOR572 • BSides Luxembourg organizer https://bsideslux.lu • Twitter: @DavidSzili • E-mail: [email protected] • Blog: http://jumpespjump.blogspot.com About Eva 6 • Managing partner at Alzette Information Security (@AlzetteInfoSec) • Web application penetration testing, source code review, security monitoring • CyberWayFinder • BSides Luxembourg organizer https://bsideslux.lu • Twitter: @EvaSzilagyiSec • E-mail: [email protected] • Blog: http://jumpespjump.blogspot.com Introduction to Elastic Stack 7 2019 Pass the SALT Workshop About Elastic Stack 8 What is Elastic Stack? Why Elastic Stack? • 4 main components: • (Free) Open Source Software • Elasticsearch • Distributed, real-time search and • Logstash analytics (very scalable) • Kibana • Parsing and data enrichment • Beats • Large Community • And several other smaller components • InfoSec Projects built around it: Security Onion • Elastic Stack Features (X-Pack) • • Moloch (Elasticsearch) • APM (Application Performance Monitoring) • SOF-ELK • SELKS • HELK • ROCK NSM Elastic Stack History 9 2019: Core security 2015: "Release 2017: Elastic features (TLS, Early 2000s: Bonanza“, Cloud RBAC) are Shay Banon’s Beats, Elastic Enterprise free, SIEM, Recipe App Cloud (AWS) (ECE) EndGame 2012: 2016: Elastic 2018: Open Elasticsearch Stack 5.0 source X-Pack, Inc. New York Stock Exchange Source: https://www.elastic.co/about/history-of-elasticsearch Elastic Stack (Very) High-Level Overview 10 • Beats: single-purpose data shippers • Logstash: server-side data processing pipeline • Elasticsearch: distributed search and analytics engine • Kibana: visualization and dashboards See also: https://www.elastic.co/assets/blt2614227bb99b9878/architecture-best-practices.pdf Beats 11 2019 Pass the SALT Workshop Beats: Lightweight Data Shippers 12 • Lightweight log agents • Written in Go • Can send to Logstash or directly to Elasticsearch • Beats Family: • Filebeat • Winlogbeat • Auditbeat • Packetbeat • Heartbeat • Metricbeat • Functionbeat • Etc. See also: https://www.elastic.co/guide/en/beats/libbeat/current/index.html Beats Configuration Examples 13 Winlogbeat Filebeat Beats Hands-On 14 2019 Pass the SALT Workshop Logstash 15 2019 Pass the SALT Workshop Logstash Overview 16 • LOTS AND LOTS of plugins! • Input: tcp, udp, syslog, beats, jdbc, kafka, rabbitmq, file, exec, cloudwatch, etc. • Filter: csv, json, xml, kv, grok, date, mutate, split, useragent, ruby, drop, etc. • Output: elasticsearch, graphite, nagios, kafka, rabbitmq, radis, file, email, irc, etc. • Easy to learn and use Input Filter Output See also: https://www.elastic.co/guide/en/logstash/current/index.html Input Plugin Examples 17 Plugin Description input { stdin { beats Events from Elastic Beats } } cloudwatch Events from AWS CloudWatch file Streams events from files input { beats { jdbc Events from JDBC data port => 5044 kafka Reads events from Kafka } } rabbitmq Pulls events from RabbitMQ input { s3 Events from files in S3 syslog { snmp Polls devices using SNMP port => 5514 } syslog Reads syslog messages } See also: https://www.elastic.co/guide/en/logstash/current/input-plugins.html Filter Plugin Examples 18 Plugin Description Plugin Description cidr Check IP against net blocks kv Parses key-value pairs csv Parses CSV data into fields mutate Performs mutations on fields date Parses dates from fields ruby Executes Ruby code dissect Extracts unstructured data split Splits multi-line messages drop Drops all events translate Replaces field contents elasticsearch Gets data from Elasticsearch truncate Truncates fields geoip Geo info about an IP urldecode Decodes URL-encoded fields grok Parses unstructured data useragent Parses user agent strings json Parses JSON data xml Parses XML data See also: https://www.elastic.co/guide/en/logstash/current/filter-plugins.html Filters - The Easy Stuff 19 JSON CSV filter { filter { ... ... json { csv { columns => ["ts", "uid", "id.orig_h", source => "message“ "id.orig_p", "id.resp_h", "id.resp_p", "proto", } "service", "duration", "orig_bytes", ... "resp_bytesconn_state", "local_orig", "local_resp", mutate { "missed_bytes", "history", "orig_pkts", "orig_ip_bytes", remove_field => [ "message" ] "resp_pkts", "resp_ip_bytes", "tunnel_parents"] } separator => " “ } } ... mutate { remove_field => [ "message" ] } } Filters - RegExp vs. Grok, Dissect (1) 20 RegExp Dissect • (?<![0-9])(?:(?:25[0-5]|2[0- • String-based split operation 4][0-9]|[0-1]?[0- • Very fast 9]{1,2})[.](?:25[0-5]|2[0-4][0- Grok 9]|[0-1]?[0-9]{1,2})[.](?:25[0- 5]|2[0-4][0-9]|[0-1]?[0- • %{IPV4:source_ip} 9]{1,2})[.](?:25[0-5]|2[0-4][0- • Pre-cooked RegExp patterns 9]|[0-1]?[0-9]{1,2}))(?![0-9]) • Custom Patterns: • (?<queue_id>[0-9A-F]{10,11}) Grok Debuggers: • Heroku App: http://grokdebug.herokuapp.com • Source: https://github.com/nickethier/grokdebug • Docker: https://hub.docker.com/r/fdrouet/grokdebug • Kibana / Dev Tools / Grok Debugger Filters - RegExp vs. Grok, Dissect (2) 21 dissect grok filter { filter { ... ... dissect { grok { mapping => { match => { "message" => "%{ts} %{+ts} "message" => %{+ts} %{src} %{prog}[%{pid}]: %{msg}“ "%{SYSLOGTIMESTAMP:syslog_timestamp} } %{SYSLOGHOST:syslog_hostname} } %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: ... %{GREEDYDATA:syslog_message}" } #"message" => "%{SYSLOGBASE2} %{GREEDYDATA:message}“ } } ... } Filters - Enrichment Examples 22 ruby geoip filter { filter { ... ... if [program] == "bro_dns" { if [resp_h_routable] == "true" { ruby { geoip { code => "event.set('query_length', source => "id.resp_h“ event.get('query').length)“ target => "geoip“ } default_database_type => "City“ } } ... geoip { } source => "id.resp_h“ target => "geoip“ default_database_type => "ASN“ } } ... } Output Plugin Examples 23 Plugin Description csv Writes events to disk in CSV output { stdout { elasticsearch Stores logs in Elasticsearch codec => rubydebug } email Sends email to an address } exec Runs a command file Writes events to files output { graphite Writes metrics to Graphite elasticsearch { kafka Writes events to Kafka hosts => ["localhost:9200"] } rabbitmq Pushes events to RabbitMQ } redis Sends events to Redis See also: https://www.elastic.co/guide/en/logstash/current/output-plugins.html Elastic Common Schema (ECS) 24 • Specification that provides a Level Description consistent and customizable way to structure your data in ECS Core Fields Fully defined set of field names Elasticsearch that exists under a defined set of • Searches can be created more ECS top-level objects narrowly ECS Extended Fields Partially defined set of field names • Field names are easier to remember that exists under the same set of • ECS Reference: ECS top-level objects https://www.elastic.co/guide/en Custom Fields Undefined and unnamed set of /ecs/current/index.html fields that exists under a user- • ECS GitHub: supplied set of non-ECS top-level https://github.com/elastic/ecs objects that must not conflict with ECS fields or objects Logstash Hands-On 25 2019 Pass the SALT Workshop Elasticsearch 26 2019 Pass the SALT Workshop Elasticsearch Overview 27 • Storage and Search More info: John Hubbard - The Elastic Stack as a SIEM: https://www.youtube.com/watch?v=v69kyU5XMFI • Built on Apache Lucene • “wrapper” written in Java • REST API • JSON over HTTP • Distributed • Real-time Elasticsearch Terms 28 • Cluster: All nodes • Node: Elasticsearch instance Node1 Node2 Node3 • Index: Set of documents (group of shards) • Shard: Index1 • Subset of documents in an index Documents • Apache Lucene instance Index2 Shard • Primary (like RAID 0) and IndexShard Index Replica (like RAID 1) Shard Shard • Document: JSON object in Elasticsearch Index3 Cluster Elasticsearch vs. Relational Database 29 • Mapping: Elasticsearch Relational Database • Defines field names and datatypes Index Database in documents • Can add new fields, but existing Mapping Schema fields cannot be changed! Document Row • Field: Field Column • Key-value pair in a document • Metadata like: _index, _id, etc. • WORM (Write Once Read Many) vs. ACID (Atomicity, Consistency, Isolation, Durability) Data Types (Few Examples) 30 Core Specialized • text • date • keyword • ip • long, integer, short, byte Complex • double, float, half_float, • array scaled_float • object • boolean • nested binary • Multi-fields Geo • Indexed as more one type • geo_point Etc. • geo_shape See also: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html Text vs. Keyword 31 Text type Keyword type • “Full-text value” • “Exact value”