SESSION ID: OST-R07 Suricata: 10 Years Strong Bringing the Best in Network Threat Detection

Eric Leblond Peter Manev Suricata Developer CSO The Open Information Stamus Networks, LLC Security Foundation (OISF) / Suricata #RSAC Suricata

10 Years Strong Bringing the Best in Network Threat Detection Who are we?

Eric Leblond

−OISF executive council

−Suricata Developer

−Stamus Networks co-founder

Peter Manev

−OISF executive council

−QA lead at OISF

−Stamus Networks co-founder OISF

The non profit foundation behind Suricata

Created more than 10 years ago to

−Receive funding

−Pay for Suricata development

−Promote Suricata

Organizing Suricon every year

−2019: 250 attendees in Amsterdam

−2020: Boston, in November

Consortium members:

−Fireeye, Proofpoint

−AlienVault, ANSSI, Juniper, Verizon, IronNet

−Bricata, Cilera, Greycortex, Indegy, Red Piranha, Ubiquiti Suricata

Intrusion Detection System

−Signature based IDS

−JSON alerts including metadata for context

Network Security Monitoring

−Port independent protocol discovery

−Board protocol support: http, smtp, tls, smb, ftp, nfs, ikev2, dhcp, ...

−JSON events

−On Disk File extraction Who’s using Suricata ?

Basically everyone you would wish to have as customer if you are a software editor

Big companies:

−Facebook, Google, Amazon, ...

Small companies:

−FireEye, Thales, ...

Governmental:

−DoD, Homeland, French army, $(put your country here) $(army|govt defense agency) Why not pick or/and

Feature side ●Political side

−Zeek will provide you a great NSM −Suricata has a real community based development −Snort will provide you a old but decent IDS −Snort is Cisco’s abandoned pet −Running both on a box can be a mayhem

−Suricata can replace both

−Suricata scalability is unmatched

80Gbps on one server

With 50000 signatures

(yes, on live traffic...) Who’s making to Suricata

●OISF paid developers ●OISF Executive director

●Victor Julien, Jason Ish, Shivani Bhardwaj, Jeff ●Kelley Misata PhD Lucovsky, Philippe Antoine, Andreas Herz ●Board of Directors ●Contributors ●Matt Jonkmann, Gene Stevens, Brian Casserly, ●133 authors Charles-H Schultz

●Private companies and governmental ●Executive team

●Mostly Europe and America ●Kelley Misata, Victor Julien, Jason Ish, Josh Stronheim, Peter Manev, Eric Leblond

●Community council

●Quarterly meeting to discuss recent evolution and perspectives Suricata 1.0

Date: 2010 (beta released Dec 2009)

Native multi threading

IDS compatible with snort rules

Protocol discovery –Identify protocol independently of port

HTTP analyzer and keywords –Match on protocol fields –Normalization

Start of NSM with HTTP logging –Apache like logging of all HTTP transactions Suricata 1.2

Date: 2012/02

File extraction –Extract file on disk –Per signature via a keyword –Globally via configuration

File analysis –Compute file hash –Detect file type –Keyword to match on file content: file_data Suricata 1.3

Date: 2012/07

TLS support

−IDS is not a network grep

−Analyse TLS handshake

−Produce Record

−Dedicated keywords: •tls.cert_subject •tls.cert_issuer •... TLS support by the example

Verify PKI usage in a company

The lazy way: one signature alert tls $SERVERS any → any any ( msg:”non PKI certificate”; tls.cert_issuer; content:!”ou=My Company”; classification:policy; sid:1; rev:1; ) Suricata 1.4

Date: 2012/12

How to deal with rapid evolution of nature of threats

−Signature language is release dependent

−Expressiveness of the language is poor

Lua signature

−Detect unexpected threats

−Implement complex algorithm Heartbleed example

Context

−Exploit based on an obscure message of TLS RFC

−Suricata support was not exposing this message to detection

Lua signature has been written a few hours after announcement

−Implement parsing code ‘like ’

−Strict test for abnormal behavior

Next release of Suricata had direct support Entropy computing in Lua

function entropy(a, verbose) function init (args) local result = {} l = #a local needs = {} V={} for i = 1, 256, 1 do needs["buffer"] = tostring(true) V[i] = 0 end return needs for i = 1, l, 1 do V[a:byte(i) + 1] = V[a:byte(i) + 1] + 1

end end c = 0

log2 = math.log(2)

for i = 1, 256, 1 do

p = V[i]/l function match(args) if p > 0 then c = c - (p * math.log(p)/log2) buffer = args["buffer"]; end end

--Ceil

bo = string.gsub(buffer, "(.*\\)(.*).exe", "%2"); if c > 4 then

return 1 return entropy(bo, 1) end return 0 end end Signature using entropy alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"High entropy in HTTP filename exe download"; flow:to_client,established; http.header; content:"Content-Transfer-Encoding|3a| binary"; http_header_names; content:"|0d 0a|Content-Disposition|0d 0a|Content-Transfer-Encoding|0d 0a|"; file.name; content:".exe"; endswith; fast_pattern; lua:entropy-detect.lua; sid:666; rev:1;) A real life result

"http": { "hostname": "d2k1gzykfsah3o.cloudfront.net", "url": "/d1zei|m28l4au/adobe_flash_player.exe", "http_user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0", "http_content_type": "application/octet-stream", "http_method": "GET", "protocol": "HTTP/1.1", "status": 200, "length": 2235 Suricata 2.0

Date: 2014/03

Bring interoperability −Get rid of file parser hell −Allow rapid extension of events −Follow switch from custom tool to generic log handling tools

JSON output −For NSM data −For Alerts

Include basic information

Enrich with protocol metadata Alert in JSON format Banker malware:spy.banker.ACUT (sid 2024099)

The signature

●alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET TROJAN Win32/Spy.Banker.ACUT CnC Checkin"; flow:established,to_server; content:"POST"; http_method; content:".php"; http_uri; content:"User-Agent|3a 20|Mozilla/3.0 (compatible|3b 20|Indy Library)|0d 0a|"; http_header; fast_pattern:12,20; content:"plugin="; http_client_body; depth:7; content:"&windows="; http_client_body; content:"&user="; http_client_body; content:"&av="; http_client_body; content:"&bs="; http_client_body; content:!"Referer|3a|"; http_header; sid:2024099; rev:2;) Sample

Exfiltrate information Username, , Antimalware presence Pimp your Banker malware

Let’s get these data They are in the POST We can extract them Just need to extend the signature Capture the value Add them to the JSON Let’s check the manual Or Victor Julien’s blog: https://blog.inliniac.net/2016/12/20/suricata-bits-ints-and-vars/ Banker malware: orange like a pcre

●Convert to Suricata 5.0

●Not needed but more convenient

●The basic match:

●http.request_body; content:"plugin="; depth:7; content:"&windows="; content:"&user="; content:"&av="; content:"&bs=";

●Add information via additional pcre keywords

●pcre:"/&user=(.+)&/G,flow:username";

●pcre:"/&windows=(.+)&/G,flow:windows";

●pcre:"/&av=(.+)&/G,flow:av"; Banker malware: enjoy

Additional information available in the alert event: Suricata 3.0

Date: 2016/01

Mitigate multi steps attacks

−Xbits to cross flow boundary Suricata 4.0

Date: 2017/07

Limit risk of compromission

−Attack on the engine itself

−Suricata parses worse user data ever

Introduction of Rust

−For protocol parsing via Nom a parser writing framework Suricata 4.1

Date: 2018/11

Continue to see things in a TLS world

−Samba addition

−TLS JA3

Support growing network speed

−eXtreme Data Path

−Bypass TLS JA3

Identify TLS client implementation

Developed by 3 JA at Salesforce The Elephant flow problem The Elephant flow problem The Elephant flow problem XDP

●eBPF

●Extended Berkeley Packet Filter

●Share data structure between user space and kernel space ●Program filter in C code ●Available in different kernel hook

●Extreme Data Path

●Run eBPF code in network driver or earlier

●On each packet

●Take decision

●Accept

●Drop

●Transfer XDP bypass

Maintain a flow table

−Shared between kernel and user spaces

Suricata adds flow to bypass

Kernel

−Drop packet from flow in table

−Do accounting Impact on performance Suricata 5.0

Date: 2019/10

Identify TLS server too

−TLS JA3s

Data sharing time with MISP and other software

−List handling for IOC and more

−Switch to sticky buffer and datasets IOC checking

●Load lists and use them in signatures:

●alert dns any any → any any (msg:”bad domain (via DNS)”; dns.query; dataset:isset,dns-bl;)

●alert http any any → any any (msg:”bad domain (via HTTP)”; http.host; dataset:isset,dns-bl;)

●Incremental update via unix socket:

●dataset-add myset string Z29vZ2xlLmNvbQ==

●Available on over 40 diff fields

●any sticky buffer

●Update for packet path

●Alert dns any any → any any (dns.query; to_sha256; dataset:set,dns-sha256-seen; noalert;) Build dataset from data path

Http user agent

−alert http any any -> any any (msg:"First seen HTTP user agent"; http.user_agent; dataset:isnotset,http-user-agent,type string,state http-user-agent.lst; dataset:set,http-user-agent,type string,state http-user-agent.lst; sid:1; rev:1;) Resulting alert events Building our data set

JQ to the rescue

−cat eve.json | jq 'select(.event_type=="alert")|{"first_seen":.timestamp, "user- agent":.http.http_user_agent}' -c Connect with Suricata

●Mob league ●Suricon

●Bring Suricata to academics ●Yearly conference (Boston 2020)

●Scholarship to attend Suricata ●From developer to user

●Outreachy

●Training

●Threat hunting with Suricata

●Advanced deployment

●On prem

●Online

●Sig Dev

●Developer Conclusion

●Suricata is a well established engine ●Bring it on

●Evolving since 10 years ●Feedback

●Community driven development ●Bug report

●Contributions Questions ?

●Suricata & OISF: ●Stamus Networks

●Homepage: http://www.suricata-ids.org/ ●https://www.stamus-networks.com/

●OISF: https://www.oisf.net/ ●Peter Manev

●Online doc: https://suricata.readthedocs.io/ ●@pevma on Twitter

● Suricon: https://suricon.net/ ●Eric Leblond

●Try Suricata: ●@regiteric on Twitter

●SELKS: https://www.stamus- networks.com/scirius-open-source

●Security Onion: https://securityonion.net/