Basics of Windows OS for Reverse Engineering Malware
Total Page:16
File Type:pdf, Size:1020Kb
Basics of Windows OS for Reverse Engineering Malware Protecting the irreplaceable | f-secure.com Copyright F-Secure 2010. All rights reserved. Applications on Windows 2 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Executable Format • Object files and executables follow the PE (Portable Executable) file format • Full specification available online • http://www.microsoft.com/whdc/system/platform/ firmware/PECOFF.mspx • Best viewed with your hex editor (HT) or specialized PE viewer (PEBrowsePro ->) 3 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Windows Executables • Filename extension hints to the executable type • EXE = executable application, anything from a DOS executable to 64-bit Windows applications • DLL = Dynamic link library, a set of callable routines compiled together as a loadable file • SYS = Driver • OBJ = Object file, input to the linker • Note that Windows does not really care much about the file extension • You can execute application.jpg just fine • All of these follow the PE/COFF specification • APPX is used for Windows 8 Windows Store apps • Not a PE, but a ZIP archive with all application contents inside 4 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Windows API • Windows API (aka. Win32 API) is the interface to the operating system for applications • Exposed by a set of system libraries: kernel32.dll, user32.dll, … • Windows 7 refactored the system libraries (“MinWin”) so you will see e.g. kernelbase.dll • On 64-bit, because of 32-bit compatibilty, c:\windows\syswow64 contains another set of libraries • Several subcategories • Administration and management (WMI, …) • Diagnostics (event logging, …) • Networking • Security • System services (processes, threads, registry…) • MSDN is the reverse engineers best friend for Windows binaries • http://msdn2.microsoft.com/en-us/library/default.aspx 5 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Native API • Undocumented interface to OS functionality • One level below Windows API • Some low-level functionality only available through Native API • Examples of interesting functions • NtSetSystemInformation • NtQuerySystemInformation • NtQueryDirectoryFile • See “Windows NT/2000 Native API Reference” by Nebbett 6 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Windows Architecture 7 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Simplified Windows Architecture System Support Service User Environment Processes Processes Applications Subsystems Subsystem DLL’s NTDLL.DLL Executive Windowing & Graphics Kernel Device Drivers HAL 8 February 18, 2013 Copyright F-Secure 2010. All rights reserved. System Mechanisms 9 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Kernel-mode & user-mode 0xFFFFFFFF System-space (Ring 0) Int 0x2e / Sysenter User-space User-space (RingUser -3)space (Ring 3) (Ring 3) 0x00000000 10 February 18, 2013 Copyright F-Secure 2010. All rights reserved. System Service Dispatching Read the file Nt! Return to caller NtReadFile Call NtReadFile Interrupt Nt! Dismiss interrupt KiSystemService Sysenter Ntdll.dll! Return to caller NtReadFile Call NtReadFile Kernel32.dll! Return to caller ReadFile ReadFile(…) Application 11 February 18, 2013 Copyright F-Secure 2010. All rights reserved. System Service Dispatching 12 February 18, 2013 Copyright F-Secure 2010. All rights reserved. System Service Dispatching 13 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Processes & Threads 14 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Processes • Abstraction of an executing program • A process has • A private virtual address space (user-mode address space) • A base image (the main executable) • A set of open handles to OS resources • An access token (“who is this process, what can it do”) • A process ID (PID) • One or more threads • Job = A group of processes that can be managed as a unit 15 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Threads • A thread is what gets scheduled for execution on the CPU • A thread has • A context (the state of execution) • A list of exception handlers • Two stacks, one for user-mode and one for kernel-mode • A thread ID (TID) • An access token • Thread-Local Storage (TLS), a per-thread storage area • Fiber = Manually scheduled unit of execution, shares the context of its owning thread (ConvertThreadToFiber, CreateFiber, SwitchToFiber) 16 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Processes & Threads 17 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Demo: Processes and Threads 18 February 18, 2013 Case Study: FU Rootkit • The FU rootkit hides processes by unlinking them from the kernels process list • Why do those processes still get execution time? 19 February 18, 2013 © F-Secure Confidential Copyright F-Secure 2010. All rights reserved. Case Study: DLL Injection • A technique to run code in the context of another process by forcing it to load a DLL • Motivation 1. Stealth: no extra processes in process list 2. Data capture using API hooks (e.g. banking trojans) 3. Making reverse engineering more difficult 4. Avoiding HIPS features (outbound network connectivity from trusted process) • Various ways to accomplish DLL injection on Windows • Registry: AppInit_DLLs • Using remote threads • SetWindowsHookEx • From the kernel using APC’s 20 February 18, 2013 Copyright F-Secure 2010. All rights reserved. What does this code do? 21 February 18, 2013 Copyright F-Secure 2010. All rights reserved. TEB & PEB • TEB = Thread Environment Block • Container for thread-specific things like the exception handler list, stack pointer, … • Windows uses the fs segment to store it (offset 0x18 has pointer to self) • mov eax, fs[18] • PEB = Process Environment Block • Container for process-specific things like the list of loaded modules • TEB has a pointer to PEB at offset 0x30 • Important when reversing code that • Enumerates loaded modules (Peb.Ldr) • Checks for an attached debugger (PEB.BeingDebugged) • Installs an exception handler (TEB.NtTib.ExceptionList) 22 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Example: Checking For a Debugger ; Call IsDebuggerPresent() call [IsDebuggerPresent] test eax, eax ; Do the same by checking PEB mov eax, large fs:18h ; Offset 18h has self-pointer to TEB mov eax, [eax+30h] ; Offset 30h has pointer to PEB movzx eax, byte ptr [eax+2] ; PEB.BeingDebugged test eax, eax 23 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Example: Installing an Exception Handler ; Install a SEH exception handler push offset_my_handler ; pointer to our handler push fs:[0] ; pointer to old exception record mov fs:[0], esp ; update TEB.NtTib.ExceptionList 24 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Memory Management 25 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Memory Manager • Each process on Windows sees a large, continuous address space • The memory manager has two important tasks 1. Mapping access to virtual memory into physical memory 2. Paging contents of virtual memory to disk as physical memory runs out, and paging data back to memory when it’s needed 26 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Virtual Memory • Each process has private, virtual address space • Paging = the process of transferring memory contents to and from disk • Amount of virtual memory can exceed physical memory 27 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Virtual Memory (x86) • Total of 4 GB virtual memory • By default, only lower 2 GB accessible from user-mode • Upper 2 GB reserved for kernel-mode and shared between all processes 28 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Management Mechanisms 29 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Registry • A directory for storing settings and configuration data for the Operating System and applications • Think of it as a huge .ini file • Basic concepts • Hive • Key • Value • Hives are just files, most under %SystemRoot%\System32\Config 30 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Registry Hive Format 31 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Registry Root Keys • HKEY_LOCAL_MACHINE (HKLM) • System-wide information • HKEY_USERS (HKU) • User-specific settings for all accounts • HKEY_CURRENT_USER (HKCU) • A link to the current user under HKEY_USERS • HKEY_CLASSES_ROOT (HKCR) • File associations and COM registration, links to HKLM\Software\Classes • HKEY_PERFORMANCE_DATA • HKEY_CURRENT_CONFIG • Current HW profile, links to HKLM\System\CurrentControlSet\Hardware Profiles\Current 32 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Registry and Malware • Malware wants to survive a reboot • Registry is the most common way of starting after reboot • Hundreds of “launchpoints” • HKLM\Software\Microsoft\Windows\CurrentVersion\Run:MyApp • HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\explorer.exe:Debugger • Malware also wants to change (security) settings for other components • Windows Firewall, IE extensions and settings, Windows File Protection, … • The registry is also a great source for forensic data, for example: • HKEY_CURRENT_USER\Software\Microsoft\Windows\ShellNoRoam\MUICache • HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\ UserAssist 33 February 18, 2013 Copyright F-Secure 2010. All rights reserved. Services • Services are background processes