Extending The Sleuth Kit and its Underlying Model for Pooled Storage File System Forensic Analysis
Fraunhofer Institute for Communication, Information Processing and Ergonomics
Jan-Niclas Hilgert* [email protected] Martin Lambertz [email protected] Daniel Plohmann [email protected]
© Fraunhofer FKIE Digital Forensic Analysis
Slide 2 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE File Systems
n Define how data is read from and written to a storage device n Utilize metadata to keep order of the data stored n File systems differ in many aspects
EXT2FS HFS NTFS
n Extensive background knowledge is required for a forensic analysis n But not always existent
Slide 3 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE File System Forensic Analysis De-Facto Standard Model
Physical Storage Media Sectors of Data Device Analysis
Volume Volume n Universal model for a file system Analysis forensic analysis n Presented by Brian Carrier in 2005 File System n Consists of four interdependent steps File Analysis n Implemented in The Sleuth Kit
Application Analysis
Slide 4 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE File System Forensic Analysis De-Facto Standard Model
Physical Storage Media Sectors of Data Device Analysis n Works great on a lot of established file systems Volume Volume Analysis EXT2FS
NTFS File System HFS File Analysis n Problem: It hasn’t been reviewed or changed since its publication Application Analysis
Slide 5 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE File System Forensic Analysis Trying TSK on ZFS
jhilgert:ZPool$ fsstat disk1 Cannot determine file system type
Slide 6 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Pooled Storage File Systems Old File System Mapping
FAT32 NTFS Ext4
Block Device Block Device Block Device
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device n Storage devices (hard drives, solid state drives ...) are somehow used to create new block devices n One file system is assigned to one of these block devices
Slide 7 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Pooled Storage File Systems New Approach
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device n Storage Devices are combined to form a storage pool n Its configuration depends completely on the implementation n File systems share the available space from the storage pool
Slide 8 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Requirements – Detect Pool Members
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device
Slide 9 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Requirements – Detect Pool Members
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device
Slide 10 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Requirements – Detect Pool Configurations
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device
Slide 11 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Requirements – Analyze a Complete Pool
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device
Slide 12 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Requirements – Access Correct Offsets on Physical Disks
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device
Slide 13 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Requirements – Access All Filesystem Data
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device
Slide 14 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Requirements – Deal with Incomplete Pools
Filesystem Filesystem Filesystem
Storage Pool
StorageStorage StorageStorage StorageStorage Storage Storage Storage DeviceDevice DeviceDevice DeviceDevice Device Device Device
Slide 15 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE File System Forensic Analysis Phyiscal Media Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis n Acquisition of data from storage devices n Data is only seen as a sequence of bytes n No change necessary File System Analysis File
Application Analysis
Slide 16 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE File System Forensic Analysis Application Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis n Performs an application-level analysis of the acquired files n Works on already recovered and File System File collected files Analysis n No change necessary
Application Analysis
Slide 17 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Volume Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis n Data is searched for its underlying volume structure n Returns detected volumes (block devices) File System Analysis File
Application Analysis
Slide 18 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Volume Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis n Pooled storage file systems implement their own “volume manager” n Established volume managers are still used File System Analysis File n Volumes can also be part of a pool n Volume analysis is still necessary
Application Analysis
Slide 19 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Pool Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis
Reconstructed File PoolSystem Pool Analysis File n Volumes or data need to be analyzed ApplicationFile System for possible pool membership File Analysis n Using the file systems volume manager results in a reconstructed pool
n Only high-level access Application Analysis n No file system analysis possible
Slide 20 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Pool Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis
Reconstructed Pool Pool (Direct Access) Pool Analysis Pool Member n File system data and metadata is File System essential for a file system analysis File Analysis n Direct access to the pool and its members is required
n Mapping schema of the file Application system needs to be known Analysis
Slide 21 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Pool Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis
Reconstructed Pool Pool (Direct Access) Pool Analysis Pool Member n Volumes do not need to be part of a pool File System File n Important: Pool analysis is file system Analysis dependent
Application Analysis
Slide 22 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model File System Analysis Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis
Reconstructed Pool Pool (Direct Access) Pool Analysis Pool Member n Each volume is searched for a file system File System File n File system analysis can be performed Analysis on the pool with direct access
Application Analysis
Slide 23 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Extension of the Standard Model Extended Model Physical Storage Media Sectors of Data Device Analysis
Volume Volume Analysis
Reconstructed Pool Pool (Direct Access) Pool Analysis Pool Member n Prototypically implemented for ZFS File System File Analysis
Application Analysis
Slide 24 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Pooled Storage File Systems ZFS – Volume Management
n First presented in 2003 (more than a decade ago) n Available for major platforms like Solaris, FreeBSD, Linux, MacOS n ZFS combines multiple volumes into a storage pool n Pool members are identified by four vdev-labels
L1 L2 Data L3 L4
Storage Device (Vdev)
Slide 25 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Pooled Storage File Systems ZFS – Volume Management
n Pools in ZFS consist of one or more top-level virtual devices (vdevs) n Data is striped across all of these top-level vdevs
Data
ZFS Pool
Top-Level Top-Level Top-Level Vdev Vdev Vdev
Slide 26 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Pooled Storage File Systems ZFS – Volume Management
n Top-level vdevs are made up of children n Different types of top-level vdevs are supported by ZFS n Each top-level vdev is addressed using its ID
ZFS Pool
Top-Level Top-Level Top-Level Mirror Disk / File RaidZ Vdev ID: 0 Vdev ID: 1 Vdev ID: 2
Child Child Child Child Child Child
Slide 27 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Pooled Storage File Systems ZFS – General Structure
n Filesystem is represented in a tree-structure n Überblock is the head of this tree Überblock Überblock n Utilizes the copy-on-write principle Metadata Metadata Metadata
Data Data Data Data Data n Block pointers refer to variable-sized blocks of data n Data is addressed by using Data Virtual Addresses (DVAs)
Top-level Vdevs Mapping Offsets
Slide 28 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Evaluation Evaluation Pool
n Use multiple numbers of disks in multiple configurations n Two simple disks n One mirror with two children n One raidz1 with three children
EvaluationPool
disk disk mirror raidz1
Slide 29 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Contribution & Future Research
n Extension of The Sleuth Kit for pooled storage file systems n Digital forensic analysis of ZFS n Accessing older versions of the ZFS copy-on-write tree n Dealing with incomplete pools n Further investigations on other ZFS structures: n ZFS Intent Log (ZIL) n L2ARC n Implementations for other pooled storage file systems n BTRFS, ReFS, APFS
Slide 30 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE Thanks for your attention!
https://github.com/fkie-cad/sleuthkit
Slide 31 August 8th, 2017 © Cyber Analysis & Defense Department, Fraunhofer FKIE