Department Informatik Technical Reports / ISSN 2191-5008 Paul Prade, Tobias Groß, Andreas Dewald Forensic Analysis of the Resilient File System (ReFS) Version 3.4 Technical Report CS-2019-05 December 2019 Please cite as: Paul Prade, Tobias Groß, Andreas Dewald, “Forensic Analysis of the Resilient File System (ReFS) Version 3.4,” Friedrich-Alexander-Universitat¨ Erlangen-Nurnberg,¨ Dept. of Computer Science, Technical Reports, CS-2019-05, December 2019. Friedrich-Alexander-Universitat¨ Erlangen-Nurnberg¨ Department Informatik Martensstr. 3 91058 Erlangen Germany · · www.cs.fau.de Forensic Analysis of the Resilient File System (ReFS) Version 3.4 Paul Prade1, Tobias Groß1 and Andreas Dewald1, 2 1Friedrich-Alexander University, Erlangen-Nuremberg, Germany 2ERNW Research GmbH, Heidelberg, Germany [email protected], [email protected] Abstract ReFS is a modern file system that is developed by Microsoft and its internal structures and behavior is not officially documented. Even so there exist some analysis efforts in deciphering its data structures, some of these findings have yet become deprecated and cannot be applied to current ReFS versions anymore. In this work, general concepts and internal structures found in ReFS are examined and documented. Based on the structures and the processes by which they are modified, approaches to recover (deleted) files from ReFS formatted file systems are shown. We also evaluated our implementation and the allocation strategy of ReFS with respect to accuracy, runtime and the ability to recover older file states. In addition, we extended The Sleuth Kit allowing it to parse ReFS partitions and build a carver based on that extend The Sleuth Kit. i List of Tables 2.1 Data categories........................................ 16 2.2 Postfixes of the names of file system tools.......................... 16 4.1 List of default ReFS attribute types.............................. 33 4.2 Data structure in the first 24 bytes of the boot sector (FSRS)................ 49 4.3 Data structure contained in the remainder of the boot sector................. 49 4.4 Structure of a page header.................................. 51 4.5 Structure of a page reference................................. 52 4.6 Structure of the superblock.................................. 53 4.7 Structure of the checkpoint.................................. 55 4.8 Structure of the index root element.............................. 57 4.9 Structure of the index header................................. 58 4.10 Structure of the header of a regular index entry....................... 58 4.11 Rows found in the Schema Table............................... 60 4.12 Rows found in the Object ID Table.............................. 60 4.13 Variable index root element used in Allocator Tables.................... 61 4.14 Rows found in Allocator Tables............................... 61 4.15 Bitmap structure used in Allocator Tables.......................... 61 4.16 Rows found in the Container Table.............................. 62 4.17 Rows found in the Parent Child Table............................ 62 4.18 Rows found in the Upcase Table............................... 62 4.19 Structure of the buffer found in the Logfile Information Table................ 63 4.20 Structure of the “Volume Label” row in the Volume Information Table........... 63 4.21 Structure of the “General Information” row in the Volume Information Table........ 63 4.22 Structure of the “General Information backup block” row in the Volume Information Table 64 4.23 Rows found in the Security Table............................... 64 4.24 Rows found in the Reparse Index Table........................... 64 4.25 Structure of the “Directory Descriptor” row type....................... 65 4.26 Structure of the “ID2” row type................................ 65 4.27 Structure of the “File” row type................................ 65 4.28 Structure of the “Directory Link” row type.......................... 66 4.29 Variable index root element used in Directory Descriptor tables............... 66 4.30 Variable index root element used in File tables........................ 67 4.31 Structure of the key in an attribute row............................ 68 4.32 Structure of the header of a standalone attribute....................... 68 4.33 Structure of the $DIR_LINK attribute type......................... 68 4.34 Structure of the $INDEX_ROOT attribute type........................ 69 4.35 Structure of the $NAMED_DATA attribute type........................ 69 4.36 Structure of the $REPARSE_POINT attribute type..................... 69 4.37 Structure of the $USN_INFO attribute type......................... 69 4.38 Rows found in the Data Run Table.............................. 70 4.39 Structure of a log page in the restart area........................... 76 4.40 Structure of a log page in the logging area.......................... 77 4.41 Structure of the redo information found in a log page in the logging area.......... 78 ii List of Tables 4.42 Structure of a table path component in a redo operation................... 78 6.1 Overview of the created test data sets............................. 91 6.2 Output of the TSK extension, compared to the final state of the file system......... 91 6.3 Final state of the file system, compared to the output of the TSK extension......... 92 6.4 State of all allocated files according to the TSK extension, compared to all actions..... 92 6.5 State of all files that could be recovered with the TSK extension compared to all actions.. 93 6.6 State of all files that could be recovered with the carver application compared to all actions 93 6.7 Runtimes of the different applications............................ 94 6.8 Experiment to analyze the recoverability of COW copies.................. 94 iii List of Figures 2.1 NTFS structure, based on [35, p. 7] and [12, p. 275].....................6 2.2 Corruption in B+-trees, according to [44]..........................8 2.3 (a) A basic b-tree (b) Inserting key 19, and creating a path of modified pages [39, p. 7]... 11 2.4 (a) A basic tree (b) Deleting key 6. [39, p. 7]......................... 11 2.5 Experiment to show, that Copy-On-Write allocations make full page copies........ 12 2.6 Abstraction layers used in The Sleuth Kit (TSK), based on [5]............... 14 4.1 Overview of the checkpoint mechanism........................... 25 4.2 Global root nodes referenced by the checkpoint....................... 26 4.3 Implementation of a table in a key-value store by using a B+-tree.............. 27 4.4 Concept of embedding tables................................. 27 4.5 Tables referenced by the Object ID Table........................... 28 4.6 Schematic view of the organization of the Object ID Table................. 29 4.7 Exemplary directory table................................... 30 4.8 Overview of the interaction between the Object ID Table and directory tables....... 32 4.9 Tables referenced by the checkpoint structure........................ 35 4.10 Exemplary view of the Upcase Table............................. 35 4.11 Overview of the Volume Information Table......................... 36 4.12 Exemplary Parent Child Table with its interpretation..................... 37 4.13 Interaction between the hierarchical allocators, according to [22, p. 41]........... 38 4.14 Schematic view of bands found in ReFS........................... 39 4.15 Necessity of the address translation process......................... 40 4.16 Exemplary address translation process............................ 41 4.17 Exemplary usage of the Block Reference Count Table.................... 41 4.18 Overview of the logging areas used in ReFS......................... 46 4.19 Example for an inconsistent checkpoint state (redo operations are necessary)........ 47 4.20 Example for a consistent checkpoint state (no redo operations are necessary)........ 47 4.21 Implementation to calculate the checksum of the FSRS................... 49 4.22 Example layout of the FSRS structure, explained in table 4.2................ 50 4.23 Example layout of the remaining boot sector, explained in table 4.3............. 50 4.24 Example layout of a page header, as explained in table 4.4................. 51 4.25 Example layout of a page reference structure, as explained in table 4.5........... 51 4.27 Calculation of a volume signature.............................. 53 4.26 Example layout of a superblock structure, as explained in table 4.6............. 54 4.28 Example layout of the first 144 bytes of a checkpoint structure, as explained in table 4.7.. 56 4.29 Example layout of the pointer list found in a checkpoint structure, as explained in table 4.7 56 4.30 Overview of the components of a Minstore B+ node.................... 57 4.32 Example layout of an Index Header structure, according to 4.9............... 58 4.33 Example layout of a key index structure........................... 58 4.31 Example layout of an index root structure, according to table 4.8.............. 59 4.34 Example layout of an index entry structure, according to 4.10................ 59 4.35 Example for the deletion of the entries e2 and e1 ....................... 71 4.36 Example for the deletion of the entries e1 and e2 ....................... 72 4.37 Example for the deletion of the entry e3 ........................... 72 4.38 Example for the insertion of the entry e5 ........................... 72 4.39 Example for the insertion of the entry e4 ........................... 73 4.40 Example for the remaining traces after a push operation................... 74