InnoDB Database Forensics Peter Frühwirt, Markus Huber Martin Mulazzani, Edgar R. Weippl Vienna University of Technology SBA Research Vienna, Austria Vienna, Austria [email protected] {mmulazzani|[email protected]} Abstract— Whenever data is being processed, there are many Clearly, database systems are bound to leave more places where parts of the data are temporarily stored; thus extensive traces than “normal” files as they not only store forensic analysis can reveal past activities, create a (partial) files but, in addition, need indexes, rollback segments and timeline and recover deleted data. While this fact is well known log files. Therefore, the analysis of the structure in the for computer forensics, multiple forensic tools exist to analyze database management system is a precondition for forensic data and the systematic analysis of database systems has only analysis [9, 10, 11]. recently begun. This paper will describe the file format of the To analyze the data, it is important to know and MySQL Database 5.1.32 with InnoDB Storage Engine. It will understand in detail, how the database is built. Only with this further explain with a practical example of how to reconstruct basic knowledge can one prove the consistency of the data. the data found in the file system of any SQL table. We will The goal of this paper is to explain how to analyze the show how to reconstruct the table as it is, read data sets from the file and how to interpret the gained information. database system MySQL with the storage engine InnoDB. This paper shows the structure and architecture of this Keywords- MySQL, InnoDB, Database and Forensic database system and how one can reconstruct data from the files in the file system. Based on these findings we provide an application that reads and interprets this basic data to I. INTRODUCTION facilitate the detection of any unauthorized manipulation. Today’s society would be unimaginable without database In the first section of this paper we will explain the basis systems. The information forms an important commodity. of a simple SQL-table and how all relevant information like Most of the information is stored and processed in databases. keys, types and others are saved and how they can be read Most of this information can be very precious for its owner from the files. In the second part of this paper we will show as it concerns data regarding patients or customer how data in tables can be saved with the help of the InnoDB information, which is covered by the relevant data storage in the files and how to interpret this data. We refer to protection. The results and information gained from database the MySQL source code in all our explanations1. While the forensics can be used for several reasons. First, for many MySQL documentation (such as [18-22]) provides a good companies it is relevant to know if the integrity of their data starting point; none of the presented material has to the best has been compromised or users’ privacy has been violated of our knowledge been documented anywhere else. [14]. This means that they need or want to prove, whether or not, their databases has been tampered with. Second, results II. SQL INTERNALS of database forensics can be used to detect and analyze Storage engines of MySQL store all information of each attacks, understand which vulnerabilities were exploited and single table in a .frm file in the directory of the database. The to develop preventive countermeasures. Third, modern file filename is the name of the table [1]. Each .frm file is systems develop in the direction of database systems and created by the function create_frm() in the file /sql/table.cc thus database forensic will also become important for file [Appendix 1]. The file size of a single .frm file is limited to forensics. 4 GB. If this limit is reached MySQL will automatically Common file systems, such as FAT, NTFS, HFS+ or truncate the file to prevent errors [13,16]. ext2/ext3 are well understood in the forensic community [8] and are supported by many forensic tools, for example A. General Table Information EnCase [23] or The Sleuth Kit [24]. Exotic or new file All general information of the file, such as the used systems are much harder to analyze as these tools do not version number, is saved at the first bytes of the .frm file. support them. This makes it increasingly difficult as new file Table 1 shows the structure of the first bytes of a .frm file systems are in constant development. Recent research has starting at position 0x0. Table 2 explains the meaning of the been focusing on more advanced file systems that partially bytes. All references to the line numbers of the source code rely on data structures found in database systems. These refer to the file /sql/table.cc [Appendix 1]. include IBM’s JFS [4] or Journaling File Systems [5], ext3 and reiserFS [15]. New and upcoming file systems include Btrfs and ext4, which are already included in many Linux distributions. Btrfs is based on the well known data structure 1 All information of the data structures and source code refers to the called B-tree [3] that is also used in database indices [2]. MySQL database management system in the version 5.1.32 for apple- darwin9.5.0 on i386 (Mac OS X 10.5.7 Leopard). TABLE I. HEXADECIMAL STRUCTURE OF .FRM FILE (0X00000000 - 0X0000004F) 0x22 4 00 00 create_info->avg_row_length 2490 00 00 0x00000000 FE 01 0A 0C 03 00 00 10 01 00 00 30 00 00 69 01 ..........0..i. 0x00000010 10 01 00 00 00 00 00 00 00 00 00 02 21 00 09 00 ...........!... 0x26 1 30 create_info->default_table_charset 2491-2492 0x00000020 00 05 00 00 00 00 30 00 02 00 00 00 00 00 00 69 ......0.......i 0x00000030 01 00 00 D4 C3 00 00 10 00 00 00 00 00 00 00 00 ............... 0x27 1 00 create_info->transactional | 2493 0x00000040 2F 2F 00 00 20 00 00 00 00 00 00 00 00 00 00 00 //.. .......... create_info->page_checksum << 2 TABLE II. EXPLANATION AND MEANING OF THE HEXADECIMAL 0x28 1 02 create_info->row_type 2495 VALUES OF THE .FRM FILE (0X00000000 - 0X00000040) 0x29 6 00 00 RAID Support 2496-2502 Offset Length Value Meaning Source 00 00 Code 00 00 0x00 1 FE fixed value 2454 0x2F 3 69 01 key_length 2503 00 00 0x01 1 01 fixed value 2455 0x33 4 D4 MYSQL_VERSION_ID (only 2504-2505 0x02 1 0A FRM_VERSION 2456 C3 00 saved to prevent a warning, because (/include/mysql_version.h 00 of unaligned key_length of 3 bytes) [Appendix 2]) + 3 + test(create_info->varchar) 0x37 4 10 00 create_info->extra_size (length of 2506 00 00 extra data sequence) 0x03 1 0C Database type (sql/handler.h Z. 258- 2458-2459 279) [Appendix 3]) 0x3B 2 00 00 extra_rec_buf_length (reservation) - z.B.: DB_TYPE_MYISAM: 9, DB_TYPE_INNODB: 12 0x3D 1 00 default_part_db_type (reservation) - 0x04 1 03 unknown 0x3E 2 00 00 create_info->key_block_size 2511 0x05 2 00 00 unknown or undefined TABLE III. STRUCTURE OF A KEY DEFINED IN /SQL/UNIREG.CC 0x07 2 10 01 IO_SIZE (4096) 2461 [APPENDIX 4] Definition in include/my_global.h [Appendix 13] Length Meaning (all keys) 0x09 2 01 00 unknown 8 key header 0x0A 4 00 30 keylength 2474-2477 00 00 (IO_SIZE+key_length+reclength+ 9 key parts(MAX_REF_PARTS) create_info->extra_size). Table 3 shows exact meaning of the key NAME_LEN name of the key length (key_length) 1 separator NAMES_SEP_CHAR (before name of key) 0x0E 2 69 01 length of the temporary key, based 2478-2479 on key_length Length Meaning (single key) 0x10 2 10 01 length of the record 2480 6 key header 0x12 4 00 00 create_info->max_rows (definition 2481 1 separator NAMES_SEP_CHAR (after name of key) 00 00 of the create_info structure HA_CREATE_INFO in sql/handler.h [Appendix 3] in line 9 padding and alignment 896-924) 0x16 4 00 00 create_info->min_rows 2482 Further information will not be saved up to the fixed 00 00 address 0x1000. This space is filled with null-values. The keys of a table will be saved at position 0x1000. At this point 0x1A 1 00 unused or padding / alignment - we want to explain how one can reconstruct the following SQL table: 0x1B 1 02 fixed value (use long pack-fields) 1) CREATE TABLE `Project` ( `idProject` int(11) NOT NULL 0x1C 2 21 00 key_info_length 1) AUTO_INCREMENT, `name` varchar(255) NOT NULL, 0x1E 2 09 00 create_info->table_options 2486-2487 `deleted` tinyint(1) DEFAULT NULL, `startTime` date DEFAULT NULL, 0x20 1 00 fixed value 2488 `endTime` date DEFAULT NULL, 0x21 1 05 fixed value (frm version number: 5) 2489 `effort` int(10) DEFAULT NULL, PRIMARY KEY (`idProject`) ) ENGINE=InnoDB DEFAULT CHARSET=latin AUTO_INCREMENT=352; Offset Len Value Meaning Source Code Appendix 5 contains the Project.frm file of this table. 0x15 1 04 T key_part->length 545 length of the key parts in bytes S B. Primary Key Table 4 shows the stored primary key in the file 0x16 1 00 unused (padding, alignment) 546 Project.frm [Appendix 5]. This can be found starting at address 0x1000. 0x17 1 FF separator NAMES_SEP_CHAR 551/555 (before name of key) TABLE IV. HEXADECIMAL STRUCTURE OF .FRM FILE (0X00000000 - 0X0000004F) 0x00001000 01 01 00 00 0A 00 00 00 04 00 01 00 00 00 01 80 ............... 0x18 7 50 52 name of key (PRIMARY) 554 0x00001010 02 00 00 1B 00 04 00 FF 50 52 49 4D 41 52 59 FF .......PRIM ARY.