File System Forensics and Data Recovery
Total Page：16
File Type：pdf, Size：1020Kb
Load more
Recommended publications
-
Ext4 File System and Crash Consistency
1 Ext4 file system and crash consistency Changwoo Min 2 Summary of last lectures • Tools: building, exploring, and debugging Linux kernel • Core kernel infrastructure • Process management & scheduling • Interrupt & interrupt handler • Kernel synchronization • Memory management • Virtual file system • Page cache and page fault 3 Today: ext4 file system and crash consistency • File system in Linux kernel • Design considerations of a file system • History of file system • On-disk structure of Ext4 • File operations • Crash consistency 4 File system in Linux kernel User space application (ex: cp) User-space Syscalls: open, read, write, etc. Kernel-space VFS: Virtual File System Filesystems ext4 FAT32 JFFS2 Block layer Hardware Embedded Hard disk USB drive flash 5 What is a file system fundamentally? int main(int argc, char *argv[]) { int fd; char buffer[4096]; struct stat_buf; DIR *dir; struct dirent *entry; /* 1. Path name -> inode mapping */ fd = open("/home/lkp/hello.c" , O_RDONLY); /* 2. File offset -> disk block address mapping */ pread(fd, buffer, sizeof(buffer), 0); /* 3. File meta data operation */ fstat(fd, &stat_buf); printf("file size = %d\n", stat_buf.st_size); /* 4. Directory operation */ dir = opendir("/home"); entry = readdir(dir); printf("dir = %s\n", entry->d_name); return 0; } 6 Why do we care EXT4 file system? • Most widely-deployed file system • Default file system of major Linux distributions • File system used in Google data center • Default file system of Android kernel • Follows the traditional file system design 7 History of file system design 8 UFS (Unix File System) • The original UNIX file system • Design by Dennis Ritche and Ken Thompson (1974) • The first Linux file system (ext) and Minix FS has a similar layout 9 UFS (Unix File System) • Performance problem of UFS (and the first Linux file system) • Especially, long seek time between an inode and data block 10 FFS (Fast File System) • The file system of BSD UNIX • Designed by Marshall Kirk McKusick, et al. -
How UNIX Organizes and Accesses Files on Disk Why File Systems
UNIX File Systems How UNIX Organizes and Accesses Files on Disk Why File Systems • File system is a service which supports an abstract representation of the secondary storage to the OS • A file system organizes data logically for random access by the OS. • A virtual file system provides the interface between the data representation by the kernel to the user process and the data presentation to the kernel in memory. The file and directory system cache. • Because of the performance disparity between disk and CPU/memory, file system performance is the paramount issue for any OS Main memory vs. Secondary storage • Small (MB/GB) Large (GB/TB) • Expensive Cheap -2 -3 • Fast (10-6/10-7 sec) Slow (10 /10 sec) • Volatile Persistent Cannot be directly accessed • Directly accessible by CPU by CPU – Interface: (virtual) memory – Data should be first address brought into the main memory Secondary storage (disk) • A number of disks directly attached to the computer • Network attached disks accessible through a fast network - Storage Area Network (SAN) • Simple disks (IDE, SATA) have a described disk geometry. Sector size is the minimum read/write unit of data (usually 512Bytes) – Access: (#surface, #track, #sector) • Smart disks (SCSI, SAN, NAS) hide the internal disk layout using a controller type function – Access: (#sector) • Moving arm assembly (Seek) is expensive – Sequential access is x100 times faster than the random access Internal disk structure • Disk structure User Process Accessing Data • Given the file name. Get to the file’s FCB using the file system catalog (Open, Close, Set_Attribute) • The catalog maps a file name to the FCB – Checks permissions • file_handle=open(file_name): – search the catalog and bring FCB into the memory – UNIX: in-memory FCB: in-core i-node • Use the FCB to get to the desired offset within the file data: (CREATE, DELETE, SEEK, TRUNCATE) • close(file_handle): release FCB from memory Catalog Organization (Directories) • In UNIX, special files (not special device files) called directories contain information about other files. -
Imagemounter Documentation Release 1.5.1
imagemounter Documentation Release 1.5.1 Ralph Broenink, Peter Wagenaar December 11, 2016 Contents 1 Contents 3 1.1 Installation................................................3 1.2 Command-line usage...........................................4 1.3 Python interface.............................................7 1.4 File and volume system specifics.................................... 18 1.5 Release notes............................................... 22 Python Module Index 31 i ii imagemounter Documentation, Release 1.5.1 imagemounter is a command-line utility and Python package to ease the mounting and unmounting of EnCase, Affuse, vmdk and dd disk images (and other formats supported by supported tools). It supports mounting disk images using xmount (with optional RW cache), affuse, ewfmount and vmware-mount; detecting DOS, BSD, Sun, Mac and GPT volume systems; mounting FAT, Ext, XFS UFS, HFS+, LUKS and NTFS volumes, in addition to some less known filesystems; detecting (nested) LVM volume systems and mounting its subvolumes; and reconstructing Linux Software RAID arrays. In its default mode, imagemounter will try to start mounting the base image on a temporary mount point, detect the volume system and then mount each volume seperately. If it fails finding a volume system, it will try to mount the entire image as a whole if it succeeds in detecting what it actually is. Note: Not all combinations of file and volume systems have been tested. If you encounter an issue, please try to change some of your arguments first, before creating a new GitHub -
File Systems
File Systems Profs. Bracy and Van Renesse based on slides by Prof. Sirer Storing Information • Applications could store information in the process address space • Why is this a bad idea? – Size is limited to size of virtual address space – The data is lost when the application terminates • Even when computer doesn’t crash! – Multiple process might want to access the same data File Systems • 3 criteria for long-term information storage: 1. Able to store very large amount of information 2. Information must survive the processes using it 3. Provide concurrent access to multiple processes • Solution: – Store information on disks in units called files – Files are persistent, only owner can delete it – Files are managed by the OS File Systems: How the OS manages files! File Naming • Motivation: Files abstract information stored on disk – You do not need to remember block, sector, … – We have human readable names • How does it work? – Process creates a file, and gives it a name • Other processes can access the file by that name – Naming conventions are OS dependent • Usually names as long as 255 characters is allowed • Windows names not case sensitive, UNIX family is File Extensions • Name divided into 2 parts: Name+Extension • On UNIX, extensions are not enforced by OS – Some applications might insist upon them • Think: .c, .h, .o, .s, etc. for C compiler • Windows attaches meaning to extensions – Tries to associate applications to file extensions File Access • Sequential access – read all bytes/records from the beginning – particularly convenient for magnetic tape • Random access – bytes/records read in any order – essential for database systems File Attributes • File-specific info maintained by the OS – File size, modification date, creation time, etc. -
Netinfo 2009-06-11 Netinfo 2009-06-11
Netinfo 2009-06-11 Netinfo 2009-06-11 Microsoft släppte 2009-06-09 tio uppdateringar som täpper till 31 stycken säkerhetshål i bland annat Windows, Internet Explorer, Word, Excel, Windows Search. 18 av buggfixarna är märkta som kritiska och elva av dem är märkta som viktiga, uppdateringarna finns för både servrar och arbetsstationer. Säkerhetsuppdateringarna finns tillgängliga på Windows Update. Den viktigaste säkerhetsuppdateringen av de som släpptes är den för Internet Explorer 8. Netinfo 2009-06-11 Security Updates available for Adobe Reader and Acrobat Release date: June 9, 2009 Affected software versions Adobe Reader 9.1.1 and earlier versions Adobe Acrobat Standard, Pro, and Pro Extended 9.1.1 and earlier versions Severity rating Adobe categorizes this as a critical update and recommends that users apply the update for their product installations. These vulnerabilities would cause the application to crash and could potentially allow an attacker to take control of the affected system. Netinfo 2009-06-11 SystemRescueCd Description: SystemRescueCd is a Linux system on a bootable CD-ROM for repairing your system and recovering your data after a crash. It aims to provide an easy way to carry out admin tasks on your computer, such as creating and editing the partitions of the hard disk. It contains a lot of system tools (parted, partimage, fstools, ...) and basic tools (editors, midnight commander, network tools). It is very easy to use: just boot the CDROM. The kernel supports most of the important file systems (ext2/ext3/ext4, reiserfs, reiser4, btrfs, xfs, jfs, vfat, ntfs, iso9660), as well as network filesystems (samba and nfs). -
File System Layout
File Systems Main Points • File layout • Directory layout • Reliability/durability Named Data in a File System index !le name directory !le number structure storage o"set o"set block Last Time: File System Design Constraints • For small files: – Small blocks for storage efficiency – Files used together should be stored together • For large files: – ConCguous allocaon for sequenCal access – Efficient lookup for random access • May not know at file creaon – Whether file will become small or large File System Design Opons FAT FFS NTFS Index Linked list Tree Tree structure (fixed, asym) (dynamic) granularity block block extent free space FAT array Bitmap Bitmap allocaon (fixed (file) locaon) Locality defragmentaon Block groups Extents + reserve Best fit space defrag MicrosoS File Allocaon Table (FAT) • Linked list index structure – Simple, easy to implement – SCll widely used (e.g., thumb drives) • File table: – Linear map of all blocks on disk – Each file a linked list of blocks FAT MFT Data Blocks 0 1 2 3 !le 9 block 3 4 5 6 7 8 9 !le 9 block 0 10 !le 9 block 1 11 !le 9 block 2 12 !le 12 block 0 13 14 15 16 !le 12 block 1 17 18 !le 9 block 4 19 20 FAT • Pros: – Easy to find free block – Easy to append to a file – Easy to delete a file • Cons: – Small file access is slow – Random access is very slow – Fragmentaon • File blocks for a given file may be scaered • Files in the same directory may be scaered • Problem becomes worse as disk fills Berkeley UNIX FFS (Fast File System) • inode table – Analogous to FAT table • inode – Metadata • File owner, access permissions, -
Lecture 03: Layering, Naming, and Filesystem Design
Lecture 03: Layering, Naming, and Filesystem Design Principles of Computer Systems Spring 2019 Stanford University Computer Science Department Lecturer: Chris Gregg PDF of this presentation 1 Lecture 03: Layering, Naming, and Filesystem Design Just like RAM, hard drives provide us with a contiguous stretch of memory where we can store information. Information in RAM is byte-addressable: even if you’re only trying to store a boolean (1 bit), you need to read an entire byte (8 bits) to retrieve that boolean from memory, and if you want to flip the boolean, you need to write the entire byte back to memory. A similar concept exists in the world of hard drives. Hard drives are divided into sectors (we'll assume 512 bytes), and are sector-addressable: you must read or write entire sectors, even if you’re only interested in a portion of each. Sectors are often 512 bytes in size, but not always. The size is determined by the physical drive and might be 1024 or 2048 bytes, or even some larger power of two if the drive is optimized to store a small number of large files (e.g. high definition videos for youtube.com) Conceptually, a hard drive might be viewed like this: Thanks to Ryan Eberhardt for the illustrations and the text used in these slides, and to Ryan and Jerry Cain for the content. 2 Lecture 03: Layering, Naming, and Filesystem Design The drive itself exports an API—a hardware API—that allows us to read a sector into main memory, or update an entire sector with a new payload. -
Forensic Toolkit (FTK)
Forensic Toolkit (FTK) User Guide | 1 AccessData Legal and Contact Information Document date: January 31, 2018 Legal Information ©2018 AccessData Group, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the publisher. AccessData Group, Inc. makes no representations or warranties with respect to the contents or use of this documentation, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, AccessData Group, Inc. reserves the right to revise this publication and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. Further, AccessData Group, Inc. makes no representations or warranties with respect to any software, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, AccessData Group, Inc. reserves the right to make changes to any and all parts of AccessData software, at any time, without any obligation to notify any person or entity of such changes. You may not export or re-export this product in violation of any applicable laws or regulations including, without limitation, U.S. export regulations or the laws of the country in which you reside. AccessData Group, Inc. 588 West 400 South Suite 350 Lindon, UT 84042 USA AccessData Trademarks and Copyright Information The following are either registered trademarks or trademarks of AccessData Group, Inc. All other trademarks are the property of their respective owners. AccessData® AD Summation® Mobile Phone Examiner Plus® AccessData Certified Examiner® (ACE®) Discovery Cracker® MPE+ Velocitor™ AD AccessData™ Distributed Network Attack® Password Recovery Toolkit® AD eDiscovery® DNA® PRTK® AD RTK™ Forensic Toolkit® (FTK®) Registry Viewer® LawDrop® Summation® | 2 A trademark symbol (®, ™, etc.) denotes an AccessData Group, Inc. -
Improving the Efficiency of Big Forensic Data Analysis Using Nosql
Improving the Efficiency of Big Forensic Data Analysis Using NoSQL Md Baitul Al Sadi Hayden Wimmer Lei Chen Kai Wang Department of Information Department of Information Department of Information Department of Computer Technology Technology Technology Science Georgia Southern University Georgia Southern University Georgia Southern University Georgia Southern University Statesboro, GA 30458, USA Statesboro, GA 30458, USA Statesboro, GA 30458, USA Statesboro, GA 30458, USA [email protected] [email protected] [email protected] [email protected] them in NoSQL (Not Only SQL) database. There is a variety of ABSTRACT tools available including Autopsy, EnCase, Foremost, FTK, The rapid growth of Internet of Things (IoT) makes the task for Registry Recon, PTK Forensics, The Sleuth Kit, The Coroner's digital forensic more difficult. At the same time, the data analyzing Toolkit, COFEE etc. to extract data from IoT devices. The technology is also developing in a feasible pace. Where traditional extracted data will be in an unstructured format, hence NoSQL is Structured Query Language (SQL) is not adequate to analyze the the best solution to analyze them. Here the document-oriented data in an unstructured and semi-structured format, Not only database program, MongoDB has been chosen to analyze the data Standard Query Language (NoSQL) unfastens the access to from Internet of Things (IoT). To our best knowledge this is pioneer analyzing the data of all format. The large volume of data of IoTs work in terms of using NoSQL and MongoDB for DF. turns into Big Data which just do not enhance the probability of attaining of evidence of an incident but make the investigation 2 BACKGROUND process more complex. -
Performance of Android Forensics Data Recovery Tools
This is author accepted copy; for final version please refer to: B.C. Ogazi-Onyemaechi, Ali Dehghantanha, Kim-Kwang Raymond Choo, “Performance of Android Forensics Data Recovery Tools”, Pages 91-110, Chapter 7, (Elsevier) Contemporary Digital Forensic Investigations Of Cloud And Mobile Applications Performance of Android Forensics Data Recovery Tools Bernard Chukwuemeka Ogazi-Onyemaechi1, Ali Dehghantanha1; Kim-Kwang Raymond Choo2 1School of Computing, Science and Engineering, University of Salford, Manchester, United Kingdom 2 Information Assurance Research Group, University of South Australia, Australia [email protected]; [email protected]; [email protected] Abstract- Recovering deleted or hidden data is among most important duties of forensics investigators. Extensive utilisation of smartphones as subject, objects or tools of crime made them an important part of residual forensics. This chapter investigates the effectiveness of mobile forensic data recovery tools in recovering evidences from a Samsung Galaxy S2 i9100 Android phone. We seek to determine the amount of data that could be recovered using Phone image carver, Access data FTK, Foremost, Diskdigger, and Recover My File forensic tools. The findings reflected the difference between recovery capacities of studied tools showing their suitability in their specialised contexts only. Keywords: Data recovery, digital forensics, deleted file recovery, mobile forensics, Android forensics. 1 1.0 INTRODUCTION Smart mobile devices, particularly smartphones, are increasingly popular in today’s Internet-connected society (1–4). For example, few years ago in 2010, shipments of smartphone grew by 74 percent to 295 million units (3,4). Unsurprisingly, sales of smartphones have been increasing since then (5,6), and it has been estimated that 1.5 billion smartphones will be sold by 2017 and 1 billion mobile subscribers by 2022 (7–15). -
The Ultimate Guide to Hard Drive Partitioning
Home SQL Server Cloud Computing Windows Server Cisco Exchange Windows 7 Active Directory Virtualization Forums IT Job Board Get Our FREE Newsletter! Search Site Gain access to white The Ultimate Guide to Hard Drive Partitioning papers, special offers and by Daniel Petri - March 11, 2009 the most recent IT tips! Printer Friendly Version Sponsors Email Address Disk partitioning divides the data storage space of a hard disk into separate areas referred to as Sign Up Now! Windows Application 'partitions'. Partitions are usually created when the hard disk is first being prepared for usage. Once a Monitoring Made Easy disk is divided into partitions, directories and files may be stored on them. Because of later changes on Quick Links the computer and operating system, often an administrator or power user will be faced with the need to Knowledge Base resize partitions, create new partitions in the unallocated space, and sometimes even undelete Take the guesswork out of Backup & Storage partitions that were previously removed or deleted. which WMI counters to use Security for apps like Microsoft® Active Directory Active Directory™ and Cisco and Routing Hard disk partitioning tools will allow you to create additional partitions from existing ones by resizing SharePoint™. SolarWinds Windows Networking FREE WMI Monitor makes it Exchange Server the existing partitions and creating empty, un-used space on which you can create new partitions. The easy! Download this FREE Virtualization desktop tool now! Windows Server 2008 most important aspect of such tools is their ability to do this without losing your data that is currently on Windows 7 your existing partitions. -
Creating Highly Specialized Fragmented File System Data Sets
CREATING HIGHLY SPECIALIZED FRAGMENTED FILE SYSTEM DATA SETS FOR FORENSIC RESEARCH A Thesis Presented in Partial Fulfillment of the Requirements for the Degree of Master of Science with a Major in Computer Science in the College of Graduate Studies at University of Idaho by Douglas Drobny May 2014 Major Professor: Jim Alves-Foss, Ph.D. ii AUTHORIZATION TO SUBMIT THESIS This thesis of Douglas Drobny, submitted for the degree of Master of Science with a Major in Computer Science and titled \Creating Highly Specialized Fragmented File System Data Sets for Forensic Research", has been reviewed in final form. Permission, as indicated by the signatures and dates given below, is now granted to submit final copies to the College of Graduate Studies for approval. Major Professor Date Dr. Jim Alves-Foss Committee members Date Dr. Paul Oman Date Dr. Marty Ytreberg Computer Science Department Administrator Date Dr. Gregory Donohoe Discipline's College Dean, College of Engineering Date Dr. Larry Stauffer Final Approval and Acceptance by the College of Graduate Studies Date Dr. Jie Chen iii ABSTRACT File forensic tools examine the contents of a system's disk storage to analyze files, detect infections, examine account usages and extract information that the system's operating system cannot or does not provide. In cases where the file system is not available, or information is believed to be outside of the file system, a file carver can be used to extract files. File carving is the process of extracting information from an entire disk without metadata. This thesis looks at the effects of file fragmentation on forensic file carvers.