DOCSLIB.ORG

UNIX and Linux Forensic Analysis DVD Toolkit.2008.Pdf

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
UNIX and Linux Forensic Analysis DVD Toolkit.2008.Pdf Chris Pogue Cory Altheide Todd Haverkos Elsevier, Inc., the author(s), and any person or firm involved in the writing, editing, or production (collectively “Makers”) of this book (“the Work”) do not guarantee or warrant the results to be obtained from the Work. There is no guarantee of any kind, expressed or implied, regarding the Work or its contents. The Work is sold AS IS and WITHOUT WARRANTY. You may have other legal rights, which vary from state to state. In no event will Makers be liable to you for damages, including any loss of profits, lost savings, or other incidental or consequential damages arising out from the Work or its contents. Because some states do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you. You should always use reasonable care, including backup and other appropriate precautions, when working with computers, networks, data, and files. Syngress Media®, Syngress®, “Career Advancement Through Skill Enhancement®,” “Ask the Author UPDATE®,” and “Hack Proofing®,” are registered trademarks of Elsevier, Inc. “Syngress: The Definition of a Serious Security Library”™, “Mission Critical™,” and “The Only Way to Stop a Hacker is to Think Like One™” are trademarks of Elsevier, Inc. Brands and product names mentioned in this book are trademarks or service marks of their respective companies. KEY SERIAL NUMBER 001 HJIRTCV764 002 PO9873D5FG 003 829KM8NJH2 004 BAL923457U 005 CVPLQ6WQ23 006 VBP965T5T5 007 HJJJ863WD3E 008 2987GVTWMK 009 629MP5SDJT 010 IMWQ295T6T PUBLISHED BY Syngress Publishing, Inc. Elsevier, Inc. 30 Corporate Drive Burlington, MA 01803 UNIX and Linux Forensic Analysis DVD Toolkit Copyright © 2008 by Elsevier, Inc. All rights reserved. Printed in the United States of America. Except as permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher, with the exception that the program listings may be entered, stored, and executed in a computer system, but they may not be reproduced for publication. Printed in the United States of America 1 2 3 4 5 6 7 8 9 0 ISBN 13: 978-1-59749-269-0 Page Layout and Art: SPi Publishing Services Copy Editor: Judy Eby For information on rights, translations, and bulk sales, contact Matt Pedersen, Commercial Sales Director and Rights, at Syngress Publishing; email [email protected]. Co-Authors Chris Pogue is an Incident Response Manager and Forensic Analyst for a major Information Technology service provider. With over ten years of administrative and security experience he was worked cases all over the globe assisting organizations from fortune 500 companies to single owner small businesses build defense in depth into their infrastructure. Prior to becoming a forensic analyst, Chris spent the five years as part of the Ethical Hacking Team for the same service provider. Tasked with emulating the actions of a malicious attacker, he assisted customers in identifying and eliminating probable attack vectors. Brining that knowledge and experience to bear to the Incident Management team, Chris specializes in incidents involving intrusion, and unauthorized access. Chris is also a former US Army Warrant Officer (Signal Corps) and has worked with the Army Reserve Information Operations Command (ARIOC) on Joint Task Force ( JTF) missions with the National Security Agency (NSA), Department of Homeland Security, Regional Computer Emergency Response Team-Continental United States (RCERT-CONUS), and the Joint Intelligence Center-Pacific ( JICPAC). Chris attended Forensics training at Carnegie Mellon University (CMU) in Pittsburgh, Pennsylvania, and was the ARIOC primary instructor for UNIX, Networking, and Incident Response for all CMU sponsored courses. Chris holds a Bachelor’s Degree in Applied Management, a Master’s degree in Information Security, is a Certified Information Systems Security Professional, (CISSP), a Certified Ethical Hacker (CEH), and a VISA PCI DSS Qualified Security Assessor (QSA). Chris resides in Tulsa, Oklahoma with his wife Michelle, and his two children, Jenna age 4, and Colin age 2. While the world of computer forensics is fast paced, and challenging … nothing compares to the rush and excitement of being a husband and father. First and foremost, I would like to thank my Lord and savior Jesus Christ, without whom nothing in my life would be possible. Since being saved, He has walked along side of me when I needed a friend, walked in front of me when I needed a perfect example of how to live my life, and carried me when I needed a Messiah. All that I am able to do is only through His grace and mercy. iii My earthly strength comes from my beautiful wife Michelle, and my two children Jenna and Colin. Next to my salvation, Michelle is the greatest gift the Lord has ever bestowed upon me, and without her I would not be the man I am today. She is a wonderful wife, and mother. Thank you so much for supporting me, loving me, and being by best friend even though I am hands down, the biggest nerd you ever met. My kids keep me young at heart. I cannot begin to count the number of times while writing this book; I played puppy with Jenna and asked Colin to kindly get off my desk. I love being a dad and hope and pray that I am half the father they deserve. Finally, I would like to thank everyone who contributed in way or another to the completion of this work. Specifically I would like to thank Harlan Carvey, Tom Millar, Jon Evans, Sam Elder, Keith Fanselow, Clint Ruoho, Barry Grundy, Mariuzz Burdach, Todd Haverkos, Cory Altheide, Andrew Rosen, Rick Van Luvender, Darrell Vydra, Mark Eifert, my mother Donna Pogue, and my sister Juli Czajkowski. Without all of you helping me, guiding me, and encouraging me along the way, I would have never finished this freaking thing. Thank you all … and stuff. Cory Altheide has been performing forensics & incident investigations for the past seven years for employers ranging from the National Nuclear Security Administration to Google. Cory has developed and presented hands-on forensics training for various government, law enforcement, and civilian entities geared towards improving inves- tigator’s skills in examining less common digital media, exotic operating systems, and live analysis and acquisition. He holds the GIAC GCIH and PCI QSA certifications and is on the technical program committee for the 8th Annual Digital Forensics Research Workshop (DFRWS). I would like to the The Macallan. Todd Haverkos has 16 years of experience in Information Technology (IT) and high technology, including 7 years with a Fortune 50 computer services company working in network security, ethical hacking roles and most recently Security Consulting and Penetration Testing roles, 6 years with an international semiconductor and communica- tions company as an integrated circuit designer for which he holds a US Patent, and 4 years with large regional electric utility in database administration, IT, and telecom. He has 6 years of dedicated experience in Information Security and has expertise with penetration testing, application security testing, vulnerability assessments and recom- mendations, security policy and procedure development, network architecture, firewall, switch and router configuration and management, and technical security documentation. iv Todd holds a bachelor’s degree in Electrical Engineering from the University of Dayton, a master’s degree in Computer Engineering from Northwestern University, and holds EC-Council Certified Ethical Hacker, and EC-Council Certified Security Analyst certifications. Thanks first to Chris Pogue for the opportunity to contribute to the book (in an albeit-small, one chapter sort of way). It’s a pleasure to have the opportunity to spread *nix thoughts! James Lee, thank you for inspiration and encouragement to jump into the water. Thanks also to the English faculty of St. Xavier High School in Cincinnati, OH, particularly Walter Koral, Jim Downie, Patricia Culley, and Mike Marchal. While I know at least one of you was quite unhappy with my choice of engineering as a career path lo, those many years ago, I would like to offer this consolation: The ability to communicate well–particularly in writing–is easily the most valuable skill I have brought to any technical job I’ve had, and a skill I use every day of this Internet-connected life. Many thanks! Thanks also to Mom who so wisely and generously found a way to make sure I had that education and the one from her, and for all those other great things Mom’s do that earn them a much-deserved place in acknowledgements! To those who like to keep a low profile, but who have been so generous in sharing their technical knowledge and friendship with me over the years, thank you! And to Phet and Boon, you make everything worth working for. v Appendix Contributor Michael Cross (MCSE, MCP+I, CNA, Network+) is an internet specialist/ programmer with the Niagara Regional Police Service. In addition to designing and maintaining the Niagara Regional Police’s Web site (www.nrps. com) and intranet, he has also provided support and worked in the areas of programming, hardware, database administration, graphic design, and network administration. In 2007, he was awarded a Police Commendation for work he did in developing a system to track high-risk offenders and sexual offenders in the Niagara Region. As part of an information technology team that provides support to a user base of over 1,000 civilian and uniformed users, his theory is that when the users carry guns, you tend to be more motivated in solving their problems. Michael was the first computer forensic analyst in the Niagara Regional Police Service’s history, and for five years he performed computer forensic examinations on computers involved in criminal investigations.
Load more

Recommended publications
  • 101 Useful Linux Commands - Haydenjames.Io
    101 Useful Linux Commands - haydenjames.io Some of these commands require elevated permissions (sudo) to run. Enjoy! 1. Execute the previous command used: !! 2. Execute a previous command starting with a specific letter. Example: !s 3. Short way to copy or backup a file before you edit it. For example, copy nginx.conf cp nginx.conf{,.bak} 4. Toggle between current directory and last directory cd - 5. Move to parent (higher level) directory. Note the space! cd .. 6. Go to home directory cd ~ 7. Go to home directory cd $HOME 8. Go to home directory (when used alone) cd 9. Set permissions to 755. Corresponds to these permissions: (-rwx-r-x-r-x), arranged in this sequence: (owner-group-other) chmod 755 <filename> 10. Add execute permission to all users. chmod a+x <filename> 11. Changes ownership of a file or directory to . chown <username> 12. Make a backup copy of a file (named file.backup) cp <file> <file>.backup 13. Copy file1, use it to create file2 cp <file1> <file2> 14. Copy directory1 and all its contents (recursively) into directory2 cp -r <directory1> <directory2>/ 15. Display date date 16. Zero the sdb drive. You may want to use GParted to format the drive afterward. You need elevated permissions to run this (sudo). dd if=/dev/zero of=/dev/sdb 17. Display disk space usage df -h 18. Take detailed messages from OS and input to text file dmesg>dmesg.txt 19. Display a LOT of system information. I usually pipe output to less. You need elevated permissions to run this (sudo).
    [Show full text]
  • BSD UNIX Toolbox 1000+ Commands for Freebsd, Openbsd
    76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page iii BSD UNIX® TOOLBOX 1000+ Commands for FreeBSD®, OpenBSD, and NetBSD®Power Users Christopher Negus François Caen 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page ii 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page i BSD UNIX® TOOLBOX 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page ii 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page iii BSD UNIX® TOOLBOX 1000+ Commands for FreeBSD®, OpenBSD, and NetBSD®Power Users Christopher Negus François Caen 76034ffirs.qxd:Toolbox 4/2/08 12:50 PM Page iv BSD UNIX® Toolbox: 1000+ Commands for FreeBSD®, OpenBSD, and NetBSD® Power Users Published by Wiley Publishing, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2008 by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-0-470-37603-4 Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 Library of Congress Cataloging-in-Publication Data is available from the publisher. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permis- sion should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4355, or online at http://www.wiley.com/go/permissions.
    [Show full text]
  • Filesystem Considerations for Embedded Devices ELC2015 03/25/15
    Filesystem considerations for embedded devices ELC2015 03/25/15 Tristan Lelong Senior embedded software engineer Filesystem considerations ABSTRACT The goal of this presentation is to answer a question asked by several customers: which filesystem should you use within your embedded design’s eMMC/SDCard? These storage devices use a standard block interface, compatible with traditional filesystems, but constraints are not those of desktop PC environments. EXT2/3/4, BTRFS, F2FS are the first of many solutions which come to mind, but how do they all compare? Typical queries include performance, longevity, tools availability, support, and power loss robustness. This presentation will not dive into implementation details but will instead summarize provided answers with the help of various figures and meaningful test results. 2 TABLE OF CONTENTS 1. Introduction 2. Block devices 3. Available filesystems 4. Performances 5. Tools 6. Reliability 7. Conclusion Filesystem considerations ABOUT THE AUTHOR • Tristan Lelong • Embedded software engineer @ Adeneo Embedded • French, living in the Pacific northwest • Embedded software, free software, and Linux kernel enthusiast. 4 Introduction Filesystem considerations Introduction INTRODUCTION More and more embedded designs rely on smart memory chips rather than bare NAND or NOR. This presentation will start by describing: • Some context to help understand the differences between NAND and MMC • Some typical requirements found in embedded devices designs • Potential filesystems to use on MMC devices 6 Filesystem considerations Introduction INTRODUCTION Focus will then move to block filesystems. How they are supported, what feature do they advertise. To help understand how they compare, we will present some benchmarks and comparisons regarding: • Tools • Reliability • Performances 7 Block devices Filesystem considerations Block devices MMC, EMMC, SD CARD Vocabulary: • MMC: MultiMediaCard is a memory card unveiled in 1997 by SanDisk and Siemens based on NAND flash memory.
    [Show full text]
  • Journaling File Systems
    Linux Journaling File Systems Linux onzSeries Journaling File Systems Volker Sameske ([email protected]) Linux on zSeries Development IBM Lab Boeblingen, Germany Share Anaheim,California February27 –March 4,2005 Session 9257 ©2005 IBM Corporation Linux Journaling File Systems Agenda o File systems. • Overview, definitions. • Reliability, scalability. • File system features. • Common grounds & differences. o Volume management. • LVM, EVMS, MD. • Striping. o Measurement results. • Hardware/software setup. • throughput. • CPU load. 2 Session 9257 © 2005 IBM Corporation Linux Journaling File Systems A file system should... o ...store data o ...organize data o ...administrate data o ...organize data about the data o ...assure integrity o ...be able to recover integrity problems o ...provide tools (expand, shrink, check, ...) o ...be able to handle many and large files o ...be fast o ... 3 Session 9257 © 2005 IBM Corporation Linux Journaling File Systems File system-definition o Informally • The mechanism by which computer files are stored and organized on a storage device. o More formally, • A set of abstract data types that are necessary for the storage, hierarchical organization, manipulation, navigation, access and retrieval of data. 4 Session 9257 © 2005 IBM Corporation Linux Journaling File Systems Why a journaling file system? o Imagine your Linux system crashs while you are saving an edited file: • The system crashs after the changes have been written to disk à good crash • The system crashs before the changes have been written to disk à bad crash but bearable if you have an older version • The sytem crashs just in the moment your data will be written: à very bad crash your file could be corrupted and in worst case the file system could be corrupted à That‘s why you need a journal 5 Session 9257 © 2005 IBM Corporation Linux Journaling File Systems Somefilesystemterms o Meta data • "Data about the data" • File system internal data structure (e.g.
    [Show full text]
  • Wildlife Management Activities and Practices
    WILDLIFE MANAGEMENT ACTIVITIES AND PRACTICES COMPREHENSIVE WILDLIFE MANAGEMENT PLANNING GUIDELINES for the Edwards Plateau and Cross Timbers & Prairies Ecological Regions Revised April 2010 The following Texas Parks & Wildlife Department staff have contributed to this document: Mike Krueger, Technical Guidance Biologist – Lampasas Mike Reagan, Technical Guidance Biologist -- Wimberley Jim Dillard, Technical Guidance Biologist -- Mineral Wells (Retired) Kirby Brown, Private Lands and Habitat Program Director (Retired) Linda Campbell, Program Director, Private Lands & Public Hunting Program--Austin Linda McMurry, Private Lands and Public Hunting Program Assistant -- Austin With Additional Contributions From: Kevin Schwausch, Private Lands Biologist -- Burnet Terry Turney, Rare Species Biologist--San Marcos Trey Carpenter, Manager, Granger Wildlife Management Area Dale Prochaska, Private Lands Biologist – Kerr Wildlife Management Area Nathan Rains, Private Lands Biologist – Cleburne TABLE OF CONTENTS Comprehensive Wildlife Management Planning Guidelines Edwards Plateau and Cross Timbers & Prairies Ecological Regions Introduction Specific Habitat Management Practices HABITAT CONTROL EROSION CONTROL PREDATOR CONTROL PROVIDING SUPPLEMENTAL WATER PROVIDING SUPPLEMENTAL FOOD PROVIDING SUPPLEMENTAL SHELTER CENSUS APPENDICES APPENDIX A: General Habitat Management Considerations, Recommendations, and Intensity Levels APPENDIX B: Determining Qualification for Wildlife Management Use APPENDIX C: Wildlife Management Plan Overview APPENDIX D: Livestock
    [Show full text]
  • State of the Art: Where We Are with the Ext3 Filesystem
    State of the Art: Where we are with the Ext3 filesystem Mingming Cao, Theodore Y. Ts’o, Badari Pulavarty, Suparna Bhattacharya IBM Linux Technology Center {cmm, theotso, pbadari}@us.ibm.com, [email protected] Andreas Dilger, Alex Tomas, Cluster Filesystem Inc. [email protected], [email protected] Abstract 1 Introduction Although the ext2 filesystem[4] was not the first filesystem used by Linux and while other filesystems have attempted to lay claim to be- ing the native Linux filesystem (for example, The ext2 and ext3 filesystems on Linux R are when Frank Xia attempted to rename xiafs to used by a very large number of users. This linuxfs), nevertheless most would consider the is due to its reputation of dependability, ro- ext2/3 filesystem as most deserving of this dis- bustness, backwards and forwards compatibil- tinction. Why is this? Why have so many sys- ity, rather than that of being the state of the tem administrations and users put their trust in art in filesystem technology. Over the last few the ext2/3 filesystem? years, however, there has been a significant amount of development effort towards making There are many possible explanations, includ- ext3 an outstanding filesystem, while retaining ing the fact that the filesystem has a large and these crucial advantages. In this paper, we dis- diverse developer community. However, in cuss those features that have been accepted in our opinion, robustness (even in the face of the mainline Linux 2.6 kernel, including direc- hardware-induced corruption) and backwards tory indexing, block reservation, and online re- compatibility are among the most important sizing.
    [Show full text]
  • GNU Coreutils Cheat Sheet (V1.00) Created by Peteris Krumins ([email protected], -- Good Coders Code, Great Coders Reuse)
    GNU Coreutils Cheat Sheet (v1.00) Created by Peteris Krumins ([email protected], www.catonmat.net -- good coders code, great coders reuse) Utility Description Utility Description arch Print machine hardware name nproc Print the number of processors base64 Base64 encode/decode strings or files od Dump files in octal and other formats basename Strip directory and suffix from file names paste Merge lines of files cat Concatenate files and print on the standard output pathchk Check whether file names are valid or portable chcon Change SELinux context of file pinky Lightweight finger chgrp Change group ownership of files pr Convert text files for printing chmod Change permission modes of files printenv Print all or part of environment chown Change user and group ownership of files printf Format and print data chroot Run command or shell with special root directory ptx Permuted index for GNU, with keywords in their context cksum Print CRC checksum and byte counts pwd Print current directory comm Compare two sorted files line by line readlink Display value of a symbolic link cp Copy files realpath Print the resolved file name csplit Split a file into context-determined pieces rm Delete files cut Remove parts of lines of files rmdir Remove directories date Print or set the system date and time runcon Run command with specified security context dd Convert a file while copying it seq Print sequence of numbers to standard output df Summarize free disk space setuidgid Run a command with the UID and GID of a specified user dir Briefly list directory
    [Show full text]
  • Outline of Ext4 File System & Ext4 Online Defragmentation Foresight
    Outline of Ext4 File System & Ext4 Online Defragmentation Foresight LinuxCon Japan/Tokyo 2010 September 28, 2010 Akira Fujita <[email protected]> NEC Software Tohoku, Ltd. Self Introduction ▐ Name: Akira Fujita Japan ▐ Company: NEC Software Tohoku, Ltd. in Sendai, Japan. Sendai ● ▐ Since 2004, I have been working at NEC Software Tohoku developing Linux file system, mainly ext3 and ● ext4 filesystems. Tokyo Currently, I work on the quality evaluation of ext4 for enterprise use, and also develop the ext4 online defragmentation. Page 2 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. Outline ▐ What is ext4 ▐ Ext4 features ▐ Compatibility ▐ Performance measurement ▐ Recent ext4 topics ▐ What is ext4 online defrag ▐ Relevant file defragmentation ▐ Current status / future plan Page 3 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. What is ext4 ▐ Ext4 is the successor of ext3 which is developed to solve performance issues and scalability bottleneck on ext3 and also provide backward compatibility with ext3. ▐ Ext4 development began in 2006. Included in stable kernel 2.6.19 as EXPERIMENTAL (ext4dev). Since kernel 2.6.28, ext4 has been released as stable (Renamed from ext4dev to ext4 in kernel 2.6.28). ▐ Maintainers Theodore Ts'o [email protected] , Andreas Dilger [email protected] ▐ ML [email protected] ▐ Ext4 Wiki http://ext4.wiki.kernel.org Page 4 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. Ext4 features Page 5 Copyright(C) 2010 NEC Software Tohoku, Ltd. All Rights Reserved. Ext4 features Bigger file/filesystem size support. Compared to ext3, ext4 is: 8 times larger in file size, 65536 times(!) larger in filesystem size.
    [Show full text]
  • Constraints in Dynamic Symbolic Execution: Bitvectors Or Integers?
    Constraints in Dynamic Symbolic Execution: Bitvectors or Integers? Timotej Kapus, Martin Nowack, and Cristian Cadar Imperial College London, UK ft.kapus,m.nowack,[email protected] Abstract. Dynamic symbolic execution is a technique that analyses programs by gathering mathematical constraints along execution paths. To achieve bit-level precision, one must use the theory of bitvectors. However, other theories might achieve higher performance, justifying in some cases the possible loss of precision. In this paper, we explore the impact of using the theory of integers on the precision and performance of dynamic symbolic execution of C programs. In particular, we compare an implementation of the symbolic executor KLEE using a partial solver based on the theory of integers, with a standard implementation of KLEE using a solver based on the theory of bitvectors, both employing the popular SMT solver Z3. To our surprise, our evaluation on a synthetic sort benchmark, the ECA set of Test-Comp 2019 benchmarks, and GNU Coreutils revealed that for most applications the integer solver did not lead to any loss of precision, but the overall performance difference was rarely significant. 1 Introduction Dynamic symbolic execution is a popular program analysis technique that aims to systematically explore all the paths in a program. It has been very successful in bug finding and test case generation [3, 4]. The research community and industry have produced many tools performing symbolic execution, such as CREST [5], FuzzBALL [9], KLEE [2], PEX [14], and SAGE [6], among others. To illustrate how dynamic symbolic execution works, consider the program shown in Figure 1a.
    [Show full text]
  • Wed04-Blagodarenkoartem-Scaling Ldiskfs for the Future. Again. LUG
    Scaling LDISKFS for the future. Again Artem Blagodarenko LUG 2017 Bloomington, Indiana LUG 2017 LDISKFS still grows As drive size increases ...8TB -> 10TB -> 12TB The maximum backend storage size increases ...16TB -> 500TB LDISKFS quickly exceeded the original design! LUG 2017 The summary of previous work Done Problems ➢code review ➢ inodes➢ count over ➢ Extending inodes testing suite UINT32_MAXcount over ➢patches with fixes ➢ largeUINT32_MAX memory blocks ➢move LDISKFS size allocation➢ Large memory limit to 256TB (LU- ➢ solutionblocks for allocation large 7592). directories➢ Large directories LUG 2017 Inode count limit (LU-1365) Example: a customer requires 16 billions of inodes on MDS Unfortunately we can not make Only 4 billions of inodes on 16 billions inodes on one MDT one MDT because of LDISKFS limitation mdt0 mdt1 We can use 4 MDTs with DNE but MDT’s space is not mdt2 mdt3 completely used 16 billions >4 billions inodes on LDISKFS LUG 2017 Inode count limit. Additional fields for ext4_dir_entry Offset Size Name Description 0x0 __le32 inode Inode number 0x4 __le16 rec_len Length of this directory entry 0x6 __u8 name_len Length of the file name 0x7 __u8 file_type File type (0x0F), Dirdata (0xF0) 0x8 __u8 lufid_len OST fid length 0x9 N fid EXT4_DIRENT_LUFID 0x8 + N __u8 hi_inode_len length, always 4 0x8 + N + 1 __le64 hi_inode EXT4_DIRENT_INODE LUG 2017 dirdata pros and cons ➕ less space for 64-bit inodes ➕ smaller dirents for 32-bit inodes ➕ more 32-bit dirents in leaf block ➕ backwards compatible with existing directories ➕ doesn’t
    [Show full text]
  • Mac OS X Server
    Mac OS X Server Version 10.4 Technology Overview August 2006 Technology Overview 2 Mac OS X Server Contents Page 3 Introduction Page 5 New in Version 10.4 Page 7 Operating System Fundamentals UNIX-Based Foundation 64-Bit Computing Advanced BSD Networking Architecture Robust Security Directory Integration High Availability Page 10 Integrated Management Tools Server Admin Workgroup Manager Page 14 Service Deployment and Administration Open Directory Server File and Print Services Mail Services Web Hosting Enterprise Applications Media Streaming iChat Server Software Update Server NetBoot and NetInstall Networking and VPN Distributed Computing Page 29 Product Details Page 31 Open Source Projects Page 35 Additional Resources Technology Overview 3 Mac OS X Server Introduction Mac OS X Server version 10.4 Tiger gives you everything you need to manage servers in a mixed-platform environment and to con gure, deploy, and manage powerful network services. Featuring the renowned Mac OS X interface, Mac OS X Server streamlines your management tasks with applications and utilities that are robust yet easy to use. Apple’s award-winning server software brings people and data together in innovative ways. Whether you want to empower users with instant messaging and blogging, gain greater control over email, reduce the cost and hassle of updating software, or build your own distributed supercomputer, Mac OS X Server v10.4 has the tools you need. The Universal release of Mac OS X Server runs on both Intel- and PowerPC-based The power and simplicity of Mac OS X Server are a re ection of Apple’s operating sys- Mac desktop and Xserve systems.
    [Show full text]
  • Evolving Ext4 for Shingled Disks Abutalib Aghayev Theodore Ts’O Garth Gibson Peter Desnoyers Carnegie Mellon University Google, Inc
    Evolving Ext4 for Shingled Disks Abutalib Aghayev Theodore Ts’o Garth Gibson Peter Desnoyers Carnegie Mellon University Google, Inc. Carnegie Mellon University Northeastern University Abstract ST5000AS0011 Drive-Managed SMR (Shingled Magnetic Recording) disks 30 ST8000AS0002 10 ST4000LM016 offer a plug-compatible higher-capacity replacement for WD40NMZW conventional disks. For non-sequential workloads, these 3 WD5000YS disks show bimodal behavior: After a short period of high 1 throughput they enter a continuous period of low throughput. 0.3 We introduce ext4-lazy1, a small change to the Linux 0.1 Throughput (MiB/s) ext4 file system that significantly improves the throughput 0.03 in both modes. We present benchmarks on four different 0.01 drive-managed SMR disks from two vendors, showing that 0 100 200 300 400 500 ext4-lazy achieves 1.7-5.4× improvement over ext4 on a Time (s) metadata-light file server benchmark. On metadata-heavy Figure 1: Throughput of CMR and DM-SMR disks from Table 1 under benchmarks it achieves 2-13× improvement over ext4 on 4 KiB random write traffic. CMR disk has a stable but low throughput under drive-managed SMR disks as well as on conventional disks. random writes. DM-SMR disks, on the other hand, have a short period of high throughput followed by a continuous period of ultra-low throughput. 1 Introduction Type Vendor Model Capacity Form Factor Over 90% of all data in the world has been generated over the DM-SMR Seagate ST8000AS0002 8 TB 3.5 inch last two years [14]. To cope with the exponential growth of DM-SMR Seagate ST5000AS0011 5 TB 3.5 inch data, as well as to stay competitive with NAND flash-based DM-SMR Seagate ST4000LM016 4 TB 2.5 inch solid state drives (SSDs), hard disk vendors are researching DM-SMR Western Digital WD40NMZW 4 TB 2.5 inch capacity-increasing technologies like Shingled Magnetic CMR Western Digital WD5000YS 500 GB 3.5 inch Recording (SMR) [20,60], Heat Assisted Magnetic Record- ing (HAMR) [29], and Bit-Patterned Magnetic Recording Table 1: CMR and DM-SMR disks from two vendors used for evaluation.
    [Show full text]