XFS Algorithms & Data Structures
Total Page:16
File Type:pdf, Size:1020Kb
XFS Algorithms & Data Structures i XFS Algorithms & Data Structures 3rd Edition XFS Algorithms & Data Structures ii Copyright © 2006 Silicon Graphics Inc. © Copyright 2006 Silicon Graphics Inc. All rights reserved. Permission is granted to copy, distribute, and/or modify this document under the terms of the Creative Commons Attribution-Share Alike, Version 3.0 or any later version published by the Creative Commons Corp. A copy of the license is available at http://creativecommons. org/licenses/by-sa/3.0/us/. XFS Algorithms & Data Structures iii REVISION HISTORY NUMBER DATE DESCRIPTION NAME 0.1 2006 Initial Release Silicon Graphics, Inc 1.0 Fri Jul 03 2009 Publican Conversion Ryan Lerch 1.1 March 2010 Community Release Eric Sandeen 1.99 February 2014 AsciiDoc Conversion Dave Chinner 3 October 2015 Miscellaneous fixes. Darrick Wong Add missing field definitions. Add some missing xfs_db examples. Add an overview of XFS. Document the journal format. Document the realtime device. 3.1 October 2015 Add v5 fields. Darrick Wong Discuss metadata integrity. Document the free inode B+tree. Create an index of magic numbers. Document sparse inodes. 3.14 January 2016 Document disk format change testing. Darrick Wong 3.141 June 2016 Document the reverse-mapping btree. Darrick Wong Move the b+tree info to a separate chapter. Discuss overlapping interval b+trees. Discuss new log items for atomic updates. Document the reference-count btree. Discuss block sharing, reflink, & deduplication. 3.1415 July 2016 Document the real-time reverse-mapping btree. Darrick Wong 3.14159 June 2017 Add the metadump file format. Darrick Wong 3.141592 May 2018 Incorporate Dave Chinnerʼs log design document. Darrick Wong Incorporate Dave Chinnerʼs self-describing metadata design document. XFS Algorithms & Data Structures iv Contents I High Level Design 1 1 Overview 3 2 Metadata Integrity 4 2.1 Introduction .................................................. 4 2.2 Self Describing Metadata ........................................... 5 2.3 Runtime Validation .............................................. 6 2.4 Structures ................................................... 7 2.5 Inodes and Dquots ............................................... 9 3 Delayed Logging 10 3.1 Introduction to Re-logging in XFS ...................................... 10 3.2 Delayed Logging Concepts .......................................... 11 3.3 Delayed Logging Design ........................................... 12 3.3.1 Storing Changes ........................................... 12 3.3.2 Tracking Changes .......................................... 13 3.3.3 Checkpoints .............................................. 14 3.3.4 Checkpoint Sequencing ....................................... 16 3.3.5 Checkpoint Log Space Accounting ................................. 17 3.3.6 Log Item Pinning ........................................... 18 3.3.7 Concurrent Scalability ........................................ 18 3.3.8 Lifecycle Changes .......................................... 19 4 Sharing Data Blocks 22 5 Metadata Reconstruction 23 6 Common XFS Types 25 XFS Algorithms & Data Structures v 7 Magic Numbers 27 8 Theoretical Limits 30 9 Testing Filesystem Changes 31 II Global Structures 32 10 Fixed Length Record B+trees 33 10.1 Short Format B+trees ............................................. 34 10.2 Long Format B+trees ............................................. 35 11 Variable Length Record B+trees 37 11.1 Block Headers ................................................. 38 11.2 Internal Nodes ................................................. 39 12 Allocation Groups 41 12.1 Superblocks .................................................. 42 12.1.1 xfs_db Superblock Example ..................................... 50 12.2 AG Free Space Management ......................................... 51 12.2.1 AG Free Space Block ......................................... 51 12.2.2 AG Free Space B+trees ........................................ 53 12.2.3 AG Free List .............................................. 55 12.2.3.1 xfs_db AGF Example ................................... 57 12.3 AG Inode Management ............................................ 59 12.3.1 Inode Numbers ............................................ 59 12.3.2 Inode Information .......................................... 59 12.4 Inode B+trees ................................................. 61 12.4.1 xfs_db AGI Example ......................................... 63 12.5 Sparse Inodes ................................................. 64 12.5.1 xfs_db Sparse Inode AGI Example .................................. 65 12.6 Real-time Devices ............................................... 67 12.7 Reverse-Mapping B+tree ........................................... 67 12.7.1 xfs_db rmapbt Example ....................................... 69 12.8 Reference Count B+tree ............................................ 72 12.8.1 xfs_db refcntbt Example ....................................... 73 XFS Algorithms & Data Structures vi 13 Journaling Log 75 13.1 Log Records .................................................. 75 13.2 Log Operations ................................................ 77 13.3 Log Items .................................................... 78 13.3.1 Transaction Headers ......................................... 79 13.3.2 Intent to Free an Extent ....................................... 80 13.3.3 Completion of Intent to Free an Extent ............................... 81 13.3.4 Reverse Mapping Updates Intent .................................. 82 13.3.5 Completion of Reverse Mapping Updates .............................. 83 13.3.6 Reference Count Updates Intent ................................... 84 13.3.7 Completion of Reference Count Updates .............................. 85 13.3.8 File Block Mapping Intent ...................................... 85 13.3.9 Completion of File Block Mapping Updates ............................ 86 13.3.10 Inode Updates ............................................. 87 13.3.11 Inode Data Log Item ......................................... 89 13.3.12 Buffer Log Item ............................................ 89 13.3.13 Buffer Data Log Item ......................................... 90 13.3.14 Update Quota File .......................................... 90 13.3.15 Quota Update Data Log Item .................................... 90 13.3.16 Disable Quota Log Item ....................................... 91 13.3.17 Inode Creation Log Item ....................................... 91 13.4 xfs_logprint Example ............................................. 92 14 Internal Inodes 95 14.1 Quota Inodes .................................................. 95 14.2 Real-time Inodes ................................................ 98 14.2.1 Real-Time Bitmap Inode ....................................... 98 14.2.2 Real-Time Summary Inode ...................................... 98 14.2.3 Real-Time Reverse-Mapping B+tree ................................. 99 14.2.3.1 xfs_db rtrmapbt Example ................................. 100 III Dynamically Allocated Structures 103 15 On-disk Inode 104 15.1 Inode Core ................................................... 105 15.2 Unlinked Pointer ................................................ 110 XFS Algorithms & Data Structures vii 15.3 Data Fork .................................................... 111 15.3.1 Regular Files (S_IFREG) ....................................... 112 15.3.2 Directories (S_IFDIR) ......................................... 112 15.3.3 Symbolic Links (S_IFLNK) ...................................... 112 15.3.4 Other File Types ........................................... 112 15.4 Attribute Fork ................................................. 113 15.4.1 Extended Attribute Versions ..................................... 113 16 Data Extents 115 16.1 Extent List ................................................... 116 16.1.1 xfs_db Inode Data Fork Extents Example .............................. 117 16.2 B+tree Extent List ............................................... 119 16.2.1 xfs_db bmbt Example ......................................... 123 17 Directories 124 17.1 Short Form Directories ............................................ 125 17.1.1 xfs_db Short Form Directory Example ............................... 127 17.2 Block Directories ............................................... 130 17.2.1 xfs_db Block Directory Example .................................. 136 17.3 Leaf Directories ................................................ 139 17.3.1 xfs_db Leaf Directory Example ................................... 143 17.4 Node Directories ................................................ 147 17.4.1 xfs_db Node Directory Example ................................... 151 17.5 B+tree Directories ............................................... 153 17.5.1 xfs_db B+tree Directory Example .................................. 154 18 Extended Attributes 157 18.1 Short Form Attributes ............................................. 157 18.1.1 xfs_db Short Form Attribute Example ................................ 159 18.2 Leaf Attributes ................................................. 163 18.2.1 xfs_db Leaf Attribute Example .................................... 168 18.3 Node Attributes ................................................ 170 18.3.1 xfs_db Node Attribute Example ................................... 171 18.4 B+tree Attributes ............................................... 174 18.4.1 xfs_db B+tree Attribute Example .................................