The Decentralized File System Igor-FS as an Application for Overlay-Networks zur Erlangung des akademischen Grades eines Doktors der Ingenieurwissenschaften der Fakult¨at f¨ur Informatik der Universit¨at Fridericiana zu Karlsruhe (TH) genehmigte Dissertation von Dipl.-Ing. Kendy Kutzner aus Karl-Marx-Stadt Tag der m¨undlichen Pr¨ufung: 14. Februar 2008 Erster Gutachter: Dr. Thomas Fuhrmann Zweiter Gutachter: Prof. Dr. Klaus Wehrle 2 Contents 1 Introduction 11 1.1 Motivation .............................. 11 1.2 Zusammenfassung in deutscher Sprache . 11 1.3 Structureofthiswork .. .. .. .. .. .. .. .. .. .. 13 I Foundations 15 2 Cryptography 17 2.1 HashFunctions ............................ 18 2.1.1 CyclicRedundancyCheck . 18 2.1.2 CryptographicHashFunctions . 19 2.1.3 HashFunctionMD5 . 20 2.1.4 HashFunctionSHA . .. .. .. .. .. .. .. .. 21 2.1.5 Hash Functions for authentication – Merkle Hashes . 22 2.1.6 Rabinhashesasrollingchecksums . 22 2.2 EncryptionandDecryption . 23 2.2.1 OneTimePad ........................ 23 2.2.2 Rijndael as Advanced Encryption Standard – AES . 24 2.2.3 RC4 .............................. 25 2.3 ModesofBlockCiphers . .. .. .. .. .. .. .. .. .. 25 2.3.1 ElectronicCodebookMode . 26 2.3.2 CipherBlockChaining. 27 2.3.3 FeedbackModes ....................... 28 2.3.4 CounterMode ........................ 28 2.4 Padding ................................ 29 2.5 Summary ............................... 30 3 Overlay Networks 31 3.1 Peer-to-PeerNetworks . 31 3.1.1 BitTorrent........................... 32 3.1.2 Freenet ............................ 33 3.2 StructuredOverlaysandKeyBasedRouting . 35 3.2.1 Kademlia ........................... 38 3 4 CONTENTS 3.2.2 Pastry............................. 38 3.2.3 Chord ............................. 39 3.2.4 ContentAddressableNetwork–CAN . 39 3.2.5 deBruijnNetworks. 40 3.3 DistributedHashTables . 40 3.3.1 BambooandotherDHTServices . 42 3.4 OptimizingOverlayNetworks . 42 3.4.1 Degreesoffreedom. 43 3.4.2 ProximityRouteSelection. 43 3.4.3 ProximityNeighborSelection . 44 3.4.4 FindingCloseNeighbors. 44 DuringBootstrapping . 45 Measuring........................... 45 Coordinate Approaches: from GNP to Vivaldi . 46 Meridian ........................... 46 3.5 Otherissues.............................. 47 3.5.1 Routeconvergence . 47 3.5.2 Bootstrapping ........................ 47 3.5.3 ImplementationIssues . 48 3.6 Summary ............................... 49 4 File Systems 51 4.1 LocalFileSystems .......................... 51 4.1.1 FileAllocationTable–FAT. 51 4.1.2 ProcessFileSystem . 52 4.1.3 FourthExtendedFileSystem . 53 4.2 DistributedFileSystems. 54 4.2.1 NetworkFileSystem. 55 4.2.2 Other Distributed File Systems . 57 4.3 DecentralizedFileSystems . 57 4.4 UserSpaceFileSystems . 57 4.4.1 FUSE ............................. 58 4.4.2 Parrot............................. 59 4.4.3 Gnome-VFSetc ....................... 60 4.5 Summary ............................... 60 II The Overlay Network Igor 63 5 Goals 65 5.1 Application .............................. 66 5.2 Igor .................................. 66 5.3 Network................................ 66 CONTENTS 5 6 Design 67 6.1 ApplicationInterface. 67 6.1.1 TheServiceConcept . 67 6.1.2 TheUpcallConcept . 67 6.2 MessageRoutingandForwarding . 69 6.2.1 Iterativevs. Recursive . 69 6.2.2 Metric............................. 70 6.2.3 AggregationTree. 71 6.2.4 ServiceRouting.. .. .. .. .. .. .. .. .. .. 71 6.2.5 Connections.......................... 72 6.3 RoutingTableMaintenance . 73 6.3.1 CreationofNewEntries . 73 6.3.2 CreationoftheFirstEntry . 73 6.3.3 EvictionofOldEntries . 73 6.3.4 ConnectionsforServices . 74 6.4 Proximity ............................... 74 6.4.1 ProximityRouteSelection. 75 6.4.2 ProximityNeighborSelection . 75 Combination of Vivaldi and Meridian . 75 Findingnewneighbors. 76 6.5 InterfacetoApplications. 76 6.5.1 Connections.......................... 76 6.5.2 Library libigor ....................... 77 6.6 Summary ............................... 78 7 Implementation 79 7.1 TheCallBackList .......................... 79 7.2 Plugins ................................ 80 7.3 IPv6.................................. 81 8 Test and Deployment 83 8.1 Testing ................................ 83 8.1.1 PlanetLab........................... 83 8.2 BuildProcess ............................. 84 8.3 RunningIgor ............................. 84 8.3.1 Start.............................. 84 8.3.2 Stop.............................. 84 8.3.3 TheConfigurationFile. 84 8.3.4 TheLogFile ......................... 85 8.4 ApplicationExamples . 86 8.4.1 Filesystem........................... 86 8.4.2 Videgor ............................ 86 Scheduling .......................... 87 VideoDataTransport . 89 ElectronicProgramGuide . 89 8.4.3 LinyPhone .......................... 90 6 CONTENTS 8.5 Summary ............................... 91 9 Conclusions 93 9.1 FutureWork ............................. 93 9.1.1 Integration with Scalable Source Routing . 93 9.1.2 ControlPlane......................... 93 9.1.3 FirewallandNATTraversal. 94 9.1.4 Bootstrapping ........................ 94 9.2 Summary ............................... 95 III The Decentralized File System IgorFS 97 10 Goals 99 10.1Security ................................ 100 10.2 Distributedness and Decentralization . 100 10.3 ScalabilityandEfficiency . 100 10.4 EasyDeployment. 101 11 Design 103 11.1 InterfacetotheApplications . 103 11.2Security ................................ 104 11.2.1 Encapsulation of Cryptographic operations . 104 11.2.2 Authentication . 106 11.2.3 Authorization. 107 11.2.4 Confidentiality . 107 11.2.5 TrustIssues. 107 11.2.6 Examples ........................... 108 11.3 HandlingofFileSystemObjects . 109 11.3.1 Files.............................. 109 11.3.2 Directories .. .. .. .. .. .. .. .. .. .. .. 111 11.3.3 DirectoryLayout . 111 11.4BlockCut............................... 112 11.4.1 Requirements . 112 11.4.2 RollingChecksums . 114 FixedBlockSize .. .. .. .. .. .. .. .. .. .. 114 Adler32 ............................ 115 CRCandRabin ....................... 115 XOR32 ............................116 11.4.3 Conclusion .. .. .. .. .. .. .. .. .. .. .. 116 11.5Snapshot ............................... 117 11.5.1 DataStructuresNecessary. 117 11.5.2 Process ............................ 117 Read-OnlyAccess . 117 Write/ModifyAccess. 118 StartofanewEpoch . .. .. .. .. .. .. .. .. 119 CONTENTS 7 ProcessSnapshot/MakePersistent . 120 11.5.3 SummaryofSnapshot . 122 11.6BlockCache.............................. 122 11.6.1 Requirements . 123 11.6.2 Design............................. 123 11.7 IgorInterface ............................. 124 11.7.1 PointerCache. 125 11.7.2 DataTransfer. 125 11.7.3 BlockTransmission. 126 11.8 TheProcSystem ........................... 127 11.8.1 Registering . .. .. .. .. .. .. .. .. .. .. 127 11.8.2 ReadingandWriting. 127 11.8.3 ExampleUses. 127 12 Implementation 129 12.1Overview ............................... 129 12.2Logging ................................ 130 12.3 Userspacetools. .. .. .. .. .. .. .. .. .. .. .. 131 12.3.1 ExportKey .......................... 132 12.3.2 Mounting and Unmounting . 132 12.4 FileSystemDaemonIgorFS. 132 12.5Modules................................ 132 12.5.1 ModuleBlockingModule . 133 12.5.2 BlockCache. .. .. .. .. .. .. .. .. .. .. 133 12.5.3 ModuleBlockCut. 133 12.5.4 ModuleBlockFetcherModule . 134 12.5.5 ModuleBlockTransModule . 135 12.5.6 ModuleFileFolderModule . 136 12.5.7 ModuleIgorInterfaceModule . 137 12.5.8 ModulePointerCache . 137 12.5.9 Module SnapShotInitiator . 138 12.5.10cProcModule . 138 12.5.11FuseInterfaceModule . 138 Overview ........................... 138 Supported Operations on File System Objects . 139 Supported Operations on the Entire File System . 140 OperationsnotSupported . 141 12.6Summary ............................... 141 13 Conclusions 143 13.1OpenIssues.............................. 143 13.1.1 Obfuscation. 143 13.1.2 DenialofService . 143 13.1.3 HashCollisions . 144 13.1.4 Reliability and Block Deletion . 144 13.1.5 ReadAhead.......................... 145 8 CONTENTS 13.1.6 XMLandOtherInterfaces. 145 13.1.7 Business/LegalAspects . 145 13.2Summary ............................... 146 IV Evaluation and Summary 147 14 Evaluation and Testing 149 14.1 ChecksumAlgorithm. 149 14.2 AdaptiveBlockSize . 150 14.3 TCPkernelinformation . 152 14.4 End-to-EndEvaluation. 155 14.5Summary ............................... 157 15 Conclusions 159 15.1 Acknowledgments. 159 A Igor Configuration options 161 B Messages in IgorFS 163 Message cBFUpdateStatus .................163 Message cCryptoMsg .....................163 cMsgEncrypt .....................163 cMsgEncrypted ....................163 cMsgDecrypt .....................163 cMsgDecrypted ....................163 cMsgHash .......................163 cMsgHashed ......................163 cMsgVerify ......................163 cMsgVerified .....................163 cMsgHashNEncryptNHash ..............164 cMsgHashedNEncryptedNHashed .......... 164 cMsgVerifyNDecryptNVerify ............164 cMsgVerifiedNDecryptedNVerified ........ 164 Message cFsoPersistenceRequest .............164 Message cIFSMessage ....................164 cMsgBlockRequestFromBfToBc ...........164 cMsgBlockResponseFromBcToBf .......... 164 cBCJobContainer ...................164 cBCDBUpdateMessage ................164 cBCDBResponse ....................164 cBCFSCKCommand ...................164 cBCFSCKSweepComplete ...............164 cBCShutdownMessage ................164 cInventoryUpdateMsg ................164 Message cIgorInterfaceMessage