SILENUS – A FEDERATED SERVICE-ORIENTED APPROACH TO DISTRIBUTED FILE SYSTEMS by MAXIMILIAN BERGER, B.S. A DISSERTATION IN COMPUTER SCIENCE Submitted to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Approved Michael Sobolewski Chairperson of the Committee Noé López-Benitez Michael Shin Per Anderson Accepted John Borrelli Dean of the Graduate School December, 2006 Copyright © 2006 Maximlilian Berger ACKNOWLEDGEMENTS I would like to thank my Advisor Dr. Michael Sobolweski. This thesis could not have been done without his permanent effort to keep me on focused on the goal. I would also like to thank Dr. Noe Lopez-Benitez, Dr. Michael Shin, and Dr. Per Anderson for serving in my comittee. They could provide me with a different viewpoint. A very special thanks goes to Barbara Hartmann for being the greatest person on earth and for making our relationship work half way around the globe for three years. I would also like to thank my best friend and roommate Nathan Larson. He was always there to support, no matter how crazy the idea. ii TABLE OF CONTENTS ACKNOWLEDGMENTS ...................................................................................... ii ABSTRACT .......................................................................................................... viii I. INTRODUCTION .............................................................................................. 1 Problem Statement ......................................................................................... 3 Dissertation Outline ....................................................................................... 4 II. BACKGROUND AND LITERATURE REVIEW ........................................... 5 Existing model for remote file storage .......................................................... 5 Model functionality ............................................................................... 6 Additional terms .................................................................................... 6 Shortcomings of the traditional model .................................................. 7 Existing network file storage solutions ......................................................... 8 Non-replicated remote file systems ....................................................... 8 Replicated file systems .......................................................................... 10 Data grid solutions ................................................................................ 11 Other existing file storage solutions ...................................................... 12 File system core features ....................................................................... 12 Architectural qualities for distributed systems .............................................. 13 Transparencies ....................................................................................... 13 Confidentiality ....................................................................................... 14 Global availability ................................................................................. 17 Disconnected Operation ......................................................................... 20 Manageability ........................................................................................ 21 Scalability .............................................................................................. 22 Reliability .............................................................................................. 22 Modifiability .......................................................................................... 23 Platform independence .......................................................................... 23 Service Orientation ........................................................................................ 24 Eight fallacies of distributed computing ................................................ 24 Generations of Remote Procedure Calls ................................................ 25 Service Oriented Architecture ............................................................... 26 iii Jini Network Technology ...................................................................... 29 Peer-to-peer networking ........................................................................ 30 SORCER ................................................................................................ 31 Eight truth of networked computing ...................................................... 32 Security in existing file storage solutions ...................................................... 32 Privileges ............................................................................................... 33 Authentication mechanisms ................................................................... 44 Privacy mechanisms .............................................................................. 45 III. REQUIREMENT ANALYSIS ........................................................................ 46 File Storage Scenarios ................................................................................... 46 Small work group .................................................................................. 46 High-Performance Computing Lab ....................................................... 47 Large network ........................................................................................ 47 Home user ............................................................................................. 48 Concurrent Engineers ............................................................................ 48 Student Computer Lab ........................................................................... 48 Astronomy ............................................................................................. 48 High-energy physics .............................................................................. 49 Host types on the network ............................................................................ 49 Server ..................................................................................................... 49 Always up client .................................................................................... 49 Work time up client .............................................................................. 49 Laptop .................................................................................................... 50 Mobile client .......................................................................................... 50 Use Case Roles .............................................................................................. 50 File system users ................................................................................... 51 Administrators ....................................................................................... 51 Optimizer services ................................................................................. 52 Service provisioners .............................................................................. 52 Intergrid service providers ..................................................................... 53 Use Case Design ........................................................................................... 53 IV. ARCHITECTURE AND DESIGN ................................................................. 65 iv A model for a grid based environment .......................................................... 66 SILENUS architectural model ....................................................................... 67 Components ................................................................................................... 70 Service user interface ............................................................................ 70 WebDAV adapter .................................................................................. 71 NFS adapter ........................................................................................... 72 File store ................................................................................................ 73 Metadata store ....................................................................................... 76 Byte store ............................................................................................... 77 Optimizer ............................................................................................... 78 Component Use Cases ................................................................................... 78 Browse files use case ............................................................................ 80 Push upload file use case ...................................................................... 80 Pull upload file use case ........................................................................ 81 Non-caching download file use case ..................................................... 82 Caching download file use case ............................................................ 82 Use cases for Service-oriented programs .............................................. 83 File system attributes ..................................................................................... 85 Transparency .......................................................................................... 85 Concurrent File Updates .......................................................................