This work is protected by copyright and other intellectual property rights and duplication or sale of all or part is not permitted, except that material may be duplicated by you for research, private study, criticism/review or educational purposes. Electronic or print copies are for your own personal, non- commercial use and shall not be passed to any other individual. No quotation may be published without proper acknowledgement. For any other use, or to quote extensively from the work, permission must be obtained from the copyright holder/s. RELIABLE FILE STORAGE IN A DISTRIBUTED COMPUTING SYSTEM by Kenneth Lunn A thesis presented in support of an application for the degree of Doctor of Philosophy in the University of Keele March 1982 1.0 ABSTRACT....................................................... 2 2.0 ACKNOWLEDGEMENTS ........................................... 4 CHAPTER 1 INTRODUCTION 1 . 1 THE P R O B L E M ................................................ 1-1 1.2 THE APPLICATION A R E A ..................................... 1-4 1.3 A SOLUTION TO THE PROBLEM OF RELIABLE FILE STORAGE 1-5 1.4 CONCLUSION.................................................. 1-9 CHAPTER 2 RELIABILITY THEORY 2.1 INTRODUCTION................................................2-1 2.2 SYSTEMS AND THEIR FAILURES ............................ 2-2 2.2.1 S y s t e m s .................................................. 2-2 2.2.2 Specification.......................................... 2-3 2.2.3 Reliability ............................................. 2-4 2.2.4 Errors, Faults And Failures ........................ 2-5 2.2.5 Fault A v o i d a n c e ........................................2-6 2.2.6 Fault T o l e r a n c e ........................................2-7 2.2.7 Fault Tolerance Techniques .......................... 2-8 2.2.7.1 Protective R e d u n d a n c y ...............................2-8 2.2.7.2 Error D e t e c t i o n ...................................... 2-9 2.2.7.3 Fault T r e a t m e n t ..................................... 2-11 2.2.7.4 Damage Assessment .................................. 2-13 2.2.7.5 Error Recovery........................................2-14 2.2.7.6 Component Dependency................................2-17 2.2.7.6.1 Independent Components...........................2-17 2.2.7.6.2 Dependent Components............................. 2-18 2.2.7.6.3 Conclusion.......................................... 2-20 2.3 CONCLUSION..................................................2-20 CHAPTER 3 LOCAL AREA NETWORKS 3.1 INTRODUCTION.................................................3-1 3.2 DEFINITIONS ................................................. 3-3 3.2.1 Communications...........................................3-3 3.2.1.1 The Cambridge R i n g ...................................3-4 3.2.2 P r o t o c o l s ................................................3-6 3.2.3 Homogeneous And Heterogeneous Systems .......... 3-7 3.2.4 Autonomy.................................................. 3-8 3.2.5 Servers And C l i e n t s ..................................3-10 3.3 PROS AND CONS OF LOCAL AREA NETWORK SYSTEMS . 3-11 3.4 SURVEY OF EXISTING LOCAL AREA NETWORK COMPUTER SYSTEMS ................................................... 3-15 3.4.1 Cambridge Model Distributed System .............. 3-15 3.4.2 Xerox Ethernet......................................... 3-18 3.4.3 Apollo Domain . ^ ..................................3-20 3.4.4 Z - N e t .................................................3-21 3.4.5 Dec N e t .................................................3-21 3.4.6 Unix Satellite Processor System ............... 3-22 3.4.7 Other Distributed Unix Systems .................. 3-23 Page 2 3.5 CONCLUSION............................................3-24 CHAPTER 4 FILE-STORES 4.1 D E F I N I T I O N S ................................................4-1 4.1.1 File-stores, File-server And Fi l e s ...................4-1 4.1.2 Da tabas e .................................................. 4-2 4.1.3 N a m i n g .....................................................4-3 4.1.4 Protection................................................4-6 4.1.5 Mutual Exclusion ...................................... 4-6 4.1.6 Deadlock.................................................. 4-8 4.1.7 Consistency ............................................. 4-9 4.1.8 Atomicity And Transactions ....................... 4-10 4.2 ISSUES IN FILE-STORE DESIGN ........................ 4-11 4.2.1 Data Placement......................................... 4-12 4.2.2 C o n s i s t e n c y ............................................4-13 4.2.3 Shared A c c e s s ......................................... 4-14 4.2.4 Shared U p d a t e ......................................... 4-15 4.2.5 N a m i n g ....................................................4-16 4.3 A SURVEY OF A NUMBER OF EXISTING FILE-STORES . 4-18 4.3.1 U n i x ...................................................... 4-18 4.3.2 GEC 0S4000 ........................................... 4-21 4.3.3 OS 360 ................................................ 4-22 4.3.4 Xerox D F S .............................................. 4-23 4.3.5 Apollo D o m a i n ......................................... 4-26 4.3.6 George I I I ...............................................4-27 4.3.7 Other Unix-like Distributed File-stores . 4-29 CHAPTER 5 REFINEMENT OF THE PROBLEM CHAPTER 6 KUDOS 6.1 THE KEELE UNIVERSITY DISTRIBUTED OPERATING SYSTEM 6-1 6.2 AIMS OF K U D O S ............................................. 6-2 6.2.1 General Objectives .................................... 6-3 6.2.2 Objectives Within Our Application ............... 6-4 6.2.3 Experience................................................6-6 6.3 KUDOS S T R U C T U R E ................................... 6-10 6.3.1 The KUDOS Message Passing Scheme ............... 6-11 6.3.2 Client-Server Architecture ....................... 6-16 6.3.3 Public/Private Domain Architecture ............. 6-16 6.3.4 The Conceptual Structure.............................6-18 6.3.4.1 The Personal Autonomous WorkStation .... 6-22 6.3.4.2 The Multi-Access Shared System ................ 6-23 6.3.4.3 The Public System Interface ................ 6-23 6.3.5 Object Naming, Addressing And Location .... 6-23 6.3.5.1 Resources ......................................... 6-24 6.3.5.2 Addressing . ....................................... 6-25 6.3.5.3 Processing .......................................... 6-25 6.3.5.4 Resource Location ............................... 6-26 6.3.5.5 Some Effects Of Node Crashes And Start-Ups . 6-29 6.3.5.6 Alternative Schemes ............................. 6-30 Page 3 6.4 KUDOS IMPLEMENTATION TO D A T E .......................... 6-32 6.4.1 File-store.............................................. 6-32 6.4.2 UCSD Filer Interface..................................6-32 6.5 EXPERIENCE AND CONCLUSIONS ......................... 6-33 6.5.1 Hardware.................................................6-33 6.5.2 Software.................................................6-35 6.6 OVERALL CONCLUSIONS ................................. 6-40 CHAPTER 7 KUDOS FILE-STORE - DESCRIPTION 7.1 THE KUDOS FI L E - S E R V E R...................................7-8 7.1.1 Introduction............................................. 7-8 7.1.2 File-server Primitives ............................. 7-10 7.1.2.1 Create And Delete ............................... 7-12 7.1.2.2 Expand And S h r i n k .................................. 7-12 7.1.2.3 Read And W r i t e ........................................7-13 7.1.2.4 Failures ............................................ 7-13 7.1.2.5 Desirable Extensions To The Primitives . 7-14 7.1.3 File-server Protection ............................. 7-17 7.1.4 File-server Reliability .......................... 7-19 7.2 THE DIRECTORY S Y S T E M .....................................7-22 7.2.1 General Description .............................. 7-22 7.2.1.1 Hierarchy ......................................... 7-22 7.2.1.2 The Active F i l e - s t o r e ............................. 7-23 7.2.1.3 Associated Volumes ............................... 7-24 7.2.1.4 Mount/dismount .................................... 7-27 7.2.1.5 Directory R e s o l v i n g ................................7-29 7.2.1.6 Timestamping ....................................... 7-31 7.2.1.7 Atomicity .......................................... 7-32 7.2.2 File P r o t e c t i o n ........................................7-35 7.2.2.1 System Defined Capabilities .................. 7-35 7.2.2.2 User Defined Capabilities ..................... 7-36 7.2.2.3 Locking ............................................ 7-37 7.2.3 Directory System Primitives .. ................ 7-40 7.2.3.1 Activate_subdirectory .......................... 7-42 7.2.3.2 Read_associated_volumes ........................ 7-44 7.2.3.3 Create_subdirectory And Delete_subdirectory 7-44 7.2.3.4 I n s e r t _ f i l e .......................................... 7-45 7.2.3.5 Find_file .......................................... 7-45 7.2.3.6 Delete_file ....................................... 7-46 7.2.3.7 Read_lock, Writ e_ lock, Refresh_lock And En d_ lo c k .............................................. 7-46 7.2.3.8 L i s t .................................................... 7-47 7.2.3.9 New_read_permit And New_write_permit .... 7-48 7.3 CLIENT VIEW OF F I L E - S T O R E ..............................7-49 7.3.1 Fi l e-server/directory Relationship With Client 7-49 7.3.2 Client Error Detection/Recovery ..............