RC24788 (W0904-091) April 21, 2009 Computer Science IBM Research Report Service Oriented File Systems Eric Van Hensbergen, Noah Evans IBM Research Division Austin Research Laboratory 11501 Burnet Road Austin, TX 78758 Phillip Stanley-Marbell IBM Research GmbH Zurich Research Laboratory 8803 Rüschlikon Switzerland Research Division Almaden - Austin - Beijing - Cambridge - Haifa - India - T. J. Watson - Tokyo - Zurich Service Oriented File Systems Eric Van Hensbergen Noah Evans Phillip Stanley-Marbell IBM Research Austin IBM Research Austin IBM Research Zurich [email protected] [email protected] [email protected] Abstract RESTful applications differ from their RPC counter- parts in relying principally on resource identifiers versus Service Oriented Architectures (SOAs) are a loose command methods. In web environments, this results in coupling of network services providing methods for sys- operational semantics being encoded as part of the URL tems development and integration. Interoperability be- of HTTP requests, and relying on HTTP methods (GET, tween different systems and programming languages is PUT, POST, DELETE) as the sole methods. This al- provided via communication protocols and well defined lows service components to access and navigate service messages. The recent development trend has been to infrastructures without requiring complete knowledge of favor RESTful approaches for these interfaces, which the resource capabilities or data structures. Existing run- encode relevant context and semantic metadata into the times each use their own language-specific bindings for URL of an HTTP GET or PUT operation. accessing and addressing these service methods. We observe that this approach is essentially a sim- plified web-instantiation of synthetic file system based It is our belief that it is the responsibility of systems service interfaces, such as those originally pioneered by software to provide a unified language- and network- UNIX and later the Plan 9 and Inferno operating sys- neutral interface to its users. As such, we believe that tems. In this paper we advocate the collapse of the the operating system should provide facilities for ad- software stack by abstracting the underlying transport dressing and interaction of SOA interfaces. We feel that and naming details, and accessing RESTful services via enabling interaction via system-provided interfaces en- standard file system interfaces. We explore the research ables a new degree of composability and flexibility in challenges and opportunities presented by taking such much the same way as pipes unlocked tool-based ap- an approach to building comprehensive dynamic dis- proaches in UNIX [10]. tributed systems appropriate for large scale cloud com- Our proposal is motivated by our observation that the puting. semantically significant URL path component of REST- ful operations is similar in many ways to synthetic file 1 Introduction system environments such as UNIX’s /proc, or those used in the Plan 9 or Inferno operating systems (Fig- Service-oriented architectures (SOAs) are networked ure 1). The introduction of the File Systems in User software infrastructures in which resources are made Space (FUSE) and 9P file systems into the mainline available through a transactional interface [19]. They Linux kernel has further enabled the use of such file typically comprise collections of reusable application system interfaces as an applications component in main- modules, each of which expose standardized interfaces. stream environments. We propose the use of such mech- The use of well-defined protocols and message for- anisms to allow RESTful interfaces to be managed, or- mats to communicate between these services decou- ganized, and interconnected by the operating system in- ples application implementations from one another, pro- stead of purely by an SOA runtime. It is our belief viding greater degrees of flexibility and composability. that using synthetic file systems to establish operating The popularity of the web has lead to many SOA run- system based uniform interface abstractions and appli- times adopting HTTP encapsulated protocols such as cation orchestration will further enable the benefits of SOAP [12]. SOAs while also enabling a seamless distributed com- The emergence of Web 2.0 and cloud computing has puting environment across platforms to enable large- extended the SOA development paradigm from busi- scale “cloud” environments. ness process services to complete computing environ- ments, involving everything from the customer-facing interfaces to the back-end database. These new classes 2 Background of application have evolved from client-server based tra- UNIX pioneered the concept of treating devices as files, ditional RPC exemplified by SOAP, to representational providing a uniform interface to system hardware. In the state transfer (REST) [6] mechanisms as their common 8th edition, this methodology was taken further through communication architecture. the introduction of the /proc synthetic file system to syscontext/ clone http://y.com/sys/spucontext mem 0/ [An HTTP transaction (e.g., GET) to achieve resource run 1/ instance creation, for instance syscontext . This returns mbox ... an identifier pointing to the new instance. ] wbox n/ http://y.com/sys/1/mem ibox ctl [URI captures instantiated identifier and resource to be data accessed. Payload of HTTP request contains data for this interaction. ] ... (a). An example "Old Unix (b). A typical dynamic (c). An example RESTful interface to a system service style" virtual filesystem, fileserver in Plan 9/Inferno; exposed over HTTP. SYSFS; data exchange via file abstractions are used files, but control and resource for resource instance instance creation via ioctl() creation ( clone ), control and create() system calls. (ctl ) and data ( data ). Figure 1: Illustration of a service–oriented file system interface manage user processes [8]. Synthetic file systems are files and read/write the necessary data to the proper lo- comprised of elements with no physical storage, i.e., the cation. files represented are not present on any disk. Instead, op- In addition to providing language portability, syn- erations on the file communicate directly with the kernel thetic file systems provide data portability as well. They sub-system or application providing the service. This can act as intermediaries between the physical data and methodology has persisted over the evolution of UNIX the presentation, interpreting data in the most appropri- and UNIX-like systems, and the Linux kernel now sup- ate way for a given application. Again this moves the ports several synthetic file systems representing devices, responsibility for ensuring data from the user to system, process control, as well as access to system services and removing much of the burden of data conversion from data structures. the user. For example files of one type, say xml data, The Plan 9 [14] and Inferno [15] operating systems can be presented as yaml and vice versa. Further exam- took the file system metaphor further, using file opera- ples of this process appear in [5]. tions as the simple, well-defined interface to all system Synthetic file systems have the unique property of be- and application services. As such, interfaces to all kernel ing implementable either within the operating system or subsystems, from the networking stack to the graphics in user space, but still providing a single consistent in- frame buffer, are represented within synthetic file sys- terface through the operating system’s file system name tems. User-space applications and services may also ex- space. Using synthetic file systems as interfaces for port their own synthetic file systems, in much the same software implementations decouples software function- way as the kernel interfaces. Common services such as ality from decisions about implementation location, pro- domain name service (DNS), authentication databases, gramming language bindings, or runtime system sup- and window management are all provided as file sys- port. tems. Even end-user applications such as editors and e-mail systems export file system interfaces as a means 3 Approach for data exchange and control. The benefits and details Following Plan 9’s example, we propose the pervasive of this approach are covered in detail elsewhere [16]. use of synthetic file systems as a resource interface ab- The intuition behind the design was based on the straction, and an augmented form of UNIX pipelines to- assumption that any programmer knows how to inter- gether with application of dynamic private name spaces act with files. Every programming language has the to achieve this goal. We believe that by collapsing the in- means of interacting with file systems, and network pro- frastructure of service-oriented architecture stacks back tocols (including HTTP) for remote access of file hier- into the system, we can return the flexibility and ele- archies are readily available. Application frameworks gance of the composable modular approach exemplified implemented as file systems eliminate many of the op- by UNIX. erating system and language portability problems that The presence of service interfaces within the operat- occur when implementers develop new protocols and ing system name space allows for natural naming and interfaces—the responsibility for the interactions occur addressability, freeing services from location-dependent at the systems level, making a new API unnecessary. numeric identifiers such as IP addresses and port num- Programs just navigate a file hierarchy open