MASARYK UNIVERSITY FACULTY}w¡¢£¤¥¦§¨  OF I !"#$%&'()+,-./012345

Filesystem based grid interface

MASTERTHESIS

Bc. Martin Bukatoviˇc

Brno, 2013

Declaration

Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Advisor: Mgr. Miroslav Ruda

iii

Abstract

The thesis explores the idea of grid user interface build on general filesystem abstrac- tion. Using this approach, operations such as directory listing, reading or writing files are translated to particular grid service requests (eg. job submission). Having desktop user in mind, experimental project called Griddir was designed to wrap grid middleware PBS/Torque and gLite with FUSE based filesystem interface.

v

Keywords grid computing, Genesis II, PBS, Torque, FUSE, filesystem user interface, Linux

vii

Contents

1 Introduction ...... 1 2 Filesystem interfaces ...... 3 2.1 FUSE ...... 3 3 Prior Work ...... 5 3.1 Genesis II ...... 5 3.2 Stroll ...... 5 3.3 Grid-Enabled Desktop Environments (Grenade) ...... 6 3.4 DropAndCompute ...... 6 3.5 Others ...... 6 4 Griddir Design ...... 7 5 Griddir Implementation ...... 9 6 Resume ...... 11 References...... 11

ix

Chapter 1 Introduction

To have a computation processed on a grid, a grid user has to adapt it’s program to par- ticular grid platform and use specialized tools to manage the computation. This hinders adoption to those willing to learn and operate rather complex grid interfaces. With bet- ter user interface, work of domain experts designing scientific simulations or processing data from experiments could be significantly simplified. One of proposed approaches to this problem is to reuse filesystem abstraction. Such interface is represented as ordinary directory tree, but some file operations (eg. creation of a file, reading file) are binded to particular commands. The reasoning behind this idea boils down to widespread familiarity with file operations among most users. While there are several project implementing this approach, unfortunately none of them has reached widespread adoption yet. In addition considering the complexity of current grid systems and multiple possibilities of binding grid commands with filesys- tem operations, there is still large area for further research. For these reasons, the thesis summarizes current state of filesystem interfaces for grid systems and proposes new design targeted to desktop user.

1

Chapter 2 Filesystem interfaces

Filesystem in userspace is a feature of OS kernel which provides ”bridging” mecha- nism so that filesystem implementation can run as ordinary process instead of being executed as a part of the kernel itself. This leads to considerably faster and less complex development process as well as less error prone code as serious prob- lem wouldn’t crash the whole kernel. Furthermore programmer is not limited to the kernel API and can use any userspace library as needed. The first implementation of this concept was introduced in Plan 9 operating system1 [PPT+90]. Although this OS never reached widespread adoption, it pioneered several interesting concepts which were later implemented by other established systems. Currently most widely used im- plementation is Linux FUSE [fus12b], with several ports for other like operating systems [Wik12]. As far as Windows systems are concerned, there is commercial imple- mentation called Callback Filesystem (CBFS) [cbf12] and open source Dokan [dok11].

2.1 FUSE

FUSE [fus12b], which stands for filesystem in userspace, is a Linux framework pro- viding a way for filesystem to be run inside userspace process (so called filesystem daemon). includes FUSE support since version 2.6.14 [ker07], although version for 2.4.x kernels also exists. FUSE consists of 2 main parts: kernel module and userspace library (libfuse). The kernel module works as any other filesystem, but instead of storing filesystem data it- self, the module just provides an interface in a form of special file /dev/fuse. FUSE filesystem daemon (the process which implements filesystem and thus stores the data) uses libfuse userspace library to maintain connection with the kernel module via the special file. So when a filesystem call is performed on FUSE based filesystem, the kernel virtual filesystem (VFS) redirects the request to FUSE kernel module, which then re- sends it to filesystem daemon responsible for the filesystem. The reply from the filesys- tem daemon is send vice versa, as shown in figure 2.1. By default libfuse works in multi threaded mode. This means that for each filesystem request (eg. read(2) call), a dedi- cated thread is created to handle it. To create new FUSE based filesystem, programmer has to provide implementations

1. So called file server in Plan 9 terminology.

3 2. FILESYSTEM INTERFACES

Figure 2.1: FUSE call diagram (image borrowed from FUSE website [fus12b]) of some FUSE operations which roughly match POSIX filesystem API. Some filesystem calls have it’s direct FUSE operation counterpart, eg. to define behaviour of write(2) call, FUSE operation of the same name has to be defined. On the other hand there are two operations for close(2) call: flush() handles every close(2) call for given file descriptor whereas release() is executed only for the last one. Programmer can also omit some fuse operations if they are not necessary. For instance read only FUSE filesystem doesn’t need to have defined just mentioned file closing operations. Besides primary FUSE library providing interface for the language, wrappers for other languages such as C++, Java, Python, Ruby and many others are also available. As a result large number of FUSE based filesystems has been created for various pur- poses [fus12a]. For instance (accessing filesystem of remote machine via ssh, C language), EncFS (file encryption wrapper, C++), s3ql (access to several on-line storage services, Python), WikipediaFS (read and edit wikipedia articles, Python) or Flickrfs (filesystem interface for Flickr, Python).

4 Chapter 3 Prior Work

The idea of using filesystem based interface for grid services is not completely new. On the one hand there are already several grid projects providing user interface inspired by filesystem philosophy to some . On the other hand since there is no natural way how to design such filesystem interface, each project uses unique design specific to particular environment. Moreover projects doesn’t directly refer to each other much. Therefore this chapter provides summary of similar projects along with description of used approach. For the purposes of this summary, a project is considered to be similar if it provides filesystem based user interface for the grid computing system with at least job submission functionality.

3.1 Genesis II

The Genesis II project [gen12] was started at The University of Virginia in 2007 with the main goal to provide first grid computing platform which follows OGSA standards [MG07]. Genesis II is unique among other grid systems because filesystem interface is inte- grated to the project design from the beginning. While the main API is based on web ser- vices, the structure follows proven filesystem abstractions, which allows simple binding between filesystem operations and grid operations.

3.2 Stroll

This project was started at University of Stavanger in 2010 [AM10]. Since then, the project evolved into universal grid interface supporting two different grid computing platforms [AM12]. Stroll is not a grid middleware but rather universal filesystem based grid wrapper designed to completely hide grid interface so the user doesn’t have to learn how to write code for particular grid middleware - all interactions are done via the filesys- tem interface, which can wrap several different grid systems. This approach has several benefits for the users: filesystem operations are easy automate in any programming en- vironment the user is used to, moreover it allows easier porting from one interface to another. Nevertheless users used to particular grid middleware can’t have their’s job

5 3. PRIOR WORK executed via stroll without significant rewrite.

3.3 Grid-Enabled Desktop Environments (Grenade)

Grenade Project [gre04] was started at University of Manchester in 2003 as a grid desk- top integration prototype for Linux using KDE 3.x desktop environment technologies: job submission is compared to sending documents to the printer and KIO slave Kon- queror plugin is used for remote GridTFP volumes [MPH+04]. Unfortunately the project was abandoned in 2004.

3.4 DropAndCompute

3.5 Others

The following projects don’t match intentions of this thesis exactly, but still provide interesting idioms:

• Plan 9: The first project to push filesystem based interfaces beyond traditional UNIX approach.

• UEM

• XtreemFS

6 Chapter 4 Griddir Design

7

Chapter 5 Griddir Implementation

9

Chapter 6 Resume

11

References

[AM10] A. Azab and H. Meling. A virtual file system interface for computational grids. Networked Services and Applications-Engineering, Control and Management, pages 87–96, 2010.

[AM12] A. Azab and H. Meling. Stroll: a universal filesystem-based interface for seamless task deployment in grid computing. In Distributed Applications and Interoperable Systems, pages 162–176. Springer, 2012.

[cbf12] Callback file system, 2012. [Online; accessed 17-09-2012]. URL: http:// www.eldos.com/cbfs/.

[dok11] Dokan — user mode file system for windows, 2011. [Online; accessed 17- 09-2012]. URL: http://dokan-dev.net/en/.

[fus12a] File systems using fuse, 2012. [Online; accessed 20-12-2012]. URL: http://sourceforge.net/apps/mediawiki/fuse/index.php? title=FileSystems.

[fus12b] , 2012. [Online; accessed 28-07-2012]. URL: http: //fuse.sourceforge.net/.

[gen12] Genesis ii wiki — standards-based grid computing, 2012. [Online; ac- cessed 18-09-2012]. URL: http://genesis2.virginia.edu/wiki/ Main/HomePage.

[gre04] Grid enabled desktop environments (grenade), 2004. [Online; accessed 10-08-2012]. URL: http://www.rcs.manchester.ac.uk/research/ grenade.

[ker07] Linux 2.6.14 – linux kernel newbies, 2007. [Online; accessed 15-12-2012]. URL: http://kernelnewbies.org/Linux_2_6_14.

[MG07] M.M. Morgan and A.S. Grimshaw. Genesis ii-standards based grid comput- ing. In Cluster Computing and the Grid, 2007. CCGRID 2007. Seventh IEEE International Symposium on, pages 611–618. IEEE, 2007.

[MPH+04] J. Marsh, S. Pettifer, D. Hanlon, S. Pickles, J. MacLaren, and M. Foster. Grenade: A grid enabled desktop environment. In Enabling Technologies:

13 6. RESUME

Infrastructure for Collaborative Enterprises, 2004. WET ICE 2004. 13th IEEE International Workshops on, pages 343–344. IEEE, 2004.

[PPT+90] R. Pike, D. Presotto, K. Thompson, H. Trickey, et al. Plan 9 from bell labs. In Proceedings of the summer 1990 UKUUG Conference, pages 1–9, 1990. URL: http://plan9.bell-labs.com/sys/doc/9.pdf.

[Wik12] Wikipedia. Filesystem in userspace — wikipedia, the free encyclopedia, 2012. [Online; accessed 17-09-2012]. URL: http://en.wikipedia.org/ w/index.php?title=Filesystem_in_Userspace.

14