Arla—A Really Likeable AFS-Client Johan Danielsson

Total Page：16

File Type：pdf, Size：1020Kb

Arla—a really likeable AFS-client Arla—a really likeable AFS-client Johan Danielsson free Parallelldatorcentrum, KTH [email protected] Assar Westerlund portable Swedish Institute of Computer Science [email protected] efficient supports disconnected operation and encryption of the data stream 1 2 What is AFS? AFS (cont.) a world-wide distributed file system files are stored at dedicated file servers also known as Andrew File System untrusted clients (cache managers) cache files on local disk originally developed at CMU clients have to prove themselves to servers later commercialised by Transarc currently 150 public cells around the world 3 4 Consistency in AFS Implementation of Arla when retrieving a file, the client gets a promise a kernel module (xfs) that the server will notify it before changing the a user-level daemon (arlad) file these communicate by sending messages over a the notification is called a callback character device allows the client to read cached files without any network activity 5 6 Advantages Kernel module (xfs) easier development small, 5000 lines, 32 KB on i386 better portability implements better development tools a system call possible to use normal libraries a character device a virtual file system Disadvantages performance? 7 8 Implementation (cont.) Performance network UFS cold Arla cold Arla warm /bin/ls arlad (I) mkdir 22 3 (II) cp 61811 User-level (III) r.stat 22 3 (IV) r.grep 55 5 (V) compile 34 36 34 VFS XFS Kernel (elapsed time in seconds of Andrew Benchmark on a ThinkPad 560) 9 10 Portability Portability (cont.) Kernel modules for: 10% operating system dependent SunOS lwp requires machine-specific context switch code Solaris NetBSD, FreeBSD, OpenBSD daemon runs on any Unix-like system Linux AIX worksonNTwithcygwin32 HP-UX Digital Unix 11 12 Security and Encryption Disconnected operation Kerberos 4-based (rxkad) allow file system operations without network connectivity implementation written outside US read from already cached data supports encryption of all data write operations to a log and replay later 13 14 Disconnected operation (cont.) Future work fetch on miss no fetch on miss consistent connected N/A more performance not consistent fetch-only disconnected more platforms more disconnected operations file + database servers 15 16 Acknowledgments Availability It’s all Bjorn¨ Gronvall’s¨ fault http://www.stacken.kth.se/projekt/arla/ Magnus Ahltorp Robert Burgess Artur Grabowski Love Hornquist-¨ Astrand˚ and lots of other people (see THANKS) 17 18.

Copy Link

Recommended publications

Love Hörnquist-Åstrand Assar Westerlund Harald Barth
Arla A Free AFS implementation from KTH Edition 0.1, for version 0.34 1999 - 2000 Love Hörnquist-Astrand˚ Assar Westerlund Harald Barth last updated $Date: 2000/10/01 19:18:41 $ Copyright (c) 1998 - 1999 Kungliga Tekniska Högskolan (Royal Institute of Technology, Stock- holm, Sweden). All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of condi- tions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by Kungliga Tekniska Högskolan and its contributors. 4. Neither the name of the Institute nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE INSTITUTE AND CONTRIBUTORS \AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PAR- TICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE INSTITUTE OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THE- ORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (IN- CLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
[Show full text]
Using the Andrew File System on *BSD [email protected], Bsdcan, 2006 Why Another Network Filesystem
Using the Andrew File System on *BSD [email protected], BSDCan, 2006 why another network filesystem 1-slide history of Andrew File System user view admin view OpenAFS Arla AFS on OpenBSD, FreeBSD and NetBSD Filesharing on the Internet use FTP or link to HTTP file interface through WebDAV use insecure protocol over vpn History of AFS 1984: developed at Carnegie Mellon 1989: TransArc Corperation 1994: over to IBM 1997: Arla, aimed at Linux and BSD 2000: IBM releases source 2000: foundation of OpenAFS User view <1> global filesystem rooted at /afs /afs/cern.ch/... /afs/cmu.edu/... /afs/gorlaeus.net/users/h/hugo/... User view <2> authentication through Kerberos #>kinit <username> obtain krbtgt/<realm>@<realm> #>afslog obtain afs@<realm> #>cd /afs/<cell>/users/<username> User view <3> ACL (dir based) & Quota usage runs on Windows, OS X, Linux, Solaris ... and *BSD Admin view <1> <cell> <partition> <server> <volume> <volume> <server> <partition> Admin view <2> /afs/gorlaeus.net/users/h/hugo/presos/afs_slides.grafﬂe gorlaeus.net /vicepa fwncafs1 users hugo h bram <server> /vicepb Admin view <2a> /afs/gorlaeus.net/users/h/hugo/presos/afs_slides.grafﬂe gorlaeus.net /vicepa fwncafs1 users hugo /vicepa fwncafs2 h bram Admin view <3> servers require KeyFile ~= keytab procedure differs for Heimdal: ktutil copy MIT: asetkey add Admin view <4> entry in CellServDB >gorlaeus.net #my cell name 10.0.0.1 <dbserver host name> required on servers required on clients without DynRoot Admin view <5> File locking no databases on AFS (requires byte range locking)
[Show full text]
The 3Ourn L of AUUG Inc. Volume 25 ¯ Number 4 December 2004
The 3ourn l of AUUG Inc. Volume 25 ¯ Number 4 December 2004 Features: A Convert to the Fold 7 Lions Commentary, part 1 16 News: Minutes of AUUG Annual General Meeting, 1 September 2004 54 AUUG 2005 annual conference: CFP 58 First Australian UNIX Developer’s Symposium: CFP 59 First Digital Pest Symposium 60 Regulars: Editorial 1 President’s Column 3 My Home Network 4 This Issue’s CD 29 The Future of AUUG CDs 30 A Hacker’s Diary 31 AUUG Corporate Members 56 Letters to AUUG 56 About AUUGN 61 Chapter Meetings and Contact Details 62 AUUG Membership Application Form 63 ISSN 1035-7521 Print post approved by Australia Post - PP2391500002 AUUGN The journal of AUUG Inc. Volume 25, Number 3 September 2004 Editor ial Frank Crawford <[email protected]> Well, after many, many years of involvement with mittee, preparing each edition. Curr ently, this AUUGN, I’ve ﬁnally been roped into writing the consists of Greg Lehey and myself, but we are editorial. In fact, AUUGN has a very long and keen to expand this by a few more, in an effort to distinguished history, providing important infor- spr ead the load. And as with previous changes, mation to generations of Unix users. During that we have a “new” approach to ﬁnding contribu- time, therehave been a range of editors all of tions. AUUG has a huge body of work, from whom have guided it through ups and downs. both the Annual Conference and regional meet- Certainly you will know many of the recent ones, ings that should be seen morewidely, especially such as David Purdue (current AUUG President), by those who weren't able to attend these events.
[Show full text]
Using the Andrew File System with BSD
Using the Andrew File System with BSD H. Meiland May 4, 2006 Abstract Since the beginning of networks, one of the basic idea’s has been sharing of files; even though with the Internet as advanced as today, simple platform independent file sharing is not common. Why is the closest thing we use WebDAV, a ’neat trick over http’, instead of a real protocol? In this paper the Andrew File System will be described which has been (and is) the file sharing core of many universities and companies world- wide. Also the reason for it’s relative unawareness in the community will be answered, and it’s actual features and performance in comparison with alternative network filesystems. Finally some information will be given on how to use it with our favorite OS: BSD. 1 History • 1984 Carnegie Mellon University, first release • 1989 TransArc Corporation formed by part of original team members • 1994 TransArc purchased by IBM • 1997 Start of Arla development at stacken.kth.se • 2000 IBM releases AFS in opensource (IBM License) • 2000 http://www.OpenAFS.org • 2006 good support for lot’s of platforms, many new features etc. 1 2 Overview 2.1 User point of view 2.1.1 Global namespace While discussing global filesystem, it is easy to dive into a organization, and explain wonderfull features like having replicas of often accessed data in branch-offices, and moving home-directories to local fileservers when mov- ing employees between departments. An essential feature of AFS is often overlooked: a common root as accesspoint of all AFS stored data.
[Show full text]
A Technical Summary
A Technical Summary Table A.1. Kerberos administrative ACL (MIT) Allow Deny Operation a A Add principals or policies d D Delete principals or policies m M Modify principals or policies c C Change passwords i I Inquiry the database l L List principals or policies s S Set key for principals * or x Allow all privileges Table A.2. Kerberos kadmin password policies (MIT) Option Meaning -history Minimum count of unusable old passwords -maxlife Maximum allowed lifetime -minclasses Minimum character classes -minlength Minimum password length -minlife Minimum lifetime 380 A Technical Summary Table A.3. LDAP access control list levels Level Privilege Coding Explanation none =0 No access at all auth =x Permits authentication attempt compare =cx Permits comparison search =scx Permits search filter application read =rscx Permits search result inspection write =wrscx Permits modification or deletion Table A.4. The OpenAFS access control list attributes Permission Meaning l List contents (lookup) i Create new files or directories (insert) d Delete files or directories a Change ACL attributes of directories (administer) r Read contents of files w Modify contents of files (write) k Lock files for reading read Equivalent to rl write Equivalent to rlidwk (no administrative rights) all Equivalent to rlidwka none Remove all ACL permissions Table A.5. The OpenAFS @sys names (excerpt) @sys Architecture alpha dux40 Digital UNIX 4 on an Alpha alpha dux50 Digital UNIX 5 on an Alpha i386 linux24 Linux Kernel 2.4 on Intel and compatible i386 linux26 Linux Kernel 2.6 on Intel and compatible i386 nt40 Microsoft Windows NT and later on Intel and compatible ppc darwin70 Apple MacOS X 10.3 on a PowerPC Macintosh ppc darwin80 Apple MacOS X 10.4 on a PowerPC Macintosh ppc darwin90 Apple MacOS X 10.5 on a PowerPC Macintosh rs aix52 IBM AIX 5.2 on a pSeries rs aix53 IBM AIX 5.3 on a pSeries sgi 65 SGI Irix 6.5 on a MPIS x86 darwin80 Apple MacOS X 10.4 on an Intel Macintosh x86 darwin90 Apple MacOS X 10.5 on an Intel Macintosh A Technical Summary 381 Table A.6.
[Show full text]
Arla—A Free AFS Client
Arla—a free AFS client Assar Westerlund Swedish Institute of Computer Science [email protected] Johan Danielsson Parallelldatorcentrum, KTH [email protected] Abstract During the development of AFS3, it was taken over by a spin-off company, Transarc. They later renamed it to the AFS is a world-wide distributed file system that supports ‘AFS File System’. When this paper only says AFS it refers several advantages over NFS, like better administration and to AFS3. availability. Arla is a free AFS-client, today available for a number of systems for which there are not any Transarc AFS is being used by organisations around the world, and clients. Porting it to new systems is not hard. Arla also there are currently approximately 150 public cells – a cell is has some disconnected functionality and supports encryp- a part of the global AFS name space managed by a partic- tion of the data stream, features that are not present in the ular organisation. All the users of AFS see the same name Transarc code. Our preliminary measurements of the un- space and can transparently access files in other cells (sub- optimized code shows that the performance of Arla is close ject to access rights). to that of UFS. 2.1 Model 1 Introduction In the AFS model, files are stored on file servers.These AFS is a world-wide distributed file system, originally servers are preferably dedicated to this task. The clients (or from Carnegie Mellon University (CMU), but now com- cache managers) do not need to be trusted at all and have to mercialised by Transarc Corporation.
[Show full text]
A UNIX Filesystem for Smartcards
CITI Technical Rep ort 98-8 SCFS: A UNIX Filesystem for Smartcards Naomaru Itoi [email protected] Peter Honeyman [email protected] Jim Rees [email protected] Abstract Smartcard software develop ers su er from the lack of a standard communication framework b etweenaworkstation and a smartcard. To address this problem, we extended the UNIX lesystem to provide ac- cess to smartcard storage, which enables us to use les in a smartcard as though normal UNIX les. Decemb er 3, 1998 Center for Information Technology Integration University of Michigan 519 West William Street Ann Arb or, MI 48103-4943 2 itoi 1 Intro duction On-chip software standards: JavaCard [14] and MULTOS [15]. Today, it is easy to purchase smartcards in rea- Although these standards provide abstrac- sonable prices, e.g., $5 - $20 for each. How- tions at a higher level than ISO-7816-4, it re- ever, smartcard software development is hard: mains a challenging task for develop ers to se- smartcard software develop ers have long suf- lect a standard, purchase all software and hard- fered from the lack of a user friendly stan- ware required, learn API and to ols, and nally dard communication proto col b etween appli- 1 implement software. Furthermore, those stan- cation software and a smartcard. The ISO- dards do not eliminate problems with interop- 7816 communication proto col [9] is so widely erability{ e.g., OCF limits the programming accepted that virtually all smartcards supp ort 2 language to Java; PC/SC is used only with it. However, the proto col is not a particularly Windows { and create their own API dep en- desirable one: dencies, b ecause software written for one stan- It is a primitive message passing proto- dard do es not run with another.
[Show full text]
Conference Reports From
THE MAGAZINE OF USENIX & SAGE August 2002 volume 27 • number 5 inside: CONFERENCE REPORTS USENIX 2002 & The Advanced Computing Systems Association & The System Administrators Guild conference reports 2002 USENIX Annual KEYNOTE ADDRESS Technical Conference THE INTERNET’S COMING SILENT SPRING Lawrence Lessig, Stanford University MONTEREY, CALIFORNIA, USA OUR THANKS TO THE SUMMARIZERS: Summarized by David E. Ott JUNE 10-15, 2002 For the USENIX Annual Technical Conference: In a talk that received a standing ova- Josh Simon, who organized the collecting of ANNOUNCEMENTS tion, Lawrence Lessig pointed out the the summaries in his usual flawless fashion Summarized by Josh Simon recent legal crisis that is stifling innova- Steve Bauer tion by extending notions of private Florian Buchholz The 2002 USENIX Annual Technical Matt Butner Conference was very exciting. The gen- ownership of technology beyond rea- Pradipta De eral track had 105 papers submitted (up sonable limits. Xiaobo Fan 28% from 82 in 2001) and accepted 25 Hai Huang Several lessons from history are instruc- Scott Kilroy (19 from students); the FREENIX track tive: (1) Edwin Armstrong, the creator Teri Lampoudi had 53 submitted (up from 52 in 2001) of FM radio technology, became an Josh Lothian and accepted 26 (7 from students). enemy to RCA, which launched a legal Bosko Milekic campaign to suppress the technology; Juan Navarro The two annual USENIX-given awards David E. Ott were presented by outgoing USENIX (2) packet switching networks, proposed Amit Purohit Board President Dan Geer. The USENIX by Paul Baron, were seen by AT&T as a Brennan Reynolds Lifetime Achievement Award (also new, competing technology that had to Matt Selsky known as the be suppressed; (3) Disney took Grimm J.D.
[Show full text]
Openbsd Frequently Asked Questions
OpenBSD Frequently Asked Questions Language: en [teams] de fr nl pl pt Documentation and Frequently Other Documents Asked Questions Upgrade Guide Following -current Commonly Encountered Issues Recent updates Port Testing Guide Using AnonCVS Stable This FAQ is supplemental documentation to the man pages, available both in the installed Using CVSup system and online. The FAQ covers the active release of OpenBSD, currently v3.8. There Manual pages are likely features and changes to features in the development version (-current) of Bug Reporting OpenBSD that are not covered in this FAQ. Mail lists PF User's Guide The FAQ in PDF and plain text form is available in the pub/OpenBSD/doc directory OpenSSH FAQ from the FTP mirrors, along with other documents. PDF files OpenBSD FAQ PF User's Guide 1 - Introduction to OpenBSD Text files ● 1.1 - What is OpenBSD? ● OpenBSD FAQ 1.2 - On what systems does OpenBSD run? ● PF User's Guide 1.3 - Is OpenBSD really free? ● 1.4 - Why might I want to use OpenBSD? ● 1.5 - How can I help support OpenBSD? Back to OpenBSD ● 1.6 - Who maintains OpenBSD? ● 1.7 - When is the next release of OpenBSD? ● 1.8 - What is included with OpenBSD? ● 1.9 - What is new in OpenBSD 3.8? ● 1.10 - Can I use OpenBSD as a desktop system? ● 1.11 - Why is/isn't ProductX included? 2 - Other OpenBSD Information Resources ● 2.1 - Web Pages ● 2.2 - Mailing Lists ● 2.3 - Manual Pages ● 2.4 - Reporting Bugs 3 - Obtaining OpenBSD ● 3.1 - Buying an OpenBSD CD set ● 3.2 - Buying OpenBSD T-Shirts ● 3.3 - Does OpenBSD provide an ISO image for download? http://www.openbsd.org/faq/index.html (1 of 7) [19/04/2006 23:03:44] OpenBSD Frequently Asked Questions ● 3.4 - Downloading via FTP, HTTP or AFS ● 3.5 - Obtaining Current Source Code 4 - OpenBSD 3.8 Installation Guide ● 4.1 - Overview of the OpenBSD installation procedure.
[Show full text]
Openbsd Frequently Asked Questions
OpenBSD Frequently Asked Questions Language: [en] [cs] [de] [es] [ja] [ko] Documentation and Frequently Other Documents Asked Questions Upgrade-MiniFAQ Ports and Packages Commonly Encountered Issues Recent updates Port Testing Guide Updating via AnonCVS Stable This FAQ is supplemental documentation to the man pages, available both in the installed CVSup system and online. The FAQ covers the active release of OpenBSD, currently v3.2. Note Manual pages that the development version (-current) of OpenBSD is not covered by this FAQ. Bug Reporting Mail lists The FAQ is available in PDF and plain text form in the pub/OpenBSD/doc directory from the FTP mirrors. PDF files FAQ in PDF form Text files 1 - Introduction to OpenBSD FAQ in Text form ● 1.1 - What is OpenBSD? ● 1.2 - On what systems does OpenBSD run? Back to OpenBSD ● 1.3 - Is OpenBSD really free? ● 1.4 - Why might I want to use OpenBSD? ● 1.5 - How can I help support OpenBSD? ● 1.6 - Who maintains OpenBSD? ● 1.7 - When will be the next release of OpenBSD? 2 - Other OpenBSD Information Resources ● 2.1 - Web Pages ● 2.2 - Mailing Lists ● 2.3 - Manual Pages ● 2.4 - Reporting Bugs 3 - Obtaining OpenBSD ● 3.1 - Buying an OpenBSD CD ● 3.2 - Buying OpenBSD T-Shirts ● 3.3 - Does OpenBSD provide an ISO image for download? ● 3.4 - Downloading via FTP or AFS ● 3.5 - Obtaining Current Source Code 4 - OpenBSD 3.2 Installation Guide http://www.openbsd.org/faq/index.html (1 of 6) [06/04/2003 11:46:48] OpenBSD Frequently Asked Questions ● 4.1 - Overview of the OpenBSD Installation Procedure.
[Show full text]
The BSD Associate Study Guide the BSD Associate Study Guide: the Beginning BSD Unix Administration Book
The Beginning BSD Unix Administration Book The BSD Associate Study Guide The BSD Associate Study Guide: The Beginning BSD Unix Administration Book November 24, 2011 Editor: Jeremy C. Reed Book Wiki: http://bsdwiki.reedmedia.net/ Copyright c 2006-2011 BSD Certiﬁcation Group, Inc. Permission to use, copy, modify, and distribute this documentation for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. THE DOCUMENTATION IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WAR- RANTIES WITH REGARD TO THIS DOCUMENTATION INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN AC- TION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS DOCUMENTATION. NetBSD and pkgsrc are registered trademarks of the NetBSD Foundation, Inc. FreeBSD is a registered trademark of the FreeBSD Foundation. Contents Introduction vii 1 Installing and Upgrading the OS and Software 1 1.1 Recognize the installation program used by each operating system . 1 1.2 Recognize which commands are available for upgrading the operating system . 5 1.3 Understand the difference between a pre-compiled binary and compiling from source . 6 1.4 Understand when it is preferable to install a pre-compiled binary and how to do so . 7 1.5 Recognize the available methods for compiling a customized binary . 7 1.6 Determine what software is installed on a system .
[Show full text]
Introduction to AFS and Its Best Practices by A. Wachsmann
IntroductionIntroduction toto AFSAFS andand itsits AdministrationAdministration Alf Wachsmann (SLAC) Introduction to AFS and its Best Practices © 2004-2009 by A. Wachsmann 1 Today's Plan 9:00 – 10:00 General Introduction to AFS 10:00 – 10:30 AFS Client Topics 10:30 – 11:00 Coffee Break 11:30 – 12:30 AFS Server Topics 12:30 – 13:30 Lunch Break 13:30 – 15:00 AFS Server Topics cont'd. 15:00 – 15:30 Coffee Break 15:30 – 17:00 Advanced AFS Topics Introduction to AFS and its Best Practices © 2004-2009 by A. Wachsmann 2 Copyright Note • This material is Copyright © 2004-2009 by Alf Wachsmann. • All rights reserved. • Use without permission is prohibited. • Supported in part by U.S. Department of Energy contract DE-AC02-76SF00515. • Version 8.0 Introduction to AFS and its Best Practices © 2004-2009 by A. Wachsmann 3 I. General Introduction to AFS Introduction to AFS and its Best Practices © 2004-2009 by A. Wachsmann 4 History of AFS • 1983: Andrew project started at CMU • 1988: AFSv3 at CMU; first use outside of CMU • 1989: original design team left CMU to form Transarc Corporation and AFS went commercial • 1993: Arla project at KTH in Sweden started • 1998: Transarc becomes wholly-owned subsidiary of IBM • 2000: IBM releases OpenAFS as open source • AFS stood for “Andrew File System” after the “Andrew Computing Project” named after Andrew Carnegie and Andrew W. Mellon, the founders of CMU Introduction to AFS and its Best Practices © 2004-2009 by A. Wachsmann 5 What is AFS? • “AFS is a distributed filesystem that enables co- operating hosts (clients and servers) to efficiently share filesystem resources across both local area and wide area networks” (AFSWiki) /afs/ openafs.org/ ..
[Show full text]