File Systems Distributed File Systems
Distributed File Systems
Tran Giang Son, [email protected]
ICT Department, USTH
Distributed File Systems Tran Giang Son, [email protected] 1 / 40 File Systems Distributed File Systems
File Systems
Distributed File Systems Tran Giang Son, [email protected] 2 / 40 • System that permanently stores data • Layered on top of a lower-level physical storage medium • Divided into logical units called “files” • Files ∈ directories • Directories ∈ volume • Directories ∈ directories • Addressable with “Path”
File Systems Distributed File Systems File Systems
Distributed File Systems Tran Giang Son, [email protected] 3 / 40 File Systems Distributed File Systems File Systems
• System that permanently stores data • Layered on top of a lower-level physical storage medium • Divided into logical units called “files” • Files ∈ directories • Directories ∈ volume • Directories ∈ directories • Addressable with “Path”
Distributed File Systems Tran Giang Son, [email protected] 3 / 40 • File • Name • Owner / group • Date • Last access date • ...
File Systems Distributed File Systems Metadata
• Volume • Available space • Formatting info • Character set • ...
Distributed File Systems Tran Giang Son, [email protected] 4 / 40 File Systems Distributed File Systems Metadata
• Volume • Available space • Formatting info • Character set • ...
• File • Name • Owner / group • Date • Last access date • ...
Distributed File Systems Tran Giang Son, [email protected] 4 / 40 File Systems Distributed File Systems Virtual File System
• / • Nested directories • Symlinks • Mount points
Distributed File Systems Tran Giang Son, [email protected] 5 / 40 File Systems Distributed File Systems Low Level Organization
• File data and metadata stored separately • File descriptors + meta-data stored in inodes • Large tree or tables • File content lookups • Replicable
Distributed File Systems Tran Giang Son, [email protected] 6 / 40 File Systems Distributed File Systems Low Level Organization
• Disks: /dev/sdX (Linux) or /dev/diskX (macOS) • Sequential array of blocks • 1KB chunks • Tree structure is flattened into blocks • Fragmentation
Distributed File Systems Tran Giang Son, [email protected] 7 / 40 File Systems Distributed File Systems Fragmentation
A B C (free space)
A B C A (free space)
A (free space) C A (free space)
A D C A D (free)
Distributed File Systems Tran Giang Son, [email protected] 8 / 40 ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems:
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS:
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows:
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices:
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 File Systems Distributed File Systems Local File System
• Very numerous • Unix-like systems: ext2/3/4 btrfs zfs ReiserFS Reiser4 JFS MinixFS UFS UFS2 XFS ZFS SquashFS • macOS: HFS HFS+ APFS • DOS/Windows: FAT FAT16 FAT32 NTFS • Flash devices: JFFS JFFS2 exFAT • Case (in)sensitive • Some has encryption supports: APFS HFS+ btrfs ZFS ext4
Distributed File Systems Tran Giang Son, [email protected] 9 / 40 File Systems Distributed File Systems Local File System on Unix-like Systems
• VFS • UID/GID • File mode for access (RWX) • Superblock, journaling, snapshots
Distributed File Systems Tran Giang Son, [email protected] 10 / 40 • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4 xfs3 ReiserFS JFS XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 • Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4 xfs3 ReiserFS JFS XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 • Where? ext2/3/4 xfs3 ReiserFS JFS XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 ext2/3/4 xfs3 ReiserFS JFS XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where?
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 xfs3 ReiserFS JFS XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 ReiserFS JFS XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4 xfs3
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 JFS XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4 xfs3 ReiserFS
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 XFS btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4 xfs3 ReiserFS JFS
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 btrfs
File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4 xfs3 ReiserFS JFS XFS
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 File Systems Distributed File Systems Journaling
• Log changes to a “journal” before it is commited • For “multiple writes” in an atomic action, e.g. append to file • Update metadata • Allocate space • Write data • Can “playback” journal to recover data quickly
• Log what? • Changes to file content: slow. twice. • Changes to metadata: fast, prone to corruption
• Where? ext2/3/4 xfs3 ReiserFS JFS XFS btrfs
Distributed File Systems Tran Giang Son, [email protected] 11 / 40 File Systems Distributed File Systems RAID
• Redundant Array of Inexpensive/Independent Disks • Levels • RAID0: Striped. Performance. • RAID1: Mirror. Reliability. • RAID5: Striped with distributed parity (no mirror) • RAID6: Striped with dual distributed parity • RAID10: Striped mirrors • Where? btrfs ZFS
Distributed File Systems Tran Giang Son, [email protected] 12 / 40 • Copy of a set of files and directories • A certain point in time
• Types? • Read-only • Read-write: copy-on-write • Supported by filesystem, or LVMs • Backup from read-only snapshots
• Where? UFS ZFS
File Systems Distributed File Systems Snapshot
• What?
Distributed File Systems Tran Giang Son, [email protected] 13 / 40 • Types? • Read-only • Read-write: copy-on-write • Supported by filesystem, or LVMs • Backup from read-only snapshots
• Where? UFS ZFS
File Systems Distributed File Systems Snapshot
• What? • Copy of a set of files and directories • A certain point in time
Distributed File Systems Tran Giang Son, [email protected] 13 / 40 • Where? UFS ZFS
File Systems Distributed File Systems Snapshot
• What? • Copy of a set of files and directories • A certain point in time
• Types? • Read-only • Read-write: copy-on-write • Supported by filesystem, or LVMs • Backup from read-only snapshots
Distributed File Systems Tran Giang Son, [email protected] 13 / 40 File Systems Distributed File Systems Snapshot
• What? • Copy of a set of files and directories • A certain point in time
• Types? • Read-only • Read-write: copy-on-write • Supported by filesystem, or LVMs • Backup from read-only snapshots
• Where? UFS ZFS
Distributed File Systems Tran Giang Son, [email protected] 13 / 40 File Systems Distributed File Systems Problems
• Size • Speed • Large file? small files? • Reads? writes? • Block size • Small : less waste, more overhead • Large : more waste, less overhead
Distributed File Systems Tran Giang Son, [email protected] 14 / 40 File Systems Distributed File Systems
Distributed File Systems
Distributed File Systems Tran Giang Son, [email protected] 15 / 40 File Systems Distributed File Systems What?
• Access to files on remote servers • Concurrency • Consistency • Locking • Support for local caching and replication
Distributed File Systems Tran Giang Son, [email protected] 16 / 40 File Systems Distributed File Systems What?
• Security • Reliability • Consistency • Parallel
Distributed File Systems Tran Giang Son, [email protected] 17 / 40 • File sharing • Backups • Centralized storage system • Size
File Systems Distributed File Systems Why?
Distributed File Systems Tran Giang Son, [email protected] 18 / 40 File Systems Distributed File Systems Why?
• File sharing • Backups • Centralized storage system • Size
Distributed File Systems Tran Giang Son, [email protected] 18 / 40 File Systems Distributed File Systems Examples
• Network File System (NFS) • Andrew File System (AFS) • GlusterFS
Distributed File Systems Tran Giang Son, [email protected] 19 / 40 File Systems Distributed File Systems
NFS • NFS • AFS • GlusterFS
• 1980s by Sun • Most widely known distributed filesystem • Presented with standard POSIX FS interface • Network drives are mounted into local directory hierarchy
Distributed File Systems Tran Giang Son, [email protected] 20 / 40 File Systems Distributed File Systems
NFS • NFS • AFS • GlusterFS
server
client client client
Distributed File Systems Tran Giang Son, [email protected] 21 / 40 File Systems Distributed File Systems
NFS Protocol • NFS • AFS • GlusterFS
• Originally stateless (similar to what?) • UDP, not TCP • File locking • All requests have enough info • E.g. write to what file? at what offset? what content? • TCP ftw.
Distributed File Systems Tran Giang Son, [email protected] 22 / 40 File Systems Distributed File Systems
NFS Overview • NFS • AFS • GlusterFS
• RPC • Client • On top of VFS • vnode to NFS RPCs • Server • Stateless • Write to storage before return
Distributed File Systems Tran Giang Son, [email protected] 23 / 40 File Systems Distributed File Systems
NFS Server • NFS • AFS • GlusterFS
• Use existing file system on server • No local disk layout • Export local disk to clients • /etc/exports • /storage 192.168.0.0/24(rw,sync)
Distributed File Systems Tran Giang Son, [email protected] 24 / 40 File Systems Distributed File Systems
NFS Server • NFS • AFS • GlusterFS
NFS server
User-visible filesystem Server filesystem
EXT3 fs EXT3 fs NFS client EXT2 fs ReiserFS
Hard Drive 1 Hard Drive 2 Hard Drive 1 Hard Drive 2
Distributed File Systems Tran Giang Son, [email protected] 25 / 40 File Systems Distributed File Systems
NFS Locking • NFS • AFS • GlusterFS
• Stateful locking • Client informs servers to lock • Server notifies clients of lock requests • Lease-based: must renew locks • Disconnect → release lock
Distributed File Systems Tran Giang Son, [email protected] 26 / 40 File Systems Distributed File Systems
NFS Summary • NFS • AFS • GlusterFS
• Very popular • Full POSIX interface • Single server • Scalability problem • Simplifies protocol, consistency • Problems • Fault tolerance • Scable performance • Consistency
Distributed File Systems Tran Giang Son, [email protected] 27 / 40 File Systems Distributed File Systems
AFS • NFS • AFS • GlusterFS
• The Andrew File System • Carnegie Mellon University, 1983 • OpenAFS • Kerberos authentication • Scalabe • Client side caching • Read only replication
Distributed File Systems Tran Giang Son, [email protected] 28 / 40 File Systems Distributed File Systems
AFS • NFS • AFS • GlusterFS
replica server
client client client
Distributed File Systems Tran Giang Son, [email protected] 29 / 40 File Systems Distributed File Systems
1 AFS vs NFS • NFS • AFS • GlusterFS
• Client/server ratio • NFS 25:1 • AFS 100s:1 • Reliability • NFS server dies → everyone dies • AFS server dies → everyone reads only from replica • Security • Keberos vs non-keberos
1Source: Morgan Stanley Distributed File Systems Tran Giang Son, [email protected] 30 / 40 File Systems Distributed File Systems
AFS Local Caching • NFS • AFS • GlusterFS
• File reads/writes on local cached copy • Modified copy is synchronized back on close • Open local copies are notified • Problem: consistency
Distributed File Systems Tran Giang Son, [email protected] 31 / 40 File Systems Distributed File Systems
AFS Replication • NFS • AFS • GlusterFS
• Read-only of file system volumes • Guaranteed to be atomic checkpoints • Changes do not propagate to existing read-only copies
Distributed File Systems Tran Giang Son, [email protected] 32 / 40 File Systems Distributed File Systems AFS Summary
• Not POSIX • Security/permissions • No write-through (no immediate sync) • High availability with replicas and local caching • Scalability for read loads
Distributed File Systems Tran Giang Son, [email protected] 33 / 40 File Systems Distributed File Systems
GlusterFS • NFS • AFS • GlusterFS
• Network-attached storage file system • Scalablity • High availibility • Ethernet / Infiniband • FUSE or NFS translator
Distributed File Systems Tran Giang Son, [email protected] 34 / 40 File Systems Distributed File Systems
GlusterFS • NFS • AFS • GlusterFS
server server
client client client
Distributed File Systems Tran Giang Son, [email protected] 35 / 40 File Systems Distributed File Systems
GlusterFS • NFS • AFS • GlusterFS
• Brick: basic unit of storage • V olume = bricks • Presents to user • Peer/node • Server hosting the bricks • Running Gluster daemons • Trusted storage pool • Group of peers • Client: the machine which mounts the volume
Distributed File Systems Tran Giang Son, [email protected] 36 / 40 File Systems Distributed File Systems
GlusterFS • NFS • AFS • GlusterFS
• Use common hardware • Use existing file system for bricks • A file system mount point • Scalability: similar to RAID0 • Striped among bricks • Performance • Capacity • Aggregated resources • No metadata server
Distributed File Systems Tran Giang Son, [email protected] 37 / 40 File Systems Distributed File Systems
GlusterFS • NFS • AFS • GlusterFS
• High availability: similar tor RAID1 • Mirrored among bricks • Can be both striped and mirrored
Distributed File Systems Tran Giang Son, [email protected] 38 / 40 File Systems Distributed File Systems
GlusterFS • NFS • AFS • GlusterFS
Distributed File Systems Tran Giang Son, [email protected] 39 / 40 File Systems Distributed File Systems Practical work 6: GlusterFS
• Install GlusterFS on your laptops, make a trusted pool • Create a distributed replicated volume • Write a short report in LATEX: • Name it « 06.glusterfs.tex » • Write the commmands for above steps • Perform benchmarks • Small files: number of accesses/s vs number of servers • Large files: read speed (MB/s) vs number of servers • Who does what. • Work in your group, in parallel • Push your report to corresponding forked Github repository
Distributed File Systems Tran Giang Son, [email protected] 40 / 40