CXFS Whitepaper
Total Page:16
File Type:pdf, Size:1020Kb
White Paper SGI® InfiniteStorage Shared Filesystem CXFS™: A High-Performance, Multi-OS Filesystem from SGI Laura Shepard and Eric Eppe 1.0 CXFS: Fulfilling the Promise of SAN Technology . 2 2.0 Challenges of a Shared Filesystem . 4 3.0 Building on Proven Technology . 5 4.0 Metadata Management and File Locking . 6 5.0 Fault Isolation and Recovery . 9 6.0 User Interface . 11 7.0 LAN-Free Backup with CXFS . .12 8.0 High Availability and Unlimited Storage: Using CXFS with FailSafe® and DMF . 13 9.0 Multi-OS Platform Support . 14 10.0 CXFS Scalability and Performance . 14 11.0 CXFS in Production . 16 11.1 CXFS in Oil and Gas Exploration . 16 11.2 CXFS in Video Post-Production (Media) . 16 12.0 Summary . 17 1 Abstract For a detailed discussion of XFS, see Scalabil- While storage area networks (SANs) give ity and Performance in Modern Filesystems." multiple computer systems high-speed con- (SGI, 2000). nections to shared storage, they do not remove the data-sharing bottlenecks associated with 1.0 CXFS: Fulfilling the Promise of direct-attached and network-attached storage. SAN Technology Even in a SAN, storage is still dedicated to a While SANs offer the benefits of consolidated specific system. This being the case, when storage, centralized management, and a high- there is a need to share data among systems, speed data network, CXFS enables true data traditional file-serving and file-sharing meth- sharing by allowing SAN-attached systems ods such as NFS, FTP, CIFS Samba®, and direct access to shared filesystems. A shared RAID controller-based copies are still being filesystem provides data access speeds well used. Each of these data-sharing mechanisms beyond what is achievable via traditional has performance, availability, and complexity methods such as NFS and FTP, solving data- penalties. sharing bottlenecks for a broad range of environments. This significantly boosts pro- To completely remove these penalties and real- ductivity when large files are shared by ize the promise of SAN technology, SGI has multiple processes in a workflow. developed InfiniteStorage Shared Filesystem CXFS, a revolutionary shared filesystem for The explosive growth in storage deployment SANs that allows multiple heterogeneous sys- has created significant challenges for tradi- tems to simultaneously access all data on a tional storage architectures. Plummeting SAN. In combination with other storage prices of storage media, increasingly data- virtualization technologies, CXFS offers het- intensive applications on ever more powerful erogeneous systems unprecedented platforms, and expanding data availability high-speed shared access to data on high- requirements are shifting the traditional para- speed storage. digms of server-attached or network-attached storage. SANs were developed in response to CXFS enables 64-bit IRIX® OS-based systems, these changing requirements. SAN technology Solaris™ 8 and Solaris™ 9, Windows NT® 4.0, allows multiple computer systems to share Windows® 2000, 64-bit Linux® for SGI® Altix®, access to storage resources using high-speed Red Hat® 7.3 for 32-bit Linux, and IBM® AIX® Fibre Channel connections. 5L systems to seamlessly and instantly share data with no copies to wait for, store, and SGI uses a three-layer model to describe SAN administer. Windows® XP and Mac OS® X will technology. Layer 1, the SAN infrastructure, be available in the second half of calendar consists of the basic hardware and software year 2003. Additional UNIX® support is in components necessary to construct a working development. SAN. CXFS leverages the proven capabilities of SGI® Layer 1 is the basic hardware infrastructure of InfiniteStorage Filesystem XFS®, the most the SAN. In a SAN, connections between scalable, high-performance, journaled 64-bit systems and storage are made through one or filesystem available. CXFS brings the full more Fibre Channel switches. The systems, feature set and performance of XFS to hetero- storage, adapters, cables, and switches make geneous SANs. This paper examines the up a SAN; the switches and cables alone are issues surrounding high-speed shared data referred to as a Fibre Channel “fabric.” access and describes how CXFS overcomes Today’s SAN technology allows up to 400MB these challenges while providing key advan- per second (full duplex) per single Fibre Chan- tages to customer applications. nel connection. 2 Value Added Layer 3 SAN Services Shared Filesystem Interoperability Testing Layer 2 Integration SAN Management Support Professional Layer 1 Services Hardware and Software Infrastructure Fig. 1. Storage area network layer model Layer 2, SAN management, includes tools to corrupted. A shared filesystem is designed to facilitate monitoring and management of the meet this need, providing direct high-speed various components of the SAN and to protect access to data at SAN speeds. SAN data. Most of the tools used in direct- attach storage environments are available for Even if consolidating storage is one of the pri- SANs. More comprehensive LAN-management- mary drivers for building a SAN, because it style tools that tie common management greatly simplifies the administration of RAID functions together are also becoming available. arrays, a high-speed shared filesystem can reduce or eliminate the need for replication, SANs provide many of the same configuration freeing disk space for application use and dra- options as traditional networks, but avoid the matically reducing management associated high overhead of traditional network protocols. with replicated data. SANs provide the ability to deploy flexible, fault-tolerant storage systems with exceptional Thus, a shared filesystem can change the way performance. Fibre Channel SANs combine that many customers work by changing and the time-tested SCSI protocol with high-speed improving the efficiency of workflow between serial data transmission for outstanding relia- systems needing to share data. In applications bility, high performance, and increased such as digital media or oil and gas exploration, flexibility. the output of one processing step becomes the input to the next. A shared filesystem elimi- The greatest potential of SAN technology lies nates the need to copy data and overcomes in Layer 3, the shared filesystem. Layer 1 and the bandwidth limitations of LANs. Layer 2 components allow a storage infrastruc- ture to be built in which all SAN-connected To satisfy the workflow requirements of cut- computer systems potentially have access to ting-edge applications such as these, SGI all storage but don t provide the ability to undertook the development of SGI® share data. Separate computer systems cannot InfiniteStorage Shared Filesystem CXFS. CXFS access data in the same filesystem, let alone is a first-of-its-kind shared filesystem for SANs share the same files. Additional software is that meets the exacting requirements of high- required to mediate and manage shared performance data sharing. CXFS fulfills the access; otherwise, data would quickly become promise of all-SAN systems directly accessing 3 LAN Application Servers SAN FC Interfaces Interconnect Switch Fabric Switches, Hubs, etc. LAN-free Backup Storage Fig. 2. Storage area network layer model all files stored on SAN-attached devices. In combined with those of high-availability solu- combination with storage virtualization tech- tions. Traditional filesystems have evolved over nologies such as hierarchical storage many years to optimize the performance of management (HSM), CXFS can provide hetero- the underlying disk pool. Application data is geneous system platforms with unprecedented typically cached in the host system’s memory high-speed access to SAN storage capacity. to speed access to the filesystem. This caching—essential to filesystem perfor- 2.0 Challenges of a Shared Filesystem mance—is the reason systems cannot simply CXFS offers the performance and features of a share data stored in traditional filesystems. traditional filesystem while providing the coor- dination needed for file sharing, distributed If multiple systems assume they have control file locking, and fault isolation and recovery. of the filesystem and write filesystem data they have in their cache, they will quickly The challenges faced by a SAN shared file- corrupt the filesystem. On the other hand, system are different from those faced by the implementing a filesystem that does not allow traditional network filesystem. For a network data caching would provide unacceptably slow filesystem, all transactions are mediated and access. Distributed data-cache management controlled by a network file server. While the for shared access is one of the key challenges same approach could be transferred to a SAN faced by the shared filesystem. using much the same protocols, this would not eliminate the fundamental limitations of the Like high-availability solutions, a shared file server or take advantage of the benefits of filesystem should preserve data access and a SAN. continuity in the event of failure of some of the systems or other components that make up the The file server is the main impediment to per- storage area network. Metadata management formance and represents a single point of among multiple systems requires a level of failure. The design challenges faced by the control that must be protected and preserved shared filesystem are more akin to the chal- in case of a failure to ensure continuing data lenges of a traditional filesystem design integrity. 4 LAN SAN File Access through File Server Direct Access to Data Local Area