559 Appendix A Xsan Security Xsan is Apple’s implementation of a Clustered File System. This means that the file system can be accessed by multiple machines concurrently. Fibre Channel is an extension of the SCSI protocol and can be accessed through fiber optic cabling. Using Xsan, multiple Macs can simultaneously access shared storage provided over a Fibre Channel network. This allows multiple users to access data striped across a large number of physical drives (let’s just say six Promise VTrak RAIDs’ worth) as though they are one volume. Each client can access data at speeds of up to 4Gbps. Xsan is commonly used high-definition video and multiuser high-bandwidth creative environments, and it is also found in web and file-sharing farms. Xsan is based on the StorNext File System, made by Quantum. Apple and Quantum both claim interoperability between StorNext and Xsan, which means that by combining the two, it’s possibly for Macs, Windows, and many Unix variants (including Solaris, AIX, Linux and IRIX) can share communicate and share storage in a fast Fibre Channel environment. When looking to secure Xsan, you may find that there are specific features of StorNext that you want to use. Because Xsan is based on StorNext, many of the features of StorNext are built into the Xsan but might not be available through Xsan Admin. To begin unlocking some of the hidden features of Xsan, you will need to dip into the command line. Xsan stores most of its data in /Library/Filesystems/Xsan. Configuration files are stored in the Config directory, and binaries are in the bin directory. The files in Config contain information about the structure of the SAN and any volumes on the SAN. The Config directory should not be world-writable, because if it is altered or deleted, the entire SAN volume could be deleted with it. The commands to control and configure Xsan are cvlabel, cvadmin, cvfsck, and cvcp. These commands are analogous to some Unix commands (and are similarly named), but they are quite specific to Xsan. Be wary of using a Unix commands in place of the Xsan- specific command. As an example, you should never, ever run fsck on an Xsan volume. 559 560 APPENDIX A: Xsan Security Metadata In an Xsan, your Metadata is the information about where on the physical disks your files are found, how many pieces they’re in, how to assemble those pieces when a client requests a file, and which clients have those files locked for writing. Metadata should be stored on a dedicated Fibre Channel LUN (logical unit number, which each logical segmentation of a target storage device has to uniquely identify it within the target storage). It is very important that it live on a mirrored RAID set, to lessen the risk of it being lost. Xsan storage is broken down into three parts: LUN: “Logical Unit Number”. A logical grouping of drives into a single entity. Usually a RAID set, but it could be a single drive. Storage pool: A group of LUNs. Each client can write to multiple LUNs through a single storage pool. Volumes: The logical volume presented to the client. It appears as a single local drive. Metadata is then managed by a metadata controller, which determines where data will go on the SAN and keeps track of where each slice of the data resides (similar to a file allocation table). Each volume should have two metadata controllers, a primary and a secondary. While you can have multiple metadata controllers, only one is ever active (per volume) at a given time. Each SAN client requires full access to all LUNs in order to write data. The metadata is simply a pointer that tells the SAN where to write data. If any of the LUNs that make up a storage pool cannot be seen over the Fibre Channel network by a metadata controller or a client, the storage pool reports “STRIPE GROUP DOWN,” and your Xsan volume will go down. For example, if you were to unplug the cable for one target, you would likely bring an entire Xsan environment down. Therefore, physical security becomes very important with Xsan environments. NOTE: If you are just using Xsan as a back-end file system for a number of bridgehead file servers then you can restrict access to the LUNs to only the WWNNs of the metadata controllers and file servers. The root user of any workstation that is connected through Fibre Channel to an Xsan environment can write directly to any accessible LUNs by writing data into /dev/<LUN>. A common path might be /dev/rdisk4. The association between and device ID and a LUN can be found in the Xsan labels for each LUN. If you write enough arbitrary data into the metadata LUN, then you will cause a volume to become inaccessible. This can be dangerous, because once the Xsan environment is restarted, it will read an invalid amount of data about itself, and therefore be unable to mount the volume. This is a denial of service attack and a tampering attack on the SAN that can be initiated by any client system that has a valid admin/root account. This is very dangerous, and there is APPENDIX A: Xsan Security 561 no workaround for it, other than restricting access to administrative or root accounts on SAN clients. Fibre Channel With Fibre Channel, you can cascade two switches by bonding (sometimes referred to as stacking) their backplanes with a 10GB to 20GB connection. Most environments use bonding in order to achieve very fast speeds between switches. From a security perspective, one nice feature of a stacked environment is that you can apply access controls across multiple switches. Affinities Storage pools can be assigned an affinity. When data is moved to the affinity, space is allocated from a specific storage pool rather than in a round-robin fashion across all the storage pools. Affinity data can be restricted to specific LUNs. Access to affinities can be limited to certain groups. Data that requires a higher level of integrity can then be given an affinity or volatile data that requires speed (such as a capture scratch) can be given the fastest possible storage (ie – SAS drives running on a RAID 0-based LUN). You can use Xsan Admin to assign an affinity to a folder at the top level of a volume. However, to assign an affinity to a nested folder, use the cvmkdir command. For example: sudo /Library/Filesystems/Xsan/bin/cvmkdir -k <affinity-name> <path-to-folder> Permissions Xsan volumes appear, to the client, as local storage. Therefore, when a file or folder is created on a volume, POSIX permissions will be assigned it based on the umask of the client user. Ownership of the files and folders will be tracked by the UID on the local system of the user who created them. A common practice in Xsan environments is to change the default umask of the client systems to 002, which will grant read and write access to owner and group, and read-only for all others. Managing permissions using only local user accounts and POSIX permissions can be very difficult. Many administrators will use Open Directory to manage users and groups, and Access Control Lists (ACLs) to manage permissions on files. ACLs greatly simplify permissions management, and they are even more valuable when other operating systems are accessing your volume. Windows and Linux clients treat ownership and permissions differently than Macs if they are tapping in through NFS, AFP and /or SMB. When ACLs are in use, new files added to the SAN volume will have the same permissions, no matter how the specific services are configured. To enable an ACL, open Server Admin, click on Sharing and browse to the root of a volume. Click Sharing, and then Volumes, and then List. Next, click the root of an Xsan volume, and then click on the Permissions tab. Check the box for Enable ACLs on 562 APPENDIX A: Xsan Security selected volume. Next, configure the permissions that should be in the ACL box, and then click the Save button. Data that is written to the Xsan volume will now have the permissions of the folder above it, provided the ACEs are set to inherit permissions (see Chapter 4). ACL entries can easily be added using Server Admin. This can be much easier than using the default umask. Quotas Quotas are also an important part of Xsan. A quota is the amount of space a user can utilize on the SAN. As an administrator, you can set hard and soft quotas for every user on the Xsan. A soft quota lets the user continue to save files when they pass their quota, but they will be warned that they are over their limit. When they reach the hard quota, they will not be able to save any more data until the administrator of the SAN gives them more space, or they delete some files. When users near their quotas, they will be alerted. Other SAN Solutions Xsan is a fast and fairly straightforward SAN product. It does not provide any capability for backup or snapshots, it is not as fault tolerant as it could be, and it is a little too latent for certain applications (such as Pro Tools by Digidesign). Other providers of SAN solutions include EMC, NetApp, and LeftHand Networks (a division of HP). These products can be made to work with the Mac OS X platform, although many do not work without using third-party software.