USING SAS® 9 AND ’S GLOBAL FILE SYSTEM2 (GFS2) SAS Institute August 2013

With the growing number of SAS GRID usage, our Red Hat (RHEL) users are looking for a clustered for the RHEL grid nodes. This short paper will discuss what is available, starting with Red Hat’s cluster file system is called Global File System 2 (GFS2) that is part of the Red Hat Resilient Storage Add- On. Note that Red Hat requires an architecture review be conducted by Red Hat prior to the implementation of a RHEL Cluster. Please work with your Red Hat account team to make this happen.

This clustered file systems works very nicely with SAS, however to get the performance enhancements that Red Hat has made to GFS2, you need to run with RHEL 6.4 + errata patches through mid-May 2013. This errata provide fixes to the tuned service as well as address a concern with irqbalance. You should not use GFS2 with any versions of RHEL prior to RHEL 6.4 and the mid-May errata due to various performance issues. Using GFS2 with a version of RHEL5 will result in severe functional problems, and these problems may exist with early versions of RHEL 6.

In addition to using the above release of RHEL, there are some other tuning requirements that should be done when setting up the for your SAS GRID environment. The penalty for not applying these tuning guidelines is very costly down time due to unacceptable performance.

 It is strongly recommended that SAS WORK directories be place on a separate GFS2 file system from the permanent SAS data file space to avoid fragmentation in the permanent SAS data file space.  Use tuned tool and tune with profile 'enterprise-storage'. Transparent Huge Pages these days does not impact performance by more than a 1-2%. The changes for GFS2 grid are in addition to what is recommended for standalone systems so this should have already been done. To see the standalone RHEL system recommendations, please review this paper: http://support.sas.com/resources/papers/proceedings11/72480_RHEL6_Tuning_Tips.pdf

 GFS2 is limited to 16 systems in a cluster.

 Use the deadline I/O scheduler and sets the dirty page ratio to 40 by setting the option VM.DIRTY_RATIO= to 40. These settings greatly improved the performance of the workloads.

 Improve the behavior of the (DLM) by making the following changes:

echo 16384 > /sys/kernel/config/dlm/cluster/dirtbl_size echo 16384 > /sys/kernel/config/dlm/cluster/rsbtbl_size echo 16384 > /sys/kernel/config/dlm/cluster/lkbtbl_size

Instructions on how to apply these tuning parameters can be found in http://www.redhat.com/resourcelibrary/datasheets/rhel-sas-deployments

 Use LVCHANGE-R to set read-ahead for the file system. Hopefully this has already been done following the standard RHEL tuning guidelines.

With RHEL 6.4 and the mid-May 2013 errata applied, SAS found that the performance of several different workloads performed very well on GFS2. If you are interested in seeing how what other clustered file systems are available on RHEL systems, please review this paper, http://support.sas.com/resources/papers/proceedings13/484-2013.pdf. And additional tuning guidelines can be found on this SAS Support Note: http://support.sas.com/kb/42/197.html

Please note that Red Hat requires an architecture review be conducted by Red Hat prior to the implementation of a RHEL Cluster. Please work with your Red Hat account team to make this happen.

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. Copyright © 2013 SAS Institute Inc., Cary, NC, USA. All rights reserved.