ORNL/TM-2015/212 Secure Storage Architectures Approved for public release; distribution is unlimited. Ferrol Aderholdt Blake Caldwell Susan Hicks Scott Koch Thomas Naughton James Pogge Stephen L. Scott Galen Shipman Lawrence Sorrillo January 2015 DOCUMENT AVAILABILITY Reports produced after January 1, 1996, are generally available free via US Department of Energy (DOE) SciTech Connect. Website: http://www.osti.gov/scitech/ Reports produced before January 1, 1996, may be purchased by members of the public from the following source: National Technical Information Service 5285 Port Royal Road Springfield, VA 22161 Telephone: 703-605-6000 (1-800-553-6847) TDD: 703-487-4639 Fax: 703-605-6900 E-mail: [email protected] Website: http://www.ntis.gov/help/ordermethods.aspx Reports are available to DOE employees, DOE contractors, Energy Technology Data Exchange representatives, and International Nuclear Information System representatives from the following source: Office of Scientific and Technical Information PO Box 62 Oak Ridge, TN 37831 Telephone: 865-576-8401 Fax: 865-576-5728 E-mail: [email protected] Website: http://www.osti.gov/contact.html This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their em- ployees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial prod- uct, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorse- ment, recommendation, or favoring by the United States Govern- ment or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. ORNL/TM-2015/212 Computing & Computational Sciences Directorate DoD-HPC Program Secure Storage Architectures Ferrol Aderholdt2, Blake Caldwell1, Susan Hicks1, Scott Koch1, Thomas Naughton1, James Pogge2, Stephen L. Scott1;2, Galen Shipman2 and Lawrence Sorrillo1 1 Oak Ridge National Laboratory Oak Ridge, TN 37831 2 Tennessee Technological University Cookeville, TN, 38501 Date Published: January 2015 Prepared by OAK RIDGE NATIONAL LABORATORY P.O. Box 2008 Oak Ridge, Tennessee 37831-6285 managed by UT-Battelle, LLC for the US DEPARTMENT OF ENERGY under contract DE-AC05-00OR22725 CONTENTS List of Figures ............................................ v List of Tables ............................................. vii Acronyms ............................................... ix Executive Summary1 1 Introduction 3 1.1 Report Outline ......................................... 3 2 Background 4 2.1 The Lustre Storage Architecture ................................ 4 2.2 The GPFS Storage Architecture ................................ 7 2.3 Supporting Security Technologies ............................... 8 2.3.1 NIST .......................................... 8 2.3.2 FIPS........................................... 9 2.3.3 GSSAPI......................................... 9 2.3.4 Kerberos ........................................ 9 2.4 Storage Related Components in OpenStack .......................... 9 2.4.1 Swift – Object Storage................................. 9 2.4.2 Cinder – Block Storage................................. 12 2.4.3 Manila – File Share Service .............................. 13 3 Security in HPC Storage 15 3.1 Lustre.............................................. 15 3.1.1 Isolation......................................... 15 3.1.2 Authentication ..................................... 15 3.1.3 Authorization...................................... 16 3.1.4 Integrity......................................... 16 3.1.5 Features in Development................................ 16 3.1.6 Gaps .......................................... 17 3.2 GPFS .............................................. 17 3.2.1 Authentication ..................................... 17 3.2.2 Authorization...................................... 18 3.2.3 Encryption ....................................... 18 3.2.4 Features & Gaps .................................... 19 3.3 Discussion............................................ 19 3.3.1 Comparisons of Security with Lustre and GPFS.................... 19 iii 3.3.2 Performance in Lustre and GPFS............................ 19 4 Bridging Technologies for Secure Storage 21 4.1 Virtualization .......................................... 21 4.2 VLAN/Network Segmentation................................. 22 4.3 I/O Forwarding ......................................... 22 4.3.1 NFS........................................... 22 4.3.2 VirtFS.......................................... 22 4.3.3 DIOD.......................................... 23 5 Secure Enclave Storage Architecture 24 5.1 Isolation-Centric Storage Architecture............................. 24 5.2 Instances of the Isolation-Centric Storage Architecture .................... 26 5.2.1 Parallel filesystem with host-based subtree limitations for VM............ 26 5.2.2 Parallel filesystem with host-based subtree limitations for VE ............ 27 6 Storage Vendor Analysis 29 6.1 Overview ............................................ 29 6.2 Seagate/Xyratex......................................... 29 6.2.1 Security......................................... 29 6.2.2 ClusterStor Platform .................................. 30 6.2.3 Physical Drive Capability................................ 30 6.2.4 Data Throughput.................................... 30 6.2.5 Specification Table ClusterStorTM 1500 ........................ 30 6.3 Oracle ZFS Storage Appliance................................. 30 6.3.1 Resiliency........................................ 31 6.3.2 Vitualization Support.................................. 31 6.3.3 Filesystem Support................................... 31 6.4 Additional Systems....................................... 32 7 Conclusion 33 7.1 Synopsis............................................. 33 7.2 Recommendations & Observations............................... 33 7.3 Future Plans........................................... 34 7.4 Acknowledgments ....................................... 34 Bibliography 35 iv LIST OF FIGURES 2.1 Illustration of Lustre configuration for basic cluster...................... 5 2.2 Diagram showing steps for Lustre client writing data ..................... 6 2.3 System structure for GPFS................................... 7 2.4 Diagram of Cinder component interactions........................... 12 2.5 Diagram of Cinder Storage Node................................ 13 2.6 Example of ‘generic’ Manila file share component....................... 14 5.1 Example illustrating layers of isolation-based storage architecture .............. 25 5.2 Diagram showing single network for storage network..................... 26 5.3 Diagram showing separate VLANs for storage network.................... 26 5.4 Example instance of Lustre, IO re-exporter with VM ..................... 27 5.5 Example instance of Lustre, bind-mount with VE....................... 27 5.6 Example instance of GPFS, bind-mount with VE ....................... 28 v LIST OF TABLES 3.1 Lustre vs. GPFS......................................... 20 6.1 Seagate ClusterStor product specifications........................... 31 6.2 Oracle ZFS Storage appliance specifications.......................... 31 vii ACRONYMS ACL Access Control List API Application Program Interface ASIC Application Specific Integrated Circuit BGP Border Gateway Protocol CLI Command Line Interface CPU Central Processing Unit C/R Checkpoint/Restart DNAT Dynamic Network Address Translation DOE Department of Energy GRE Generic Routing Encapsulation HPC High-Performance Computing HPCCG High-Performance Computing Conjugate Gradient HPL High Performance Linpack I/O Input/Output IPC Inter-Process Communication LACP Link Aggregation Control Protocol LUN Logical Unit LXC Linux Containers MD5 Message Digest Algorithm V5 MLAG Multichassis Link Aggregation MPI Message Passing Interface NAT Network Address Translation NFV Network Function Virtualization NIDS Network Intrusion Detection System ORNL Oak Ridge National Laboratory OS Operating System OSPF Open Shortest Path First OVS Open Virtual Switch PC Personal Computer POSIX Portable Operating System Interface for Unix PVM Parallel Virtual Machine QOS Quality of Service RAS Reliability, Availability, and Serviceability SDM Security Device Manager SDN Software Defined Networking SNAT Source Network Address Translation SNMP Simple Network Management Protocol SSH Secure Shell VLAN Virtual Local Area Network VMS Virtual Modular Switch VNIC Virtualized Network Interface Control VPC Virtual Port Channel ix VRF Virtual Routing and Forwarding VXLAN Virtual eXtensible Local Area Network XML Extensible Markup Language XMPP Extensible Messaging and Presence Protocol x Executive Summary Secure Storage Architectures The purpose of this report is to clarify the challenges associated with storage for secure enclaves. The major focus areas for the report are: • review of relevant parallel filesystem technologies to identify assets and gaps; • review of filesystem isolation/protection mechanisms, to include native filesystem capabilities and auxiliary/layered techniques; • definition of storage architectures that can be used