IP SAN Fundamentals: An Introduction to IP SANs and iSCSI Updated April 2007 Sun Microsystems, Inc. © 2007 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, CA 95054 USA All rights reserved. This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Parts of the product may be derived from Berkeley BSD appliances, licensed from the University of California. Sun, Sun Microsystems, Sun StorEdge, the Sun logo, and Solaris are trademarks, registered trademarks, or service marks of Sun Microsystems, Inc. in the U.S. and other countries. UNIX is a registered trademark in the United States and other countries, exclusively licensed through X/Open Company, Ltd. Windows is a registered trademark of Microsoft Corporation in the United States and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. The OPEN LOOK and Sun's Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun's licensees who implement OPEN LOOK Guise and otherwise comply with Sun's written license agreements. RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and FAR 52.227- 1987), or DFAR 252.227-7015(b)(6/95) and DFAR 227.7202-3(a). DOCUMENTATION IS PROVIDED ÒAS ISÓ AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON- INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. Sun Microsystems, Inc. Table of Contents Introduction: Why Build Storage Area Networks?................................................................................................1 Terminology.....................................................................................................................................................2 iSCSI Fundamentals..........................................................................................................................................3 Internet Storage Name Service..........................................................................................................................5 LUN Masking....................................................................................................................................................6 IP SAN Security.................................................................................................................................................7 iSCSI Host Bus Adapters....................................................................................................................................8 Booting Over iSCSI............................................................................................................................................9 iSCSI Targets and iSCSI Routers........................................................................................................................10 Extending IP SANs Over Wide Area Networks.....................................................................................................11 Positioning IP SANs with NAS and Fibre Channel SANs........................................................................................12 Making a Choice between an IP SAN or a Fibre Channel SAN................................................................................13 References.....................................................................................................................................................15 1 Introduction: Why Build Storage Area Networks? Sun Microsystems, Inc. Introduction: Why Build Storage Area Networks? In a Direct Attached Storage (DAS) environment, more storage capacity than applications require is commonly purchased as insurance against running out of space: this is called over-provisioning. Storage Area Networks (SANs) allow multiple heterogeneous hosts to share storage sub-systems. Disks are pooled behind an array controller; RAID volumes are created from the disks; the RAID volumes are carved up into volumes; and the volumes are presented to hosts. This reduces the need to over-provision resulting in: fewer sub-systems in the Data Center; higher utilization of the available storage; and savings on space, heat and power. Fibre Channel SANs became popular in the late 1990s and are widely used today but, despite prices falling in recent years, the costs of a Fibre Channel Switch infrastructure, Fibre Channel Host Bus Adapters and training staff in the new skills required to manage a SAN remain a barrier to entry for many organizations and IT groups: IP based SANs offer many of the benefits of Fibre Channel SANs, but at a lower cost. This document provides a high level technical overview of IP SANs and iSCSI, and positions IP SANs relative to Fibre Channel SANs and NAS. 2 Terminology Sun Microsystems, Inc. Terminology iSCSI iSCSI (Internet Small Computer System Interface) is a data transport protocol used to carry block-level data over IP networks. IP SAN An IP SAN is a Storage Area Network that uses the iSCSI protocol to transfer block-level data over a network, generally Ethernet. Initiator In this document the term "initiator" is used interchangeably to refer to a server, host or device driver that initiates (i.e. begins) iSCSI command sequences. Target iSCSI targets break down iSCSI command sequences from initiators and process the SCSI commands. Examples of iSCSI targets are a disk or tape device with an iSCSI port and a NAS appliance with iSCSI target support. Fibre Channel SAN A Fibre Channel SAN is a Fibre Channel network over which the SCSI-FCP (Fibre Channel SCSI) protocol runs. See References for a resource that discusses Fibre Channel SANs in detail. 3 iSCSI Fundamentals Sun Microsystems, Inc. iSCSI Fundamentals iSCSI Enabling Hosts Software based iSCSI initiators are available for most operating systems including: the SolarisTM OS; Microsoft Windows; AIX, HP-UX; and Linux. These work with standard NICs. iSCSI Host Bus Adapters (HBAs) are available to provide iSCSI support for some operating systems (or versions of operating systems) that do not have software initiators available. See reference (v) for a list of Solaris Ready iSCSI HBAs. iSCSI Node Names Every iSCSI initiator and target has a worldwide unique identifier called a node name. A target's node name is assigned by the manufacturer. On a host, a worldwide unique node name is generated by the iSCSI initiator software when it is first enabled; the node name can also be set manually. A host's node name is used to represent all of the network ports for that host, i.e. it is unique to the host, not to an individual network port on the host. There are two formats for iSCSI node names: Extended Unique Identifier (EUI) and iSCSI Qualified Name (IQN). EUI's look very like Fibre Channel Worldwide Numbers. e.g. eui.02004567A425678A. IQN's look like an unusual DNS name. An example of an IQN is iqn.1986-03.com.sun:01:e00000000000.44180a08. The IQN format is proving to be more popular. Note that the domain name style information in an IQN node name is not derived from the host's actual domain but is defined by the company which wrote the initiator stack; the above example is from the iSCSI software initiator in the Solaris 10 OS. iSCSI Log In An iSCSI log in is the process of establishing an iSCSI session. iSCSI Sessions and Portals The association between an iSCSI initiator and an iSCSI target is known as an iSCSI session. To establish an iSCSI session the initiator logs into the target using the target's IP address and a TCP port number. This IP address and TCP port number pair is known as an iSCSI Portal. 4 iSCSI Fundamentals Sun Microsystems, Inc. iSCSI Discovery iSCSI Discovery is the process by which an iSCSI initiator can learn which target iSCSI node names are available to it. There are a number of different methods of discovery: Static Configuration The initiator is told the complete target name including portal addresses, etc. This information is configured manually. Send-Targets The initiator is told to query a discovery IP address. The initiator communicates with the discovery address to receive all the configuration data available to this initiator for that target, e.g. all of the volumes it has access to. This needs to be repeated for each target. Internet Storage Name Service For small IP SANs the methods described above will suffice. For larger IP SANs the Internet Storage naming Service (iSNS) removes the need to manually enter discovery information on each initiator by providing centralized naming services, iSNS is a large topic and is covered in a later section. iSCSI Multipathing Where iSCSI multipathing is required,