Target_Core_ModTarget_Core_Mod v3.0,v3.0, aa ConfigFSConfigFS enabledenabled SCSISCSI targettarget infrastructureinfrastructure LinuxLinux StorageStorage andand FilesystemFilesystem Workshop,Workshop, ©09©09 NicholasNicholas A.A. Bellinger,Bellinger, Linux-iSCSI.orgLinux-iSCSI.org ChangesChanges fromfrom LIOLIO v2.9v2.9 toto v3.0v3.0 sincesince LSFLSF ©08©08 ● CodeCode hashas beenbeen importedimported intointo kernel.org/lio-core-2.6.gitkernel.org/lio-core-2.6.git whichwhich trackstracks linux-2.6.gitlinux-2.6.git (currently(currently atat v2.6.29)v2.6.29) ● GenericGeneric TargetTarget EngineEngine codecode andand subsystemsubsystem pluginsplugins (interface(interface toto Linux/SCSI,Linux/SCSI, Linux/BLOCKLinux/BLOCK andand Linux/VFS)Linux/VFS) hashas beenbeen seperatedseperated intointo target_core_modtarget_core_mod thatthat liveslives inin drivers/targetdrivers/target andand include/targetinclude/target ● AllAll IOCTLIOCTL codecode forfor thethe GenericGeneric TargetTarget EngineEngine andand LIO-LIO- TargetTarget (iSCSI(iSCSI TargetTarget fabricfabric module)module) hashas beenbeen convertedconverted toto 100%100% upstreamupstream ConfigFSConfigFS infrastructureinfrastructure ● Target_Core_Mod/ConfigFSTarget_Core_Mod/ConfigFS hashas beenbeen submittedsubmitted forfor reviewreview andand inclusioninclusion inin v2.6.30,v2.6.30, LIO-Target/ConfigFSLIO-Target/ConfigFS willwill bebe submittedsubmitted seperately.seperately. ChangesChanges fromfrom v2.9v2.9 toto v3.0,v3.0, ContinuedContinued ● ExhaustiveExhaustive supportsupport forfor SPC-3SPC-3 compliantcompliant PersistentPersistent ReservationsReservations ● InitialInitial supportsupport forfor implictimplict (out(out ofof band)band) AsymmetricAsymmetric LogicalLogical UnitUnit AssignmentAssignment LogicalLogical UnitUnit andand TargetTarget PortPort GroupsGroups throughthrough ConfigFSConfigFS ● 4k4k sectorsector supportsupport (physical(physical andand emulatedemulated support)support) ● AdditionalAdditional EVPDEVPD 0x830x83 DeviceDevice IdentifiersIdentifiers includingincluding NAANAA IEEEIEEE RegisteredRegistered ExtendedExtended AssignedAssigned designatordesignator formatformat fromfrom ConfigFSConfigFS providedprovided WWNWWN informationinformation ● UnitUnit AttentionAttention supportsupport ConfigFSConfigFS ● target_core_modtarget_core_mod createscreates thethe ConfigFSConfigFS groupgroup /sys/kernel/config/target/core/sys/kernel/config/target/core ● Linux/SCSI,Linux/SCSI, Linux/BLOCKLinux/BLOCK andand Linux/VFSLinux/VFS storagestorage HBAHBA andand objects/devicesobjects/devices areare registered/unregisteredregistered/unregistered mkdir(2),mkdir(2), rmdir(2)rmdir(2) andand echoecho throughthrough /sys/kernel/config//sys/kernel/config/ target/core/$HBA/$DEVtarget/core/$HBA/$DEV ● ConfigFSConfigFS symlinkssymlinks areare usedused toto createcreate SCSISCSI TargetTarget PortsPorts fromfrom storagestorage objectsobjects inin /sys/kernel/config/target//sys/kernel/config/target/ core/core/ toto SCSISCSI fabricfabric modulesmodules inin /sys/kernel/config/target/$FABRIC/sys/kernel/config/target/$FABRIC ConfigFSConfigFS layoutlayout /sys/kernel/config/target/core/$HBA/$DEV/sys/kernel/config/target/core/$HBA/$DEV groupsgroups andand attributes:attributes: alua_lu_gp/alua_lu_gp/ :: UsedUsed forfor ALUAALUA logicallogical unitunit groupsgroups attrib/:attrib/: AttributesAttributes likelike block_size,block_size, emulate_tas,emulate_tas, emulate_ua_intrlck_ctrl,emulate_ua_intrlck_ctrl, queue_depth,queue_depth, etc.etc. controlcontrol :: UsedUsed toto passpass parametersparameters toto subsystemsubsystem pluginsplugins enableenable :: UsedUsed toto enableenable storagestorage objectobject fd/fd/ :: UsedUsed toto passpass filefile descriptordescriptor toto subsystemsubsystem pluginsplugins pr/pr/ :: UsedUsed forfor SPC-3SPC-3 persistentpersistent resevationsresevations informationinformation wwn/wwn/ :: UsedUsed forfor T10T10 worldworld widewide uniqueunique namingnaming informationinformation ConfigFSConfigFS andand Linux/SCSILinux/SCSI RegisteringRegistering aa Linux/SCSILinux/SCSI StorageStorage Object:Object: mkdirmkdir -p-p /sys/kernel/config/target/core/pscsi_0/sdd/sys/kernel/config/target/core/pscsi_0/sdd /*/* UsingUsing FileFile DescriptorDescriptor method,method, alsoalso cancan useuse UDEVUDEV pathpath */*/ execexec 3<>/dev/sdd3<>/dev/sdd echoecho 33 >> /sys/kernel/config/target/core/pscsi_0/sdd/fd/sys/kernel/config/target/core/pscsi_0/sdd/fd execexec 3>&-3>&- /*/* Or,Or, usingusing parameterparameter methodmethod */*/ echoecho ©scsi_target_id=0,scsi_channel_id=0,scsi_lun_id=0©©scsi_target_id=0,scsi_channel_id=0,scsi_lun_id=0© // >> /sys/kernel/config/target/core/pscsi_0/sdd/control/sys/kernel/config/target/core/pscsi_0/sdd/control echoecho 11 >> /sys/kernel/config/target/core/pscsi_0/sdd/enable/sys/kernel/config/target/core/pscsi_0/sdd/enable ConfigFSConfigFS andand Linux/BLOCKLinux/BLOCK RegisteringRegistering aa Linux/BlockLinux/Block LVMLVM StorageStorage Object:Object: mkdirmkdir -p-p /sys/kernel/config/target/core/iblock_0/lvm_test0/sys/kernel/config/target/core/iblock_0/lvm_test0 /*/* UsingUsing FileFile DescriptorDescriptor method,method, alsoalso cancan useuse UDEVUDEV pathpath */*/ execexec 3<>/dev/lio-test/test03<>/dev/lio-test/test0 echoecho 33 >> /sys/kernel/config/target/core/iblock_0/lvm_test0/fd/sys/kernel/config/target/core/iblock_0/lvm_test0/fd execexec 3>&-3>&- /*/* Or,Or, usingusing parameterparameter methodmethod */*/ echoecho ©iblock_major=252,iblock_minor=2©©iblock_major=252,iblock_minor=2© // >> /sys/kernel/config/target/core/iblock_0/lvm_test0/control/sys/kernel/config/target/core/iblock_0/lvm_test0/control echoecho 11 >> /sys/kernel/config/target/core/iblock_0/lvm_test0/enable/sys/kernel/config/target/core/iblock_0/lvm_test0/enable ConfigFSConfigFS andand Linux/VFSLinux/VFS RegisteringRegistering aa Linux/VFSLinux/VFS FILEIOFILEIO storagestorage objectobject mkdirmkdir -p-p /sys/kernel/config/target/core/fileio_0/my_file0/sys/kernel/config/target/core/fileio_0/my_file0 /*/* UsingUsing parameterparameter method,method, alsoalso willwill automaticallyautomatically detectdetect sizesize fromfrom fd_dev_name=fd_dev_name= thatthat referencesreferences underlyingunderlying structstruct block_deviceblock_device */*/ echoecho ©fd_dev_name=/tmp/my_file,fd_dev_size=10000000©©fd_dev_name=/tmp/my_file,fd_dev_size=10000000© // >> /sys/kernel/config/target/core/fileio_0/my_file0/control/sys/kernel/config/target/core/fileio_0/my_file0/control echoecho 11 >> /sys/kernel/config/target/core/fileio_0/my_file0/enable/sys/kernel/config/target/core/fileio_0/my_file0/enable SPC-3SPC-3 PersistentPersistent Reservations:Reservations: WhatsWhats implemented..?implemented..? ● PROUTPROUT ServiceService Actions:Actions: REGISTER,REGISTER, RESERVE,RESERVE, RELEASE,RELEASE, CLEAR,CLEAR, REGISTER_AND_IGNORE,REGISTER_AND_IGNORE, PREEMPT,PREEMPT, andand PREEMPT_AND_ABORTPREEMPT_AND_ABORT ● AllAll PROUTPROUT ReservationReservation TypesTypes areare supported:supported: WriteWrite Exclusive,Exclusive, ExclusiveExclusive Access,Access, WriteWrite ExclusiveExclusive RegistrantsRegistrants Only,Only, ExclusiveExclusive AccessAccess RegistrantsRegistrants Only,Only, WriteWrite ExclusiveExclusive AllAll Registrants,Registrants, ExclusiveExclusive AccessAccess AllAll RegistrantsRegistrants ● AllAll PRINPRIN ServiceService Actions:Actions: READ_KEYS,READ_KEYS, READ_RESERVATION,READ_RESERVATION, REPORT_CAPABILITIES,REPORT_CAPABILITIES, READ_FULL_STATUSREAD_FULL_STATUS SPC-3SPC-3 PersistentPersistent Reservations:Reservations: WhatWhat clientsclients havehave beenbeen tested?tested? ● RHELRHEL v5u3v5u3 usingusing SCSISCSI FencingFencing (uses(uses WriteWrite Exclusive,Exclusive, RegistrantsRegistrants OnlyOnly andand PREEMPT_AND_ABORT)PREEMPT_AND_ABORT) usingusing ext3ext3 mounts.mounts. TestingTesting withwith GFSGFS (with(with multiplemultiple writers)writers) isis underwayunderway ● MSFTMSFT ClusterCluster 20082008 (uses(uses WriteWrite Exclusive,Exclusive, RegistrantsRegistrants OnlyOnly andand PREEMPT)PREEMPT) usingusing thethe MSFTMSFT domaindomain validationvalidation suitesuite SPC-3SPC-3 PersistentPersistent Reservations:Reservations: WhatsWhats leftleft toto implement..?implement..? ● ActivateActivate PersistPersist ThroughThrough PowerPower LossLoss (APTPL)(APTPL) usingusing /var/target/$HBA/$DEV/persist/var/target/$HBA/$DEV/persist fromfrom ConfigFSConfigFS storagestorage objectobject layoutlayout forfor registration/reservationregistration/reservation metadatametadata ● PROUTPROUT REGISTER_AND_MOVEREGISTER_AND_MOVE ServiceService ActionAction (Register(Register andand movemove reservation)reservation) ● SPEC_I_PTSPEC_I_PT (Allows(Allows multiplemultiple InitiatorsInitiators toto bebe registeredregistered withwith aa singlesingle PROUTPROUT ServiceService Action)Action) AsymmetricAsymmetric LogicalLogical UnitUnit Assignment:Assignment: WhatsWhats implemented?implemented? ● LogicalLogical UnitUnit GroupsGroups (per(per storagestorage object)object) ● TargetTarget PortPort GroupsGroups (per(per SCSISCSI targettarget port)port) ● ImplictImplict ALUAALUA (through(through ConfigFS)ConfigFS) ● OptmizedOptmized andand NonNon OptimizedOptimized ALUAALUA accessaccess statesstates ● REPORT_TARGET_PORT_GROUPSREPORT_TARGET_PORT_GROUPS AsymmetricAsymmetric LogicalLogical UnitUnit Assignment:Assignment: WhatWhat clients..?clients..? ● LinuxLinux usingusing thethe Open/iSCSIOpen/iSCSI InitiatorInitiator withwith thethe genericgeneric ALUAALUA handlerhandler (scsi_dh_alua)(scsi_dh_alua)