HDF5 Portable Parallel - Juelich.De I/O and I/O Data Formats Member of the Helmholtz-Association March March 14Th, 2017 Outline

Member of the Helmholtz-Association Jülich, Jülich, March 14th, 2017 Forschungszentrum Jülich GmbH Jülich Supercomputing Centre s.luehrs@fz Sebastian Lührs HDF5 Portable Parallel - juelich.de I/O and I/O Data Formats Member of the Helmholtz-Association March March 14th, 2017 Outline . . . HDF5 Introduction Parallel ...... HDF5 Closing Creating Terms Structure Partial Row Writing/ Creating Creating - programming HDF5 major predefined and I/O reading HDF5 / opening datasets dataspaces of definitions / the column files data HDF5 datatypes HDF5 model and major library other files and objects API 2 Member of the Helmholtz-Association March March 14th, 2017 What . . Tools API, is data suite HDF5? model for H ierarchical accessing and file format data D ata in HDF5 F for ormat I/O management format 3 Member of the Helmholtz-Association March March 14th, 2017 HDF5 ...... Provides Supports Available Portable management Self I/O parallelSupports . - describing Pythonic Features file tools C on a , C++, format interfaces of variety to data complex operate Fortran model of also platforms data on HDF5 90 available which and sets Java allows files and the data 4 Member of the Helmholtz-Association March March 14th, 2017 Layers stdio of Object the files split API (C, F90, C++, Java, Python) HDF5 Library Library VirtualI/O File Storage Application Internals device MPI I/O MPI Custom

5 Your HDF5 Library Application Member of the Helmholtz-Association March March 14th, 2017 File organization . Unix/Linux file system file system (fs) Unix/Linux file structure corresponds inmanyrespects HDF5 to a Data Data Group HDF set 5 ↔ ↔ Unix/Linux Directory File fs 6 Member of the Helmholtz-Association March March 14th, 2017 Terminology Multi Dataset data pointsrespectively, i.e. the it describes shape of of the the data array dimensionality Describes and the shape ofthe Dataspace to Can beused datasets describe and isattached to them Attribute datasets objects, e.g. may Structure contain HDF5datasets, which attributes, Group Container File - dimensional array of data elements dimensional for storing data a dataset 7 Member of the Helmholtz-Association March March 14th, 2017 Library specific types . . Fortran C Own Defined INTEGER(HSSIZE_T) INTEGER(HSIZE_T) INTEGER(HID_T) use hdf5 hvl_t hssize_t hsize_t herr_t hid_t #include hdf5.h defined types Variable length datatype Used for dimensions Object identifier Used for coordinates and dimensions Function return value types are integers ensure Used for coordinates Used for coordinates and dimensions Used Object identifier portability of for dimensions different size 8 Member of the Helmholtz-Association March March 14th, 2017 Fortran HDF5 open . . Fortran status code variables) The HDF5 H5CLOSE_F H5OPEN_F INTEGER, INTENT(OUT) :: STATUS INTEGER, INTENT(OUT) :: STATUS and returns closed (STATUS) by library (STATUS) calling ( 0 H5CLOSE_F interface if H5OPEN_F successful needs ) at before the to be end initialized it . can be used (e.g. global in your 9 Member of the Helmholtz-Association March March 14th, 2017 API ...... H5P H5F H5G H5D H5 H5S naming scheme(excerpt) ...... Group Dataset interface: dataset accessand manipulation Library functions: general routines Dataspace interface: dataspace definition and access routines Property listinterface: property object list manipulation File interface: file accessroutines interface: group group creation andmanipulation routines - purpose functions routines 10 Member of the Helmholtz-Association March March 14th, 2017 General H5 F create, H5 Procedure H5 H5 F S H5 close create_simple H5 D S create, H5 F close H5 H5 open D D read, H5 close D open D write

create/close Dataset

create/close Dataspace

create/close HDF5 File 11 Member of the Helmholtz-Association March March 14th, 2017 Creating . . . . Fortran C access_flags name Fortran uses property list, creation_prp H5FCREATE_F hid_t INTEGER(KIND=HID_T), OPTIONAL, INTENT(IN) :: INTEGER, INTENT(OUT) :: HDFERR INTEGER(KIND=HID_T), INTENT(OUT) :: INTEGER, INTENT(IN) :: ACCESS_FLAGS CHARACTER(*), INTENT(IN) :: NAME : of Name the file CREATION_PRP H5Fcreate an HDF5 H5P_DEFAULT[_F] file_id (NAME, ACCESS_FLAGS, FILE_ID, HDFERR, CREATION_PRP, ACCESS_PRP) : File access access flags : File and ( , hid_t access_flag const access_prp ACCESS_PRP as return value file access_prp char *name, unsigned , if not : File creation and access : creation andaccess File hid_t specified ) creation_prp FILE_ID , 12 Member of the Helmholtz-Association March March 14th, 2017 Opening . . . . Fortran C not access_prp name Avoid Fortran uses H5FOPEN_F hid_t INTEGER(KIND=HID_T), OPTIONAL, INTENT(IN) :: INTEGER, INTENT(OUT) :: HDFERR INTEGER(KIND=HID_T), INTENT(OUT) :: INTEGER, INTENT(IN) :: FLAGS CHARACTER(*), INTENT(IN) :: NAME specified : of Name the file ACCESS_PRP multiple H5Fopen an (NAME, FLAGS, FILE_ID, HDFERR, ACCESS_PRP) file_id : File access access : File existing opens ( hid_t const of as return value the access_prp char *name, unsigned flags, property list, HDF5 same file file ) H5P_DEFAULT[_F] FILE_ID if 13 Member of the Helmholtz-Association March March 14th, 2017 Access modes Access ...... access_prp creation_prp ( property More irrelevant for H5F_ACC_RDONLY for H5F_ACC_RDWR exists file already H5F_ACC_EXCL file existing H5F_ACC_TRUNC[_F] files access_prp H5Fcreate[_f] specific list ( creation_prp H5Fcreate[_f] settings controls ) which [_F] [_F] controls [_F] : : defaults are different Open fileinread Create anewfile, : Create anewfile, : file Open fileinread controlled metadata ) to and methods H5P_DEFAULT[_F] file through access - write mode,irrelevant of H5Fcreate - overwrite only mode, only performing file property creation an fails if fails I/O on lists 14 Member of the Helmholtz-Association March March 14th, 2017 Group . . . . Fortran C loc_id use lcpl_id name H5GCREATE_F hid_t INTEGER(KIND=HID_T), OPTIONAL, INTENT(IN) :: INTEGER(KIND=SIZE_T), OPTIONAL, INTENT(IN) :: INTEGER, INTENT(OUT) :: HDFERR INTEGER(KIND=HID_T), INTENT(OUT) :: GRP_ID CHARACTER(LEN=*), INTENT(IN) :: NAME INTEGER(KIND=HID_T), INTENT(IN) :: LOC_ID H5Gclose LCPL_ID, SIZE_HINT creation can bean absolute orrelative path H5Gcreate : be the Can , gcpl_id (LOC_ID, NAME, GRP_ID, HDFERR, SIZE_HINT, LCPL_ID, GCPL_ID, GAPL_ID) [_f] GCPL_ID, ( hid_t hid_t hid_t file_id , to finalize gapl_id GAPL_ID gapl_id lcpl_id loc_id or another group : , Property , ) const hid_t access group_id lists lists char *name, gcpl_id for link/group , 15 Member of the Helmholtz-Association March March 14th, 2017 Closing Fortran C H5FCLOSE_F herr_t INTEGER, INTENT(OUT) :: HDFERR INTEGER(KIND=HID_T), INTENT(IN) :: an HDF5 H5Fclose (FILE_ID, HDFERR) ( hid_t file file_id ) FILE_ID 16 Member of the Helmholtz-Association March March 14th, 2017 Exercise

Exercise 1 – HDF5 hello world . mpif90 helloworld_hdf5.f90 module load HDF5/1.8.16 module load intel mpicc module load HDF5/1.8.16 module load intel h5dump the file using: Check resulting . . Solutions Solutions and templates: Create agroupof “data” inside thisfile file anHDF5closes Write aserialprograminCorFortrancreates which and helloworld_hdf5.c - - para para /work/ - lhdf5 hpclab - lhdf5_fortran /train001 17 Member of the Helmholtz-Association March March 14th, 2017 HDF5 . . . Fortran C ones Native platform HDF5 Native HDF5 real integer double float int F type C type pre which files file datatypes datatype H5T_IEEE_F64[BE,LE] H5T_IEEE_F32[BE,LE] H5T_STD_I32[BE,LE] HDF5 file type (pre type - . defined H5T_IEEE_F32[BE,LE] H5T_STD_I32[BE,LE] HDF5 file type (pre are depends referred might are not in anHDF differ datatypes on to by - compiler defined) from - defined) native platform file datatypes switches H5T_NATIVE_DOUBLE H5T_NATIVE_FLOAT H5T_NATIVE_INT HDF5 memory type (native) H5T_NATIVE_REAL H5T_NATIVE_INTEGER HDF5 memory type (native) ( but excerpt to the platform and appear pre - underlying defined ) in the 18 Member of the Helmholtz-Association March March 14th, 2017 Dataspace . . . maximum_size current_size rank: The Metadata dataset The Simple Simple dataspace dataspace dataspace int . . . . Storage Attributes Datatype Dataspace are : : hsize_t hsize_t : info describes is part [rank] [rank] of the the metadata size rank = and 2 , of shape dimensions the underlying of the = 2x5 dataset 19 Member of the Helmholtz-Association March March 14th, 2017 Creating . . . . Fortran C current_dims maximum_dims rank use to “infinite” dimensions H5S_UNLIMITED[_F] H5SCREATE_SIMPLE_F hid_t INTEGER(KIND=HISIZE_T)(*), OPTIONAL, INTEGER, INTENT(OUT) INTEGER(KIND=HID_T), INTENT(OUT) :: SPACE_ID INTEGER(KIND=HISIZE_T)(*), INTENT(IN) :: DIMS INTEGER, INTENT(IN) :: RANK H5Sclose[_f] : INTENT(OUT) Number H5Screate_simple a dataspace MAXDIMS) of are may dimensions :: MAXDIMS the be to finalize dataspace access to finalize (RANK, DIMS, SPACE_ID, HDFERR, size can be same NULL ( :: HDFERR const const int . used as Then rank, hsize_t hsize_t maximum_dims maximum_dims * * maximum_dims current_dims and to set ) , 20 Member of the Helmholtz-Association March March 14th, 2017 Creating . Fortran C classtype H5SCREATE_F hid_t INTEGER(HID_T), INTEGER, INTENT(IN) :: CLASSTYPE INTEGER, INTENT(OUT) :: HDFERR H5Screate a : dataspace (CLASSTYPE, SPACE_ID, HDFERR) H5S_SCALAR[_F] (H5S_class_t type) INTENT(OUT) :: SPACE_ID or H5S_SIMPLE[_F] 21 Member of the Helmholtz-Association March March 14th, 2017 Creating an Attribute . . . Fortran C loc_id use ACPL_ID, AAPL_ID committed datatype) file or an HDF5 H5ACREATE_F hid_t INTEGER(KIND=HID_T), OPTIONAL, INTENT(IN) :: INTEGER, INTENT(OUT) :: HDFERR INTEGER(KIND=HID_T), INTENT(OUT) :: ATTR_ID INTEGER(KIND=HID_T), INTENT(IN) :: TYPE_ID, CHARACTER(LEN=*), INTENT(IN) :: NAME INTEGER(KIND=HID_T), INTENT(IN) :: LOC_ID H5Aclose[_f] ACPL_ID, AAPL_ID SPACE_ID H5Acreate may be object identifierany HDF5 (group, dataset, or (LOC_ID, NAME, TYPE_ID, SPACE_ID, ATTR_ID, HDFERR, ( hid_t hid_t hid_t : to finalize H5P_DEFAULT[_F] acpl_id type_id loc_id the attribute access , ACPL_ID, AAPL_ID) , , identifier const hid_t hid_t if not char * aapl_id space_id specified attr_name ) , , 22 Member of the Helmholtz-Association March March 14th, 2017 Writing an Attribute . Fortran C data buffer Fortran H5AWRITE_F herr_t INTEGER, INTENT(OUT) :: HDFERR INTEGER(KIND=HSIZE_T)(*), INTENT(IN):: TYPE, INTENT(IN) INTEGER(KIND=HID_T), INTENT(IN) :: MEMTYPE_ID INTEGER(KIND=HID_T), INTENT(IN) :: ATTR_ID : DIMS H5Awrite buf (ATTR_ID, MEMTYPE_ID, BUF, DIMS, HDFERR) array (new since 1.4.2) (new since ( const hid_t to hold corresponding dimension sizes of sizes dimension to corresponding hold :: BUF void * attr_id buf , ) hid_t mem_type_id DIMS , 23 Member of the Helmholtz-Association March March 14th, 2017 Writing . . call H5Tclose_f( … call H5Tset_strpad_f(atype,H5T_STR_NULLTERM_F) call H5Tset_size_f( call H5Tcopy_f(H5T_C_S1,atype,status) StringType H5Tclose( … H5Tset_strpad(atype,H5T_STR_NULLTERM); H5Tset_size( atype StringType an Attribute = H5Tcopy(H5T_C_S1); Example (C): Example atype atype ); (Fortran): atype , 5); int atype , status) (5,HSIZE_T),status) , H5T_STR_NULLPAD H5T_STR_SPACEPAD alternative: 24 Member of the Helmholtz-Association March March 14th, 2017 Dataset ( Dataset - 1 4 0 09 - 1 4 0 09 - 1 4 0 0 9 2 45 2 4 5 2 45 metadata

Data + data Datatype Dataspace • • • • • ) Integer dim dim dim rank =3 [2] = 3 [1] = 4 [0] = 2 Metadata Storage Attributes • • • Contiguous Temp Time =2.1 = 122 25 Member of the Helmholtz-Association March March 14th, 2017 Creating aDataset . Fortran C use H5DCREATE_F hid_t INTEGER(KIND=HID_T), OPTIONAL, INTENT(IN) :: INTEGER, INTENT(OUT) :: HDFERR INTEGER(KIND=HID_T), INTENT(OUT) :: DSET_ID INTEGER(KIND=HID_T), INTENT(IN) :: TYPE_ID, CHARACTER(LEN=*), INTENT(IN) :: NAME INTEGER(KIND=HID_T), INTENT(IN) :: LOC_ID H5Dclose[_f] DCPL_ID, LCPL_ID, DAPL_ID SPACE_ID H5Dcreate DSET_ID, HDFERR, DCPL_ID, LCPL_ID, DAPL_ID) (LOC_ID, NAME, TYPE_ID, SPACE_ID, ( hid_t hid_t hid_t hid_t to finalize the to dataset access finalize dapl_id lcpl_id dtype_id loc_id , ) , , const hid_t hid_t char *name, dcpl_id space_id , , 26 Member of the Helmholtz-Association March March 14th, 2017 Creating aDataset . . . . . dapl_id lcpl_id dcpl_id space_id type_id : list access propertyDataset : list propertycreationLink : list propertycreationDataset : Datatypeidentifier : identifier Dataspace 27 Member of the Helmholtz-Association March March 14th, 2017 Property Lists ...... Access . . Creation Default: in HDF5 handlingdata internal Propertylists ( . Transferproperties . The driver used The driver to open afile (e.g. MPI compression orchecksum Specify chunked Whether a dataset stored is a compact, in or contiguous, Collective Optimization settings in specialized properties H5P_DEFAULT[_ properties filters filters to to a datasetbe applied layout or independent H5P ) the to changecan beused I/O evaluation) F] environments - I/O or (e.g. g Posix zip ) 28 Member of the Helmholtz-Association March March 14th, 2017 Recipe 6. 5. 4. 3. 2. 1. Close all Close Create Specify Define Specify Get identifier : Creating dataspace dataset property datatype opened for dataset lists (integer, objects an empty ( or location H5P_DEFAULT[_F] composite dataset etc.) ) 29 Member of the Helmholtz-Association March March 14th, 2017 Exercise

Exercise 2 – HDF5 metadata handling h5dump the file using: Check resulting . . . . Write astring into this attribute value Add astring attribute connected to this integer values be atwo should dimensional the Create inside“data” group anempty dataset which Extend yourserialprogram array (5x20 elements) of dataset 30 Member of the Helmholtz-Association March March 14th, 2017 Writing . . Fortran C or H5S_ALL[_F] specify collective or independent parallel I/O) or parallel independentspecify collective xfer_plist_id H5DWRITE_F herr_t file_space INTEGER(HID_T), OPTIONAL, INTENT(IN) :: INTEGER, INTENT(OUT) :: HDFERR DIMENSION(*), INTEGER(HSIZE_T), INTENT(IN) :: DIMS TYPE, INTENT(IN) :: BUF INTEGER(HID_T), INTENT(IN) :: DSET_ID, MEM_TYPE_ID MEM_SPACE_ID, FILE_SPACE_ID, XFER_PRP to H5Dwrite a (DSET_ID, MEM_TYPE_ID, BUF, DIMS, HDFERR, MEM_SPACE_ID, FILE_SPACE_ID, XFER_PRP) dataset can be used tospecifyno special identifier / ( xfer_prp const file_space_id hid_t hid_t void * mem_space_id dataset_id is atransfer property (e.g. to buf , hid_t , ) , hid_t hid_t xfer_plist_id mem_type_id mem_space , , 31 Member of the Helmholtz-Association March March 14th, 2017 Writing mem_space_id dataspace id dataspace id H5S_ALL H5S_ALL to a dataset file_space_id dataspace id dataspace id H5S_ALL H5S_ALL mem_space use default use selection) mem_space use given use dataspaces all selection selection for default file_space file_space , set dataspace (including the Behaviour as is all selection selection for both dataspace also for also for file_space 32 Member of the Helmholtz-Association March March 14th, 2017 Open . Fortran C dapl_id H5DOPEN_F hid_t INTEGER(HID_T), INTENT(IN) :: LOC_ID INTEGER(HID_T), OPTIONAL, INTENT(IN) :: DAPL_ID INTEGER, INTENT(OUT) :: HDFERR INTEGER(HID_T), INTENT(OUT) :: DSET_ID CHARACTER(LEN=*), INTENT(IN) :: NAME a existing H5Dopen : Dataset (LOC_ID, NAME, DSET_ID, HDFERR) ( dapl_id hid_t access access property list dataset loc_id ) , const char *name, hid_t 33 Member of the Helmholtz-Association March March 14th, 2017 Dataspace . . Fortran C dimension information dimension H5Sget_simple_extent_dims H5Sget_simple_extent_ndims Returns anidentifierfor a copyof the dataspace for a dataset H5DGET_SPACE_F hid_t INTEGER(HID_T), INTENT(OUT) :: DATASPACE_ID INTEGER(HID_T), INTENT(IN) :: DATASET_ID INTEGER, INTENT(OUT) :: HDFERR H5Dget_space inquiry (DATASET_ID, DATASPACE_ID, HDFERR) ( hid_t dataset_id can be used toextract and ) . 34 Member of the Helmholtz-Association March March 14th, 2017 Reading a . . Fortran C or H5S_ALL[_F] specify collective or independent parallel I/O) or parallel independentspecify collective xfer_plist_id H5DREAD_F herr_t file_space INTEGER(HID_T), OPTIONAL, INTENT(IN) :: INTEGER, INTENT(OUT) :: HDFERR DIMENSION(*), INTEGER(HSIZE_T), INTENT(IN) :: DIMS TYPE, INTENT(IN) :: BUF INTEGER(HID_T), INTENT(IN) :: DSET_ID, MEM_TYPE_ID MEM_SPACE_ID, FILE_SPACE_ID, XFER_PRP H5Dread (DSET_ID, MEM_TYPE_ID, BUF, DIMS, HDFERR, dataset MEM_SPACE_ID, FILE_SPACE_ID, XFER_PRP) can be used tospecifyno special identifier ( / void * file_space_id hid_t hid_t xfer_prp mem_space_id dataset_id buf is atransfer property (e.g. to ) , hid_t , , hid_t hid_t xfer_plist_id mem_type_id mem_space , , 35 Member of the Helmholtz-Association March March 14th, 2017 Exercise

Exercise 3 – HDF5 write data h5dump the file using: Check resulting . . . Write thisarray dataset into theexistingempty HD5 … 41 4243444546 21 222324252627… 1 array values with Create atwo dimensional Extend yourserialprogram 1 234567… up to 100 4 7 … 36 Member of the Helmholtz-Association March March 14th, 2017 Excursion Storing data Storing dataina data “Logical” view: Value Value Adress Fortran in a : C row 2x3 3x2 C: 1 1 1 dimensional dataset: dimensional dataset: HDF5 dimensional - major / Fortran: 5 3 1 M[ 6 4 2 3 2 2 i,j ] = Fortran: 2 1 5 3 1 column 5 3 3 4 3 6 4 2 6 5 4 5 1 6 2 3 2 4 4 - major 4 5 5 order 6 6 6 37 Member of the Helmholtz-Association March March 14th, 2017 Partial I/O - H yperslabs 38 Member of the Helmholtz-Association March March 14th, 2017 Partial I/O - H yperslabs 39 Member of the Helmholtz-Association March March 14th, 2017 Partial I/O . . Hyperslabs Hyperslabs ...... start B R C to Dimension block count stride locks locks egular ontiguous dimension : ( - : : : or H size number separation are are in a pattern yperslabs offset of dataspace described portions collection of block ): of these of of starting the blocks blocks points to underlying four be of of by location to points to selected datasets in a parameters be be four dataspace selected selected in a parameters dataspace from dataspace corresponds dataspace : 40 Member of the Helmholtz-Association start start March March 14th, 2017 Hyperslab [0,0] [1]=1 [0]=0 [8,0] example count [1]=4 [0,12] [8,12]

count[0]=2 block[0]=3 block[1]=2 stride [1]=3 41 stride[0]=4 Member of the Helmholtz-Association March March 14th, 2017 Creating Fortran C H5SSELECT_HYPERSLAB_F herr_t INTEGER(HSIZE_T), DIMENSION(*), OPTIONAL, INTEGER, INTENT(OUT) :: HDFERR INTEGER(HSIZE_T), DIMENSION(*), INTENT(IN) :: INTEGER, INTENT(IN) :: OP INTEGER(HID_T), INTENT(IN) :: SPACE_ID INTENT(IN) :: STRIDE, BLOCK START, H5Sselect_hyperslab hyperslabs COUNT, HDFERR, STRIDE, BLOCK) COUNT * const H5S_seloper_t count, hsize_t const (SPACE_ID, OPERATOR, START, hsize_t * op, ( stride, hid_t const space_id * block ) const hsize_t hsize_t , * start, 42 Member of the Helmholtz-Association March March 14th, 2017 Creating ...... selections: The H5S_SELECT_NOTB elements that aremembers of both members of thenew selection or the existing selection, excluding H5S_SELECT_XOR new selectionand the existing H5S_SELECT_AND selection. H5S_SELECT_OR this operator. parameters from thiscall.Overlapping blocks arenotsupported with H5S_SELECT_SET[_F] that are not in theexistingselection. H5S_SELECT_NOTA selection that are not in the new selection. following following operators operators ( hyperslabs [_F] [_F] [_F] [_F] [_F] op : Adds ) : Retains : : Replaces are : Retains : Retains Retains Retains supported to combine old andnew the new selection to the existing selection. only the elements that are only the overlapping only the overlapping portions of the only elements of the newselection only elements of theexisting the existingselection with the selections. 43 Member of the Helmholtz-Association Jülich, Jülich, March 14th, 2017 Forschungszentrum Jülich GmbH Jülich Supercomputing Centre s.luehrs@fz Sebastian Lührs Parallel HDF5 Portable Parallel - juelich.de I/O and I/O Data Formats Member of the Helmholtz-Association March March 14th, 2017 Factoids . . . . different A to all processes file image Single files HDF5 with serial PHDF5 filescompatible Supports standard . . One fileperprocess is undesirable design between differentShareable platforms or serial parallel platforms MPI parallel I/O parallel programming . interface must be portable to 45 Member of the Helmholtz-Association March March 14th, 2017 Implementation Parallel HDF5 + HDF5MPIParallel Parallel MPI I/O Application layers (GPFS, Parallel Parallel I/O HDF5 User Application library Lustre file systems layer , …) 46 Member of the Helmholtz-Association March March 14th, 2017 Important toknow . . . communicator After file withacommunicator a parallelPHDF5 opens Most ...... a file is opened by the processes of aby theprocesses a fileisopened functions communicators Different differentfiles canbeopenedvia Future access to the the file via Returns a file i.e. all Each process may write to an individual data array Each process may write to an individual processes may write to Multiple the same data array processes inthe by all objects file areaccessible All parts processes All of by all file areaccessible processes of - handle the of PHDF5 PHDF5 API the communicator are - handle collectives must participate 47 Member of the Helmholtz-Association March March 14th, 2017 MPI . . . Fortran C cls_id template and sets access it up with MPI parallel Each process of the MPI communicator creates an access file creation/access Property isused during H5PSET_FAPL_MPIO_F H5PCREATE_F herr_t hid_t - IO template access INTEGER, INTENT(OUT) :: HDFERR INTEGER, INTENT(IN) :: INFO INTEGER, INTENT(IN) :: COMM INTEGER(HID_T), INTENT(IN) :: PRP_ID INTEGER, INTENT(OUT) :: HDFERR INTEGER(HID_T), INTEGER, INTENT(IN) :: CLASSTYPE H5Pcreate / H5Pset_fapl_mpio classtype (CLASSTYPE, PRP_ID, HDFERR) ( hid_t (PRP_ID, COMM, INFO, HDFERR) INTENT(OUT) :: PRP_ID must cls_id ( be comm hid_t H5P_FILE_ACCESS[_F] , ); MPI_Info fapl_id , info) MPI_Comm information 48 Member of the Helmholtz-Association March March 14th, 2017 Dataset . . Fortran C xfer_modes cls_id H5PSET_DXPL_MPIO_F H5PCREATE_F herr_t hid_t INTEGER, INTENT(OUT) :: HDFERR INTEGER, INTENT(IN) :: DATA_XFER_MODE INTEGER(HID_T), INTENT(IN) :: PRP_ID INTEGER, INTENT(OUT) :: HDFERR INTEGER(HID_T), INTEGER, INTENT(IN) :: CLASSTYPE . . H5FD_MPIO_COLLECTIVE[_F] access (default) H5FD_MPIO_INDEPENDENT[_F] H5Pcreate / transfer H5Pset_dxpl_mpio classtype : (CLASSTYPE, PRP_ID, HDFERR) ( hid_t property (PRP_ID, DATA_XFER_MODE, HDFERR) INTENT(OUT) :: PRP_ID must be cls_id ( H5FD_mpio_xfer_t hid_t H5P_DATASET_XFER[_F] ); : dxpl_id Use collective I/O access : Use independent Use independent I/O , xfer_mode ) 49 Member of the Helmholtz-Association March March 14th, 2017 Performance hints . . C Additional chunk cache ispossible chunk cache Additional are allstored separately inthe the whichfile, chunkeddatasets chunks are splitintomultiple Chunking: H5Pset_chunk dcpl_id = Contiguous H5Pcreate ( dcpl_id (H5P_DATASET_CREATE); datasets datasets are stored in a singleblock in , 2, file. chunk_dims https://www.hdfgroup.org/HDF5/doc/Advanced/Chunking/ ); 50 Member of the Helmholtz-Association March March 14th, 2017 Performance hints ...... h5perf Example Options ( Example Chunking I/O Independent andcollective C Part of the installation standard HDF5 1D or2Ddataset I/O HDF5 Simple ontiguous or interleaved access access pattern or interleaved ontiguous . . . . . Number Number of datasets ( Transfer size ( Block size( Bytes perProcess( 1D /2D( - g - - ) benchmark application h5perf B ) - x / - e - - ) d X - ) ) h ): 51 Member of the Helmholtz-Association March March 14th, 2017 Performance hints ...... Example interleaved contiguous transfer block bytes num 0 0 - processes 0 0 (1D): - - size =2 per 0 1 0 1 - buffer - 0 2 process 0 2 0 0 = 3 0 0 - size =4 1 1 https://www.hdfgroup.org/HDF5/doc/Tools/h5perf_parallel/h5perf_parallel.pdf 1 1 = 8 1 2 1 2 1 0 1 0 1 1 1 1 2 2 2 2 2 1 write operation 1 writeoperation per transfer write operations per transfer 2 0 2 0 2 1 2 1 2 2 2 2 52 Member of the Helmholtz-Association interleaved March March 14th, 2017 Performance hints . . . . Example 0 0 0 0 0 0 0 0 transfer block bytes num 0 0 0 0 0 0 0 0 - 1 1 1 1 1 1 1 1 processes (2D): - - 1 1 1 1 1 1 1 1 size =2 per - 0 0 0 0 0 0 0 0 buffer - process 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 = 2 1 1 1 1 1 1 1 1 - size =8 8 write operations 8 writeoperations per transfer https://www.hdfgroup.org/HDF5/doc/Tools/h5perf_parallel/h5perf_parallel.pdf = 4 contiguous 1 1 1 1 0 0 0 0 1 write operation 1 writeoperation per transfer 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 53 Member of the Helmholtz-Association March March 14th, 2017 Exercise

Exercise 4 – parallel HDF5 ...... Mandelbrot hdf5 implementation (use Time left? Try to Templateis available: the file using: Check resulting Write the data collectively … 1 … 1 … … 0 … 0 … view: Logical Create acombineddatasetof involved processes all array with the rank numberyour two dimensional Fill Extend yourserialprogramto a parallel h5perf - h run a to see all available options) or run the to available see all /work/ h5perf into the file #cores x5 h5dump hpclab benchmark /train001 program 54