The Thrust library Will Landau Getting started Iterators Containers The Thrust library Algorithms Will Landau Iowa State University November 11, 2013 Will Landau (Iowa State University) The Thrust library November 11, 2013 1 / 42 Outline The Thrust library Will Landau Getting started Iterators Containers Getting started Algorithms Iterators Containers Algorithms Will Landau (Iowa State University) The Thrust library November 11, 2013 2 / 42 Getting started Outline The Thrust library Will Landau Getting started Iterators Containers Getting started Algorithms Iterators Containers Algorithms Will Landau (Iowa State University) The Thrust library November 11, 2013 3 / 42 Getting started Background The Thrust library Will Landau Getting started Iterators I Thrust is the CUDA analog of the Standard Template Containers Library (STL) of C++. It comes with any installation of Algorithms CUDA 4.2 and above and features: I Dynamic data structures I An encapsulation of GPU/CPU communication, memory management, and other low-level tasks. I High-performance GPU-accelerated algorithms such as sorting and reduction I Brief history: I Emerged from Komrade (deprecated) in 2009 I Maintained primarily by Jared Hoberock and Nathan Bell of NVIDIA. Will Landau (Iowa State University) The Thrust library November 11, 2013 4 / 42 Getting started vector1.cu The Thrust library Will Landau 1 #include <thrust/host vector .h> Getting started 2 #include <thrust/device vector .h> 3 #include <iostream> Iterators 4 5 int main( void) Containers 6 { 7 // H has storage for 4 integers Algorithms 8thrust::hostvector<int> H(4) ; 9 10 // initialize individual elements 11 H [ 0 ] = 1 4 ; 12 H [ 1 ] = 2 0 ; 13 H [ 2 ] = 3 8 ; 14 H [ 3 ] = 4 6 ; 15 16 // H. size () returns the size of vector H 17 s t d : : cout << ”H has size ” << H. size () << std :: endl ; 18 19 // print contents of H 20 for( int i=0;i< H. size () ; i++) 21 s t d : : cout << ”H[ ” << i << ” ] = ” << H[ i ] << std :: endl ; 22 23 // resize H 24 H. r e s i z e ( 2 ) ; 25 s t d : : cout << ”H now has size ” << H. size () << std :: endl ; 26 27 // Copy host vector H to device vector D 28 t h r u s t : : d e v i c e vector<int> D=H; Will Landau (Iowa State University) The Thrust library November 11, 2013 5 / 42 Getting started vector1.cu The Thrust library Will Landau Getting started 29 // elements of D can be modified Iterators 30 D[ 0 ] = 9 9 ; 31 D[ 1 ] = 8 8 ; Containers 32 33 // print contents of D Algorithms 34 for( int i=0;i< D. size () ; i++) 35 s t d : : cout << ”D[ ” << i << ” ] = ” << D[ i ] << std :: endl ; 36 37 // H and D are automatically deleted when the function returns 38 return 0; 39 } 1 > nvcc vector1 . cu ovector1 2 > ./ vector1 − 3Hhassize4 4H[0]=14 5H[1]=20 6H[2]=38 7H[3]=46 8Hnowhassize2 9D[0]=99 10 D[ 1 ] = 88 Will Landau (Iowa State University) The Thrust library November 11, 2013 6 / 42 Getting started Notes The Thrust library Will Landau Getting started Iterators I Thrust takes care of malloc(), cudaMalloc(), Containers free(), and cudaFree() for you without sacrificing Algorithms performance. I The “=” operator does a cudaMemcpy() if one vector is on the host and one is on the device. I thrust:: and std:: clarify the namespace of the function after the double colon. For example, we need to distinguish between thrust::copy() and std::copy(). I The “<<” operator sends a value to an output stream, the C++ alternative to printf(). Will Landau (Iowa State University) The Thrust library November 11, 2013 7 / 42 Getting started vector2.cu The Thrust library Will Landau 1 #include <thrust/host vector .h> 2 #include <thrust/device vector .h> Getting started 3 #include <thrust/copy.h> 4 #include <thrust/ fill .h> Iterators 5 #include <thrust/sequence .h> Containers 6 #include <iostream> 7 Algorithms 8 int main( void) 9 // initialize{ all ten integers of a device vector to 1 10 t h r u s t : : d e v i c e vector <int> D(10 , 1) ; 11 12 // set the first seven elements of a vector to 9 13 thrust:: fill(D.begin(), D.begin() + 7, 9); 14 15 // initialize a host vector with the first five elements of D 16 t h r u s t : : h o s t vector<int> H(D.begin(), D.begin() + 5); 17 18 // set the elements of H to 0, 1, 2, 3, ... 19 thrust::sequence(H.begin(), H.end()); 20 21 // copy all of H back to the beginning of D 22 thrust::copy(H.begin(), H.end(), D.begin()); 23 24 // print D 25 for( int i=0;i< D. size () ; i++) 26 s t d : : cout << ”D[ ” << i << ” ] = ” << D[ i ] << std :: endl ; 27 28 return 0; 29 } Will Landau (Iowa State University) The Thrust library November 11, 2013 8 / 42 Getting started vector2.cu The Thrust library Will Landau Getting started Iterators Containers 30 [landau@impact1 vector2]$ make Algorithms 31 nvcc v e c t o r 2 . cu ovector2 32 [landau@impact1− vector2]$ ./vector2 33 D[ 0 ] = 0 34 D[ 1 ] = 1 35 D[ 2 ] = 2 36 D[ 3 ] = 3 37 D[ 4 ] = 4 38 D[ 5 ] = 9 39 D[ 6 ] = 9 40 D[ 7 ] = 1 41 D[ 8 ] = 1 42 D[ 9 ] = 1 43 [landau@impact1 vector2]$ Will Landau (Iowa State University) The Thrust library November 11, 2013 9 / 42 Getting started Assignment The Thrust library Will Landau Getting started Iterators Containers Algorithms I thrust::copy() copies a section of one vector into a section of another. I thrust::fill() sets a range of elements to some fixed value. I thrust::sequence() assigns equally-spaced values to a section of a vector. Will Landau (Iowa State University) The Thrust library November 11, 2013 10 / 42 Getting started The vector template classes The Thrust library Will Landau Getting started Iterators I Declaring vectors: Containers I thrust::device vector<T> D; creates a vector D Algorithms with entries of data type T on the device. I The analogous declaration for host vectors is thrust:: host vector<T> H;. I An object D of the vector template class includes the following features: I A dynamic linear array of elements of type T. I Two iterators: I D.begin() I D.end() Will Landau (Iowa State University) The Thrust library November 11, 2013 11 / 42 Iterators Outline The Thrust library Will Landau Getting started Iterators Containers Getting started Algorithms Iterators Containers Algorithms Will Landau (Iowa State University) The Thrust library November 11, 2013 12 / 42 Iterators Basic iterators The Thrust library Will Landau Getting started I An iterator is a pointer with a C++ wrapper around it. Iterators The wrapper contains additional information, such as Containers whether the vector is stored on the host or the device. Algorithms 1 // allocate device vector 2thrust::devicevector<int> d vec (4) ; 3 4dvec . begin () ; // returns iterator at first element of d vec 5dvec .end() ; // returns iterator one past the last element of d vec 6 7 // [ begin , end) pair defines a sequence of 4 elements Will Landau (Iowa State University) The Thrust library November 11, 2013 13 / 42 Iterators Iterators act like pointers. The Thrust library Will Landau Getting started Iterators 1 // allocate device vector 2thrust::devicevector<int> d vec (4) ; Containers 3 4thrust::devicevector<int >:: iterator begin = d vec . begin () ; Algorithms 5thrust::devicevector<int >:: iterator end = d vec .end() ; 6 7 int length = end begin ; // compute the length of the vector 8 − 9end=dvec . begin () + 3; // define a sequence of 3 elements Will Landau (Iowa State University) The Thrust library November 11, 2013 14 / 42 Iterators Using iterators The Thrust library Will Landau Getting started Iterators Containers Algorithms 1 // allocate device vector 2thrust::devicevector<int> d vec (4) ; 3 4thrust::devicevector<int >:: iterator begin = d vec . begin () ; 5 6 begin = 13; // same as d vec [0] = 13; 7 ⇤int temp = begin ; // same as temp = d vec [0]; 8 ⇤ 9begin++;// advance iterator one position 10 11 begin = 25; // same as d vec [1] = 25; ⇤ Will Landau (Iowa State University) The Thrust library November 11, 2013 15 / 42 Iterators Wrap pointers to make iterators. The Thrust library Will Landau Getting started Iterators Containers 1 int N= 10; 2 Algorithms 3 // raw pointer to device memory 4 int raw ptr ; 5cudaMalloc((⇤ void )&rawptr , N sizeof( int)); 6 ⇤⇤ ⇤ 7 // wrap raw pointer with a device ptr 8thrust::deviceptr<int> dev iter(raw ptr) ; // dev iter is now an iterator pointing to device memory 9thrust::fill(deviter , dev iter + N, (int)0);// access device memory through device ptr 10 11 d e v iter [0] = 1; 12 13 // free memory 14 cudaFree ( r a w ptr) ; Will Landau (Iowa State University) The Thrust library November 11, 2013 16 / 42 Iterators Unwrap iterators to extract pointers. The Thrust library Will Landau Getting started Iterators Containers Algorithms 1 // allocate device vector 2thrust::devicevector<int> d vec (4) ; 3 4 // obtain raw pointer to device vector s memory 5 int ptr = thrust :: raw pointer cast(&d vec [0]) ; // use ptr in a CUDA C ⇤ kernel 6mykernel<<<N/256 , 256>>>(N , p t r ) ; 7 8 // Note : ptr cannot be dereferenced on the host ! Will Landau (Iowa State University) The Thrust library November 11, 2013 17 / 42 Iterators constant iterator The Thrust library Will Landau Getting started Iterators Containers I A constant iterator is a pointer with some constant value associated with it. Algorithms 1 #include <thrust/iterator/constant iterator .h> 2... 3 // create iterators 4thrust::constantiterator <int> first (10); 5thrust::constantiterator <int> last = first + 3; 6 7first[0];// returns 10 8first[1];// returns 10 9first[100];// returns 10 10 11 // sum of [ first , last) 12 thrust::reduce(first , last); // returns 30 ( i .