Gpucv: a GPU-Accelerated Framework for Image Processing and Computer Vision
Total Page:16
File Type:pdf, Size:1020Kb
GpuCV: A GPU-accelerated framework for image processing and Computer Vision Y. Allusse, P. Horain Outline Why accelerating Computer Vision and Image Processing? How can GPUs help? GpuCV description Results Future works Conclusion page 2 6 mai 2010 Y. Allusse, P. Horain Motivation statement Why accelerating Computer Vision and Image Processing? Processing large images & HD videos Increasing data weight • Microscopy • Satellite images • Fine arts, printing • Video databases Images up to 100 GBytes ! page 4 6 mai 2010 Y. Allusse, P. Horain Real time computer vision Interactive applications, security… • Biometry • Multimodal applications • 3D motion tracking P A B / 4 G 3D/2D registration E P M page 5 6 mai 2010 Y. Allusse, P. Horain page page 6 Example application: Virtual conference Example application: Virtual Y. Allusse, P. Horain P. Allusse, Y. http://MyBlog3D.com demos page page 7 Example application: Virtual conference Example application: Virtual Y. Allusse, P. Horain P. Allusse, Y. http://MyBlog3D.com demos How to accelerate image processing? Available technologies for acceleration: • Extended instruction sets (ex.: Intel OpenCV / IPP) • Multiple CPU cores (ex.: IBM Cell) • Co-processor: - FPGA (field-programmable gate array) - GPU (graphical processing unit) GPUs available (almost) for free in consumer PCs page 8 6 mai 2010 Y. Allusse, P. Horain Graphics Processing Units GPU: what? Graphics Processing Unit Designed for advanced rendering • Multi-texturing effects. • Realistic lights and shadows effects. • Post processing visual effects. Originally in consumer PCs for gaming Image rendering device : Highly parallel processor High bandwidth memory page 10 6 mai 2010 Y. Allusse, P. Horain page page 11 GPU history: The power race 6 mai 2010mai6 Y. Allusse, P. Horain P. Allusse, Y. Source: GPU4Vision, http://gpu4vision.icg.tugraz.at GPU vs CPU: Specifications ] Effective Nbr. of u Avg Price in € Millions of j a Model processing Power in Watts processing r (12/01/2008) transistors a d n power in Gflops units i v o G a g a N : e c r NVIDIA u 300 336 110 754 112 o S GeForce 8800 GT [ ATI 300 475 215 700 320 Radeon HD 2900 XT Intel 169 17 65 291 2 Core 2 Duo E6700 AMD 168 19 125 227,4 2 64 x2 6000+ page 12 6 mai 2010 Y. Allusse, P. Horain GPU vs. CPU Processing power ratio on price and power consumption gFlops per € 4 Different purpose 2 Different architecture Different efficiency 0 gFlops per Watts gFlops NVIDIA GeForce 8800 GT ATI Radeon HD 2900 XT Intel Core 2 Duo E6700 AMD 64 x2 6000+ page 13 6 mai 2010 Y. Allusse, P. Horain GPU history: towards programming flexibility Until 2000: fixed architecture (not programmable) 2000-01: Pixel and Vertex shaders GeForce 3 and ATI R200. 2005: Geometry shaders GeForce 6800 and ATI Radeon X800. 2006: ATI CTM™ (for "Close To Metal"). ATI Radeon based GPUs. 2007: NVIDIA CUDA, ATI Stream SDK NVIDIA GeForce 8, AMD ATI FireStream. 2009 (Q3): OpenCL drivers & SDK available Apple Inc., AMD, IBM, Intel, Nvidia… page 14 6 mai 2010 Y. Allusse, P. Horain GPU pipeline Shaders allow application to run their own code in the graphic pipeline page 15 6 mai 2010 Y. Allusse, P. Horain CUDA thread batching GPU processing library from NVIDIA. Subdivide processing tasks in thousands of threads using blocks. C Style programming Full memory access page 16 6 mai 2010 Y. Allusse, P. Horain Cuda memory model Threads share memory => Synchronization mechanism Cache memory really fast (Shared Memory, Registers, Local Memory) Texture and constant Memory are fast but READ ONLY. Global Memory is slower but WRITABLE. page 17 6 mai 2010 Y. Allusse, P. Horain GPU vs. CPU(2) Benefits of GPU: • Processing power: - increasing faster than CPU, - cheaper than CPU, - highly parallel, • Easily upgradable. Benefits of CPU: • Flexible and general processing unit, • Stable programming languages. page 18 6 mai 2010 Y. Allusse, P. Horain GPU programming challenges Algorithms • Highly parallel • Coding limitations Dedicated APIs • OpenGL, shading languages, • Brook, CUDA, OpenCL… Development tools • Rapidly evolving APIs • Heterogeneous and scattered documentation GPU complexity to be hidden for wide acceptance page 19 6 mai 2010 Y. Allusse, P. Horain Open GPU libraries for image processing • OpenVIDIA: GPU Computer Vision (since 2002) http://openvidia.sourceforge.net • NVPP: NVIDIA Performance Primitives http://www.nvidia.com/nvpp • GPU4Vision: research in GPU-based vision algorithms http://gpu4vision.icg.tugraz.at • …and many more http://www.nvidia.com/cuda... Collections of image processing primitives and algorithms Basic framework page 20 6 mai 2010 Y. Allusse, P. Horain GpuCV Overview GpuCV in a few words: Aim: Accelerate computer vision with GPUs Initiated in 2005 ~ 2.5 person∙years effort Publications in major international conferences: • ACM Multimedia 08 (Open Source competition) • ISCV08 (Int. Symposium on Visual Computing) • IEEE ICME06 (Int. Conf. on Multimedia & Expo) Open source: picoforge.int-evry.fr/projects/gpucv ~ 100 visitors worldwide daily page 22 6 mai 2010 Y. Allusse, P. Horain GpuCV: Description A collection of GPU-accelerated operators for image processing and computer vision Initiated in 2005 A framework friendly to computer vision developers • Activation of low level GPU codelets: OpenGL & GLSL, NVIDIA CUDA (OpenCL on the way) • Data transfers between CPU and GPU memories • Compatible with OpenCV applications migration • Self-adaptive to the hardware platform spectrum of consumer PCs page 23 05/05/2010 Y. Allusse, P. Horain GpuCV: main features Compatible on multiple OS such as MS Windows XP and LINUX (Mac OSX on the way). Help developers to manage: Hardware compatibilities Data transfers between CPU and GPU memories Activation of low level GPU codelets: OpenGL & GLSL, NVIDIA CUDA, OpenCL (on the way) page 24 6 mai 2010 Y. Allusse, P. Horain GpuCV Operators plugins & Integration with OpenCV direction ou services GpuCV: Plugins Plugins are a set of operators sharing the same programming scheme or technologies: CPU: OpenMP / SSE / MMX … GPU: CUDA / OpenCL / OpenGL … They are in correspondance with their OpenCV counter-part: CXCORE / CV / HIGHGUI / CVAUX page 26 06/05/2010 Y. Allusse, P. Horain GpuCV: Plugins GpuCV = GPGPU framework + GPU-accelerated Computer Vision plugins OpenCV GPU-accelerated application GpuCV switch mechanism 3rd parts CUDA OpenGL plugins plugins plugins OpenCV library GpuCV framework page 27 06/05/2010 Y. Allusse, P. Horain GpuCV: Integration with OpenCV A strict naming convention is used to distinguish operators: CvAdd(): original OpenCV operator CvgAdd(): OpenGL accelerated operator CvgCudaAdd(): CUDA accelerated operator CvgswAdd(): switch operator Operators definition is stricly based on OpenCV definitions Function wrapper definitions in headers: #define cvAdd cvgswAdd page 28 06/05/2010 Y. Allusse, P. Horain GpuCV: Integration with OpenCV To get an existing OpenCV application on GPU: – Replace OpenCV headers by GpuCV headers – Link with GpuCV switch libraries – Add an init function 'cvgswInit(,)' A few coding rules to respect – NO direct access to buffer 'IplImage::imagedata' – Re-use images to avoid new allocations page 29 06/05/2010 Y. Allusse, P. Horain GpuCV: Example of OpenCV application #include … //other includes //size 3 cvMorphologyEx( imgSrc, imgTmp2, imgTmp, #include <cv.h> elemStrut3,CV_MOP_OPEN, 1); #include <cxcore.h> cvMorphologyEx( imgTmp2, imgDst, imgTmp, #include <highgui.h> elemStrut3,CV_MOP_CLOSE, 1); int main(int argc, char **argv) //size 5 { cvMorphologyEx( imgDst, imgTmp2, imgTmp, //load source image elemStrut5,CV_MOP_OPEN, 1); IplImage * imgSrc = cvLoadImage(MY_IMAGE_PATH); cvMorphologyEx( imgTmp2, imgDst, imgTmp, //create structuring elements of size 3 and 5 elemStrut5,CV_MOP_CLOSE, 1); IplConvKernel* elemStrut3 = cvCreateStructuringElementEx(3,3, 1, 1, CV_SHAPE_RECT, NULL); //show results IplConvKernel* elemStrut5 = cvCreateStructuringElementEx(5,5, 1, cvNamedWindow(« ResultImage », 1); 1, CV_SHAPE_RECT, NULL); cvShowImage(« ResultImage », imgDst); unsigned int depth = imgSrc->depth; cvWaitKey(0); unsigned int depth = imgSrc->nChannels; cvDestroyWindow(« ResultImage »); CvSize size = cvGetSize(imgSrc); //release data //create temporary images to be processed cvReleaseStructuringElement(&elemStrut3); IplImage * imgDst = cvReleaseStructuringElement(&elemStrut5); cvCreateImage(size, depth, channels); cvReleaseImage(&imgSrc); IplImage * imgTmp = cvReleaseImage(&imgDst); cvCreateImage(size, depth, channels); cvReleaseImage(&imgTemp); IplImage * imgTmp2 = cvReleaseImage(&imgTemp2); cvCreateImage(size, depth, channels); } page 30 May 6, 2010 Y. Allusse, P. Horain GpuCV: Example of OpenCV application #include … //other includes //size 3 cvMorphologyEx( imgSrc, imgTmp2, imgTmp, elemStrut3,CV_MOP_OPEN, 1); #include <cv_switch/cv_switch_wrapper.h> cvMorphologyEx( imgTmp2, imgDst, imgTmp, #include <cxcore_switch/cxcore_switch_wrapper.h> elemStrut3,CV_MOP_CLOSE, 1); #include <highgui_switch/highgui_switch_wrapper.h> //size 5 int main(int argc, char **argv) cvMorphologyEx( imgDst, imgTmp2, imgTmp, { elemStrut5,CV_MOP_OPEN, 1); //set path to GpuCV data files cvMorphologyEx( imgTmp2, imgDst, imgTmp, cvgSetShaderPath(MY_GPUCV_DATA_PATH); elemStrut5,CV_MOP_CLOSE, 1); //init GpuCV switch cvgswInit(true, false); //Create OpenGL context, simple thread //show results cvNamedWindow(« ResultImage », 1); //load source image cvShowImage(« ResultImage », imgDst); IplImage * imgSrc = cvLoadImage(MY_IMAGE_PATH); //create structuring elements