Cuda by Example.Book.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
ptg Download from www.wowebook.com CUDA by Example ptg Download from www.wowebook.com This page intentionally left blank ptg Download from www.wowebook.com CUDA by Example AN INTRODUCTION TO GENERAL!PUR POSE GPU PROGRAMMING ptg JASON SANDERS EDWARD KANDROT 8SSHU6DGGOH5LYHU1-ǩ%RVWRQǩ,QGLDQDSROLVǩ6DQ)UDQFLVFR 1HZ<RUNǩ7RURQWRǩ0RQWUHDOǩ/RQGRQǩ0XQLFKǩ3DULVǩ0DGULG &DSHWRZQǩ6\GQH\ǩ7RN\Rǩ6LQJDSRUHǩ0H[LFR&LW\ Download from www.wowebook.com 0DQ\RIWKHGHVLJQDWLRQVXVHGE\PDQXIDFWXUHUVDQGVHOOHUVWRGLVWLQJXLVKWKHLUSURGXFWVDUH FODLPHGDVWUDGHPDUNV:KHUHWKRVHGHVLJQDWLRQVDSSHDULQWKLVERRNDQGWKHSXEOLVKHUZDV DZDUHRIDWUDGHPDUNFODLPWKHGHVLJQDWLRQVKDYHEHHQSULQWHGZLWKLQLWLDOFDSLWDOOHWWHUVRULQDOO FDSLWDOV 7KHDXWKRUVDQGSXEOLVKHUKDYHWDNHQFDUHLQWKHSUHSDUDWLRQRIWKLVERRNEXWPDNHQRH[SUHVVHG RULPSOLHGZDUUDQW\RIDQ\NLQGDQGDVVXPHQRUHVSRQVLELOLW\IRUHUURUVRURPLVVLRQV1ROLDELOLW\LV DVVXPHGIRULQFLGHQWDORUFRQVHTXHQWLDOGDPDJHVLQFRQQHFWLRQZLWKRUDULVLQJRXWRIWKHXVHRIWKH LQIRUPDWLRQRUSURJUDPVFRQWDLQHGKHUHLQ 19,',$PDNHVQRZDUUDQW\RUUHSUHVHQWDWLRQWKDWWKHWHFKQLTXHVGHVFULEHGKHUHLQDUHIUHHIURP DQ\,QWHOOHFWXDO3URSHUW\FODLPV7KHUHDGHUDVVXPHVDOOULVNRIDQ\VXFKFODLPVEDVHGRQKLVRU KHUXVHRIWKHVHWHFKQLTXHV 7KHSXEOLVKHURIIHUVH[FHOOHQWGLVFRXQWVRQWKLVERRNZKHQRUGHUHGLQTXDQWLW\IRUEXONSXUFKDVHV RUVSHFLDOVDOHVZKLFKPD\LQFOXGHHOHFWURQLFYHUVLRQVDQGRUFXVWRPFRYHUVDQGFRQWHQW SDUWLFXODUWR\RXUEXVLQHVVWUDLQLQJJRDOVPDUNHWLQJIRFXVDQGEUDQGLQJLQWHUHVWV)RUPRUH LQIRUPDWLRQSOHDVHFRQWDFW 86&RUSRUDWHDQG*RYHUQPHQW6DOHV (800) 382-3419 FRUSVDOHV#SHDUVRQWHFKJURXSFRP )RUVDOHVRXWVLGHWKH8QLWHG6WDWHVSOHDVHFRQWDFW ,QWHUQDWLRQDO6DOHV LQWHUQDWLRQDO#SHDUVRQFRP 9LVLWXVRQWKH:HELQIRUPLWFRPDZ Library of Congress Cataloging-in-Publication Data ptg 6DQGHUV-DVRQ &8'$E\H[DPSOHDQLQWURGXFWLRQWRJHQHUDOSXUSRVH*38SURJUDPPLQJ -DVRQ6DQGHUV(GZDUG.DQGURW S FP ,QFOXGHVLQGH[ ,6%1 SENDONSDSHU $SSOLFDWLRQVRIWZDUHǟ'HYHORSPHQW&RPSXWHUDUFKLWHFWXUH 3DUDOOHOSURJUDPPLQJ &RPSXWHUVFLHQFH ,.DQGURW(GZDUG,,7LWOH 4$$6 ǟGF &RS\ULJKWk19,',$&RUSRUDWLRQ $OOULJKWVUHVHUYHG3ULQWHGLQWKH8QLWHG6WDWHVRI$PHULFD7KLVSXEOLFDWLRQLVSURWHFWHGE\FRS\- ULJKWDQGSHUPLVVLRQPXVWEHREWDLQHGIURPWKHSXEOLVKHUSULRUWRDQ\SURKLELWHGUHSURGXFWLRQ VWRUDJHLQDUHWULHYDOV\VWHPRUWUDQVPLVVLRQLQDQ\IRUPRUE\DQ\PHDQVHOHFWURQLFPHFKDQLFDO SKRWRFRS\LQJUHFRUGLQJRUOLNHZLVH)RULQIRUPDWLRQUHJDUGLQJSHUPLVVLRQVZULWHWR 3HDUVRQ(GXFDWLRQ,QF 5LJKWVDQG&RQWUDFWV'HSDUWPHQW %R\OVWRQ6WUHHW6XLWH %RVWRQ0$ )D[ ,6%1 ,6%1 7H[WSULQWHGLQWKH8QLWHG6WDWHVRQUHF\FOHGSDSHUDW(GZDUGV%URWKHUVLQ$QQ$UERU0LFKLJDQ )LUVWSULQWLQJ-XO\ Download from www.wowebook.com To our families and friends, who gave us endless support. To our readers, who will bring us the future. And to the teachers who taught our readers to read. ptg Download from www.wowebook.com This page intentionally left blank ptg Download from www.wowebook.com Contents )RUHZRUG [LLL 3UHIDFH [Y $FNQRZOHGJPHQWV [YLL $ERXWWKH$XWKRUV [L[ 1 WHY CUDA? WHY NOW? 1 &KDSWHU2EMHFWLYHV 2 7KH$JHRI3DUDOOHO3URFHVVLQJ 2 ptg &HQWUDO3URFHVVLQJ8QLWV 2 7KH5LVHRI*38&RPSXWLQJ 4 $%ULHI+LVWRU\RI*38V 4 (DUO\*38&RPSXWLQJ &8'$ :KDW,VWKH&8'$$UFKLWHFWXUH" 8VLQJWKH&8'$$UFKLWHFWXUH $SSOLFDWLRQVRI&8'$ 8 0HGLFDO,PDJLQJ 8 &RPSXWDWLRQDO)OXLG'\QDPLFV 9 (QYLURQPHQWDO6FLHQFH 10 &KDSWHU5HYLHZ 11 vii Download from www.wowebook.com CONTENTS 2 GETTING STARTED 13 &KDSWHU2EMHFWLYHV 14 'HYHORSPHQW(QYLURQPHQW 14 &8'$(QDEOHG*UDSKLFV3URFHVVRUV 14 19,',$'HYLFH'ULYHU 1 &8'$'HYHORSPHQW7RRONLW 1 6WDQGDUG&&RPSLOHr 18 &KDSWHU5HYLHZ 19 3 INTRODUCTION TO CUDA C 21 &KDSWHU2EMHFWLYHV 22 $)LUVW3URJUDP 22 ptg +HOOR:RUOG! 22 $.HUQHO&DOO 23 3DVVLQJ3DUDPHWHUV 24 4XHU\LQJ'HYLFHV 2 8VLQJ'HYLFH3URSHUWLHV 33 &KDSWHU5HYLHZ 3 4 PARALLEL PROGRAMMING IN CUDA C 37 &KDSWHU2EMHFWLYHV 38 &8'$3DUDOOHO3URJUDPPLQJ 38 6XPPLQJ9HFWRUV 38 $)XQ([DPSOH 4 &KDSWHU5HYLHZ viii Download from www.wowebook.com CONTENTS 5 THREAD COOPERATION 59 &KDSWHU2EMHFWLYHV 0 6SOLWWLQJ3DUDOOHO%ORFNV 0 9HFWRU6XPV5HGX[ 0 *385LSSOH8VLQJ7KUHDGV 9 6KDUHG0HPRU\DQG6\QFKURQL]DWLRQ 'RW3URGXFW 'RW3URGXFW2SWLPL]HG ,QFRUUHFWO\ 8 6KDUHG0HPRU\%LWPDS 90 &KDSWHU5HYLHZ 94 6 CONSTANT MEMORY AND EVENTS 95 ptg &KDSWHU2EMHFWLYHV 9 &RQVWDQW0HPRU\ 9 5D\7UDFLQJ,QWURGXFWLRQ 9 5D\7UDFLQJRQWKH*38 98 5D\7UDFLQJZLWK&RQVWDQW0HPRU\ 104 3HUIRUPDQFHZLWK&RQVWDQW0HPRU\ 10 0HDVXULQJ3HUIRUPDQFHZLWK(YHQWV 108 0HDVXULQJ5D\7UDFHU3HUIRUPDQFH 110 &KDSWHU5HYLHZ 114 7 TEXTURE MEMORY 115 &KDSWHU2EMHFWLYHV 11 7H[WXUH0HPRU\2YHUYLHZ 11 L[ Download from www.wowebook.com CONTENTS 7.3 Simulating Heat Transfer . 117 7.3.1 Simple Heating Model . 117 7.3.2 Computing Temperature Updates . 119 7.3.3 Animating the Simulation . 121 7.3.4 Using Texture Memory . 125 7.3.5 Using Two-Dimensional Texture Memory . 131 7.4 Chapter Review . 137 8 GRAPHICS INTEROPERABILITY 139 8.1 Chapter Objectives . 140 8.2 Graphics Interoperation . 140 8.3 GPU Ripple with Graphics Interoperability . 147 8.3.1 The GPUAnimBitmap Structure . 148 ptg 8.3.2 GPU Ripple Redux . 152 8.4 Heat Transfer with Graphics Interop . 154 8.5 DirectX Interoperability . 160 8.6 Chapter Review . 161 9 ATOMICS 163 9.1 Chapter Objectives . 164 9.2 Compute Capability . 164 9.2.1 The Compute Capability of NVIDIA GPUs . 164 9.2.2 Compiling for a Minimum Compute Capability . 167 9.3 Atomic Operations Overview . 168 9.4 Computing Histograms . 170 9.4.1 CPU Histogram Computation . 171 9.4.2 GPU Histogram Computation . 173 9.5 Chapter Review . 183 x Download from www.wowebook.com CONTENTS 10 STREAMS 185 &KDSWHU2EMHFWLYHV 18 3DJH/RFNHG+RVW0HPRU\ 18 &8'$6WUHDPV 192 8VLQJD6LQJOH&8'$6WUHDP 192 8VLQJ0XOWLSOH&8'$6WUHDPV 198 *38:RUN6FKHGXOLQJ 20 8VLQJ0XOWLSOH&8'$6WUHDPV(IIHFWLYHO\ 208 &KDSWHU5HYLHZ 211 11 CUDA C ON MULTIPLE GPUS 213 &KDSWHU2EMHFWLYHV 214 ptg =HUR&RS\+RVW0HPRU\ 214 =HUR&RS\'RW3URGXFW 214 =HUR&RS\3HUIRUPDQFH 222 8VLQJ0XOWLSOH*38V 224 3RUWDEOH3LQQHG0HPRU\ 230 &KDSWHU5HYLHZ 23 12 THE FINAL COUNTDOWN 237 &KDSWHU2EMHFWLYHV 238 &8'$7RROV 238 &8'$7RRONLW 238 &8))7 239 &8%/$6 239 19,',$*38&RPSXWLQJ6'. 240 [L Download from www.wowebook.com CONTENTS 12.2.5 NVIDIA Performance Primitives . 241 12.2.6 Debugging CUDA C . 241 12.2.7 CUDA Visual Profiler . 243 12.3 Written Resources . 244 12.3.1 Programming Massively Parallel Processors: A Hands-On Approach . 244 12.3.2 CUDA U . 245 12.3.3 NVIDIA Forums . 246 12.4 Code Resources . 246 12.4.1 CUDA Data Parallel Primitives Library . 247 12.4.2 CULAtools . 247 12.4.3 Language Wrappers . 247 12.5 Chapter Review . 248 ptg A ADVANCED ATOMICS 249 A.1 Dot Product Revisited . 250 A.1.1 Atomic Locks . 251 A.1.2 Dot Product Redux: Atomic Locks . ..