Parallelization of Dynamic Programming Recurrences in Computational Biology Arpith Jacob Washington University in St
Total Page:16
File Type:pdf, Size:1020Kb
Washington University in St. Louis Washington University Open Scholarship All Theses and Dissertations (ETDs) January 2010 Parallelization of dynamic programming recurrences in computational biology Arpith Jacob Washington University in St. Louis Follow this and additional works at: https://openscholarship.wustl.edu/etd Recommended Citation Jacob, Arpith, "Parallelization of dynamic programming recurrences in computational biology" (2010). All Theses and Dissertations (ETDs). 169. https://openscholarship.wustl.edu/etd/169 This Dissertation is brought to you for free and open access by Washington University Open Scholarship. It has been accepted for inclusion in All Theses and Dissertations (ETDs) by an authorized administrator of Washington University Open Scholarship. For more information, please contact [email protected]. WASHINGTON UNIVERSITY IN ST. LOUIS School of Engineering and Applied Science Department of Computer Science and Engineering Thesis Examination Committee: Jeremy Buhler, Chair Michael Brent Ron Cytron Mark Franklin Robert Morley Bill Smart David Taylor PARALLELIZATION OF DYNAMIC PROGRAMMING RECURRENCES IN COMPUTATIONAL BIOLOGY by Arpith Chacko Jacob A dissertation presented to the Graduate School of Arts and Sciences of Washington University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY December 2010 Saint Louis, Missouri copyright by Arpith Chacko Jacob 2010 ABSTRACT OF THE THESIS Parallelization of dynamic programming recurrences in computational biology by Arpith Chacko Jacob Doctor of Philosophy in Computer Science Washington University in St. Louis, 2010 Research Advisor: Professor Jeremy Buhler The rapid growth of biosequence databases over the last decade has led to a per- formance bottleneck in the applications analyzing them. In particular, over the last five years DNA sequencing capacity of next-generation sequencers has been doubling every six months as costs have plummeted. The data produced by these sequencers is overwhelming traditional compute systems. We believe that in the future com- pute performance, not sequencing, will become the bottleneck in advancing genome science. In this work, we investigate novel computing platforms to accelerate dynamic pro- gramming algorithms, which are popular in bioinformatics workloads. We study algorithm-specific hardware architectures that exploit fine-grained parallelism in dy- namic programming kernels using field-programmable gate arrays (FPGAs). We ad- vocate a high-level synthesis approach, using the recurrence equation abstraction to represent dynamic programming and polyhedral analysis to exploit parallelism. ii We suggest a novel technique within the polyhedral model to optimize for throughput by pipelining independent computations on an array. This design technique improves on the state of the art, which builds latency-optimal arrays. We also suggest a method to dynamically switch between a family of designs using FPGA reconfigura- tion to achieve a significant performance boost. We have used polyhedral methods to parallelize the Nussinov RNA folding algorithm to build a family of accelerators that can trade resources for parallelism and are between 15-130x faster than a modern dual core CPU implementation. A Zuker RNA folding accelerator we built on a single workstation with four Xilinx Virtex 4 FPGAs outperforms 198 3 GHz Intel Core 2 Duo processors. Furthermore, our design running on a single FPGA is an order of magnitude faster than competing implementations on similar-generation FPGAs and graphics processors. Our work is a step toward the goal of automated synthesis of hardware accelerators for dynamic programming algorithms. iii Acknowledgments I am deeply indebted to my advisor Dr. Jeremy Buhler, who has patiently guided me through my entire graduate career. I am thankful for the hours of discussions, all the comments and suggestions, and for helping me successfully reach this stage in my career. I would like to thank Dr. Buhler for taking a chance on me all these years ago, and for his continued support through the \known unknowns" and the \unknown unknowns." Dr. Buhler has spent countless hours reviewing and perfecting my work. I am eternally grateful for your support. I would like to thank Dr. Roger Chamberlain with whom I have also worked closely during my entire graduate career. I am extremely grateful for his confidence in me and the many hours spent helping me improve my work. Dr. Buhler and Dr. Chamberlain have always afforded me great independence, and even played along with some of my crazier ideas. You have taught me how to do research and I have thoroughly enjoyed working with both of you. I am extremely grateful for financial support provided by an NIH award R42 HG003225 and an ACM/IEEE-CS HPC Ph.D. fellowship in 2007. This support allowed me to focus entirely on my research. I would like to thank members of my committee: Dr. Mark Franklin, Dr. Ron Cytron, Dr. Michael Brent, Dr. Bill Smart, Dr. Robert Morley, and Dr. David Taylor. You have asked the hard questions and forced me to consider my research in the larger context. In particular, I would like to thank Dr. Franklin, who showed me the importance of communicating research results through the (always merciless, sometimes frustrating) culling of technical detail in presentations. Thank you all for sacrificing your valuable time for me. I would like to thank members past and present of the High Performance Computa- tional Biology group at Washington University. Special thanks to Joseph Lancaster with whom I have worked for the last six years. I am especially grateful for your pa- tience in the early years as I came to grips with the world of hardware programming|I can only imagine how frustrating I might have been! I would also like to thank Praveen Krishnamurthy, Eric Tyson, Brandon Harris, Peng Li, Lin Ma, Rahav Dor, iv and Jwalant Ahir for the pleasant work environment you created. I would also like to thank Hongtao Sun and Stephen Cole for many enjoyable interactions. I spent a summer doing an internship with Dr. Maya Gokhale at the Lawrence Livermore National Laboratory. I was able to learn a great deal from her perspective on reconfigurable computing and she has helped my career a great deal. Exegy Inc., provided the FPGA system we used in this dissertation. Their (relatively easy to use) platform and gracious support was critical in achieving the results in this dissertation. In particular, I would like to thank Mr. Berkely Shands who has patiently answered all my questions on using the system. I would like to acknowledge the enormous support of the staff members of the com- puter science department. Your efficiency in dealing with administrative concerns and the congenial environment in the department is much appreciated. I would like to thank members of International Friends who have provided community during the last six years. Finally, I would like to thank my Father and Mother who along with my brother have been unceasing in their support and encouragement throughout my life. I dedicate this work to my family. Arpith Chacko Jacob Washington University in Saint Louis December 2010 v Unless the LORD builds the house, its builders labor in vain. Unless the LORD watches over the city, the watchmen stand guard in vain. In vain you rise early and stay up late, toiling for food to eat| for He grants sleep to those He loves. Psalm 127:1-2 He named it Ebenezer, saying, \Thus far has the LORD helped us." 1 Samuel 7:12b vi Contents Abstract :::::::::::::::::::::::::::::::::::::: ii Acknowledgments :::::::::::::::::::::::::::::::: iv List of Tables ::::::::::::::::::::::::::::::::::: x List of Figures :::::::::::::::::::::::::::::::::: xii 1 Introduction :::::::::::::::::::::::::::::::::: 1 1.1 Dynamic Programming in Computational Biology...........2 1.2 A Bottleneck in Computational Biology Analysis............5 1.3 Rise of Non-traditional Computing Architectures........... 12 1.4 Challenges of Development in Hardware Languages.......... 17 1.5 A New Approach to an Old Problem.................. 19 1.6 Dissertation Overview.......................... 20 1.6.1 Brief Background......................... 21 1.6.2 Summary of our Approach.................... 22 1.6.3 Research Questions........................ 26 1.6.4 Contributions........................... 27 1.6.5 Outline.............................. 30 2 Related Work ::::::::::::::::::::::::::::::::: 32 2.1 High-Level Code-Generation Tools for Computational Biology.... 34 2.2 High-Level Hardware Synthesis Tools for Computational Biology... 39 2.3 High-Level Synthesis Tools for Recurrence Equations......... 40 3 Background: Parallelization of Dynamic Programming ::::::: 43 3.1 Dynamic Programming.......................... 43 3.2 Systolic Arrays.............................. 44 3.2.1 History of Systolic Arrays.................... 45 3.3 Systems of Recurrence Equations.................... 46 3.4 Dynamic Programming as Systems of Affine Recurrence Equations.. 48 3.4.1 Classification of Dynamic Programming............ 49 3.5 Systolic Arrays as Systems of Uniform Recurrence Equations..... 50 3.6 Parallelization of Uniform Recurrences................. 52 3.6.1 Preliminaries of the Polyhedral Representation......... 53 3.6.2 Array Mappings for Recurrence Equations........... 54 vii 3.6.3 Latency-Space Optimal Array Mappings............ 56 3.6.4 Localization............................ 59 3.7 Summary................................. 63 4 Accelerating the Nussinov RNA Folding Recurrence :::::::: 64 4.1