DOCSLIB.ORG

Da Li July 2016

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Da Li July 2016 FACILITATING EMERGING APPLICATIONS ON MANY-CORE PROCESSORS ________________________________________________ A Dissertation Presented to the Faculty of the Graduate School at the University of Missouri-Columbia ___________________________________________________ In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy ______________________________________________ by DA LI Dr. Michela Becchi, Supervisor JULY 2016 The undersigned, appointed by the dean of the Graduate School, have examined the dissertation entitled FACILITATING EMERGING APPLICATIONS ON MANY-CORE PROCESSORS presented by Da Li, a candidate for the degree of doctor of philosophy and hereby certify that, in their opinion, it is worthy of acceptance. Dr. Michela Becchi Dr. Tony Han Dr. Zhihai He Dr. William Harrison Dr. Jason Xu ACKNOWLEDGEMENTS First and foremost, I would like to deeply thank my adviser, Dr. Michela Becchi, for her valuable guidance and advice and for her vast reserve of patience and knowledge. I am fortunate and grateful to have worked with her during my studies. I also appreciate her generosity and flexibility in offering me opportunities to interact with other scientists and explore various projects through internships from industries. I would like also to express my sincere gratitude to Dr. Tony Han, Dr. Zhihai He, Dr. William Harrison and Dr. Jason Xu for their teaching, mentoring and assistance. I want to thank Dr. Ziliang Zong and Xinbo from Texas State University for their expertise, feedback and overall support. I would like to thank all the current and previous members of the Networking and Parallel System laboratory, especially Kittisak, Huan, Ruidong and Henry who helped me with my research. Finally, I would like to acknowledge my family and friends for their encouragement and devotion. I am very grateful to have a supportive family who puts my education as the first priority. I would express my deepest appreciation and thanks to my wife, Xin Tong, who is better at science than me, for her love and support. This work has been supported by National Science Foundation awards CNS-1216756, CCF-1452454, CNS-1305359, and by gifts and equipment donations from NEC Labs America and Nvidia Corporation. ii Table of Contents ACKNOWLEDGEMENTS .......................................................................................... ii LIST OF FIGURES .................................................................................................... vii LIST OF TABLES ....................................................................................................... xi ABSTRACT ................................................................................................................ xii Chapter 1 Introduction .................................................................................................. 1 1.1 Introduction ......................................................................................................... 1 1.2 Contributions ...................................................................................................... 4 1.3 Dissertation Organization ................................................................................... 5 Chapter 2 Background .................................................................................................. 6 2.1 GPU Architecture and Programming Model ...................................................... 6 2.2 Needleman-Wunsch Algorithm .......................................................................... 9 2.3 Irregular applications ........................................................................................ 11 2.4 Convolutional neural networks ......................................................................... 12 Chapter 3 Bioinformatics Applications on CPU-GPU Clusters ................................. 15 3.1 Motivation ......................................................................................................... 15 3.2 Related Work .................................................................................................... 16 3.2.1 Early Works on Sequence Alignments ...................................................... 16 3.2.2 Acceleration of Bioinformatics Application on GPUs .............................. 17 iii 3.2.3 Sequence Alignments on GPUs ................................................................. 18 3.3.4 Rodinia-NW: Needleman-Wunsch on GPU .............................................. 19 3.3 Design of GPU-workers .................................................................................... 21 3.3.1 TiledDScan-mNW: Multiple alignments with tiling ................................. 21 3.3.2 DScan-mNW: Single-kernel diagonal scan ............................................... 22 3.3.3 RScan-mNW: Row scan via single CUDA core ........................................ 24 3.4 Experimental Evaluation ................................................................................... 28 3.4.1 Experimental setup ..................................................................................... 28 3.4.2 Performance on single GPU ....................................................................... 28 3.5 Other Applications with Similar Computation Pattern ..................................... 35 Chapter 4 Irregular Applications on Many-Core Processors ...................................... 40 4.1 Related Work .................................................................................................... 41 4.2 Unified Programming Interface ........................................................................ 44 4.2.1 General Design & Methodology ................................................................ 46 4.2.2 Programming API ...................................................................................... 47 4.2.3 Case Study ................................................................................................. 50 4.2.4 Compiler Design ........................................................................................ 54 4.2.5 Runtime Library Design ............................................................................ 58 4.2.6 Performance Evaluation ............................................................................. 61 4.3 Adaptive Computing ......................................................................................... 68 iv 4.3.1 Motivation .................................................................................................. 69 4.3.2 Exploration Space ...................................................................................... 74 4.3.3 Implementation .......................................................................................... 81 4.3.4 Adaptive Runtime ...................................................................................... 86 4.3.5 Experimental Evaluation ............................................................................ 91 4.4 Parallelization Templates .................................................................................. 99 4.4.1 Motivation ................................................................................................ 100 4.4.2 Irregular Nested Loop .............................................................................. 101 4.4.3 Experimental Evaluation .......................................................................... 105 4.5 Workload Consolidation ................................................................................. 113 4.5.1 Dynamic Parallelism ................................................................................ 114 4.5.2 Application Characterization ................................................................... 115 4.5.3 Motivation ................................................................................................ 116 4.5.4 Methodology ............................................................................................ 121 4.6.4 Experimental Evaluation .......................................................................... 132 Chapter 5 Deep Neural Network on CPU-GPU ....................................................... 141 5.1 Related Work .................................................................................................. 141 5.2 Energy Efficiency ........................................................................................... 142 5.2.1 Motivation ................................................................................................ 142 5.2.2 Methodology ............................................................................................ 144 v 5.2.3 Overall Results on CPU & GPU .............................................................. 146 5.2.4 Effect of NN and Batch Size Configuration ............................................ 151 5.2.5 Effect of Hardware Settings ..................................................................... 153 5.3 CNN on CPU-GPU heterogeneous architecture ............................................. 158 5.3.1 Motivation ................................................................................................ 158 5.3.2 HetNet ...................................................................................................... 159 5.3.3 HybNet ..................................................................................................... 161 5.3.4 Experimental Evaluation .......................................................................... 162
Load more

Recommended publications
  • Download The
    LEADING THE FUTURE OF TECHNOLOGY 2016 ANNUAL REPORT TABLE OF CONTENTS 1 MESSAGE FROM THE IEEE PRESIDENT AND THE EXECUTIVE DIRECTOR 3 LEADING THE FUTURE OF TECHNOLOGY 5 GROWING GLOBAL AND INDUSTRY PARTNERSHIPS 11 ADVANCING TECHNOLOGY 17 INCREASING AWARENESS 23 AWARDING EXCELLENCE 29 EXPANSION AND OUTREACH 33 ELEVATING ENGAGEMENT 37 MESSAGE FROM THE TREASURER AND REPORT OF INDEPENDENT CERTIFIED PUBLIC ACCOUNTANTS 39 CONSOLIDATED FINANCIAL STATEMENTS Barry L. Shoop 2016 IEEE President and CEO IEEE Xplore® Digital Library to enable personalized importantly, we must be willing to rise again, learn experiences based on second-generation analytics. from our experiences, and advance. As our members drive ever-faster technological revolutions, each of us MESSAGE FROM As IEEE’s membership continues to grow must play a role in guaranteeing that our professional internationally, we have expanded our global presence society remains relevant, that it is as innovative as our THE IEEE PRESIDENT AND and engagement by opening offices in key geographic members are, and that it continues to evolve to meet locations around the world. In 2016, IEEE opened a the challenges of the ever-changing world around us. second office in China, due to growth in the country THE EXECUTIVE DIRECTOR and to better support engineers in Shenzhen, China’s From Big Data and Cloud Computing to Smart Grid, Silicon Valley. We expanded our office in Bangalore, Cybersecurity and our Brain Initiative, IEEE members India, and are preparing for the opening of a new IEEE are working across varied disciplines, pursuing Technology continues to be a transformative power We continue to make great strides in our efforts to office in Vienna, Austria.
    [Show full text]
  • Diana Marculescu
    DIANA MARCULESCU Carnegie Mellon University Phone: (412) 268-1167 Dept. of ECE, 5000 Forbes Ave E-mail: [email protected] Pittsburgh, PA 15213-3890 URL: http://users.ece.cmu.edu/~dianam David Edward Schramm Professor of Electrical and Computer Engineering Founding Director, Center for Faculty Success, College of Engineering RESEARCH INTERESTS Sustainability- and energy-aware computer system modeling and optimization Energy-aware machine learning; Fast and accurate power modeling, estimation, optimization for multi- core systems; Modeling and optimization for sustainability in computing and renewable energy Reliability- and variability-aware system design Modeling, analysis, and optimization of soft-error rate in large digital circuits; Microarchitecture to system level design variability modeling and mitigation; 3D integration and impact of process variations Discrete modeling and analysis of non-silicon networks Logical models and hardware emulation for efficient nonlinear system analysis; Efficient models for computational biology; Electronic textiles and smart fabrics EDUCATION Ph.D. in Computer Engineering, University of Southern California - August 1998 Dissertation: Information-theoretic and Probabilistic Measures for Power Analysis of Digital Circuits Advisor: Prof. Massoud Pedram M.S. in Computer Science (Eng. Dipl.), Polytechnic Institute of Bucharest, Romania - June 1991 Dissertation: Fault-tolerant Database System Design Advisor: Prof. Irina Athanasiu PROFESSIONAL EXPERIENCE David Edward Schramm Professor, Dept. of ECE, Carnegie Mellon University, Oct. 2016 – Present Founding Director, Center for Faculty Success, College of Engineering, Carnegie Mellon University, Nov. 2014 – present Founded and led the only CMU center focused on faculty development and support. Developed and ran programs for faculty recruiting, mentoring, development, and diversity/inclusion awareness for over 600 faculty, staff, and students at CMU and other institutions.
    [Show full text]
  • Arch2030: a Vision of Computer Architecture Research Over
    Arch2030: A Vision of Computer Architecture Research over the Next 15 Years This material is based upon work supported by the National Science Foundation under Grant No. (1136993). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Arch2030: A Vision of Computer Architecture Research over the Next 15 Years Luis Ceze, Mark D. Hill, Thomas F. Wenisch Sponsored by ARCH2030: A VISION OF COMPUTER ARCHITECTURE RESEARCH OVER THE NEXT 15 YEARS Summary .........................................................................................................................................................................1 The Specialization Gap: Democratizing Hardware Design ..........................................................................................2 The Cloud as an Abstraction for Architecture Innovation ..........................................................................................4 Going Vertical ................................................................................................................................................................5 Architectures “Closer to Physics” ................................................................................................................................5 Machine Learning as a Key Workload ..........................................................................................................................6 About this
    [Show full text]
  • Sensible Machine
    Rebooting Computing Summit 4 : Sensible Machine R. Stanley Williams Senior Fellow Hewlett Packard Labs December 10, 2015 US National Grand Challenge in Future Computing: Sensible Machine • US BRAIN Initiative - April 22, 2013 • OSTP RFI: “Nanotechnology-Inspired Grand Challenges for the Next Decade” – June 17 • Submitted a response to RFI entitled “Sensible Machines” – June 24 • Presidential Executive Order: National Strategic Computing Initiative – July 29 • OSTP shortlisted ‘Sensible Machines’, asked to ‘develop a program’ – July 30 • Worked with IEEE Rebooting Computing and ITRS – Big thank you to Erik DeBenedictis, Tom Conte, Dave Mountain and many others! • Review of the Chinese Brain-Inspired Computing Research Program – Oct 15 • Tom Kalil announces Future Computing Grand Challenge at NSCI workshop – Oct. 20 The “Sensible Machine” response to OSTP RFI “The central thesis of this white paper is that although our present understanding of brains is limited, we know enough now to design and build circuits that can accelerate certain computational tasks; and as we learn more about how brains communicate and process information, we will be able to harness that understanding to create a new exponential growth path for computing technology.” Our challenge as a community is now to continuously perform more computation per unit energy rather than manufacture more transistors per unit area. URLs for further information • White House announcement of Future Computing Grand Challenge: https://www.whitehouse.gov/blog/2015/10/15/nanotechnology-inspired-grand-challenge-future-computing
    [Show full text]
  • IEEE Future Directions Newsletter
    ISSUE 22 | December 2019 In this Issue: IEEE Technology Roadmaps Grow IEEE Future Directions at IEEE GLOBECOM 2019 IEEE Future Directions Technical Communities Technology Policy and Ethics Current Activities in our Technical Communities IEEE Future Directions Events IEEE Technology Roadmaps Grow Interest in technology roadmaps is growing. A number of IEEE organizational units are participating in the development of new technology roadmaps or continuing the work of established roadmaps. The start of some recent IEEE roadmap activity began as an effort to continue the work of the International Technology Roadmap for Semiconductors 2.0 (ITRS). The Heterogeneous Integration Roadmap (HIR), sponsored by IEEE Electronics Packaging Society (EPS), SEMI, IEEE Electron Devices Society (EDS), IEEE Photonics Society, and the ASME Electronic & Photonic Packaging Division (EPPD), published the first chapters of its 2019 edition in October, with additional chapters being released on a rolling basis. Another recently published technology roadmap, the International Technology Roadmap on Wide Bandgap Semiconductors (ITRW), has made the first edition of its roadmap free to members of the sponsoring society, IEEE Power Electronics Society (PELS). A third technology roadmap, which began in the IEEE Standards Association Industry Connections program and has found a home in the IEEE Rebooting Computing Initiative, IEEE International Roadmap for Devices and Systems™ (IRDS), published its second edition this year. In order to support IEEE’s technology roadmap activities, the IEEE Roadmaps Strategy and Governance (IRSG) Ad Hoc Committee was formed in May 2018. The committee, led by Chair Rakesh Kumar, consists of a steering committee and a user group. The user group serves as a forum for communication between the representatives of various published and developing IEEE roadmaps, as well as a resource for capturing best practices and guidelines.
    [Show full text]
  • RCS 2 2Nd Rebooting Computing Summit Summary Report
    RCS 2 2nd Rebooting Computing Summit Summary Report The Chaminade Santa Cruz, CA May 14-16, 2014 Prepared By: Alan M. Kadin And the IEEE Rebooting Computing Committee http://rebootingcomputing.ieee.org/ http://rebootingcomputing-ieee.blogspot.com/ June 2014 1 | P a g e Contents Foreword ....................................................................................................................................................... 4 What Is “Rebooting Computing”? ................................................................................................................. 5 RCS 1: Future Vision and Pillars of Computing ............................................................................................. 6 Future Vision of Intelligent Mobile Assistant ............................................................................................ 6 Three Pillars of Future Computing ............................................................................................................ 6 Human/Computer Interface and Applications...................................................................................... 6 Energy Efficiency ................................................................................................................................... 6 Security ................................................................................................................................................. 6 RCS 2: Future Computer Technology – The End of Moore’s Law? .............................................................
    [Show full text]
  • Rebooting Computing Week 2017 Impressions by D. Scott Holmes, SNF Co-Editor Electronics January 28, 2017
    IEEE/CSC & ESAS SUPERCONDUCTIVITY NEWS FORUM (global edition), February 2018. Rebooting Computing Week 2017 Impressions By D. Scott Holmes, SNF Co-editor Electronics January 28, 2017 (HE114, STH625). Rebooting Computing Week (November 6-10, 2017) included three related meetings focusing on the next era of computing. This year superconductor electronics, while still far from central, was noticeably present and actively discussed. Links to videos from the meetings are available here. Nov. 6-7 International Roadmap for Devices and Systems (IRDS) Fall Conference The International Technology Roadmap for Semiconductors (ITRS) projected technology requirements and potential solutions for semiconductors from 2001 to 2014. The ITRS used transistor feature sizes, density, clock rate, and other metrics to roadmap the future of integrated circuits. In 2015, the ITRS committee presented a new roadmap, called ITRS 2.0, for key systems that contain integrated circuits and drive process, design, and integration technologies. Subsequent partnering of ITRS 2.0 with the IEEE Rebooting Computing (IEEE RC) Initiative resulted in the International Roadmap for Devices and Systems (IRDS). The IRDS mission is to “Identify the roadmap of electronic industry from devices to systems and from systems to devices”, which represents a broadening of the scope. For an overview, see Paolo Gargini's presentation given at the November 10 Industry Summit and described below. “Beyond CMOS” is one of the IRDS focus topics and includes technologies other than Complementary Metal-Oxide Semiconductor (CMOS) electronics such as memristors, spintronics, straintronics, and superconductor electronics. As participants in the Beyond CMOS International Focus Team, D. Scott Holmes and Erik DeBenedictis started a Cryogenic Electronics group, thus initiating superconductor electronics as an activity within the IRDS.
    [Show full text]
  • Software > Blockchain > Augmented
    > Software > Blockchain > Augmented and Virtual Reality > Cyber-Physical Systems SEPTEMBER 2019 www.computer.org Keep Your Career Options Open Upload Your Resume Today! Whether your enjoy your current position or you are ready for change, the IEEE Computer Society Jobs Board is a valuable resource tool. Take advantage of these special resources for job seekers: JOB ALERTS TEMPLATES CAREER RESUMES VIEWED ADVICE BY TOP EMPLOYERS No matter your career WEBINARS level, the IEEE Computer Society Jobs Board keeps you connected to workplace trends and exciting new career prospects. www.computer.org/jobs IEEE COMPUTER SOCIETY computer.org • +1 714 821 8380 STAFF Editor Publications Portfolio Managers Cathy Martin Carrie Clark, Kimberly Sperka Publications Operations Project Specialist Publisher Christine Anthony Robin Baldwin Publications Marketing Project Specialist Meghan O’Dell Senior Advertising Coordinator Debbie Sims Production & Design Carmen Flores-Garvey Circulation: ComputingEdge (ISSN 2469-7087) is published monthly by the IEEE Computer Society. IEEE Headquarters, Three Park Avenue, 17th Floor, New York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720; voice +1 714 821 8380; fax +1 714 821 4010; IEEE Computer Society Headquarters, 2001 L Street NW, Suite 700, Washington, DC 20036. Postmaster: Send address changes to ComputingEdge-IEEE Membership Processing Dept., 445 Hoes Lane, Piscataway, NJ 08855. Periodicals Postage Paid at New York, New York, and at additional mailing offices. Printed in USA. Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in ComputingEdge does not necessarily constitute endorsement by the IEEE or the Computer Society.
    [Show full text]
  • October Edition
    [This page intentionally left blank] 2 | P a g e CONTENTS WELCOMING MESSAGES ......................................................................................................................................... 5 MESSAGE FROM IEEE REGION 10 DIRECTOR .................................................................................................................................. 5 MESSAGE FROM IEEE REGION 10 NEWSLETTER COORDINATOR .......................................................................................................... 6 1. 2020 IEEE REGION 10 AWARDS RECIPIENTS .............................................................................................................. 8 INDIVIDUAL AWARDS ........................................................................................................................................................... 8 ORGANIZATIONAL UNIT AWARDS ................................................................................................................................................ 12 2. R10 PERSONALITIES OF THE MONTH ...................................................................................................................... 17 R10 PERSONALITY OF THE MONTH – SUDEENDRA THIRTHA KOUSHIK ................................................................................................ 17 R10 WIE PERSONALITY OF THE MONTH – DIVYA M.G. .................................................................................................................. 19 R10 YP PERSONALITY OF THE MONTH
    [Show full text]
  • IEEE Rebooting Computing Initiative & International Roadmap of Devices
    IEEE Rebooting Computing Initiative & International Roadmap of Devices and Systems Tom Conte, 2015 IEEE Computer Society President, Co-Chair, IEEE Rebooting Computing Initiative, Schools of CS & ECE, Georgia Institute of Technology What the problem is Transistors are getting smaller but not faster – From a microarchitect’s perspective: 10nm isn’t any better than 14nm, which was only marginally better than 22nm Moore's Law for 2D really ends in 2021 Single thread exponential performance scaling ended in 2005 –Multicore didn’t continue scaling 2 How we got here 1945: Von Neumann’s report describing computer arch. 1965: Software industry begins (IBM 360), Moore #1 1975: Moore’s Law update; Dennard’s geo. scaling rule 1985: “Killer micros”: HPC, general-purpose hitch a ride on Moore’s law 1995: Slowdown in CMOS logic: superscalar era begins 3 Superscalar to the rescue Processor performance Moore’s law 4 Source: Sanjay Patel, UIUC (used with permission) How we got here, part 2 1945: Von Neumann’s report describing computer arch. 1965: Software industry begins (IBM 360) 1975: Moore’s Law; Dennard’s geometric scaling rule 1985: “Killer micros”: HPC, general-purpose hitch a ride on Moore’s law 1995: Slowdown in CMOS logic: superscalar era begins 2005: The Power Wall: Single thread exponential scaling ends (Intel Prescott) … 5 What did we do next? Kept on getting more transistors, but couldn’t build a bigger superscalar: The Multicore era began Superscalar Superscalar Superscalar Superscalar Two problems:Core #1 Core #2 Core #3 Core #4 SequentialL2 cache programsL2 cache dominateL2 cache L2 cachethe industry Network On Chip Not all problems solvable via parallel algorithms Shared L3 cache 7 6/19/16 How we got here, part 3 1945: Von Neumann’s report describing computer arch.
    [Show full text]
  • (Bud) Lawson Written
    Experiences and Reflections of a Computer and Systems Engineering Pioneer Written by: Harold (Bud) Lawson Written: February, 2018 CHM Reference number: X8505.2018 © 2018 Computer History Museum Experiences and Reflections of a Computer and Systems Engineering Pioneer Introduction I have divided this personal history into three phases; namely computer industry, computer- based systems and complex systems. While the years indicated are approximate and there is significant overlap, it provides a structure to conveying the essence of my career. I have worked with many talented people during my career and we have learned from each other. I thank them all and identify many of them. Further, I take the opportunity to provide some reflections that may have occurred during the experience or in several cases later in my career. I also provide a summary of what I consider to be my most important publications. Phase 1 (1959 to 1974) – Computer Industry The period involved is from June 1959 to sometime in early 1974. During this time, I was employed first by Remington-Rand Univac, then by International Business Machines, Inc, later by Standard Computer Corporation (Costa Mesa, California) and finally as a consultant to Datasaab (Linköping, Sweden). Also during this period starting in 1967, I began a parallel academic career, first as Associate Professor and later as Professor at the Polytechnic Institute of Brooklyn (Now the Polytechnic Institute of New York) as well as guest professorships at the University of California at Irvine and Linköpings University in Sweden. This was a very exciting period in the computer industry and it turned out to be a very productive period of my life providing a breadth of experiences and knowledge.
    [Show full text]
  • Advances in Celebration
    2021 Issue 2 // Volume 117 THE BRIDGE The Magazine of IEEE-Eta Kappa Nu Modeling of organic semiconductor conduction parameters with reference to inorganic semiconductors Modeling of organic semiconductor metallic contact and optoelectronic parameters with reference to inorganic semiconductors A review of Schottky junction solar cells Register now for our Graduation Advances in Celebration 2020 Outstanding Chapter Award Winners SOLAR CELL Celebrating 50 Years with IEEE-HKN IEEE-HKN AWARDS PROGRAM THE BRIDGE As the Honor Society of IEEE, IEEE-Eta Kappa Nu provides opportunities The Magazine of IEEE-Eta Kappa Nu to promote and encourage outstanding students, educators and members. Visit our new website to view the awards programs, awards committees, 2021 ISSUE 2 | ADVANCES IN SOLAR CELL DESIGN list of past winners, nomination criteria and deadlines. 4 5 Editors-in-Chief Guest Editor ALTON B. ZERBY AND CARL T. KOERNER OUTSTANDING CHAPTER AWARD (OCA) Dr. Sahra Sedigh Sarvestani Dr. Amirhossein Ghods OUTSTANDING STUDENT AWARD (OSA) Recognizes chapters for excellence in activities and and Dr. Steve E. Watkins Presented annually to a senior who has proven service at the department, university and community outstanding scholastic excellence and high moral levels. The award is based on the content contained character, and has demonstrated exemplary service in their Annual Chapter Report for the preceding Features to classmates, university, community, and country. academic year. 14 20 27 (Deadline: 30 June) (Deadline: 31 July) Modeling of organic
    [Show full text]