Doctoral Dissertation Processing Multimedia Workloads on Heterogeneous Multicore Architectures H˚akon Kvale Stensland February 2015 Submitted to the Faculty of Mathematics and Natural Sciences at the University of Oslo in partial fulfilment of the requirements for the degree of Philosophiae Doctor © Håkon Kvale Stensland, 2015 Series of dissertations submitted to the Faculty of Mathematics and Natural Sciences, University of Oslo No. 1601 ISSN 1501-7710 All rights reserved. No part of this publication may be reproduced or transmitted, in any form or by any means, without permission. Cover: Hanne Baadsgaard Utigard. Printed in Norway: AIT Oslo AS. Produced in co-operation with Akademika Publishing. The thesis is produced by Akademika Publishing merely in connection with the thesis defence. Kindly direct all inquiries regarding the thesis to the copyright holder or the unit which grants the doctorate. Abstract Processor architectures have been evolving quickly since the introduction of the central processing unit. For a very long time, one of the important means of increasing per- formance was to increase the clock frequency. However, in the last decade, processor manufacturers have hit the so-called power wall, with high heat dissipation. To overcome this problem, processors were designed with reduced clock frequencies but with multiple cores and, later, heterogeneous processing elements. This shift introduced a new challenge for programmers: Legacy applications, written without parallelization in mind, gain no benefits from moving to multicore and heterogeneous architectures. Another challenge for the programmers is that heterogeneous architecture designs are very different with respect to caches, memory types, execution unit organization, and so forth and, in the worst case, a programmer must completely rewrite the application to obtain the best performance on the new architecture. Multimedia workloads, such as video encoding, are often time sensitive and interac- tive. These workloads differ from traditional batch processing workloads with no real-time requirements. This work investigates how to use modern heterogeneous architectures ef- ficiently to process multimedia workloads. To do so, we investigate both simple and complex workloads on multiple architectures to learn about the properties of these archi- tectures. When programing multimedia workloads, it is very important to know how the algorithms perform on the target architecture. In addition, achieving high performance on heterogeneous architectures is not a trivial task, often requiring detailed knowledge about the architecture. We therefore evaluate several optimizations so we can learn how best to write programs for these architectures and avoid potential pitfalls. We later use the knowledge gained to propose a framework design and language called Parallel Pro- cessing Graph (P2G). The P2G framework is designed for multimedia workloads and supports heterogeneous architectures. To demonstrate the feasibility of the framework, we construct a proof-of-concept implementation. Two simple workloads show that we can express multimedia workloads in the system. We also demonstrate the scalability of the designed solution. iii iv Acknowledgements Working with the PhD has been a long journey, at times, it has been both frustrating and stressful, but it has mostly been lots of fun. I would like to thank my supervisors, Professor Carsten Griwodz and Professor P˚alHalvorsen for interesting discussions, their deep insights and valuable feedback over the years. I would also like to thank my colleagues H˚avard Espeland and Paul Beskow who I have shared office with for good collaboration and inspiring discussions on the topic of processing multimedia workloads. During the work with this thesis I have supervised several master students. I would like to thank all of them, the discussions have been very inspiring, and have helped me with this thesis. The work environment at Simula Research Laboratory and the Media-department has also been excellent. Here, I would like to thank Andreas Petlund, Kristian Evensen, Ragnhild Eg, Preben Olsen and Vamsidhar Gaddam for making Simula a great place to be. Finally I would like to thank my family, friends and especially my wife Marianne, for being patient and always supporting me no matter what I decide to do. v vi Contents I Overview 1 1 Introduction 3 1.1 Background and Motivation . .3 1.1.1 Heterogeneous Architectures . .4 1.1.2 Multimedia Workloads . .5 1.2 Problem Statement . .6 1.3 Limitations . .7 1.4 Research Method . .7 1.5 Main Contributions . .8 1.6 Outline . 10 2 Heterogeneous Computing 11 2.1 Hardware Architectures . 11 2.1.1 Intel x86 Processor Architecture . 11 2.1.2 Intel IXP Network Processor . 16 2.1.3 Nvidia Graphics Processing Units . 18 2.1.4 STI Cell Broadband Engine . 22 2.1.5 Other Hardware Architectures . 24 2.1.6 Summary . 26 2.2 Hardware Abstractions and Programming Models . 27 2.2.1 SMT . 27 2.2.2 SIMD . 28 2.2.3 SIMT . 29 2.2.4 Summary . 30 2.3 Summary . 30 3 Using Heterogeneous Architectures for Simple Tasks 33 3.1 Intel IXP Network Processor . 33 3.1.1 Case Study: Network Protocol Translation . 34 3.1.2 Implications . 38 3.2 x86 Processor Architecture . 39 3.2.1 Case study: Motion JPEG Encoding . 39 3.2.2 Case Study: Multi-Rate Video Encoding with VP8 . 42 3.2.3 Case Study: Parallel Execution of a Game Server . 46 3.2.4 Implications . 52 3.3 Graphics Processing Units . 53 vii 3.3.1 Case Study: GPU Memory Spaces and Access Patterns . 53 3.3.2 Case Study: Host{Device Communication Optimization . 56 3.3.3 Case Study: Cheat Detection . 58 3.3.4 Case Study: MJPEG Encoding . 64 3.3.5 Implications . 68 3.4 Cell Broadband Engine . 69 3.4.1 Case Study: MJPEG Encoding . 69 3.4.2 Implications . 73 3.5 Architecture Comparison . 74 3.6 Summary . 75 4 Using Heterogeneous Architectures for Complex Workloads 77 4.1 Bagadus Sports Analysis System . 77 4.1.1 Bagadus: The Basic Idea . 78 4.1.2 Video Subsystem . 80 4.2 The Real-Time Bagadus Video Pipeline . 83 4.2.1 Performance Analysis . 90 4.2.2 Discussion . 92 4.3 Summary . 94 5 The P2G Framework and the Future 97 5.1 Summary of Challenges . 97 5.2 Design Ideas for a New Processing Framework . 98 5.3 Existing Processing Frameworks . 98 5.4 The P2G Framework . 99 5.4.1 Architecture . 102 5.4.2 Programming Model . 103 5.4.3 Prototype . 108 5.4.4 Workloads . 109 5.4.5 Evaluation . 110 5.4.6 Summary . 113 5.5 The Future . 113 6 Papers and Author's Contributions 115 6.1 Overview of Research Papers . 115 6.2 Paper I: Transparent Protocol Translation for Streaming . 115 6.3 Paper II: Evaluation of Multi-Core Scheduling Mechanisms for Heteroge- neous Processing Architectures . 116 6.4 Paper III: Tips, Tricks and Troubles: Optimizing for Cell and GPU . 117 6.5 Paper IV: Cheat Detection Processing: A GPU versus CPU Comparison . 118 6.6 Paper V: Reducing Processing Demands for Multi-Rate Video Encoding: Implementation and Evaluation . 119 6.7 Paper VI: LEARS: A Lockless, Relaxed-Atomicity State Model for Parallel Execution of a Game Server Partition . 120 6.8 Paper VII: P2G: A Framework for Distributed Real-Time Processing of Multimedia Data . 120 6.9 Paper VIII: Bagadus: An Integrated Real-Time System for Soccer Analytics122 viii 6.10 Paper IX: Processing Panorama Video in Real-Time . 122 6.11 Supervised Master's Students . 123 6.12 Other Publications . 127 7 Conclusion 129 7.1 Summary . 129 7.2 Concluding Remarks . 130 7.3 Future Work . 131 II Research Papers 145 Paper I: Transparent Protocol Translation for Streaming 147 Paper II: Evaluation of Multi-Core Scheduling Mechanisms for Hetero- geneous Processing Architectures 153 Paper III: Tips, Tricks and Troubles: Optimizing for Cell and GPU 161 Paper IV: Cheat Detection Processing: A GPU versus CPU Compari- son 169 Paper V: Reducing Processing Demands for Multi-Rate Video Encod- ing: Implementation and Evaluation 177 Paper VI: LEARS: A Lockless, Relaxed-Atomicity State Model for Par- allel Execution of a Game Server Partition 199 Paper VII: P2G: A Framework for Distributed Real-Time Processing of Multimedia Data 209 Paper VIII: Bagadus: An Integrated Real-Time System for Soccer An- alytics 223 Paper IX: Processing Panorama Video in Real-Time 247 Posters and live demonstrations 269 Other research papers 273 A BNF Grammar of the P2G Kernel Language 277 ix x List of Figures 1.1 The real-time panorama video stitching pipeline in the Bagadus soccer analysis system [114]. .6 2.1 Comparison of SMP architectures. 13 2.2 Intel Haswell architecture diagram. 14 2.3 Intel Xeon Phi MIC architecture. 16 2.4 Intel IXP2400 architecture diagram. 17 2.5 Comparison of transistor usage on a CPU and on a GPU. 18 2.6 A pre-DirectX 10 graphics pipeline [43], with a programmable vertex pro- cessor and fragment processor. 19 2.7 Nvidia GK110 SMX architecture [90], slightly modified. 21 2.8 The Kepler memory hierarchy. 22 2.9 CBE architecture. 23 2.10 Overview of an SPE. 24 2.11 A superscalar processor design with and without SMT. 28 2.12 SIMD programming model. 28 2.13 Nvidia CUDA programming model. 29 3.1 Overview of the streaming scenario. 35 3.2 Packet flow on the Intel IXP2400. 36 3.3 Achieved bandwidth, varying drop rate and link latency with 1% server{ proxy loss. ..