UNIVERSIDADE DE LISBOA INSTITUTO SUPERIOR TÉCNICO Speculative Parallelization of Object-Oriented Applications Ivo Filipe Silva Daniel Anjo Supervisor: Doctor João Manuel Pinheiro Cachopo Thesis approved in public session to obtain the PhD Degree in Information Systems and Computer Engineering Jury final classification: Pass with Merit Jury Chairperson: Chairman of the IST Scientific Board Members of the Committee: Doctor Thomas Richard Gross Doctor Bruno Miguel Brás Cabral Doctor Luís Manuel Antunes Veiga Doctor João Manuel Pinheiro Cachopo 2015 UNIVERSIDADE DE LISBOA INSTITUTO SUPERIOR TÉCNICO Speculative Parallelization of Object-Oriented Applications Ivo Filipe Silva Daniel Anjo Supervisor: Doctor João Manuel Pinheiro Cachopo Thesis approved in public session to obtain the PhD Degree in Information Systems and Computer Engineering Jury final classification: Pass with Merit Jury Chairperson: Chairman of the IST Scientific Board Members of the Committee: Doctor Thomas Richard Gross, Full Professor, ETH Zürich, Switzerland Doctor Bruno Miguel Brás Cabral, Professor Auxiliar da Faculdade de Ciências e Tecnologia, Universidade de Coimbra Doctor Luís Manuel Antunes Veiga, Professor Auxiliar do Instituto Superior Técnico, Universidade de Lisboa Doctor João Manuel Pinheiro Cachopo, Professor Auxiliar do Instituto Superior Técnico, Universidade de Lisboa Funding Institutions Fundação para a Ciência e a Tecnologia 2015 Resumo Apesar da ubiquidade dos processadores multicore, e ainda que novas aplicações informáti- cas estejam a ser desenvolvidas para tirar partido das suas capacidades de processamento paralelo, muitas das aplicações mais comuns são ainda sequenciais. Como consequência, muitos dos designers de processadores, ainda que activamente publicitando e recomendando a utilização de programação paralela, dedicam ainda muitos dos seus recursos à obtenção de melhores velocidades de processamento para código sequencial. Esta persistência mostra que o desempenho na execução sequencial é ainda um importante desafio em sistemas modernos. Nesta dissertação, proponho a criação de uma framework para paralelização automática com o objectivo de acelerar a execução — em multicores — de aplicações sequenciais orientadas a objectos que se executem em plataformas managed. A natureza fortemente dinâmica e irregular deste tipo de aplicações e das plataformas em que assentam derrota muitas das técnicas tradicionais de paralelização, requerendo uma aproximação diferente para se conseguir ter sucesso. Para permitir a execução especulativa de código com baixo impacto na performance, pro- ponho um novo modelo de Memória Transaccional por Software especializado para paralelização especulativa. Para permitir à framework proposta a utilização de Especulação ao Nível do Método, proponho um novo algoritmo para transformar chamadas a métodos em pontos de início de especulação que retornam futuros. Examino ainda os problemas com filas de exe- cução e mapeamento de tarefas para threads, e introduzo um conjunto de novas técnicas que permitem reduzir os tempos de espera e que aumentam a eficiência da utilização de recursos da máquina. Finalmente, descrevo a implementação das técnicas propostas na criação de um sistema coerente — a framework JaSPEx-MLS — que proporciona uma solução integrada para a aceleração da execução de aplicações Java/JVM sequenciais em hardware comum, sem necessidade de alterações prévias às mesmas, e com performance capaz de superar JVMs comerciais. v Abstract Despite the ubiquity of multicores, and although new applications are being developed to take advantage of multicore systems, many commonly-used applications are still sequential. As a consequence, many chip designers, while actively advocating parallel programming, still dedicate many of their resources to providing faster sequential code execution. This shows that sequential code execution performance is still an important challenge for modern computing systems. In this dissertation, I propose the creation of an automatic parallelization framework that boosts the execution of sequential object-oriented applications running atop managed runtimes. The highly dynamic and irregular nature of these applications and their supporting runtimes requires a different approach to parallelization that defeats many traditional parallelization techniques. To support low-overhead speculative code execution, I propose a novel Software Transac- tional Memory model that is specialized for speculative parallelization. To allow the proposed framework to use Method-Level Speculation as a task identification strategy, I propose a new algorithm that allows the safe transformation of method call sites into spawn points that return futures. Then, I examine the issues of runtime task queuing and thread mapping, and introduce several new techniques to reduce stalling and increase resource usage efficiency. Finally, I describe the combination of the proposed techniques into a coherent system — the JaSPEx- MLS framework — that provides an end-to-end solution for boosting sequential Java/JVM applications on commodity hardware, without requiring applications to be previously modified, and yielding performance capable of surpassing production Java runtimes. vii Palavras-chave Paralelização Automática Especulação ao Nível do Método Aplicações Orientadas a Objectos Memória Transaccional em Software Máquina Virtual Java OpenJDK HotSpot Arquitecturas Multicore Keywords Automatic Parallelization Method-Level Speculation Object-Oriented Applications Software Transactional Memory OpenJDK HotSpot JVM Multicore Architectures ix Acknowledgments How much can I change on this journey? — A question that I didn’t consciously ask myself when I decided to embark on this challenge, but that I feel has been hoisted on the flagship since day one. A lot, as it turns out. I have become a better engineer, a better scientist, and a better person. The trip has been long and hard, but in the end I feel that this work is really my own, and it shows all my ideas, enthusiasm, and skills for getting it working. For a number of years I will be able to brag about knowing things about Java and the Java platform that no-one should have to know. I hope that others will be able to learn from both my successes and my mistakes, and that future research will be better off than when I started my work, all the way back in 2008. I would like to extend greetings and general kudos to my colleagues for putting up with me for the best part of 6 years (10! for some): Hugo, Stoyan, Luís and Sérgio; also other guys from ESW, GSD, the room across, and the new blood from INESC-ID room 138. I put up with you too, so don’t complain too much :) A big hug and thanks goes to Telma. Thanks for keeping me company all those lunches, thanks for kicking my butt when I deserved, and thanks for listening when I needed someone to talk to. We’re almost out of here, home stretch! I would not be here if it were not for the support from my advisor, Professor João Cachopo. His insights and steering were invaluable for the success of this work, and his technical skills and engineer mind are among the best I had the pleasure to cross paths with. I wish you the best on your UK adventure, and well, here’s hoping I’ll get to it someday too ;) Last, but by no stretch of mind least, my friends and family. Friends, past and present: Thanks are never enough. Francisco, Eduardo, Tiago, Hugo, Camila, João², Ivo, Rafael, Luena, Patrícia, Ana. I love you seems to not really cover it. As I sit here writing this text, I fail to see how these characters can convey my deep thanks for your support. You know who you are. Thanks for being a part of my life, and hope you’ve enjoyed doing so. Thanks to my younger sister, Mafalda. You’ve grown so much these past years — I have to say I kinda envy you! Thanks to my father, Carlos, and to Dulce, for believing in me and giving me a place where I can always go back to. Thanks to Mário, who doesn’t get enough credit for setting me straight one too many times and for teaching me to go further. A wink goes to Márcia, the latest entry on this list. Where were you? You almost didn’t make it in time to be present on this list, you late baka-head. I love you! xi Finally, I dedicate this dissertation to my mother, Isabel, and to my grandfather, Ivo, who are no longer amongst us. Your departure left me with great sorrow, but I like to think that you’d approve of where I am now and what I decided to do with my life. I daresay you’d be proud. Ivo Anjo July 2014 xii Contents List of Figures xvii List of Tables xix List of Listings xxi 1 Introduction 1 1.1 Thesis Statement..................................... 3 1.2 Contributions....................................... 4 1.3 Outline of the Dissertation ............................... 5 2 Motivation, Problem Statement, and Approach7 2.1 Towards the Parallelization of Object-Oriented Applications............ 8 2.2 Case Study: The Java Platform............................. 10 2.2.1 Java or JVM Applications? ............................ 10 2.3 Problem Statement.................................... 11 2.4 Approach: Creation of a Speculative Parallelization Framework.......... 12 2.4.1 Solution Overview ................................. 12 2.4.2 JaSPEx-MLS System Overview.......................... 13 2.4.3 Why Work Atop the HotSpot JVM?........................ 15 2.4.4 Method-Level Speculation............................. 16 2.4.5 Threads and Tasks................................. 17 2.5 Summary ........................................