UNIVERSIDAD COMPLUTENSE DE MADRID FACULTAD DE INFORMÁTICA DEPARTAMENTO DE ARQUITECTURA DE COMPUTADORES Y AUTOMÁTICA TESIS DOCTORAL System Level Management of Hybrid Memory Systems Gestión de jerarquías de memoria híbridas a nivel de sistema MEMORIA PARA OPTAR AL GRADO DE DOCTOR PRESENTADA POR Manu Perumkunnil Komalan DIRECTORES José Ignacio Gómez Pérez Christian Tomás Tenllado Francky Catthoor Madrid, 2018 © Manu Perumkunnil Komalan, 2017 ARENBERG DOCTORAL SCHOOL Faculty of Engineering Science Universidad Complutense de Madrid Facultad de Informática Departamento de Arquitectura de Computadores y Automática System Level Management of Hybrid Memory Systems Gestión de jerarquías de memoria híbridas a nivel de sistema Manu Perumkunnil Komalan Supervisors Prof. dr. ir. José Ignacio Gómez Pérez (UCM) Prof. dr. ir. Christian Tomás Tenllado (UCM) Prof. dr. ir. Francky Catthoor (KU Leuven) March 2017 System Level Management of Hybrid Memory Sys­ tems Gestión de jerarquías de memoria híbridas a nivel de sistema Manu Perumkunnil KOMALAN Examination committee: Prof. dr. ir. José Ignacio Gómez Pérez Prof. dr. ir. Christian Tomás Tenllado Prof. dr. ir. Francky Catthoor Prof. dr. ir. Wim Dehaene Prof. dr. ir. Dirk Wouters Prof. dr. ir. Manuel Prieto Matías Prof. dr. ir. Luis Piñuel Prof. dr. ir. José Manuel Colmenar March 2017 2017, UC Madrid, KU Leuven, – Manu Perumkunnil Komalan Acknowledgments There is an impossibly long list of people I want to thank for helping me in the pursuit of my PhD and making it worth much more than simple technical jargon. Like I’ve been counseled many a times and experienced, my PhD like every other PhD has followed the trajectory of a sine wave. There have been crests and there have been troughs. At the end, I’m glad the positive aspects won out and the journey provided me with the experience of a lifetime. I want to start off by thanking my teachers who taught and guided me at different stages of my academic and work life before I started my PhD. It was their vision and guidance that finally culminated in this. A few names that come to mind are Mr. Robin Kumar, Dr. O.P. Sinha, Dr. Susmita Mitra, Dr. Sandip Chakraborty, Dr. K. S. R Koteswara Rao, Dr. R. P. Singh, Dr. Subhasis Ghosh, Dr. Shanu Chacko, Dr. John Joseph, Dr. Jaya Lal Ji and Dr. Nancy George. I have been blessed to have a family who has always supported me, loved me unconditionally, inspired me and been the bedrock on which my life is based. My parents have provided me with strength, whenever it was lacking and it is only due to their upbringing, constant care and steadfast confidence in me that I’ve been able to reach where I have. Through the toughest times of my PhD they have braced me without questions. My younger brother has been another source of support and love for me who has kept me grounded and true to my self. His cheerful nature and solutions to stressful situations have always picked me up. They say that each friendship offers something totally unique and irreplaceable. Each friendship ultimately makes us who we are. I’ve been lucky enough to have a plethora of friends who have helped me when I was constantly shuffling between Madrid and Leuven and also helped me whenever I was down by cheering me up and offering consolation whenever required. First and foremost, I’d like to thank my close friends in Leueven without whom, it would have been i ii ACKNOWLEDGMENTS impossible to get through the PhD. Arul, Manisha, Phalguni, Diana, Sajin, Ameya, Ashwini and Sriram... without you guys it wouldn’t have been worth it. I’d like to thank Ricardo and Jorge Val Calvo for being the very best roommates during my stay in Madrid. They helped me out with the bureaucracy whenever it came up and were always up for a drink to lift my spirits. I’d like to thank all the lab mates in Madrid: Jorge Quintas, Daniel Tabas, Jose Luis, Fermin and Javier for making my stay at UCM thoroughly enjoyable. I’d also like to thank my friends and colleagues at imec, namely, Matthias, Prashant, Arindam, Trong, Bharani for making my stay at imec a wonderful experience. I’d like to especially thank Sushil Sakhare and Oh HyungRock for being more than just friends and also mentoring me in whatever way possible. I would also like to thank the numerous mentors who have helped me over the course of my PhD like Dr. Dirk Wouters, Dr. Stefan Cosemans, Dr. Julien Ryckaert, Dr. Praveen Raghavan, Prof. Manuel Prieto and Prof. Jose Francisco Tirado. I would also like to thank Prof. Wim Dehaene for taking the time to review my PhD thesis and being an invaluable part of my supervisory committee. Finally, I would like to thank the three most important people who have been responsible for me completing this PhD: Prof. Francky Catthoor, Prof. José Ignacio Gómez and Prof. Christian Tenllado. Francky has mentored me not only technically but also on various aspects of life and broadened my horizons. He has always had a solution to any problem and with unending energy helped me whenever possible and in whatever way. I will always be grateful for that. I cannot stress how much of a help Nacho (José Ignacio) has been to me these last 5-6 years. He has been more of an elder brother to me than an advisor and helped me in innumerable ways, be it technically, financially or any other. It is only due to his help that I was able to navigate through one of the darkest phases of my PhD. Christian has been the same and helped me immensely whenever possible. Besides work, living in Europe has also been a fascinating experience. After about 6 years, I feel like I can truly call Leuven my second home. Contents Acknowledgments i Contents iii List of Figures vii List of Tables xv Abstract xvii Resumen xxi Beknopte samenvatting xxv 1 Introduction 1 1.1 Context and Motivation ...................... 1 1.1.1 Global Context ....................... 1 1.1.2 The Memory Perspective . 4 1.2 PhD Focus and Objectives ..................... 10 1.3 List of Simulators Used ...................... 13 1.4 List of Contributions ........................ 13 1.5 PhD structure ............................ 17 iii iv CONTENTS 2 NVMs: State of the Art 21 2.1 General Overview of Non Volatile Memories . 21 2.2 Overview of NVM device characteristics . 22 2.2.1 Flash Memory ....................... 22 2.2.2 MRAM ........................... 23 2.2.3 RRAM/ReRAM ...................... 29 2.2.4 PCRAM/PRAM ...................... 34 2.2.5 FeRAM ........................... 35 2.2.6 DWM ............................ 37 2.3 Other new memory technologies and the memory roadmap . 40 2.4 System Architecture Level changes and Hybrid memory systems 44 2.4.1 Classification of Hybrid Memory Organizations . 44 2.4.2 Hybrid Memory Systems SOTA . 46 3 eNVM based Instruction memory for an Ultra Low Power Platform 52 3.1 Introduction ............................. 52 3.2 Related Work ............................ 55 3.3 NVM array design ......................... 57 3.3.1 OxRAM: Cell, Characteristics and Parameters . 58 3.3.2 xRAM periphery vs traditional SRAM periphery . 59 3.4 System Overview .......................... 63 3.5 OxRAM based Instruction Memory Exploration . 64 3.5.1 Proposed Architectural Changes . 65 3.5.2 Addressing the OxRAM limitations . 66 3.6 Energy Modeling .......................... 72 3.7 Exploration and Analysis ..................... 74 3.7.1 Exploration of real Benchmarks . 76 CONTENTS v 3.7.2 Exploration by means of synthetic benchmarks . 83 3.8 Conclusions ............................. 88 4 eNVM based Configuration Memory Organisation 93 4.1 Introduction ............................. 93 4.2 Related Work ............................ 95 4.3 CGRA Basics and Base architecture . 96 4.4 eNVM based memory architecture exploration . 100 4.4.1 OxRAM architectural modifications to address technol­ ogy limitations ....................... 100 4.4.2 CGRA interface exploration . 101 4.4.3 Alternatives to further reduce the energy consumption . 103 4.5 Energy Modeling .......................... 105 4.5.1 SRAM, Configuration cache and VWR energy . 105 4.5.2 OxRAM memory configuration . 106 4.6 Results and Discussion . 106 4.6.1 Assumptions ........................ 106 4.6.2 Discussion of results .................... 108 4.7 Conclusions and Future work . 116 5 NVM based I-Cache for High Performance General Purpose Processors 117 5.1 Introduction . 117 5.2 Related Work . 118 5.3 Replacing SRAM with NVMs . 120 5.3.1 STT-MRAM vs ReRAM . 124 5.3.2 NVM drop-in ........................ 125 5.4 Extended MSHR to address NVM limitations . 126 vi CONTENTS 5.5 Exploration and Analysis ..................... 131 5.6 Conclusion ............................. 141 6 STT-MRAM based D-cache explorations for High Performance General Purpose Processors 143 6.1 Introduction . 143 6.2 Related Work . 145 6.3 Replacing SRAM with STT-MRAM: Comparison with existing solutions . 145 6.4 Architectural modifications to address NVM limitations . 147 6.5 Code Transformations and Optimization . 150 6.6 Exploration and Analysis ..................... 152 6.7 Conclusion ............................. 156 7 Conclusions and Future Work 158 7.1 Summary of Conclusions . 158 7.1.1 eNVM based Instruction memory for an Ultra Low Power Platform . 159 7.1.2 eNVM based Configuration Memory Organization . 160 7.1.3 NVM based I-Cache for High Performance General Purpose Processors . 161 7.1.4 STT-MRAM based D-cache explorations for High Per­ formance General Purpose Processors . 161 7.1.5 Case study: Cross Layer Exploration of a STT-MRAM based L2 cache memory organization . 162 7.1.6 Impact of multicore architectures . 166 7.2 Future Work ............................ 168 Curriculum vitae 171 Bibliography 173 List of Figures 1.1 The different NVM applications in the electronics industry with respect to market size in 2012.