Application-Level Virtual Memory for Object-Oriented Systems Mariano Martinez Peck
Total Page：16
File Type：pdf, Size：1020Kb
Application-Level Virtual Memory for Object-Oriented Systems Mariano Martinez Peck To cite this version: Mariano Martinez Peck. Application-Level Virtual Memory for Object-Oriented Systems. Program- ming Languages [cs.PL]. Université des Sciences et Technologie de Lille - Lille I, 2012. English. tel-00764991 HAL Id: tel-00764991 https://tel.archives-ouvertes.fr/tel-00764991 Submitted on 26 Dec 2012 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. N° d’ordre : 40886 THESE présentée en vue d’obtenir le grade de DOCTEUR en Spécialité : informatique par Mariano MARTINEZ PECK DOCTORAT DELIVRE CONJOINTEMENT PAR MINES DOUAI ET L’UNIVERSITE DE LILLE 1 Titre de la thèse : Application-Level Virtual Memory for Object-Oriented Systems Soutenue le 29/10/2012 à 10h devant le jury d’examen : Président Jean-Bernard STEFANI (Directeur de recherche – INRIA Grenoble- Rhône-Alpes) Directeur de thèse Stéphane DUCASSE (Directeur de recherche – INRIA Lille) Rapporteur Robert HIRSCHFELD (Professeur – Hasso-Plattner-Institut, Universität Potsdam, Allemagne) Rapporteur Christophe DONY (Professeur – Université Montpellier 2) Examinateur Roel WUYTS (Professeur – IMEC & Katholieke Universiteit Leuven, Belgique) co-Encadrant Noury BOURAQADI (Maître-Assistant – Mines de Douai) co-Encadrant Marcus DENKER (Chargé de recherche – INRIA Lille) co-Encadrant Luc FABRESSE (Maître-Assistant – Mines de Douai) Laboratoire(s) d’accueil : Dépt. IA, Mines Douai + RMoD INRIA Lille Nord de France Ecole Doctorale SPI 072 (Lille I, Lille III, Artois, ULCO, UVHC, Centrale Lille) Contents Acknowledgments ix Abstract xiii Résumé xv 1 Introduction1 1.1 Context: The Memory Used by Running Object-oriented Programs.....2 1.2 The Problem of Using Unnecessary Memory................2 1.3 Shortcomings of Existing Approaches....................3 1.4 Our Solution in a Nutshell..........................5 1.5 Contributions.................................6 1.6 Structure of the Dissertation.........................7 2 Unused Memory in OO Programs: The Problem9 2.1 Introduction.................................. 10 2.2 Glossary.................................... 10 2.3 Analyzing Memory Usage.......................... 11 2.3.1 Detecting Used and Unused Objects................. 11 2.3.2 Applications for Analysis...................... 12 2.3.3 Unused Objects: Experiments.................... 13 2.4 Requirements for Application-Level Virtual Memory Manager....... 14 2.5 Summary................................... 15 3 Approaches to Unused and Application-Level Virtual Memory 17 3.1 Introduction.................................. 18 3.2 Virtual Memory and Garbage Collected Languages............. 18 3.2.1 Operating System Virtual Memory.................. 18 3.2.2 Application-Specific Operating System Virtual Memory...... 19 3.2.3 LOOM: Large Object-Oriented Memory............... 21 3.2.4 Solving the OS Thrashing Problem................. 22 3.3 Reducing the Memory Occupied by Code.................. 23 3.3.1 Reduced and Specialized Runtimes................. 23 3.3.2 Custom and Specific Runtimes.................... 24 3.4 Object Faulting, Orthogonal Persistence and Object Databases....... 25 3.4.1 Object Faulting............................ 25 3.4.2 Orthogonal Persistence and Object Databases............ 26 3.5 Melt: Tolerating Memory Leaks....................... 27 3.6 General-Purpose Object Graph Swappers.................. 28 3.6.1 ImageSegment Object Swapping Principles............. 28 iv Contents 3.6.2 Swapping Out and In......................... 29 3.6.3 Evaluation of ImageSegment..................... 29 3.7 Summary................................... 31 4 Marea, A Model for Application-Level Virtual Memory 33 4.1 Introduction.................................. 34 4.2 Marea Overview............................... 34 4.2.1 Marea Subsystems.......................... 35 4.2.2 Evaluating Requirements on Marea................. 36 4.3 The Main Challenge: Dealing Efficiently with Shared Objects....... 37 4.3.1 The Case of Shared Objects..................... 37 4.3.2 Handling Shared Objects....................... 39 4.4 Marea’s Object Swapper Algorithms..................... 39 4.4.1 Swapping Out............................. 39 4.4.2 Swapping In............................. 42 4.5 Handling Graph Swapping Intersection.................... 45 4.5.1 Problem of Shared Proxies...................... 45 4.5.2 Solution part 1: Swapping Out with Shared Proxies......... 45 4.5.3 Solution part 2: Swapping In with Shared Proxies.......... 46 4.5.4 Cleaning SharedProxiesTable.................... 47 4.6 Summary................................... 47 5 Marea’s Validation 49 5.1 Introduction.................................. 50 5.2 How to Use Marea?.............................. 50 5.3 Experimental Setup.............................. 51 5.4 Benchmarks and Case Studies........................ 52 5.4.1 Swapping Out Code......................... 52 5.4.2 Swapping Out Data.......................... 54 5.5 Measuring Unused Objects and Speed.................... 55 5.5.1 Measuring Unused Objects...................... 55 5.5.2 Speed Analysis............................ 56 5.6 Conclusion.................................. 57 6 Fuel: A Fast and Universal Serializer 59 6.1 Introduction.................................. 60 6.1.1 Motivation and Desired Properties.................. 60 6.1.2 Overview of Fuel........................... 61 6.2 Serializer Required Concerns and Challenges................ 62 6.2.1 Serializer concerns.......................... 63 6.2.2 Serializer challenges......................... 64 6.3 Fuel’s Foundation............................... 66 6.3.1 Pickle Format............................. 66 6.3.2 Grouping objects in clusters..................... 67 Contents v 6.3.3 Analysis phase............................ 69 6.3.4 Two phases for writing instances and references........... 69 6.3.5 Iterative graph recreation....................... 70 6.3.6 Breadth-first traversal......................... 70 6.4 Fuel’s Features................................ 71 6.4.1 Fuel serializer concerns........................ 71 6.4.2 Addressing Challenges with Fuel.................. 73 6.4.3 Discussion.............................. 74 6.5 Fuel Design and Packages.......................... 75 6.5.1 Fuel Design.............................. 75 6.5.2 Fuel Packages and Libraries..................... 76 6.6 Benchmarks.................................. 78 6.6.1 Benchmarks constraints and characteristics............. 78 6.6.2 Benchmarks serializing primitive and large objects......... 80 6.6.3 ImageSegment results explained................... 82 6.6.4 Different graph sizes......................... 83 6.6.5 Differences while using CogVM................... 83 6.6.6 General benchmarks conclusions................... 84 6.7 Real Cases Using Fuel............................ 85 6.8 Related Work................................. 87 6.9 Conclusion.................................. 88 7 Ghost: Uniform and Flexible Proxies 89 7.1 Introduction.................................. 90 7.2 Proxy Evaluation Criteria........................... 91 7.2.1 Vocabulary and Roles......................... 91 7.2.2 Proxies Implementation Criteria................... 92 7.3 Common Proxy Implementation....................... 94 7.3.1 Typical Proxy Implementation.................... 94 7.3.2 Evaluation.............................. 96 7.4 Pharo Support for Proxies........................... 97 7.4.1 Pharo Reflective Model and VM Overview............. 97 7.4.2 Hooks and Features Provided by Pharo............... 98 7.5 Ghost’s Design and Implementation..................... 99 7.5.1 Overview Through an Example................... 99 7.5.2 Proxies for Regular Objects..................... 101 7.5.3 Extending and Adapting Proxies and Handlers........... 102 7.5.4 Intercepting Everything or Not?................... 103 7.5.5 Messages to be Answered by the Handler.............. 103 7.6 Proxies for Classes and Methods....................... 105 7.6.1 Proxies for Methods......................... 105 7.6.2 Proxies for Classes.......................... 106 7.7 Special Messages and Operations....................... 109 7.7.1 Inlined Methods........................... 110 vi Contents 7.7.2 Special Associated Bytecodes and VM optimization........ 111 7.8 Evaluation................................... 112 7.9 Discussing VM Support for Proxies..................... 114 7.10 Case Studies.................................. 115 7.10.1 Marea................................. 115 7.10.2 Method Wrappers........................... 116 7.11 Related Work................................. 117 7.11.1 Proxies in dynamic languages.................... 117 7.11.2 Proxies in static languages...................... 119 7.11.3 Comparison.............................. 120 7.12 Conclusion.................................. 121 8 Marea’s Implementation Details 123 8.1 Introduction.................................. 124 8.2 Marea Implementation............................ 124 8.2.1 Marea Design............................. 124 8.2.2 Requirements