Memory Pool System Documentation Release 1.111.0
Total Page:16
File Type:pdf, Size:1020Kb
Memory Pool System Documentation Release 1.111.0 Ravenbrook Limited May 23, 2013 CONTENTS i ii CHAPTER ONE GUIDE 1.1 Overview of the Memory Pool System The Memory Pool System is a very general, adaptable, flexible, reliable, and efficient memory management system. It permits the flexible combination of memory management techniques, supporting manual and automatic memory management, inline allocation, finalization, weakness, and multiple concurrent co-operating incremental generational garbage collections. It also includes a library of memory pool classes implementing specialized memory management policies. The MPS has been in development since 1994 and deployed in successful commercial products since 1997. Bugs are almost unknown in production. It is under continuous development and support by Ravenbrook. The MPS is distributed under an open source license. The license is designed to make it possible for you to use the MPS in your own projects, provided that you either don’t distribute your product, or your product is open source too. If the licensing terms aren’t suitable for you (for example, you’re developing a closed-source commercial product or a compiler run-time system) you can easily license the MPS under different terms from Ravenbrook by arrangement. Please contact us at [email protected] for details. 1.1.1 Supported target platforms The MPS is currently supported for deployment on: • Windows XP or later on IA-32 and x86-64, using Microsoft Visual C/C++; • Linux (Ubuntu 11 and RHEL 6.3 known good, otherwise YMMV) on IA-32 and x86-64, using GCC; • FreeBSD 7 or later, on IA-32 and x86-64, using GCC; • OS X 10.4 or later, on IA-32 and x86-64 (single threaded only), using Clang/LLVM. The MPS will not work in a multi-threaded 32-bit application on 64-bit Windows 7. This is due to a serious fault in Microsoft’s WOW64 emulator that we are powerless to correct. It is due to be fixed in Windows 8. See WOW64 bug: GetThreadContext() may return stale contents. The MPS is highly portable and has run on many other processors and operating systems in the past (see Building the Memory Pool System). Most of the MPS is written in very pure ANSI C and compiles without warnings on anything. 1.1.2 Technical introduction The figure below gives a simplified picture of a program’s memory from the point of view of the Memory Pool System. 1 Memory Pool System Documentation, Release 1.111.0 Figure 1.1: Overview of the Memory Pool System. The arena is the top-level data structure in the MPS. An arena is responsible for requesting memory(3) from the operating system (and returning it), for making memory available to pools, and for garbage collection. Multiple arenas are supported, but it’s usually best to have only one arena in your program, because the MPS can’t collect cyclic structures that span multiple arenas. See Arenas. The MPS is designed to co-operate with other memory managers (for example malloc and free(2) in C, or operators new and delete in C++), so you need not move all your memory management to the MPS at once, and you can co-operate with libraries that use other allocation mechanisms. Within the arena you create one or more pools.A pool is responsible for requesting memory from the arena and making it available to your program. See Pools. Pools belong to pool classes that specify policies for how their memory is managed. Some pools are manually managed (you must explicitly return memory to the pool, for example by calling mps_free()) and others are automatically managed (the garbage collector reclaims unreachable blocks). See Pool reference. Formatted pools need you to tell them how to scan for references to allocated blocks. See Scanning. The arena needs you to tell it how to find your roots: references to allocated blocks that are stored in static data, in memory not managed by the MPS, or on your program’s registers or control stack. See Roots. The MPS is designed to work with multi-threaded programs. Functions in the C interface are thread safe, except in a few documented cases. See Threads. The allocation point protocol provides fast lock-free allocation on multiple threads simultaneously. See Allocation. The garbage collector is incremental: it proceeds in small steps interleaved with the execution of your program, so there are no long waits. See Garbage collection. 1.1.3 What next? For a much more detailed technical overview of the MPS, see Brooksby (2002). If you’re going to try it out, see Building the Memory Pool System. If you have a program in need of memory management, then you’ll want to learn how to integrate it with the Memory Pool System. See Garbage collecting a language with the Memory Pool System. If you want to know more technical details, they appear in the Reference. 1.2 Building the Memory Pool System 1.2.1 Introduction This document describes the various ways in which you can build the MPS, its libraries, and the tests and tools that come with it. You may be building the MPS for a number of different purposes. 1.2.2 Getting hold of the MPS Kit Download the latest MPS Kit release from http://www.ravenbrook.com/project/mps/release/. 2 Chapter 1. Guide Memory Pool System Documentation, Release 1.111.0 1.2.3 Compiling the MPS for your project It is easy to compile the MPS. You can do it separately, or include the source in your own project’s build system. This section describes compilation in terms of command lines, but you can equally add the files to a project in an IDE. The MPS also comes with Makefiles and IDE project files for building libraries, tools, and tests. See “Building the MPS for development”. Compiling for production In the simplest case, you can compile the MPS to an object file with just: cc-c mps.c (Unix/Mac OS X) cl/c mps.c (Windows) This will build a “hot” variety (for production) object file for use with mps.h. You can greatly improve performance by allowing global optimization, for example: cc-O2-c mps.c (Unix/Mac OS X) cl/O2/c mps.c (Windows) Compiling for debugging You can get a “cool” variety MPS (with more internal checking, for debugging and development) with: cc-g-DCONFIG_VAR_COOL-c mps.c (Unix/Mac OS X) cl/Zi/DCONFIG_VAR_COOL/c mps.c (Windows) Optimizing for your object format If you are using your own object format, you will also get improved performance by allowing the compiler to do global optimizations between it and the MPS. So if your format implementation is in, say, myformat.c, then you could make a file mymps.c containing: #include "mps.c" #include "myformat.c" then: cc-O2-c mymps.c (Unix/Mac OS X) cl/O2/c mymps.c (Windows) This will get your format code inlined with the MPS garbage collector. Compiling without the C library If you’re building the MPS for an environment without the standard C library, you can exclude the plinth component of the MPS with: cc-DCONFIG_PLINTH_NONE-c mps.c cl/Gs/DCONFIG_PLINTH_NONE/c mps.c 1.2. Building the Memory Pool System 3 Memory Pool System Documentation, Release 1.111.0 but you must then provide your own implementation of mpslib.h. You can base this on the ANSI plinth in mpsliban.c. If you want to do anything beyond these simple cases, use the MPS build as described in the section “Building the MPS for development” below. 1.2.4 Building the MPS for development If you’re making modifications to the MPS itself, want to build MPS libraries for linking, or want to build MPS tests and tools, you should use the MPS build. This uses makefiles or Xcode projects. [Coming soon, Microsoft Visual Studio solutions.] Prerequisites For Unix-like platforms you will need the GNU Make tool. Some platforms (such as Linux) have GNU Make as their default make tool. For others you will need to get and install it. (It’s available free from ftp://ftp.gnu.org/gnu/make/.) On FreeBSD this can be done as root with pkg_add -r gmake. On Windows platforms the NMAKE tool is used. This comes with Microsoft Visual Studio C++ or the Microsoft Windows SDK. On Mac OS X the MPS is built using Xcode, either by opening mps.xcodeproj with the Xcode app, or using the command-line “xcodebuild” tool, installed from Xcode ! Preferences ! Downloads ! Components ! Command Line Tools. Platforms The MPS uses a six-character platform code to express a combination of operating system, CPU architecture, and compiler toolchain. Each six-character code breaks down into three pairs of characters, like this: OSARCT Where OS denotes the operating system, AR the CPU architecture, and CT the compiler toolchain. Here are the platforms that we have regular access to and on which the MPS works well: Platform OS Architecture Compiler Makefile fri3gc FreeBSD IA-32 GCC fri3gc.gmk fri6gc FreeBSD x86_64 GCC fri6gc.gmk lii3gc Linux IA-32 GCC lii3gc.gmk lii6gc Linux x86_64 GCC lii6gc.gmk xci3ll Mac OS X IA-32 Clang mps.xcodeproj xci6ll Mac OS X x86_64 Clang mps.xcodeproj xci3gc Mac OS X IA-32 GCC (legacy) xci3gc.gmk w3i3mv Windows IA-32 Microsoft C w3i3mv.nmk w3i6mv Windows x86_64 Microsoft C w3i6mv.nmk Historically, the MPS worked on a much wider variety of platforms, and still could: IRIX, OSF/1 (Tru64), Solaris, SunOS, Classic Mac OS; MIPS, PowerPC, ALPHA, SPARC v8, SPARC v9; Metrowerks Codewarrior, SunPro C, Digital C, EGCS.