Elephant User Manual Elephant version 0.9 By Ian Eslick with Robert Read and Ben Lee Elephant System Original Version, Copyright c 2004 Ben Lee and Andrew Blumberg. Version 0.5, Copyright c 2006 Robert L. Read. Versions 0.6-0.9, Copyright c 2006-2007 Ian Eslick and Robert L. Read Portions copyright respective contributors (see `CREDITS'). Elephant Manual Original Version, Copyright c 2004 Ben Lee. Versions 0.5-0.6, Copyright c 2006 Robert L. Read. Current Version, Copyright c 2006-2007 Ian Eslick and Robert L. Read Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License. See the Copyright and License chapter for details about copyright, license and warranty for this manual and the Elephant system. i Short Contents 1 Introduction ::::::::::::::::::::::::::::::::::::::::: 1 2 Tutorial ::::::::::::::::::::::::::::::::::::::::::::: 3 3 Installation ::::::::::::::::::::::::::::::::::::::::: 23 4 User Guide ::::::::::::::::::::::::::::::::::::::::: 33 5 User API Reference :::::::::::::::::::::::::::::::::: 57 6 Design Patterns:::::::::::::::::::::::::::::::::::::: 69 7 Elephant Architecture :::::::::::::::::::::::::::::::: 85 8 Data Store API Reference ::::::::::::::::::::::::::::: 91 9 Copyright and License :::::::::::::::::::::::::::::::: 99 A Concept Index :::::::::::::::::::::::::::::::::::::: 101 B Object Index ::::::::::::::::::::::::::::::::::::::: 103 C Function / Macro Index :::::::::::::::::::::::::::::: 105 D Variable Index :::::::::::::::::::::::::::::::::::::: 107 Colophon:::::::::::::::::::::::::::::::::::::::::::::: 109 ii Elephant User Manual iii Table of Contents 1 Introduction::::::::::::::::::::::::::::::::::::: 1 1.1 History :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 1 1.2 Elephant Goals::::::::::::::::::::::::::::::::::::::::::::::::: 1 1.3 More Information :::::::::::::::::::::::::::::::::::::::::::::: 2 2 Tutorial :::::::::::::::::::::::::::::::::::::::::: 3 2.1 Overview :::::::::::::::::::::::::::::::::::::::::::::::::::::: 3 2.2 Getting Started :::::::::::::::::::::::::::::::::::::::::::::::: 4 2.3 The Store Root :::::::::::::::::::::::::::::::::::::::::::::::: 4 2.4 Serialization:::::::::::::::::::::::::::::::::::::::::::::::::::: 5 2.5 Persistent Classes :::::::::::::::::::::::::::::::::::::::::::::: 6 2.6 Rules about Persistent Classes:::::::::::::::::::::::::::::::::: 8 2.7 Persistent collections ::::::::::::::::::::::::::::::::::::::::::: 9 2.7.1 Using PSets ::::::::::::::::::::::::::::::::::::::::::::::: 9 2.7.2 Using BTrees::::::::::::::::::::::::::::::::::::::::::::: 11 2.8 Indexing Persistent Classes :::::::::::::::::::::::::::::::::::: 12 2.9 Using Transactions :::::::::::::::::::::::::::::::::::::::::::: 15 2.9.1 Why do we need Transactions? ::::::::::::::::::::::::::: 15 2.9.2 Using with-transaction :::::::::::::::::::::::::::::::: 17 2.9.3 Nesting Transactions ::::::::::::::::::::::::::::::::::::: 17 2.9.4 Idempotent Side Effects :::::::::::::::::::::::::::::::::: 18 2.9.5 Transactions and Performance :::::::::::::::::::::::::::: 19 2.9.6 Transactions and Applications :::::::::::::::::::::::::::: 20 2.10 Advanced Topics ::::::::::::::::::::::::::::::::::::::::::::: 21 3 Installation ::::::::::::::::::::::::::::::::::::: 23 3.1 Requirements ::::::::::::::::::::::::::::::::::::::::::::::::: 23 3.1.1 Supported Lisp, Platform and Data store combinations ::: 23 3.1.2 Library dependencies ::::::::::::::::::::::::::::::::::::: 23 3.2 Configuring Elephant ::::::::::::::::::::::::::::::::::::::::: 24 3.3 Loading Elephant ::::::::::::::::::::::::::::::::::::::::::::: 24 3.3.1 Loading Elephant via ASDF:::::::::::::::::::::::::::::: 25 3.3.2 Two-Phase Load Process ::::::::::::::::::::::::::::::::: 25 3.3.3 Packages ::::::::::::::::::::::::::::::::::::::::::::::::: 25 3.3.4 Opening a Store:::::::::::::::::::::::::::::::::::::::::: 25 3.4 Berkeley DB :::::::::::::::::::::::::::::::::::::::::::::::::: 26 3.5 Setting up Berkeley DB ::::::::::::::::::::::::::::::::::::::: 26 3.6 Upgrading Berkeley DB Databases :::::::::::::::::::::::::::: 26 3.6.1 Upgrading to 0.9 ::::::::::::::::::::::::::::::::::::::::: 27 3.6.2 Upgrade from Elephant 0.5 ::::::::::::::::::::::::::::::: 27 3.7 CL-SQL :::::::::::::::::::::::::::::::::::::::::::::::::::::: 27 3.8 CL-SQL Example ::::::::::::::::::::::::::::::::::::::::::::: 28 iv Elephant User Manual 3.9 Elephant on Windows ::::::::::::::::::::::::::::::::::::::::: 29 3.10 Test Suites::::::::::::::::::::::::::::::::::::::::::::::::::: 29 3.11 Documentation :::::::::::::::::::::::::::::::::::::::::::::: 30 4 User Guide::::::::::::::::::::::::::::::::::::: 33 4.1 The Store Controller :::::::::::::::::::::::::::::::::::::::::: 33 4.2 Serialization details ::::::::::::::::::::::::::::::::::::::::::: 34 4.2.1 Restrictions of Store-by-Value :::::::::::::::::::::::::::: 34 4.2.2 Atomic Types :::::::::::::::::::::::::::::::::::::::::::: 35 4.2.3 Aggregate Types ::::::::::::::::::::::::::::::::::::::::: 36 4.3 Persistent Classes and Objects :::::::::::::::::::::::::::::::: 37 4.3.1 Persistent Class Definition:::::::::::::::::::::::::::::::: 37 4.3.2 Instance Creation :::::::::::::::::::::::::::::::::::::::: 39 4.3.3 Persistent Instance Lifecycle :::::::::::::::::::::::::::::: 40 4.3.4 Using Transient Slots::::::::::::::::::::::::::::::::::::: 40 4.3.5 Using Persistent Slots :::::::::::::::::::::::::::::::::::: 41 4.3.6 Class Redefinition :::::::::::::::::::::::::::::::::::::::: 42 4.3.7 Support for change-class ::::::::::::::::::::::::::::::: 43 4.4 Class Indices :::::::::::::::::::::::::::::::::::::::::::::::::: 43 4.4.1 Synchronizing Classes and Data Stores ::::::::::::::::::: 44 4.5 Persistent Sets :::::::::::::::::::::::::::::::::::::::::::::::: 44 4.6 Persistent BTrees ::::::::::::::::::::::::::::::::::::::::::::: 44 4.7 BTree Cursors :::::::::::::::::::::::::::::::::::::::::::::::: 45 4.8 BTree Indexing ::::::::::::::::::::::::::::::::::::::::::::::: 46 4.9 Index Cursors ::::::::::::::::::::::::::::::::::::::::::::::::: 48 4.10 Multi-threaded Applications:::::::::::::::::::::::::::::::::: 48 4.10.1 Shared Resources ::::::::::::::::::::::::::::::::::::::: 49 4.10.2 Data Store Thread Safety and Transactions:::::::::::::: 49 4.10.3 Minimize Dependency on Thread-Local Specials ::::::::: 49 4.11 Transaction Details :::::::::::::::::::::::::::::::::::::::::: 49 4.11.1 with-transaction internals :::::::::::::::::::::::::::: 50 4.11.2 execute-transaction internals ::::::::::::::::::::::::: 50 4.11.3 Building your own transactional framework :::::::::::::: 50 4.11.4 Analyzing Dynamic Transaction Behavior ::::::::::::::: 51 4.12 Multi-repository Operation::::::::::::::::::::::::::::::::::: 51 4.13 Multiple Processes and Distributed Applications:::::::::::::: 51 4.14 Repository Migration and Upgrade ::::::::::::::::::::::::::: 51 4.15 Performance Tuning ::::::::::::::::::::::::::::::::::::::::: 53 4.16 Garbage Collection::::::::::::::::::::::::::::::::::::::::::: 53 4.17 Berkeley DB Data Store ::::::::::::::::::::::::::::::::::::: 53 4.17.1 Architecture Overview :::::::::::::::::::::::::::::::::: 54 4.17.2 Opening a Store::::::::::::::::::::::::::::::::::::::::: 54 4.17.3 Starting a Transaction :::::::::::::::::::::::::::::::::: 54 4.17.4 Special Commands :::::::::::::::::::::::::::::::::::::: 55 4.17.5 Performance Tuning::::::::::::::::::::::::::::::::::::: 55 4.18 CLSQL Data Store::::::::::::::::::::::::::::::::::::::::::: 56 4.18.1 Basic CLSQL Implementation ::::::::::::::::::::::::::: 56 4.19 Postmodern Data Store :::::::::::::::::::::::::::::::::::::: 56 v 4.20 Native Lisp Data Store::::::::::::::::::::::::::::::::::::::: 56 5 User API Reference ::::::::::::::::::::::::::: 57 5.1 Store Controllers :::::::::::::::::::::::::::::::::::::::::::::: 57 5.2 Persistent Objects::::::::::::::::::::::::::::::::::::::::::::: 58 5.3 Persistent Object Indexing :::::::::::::::::::::::::::::::::::: 58 5.3.1 Indexed Object Accessors :::::::::::::::::::::::::::::::: 58 5.3.2 Direct Class Index Manipulation:::::::::::::::::::::::::: 59 5.3.3 Dynamic Indexing API ::::::::::::::::::::::::::::::::::: 59 5.4 Persistent Sets :::::::::::::::::::::::::::::::::::::::::::::::: 60 5.5 BTrees :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 61 5.6 Btree Cursors ::::::::::::::::::::::::::::::::::::::::::::::::: 62 5.7 Index Cursors ::::::::::::::::::::::::::::::::::::::::::::::::: 64 5.8 Transactions :::::::::::::::::::::::::::::::::::::::::::::::::: 65 5.9 Migration and Upgrading ::::::::::::::::::::::::::::::::::::: 66 6 Design Patterns ::::::::::::::::::::::::::::::: 69 6.1 Persistent System Objects::::::::::::::::::::::::::::::::::::: 69 6.2 File System Replacement:::::::::::::::::::::::::::::::::::::: 70 6.3 Checkpointing Conventional Program State:::::::::::::::::::: 72 6.3.1 Assumptions ::::::::::::::::::::::::::::::::::::::::::::: 72 6.3.2 Snapshot Set ::::::::::::::::::::::::::::::::::::::::::::: 73 6.3.3 Snapshot Set Implementation::::::::::::::::::::::::::::: 75 6.3.4 Isolating multiple snapshot sets::::::::::::::::::::::::::: 78 6.4 Elephant as Database ::::::::::::::::::::::::::::::::::::::::: 80 6.5 Multithreaded Web Applications :::::::::::::::::::::::::::::: 81 6.6 Real-World Application Examples ::::::::::::::::::::::::::::: 81 6.6.1 Konsenti ::::::::::::::::::::::::::::::::::::::::::::::::: 81 6.6.2 Conceptminer :::::::::::::::::::::::::::::::::::::::::::: 81 7 Elephant Architecture :::::::::::::::::::::::: 85 7.1 Initializing a store controller::::::::::::::::::::::::::::::::::: 85 7.2 Persistent Object Creation