Berkeley DB 1 Berkeley DB Berkeley DB Original author(s) Margo Seltzer and Keith Bostic of Sleepycat Software Developer(s) Sleepycat Software, later Oracle Corporation Stable release 5.1.19 / September 17, 2010 Development status production Written in C Operating system Unix, Linux, Windows, AIX, Sun Solaris, SCO Unix, Mac OS Size ~700KB compiled on x86 Type Database License Sleepycat License Website [1] Berkeley DB (BDB) is a computer software library that provides a high-performance embedded database for key/value data. Berkeley DB is a programmatic software library written in C with API bindings for C++, PHP, Java, Perl, Python, Ruby, Tcl, Smalltalk, and most other programming languages. BDB stores arbitrary key/data pairs as byte arrays, and supports multiple data items for a single key. Berkeley DB is not a relational database.[2] BDB can support thousands of simultaneous threads of control or concurrent processes manipulating databases as large as 256 terabytes, on a wide variety of operating systems including most Unix-like and Windows systems, and real-time operating systems. Berkeley DB is also used as the common name for three distinct products; Oracle Berkeley DB, Berkeley DB Java Edition, and Berkeley DB XML. These three products all share a common ancestry and are currently under active development at Oracle Corporation. Origin Berkeley DB originated at the University of California, Berkeley as part of the transition (1986 to 1994) from 4.3BSD to 4.4BSD and of the effort to remove AT&T-encumbered code[3] . The first code, due to Seltzer and Yigit[4] , attempted to create a disk hash table that performed better than any of the existing Dbm libraries. In 1996 Netscape requested that the authors of Berkeley DB improve and extend the library, then at version 1.86, to suit Netscape's requirements for an LDAP server[5] and for use in the Netscape browser. That request led to the creation of Sleepycat Software. This company was acquired by Oracle Corporation in February 2006, which continues to develop and sell Berkeley DB. Since its initial release, Berkeley DB has gone through various versions. Each major release cycle has introduced a single new major feature generally layering on top of the earlier features to add functionality to the product. The 1.x releases focused on managing key/value data storage and are referred to as "Data Store" (DS). The 2.x releases added a locking system enabling concurrent access to data. This is what is known as "Concurrent Data Store" (CDS). The 3.x releases added a logging system for transactions and recovery, called "Transactional Data Store" (TDS). The 4.x releases added the ability to replicate log records and create a distributed highly available single-master multi-replica database. This is called the "High Availability" (HA) feature set. Berkeley DB's evolution has sometimes led to minor API changes or log format changes, but very rarely have database formats changed. Berkeley DB HA supports online upgrades from one version to the next by maintaining the ability to read and apply the prior release's log records. Berkeley DB 2 The FreeBSD and OpenBSD operating system continue to use Berkeley DB 1.8x for compatibility reasons;[6] Linux-based operating systems commonly include several versions to accommodate for applications still using older interfaces/files. Berkeley DB is redistributed under the Sleepycat Public License, which is an OSI-approved open source license as well as an FSF-approved free software licence.[7] [8] The product ships with complete source code, build script, test suite, and documentation. The code quality and general utility along with the licensing terms have led to its use in a multitude of free and open source software. Those who do not wish to abide by the terms of the Sleepycat Public License have the option of purchasing another proprietary license for redistribution from Oracle Corporation. This technique is called dual licensing. Berkeley DB includes compatibility interfaces for some historic Unix database libraries: dbm, ndbm and hsearch (a System V library for creating in-memory hash tables). Architecture Berkeley DB has an architecture notably simpler than that of other database systems like Microsoft SQL Server and Oracle. For example, like SQLite, it does not provide support for network access — programs access the database using in-process API calls. Oracle added support for SQL in 11g R2 release based on the popular SQLite API by including a version of SQLite in Berkeley DB.[9] . Consequently, as is common with SQLite, there is now 3rd Party support for PL/SQL in Berkeley DB through Metatranz StepSqlite[10] . A program accessing the database is free to decide how the data is to be stored in a record. Berkeley DB puts no constraints on the record's data. The record and its key can both be up to four gigabytes long. Despite having a simple architecture, Berkeley DB supports many advanced database features such as ACID transactions, fine-grained locking, hot backups and replication. Editions The name Berkeley DB encompasses three different products: 1. Berkeley DB 2. Berkeley DB Java Edition 3. Berkeley DB XML Each edition has separate database libraries, despite the common branding. The first is the traditional Berkeley DB, written in C. Berkeley DB Java Edition (JE) is a pure Java database. Its design resembles that of Berkeley DB without replicating it exactly, and has a feature set that includes many of those found in the traditional Berkeley DB and others that are specific to the Java Edition. Since it is written in pure Java, no native code is required. It has a log structured storage architecture, which gives it different performance and concurrency characteristics. Three APIs are available—a Direct Persistence Layer which is "Plain Old Java Objects" (POJO); one which is based on the Java Collections Framework (an object persistence approach); and one based on the traditional Berkeley DB API. The Berkeley DB Java Edition High Availability option (Replication) is available. Note that traditional Berkeley DB also supports a Java API, but it does so via JNI and thus requires an installed native library. The Berkeley DB XML database specializes in the storage of XML documents, supporting XQuery via XQilla. It is implemented as an additional layer on top of (a legacy version of) Berkeley DB and the Xerces library. DB XML is written in C++ and supports multiple language bindings, including C++, Java (via JNI), Perl and Python. Berkeley DB 3 Programs that use Berkeley DB Berkeley DB provides the underlying storage and retrieval system of several LDAP servers, database systems, and many other proprietary and free/open source applications. Notable software that use Berkeley DB for data storage include: • ARC - Advanced Resource Connector open source grid middleware introduced by NorduGrid, use BerkleyDB in several services. • Asterisk PBX - A free/open source PBX. • Bogofilter – A free/open source spam filter that saves its wordlists using Berkeley DB. • Carbonado – An open source relational database access layer. • Cfengine – A free/open source configuration management system, developed by Mark Burgess of Oslo University College. • Citadel – A free/open source groupware platform that keeps all of its data stores, including the message base, in Berkeley DB. • Cyrus IMAP Server – A free/open source IMAP and POP3 server, developed by Carnegie Mellon University • Evolution - A free/open source mail client; contacts are stored in addressbook.db using Berkley DB • Figaro DB - An embedded, native XML Database for the Microsoft's .NET Framework built upon Berkeley DB. • GlusterFS - A cluster filesystem which leverages industry-standard commodity hardware to build highly scalable, non-stop storage systems. • GRAMPS - "Genealogical Research and Analysis Management Programming System", free genealogy software. • HyperGraphDB - An open-source OO graph database based on generalized hypergraphs. • IPD OS - Oracle's Instantaneous Problem Detection OS Tool • Jabberd2 – A Extensible Messaging and Presence Protocol server • J-ISIS (Java ISIS) - A new version of Winisis dtabase programme (of Unesco) being developed for library use • KDevelop – An IDE for Linux and other Unix-like operating systems • KLibido – A free/open source Newsgroup reader tailored for binary downloads • Movable Type (until version 4.0) – A proprietary weblog publishing system developed by California-based Six Apart • memcachedb - A persistence enabled variant of memcached • MySQL database system – Prior to v5.1, MySQL included a BDB data storage backend. • OpenCOBOL - A free/open source COBOL implementation. • OpenDS - an open source directory server. • OpenLDAP – A free/open source implementation of the Lightweight Directory Access Protocol (LDAP) • Papyrus Platform – The ISIS Papyrus Platform uses BerkeleyDB for its distributed object-relational transaction kernel for adaptive content and process management in enterprise applications • PiSi The package management system of Pardus • Postfix – A fast, secure, easy-to-administer MTA for Linux/Unix systems • Redland – A RDF Application Framework can use BDB for persistent storage (triplestore) • RPM – The RPM Package Manager uses Berkeley DB to retain its internal database of packages installed on a system • Scalien Keyspace - A consistently replicated, fault-tolerant key-value store • Spamassassin – An anti-spam application • Subversion – A version control system designed specifically to replace CVS • Sun Grid Engine – A free and open