Project no. 004758 GORDA Open Replication Of Databases Specific Targeted Research Project Software and Services GORDA Architecture Definition GORDA Deliverable 2.2 Due date of deliverable: 2006/03/31 Actual submission date: 2006/04/24 Revision 1.1 date: 2007/09/29 Start date of project: 1 October 2004 Duration: 36 Months FFCUL Revision 1.1 Project co-funded by the European Commission within the Sixth Framework Programme (2002-2006) Dissemination Level PU Public X PP Restricted to other programme participants (including the Commission Services) RE Restricted to a group specified by the consortium (including the Commission Services) CO Confidential, only for members of the consortium (including the Commission Services) Contributors Sara Bouchenak, INRIA Alfranio Correia Jr., U. Minho Nuno Carvalho, U. Lisboa Nuno A. Carvalho, U. Minho Emmanuel Cecchet, Continuent Susana Guedes, U. Lisboa Adrian Mos, INRIA Rui Oliveira, U. Minho José Pereira, U. Minho Luís Rodrigues, U. Lisboa Luís Soares, U. Minho Ricardo Vilaça, U. Minho —————————————————— (C) 2006 GORDA Consortium. Some rights reserved. This work is licensed under the Attribution-NonCommercial-NoDerivs 2.5 Creative Commons License. See http://creativecommons.org/licenses/by-nc-nd/2.5/legalcode for details. Abstract This document describes the GORDA Architecture, which enables independent development of database management systems (DBMS) and database replication systems. The architecture builds on the classic database management system’s architecture with remote database access, a standard call-level interface and sup- porting SQL requests. The proposed architecture models the system through two articulated plans: one that reflects the standard transaction pipeline and the other a set of contexts reflecting the scope of the various operations through the transac- tion processing pipeline. The replicated database management system architecture is complemented by a generic management architecture. Contents 1 Introduction 2 1.1 Relationship With Other Deliverables . 2 2 Assumptions and approach 4 2.1 Generic database architecture . 4 2.2 Relevant standards . 5 2.3 Design principles . 6 3 Architecture 7 3.1 Reflective architecture for replication . 7 3.1.1 Target Reflection Domain . 7 3.1.2 Processing Stages . 9 3.1.3 Processing Contexts . 10 3.1.4 Base-level and Meta-level Calls . 11 3.1.5 Design Patterns . 12 3.2 Generic architecture and composition scenarios . 12 3.3 Architecture components and usage . 14 3.3.1 Reflection and Replication . 14 3.3.2 Management . 17 3.3.3 Recovery and security aspects . 19 1 Chapter 1 Introduction This document describes the GORDA Architecture, which enables independent development of database management systems (DBMS) and database replication systems. The document is structured as follows. Chapter 2 summarizes concepts and assumptions, referring to available standards where appropriate. This includes de- scribing the architecture of a non-replicated DBMS that serves as a baseline for comparison. Chapter 3 describes the generic GORDA Architecture, its compo- nents and relations. It then describes several concrete refinements for different implementation scenarios and shows how to use the presented architecture in a specific use case scenario. We do not include information on database management systems, communica- tion protocols, or tools themselves. 1.1 Relationship With Other Deliverables This document, along with its companion deliverable D2.3 - APIs Definition Re- port, refines deliverable D2.1 - Preliminary Architecture and API and serves as the basis for most of the development of workpackages 3 (Replication Protocols) and 4 (Database Support). With respect to the preliminary report D2.1, it obsoletes its sections 2 and 3 (the companion document D2.3 does the same for the remaining sections 4 and 5). Since the general architecture and guidelines proposed in D2.1 have been con- sidered to be correct, the current document makes only minor updates that reflect experience earned while implementing both the replication protocols and database support components. Namely: • A new processing context, Database Context, has been added to support mul- tiple active databases within the same DBMS. • The Statement Context has been renamed Request Context and its meaning changed slightly to better handle different database implementations. 2 • The meaning of the Physical Storage Stage has been changed. While initially it was supposed to allow synchronization of commit requests, it is now used only to inspect logs after transaction commit. Synchronization with commit requests is now done through the Transaction Context. • The scope of processing contexts has been changed to be consistent with the revised definition of the Physical Storage Stage. 3 Chapter 2 Assumptions and approach 2.1 Generic database architecture Management Application Driver DBMS Application Driver Figure 2.1: Generic database architecture. The GORDA Architecture builds on the assumption of the generic database architecture with remote database access, a standard call-level interface, and SQL as shown in the Figure 2.1. The main components of this model are: • The Application, which might be the end-user application or a tier in a multi- 4 tiered application. • The Driver provides a standard call-level interface (CLI) for the application and remotely accesses the database itself using a communication mecha- nism. The communication protocol is hidden from the application and can be proprietary. • The DBMS holds the database content and handles remote requests expressed in standard SQL to query and modify data. • Management tools are able to control the Driver and DBMS components independently from the Application using a mixture of standard and propri- etary interfaces. Further assumptions on these systems are that: • The call-level interface and SQL should not be changed, and cannot be changed at all in a backward incompatible manner. • Some DBMS implementations can be modified in a backward compatible manner, but some others cannot be modified at all. • The remote database access protocol should not be changed to maintain com- patibility with third party tools. • The driver can easily be changed with minor impact. This simple architecture can easily be mapped to a Java system, using JDBC as the call-level interface and driver specification, any remote database access protocol encapsulated by the driver and a DBMS, and an external configuration tool for the JDBC driver. 2.2 Relevant standards The GORDA Architecture and Programming Interface (GAPI) are based on exist- ing data management standards. Namely: • ISO/IEC 10032:1995 - Reference Model of Data Management (RMDM) specifies typical data management architectures and nomenclature. • ISO/IEC 9075-3:1995 - Call Level Interface (SQL/CLI) and X/Open XA Distributed Transaction Processing (DTP) specify client interfaces. • ISO/IEC JTC1/SC32 working drafts on Distributed Database Access and Schemas for client information. The rendering of interfaces in the Java language is therefore based on existing implementations of such standards in the Java platform, in particular, in Java En- terprise Edition (JEE). 5 2.3 Design principles The design of the GORDA Architecture stands on the following general principles: • Independence of operation and configuration. All configuration interfaces are assumed to be out of the scope of the present specification. Configu- ration of components and relevant parameters is available by means of an embedded directory service and the factory design patterns. • Variable geometry interfaces. Each implementer is free to provide only a subset of the whole GAPI that is adequate for each situation. Each compo- nent should therefore explicitly state requirements and check for the avail- ability of all requirements. This is however part of configuration and thus out of the scope of this specification. • Facade interfaces that allow manipulation of the internal state of the DBMS without forward and backward format conversions. Conversion to a DBMS independent representation if necessary is achieved by an optional layer on basic interfaces. 6 Chapter 3 Architecture The GORDA Architecture specifies the building blocks for a replicated DBMS. At an abstract level, these building blocks can be mapped to existing monolithic implementations regardless of the implementation. The GORDA Architecture is however the basis for the definition of interfaces between such components that allows them to be reused in different contexts. This Section discusses the GORDA architecture. Implementation issues are discussed in the Deliverables D4.3, D4.4 and D4.5 of the project. 3.1 Reflective architecture for replication In this section we outline the reflection usage in the GORDA architecture, as well as the underlying rationale. The GORDA architecture considers a multi-stage model of transaction processing that can be replicated by observing and modify- ing it through a reflector interface. The main intent of the reflection is to allow the replication component to observe the state of the DBMS. Specifically, the re- flector exposes transaction-processing concepts such as parse trees, write sets, or transactions as first class entities in the target programming language. Replication protocols can register for significant events to be notified of relevant state transi- tions and call methods to alter the state. 3.1.1 Target Reflection Domain Existing reflective facilities in database management systems are targeted at application programmers using a relational model. Its