Using a Metadata Software Layer in Information Systems Integration Mark Roantree1, Jessie B. Kennedy2, and Peter J. Barclay2 1 School of Computer Applications, Dublin City University, Dublin, Ireland [email protected] 2 School of Computing, Napier University, Edinburgh, Scotland Abstract. A Federated Information System requires that multiple (of- ten heterogenous) information systems are integrated to an extent that they can share data. This shared data often takes the form of a federated schema, which is a global view of data taken from distributed sources. One of the issues faced in the engineering of a federated schema is the continuous need to extract metadata from cooperating systems. Where cooperating systems employ an object-oriented common model to inter- act with each other, this requirement can become a problem due to the type and complexity of metadata queries. In this research, we specified and implemented a metadata software layer in the form of a high-level query interface for the ODMG schema repository, in order to simplify the task of integration system engineers. Two clears benefits have emerged: the reduced complexity of metadata queries during system integration (and federated schema construction) and a reduced learning curve for programmers who need to use the ODMG schema repository. 1 Introduction Many database applications require a mechanism by which ‘generic’ applications can determine a database’s structure at runtime, for functions such as graphical browsers, dynamic queries and the specification ofview schemata. This property, often referred to in programming languages as reflection, has been a feature of databases for many years, and the 2.0 specification of the ODMG metamodel [3] has provided a standard API for metadata queries in object-oriented databases. As part ofour research into federateddatabases, we specified and implemented a global view mechanism to facilitate the creation of views for ODMG databases, and the subsequent integration of view schemata to form federated schemata. Please refer to [6,10] for a complete background on federated databases. In this paper we do not concentrate on the topic offederateddatabases but instead focus on the construction of a metadata interface to ODMG information systems. This paper is structured as follows: the remainder of this section provides a brief overview ofthe nature ofour research, the importance ofmetadata to our view mechanism, and the motivation for this research; in Sect. 2 the main concepts in Supported by Forbairt Strategic Research Programme ST/98/014 K.R. Dittrich, A. Geppert, M.C. Norrie (Eds.): CAiSE 2001, LNCS 2068, pp. 299–314, 2001. c Springer-Verlag Berlin Heidelberg 2001 300 Mark Roantree, Jessie B. Kennedy, and Peter J. Barclay this form of research are discussed, together with an informal description of the metadata layer; in Sect. 3 the pragmatics ofthe language are presented through a series ofexamples; in Sect. 4 we present details ofthe implementation; and finally in Sect. 5 we offer some conclusions. In this paper we use the term view (or ODMG view) to refer to an ODMG subschema which may contain multiple classes, and is defined on an ODMG database, or on another view which has been defined on an ODMG database. 1.1 Background and Motivation The main focus of our research was to extend the ODMG 2.0 model to provide views in a federated database environment. This work yielded the specification and implementation ofa global view mechanism, using the ODMG model as the common model for a federation ofdatabases. The concept ofa federationof databases [10] is one where heterogeneous databases (or information systems) can communicate with one another through an interface provided by a com- mon data model. In our case, the common data model is the ODMG model, the standard model for object-oriented databases since 1993 [3]. The most com- mon architecture for these systems is as follows: data resides in many (generally heterogeneous) information systems or databases; the schema of each Informa- tion System (IS) is translated to an O-O format, and this new schema is called the component schema; view schemata are defined as shareable subsets ofthe component schema; the view schemata are exported to a global or federated server where they are integrated to form many global or federated schemata. Our focus was to extend the ODMG model so that it was possible to define the view schemata on top ofeach component schema, and define integration oper- ators which facilitated the construction of federated schemata. This extension provided a layer ofODMG views on top ofthe component schema. However, it was also necessary to provide a mapping language which could bind the compo- nent schema to its local model representation. This facilitates the translation of ODMG queries (to the their local IS equivalent), and enables data transfer to the ODMG database when views are defined. The classes which comprise the database schema are used to model the real world entities which are stored in the database. Additionally, there is a set of metaclass instances which are used to describe the database classes. Thus, we can think ofan ODMG database as having two distinct sets ofclasses: those which reside in the database schema, and the abstract classes (metaclasses) which reside in the schema repository. Whenever we process and store a view defini- tion, a set ofmetaclass instances are stored in the database. Where a view definition involves multiple classes, each with their own extents, this combina- tion ofmeta-objects can become quite complex. Thus, the request to display a view, or extract data from the local IS often requires powerful query facili- ties in order to retrieve the required meta-information. In Fig. 1 the role of the schema repository within a federated database environment is illustrated. Both the Component Database and Federated Database Server are ODMG databases. The schema repository contains a description ofthe database classes, hence the Using a Metadata Software Layer in Information Systems Integration 301 Software Layer Information Systems Layer Descriptions of imported views Metadata and federated view definitions Query (metaclasses) Interface Global Descriptions of Query database classes, Interface S.R. D.S. wrapper definitions to local IS, and view definitions (metaclasses) Federated Database Server Local Vi ew definitions Query Interface ODMG representation S.R. D.S. of local information system (database classes) Component Database mappings S.R. : Schema Repository D.S. : Database Schema Local information system Fig. 1. Metadata architecture within ODMG information systems. arrow towards the database schema. However, in this type ofarchitecture, the schema repository will contain a large amount ofadditional data required by view definitions. This research involved extending the ODMG metamodel in order to construct view schemata. However, due to the complex nature ofboth the base metamodel and extensions, many ofthe OQL queries which were required both to retrieve base and view class metadata would necessitate long expressions. For this reason we specified some language extensions to OQL for the specific purpose of easy retrieval ofmetadata fromthe schema repository. The contribution ofthis work is to provide a software metadata layer which facilitates the easier expression ofODMG metadata queries. In factall metadata queries can be expressed in a single line. In addition, we believe it is possible to improve the performance of metadata queries as some ofour experiments (discussed briefly in Sect. 4) have shown. 2 Metadata Objects and Queries In this section we provide a description ofthe main concepts involved in this research: the metadata objects and the queries used to manipulate metadata. Metadata objects are used to describe both the structural elements ofthe par- ticipating systems (base metadata objects), and the virtual elements which are defined using a view language and are mapped to base metadata objects. Meta- data queries are categorized into groups representing the type ofmetadata in- formation required. 302 Mark Roantree, Jessie B. Kennedy, and Peter J. Barclay d_Scope d_Meta_Object d_Property d_Type d_Constant d_Relationship d_Attribute d_Derived d_Inheritance d_Class d_Ref_Type d_Collection_type d_Primitive_Type Fig. 2. The ODMG metamodel (subset). 2.1 Metadata Elements In Fig. 2 a briefoutline ofthe ODMG schema repository interface(or ODMG C++ metamodel) is illustrated. For integration engineers the main points ofin- terest are classes and their properties, and issues such as scoping and inheritance. The illustration attempts to show a hierarchy ofmetaclasses with the d Scope, d Inheritance and d Meta Object metaclasses at the top ofthe hierarchy. Oth- erwise, all metaclasses derive from d Meta Object (identified by arrows) and multiple inheritance occurs where some metaclasses derive from d Scope.By deriving from a d Scope object, meta-objects can place other meta-objects in- side a (conceptual) container class. For example, a d Class object derives from d Scope, and can use its properties to bind a set of d Attribute objects to it. Specifics ofthe roles ofeach ofthe metaclasses is best described in [5]. In an ODMG database that contains a view mechanism, it is necessary to be able to distinguish between base and virtual classes, and for virtual classes one must be capable ofobtaining the same informationregarding structure as can be obtained for base classes. The view mechanism, its specification language, and implementation