Development of a Database Abstraction Layer
Master Thesis by Eelco Heerschop, 2004 Supervised by: Drs. J. H, Jongejan & Ir. S. Achterop
RuG Development of a Database Abstraction Layer
Master Thesis by Eelco Heerschop, 2004 Supervised by: Drs. J. H, Jongejan & Ir. S. Achterop
RuG F
Development of a Database Abstraction Layer
from http://adodb.com:
We are very sorry to say, we lost our entire database.
We have roughly 1000uniqueand returning visitors per day and apologize to all of you for this interruption.
If you have any ideas please email me
jason ®idma.com
from http://msdn.microsoft.com:
To add to the confusion, Microsoft introduced another data-access object model: ADO
Page 2 of 92 Development of a Database Abstraction Layer
Abstract
Databases play an important role in the current information based societyand it will even grow more and more important in the nearfuture. This report describes an approach to create a more abstract model ofdatabases and its connections in order to make database connections truly independentof the database management systems used.
The thesis consists of three parts. The first part of this thesis consistsof the problem definition and an analysis of the most popular database access technologiesused today. The second part describes the creation of an abstract model for accessingdatabases and the third and final part describes a practical implementation of thismodel in the Java programming language.
Before we proceed with the problem definition, we start with a smallintroduction to get more familiar with the world of databasesand computer architectures.
Page 3 of 92 Development of a Database Abstraction Layer
Table of Contents
Abstract 3 Introduction 6 Part I, problem definition and analysis 10 Problem definition 10 Analysis of methods for database access 11 ODBC 11 ADODB 12 JDBC 13 CORBA 15 DCOM 20 Part II, a solution by abstraction 22 Object oriented approach 24 System overview 31 The database abstraction layer 31 The abstract database 32 The database table model 32 The database tree model 32 Part III, a practical implementation 34 Choosing a programming language and programming tools 34 Implementation 36 The abstract database 36 The database abstraction layer 37 The database drivers 38 The database explorer 41 Conclusions 43 History 43 Glossary 44
Page 4of 92 Development of a Database Abstraction Layer
References 46 Appendices 50
A Developers guide 50 Overview 50 Getting started 50 Registering database drivers 51 Creating connections 52 Gathering information 53 Executing statements 54 Viewing results 56 A programming example 57 Developing database drivers 62 More information 69
B User guide 70 Overview 70 Getting started 70 Managing database connections 71 Browsing 73 Executing commands 74 Viewing results 75 The message window 76 More information 77 C Source code (partial) 78 AbstractDatabase 78 DatabaseAbstractionLayer 84 DatabaseTableModel 87 DatabaseTreeModel 90
Page 5of92 Development of a Database Abstraction Layer
Introduction
TV, newspapers,magazines,Internet...Intoday'ssociety,people tendtoget overwhelmed by information. Information, the key property of today's society. All those huge quantities of information must be managed in some way. Users of information sources want to be able to retrieve, store and search information. This can be achieved by databases or more correct (after all databases are basically just a place to put your stuff in): by database management systems(DBMS'). People use databases more and more, often without even knowing they are using them. If you visit an ATM, search the Internet or shop in your local supermarket, in all cases databases are involved. Therefore, databases are playing an increasingly important role. In earlier systems centralized architectures (see figure1) were used, which operated on mainframe computers to provide processing for all the functions, including user applications, user interfaces, as well as all the DBMS functionality. The reason for using centralized systems was that most users accessed such systems via computer terminals that did not have any processing power and only provided display capabilities. Gradually, most users replaced their computer terminals with personal computers that had their own processing power. At first database management systems did not make use of this huge potential of processing power, but that changed with the introduction of a new architecture: the client- server architecture.
Display Display Display Display T-minais Monitor Monitor Monitor Monitor j
Mainframe
Figure 1: model of a centralized database architecture
'Throughoutthis thesis lots of abbreviations will be used. An extensive list of all abbreviations and its description can be found in theglossary.
Page 6 of 92 Development of a Database Abstraction Layer
The client-server architecture (see figure 2) is built on a framework of many computers connected via local networks and other types of computer networks. A client in this framework isa user machine that provides (graphical) user interfaces andlocal processing. A smaller number of computers in this framework are so called servers. A server is a machine that basically provides services to the client machines,such as printing, archiving, or database access.
cilent Server
Figure 2: model of a client-server architecture
Although most systems are still based on the client-server architecture, we see another architecture rising in recent years: 3-tier architectures or multi-tier systems (see figure 3). In a 3-tier architecture functions are even more separated than in a client-server architecture. The client and server cannot connect with each other directly, but are separated by the middleware layer. Conceptually the middleware processes requests from the client and can pass them on to the (database) server. Responses from the (database) server are then interpreted again, and the filtered and formatted data ispassed on to the client again. As one already might guess, the very existence of this architecture lies in the exponential growth of the Internet and the shift from static to dynamic web pages. If you look closer at a typical implementation of a 3-tier architecture, the client part of such an architectureis just a standard web browser like Mozilla or Internet Explorer, the (database) server side is some sort of DBMS and the middleware consists of a web server, usually Apache or uS.Most web servers are capable of utilizing scripting languages like JSP, PHP and ASP that make it easy to connect with databases and render dynamic web content. One might take notice of the fact that the role of the client has become less important again. You can regard the client, which is only running a web browser, as a dumb client, just as it used to be in the first generation of database management systems.
Page 7of92 Development of a Database Abstraction Layer
Client Det.baae Server
webbrow*er
[
Web Server answer http result
Figure 3: model of a 3-tier architecture
The 3-tier architecture arose as a merger of two technologies: databasetechnology and networkand data communication technology. The latter has made tremendousstrides in terms of wired and wireless technologies includingsatellite and mobile communications and the standardization of protocols like Ethernet, TCPIIP and ATM. Currently, we can see an immense increase in both the usage of dynamic content1but also in the absolute number of pages viewed on the World Wide Web. Nowadays,both web servers as well as web browsers are matureproducts. This development has not passed companies unnoticed. Web servers are used more and more on internal company networks, not only for providing informationtotheir employees, but even for administration purposes. And an average employee cannot do without a webbrowser anymore. Not that long ago the slogan —orvision —ofthe world's leading manufacturer of software products was: "one PC at every desk". In recent years they changed their slogan to: "Connect anything with everything" and they are not the only one. Even one of the largest photo camera manufacturers in the world has been caught by the networking madness and advertises with this phrase: "Imaging across networks". In short., we see that an increasing amount of data is stored and accessed using network connections. One of the more recent developments is the usage of databases by 'ordinary' programs that before did not make use of databases. For example, new office and iDE (Integrated Development Environments) suites make use of databases for storage, often without telling the end-users of those products. Databases might even be used by file system drivers in order to make fast storage, retheval and searching of files possible on the increasingly growing capacity of hard drives. Because current database architectures are not based on an all-in-one proprietary system, but are based on client-server or client-middleware-server architectures, methods are needed for accessing data from these database management systems. All DBMS vendors have developed their own methods for providing database connectivity including
Page 8of92 Development of a Database Abstraction Layer complete proprietary languages, systems for creating user interfaces and APIs such as ODBC, JDBC, .NET and ADO/OLEDB. One hastoremark that DBMS vendors have the tendency to invent the most beautiful abbreviations and names for the simplest things. Sometimes they even give their own names to other (well known) products. For example: "The OracleH77P Server provides a productive and high performance Web server environmentfor Oracle9i Application Server", while in fact this so called 'special' web server is just the popular Apache web server. In the past, developers had to add code to their applicationthat talks to a particular database using a proprietary language, but in recent years this has changed with arrival of non-proprietary APIs. We will discuss the benefits and drawbacks of these APIs later. Today, we can connect our newly created application with a database in lots of different ways and every DBMS provides developers with their own wayof connecting. While this is not necessarily wrong, we would like to interface with our databases in a better and easier way. We would like to use our databases in such a way that we do not have to know anything about the DBMS or even about the specific API used.
Page 9 of 92 Development of a Database Abstraction Layer —PartI
Part I, problem definition and analysis
Problem definition
We want to be able to connect to databases in an absolutely transparent manner, meaning that we are able to communicate with our databases in exactly the same way independent of the actual APIs and database management systems used. This goal can be realized by the creation of an abstract model of a database. Before we start with the creation of such a model, we first analyze a number of popular APIs for providing database connectivity followed by an analysis of more generic systems for sharing information. The second part describes the creation of an abstract model and the last part of this thesis describes a practical implementation of the created model in the Java programming language, which is suitable for both the client-server architecture as well as the 3-tier architecture.
Page 100192 Development of a Database Abstraction Layer —PartI
Analysis of methods for database access
The obvious way to reach the goals described in the previous section is by inserting a new middleware layer. By middleware we mean software that connects twootherwise separate applications. Middleware serves as the glue between those applications. (See figure 3 where the web server acts as a middleware layer.) The first step in the development of our new middleware layer is taking a closer look at where we will insert itin. Middleware is nothing new: in most cases we will connect to some kind of middleware instead of the DBMS directly! Many different API's for connecting with databases have been developed, including ODBC, JDBC, NET and ADO/OLE DB. Besides middleware that specifically has been developed to interact with databases, also more generic middleware exists. Examples are CORBA andMicrosoft's Component Object Model. Both are examples of Distributed Object Systems. Distributed Object Systems are systems that allow applications to use objects on remote systems, as if the objects were local. Infact,in some systems there is absolutely no difference in programming for a local object versus a remote one. One might ask why a new middleware layer is needed. After all, there is enough to choose from. This is exactly why! We would like to have one generic abstraction, independent of the actual interface and version used. Vendors have the tendency to bring out new application programming interfaces or new versions of those interfaces every once in a while, making it hard for developers to keep their applications up to date. Another advantage of using one generic abstraction is the ability to bridge many different database connections over different application programming interfaces. To get a bit more familiar with database connection technologies and Distributed Object Systems, we first take a look at a few popular application-programming interfaces, followed by a closer look at two examples of Distributed Object Systems.
ODBC
ODBC is the acronym for Open Data Base Connectivity, a Microsoft Universal Data Accessing standard that started life as the Windows implementation of the X/Open SQL Call Level Interface specification. Since its inception in 1992 it has rapidly become the industry standard interface for developing database independent applications. It is also the emerging standard interface for SQL based database engines replacing many of the first generation Embedded SQL and proprietary call level interfaces provided by database engine and database connectivity middleware vendors alike. In other words: ODBC isan open application-programming interface that allows developers to access a database in a predictable way. When writing code to interact with a database, you usually have to add code to your application that talks to a particular database using a proprietary language. If you want your program to talk to Access, SQL- Server and Oracle databases you have to code your program with three different database languages. However, when a developer uses ODBC, he only needs to talk the ODBC language (a combination of ODBC API function calls and the SQL language). The ODBC Manager will figure out how to connect with the database you want to use. From a programmer's point of view, databases are not directly available through the ODBC
Page 11 of92 Development of a Database Abstraction Layer —PartI driver, but only indirectly as so-called Data Sources that have to beconfigured in the ODBC Manager. Since version 2.0, the ODBC standard supports SAG SQL. The listing below shows an example how database can be accessed usingODBC.
Dim conn As New rdoconnection Dimrs AS rdoRecordset
'Attempt to create a database connection conn.Connect = "Driver={SQL Server); Server=MyServer; " &— "Database=Pubs;Uid=sa; Pwth" conn. EstablishConnectiOfl
'Execute query conn.Execute("SELECT* FROMCustomers")
'While more rows exist, print customer id's WHILE NOT rs.EOF Response .Write (rs (?CustomerlD?) .Value) rs .MoveNext() END WHILE
Listing1: code that uses ODBC in VisualBasic2
ADODB
Another programming interface to access data in a database is ADO DB.ADO is a Microsoft technology that stands for ActiveX Data Objects and is —surprisingly-a Microsoft Active-X component. In terms of usage, ADO looks very similar toODBC. This comes very clear if we compare listing 1 and 2. However, ADO DBis focused more towards the creation of dynamic web content. This explains why ADO isautomatically installed with Microsoft uS.
Dimconn AS New ADODB.Connection() Dimrs AS ADODB.Recordset
'Attempt to create a database connection 1 ;UserID=sa;mi tialCatalog=Northwind; corin .Open( Provider=SQLOLEDB.
DataSource=hOst; )
'Execute query rs= conn.Execute("SELECT* FROMCustomers") 'While more rows exist, print customer id's WHILE NOT rs.EOF Response.Write(rS(?CuStolflerlD?) .Value) rs .MoveNext() ENDWHILE
conn =Nothing rs = Nothing
[13j Listing 2: codethat uses ADOin .NET
Page120192 Development of a Database Abstraction Layer —PartI
JDBC
JavaDatabase Connectivity (JDBC) provides Java developers with a standard API that is used to access databases. JDBC has capabilities to connect to a database and to retrieve the results of a query using the Java classes Connection, Statement and ResuitSet respectively. JDBC tries to present a uniform interface to databases —afterchange of database management system your applications only need to change their driver. Unfortunately this is not entirely true, because of the different versions of the API and the different implementations of the actual JDBC drivers. Not all drivers act in the same way under certain circumstances. Typical differences between drivers lie in the methods connections with the DBMS are being made, errors are handled and result sets are accessed. Another important difference is the availability and correctness of the (meta) information about a DBMS and its databases.
JDBC driver implementations fit into one of four categories [*4]:
1.A JDBC-ODBC bridge provides JDBC API access via one or more ODBC drivers. Note that some ODBC native code and in many cases native database client code must be loaded on each client machine that uses this type of driver. Besides the JDBC-ODBC bridge driver provided by Sun, there are also several commercial implementations.
2. A native-API (partly Java technology-enabled) driver converts JDBC calls into calls on the client API for Oracle, Sybase, Informix, DB2, or other DBMS. Note that, like the bridge driver, this style of driver requires that some binary code be loaded on each client machine.
3. A net-protocol (fully Java technology-enabled) driver translates JDBC API calls into a DBMS-independent net-protocol which is then translated to a DBMS protocol by a server. This net server middleware is able to connect all of its Java technology-based clients to many different databases. The specific protocol used depends on the vendor. In general, this is the most flexible JDBC API alternative. It is likely that all vendors of this solution will provide products suitable for Intranet use. In order for these products to also support Internet access they must handle the additional requirements for security, access through firewalls, etc., that the Web imposes. Several vendors are adding JDBC technology-based drivers to their existing database middleware products.
4. A native-protocol(fullyJavatechnology-enabled)driverconverts JDBC technology calls into the network protocol used by DBMSs directly. This allows a direct call from the client machine to the DBMS server and is a practical solution for Intranet access. Since many of these protocols are proprietary the database vendors themselves will be the primary source for this style of driver.
Page 13 of 92 Development of a Database Abstraction Layer —PartI
One major advantage of JDBC is the availability of plenty of drivers that support many popular databases. In many cases a JDBC driver that works specifically for one DBMS is available. If not, Sun provides a driver that is compatible with ODBC, so connections to any ODBC compliant DBMS should be possible. In recent years,commercial JDBC drivershave become availableforconnectingwith ODBC compliant database management systems directly too.
Listing 3 shows an example of making a database connection and execution a query using JDBC.
//Attempt to load database driver try Class.forName("sun.jdbc.odbc.JdbCOdbCDriVer') .newlnstanceU;
} catch (ClassNotFoundExceptiOn cnfe) ( System.err.println(Unable to load database driver: +cnfe); Systein.exit(O);
// Now attempt to create a database connection String un ="jdbc:odbc:localhost; Connection conn =DniverManager.getConnectiofl(Url,uu,nsqln); //Create a statement to send SQL Statement stint =conn.createStatementO;
IIExecutequery ResuitSetrs =stmt.executeQuery(select *fromCustomers);
// While more rows exist, print customer id's while (ns.next()
{ System.out.println(1D : " + rs.getlnt(CustomerlD));
Listing3: code that uses JDBC in Java1
Page 14 of 92 r
Development of a Database Abstraction Layer —PartI
CORBA
Specified by the Object Management Group, CORBA is the acronym for Common Object Request Broker Architecture. CORBA is an architecture for an open software bus. The central component of this architecture is the Object Request Broker (ORB), on which object components written by different vendors can interoperate across networks and operating systems without knowing where the objects they access reside or in what language the requested objects are implemented. The ORB interacts and makes requests to those objects and negotiates between request messages from objects or object servers and the affiliated data sets.
The most important part of the OMG"7 specification isthe Interface Definition Language (ll)L)8 which is used to define the interfaces to CORBA objects. CORBA objects are quite different from typical programming language objects because CORBA objects can be located anywhere on a network and can operate with objects written in other languages or on other platforms.
Since CORBA was first introduced in 1991, two new versions of the specification followed. Nowadays several implementations are available including free (open source) implementations on a variety of platforms. However, not all implementations provide the same level of functionality and robustness.
The idea of sharing objects across networks, platforms and programming languages is not a bad idea at all. Therefore it is worthwhile to look a bit deeper inside CORBA. CORBA applications are composed of objects, individual pieces of software that combine functionality and data. Typically, there are many instances of an object of a single class, all identical in functionality but differing in that each has its own state. For each class one has to define an interface in the Interface Definition Language. Interface definitions are independent of programming language, but they do have to be compiled to the actual programming language used. Currently there are standardized mappings from the JDL to C, C++, Java, COBOL, Smalitalk, Ada, Lisp, Python, and lDLscript. Compiling an interface generates two implementations of the defined interface. The first one is the client stub, which is the interface used by applications to perform operations on the object and the second one is the object skeleton, which serves as a framework for the actual implementation of the object. At the moment a client wants to make use of a CORBA object, it has to use the objects' DL interface (implemented in the client stub) to specify which operations it wants to perform. Requests from the client are routed through the ORB, which uses the same DL interface. When requests are processed by the object implementation, the results are sent back through the ORB using the same interface again. (See figure 4)
Page 15 of 92 Development of a Database Abstraction Layer —PartI
Figure 4: A request passing from client to object implementation
In CORBA, every object instance has its own unique object reference ID. Clients use the object references to direct their invocations, identifying to the ORB the exact instance they want to invoke.
An important feature of CORBA is its interoperability between ORBs. A request can be routed from one ORB to another ORB when the first ORB detects a request cannot be handled locally. This so-called 'remote invocation' is achieved by agreeing on a common protocol. Although other protocols could be used for this task, the 0MG has defined a standard protocol for the ORB-to-ORB communication: flop.
From the client's point of view, remote object invocations are no different from normal object invocations. Because hOP tunnels the request and results from and to the ORBs transparently, there is no need for a central server. Instead data and functionality can be distributed transparently across networks. (See figure 5)
r0iImplementation [1 IDL Skeleton Skeipton
'lop Request--- Protocol Object Request Broker 1 Object Request Broker 2 I Figure 5:Interoperabilityusing ORB to ORB communication
Page 16 of 92 Development of a Database Abstraction Layer —PartI
Developing CORBA applications is quite complicated and even the famous "HelloWorid" example requires a couple of files to be implemented. The first step is the definition of the interface (listing 4).
module HelloApp
interface Hello
string sayHello 0;
Listing 4: definition of the interface for the HelloWorid example (Hello.idl)L'9J
Next, the IDL compiler is used to generate the stubs and skeletons. As stated before, we can map the interface to different programming languages, butfor simplicity we only perform a mapping to the Java language. To do this, we perform the command: iditojava Hello.idl which generates five files including a mapping of the interface to the Java programming language (listing 5).
packageHelloApp; public interfaceHello extends org.omg.CORBA.Object { String sayHellou; ) Listing 5: the interface mapped to the Java programming language (Hello.java)
If we compare listings 4 and 5,wecan easily see how the 1DL statements map to the generated Java statements. (table 1)
LDL Statement Java Statement module HelloApp package HelloApp; interface Hello public interface Hello string sayHelloO; String sayHello; Table 1: mapping of DL to java
After the compilation of the interface to the target programming language, we proceed by implementing the server part of the program (Listing 6)
II Step1: import required packages II The package containing our stubs. import HelloApp. *; //HelloServer will use the naming service. import org. omg .CosNaming.*; IIThe package containing special exceptions thrown by the name service. *; import org. omg .CosNaming .NamingContextPackage. //All CORBA applications need these classes. import org.omg.CORBA.*;
Page 17of92 Development of a Database Abstraction Layer —PartI
//Step 2: declare the serverclass: public class HelloServer
II Step 3: define the main() method: public static void main(String args[])
IIStep4: handle CORBA system exceptions try //Step 5: create and initialize the ORB ORB orb =ORB.init(args,null);
IICreatethe servant and register it with the ORB HelloServant helloRef =newHelloServant(); orb.connect(helloRef);
IIGetthe root naming context org.omg.CORBA.Object objRef = orb.resolve_initial_references("NameService"); NamingContext ncRef =NamingContextHelper.narrow(objRef);
IIBindthe object reference in naming NaineComponent nc =newNameComponent C"Hello", " NatneComponentpath(] = (nc); ncRef.rebind(path, helloRef);
IIWaitfor invocations from clients java.lang.Object sync =newjava.lang.ObjectO; synchronized ( sync) sync.waitO;
} catch(Exception e) ( IIStep4, continued System.err.printlriY'ERROR: " + e); e .printStackTrace(Systein.out);
}
I/Step6: manage the Servant object II note:the HelloServer needs a HelloServant. II Theservant implements the interface generated by II idltojavaand actually performs the work of the II operationson that interface. The servant will be II instantiated by the HelloServer class HelloServant extends HellolmplBase
public String sayHello()
{ return"\nHello world !!\n";
uiUj Listing6: implementation of the server (HelloServer.java)
Page18 of 92 F
Development of a Database Abstraction Layer — Part I
Thefinalstepinour sample implementation is the creation of the client: importHelloApp.*; II The package containing our stubs. import org.omg.CosNaming.*; II HelloClient will use the naming srv. import org.omg.CORBA.*; // Required CORBA classes.
public class HelloClient
public static void main(String args[J)
try
If Createand initialize the ORB ORB orb =ORB.init(args,null);
// Get the root naming context org.omg.CORBA.Object objRef = orb.resolve_initial_references("NameService"); NainingContext ncRef =NamingContextHelper.narrow(objRef);
IIResolvethe object reference in naming NaineComponent nc =newNameComponent(Hello', U); NameComponentpath[] = {nc}; Hello helloRef =HelloHelper.narrow(ncRef.resolve(path));
II Call the Hello server object and print results String Hello =helloRef.sayHelloi;
System .out .printin(Hello)
catch(Exception e) C
System.out.println(ERROR : +e); e .printStackTrace(System.out)
Listing7: implementation of the client (HelloClient.java) u' U
Aftercompilation of the source files, we can finally test-drive our application by starting the client and server. The output of the client should be: "Hello World! !"(Thismust be the most complicated "Hello World" in the world!)
Page19of92 Development of a Database Abstraction Layer —Part!
On paper CORBA looks very promising and well suited for our needs, but nevertheless there are several disadvantages:
•Complex —becauseof the fact that CORBA tries to be entirely platform and language independent relying on the Interface Definition Language, the interfaces and generated stubs and skeletons can become really complex. The development time is therefore rather high too. •Slow improvement —committeesgenerally tend to move slowly, from specification to implementation currently takes 18 months. • Cost —althoughsome free implementations do exist, most are not free at all. On the contrary: ORBs for various platforms are very expensive.
We can conclude that platform independence and cross platform communication is surely possible with CORBA, but at a high price: rigid structures and a long development time.
DCOM
DCOM is an extension of Microsoft's Component Object Model (COM) that allows interaction between objects executing on separate hosts in a network. The original COM model provides a framework for dynamically integrating components that interact within a single address space or between processes on a single host. The implementations of these components were packaged in Dynamic Link Libraries (DLLs). COM is essentially a binary integration scheme. It adopted the structure of C++ virtual function tables as the binary representation of an interface. COM defines an APItoallow for the creation of components for use in integrating custom applications or to allow diverse components to interact. However, in order to interact, components must adhere to a binary structure specified by Microsoft. As long as components adhere tothis binary structure, components written in different languages can interoperate(121•Alot of Microsoft's client services such as Microsoft Transaction Server (MTS), ActiveX and OLE depend on the COM infrastructure as shown in figure 6.
MTS Components I
Compound Documents (OLE)
MTS
COMInfrastructure Figure 6: Component Object Model Architecture
Page 20 of 92 Development of a Database Abstraction Layer —PartI
DCOM shares a lot of similarities with CORBA. Both provide interface definition languages, but their roles are fundamentally different. While the CORBA IDLis rigorously defined and a fundamental part of CORBA, Microsoft's IDL is only a 'tool'. It is only one of several ways to define COM interfacesU13)• COMis a typical example of a desktop architecture and extending it to an enterprise architectureis problematic. Over the years COM has changed dramatically inits evolution from compound documents to distributed objects. The changes in design have not been graceful and really show the ad-hoc nature ofMicrosoft's solutions. COM is fundamentally not a well-partitioned architecture and relies on a key optimization for a single language and platform.
Page21 of 92 Development of a Database Abstraction Layer —PartII
Part II, design
A solution by abstraction
Before we proceed with a possible solution for our problem we summarize the first chapters briefly. We have seen that there are several APIs for accessing databases. Access can be either directly or through some form of middleware (for example: through an ODBC manager). We have also seen that there are more generic ways of exchanging information. In this case we do not talk about sharing data, but about distributing components or objects that include both functionality as well as data. An example of an architecture that uses the principles of the distribution of objects is CORBA. We have also mentioned before that we want to achieve our goals by two key concepts: abstraction and middleware. Abstraction is an important aspect, because it enables us to hide actual implementations and APIs from the outside world. The solution will be in the form of middleware, because the new software layer will be positioned right in the middle of user-applications and DBMS drivers, gluing everything together.
Now, let's take a closer look at the position where our new middleware fits in. If we look at figure 7 below, we can see our "database abstraction layer" positioned right between the application and the middleware supplied by DBMS vendors. Therefore, the developers of a database-connected application don't have to pay attention to a specific DBMS or interface anymore.
ckent Midewaze Sever
Figure 7: overview of a DAL connected application
Page 22 of 92 Development of a Database Abstraction Layer —PartII
So far, we have looked at different database connection technologies and atthe place where our new middleware will be positioned. If we compare the examplesabove, one has to remark that the APIs all look very similar. Of course, we canonly use one programming language at the time, but still, a truly independent interface for connecting with databases, regardless of the database drivers and DBMS used should be possible.
Page 230192 Development of a Database Abstraction Layer —PartII
An Object Oriented Approach
Instead of just creating a new plain function-oriented application-programming interface, we are going for another approach: the object-oriented approach. Object-oriented programming has its roots in the SIMULA programming language. The version of SIMULA that was introduced in 1967 offered most of the key concepts of object-oriented programming such as objects, classes and subclasses. In SIMULA, the concept of a class groups together the internal data structures of an object as well as the implementation of its functionality. The next step in the development of object-oriented languages was SMALLTALK. This language developed at Xerox PARC in the 1970s was the first real programming language explicitly designed to be object-oriented. The introduction of C++ in the early 1980s extended C with the key concepts of SIMULA. However, the first commercial version of C++ became only available in 1995 and it took until 1998 before an ISO standard version became available51•Probably the best example of a present- day object-oriented language is Java. In 1991 James Gosling worked on embedded systems software at Sun Microsystems. Because of the frustrations he had working with C++ he began developing a language called "Oak" to be a safe, object-oriented systems language. Although he was frustrated working with C++, the resulting language was still strongly influenced by it. By 1993 this language had been renamed to Java and several prototype devices using Java where available. The market however didn't seem to be interested. Around this time, the usage of the Worldwide Web grew extensively and Sun began to see uses for Java's small, safe and platform independent code in the internet community. Today many people still associate Java with the internet. The first official 1.0 release was in 1995 and Java has continually been improved and adapted at new requirements. At the time of writing, the newest version is 1.4. Java is different from other languages in the sense that programs written in Java on one hand have to be compiled and on the other have to be interpreted at runtime. This is because the Java compiler does not translate the program to machine code, which can be executed directly by the computer, but to Java byte code. Although somewhat slow, this code can be executed by a Java interpreter. Nowadays there are Java interpreters for almost every platform imaginable including PDA's and cellular phones.
It is very difficult to give an exact description of what an object-oriented programming language is. According to Webordia. an online encyclopedia on computer related terms, object oriented programming is
"A type of programming in which programmers define not only the data type of a data structure, but also the types of operations (functions) that can be applied to the data structure. In this way, the data structure becomes an object that includes both data and functions. In addition, programmers can create relationships between one object and another. For example, objects can inherit characteristics from other objects."
Thus, according to the definition above, an object-oriented language must support the programmer in such a way that both data structures and operations on those data structures can be defined. I think this is a rather limited definition, because this makes nearly every programming language a potential object-oriented language.
Page 240192 Development of a Database Abstraction Layer —PartII
Bjarne Stroustrup, the developer of C++ states(221:
"Ifthe term "object-oriented programming language means anything it must mean a programming language that provides a mechanism that supports the object-oriented style of programming as well."
Maybe it is better to state that object-oriented programming languages are onlythose languages that actively support object oriented programming. This does not mean that every program written in an object-oriented languageis an object-oriented program. Moreover(2.
•A design can be Object-oriented, even if the resulting program isn't. •A program can be Object-oriented, even if the language it's written in isn't
But what then makes programming languages object-oriented? What characteristics must a programming language have toactivelysupport object-oriented programming? [2,2-5, 2.6] Summarizing thefollowing features should be supported by an object-oriented programming language:
•Autonomous entities called objects •Interaction by messaging, with no assumption of implementation •Object organization, including an inheritance mechanism •Programs as models
Objects are autonomous software entities modeled after real-world objects in the sense that they both combine state and behavior. An object's state is maintained in its variables and its behavior is implemented in its methods.
Methods (behavior)
Variables (state)
[271 Figure8: visual representation of a software object
Asingle object does not offer much more functionality compared to a normal program. However, it gets more interesting if we let several objects communicate with each other. Objects are able to interact with each other by sending messages. Because the object's methods define its behavior, message passing supports all possible interactions between objects without direct manipulation of the variables. Message passing makes it possible for objects to communicate even if they do not reside in the same process or even on the same machine.
Page 25 of 92 Development of a Database Abstraction Layer —PartII
Object B
Object A
Figure 9: messaging between objects[2-81
An object has a public interface that other objects can use to communicate with it. But the object can maintain private information and methods that can be changed at any time without affecting other objects that depend on it. Other objects do not have to know how an object is implemented in order to use it. Thus, abstraction, an important element of our abstract database layer, arises naturally with object-orientated programming.
Objects are organized in hierarchical structures and a proper inheritance mechanism should be made available by the programming language. Inheritance provides a powerful and natural mechanism for organizing and structuring software programs. In object- oriented programming inheritance means the ability of one object to be defined in terms of other objects. Although each descendant inherits both state and behavior of its ancestor, new state and behavior can be added easily and inherited methods can also be overridden.
Anc.stor
D.sc.r,d.nt I Ds.csndsnt
Figure 10: schematic representation of the inheritance mechanism[29j
Using the inheritance mechanism, programmers can create so called abstract classes that define generic behavior. An abstract superclass defines and partially implements the
Page 260192 r
Development ofa DatabaseAbstractionLayer —PartII
behavior of the object. Descendants of this superclass define and implementthe missing parts. Thus, inheritance takes abstraction to the nextlevel, making it even more powerful.
Today object-oriented concepts are applied inthe area of software engineering, knowledge bases, artificial intelligence and even in the area of databases.The Object- oriented approach has several advantages over the function-orientedapproach including:
1.Data abstraction: The details of a class's representation are visibleonly to its methods; different implementations of a class can be used with no changes tothe code that uses that class. (Data abstraction is not unique to object orientationbut arises naturally with it) 2.Compatibility: Heuristics for constructing classes and their interfaces make it easier to combine software components. 3.Reuse: combining methods with data representations to construct classes,makes it easier to reuse existing code. 4.Extensibility: Software built using object-oriented techniques tends to be easier to extend. There are two reasons for this: Inheritance enables new classes to be built from old ones, while still participating in all the original relationships; and the classes form a loosely coupled structure that is easier to modify. 5.Maintenance:The natural modularity of the class structures make it easier to contain the effects of changes, and the use of inheritance reduces the numberof disparate concepts to understand the code.
We can conclude that these basic concepts of object-oriented programming areable to provide a really powerful framework for the implementation of our database abstraction layer. Of course, choosing an object-oriented approach, also introduces a fewdrawbacks. For example, function-oriented methods seem to be inappropriate. This might be a serious problem, because database management systems are usually equippedwith function-oriented interfaces. There is no way to get around this problem in a proper way, but fortunately, Madsen states[23j:"Thinking object-oriented does not have to exclude functional expressions when that is more natural. Functions, types and values are in fact needed in order to describe measurable properties of objects." .. .AndI cannot agree with him more!
In the past few years quite a few object-oriented database management systems have been developed. As the number of object-oriented DBMSs grew, the need for a standard language and model was recognized and a consortium of vendors and users proposed a standard called ODMG-932°' which gradually evolved to the ODMG 3.0 standard(2h1,2 I2]Unfortunately, after the completion of the ODMG 3.0 standard, the group was disbanded. Today, most database management systems are still of the relational kind. So, even if we choose for an object-oriented approach and we do just that, we have to support the function-oriented methods for accessing databases.
Page27of92 Development of a Database Abstraction Layer —PartII
Designing an abstract database layer
Designing the new layer is a difficult task and we have to work very carefully making extensive use of the features object-oriented programming offers. Perhaps the most difficult part is how we perform the mapping from the (relational) database to the user application. One approach consists of extracting the relational data out of the database and mapping this data to objects. In this case the mapping should be preferably performed in such a way that the resulting objects are directly accessible and fully compatible with normal objects in the programming language used and operations on those objects should result on operations on the data in the database. Actual examples of sucha higher-levelabstractionare'transparent persistentobjects'inthe Java programming language[2.13, 2-14J•Whereas persistent data is information that can outlive the program that creates it, transparent persistence is the automatic storage and retrieval of persistent data. From a programmer's point of view, persistent objects are treated no differently than transient objects (or instances which only reside in memory and do not persist outside of an application). In Java, transparent persistence is available through the JDO interface. Although the specification is still under development, several preview implementations do exist. At a first glance this mechanism looks like a very good candidate for our relational/object mapping, but if we look a little bit closer at the specification we see that the emphasis is more towards mapping objects (or actually the state of objects) to databases and back again, instead of mapping existing databases to objects and back. There are layers build on top of JDO that actually do suport 'reverse engineering' of existing databases. Examples of such layers are Cayenne[2- and KODO
The question remains of course, whether these layers can or cannot manage huge quantities of data. For example: real life production databases can hold thousands or even millions of records and mapping all those records can be a very resource-consuming task. The makers of these layers claim the actually can do it using a caching mechanism (see figure 11). A somewhat extreme example of 'caching' is Prevayler(2.171,This software layer features transparent persistence of Java objects, but instead of mapping and caching objects to RAM if needed, it keeps a copy of all objects in RAM at all times.
Application (1) Server
I) .c (4) (U - ---- (6) (5) - -
1) request from application 5) applicationreads/writes to the cache 2) server queries database 6) changes are passed to the server 3) results are passed to server 7) the server applies the changes to the database 4) server passes result to the application cache Figure11: caching of transparent persistent objects
Page28 of 92 Development of a Database Abstraction Layer —PartII
One problem with caching is the introduction of serious concurrency issues. Another problem is the size of the RAM. Most DBMSs contain more data than fits in RAM. The database to object mapping is performed by mapping a table from the database to a class in the object-oriented programming language. Every record in the table is mapped to an instance of the class. One might ask how records can be compared, manipulated and sorted. One way is by using the standard programming techniques, which however can be very costly.Another approach consists of making use of some special kind of database query language. In this particular case queries can be done with 'JDO Queries'.
Perhaps it is better to take a slightly different approach. In my view it is better not to model the data itself, but to model only a representation of the data. For example: if we want to create a table of all customers in a customer database, we first define an object 'table' which will contain properties like the number of columns, and the column names. We also define methods how we can retrieve records or even single items from the database. What we do not want to do is to copy records from the database to the table object.
Besides connecting to a DBMS and being able to map our data, we also have to determine how we retrieve the data we want to have. If we had chosen for transparent persistent objects we just could have used our programming language. At first sight this seems to be a very elegant approach. However, in practice, it can be a very difficult and time-consuming task to find and sort the appropriate records. However, since we use a different approach, we have to choose either for some sort of self defined language like 'JDO queries' or for the general accepted database language SQL. Nowadays, almost every DBMS does support some form of SQL. Unfortunately, there are still differences in the various SQL dialects and while some vendors have added their own functionality, others do not even support all the standard SQL functions. Choosing for a query language to be newly defined also introduces serious problems. Besides defining a new language, we also have to parse the input and add the appropriate actions to our defined commands (probably in the form of SQL queries). The resulting queries are not the same for all DBMSs. Therefore every driver has to implement the appropriate actions. This does not mean that itis not possible, but the final result undoubtedly will be much slower and only very basic operations will be available. The major benefit is of course that the query language can be very strict and small and that all queries will result in valid SQL-queries for the target DBMS. Perhaps it is best to give a small overview of the pros and cons of both approaches:
Usage of SQL:
Pros: •Very well known by the majority of database developers •Short development time for the software layer Cons •DBMS vendors have added their own functions •Some standard ANSI SQL functions are not implemented in all DBMSs
Page 290192 Development of a Database Abstraction Layer —PartII
Usage of self-defined language:
Pros: •small language •strict non ambiguous definition Cons •limited functionality •slower •special DBMS specific features cannot be used •longer development time
Eventually I decided to use SQL as the query language for the drivers. The primary reason is simple: it had to work in practice. People are familiarwith SQL and they know that it works. They just wouldn't be happy working with some kind of self-defined language, how simple, straightforward and unambiguous it might be. Moreover, using another language might imply that not all functions a DBMS offers are available. While these products cost serious amounts of money, it is simply not explainable why only a 'crippled' version can be used. The question remains how we solve the differences between the various SQL dialects. The answer is very simple: we don't. What we are going to do is to give the user two different kinds of access modes: a native mode, which allows the user to talk to the DBMS directly with all features the DBMS offers and a compatibility mode which offers only a basic subset of the SQL-language. Furthermore we can try to solve compatibility problems by supplying the 'abstract database' with extra information. This extra information or 'metadata' tells what a particular DBMS can or cannot do and how certain things should be done.
Page 30 of 92 Development of a Database Abstraction Layer —Part11
System overview
Now we know more about the object-oriented approach and the way we want to access data in the database we can elaborate our architecture depicted in figure 7. Figure 7 shows the "database abstraction layer" as one component positioned between the application and the database connection technologies. We divide this component into five smaller portions:
•Database Abstraction Layer —managesall connections •Abstract Database —abstractobject representing a database •Database Driver —levelsdifferences between the abstract database and the actual driver •Database Table Model —returnsa table model of the results of a query •Database Tree Model —returnsa tree model of the database
These components will be explained in detail later, but first we give new overview of the entire architecture (figure 12).
Database Abstraction Layer
Actual DBMS ase
Database Mod&Tree Figure12: Overview of the database abstraction layer
TheDatabase Abstraction Layer
The heart of the system is the database abstraction layer class. This is the class that manages all the abstract databases currently in use or stored for later usage. Through the database abstraction layer users are able to connect with databases (without specifying a specific DBMS), register new database drivers, and select the default database to use. The methods defined for this class are:
•openO, try to open a database connection with the given properties •closeO, close all database connections •storeO, store a specified connection for later usage •selectO, select the default database to use •getO, get the current selected database •registerDriverO, registers a new database driver
Page 31 0192 Development ofa DatabaseAbstractionLayer —PartII
The registerDriver() method registers a new database driver. (In this context adatabase driver is an implementation of the abstract database class) Because all API and DBMS specific code are linked to such a database driver and we are able to load it when needed, we are able to save memory and plug-in new drivers asrequired.
The Abstract Database
The abstract database object is the key component of the database abstractionlayer. This object will be implemented as an abstract class. This abstract class defines the common representation and behavior of all the database drivers. The database drivers implement the actual connection with the database connection API's which in their turn connectwith the DBMS. The most important methods to be defined in the abstract databaseclass are:
•openO, open the connection with the database •closeO, close the connection with the database •queryO, query the database •updateO, insert data in the database •toStringO, returns a human readable description of the database
Besides the methods above, we also need some methods to set and retrieve properties like the hostname of the database server, the port number and the username.
The Database Table Model
Besides the basic functions for opening and closing connection and the handlingof data, we also want to be able to represent the data inthe database. Therefore we create a model of the data we want to represent. Unless it is absolutely necessary, we do not want to manage the actual data, but we only create a way to representthe data. This means we are able to tell how a cell is defined, how manycolumns are used, what the column titles are and how we can retrieve a specific cell.The way we do this is tightly coupled to the used programming language and accompanyinglibrariesand assuch,a more comprehensive description of its internals can be found in the implementation part of this thesis.
The Database Tree Model
Both developers and users of databases are usually interested in what databases can offer them. With a database tree model, users of the abstraction layer have a simple but very efficient mechanism to retrieve everything they possibly want to know. The database tree model models all information including databases, tables, columns and its accompanying properties, in a tree structure that is easily accessible and searchable. One advantage of a tree structure is that it also can be presented to end-users verywell.
Page 320192 Development of a Database Abstraction Layer —PartII
One mightaskhow all this information can be reirieved. There is no driver independent approach for this. Some DBMSs support special functions like getCatalogs() or getSchemas() others support special SQL queries. Therefore the database driver, the device dependent implementation part of an abstract database, defines how we should do this. When a database tree model is created, the returned object will be compatible with the used GUI. This makes it possible to create a visual representation of the tree structure with just a single statement.
Page 330192 Development of a Database Abstraction Layer —Partill
Part III, a practical implementation
Choosing a programming language andprogrammingtools
Before I could start with the implementation of the abstraction layer I had to decide over the programming language and tools to use. I chose the Java programming languagefor several reasons including:
•Time —theimplementation had to be realized in a reasonable amount of time •Database support —Javaalready supports the use of databases out-of-the box in the form of JDBC. Besides database drivers no extra libraries of third parties are required. •Driver availability -formost DBMSs (free) JDBC drivers areavailable.' •Platform independence —programswritten in Java can be used on various platforms without recompilation.2 •Language constructs —theJava programming language supports all features we need for a truly object-oriented implementation.
1) There are different versions of JDBC. The implementation must be able to handle those differences. Besides IDBC we have to make sure ODBC drivers can be used too. 2) Some database drivers are NOT platform independent and use the JavaJNI interface to load themselves. These drivers are mainly commercial ones.
Although my previous experiences with IDEs for the Java programminglanguage, such as Jdesigner and Jdeveloper, where not sopositive at all, it seemed a wise decision to make use of an IDE after all, because of the size of this project. The companywhere I followed my internship had used IBM Visual Age before and becauseof its availability I decided to use it too. Visual Age is a much more stable IDE than the onesI used before and I have not experienced any memory leaks or instability of anykind. The IDE also includes a pretty good versioning system. A free version of this IDE is availablefor download at the W'WW(311•Figure1 shows an example of how the IDE looks like. A major advantage of this IDE is its ability to use visual composition for the user interfaces. (See figure 2) For the database abstraction layer this is not an issue at all, but for user applications this can be a big benefit. The DatabaseExplorer, a test application that is going to be build on top of the database abstraction layer, will be made using the visual composition editor of the Visual Age IDE.
Page 34 of 92 r
Development of a Database Abstraction Layer —PartIII
RWorkbench IAdmrnistratorl I •
I [AiForct:
C J- 0 — 0 DathaeTeMode —0 DatthaTreeModel -t30JORQDtzrl - 0 JDBC_MySQL -±0JDBc_oooc —.0 JDBC...SQL_Seivi 0 bet4bo9i. —0 D.E.He,Sceen ± neôjthQ
-. 0 F,iben 0 G,1kiSean --± Gti1Sbeam 0Imageac*oiy ,
So.sct L1PO upon
Figure 1: example view of the IDE
1-igure z: exampie view or tne visuai composition editor
Page 35 of 92 Development of a Database Abstraction Layer —PartIII
Implementation
The Abstract Database
The first and probably most important step in our implementation is the creation of an abstract class 'AbstractDatabase'. In theory this abstract class serves just as a guideline for the implementation of the actual database drivers, but because a lot of database primitives are already available in the Java programming language it implements most methods by default. This does not mean database drivers have to make use of these 'default'implementations. The Java programming language permitstheuse of overloading methods. When default implementations are overloaded they will not be visible any more by default, however they scan still be called by calling the drivers super() (which is of course an instance of an AbstractDatabase). See the accompanying developers guide for an example implementation of a driver that uses such methods.
public abstract class AbstractDatabase public abstract boolean open(String connection_string); public abstract void closeO; public abstract boolean open(String server, String database, String user, String password); Finalize() public mt getAccessMethod() public mt getColumnCountO public Connection getConnectionO public String getDatabase() II returns database name public mt getFetchSizeO public String[] getHeadersO public DatabaseMetaData getMetaData() public mt getNumberO public String getPassword() public String getPort() public mt getRetrieveMethod() public String getServer() public TableModel getTableModel() public String getType() public String getUserO public String[] nextRowO public void printMessage(String tp, String msg) public void setFetchSize(int rows) public void setProperties(String s, String o, String t, String d, String u, String p, boolean r, mt n) public boolean setResultSet(ResultSet r) public String toString() public boolean succesfullO public boolean query(String queryStr) public boolean update(String updateStr) public boolean execute(String statement) public mt check(String statement)
Listing 8: the structure of the abstract database
Page 360192 Development of a Database Abstraction Layer —PartIll
Listing 1 shows the structure of the abstract database class. I decided to make thisan abstract class, because it makes the design a lot cleaner. By usingan abstract class, users use the same interface independent of the actual database driver used. The abstract methods openO, close() and finalize() ensures that the database drivers will implement these methods.
The source of this class can be found at the project page [3-21AJavaDoc description of this class can be found there too
The Abstract Database Layer
The abstract database layer class is the class that manages all abstract databases and thus all database connections. It keeps track of all connections being made and also has the ability to store connections for later usage. Normally users do notuse an abstract database directly, but access them through the abstraction layer. The listing below shows the structure of the class:
public DatabaseAbstractionLayer() { public void close() II closes all connections protected void finalize() II makes sure exit cleanly public DefaultListModel getCurrentConnections() II returnsopen connections public DefaultListModel getStoredConnections() II returns stored connections public AbstractDatabase getDatabase() II returns current selected database public String getDefaultServerType() II returns driver to use by default public DefaultListModel getDrivers() II returns list of registered drivers public boolean open(connection parameters) II open connection public boolean registerDriver(String driver) II registersa database driver public boolean selectDatabase(int index) II selects default database touse public boolean store(String server, String port, String type, String database, String user, String password, boolean remember_password, mt mode)
}
Listing 9: the structure of the database abstraction layer
1) Connections can be opened in severalways. A user can either specify a connection string or pass a list of connection parameters including the hostname, username and password to use. See the JavaDoc on the project page (331foran exact specification of the parameters that can be used.
Page 37of92 Development of a Database Abstraction Layer —PartIII
Before the abstraction layer can use a database driver, the driver has to be registered first. The RegisterDriver() method (see listing 3) tries to register the specified driver.
public boolean registerDnver(String driver) { try{ Stringpackage_name = this.getClassO.getNameO.substring(O,
this.getClassO.getNameO.lastlndexOf(".")÷1); ClassLoader loader =this.getClassO.getClassLoaderO;
if (loader.loadClass(package_name +driver)!=null){ 1* NOTE: method findClass not visible, using loadClass instead *1 /* alternative method: Class.forName(package +driver).newlnstanceO; *1 registeredDrivers.addElement(new String(packag&..name + driver)); System.err.println(" [DatabaseAbstractionLayerl Registering driver: "+ driver);
} return true; } catch (Exception e) ( System.err.println("[DatabaseAbstractionLayerl Unable to register driver: "+ driver+" ("÷e+")"); returnfalse;
}
}
Listing 10: the registerDriver method
The method first retrieves the same ClassLoader used for loading the database abstraction layer. Next it uses the ClassLoader 'loader' to load the specified driver. If the specified driver can be loaded, it will be added to the list of registered drivers and the driver is ready to be used.
The Database Drivers
Databasedrivers handle theDBMS specific part of an abstract database. Because the abstract database has alreadyimplemented most functions for you itis generally not hard to implement a driver for a specific DBMS, especially if a newerJDBC (version 2 or higher) driver exists. The only methods we must implement are the methods that are defined abstract in the AbstractDatabase and a finalize() method. The finalize() method is also obligatory because we want to be sure that the database connection is closed properly after usage. To create a framework for the database drivers we first implement a dummy driver. This framework makes it easier to implement real drivers. There is also a second reason: the database abstraction layer and user applications might want to have a database driver object without having an actual connection with a real database.
Page 38 of 92 Development of a Database Abstraction Layer —PartIII
public class DR V_Dummy extends AbstractDatabase {
1* * * DRV_Dummy constructor *1 publicDRV_Dummy() { superO; I
1* * * Closesthe connection in a proper manner *Catchesexception if this connection is already closed *1 publicvoid close() { try{ con.closeO; )catch(Exception e) {}
1* * * Makessure to close the connection. *ifthis object is going to be removed. *1 publicvoid finalize() { closeO;
1** * Triesto create a connection with the database. *1 publicboolean open(String dbURL) { returnfalse; I
1* * * Triesto open the connection with the given parameters. *1 publicboolean open(String server, String database, String user, String password) ( returnfalse;
I
Usting 11: the dummy driver
Page39 of 92 Development ofa DatabaseAbstractionLayer —PartIII
The listing above shows the full source code of the Dummy driver. As you can see, the structureofthisdriver(actuallyeverydatabasedriver)isverysimpleand straightforward. The sole purpose of the database drivers is to glue the abstract database to a real database driver. Because of its simple structure it is easy for developers to implement their own drivers making it possible to access almost every database as an abstract database. Currently database drivers for various DBMSs are already available including support for SQL Server, MySQL and PostgresSQL. The sources for these drivers are available at the WWW[3.4]
Page40 of 92 Development of a Database Abstraction Layer —Part111
The Database Explorer
The Database Explorer is a graphical user application demonstrating the capabilities of the abstract database layer. Originally this application started as a small application to test the abstraction layer. This application steadily evolved in a user-friendly application embodying the powerful capabilities of the underlying software layer. One can say that it now serves both as a test case for the Database Abstraction Layer as well as an easy-to- use user application. You can take a look at the user guide for more information on how you can use this application. The interface of the Database Explorer has been created using the composition editor of Visual Age for Java. I have tried to match the user interface with the basic functions of the database abstraction layer. The main window of the user interface has 5tabs.The first tab allows you to connect to a DBMS, the second one lets you browse your databases, the third one lets you execute SQL statements and the fourth one shows you the results of the performed SQL queries. See the figure below
IIFileEditViewHelp ConnectionsBrowseExecuteResultseti MessaesI
New connection: Stored connections:
Server J127.O.0.1 Port
Type: JMYSQL
Database: JMediaCenter User [ Current connections: Password: [ Iocalhost Access mode: ( ANSI SQL C Native 127.0.0.1 :HJB Remember password: r 127.0.1 :MediaCeriter Store if connection succeeds: r
IiConnect Store Close Delete
_!i!iiI1IISELECT*FROMneuke_aibumsl -— Figure 3: the Database Explorer main window
Page 410192 Development of a Database Abstraction Layer —PartII!
The fifth tab shows all error, warning and information messages generated by the abstraction layer or the Java subsystem. This is implemented by redirecting the standard error and standard output to the message windows. Technically this works by creating a stream to the text area of our message window (See listing 5).
llredirect standard error/out output to the message tab GuiPrintStream gos =newGuiPrintStream(new GuiOutputStream(getTextArea_MessagesO)); System.setOut((PrintStream) gos); System.setErr((PrintStream) gos);
Listing 12: redirecting the standard error/output
Generated user interfaces have the tendency to favor the ease of use at the cost of readability of the source code. I tried to keep the source as clear as possible. All action on components in the user interface are linked to methods starting with 'action' in the source code. All components start with 'get' followed by its name. All connections between the components and actions start with 'conn'. At the startup of this application the method 'initialize'is being executed. This method initializes all components, redirects the standard error and output as described above, creates a database abstraction layer, registers all available database drivers and makes a list of registered database drivers available to the user. Information on how to use this program can be found in the users guide[351•
Page42 of 92 Development of a Database Abstraction Layer — Conclusions & History
Conclusions
The abstraction layer is an example of software that emerged from real lifeexperiences. From previous projects I experienced that there was a need for a mechanism to access databases in a simple and transparent manner. By looking closely to the various database connection technologies and to object-oriented programming in general, it waspossible to implement a functional abstraction layer that suits theseneeds. One might ask why such an implementation is useful after all there are alreadylots of database connection technologies. All though that is certainly true you do not want torely on one single connection technology, because asingle connection technology is no guarantee that all DBMSs that you want to use are supported.Your DBMS might not be supported that well or even at all! Although the current implementationisfullyfunctional thereis roomstillfor improvement. Possible improvements are for example the implementationof more drivers (especially a CORBA client and a CORBA ORB driver, that wouldmake the abstraction layer CORBA enabled), better SQL syntax checking and the additionof JDO support (making it possible to map tables from the database to objectsand back again).
History
My first real encounter with applications that make extensible use of databases wasfor the course Software Engineering. During this course we had to design andimplement an online reservation system. Because the system had to make databaseconnections on various places in the implementation, we decided to make a separate classhandling the database connection us. When I later encountered the same issues during myinternship for Aegon Inc in Taiwan, I decided to create a more generic set of classes tohandle database connections, which resulted in the first steps of theDatabaseAbstractionLayer. After my internship the abstraction layer evolved gradually until it becamewhat it is today. More features where added and a test application wasmade.
Page43 of 92 Development of a Database Abstraction Layer —Glossary
Glossary
Abbreviationsused in this document
ADO - ActiveXData Objects, programming interface from Microsoft to access data in a database ANSI - AmericanNational Standards Institute API - ApplicationProgramming Interface, enables programmers to use (external) functionality in a structured manner ASP - ActiveServer Pages, Microsoft scripting language for the creation of dynamic web pages ATM - AsynchronousTransfer Mode, a network technology based on transferring data in packets. ATM differs from TCP/IP, because ATM creates a fixed channel or route whenever data transfer begins. COM - ComponentObject Model, a software architecture developed by Microsoft to build component-based applications. COM was first released in 1993. Microsoft's infamous ActiveX is based on COM. CORBA - CommonObject Request Broker Architecture, OMG's open, vendor-independent specification for an architecture and infrastructure that computer applications use to work together over networks. DB - Database,a collection of related data DBMS - DatabaseManagement System, a collection of programs that enable users to create and maintain a database. DCOM - DistributedComponent Object Model, an extension of the Component Object Model (COM) that allows COM components to communicate across networks. DCOM uses the RPC mechanism to transparently send and receive information between COM components. DCOM was introduced in 1995 with the introduction of Windows NT 4 DLL - DynamicLink Library, a library of functions and other information that can be accessed by a program. IDL - InterfaceDefinition Language, used to define the interfaces to CORBA objects. hOP - InternetInter Orb Protocol, a protocol developed by the 0MG to implement CORBA solutions over networks. US - InternetInformation Services, a web server from Microsoft IP - InternetProtocol, specifies the format of network data packets and the addressing scheme. ISO - InternationalStandards Organization JSP - JavaServer Pages, server side scripting in Java used to create dynamic Web pages
Page 44 of 92 Development of a Database Abstraction Layer —Glossary
JDBC - JavaDatabase Connectivity, set of Java classes developed by Sun Microsystems to allow access to relational database through the execution of SQL-statements. MDAC - MicrosoftData Access Components, package containing several components that provide various database technologies including JDBC ADO/OLE DB MTS - MicrosoftTransaction Server, a server program that manages application and database transaction requests on behalf of a client. .NET - AMicrosoft platform that incorporates applications, tools and services in order to erase the boundaries between applications and the Internet. ODBC - ObjectDatabase Connectivity, a standard database access method. The goal of ODBC is to make it possible to access databases from applications. This goal is achieved by inserting a middleware layer, the ODBC manager, between an application and the DBMS. ODMG - ObjectDatabase Management Group, a consortium of object- oriented DBMS vendors and users. Not to be confused with 0MG. The ODMG stopped after the completion of its work on object data management standards in 2001. OLE - Abbreviationof Object Linking and Embedding, a compound document standard developed by Microsoft Corporation. It enables you to create objects with one application and then link or embed them in a second application. 0MG - ObjectManagement Group, a consortium with a membership of more than 700 companies. The organization's goal is to provide a common framework for developing applications using object- oriented programming techniques. PHP - Self-referentiallyshort for PHP: Hypertext Preprocessor, an open source, server-side, HTML embedded scripting language used to create dynamic Web pages. ROI - Returnon Investment, analyzes the derived benefits of an investment relative to its costs. RPC - RemoteProcedure Call, a type of protocol that allows a program on one computer to execute a program on a server. The client program sends a message to the server with the appropriate arguments and the server returns a message containing the result of the program executed. SAG - SQLAccess Group, an organization of leading hardware and software developers committed to universal database access.
Pate 45 of 92 Development of a Database Abstraction Layer —Glossary
SQL - StructuredQuery Language, the standard language for commercial relational DBMSs. Originally called SEQUEL and designed and implemented at IBM Research. Standardized by the joint effort of ISO and ANSI. In 1986, ANSI approved a rudimentary version of SQL as the official standard, but most versions of SQL since then have included many extensions to the ANSI standard. In 1991, ANSI updated the standard. Because the SQL Access Group developed the draft for this new standard, it is known as SAG SQL. TCO - TotalCost of Ownership, reveals the total cost of a solution over time, including ongoing maintenance, integration and support costs. TCP - TransmissionControl Protocol, enables two hosts to establish a connection and exchange streams of data.
Page 46 of 92 U
Development of a Database Abstraction Layer —References
References
General:
[gi] Elmasri & Navate [2000], Fundamentals of Database SystemsYcedition, Addison-Wesley, 2000
[g2] Derek Coleman, Patrick Arnold, Stephanie Bodoff, Chris Dollin, Helena Gilchrist, Fiona Hayes, Paul Jeremaes [1994], Object-Oriented Development, The Fusion Method, Prentice-Hall, 1994
[g3] Cay Horstmann, Gary Cornell, Core Java, Volume 1 —Fundamentals, Prentice-Hall, 1998
[g41 H.M. Deitel, P.J. Deitel, Java, How to program 2' edition, Prentice-Hall, 1998
Part I:
[1-1]ODBC information http://www.iodbc.org
[1-2]ODBC Visual Basic example Bretschneider, Udo translation by Wessels, Paul Visual Basic 4.0 Databecker Gmbh & Co. KG, 1996
[1-3] ADO DB example http :/Iwww.speech.cs.cmu.edu/—sburke/PUb/tPI 17.html
[1-4]Types of JDBC technology drivers http://java.sun.com/products/idbc/driverdesc.html
[1-5]JDBC example; http://www.wdvl.com/Authoring/Java/ServletSIJDBC example.html
[1-6]CORBA information http://www.corba.org
Page 460192 Development ofaDatabase AbstractionLayer —References
[1-710MG: The Object Management Group http://www.omg.org
[1-8] 0MG IDL Specifications http:llwww.omg.orgltechnologv/documents/idl2x spec catalog.htm
[1-9]CORBA IDL example http://iava.sun.com/products/idk/l .2/docs/guidelidl/tutorialJGSIDL.html
[1-10] CORBA Server example http://java.sun.com/products/jdk/l.2/docs/guide/idi/tutori al/appfHelloClient.j ava
[1-11] CORBA Client example http://java.sun.com/products/idk/l.2/docs/guide/id1/tutori al/appfHelloCl ient. java
[1-12] Carnegie Mellon Software Engineering Institute Component Object Model (COM), DCOM, and Related Capabilities http://www.sei.cmu.edu/str/descriptions/com.html
[1-13] Tallman, Owen and Kain, J. Bradford COM versus CORBA: A Decision Framework Printed in Distributed Computing, September—December 1998 http://www.scit.wlv.ac.uk/—cm 1 924/cp3025/distrib/readinWcorba/corba3/corba6.html
Part H:
[2-1]: Object oriented programming explained http://www.webopedia.comfl'ERM/o/object oriented programming OOP.html
[2-2]:Stroustrup, Bjarne, What is "Object-Oriented Programming"?, 1991 http://www.research.att.com/—bs/whatis.ps
[2-3]: Ole Lehrmann Madsen, "What object-oriented programming may be —andwhat it does not have to be" Lecture Notes in Computer Science Ed: G. Goos and J. Hartmanis, 1988
[2-4]: Campione, Mary and Walrath, Kathy Object-Oriented Programming Concepts: A Primer, 1996 http://www.ecs.umass.edu/ece/wireless/people/emn-ianuelliava/j ava/objects/ http://www.ecs.umass.edu/ece/wireIess/peopleJemmanuel/java/j ava/iavaOO/
[2-5]:Hostetter, Chris, Survey of Object Oriented Programming Languages, 1998 http://www .rescomp.berkely.edu/—hossman/cs263/paper.html
Page 47of 92 Development of a Database Abstraction Layer — References
[2-61:Nguyen, Van and Hailpern, Brent A Generalized Object Model ACM SIGPLAN NOTICES —v21,nlO, Ed: Richard Wexeiblat, 1986
[2-7]: What Is an Object? Learning the Java Language, Sun Microsystems, Inc., 2003 http://java. sun .com/docslbooks/tutorial/iava/concepts/Obiect.html
[2-81: What Is a Message? Learning the Java Language, Sun Microsystems, Inc., 2003 http://isun.com/docs/books/tutorial/i ava/concepts/message.html
[2-9]: What Is Inheritance? Learning the Java Language, Sun Microsystems, Inc., 2003 http://i.com/docs/books/tutoriallj ava/concepts/inheritance.html
[2-10]: Cattell, R. G. G. The Object Database Standard: ODMG-93 Morgan Kaufmann 1993
[2-1 1]: Cattell, R. G. G., Barry, Douglas K. The Object Data Standard: ODMG 3.0 Morgan Kaufmann 2000, ISBN: 1558606475
[2-12]: Schmitt, Ingo Der ODMG 3.0-Standard http://www.mycosima.com/dbp2/vorlesungJOdmu.Pdf
[2-13]: Transparent Persistence with Java Data Objects http://developers.sun.com/tool s/javatools/articles/transparent.html
[2-14]: Transparent Persistence http ://www.service-architecture.com/obiect-orieflted-databases/ articles/transparent persistence.html
[2-15]: Cayenne http://www.objectstvle.org/cayenne/
[2-16]: SolarMetric KODO JDO htp://www.solarmetric.com/SoftwareIDoCUmentatiOflllateStldoCs/ifldeX.html
[2-17]: Prevayler, a software prevalence layer for Java http://www.prevayler.org
Page48 of 92 Development of a Database Abstraction Layer —References
Part III:
[3-11: IBM Visual Age for Java http://software.ibm.com/ad/vaiava
[3-2]:Project Page, Source: AbstractDatabase.java http://www.fmf.nl/—eelco/afstuderen/ files/source/rugleduldatabase/AbstractDatabase.java
[3-3]:Project Page, Javadoc http://www.fmf.nI/—eelco/afstuderen/files/javadoc/
[3-4]:Project Page, Source: abstract database and drivers http://www.fmf.nJ/—eelco/afstuderen/fjles/javadoc/rug/eduJdatabase/
[3-5]:Project Page, Documents: users guide http://www.fmf.nl/—eejco/afstuderen/fjles/documents/users guide.ps
Page 49 of 92 Development of a Database Abstraction Layer —DevelopersGuide
Appendix A: Developers Guide
Overview
• Getting started • Registering database drivers • Creating connections • Gathering information • Executing statements • Viewing results • A programming example • Developing database drivers • More information
Getting started
The Database Abstraction Layer isa software layer positioned betweenthe user application and the database making it easier to access databases from differentDBMS vendors.
This guide is intended for developers who want more information on how they canmake their applications 'Abstract Database'-enabled.
There are two sample applications bundled with the sources of this package. The first application is a full-blown program called 'Database Explorer' which makes extensive use of the capabilities of this software layer.The second application is a small sample application simply called:'Example'. This example is a good starting point for integrating the Database Abstraction Layer in your application.
Before you can use this package the first time it is important to verify if you have the Java runtime environment (version 1.2.2 or higher) installed. Versions before 1.4.1 might require the setting of the correct $CLASSPATh variable. From version 1.4.1 onwards this should not be necessary.
Page 50 of 92 Development of a Database Abstraction Layer —DevelopersGuide
Registering database drivers
Before we can open and use database connections we have to create an abstraction layer and register the database drivers we want to use. (Listing 1)
IISTEP 1:create the database abstraction layer dbl =newDatabaseAbstractionLayerO;
IISTEP2: register the driver(s) you want to use dbl .registerDriver("JDBC_MySQL"); dbl .registerDriver('More Drivers");
Listing 1: create an abstraction layer and register drivers
A database driver is the DBMS specific part of an 'Abstract Database'. There is no limitation in the amount of drivers that can be registered. Multiple drivers may even use the same DBMS. For example: you can create a driver accessing a SQL Server using an ODBC connection, but you can also implement a driver accessing the same SQL Server with a JDBC connection. Internally, the registerDriver() method tries to find the class specified by the given parameter in the available packages or on the local file system and loads it into memory. This enables you to add and use drivers on the fly without recompilation of all your classes.
Note: the registerDriver() method returns a Boolean which enables you to check if the specified driver was registered successfully.
Page 510192 Development of a Database Abstraction Layer —DevelopersGuide
Creating connections
After you have registered one or more drivers, you can try to make a connection with a database. There are three approaches that can be used to open a connection. The first approach is to open a connection without specifying the driver to use. The abstraction layer will try to figure out what driver to use by creating connections and looking for the specified database. The second method tells the abstraction layer that is has to use the given driver. Note: all parameters are Strings and are not required to have a value. When a parameter is empty the system will use its default value. See the userguide for a table containing all default values. The third method is driver dependent and intended for testing purposes only. It is not recommended to use this approach for normal use. (See listing 2)
IIApproachA: open a connection with auto detection of the driver to use dbl.open(server, database, user, password);
IIApproachB: open a connection through the specified driver dbl.open(server, database, user, password, driver);
IIApproachC: open a connection using the supplied connection string dbl .open("jdbc:mysql ://server/database?user=XXX&passwordXXX");
Listing 2: connecting to a database
Page 52 of 92 Development of a Database Abstraction Layer —DevelopersGuide
Gathering information
You are probably curious what an established connection has to offer you. You can find out everything you want to know about your DBMS and databases by requesting a 'DatabaseTreeModel' or a 'DatabaseMetaData' object. The DatabaseTreeModel is a tree structure containing all accessible databases, tables and columns. Furthermore this structure contains a list of all metadata properties describing the capabilities of the DBMS. The DatabaseMetaData object contains all database metadata properties. It tells you, for example, whether you can or cannot write to the database, use subselect statementsorusebatchupdates.The DatabaseMetaDataobjectisinfacta java.sql.DatabaseMetaData object, already available in every version of Java. See [urli] for a comprehensive description of this class. The following example shows you how you can create and use a tree and a metadata object:
II Create a tree model DatabaseTreeModel dUn =newDatabaseTreeModel(dbl);
IICreatea graphical representation of the tree using a java.swing.JTree IISee[url2] for a description of the usage of Jtrees II Note: the DatabaseTreeModel extends the DefaultTreeModel and therefore II it can be used directly by Java Swing components JTree aTree =newJTreeO; aTree.setName("Graphic representation of the DatabaseTreeModel "); aTree.setModel(dtm);
IIGetthe DatabaseMetaData object belonging to the current selected database DatabaseMetaData dmd =dbl.getDatabaseO.getMetaDataO;
IISimplemeta data test: check if DBMS supports the ODBC Minimum II SQL grammar. See [urli] for all available methods Boolean supportMinSQL =dmd.supportsMinimumSQLGrammar()
Listing 3: gathering information
NOTE: a summary of all methods and fields can be found in the generated JavaDoc files. These documents are available at the project page [url3].
Page 53 of 92 Development of a Database Abstraction Layer —DevelopersGuide
Executing statements
Connecting and retrieving information is one thing, but it becomes really interesting when you can actually communicate with your databases. The Database Abstraction Layer distinguishes between two types of database access. The first type, read-only access,assures that data can only be retrieved from the database. Insertions and deletions are not possible. The corresponding method is 'queryO'. The second type supports write access and can be used for deletions and insertions. The corresponding method iscalled 'updateO'. Normally, you do not have to distinguish between these two types of access and you can just use the 'executeO' method. The 'checkO' method enables you to check what kind of access you should use with a given SQL staiement. The current implementation of this method performs only basic checks like searching for keywords that need write access. You are free to overload this method with a better implementation. Listing 4 gives you examples how to check and execute SQL statements.
IIGetthe selected database from the database abstraction layer AbstractDatabase adb =dbl.getDatabaseO;
IICreatea sample SQL query String statement ="SELECT*FROMCustomerData";
IICheckwhat kind of query we have IIAfterexecution 'statement' can contain one of the following values: IIAbstractDatabase.isQuery:a valid query (read-only) statement detected IIAbstractDatabase.isUpdate:a valid update (write) statement detected IIAbstractDatabase.isError:not a valid SQL statement detected IIinthis particular case 'statement' equals AbstractDatabase.isQuery mt check =adb.check(statement);
IIperformquery or update, based on our check if (check==AbstractDatabase.isError) { System.err.pnntln("Error in SQL Statement"); } else{ if (check==AbstractDatabase.isQuery) { adb.query(statement);
if (check==AbstractDatabase.isUpdate) { adb.update(statement);
Page 54 of 92 Development of a Database Abstraction Layer —DevelopersGuide
IIAlternativeexecution of our SQL statement IINote:this is the simplest and preferred way to perform an SQLquery IITheexecute method decides automatically which methods to call IINote:Returns true if execution was successful Boolean ok =adb.execute(statement);
Listing 4: examples of checking and performing queries
Warning: all update statements will be executed on your database. When used without caution, you can seriously harm the contents ofyour databases.
Page 55 of 92 Development of a Database Abstraction Layer —DevelopersGuide
Viewing results
The results of your queries are stored in a ResuitSet object. The simplest way to visualize your results is by asking the database abstraction layer to create a model of your data. The returned model can be directly inserted in a javax.swing.JTable (See listing 5) because itis compatible with a javax.swing.table.TableModel [url4J. The database abstraction layer tries to minimize the amount of network traffic between the DBMS and your application by only fetching the data rows that are actually displayed. Unfortunately, this is not possible when you are using a JDBC version 1.0 driver. If you use such a driver, the database abstraction layer will build a different, but still compatible table model, which does contain all your returned data. The amount of data that should be prefetched can be manually tuned with the 'getFetchSize' and 'setFetchSize' methods (See Listing 5).
String query ="SELECTfilename FROM test.files";
II STEP 4: perform the query dbl .getDatabaseO.execute(query);
II Use specified fetch size of 10 rows at a time dbl .getDatabaseO.setFetchSize( 10);
IISTEP5:displaythe results getTableO.setModel(dbl.getDatabaseO.getTableModelO);
Listing 5: visualizing query results
Page 560192 I
Development of a Database Abstraction Layer—DevelopersGuide
A programming example
We can summarize the code examples above to createa simple, but good example showing the capabilities of the Database Abstraction Layer. The following listing contains the full source code of an example program that is able to connectto a database, retrieve an entire table and view the contents of the retrieved table.
import java.awt.*; import javax.swing.*; import rug.edu.database.*;
/** * Exampleapplication for *theDatabase Abstraction Layer * *1 public class Example extends JFrame private JTable ivjTable =null; private JScrollPane ivjScrollPane =null; private rug.edu.database.DatabaseAbstractioayer dbl=null; /** * Exampleconstructor comment. *1 public Example() { superO; initializeO;
Page 57 of 92 Development of a Database Abstraction Layer —DevelopersGuide
/** * Triesto: *- createsa database connection *- perfonna query *- displaythe results *
*Creationdate: (2/2/2004 1:59:07 PM) *1 public void doQuery() II Configuration String driver ="JDBC_MySQL'; String server= ""; Stringuser = Stringpassword = Stringdatabase = Stringquery ="SELECTfilename FROM test.files";
II STEP 1: create the database abstraction layer dbl =newDatabaseAbstractionLayerO;
II STEP 2: register the driver(s) you want to use dbl.registerDnver("JDBC_MySQL");
II STEP 3: open the connection dbl.open(server, database, user, password);
II STEP 4: perform the query dbl.getDatabaseO.execute(query);
II STEP 5: display the results getTableO.setModel(dbl.getDatabaseO.getTableMOdelO);
/** * Createsthe scrollable pane *Returnsthe ScrollPane property value. *@returnjavax.swing.JScrollPane *1 private javax.swing.JScrollPane getScrollPane() if (ivjScrollPane =null)
ivjScrollPane =newjavax.swing.JScrollPaneO; ivjScrollPane.setName("ScrollPane"); ivjScrollpane.setVerticalScrollBarPolicy (JScrollPane.VERTICAL_SCROLLBAR_ALWAYS); ivjScrollPane.setHorizontalScrollBarPolicy (JScrollPane.HORIZONTAL_SCROLLBAR_ALWAYS) getScrolwaneO.setViewportView(getTableO)
)catch(java.lang.Throwable ivjExc) { handleException(ivjExc);
return ivjScrollPane;
Page 58 of 92 Development of a Database Abstraction Layer —DevelopersGuide
/** * Createsthe table in the scrolipane *Returnsthe Table property value. *@returnjavax.swing.JTabIe *1 private javax.swing.JTable getTable() if (ivjTable =null) try( ivjTable =newjavax.swingiTableo; ivjTable.setNarne("Table"); getScrolIPaneO.setColumnHeaderView(ivjTabIe.getTabIeHeader); getScrollPaneO.getViewportO.setBackingStoreEnabled(true); ivjTable.setBounds(0, 0, 200, 200);
}catch(java.lang.l'hrowable ivjExc) handleException(ivjExc);
return ivjTable;
/** * Calledwhenever the part throws an exception. *@paramexception java.Iang.Throwable *1 private void handleException(java.lang.Throwable exception) {
1*Uncomment the following lines to print uncaught exceptions to stdout *1 II System.out.println("-——---- UNCAUGHT EXCEPTION —------"); IIexception.pnntStacklrace(System.out);
/** * Initializethe class. *1 private void initialize() 1 try( setName("Example"); setDefaultcloseOperation(javax.swing.WindowConstants.DJSpOSE_ON_CLOSE); setSize(600, 400); setTitle('Database Abstraction Layer -Example"); setContentPane(getScrollPaneO); catch (java.lang.Throwable ivjExc) { handleException(ivjExc);
II make the connection, perform the query and show the results doQueryO;
Page 59 of 92 Development of a Database Abstraction Layer —DevelopersGuide
,** * Starts the application. * ®param args an array of command-line arguments *1 public static void main(java.lang.String[] args) tiy{ /*Setnative look and feel */ UlManager.setLookAndFeel(UlManager.getSystenlLookAndFeelClasSNaflleO); /*Createthe frame *1 Example aExample =newExampleO;
/*Askjava.awt.Toolkit for the current screen size / Dimension screenSize =Toolkit.getDefaultToolkitO.getScreenSizeO;
/*Packframe on the screen / aExample.packO;
/*Centerframe on the screen *1 Dimension frameSize =aExample.getSizeO; if (frarneSize.height> screenSize.height) frameSize.height =screenSize.height; if (frameSize.width> screenSize.width) frameSize.width =screenSize.width; aExample.setLocation((screenSize.width -frameSize.width)/2, (screenSize.height -frameSize.height)/2);
/*Adda windowListener for the windowClosedEvent *1 aExample.addWindowListener(new java.awt.event.WindowAdapterO ( public void windowClosed(java.awt.event.WindowEvent e) System.exit(O);
aExample.setVisible(true); catch (Throwable exception) System.err.println("Exception occurred in main() of Example"); exception.printStackTrace(System.out);
Listing 6: a complete example.
If you take a closer look at this example, you will see that there is only one method performing all database related actions. This method, doQueryO, creates the database abstraction layer, registers the database driver you want to use, opens the connection with the database, performs the query and finally displays the results. The other methods in this example are just to initialize and glue the GUI-components together.
Page 600192 ______. U
Development of a Database Abstraction Layer —DevelopersGuide
In order to run this example, make sure to edit the variables 'server', 'user', 'password' 'driver', 'database' and 'query' in the 'doQueryO'-method. When we run this example, the resulting table looks like the figure below:
- unaato bach lataimusicIAimee . .Vise' lataimusic/Alr - AllI Neemp3 Iataimusic/Arnerican Desi - Pijrijauu Iatalmusic(Beach Boys - The Uon SIee,.. TonI-.hf nn1 lataimusiclBert and Ernie - Rubber Duckymp3 IataImuscfBhangra - Dj Sanj - Next Episodem IataimusicfBill Withers - Aini NO Sunshine.mp3 -Iata/muscIBiII Withers - Just the Two of Us mp lata/music/Boney M - Daddy Coolmp3 lata/music/Brainpower- Dansplaatmpi lata/music/Bus Stop - Kung Fu Fighting.r latafmuslc/Commodores - Disco lnhmo latalmusic/Coyote Ugly- Leanne latalmusic/De DIjk - Als Ze Er Nk lataimusic/Delite - Groove is in the Heart lata/mustcJEales - Hotel Califomla.m lataimusic/Flashdance - What a Feplir lataimusic/Galleon- So I lataimusklOroove Armada lidataimusiciJackson Five - biamu It C Thel IidataimusiclKeeye I- Geef Me Dat lidatalrnusiciLL Cool J - Doln ltmp j-i - - . rigure 1: example output of the programming campic.
Page 61 0192 Development of a Database Abstraction Layer —DevelopersGuide
Developing database drivers
Although the source package contains drivers for various DBMSs it ispossible that your DBMS is not supported yet. If this is the case, you will have to add adatabase driver yourself. First you will have to find out what kind of connections yourDBMS supports. Some DBMSs allow 'native' access directly from Java; othersrequire the use of the JDBC or ODBC API. It is best to use the following guideline in case youhave more than one option:
•First try to find a JDBC driver for your DBMS, preferably a versionsupporting JDBC 2.0 or higher. Implement the database driver using this JDBCdriver. (See JDBC_Durnmy for a generic implementation of such a driver.) Usuallythis results in a robust and fast combination. •Use a 'native' Java implementation for accessing your DBMS.This can result in a very fast driver, but it might take some time todevelop the driver. •Use a 'native' ODBC driver. If you have an ODBC compliantDBMS and you can find a native ODBC driver for that particular DBMS, you can use that too. Note: in most cases these drivers are commercial products. •Use the supplied data bridge. The supplied JDBC_ODBC driver,which implements an AbstractDatabase using Sun's ODBC bridge, enables you to use your ODBC compliant DBMS directlywithout the need for a new driver. Because of the many translation steps between the database and the applicationthis is not the recommended thing to do for production systems.
In short, you can say JDBC is preferred over ODBC. The reasonfor this is twofold: we use Java for our programming language andJDBC has evolved to an integral part of that language and the second reason is that there are a lot of goodand free JDBC drivers available. There is only one exception: Microsoft products generallywork better with ODBC than with JDBC. There is a JDBC implementation forMicrosoft's SQL-server, however you will not be able to use the JDBC 2.0 features.
Besides creating a totally new database driver, you can alsoimplement an alternate database driver. For example: you already have a databasedriver, but you would like to be able to log and analyze all queries and the returned data.Now you have two options: create and insert a dummy 'pass through' driver ormake a copy of an original driver and alter it.
During the following example we will create a generic 'passthrough' driver logging all actions to a database.
Page 62 of 92 Development of a Database Abstraction Layer —DevelopersGuide
Westartwithcreatingacopyofthe Dummy driverandrenameitto 'DRV_PassThrough'. Now, when we open the driver in an editorwe see that we must implement the following methods:
•DRV_PassThroughO, the constructor •openO, defines how to open the connection to the database' •closeO, defines how to close the connection to the database •finalizeO, makes sure to close the connection to the database when a instance of this driver is removed.
There are two versions of the open() method to implement; the first version has only one parameter representing a 'connection string'. This string defines exactly how and where we should connect to. Unfortunately, this string is not exactly defined in the JDBC standard and different drivers use different schemes. The second version has four parameters representing the server, database, user and password to use. The primary goal of this version of the open() method is to level out the driver dependent differences mentioned above.
The primary task of the constructor is to initialize its superclass the 'AbstractDatabase'. Next, it initializes the driver used for logging and real database driver (see listing 7)
1** * TheDRV_PassThrough constructor. *Initializesthe superclass AbstractDatabase *1 publicDRV_PassThrough() { superO; dal =newDatabaseAbstractionLayer; IInewhidden abstraction layer dal.registerDriver(logDriver); IIdriverused for logging if (!adbDriver.equals(logDriver)) { dal.registerDriver(adbDriver); IIrealdriver to use
} if(initializeLogO) { printMessage(DRV+ NOTE, "Log initialized");
)else{ pnntMessage(DRV+ NOTE, "Log is unavailable");
Listing 7: the DRV_PassThrough constructor
Page 630192 Development of a Database Abstraction Layer —DevelopersGuide
The implementation of the open() method is somewhat special in this particular case. We do not open the database connection ourselves, because we do not exactly know which driver we should use. Instead we just ask the (hidden) database abstraction layer to do the job for us. If we have successfully opened the connection, we point the connection object and retrieve method of our database driver to the actual object. This ensures us that the user will see the real driver instead of our 'pass through' driver. (See listing 8)
1** * openthe database connection *....butwe are just a fake pass through driver ?!!!! * * Q: howdo we know where we should connect to and * whatdriver should be used? * * A:just let the database abstraction layer do all the work! *1 publicboolean open(String server, String database, String user, String password) {
booleansuccess =dal.open(server,database, user, password);
if (success) { con=daLgetDatabaseO.getConnectionO; retrieve_method =dal.getDatabaseO.getRetrieveMethodO;
printMessage(DRV +NOTE,"PassThrough driver initialized\n"); return success;
Listing 8: opening the database connection
Page 640192 Development of a Database Abstraction Layer —DevelopersGuide
The next method we must implement is the closeO method. Normally we only have to close one connection, but because we log our actions to a database we have to close two connections. (See listing 9)
/** *Closethe connection in a proper manner *Catchexception if this connection is already closed * * Closingthe connection is important, because most *DBMS'scan handle only a limited amount of connections. *1 publicvoid close() { II closeconnection to the database try{ con.closeO;
}catch(Exception d) (} II closeconnection to the database we are logging to
log.closeO;
}catch(Exception 1) (} }
Listing9: the closeO method
The last method that we are required to implement is the finalize() method. This method makes sure all connections are closed before this instance is removed. (See listing 10)
1** * Makessure to close the connection *ifthis object is going to be removed. *1 publicvoid finalize() { this.closeO;
Listing 10: the finalize() method
Now that we have implemented all methods that are required for every driver, we can adapt our driver by adding extra methods and overloading some default methods of the AbstractDatabase class. An important fact in the Java programming is that overloaded methods still can be called by making its call to its super class. This saves us a lot of work and eases the implementation of the other methods specific for our example pass through driver (See listing 11)
Page 650192 Development of a Database Abstraction Layer —DevelopersGuide
1* * * Executethe given statement. *Creationdate: (3/2/2004 1:52:07 PM) *@returnboolean *@paramstatement java.lang.String *1 public boolean execute(String statement) { printMessage(DRV+NOTE,"Execution caught"); boolean res =super.execute(statement); return res;
1* * * Initializeour log. *Creationdate: (4/2/2004 11:31:38 PM) *@returnboolean *1 public boolean initializeLog() { log=dal.open("","log","",""); if (log) { Datedt =newDateO; logID = dt.getTimeO; String dbCreate = "CREATE TABLE 'log_" + logID ÷ ('id' INT NOT NULL AUTO_INCREMENT PRIMARY KEY, 'action' VARCHAR(255) DEFAULT' 'NOT NULL, 'success' INT NOT NULL, 'starttime' VARCHAR(255) NOT NULL, 'duration' VARCHAR(255) NOT NULL);"; log = dal.getDatabaseo.update(dbCreate);
I returnlog; I
1** * Performquery. *Creationdate: (3/2/2004 2:06:25 PM) *@returnboolean *@paramqueryStrjava.lang.String *1 public boolean query(String queryStr) {
longsTime = new DateO.getTimeO; boolean res = super.query(queryStr); long eTime = new DateO.getTimeO;
wnteLog(queryStr, res, sTime, eTime-sTime);
Page 66 of 92
-j Development of a Database Abstraction Layer —DevelopersGuide
if(res) { printMessage(DRV+ "Query caught:", "query was successful; (executed in "+(eTime-sTime)+ "ms)");
)else{ printMessage(DRV+"Querycaught:", "query was unsuccesful; (returned in "+(eTime-sTime)+"ms)"); I returnres;
}
1* * * Returna description of this driver. *Creationdate: (4/2/2004 10:37:41 PM) *@returnjava.lang.String *1 publicString toString() ( return"pass through ->" + super.toStringo; I
1* * * Performupdate *Creationdate: (3/2/2004 2:06:03 PM) *@returnboolean *@paramupdateStrjava.lang.String *1 publicboolean update(String updateStr) {
longsTime =newDateo.getTimeo; boolean res =super.update(updateStr); long eTime =newDateO.getTimeO;
writeLog(updateStr, res, sTime, eTime-sTime);
if (res) { printMessage(DRV+ "Update caught:", "update was successful; (executed in "+(eTime-sTime)+ "ms)");
}else{ printMessage(DRV+ "Update caught:", "update was unsuccesful; (returned in "+(eTime-sTime)+ "ms)"); I returnres; I
Page67of92 Development of a Database Abstraction Layer —DevelopersGuide
1** * Writeactions to our log. *Creationdate: (5/2/200412:04:57AM) *1 publicvoid wnteLog(String action, boolean success, long starttime, long duration) { if(log) { StringsuccessStr ="0"; if (success) { successStr="1"; I StringactionStr =action;IINOTE:should be escaped?? String logEntry ="INSERTINTO 'log_" + logiD ÷ rid', 'action', 'success', 'starttime', 'duration') VALUES (","+actionStr + UI +successStr + ", +starttime + ",'" +duration + ");"; dal.selectDatabase(0); IImakesure we select the log database boolean res =dal.getDatabaseO.update(logEntry); if(res) { printMessage(DRV+ "Logged:", actionStr);
}else{ printMessage(DRV+ "Failed to log the following statement:", actionStr); I
Listing11: other implemented methods
The full source of this example can be found on the WWW [url5]. The example driver gives you an idea of a possible driver implementation. Another possible driver implementation is for example an analyzer driver, analyzing errors by contacting a validation web service. [urI6]
Page 68 of 92 Development of a Database Abstraction Layer —DevelopersGuide
More information
More information can be found on the World Wide Web at: http://www.fmf.nl/—eelco/afstuderen/
You can contact me by e-mail at: [email protected]
The source code for the Database Abstraction Layer: http://www.fmf.nlJ—eelco/afstuderen/files/sources.zip
Note: The downloadable package includes programming examples and all the Java sources.
Links on the Word Wide Web: [un 1]: http://java.sun.com/j2se/1.4.1/docs/api/j ava/spllDatabaseMetaData.html [url2]: http://iava. sun.cornl j2se/ 1.4. 1/does/api/i avax/swing/JTree.html [url3J: http://www.fmf.nL/—eelco/alstuderen/files/javadoc/index .html [uri4IJ: http://java. sun .coml j2se/ 1.4. 1/docs/apilj avaxlswing/tablelTableModel .html [url5]: http://www.fmf.nl/—eelco/afstuderenlfiles/source/rug/eduldatabase/ [url6]: http://developer.mimer.se/validator/
Page 69of 92 Development of a Database Abstraction Layer —UserGuide
Appendix B: User Guide
Overview
• Getting started • Managing database connections • Browsing • Executing commands • Viewing results • The message window • More information
Getting started
The Database Explorer enablesyoutoexplore databases onvariousdatabase management systems.
Originally this application started as a small application to test the Database Abstraction Layer. The Database Abstraction Layer is a software layer positioned between the user application and the database making it easier to access databases from different DBMS vendors. The application steadily evolved in a user-friendly application embodying the powerful capabilities of the underlying software layer. One can say that it now serves both as a test case for the Database Abstraction Layer as well as an easy-to-use user application.
Before you run this application the first time it is important to verify wheter you have the Java runtime environment (version 1.2.2 or higher) installed. Versions before 1.4.1 might require the setting of the correct $CLASS_PATh variable. From version 1.4.1 onwards this should not be necessary. You can start the application by running start.bat or start.sh depending on your platform. While the application initializes, a splash screen will be shown and finally the main window will come up. The main window consists of various tabbed pages that can be brought up by clicking the corresponding tab. By default the application will start with the 'connection' page, ready to make a connection.
Page 7Oof 92 Development of a Database Abstraction Layer —UserGuide
Managing database connections
To open a new connection, click on the 'connections'-tab to bring up the 'connection'- page. In the 'connection'-page you will find the various connection properties at the left side of the page. Here you can fill in the properties required for your connection. When a property is left blank, the application will substitute the blank property with the default value for that property (see table 1).
Property Description Default Server hostname or IP-address of the server localhost or 127.0.0.1 Port port number of the database server default port number, specified by the driver Type the server/DBMS type to connect to AutoDetect, tries to auto detect specifies which driver should be used the correct driver Database selects the initial database none, connect to the DBMS, but no to a specific database User the usemame used for authentificationempty string1) Password the password used for verification empty string1) Access modeaccess the database in compatibility 'Entry Level SQL-92' mode or with full access control2) Remember remember the password if you store thedo not remember password password connection properties Store if stores connection for later usage do not store connection connection succeeds Table 1: overview of the connection properties.
'Some older versions of Java (1.2.2) andJDBC database drivers permit the use of empty usernames and passwords.
2) Youcan select 'Entry Level SQL-92' to access the database in compatibility mode or 'Full Native Access' for full access control. Note: in compatibility mode, statements will be checked for keywords before execution.
If you choose to select the 'store if connection succeeds' property, the connection properties will be saved for later usage in this session and can be made available again by clicking on the corresponding entry in the 'stored connections'-list at the right side of the 'connection'-page. It is also possible to save your stored connections to a file,so you can use it again the next time you use the application. You can do so by saving your session with the 'save session' command in the file menu.
Pare 71 of 92 Development of a Database Abstraction Layer —UserGuide
When have set all the necessary properties, you can now connect with your database by clicking the 'connect'-button. (Note: you can also choose to store your connection properties without making the connection by clicking the 'store'-button.) If all goes well, the status message at the bottom of the screen will be updated to 'connected' and the new connection will be added to the 'current connections'-list. If the status message says 'error', the application failed to connect to your database. You can find out what went wrong by clicking on the 'messages'-tab. The'message'-page shows all error- and warning-messages generated by the underlying software layers.
If you are finished using a certain database you can close it by selecting the database you want to close in the 'current connections'-list, followed by clicking the 'close'-button on the bottom of the tabbed page.
It is also possible to remove stored connections. You can delete stored connections by selecting the connections you want to remove in the 'stored connections'-list followed by clicking the 'delete'-button on the bottom of the 'connection'-page.
Note: the easiest way to close all open connections and to remove all stored connections is by starting a new session. You can do this by issuing the 'new session' command, which can be found in the file menu. Of course it is also possible to reopen saved sessions. You can reopen saved sessions with the 'load session' -command, which too can be found in the file menu.
Page 720192 I
Development of a Database Abstraction Layer —UserGuide
Browsing
You are probably curious what the current connections have to offer you. You can find out by selecting the 'browse'-page in the main window. This page shows a tree structure similar to the figure below:
onnections E locaihost [3 Properties
• DatabaseProductName: MySQL DatabaseProductVersion: 3.23.49-nt DriverName: Mark Matthews MySQLDriver Driveiversion: 2.0.14
• MaxConnections: 0 ProcedureTerm: Schemalerm: Non SQL92 keywords: AUTOJNCREMENTBINARV,BLOB,ENUM,INFILE,LOAD,MEDIUNINTOP1 SthnFunctlons: ACII.CHAR,CHAR_LENOTH.CHARACTER_LENOTh,CONCAT,ELT,FIELD,FIND SystemFunctions: DATABASE.USER.SYSTEM_USER,SESSION_USER.PASSWORD.ENCRYPT. TimeDateFunclions: DAYOFWEEKWEEKDAV,DAYOFMONTH.DAYOFYEAR,MONTH,DAYNAME.M UserName: nobody ConnectionURL IdentilterouoteString:'
Database does not support subquertes in IN statements. Database does NOT support subquerles In quantified expressions. Database supports table correlation names. Database supports transactions. Database does NOT support SQL UNION. Database does NOT support SQL UNION ALL El Databases ElCustomerOata cd_customer -clD(INT11) cPID(1NT11) --cName (VARCHAR 50) Iceirthday(DATE1O) -cSex (SET 2) cSmoke (SET 2) —cClass (CHAR 1) -cLastSent (DATE 10) El cd_email E cd_source Elcd_sourcedescription El MediaCenter Elemule
Figure1: browsing through your connections
In the root of this tree you will find all available connections. Each connection has two branches: one branch called 'properties' and another one called 'databases'. In the properties branch you will find all relevant properties for this specific connection.
Page 73 of 92 Development of a Database Abstraction Layer —UserGuide
Properties include basic settings like read only access and the used username, but also descriptions of the DBMS dependent features. Besides telling you what you can do, it also tells you what you cannot do! This is important because not all DBMSs support all standard SQL-92 functions. For example in the figure above, you can see that MySQL does not support UNIONS or sub queries. The databases branch shows all databases that can be accessed through the connection. When you click on a database, the nextbranch containing all the tables will be shown. Now, when you click on a table, the tree is expanded even further and the individual columns become visible. (See figure 1)
A pop-up menu is brought up if you double click a table or column. This pop-up menu allows you to perform the basic commands 'view' and 'drop' on the selected table or column. The 'view' command retrieves all records from the selected table or column and shows them in the 'result set'-page. The 'drop' command allows you to delete the selected column or table.
Note: some database drivers let you show and browse all the databases that are managed by the DBMS you are connected to, even if you made a connection with a specific database. Others show only the database where you made a connection to in the first place. A third type of database driver does show all available databases, but only permits you to browse the selected database. The exact behavior varies on thecombination of drivers, DBMSs and configuration used. For example: a database administrator can choose to allow a specific user only read-only access to a certain database while hiding all other databases contained in the DBMS.
Executing commands
Besides browsing through your database, you can also build and execute SQL queries. You can type your queries or updates in the text editor of the 'execute'-page. The statements can be executed by clicking the 'execute'-button
Warning: all statements typed in the editor will be executed on your database. When used without caution, you can seriously harm the contents of your database.
Page 74 of 92 Development of a Database Abstraction Layer —UserGuide
Viewing results
The 'result set'-page shows the results of the queries you executed from the 'execute'- or 'browse'-page. After you perform a query, the application will try to retrieve the query results as efficiently as possible by first creating a model of the data to represent. When possible, records are only sent from database server to the application when they are actually needed, thus saving network bandwidth. The created model also determines how your results are formatted. Although the application tries to format the resultsin the best possible way, it is possible that the formatting does not suit your needs. You can change the formatting easily. For example: if you want to make a certain column wider, go to one of the side edges of the column header with your mouse, click on it and move your mouse towards the neighboring column. You can also easily exchange columns by clicking on the header of the column you want to move and dragging the column to the place you want to position it. (See figure 2)
Data base File Edit ViewHelp
iai I Connecilonsi Browse ExecuteResuitseti Messages
Id name track_cnt
HIpF *" Unhiot 443 Rap 23 Blues 55 1 Techno 17 8 Other 148 9 Dsncr 30 iii R&D 38 Ciock 16 : FOp 141 13 ththe 1 14 rock 110 13 Rap/R&D 1 16 Punk I -______11 ,undtrk 54 - 18 Oeneral Rock 6 19 Trip-I-bp 9 j ii — . -
Retrieved 1 (SELECT' FROM 'neflukQjenresi Figure 2: swapping columns
Page 75 of 92 .
Development of a Database Abstraction Layer — User Guide
The message window
The message window shows all errors, warnings and otherinformation generated by the underlying software components. This is important because itenables you to keep track of all the operations performed 'behind the scenes'. If anything goes wrong you cantrace back the operations performed and where it went wrong. Eachline in the message window represents exactly one message with the following syntax (Seefigure 3): [software component] [action Itypeof message]: [message]
IViewHelp
ConnectionsBrowseExecuteResuitset Messages [DatabaselbstractiOflLayer]Registering driver: JDBC_IIYSQL [DatabaseithstraCtiOflLayet] Registering driver: JDBC_SQL_Servet (DatabaseAbstraCtionLayer]Unable toregister driver: JDBC_Oracle (java.lang.CIaZSNO (DatabaseExplorer]Driver not registered [DatabaseAbstractionlayer] Registering driver: JDBCPostgresSQL (DatabaseAbstractionLayet]Registering driver: JDBC_ODBC [DatabaseExplorer] Opening:8:(AutoDetect) [AbstractDatabaseLayer]Trying: rug. edu. database. JDBC_MYSQL (DatabaseExplorer] Connected with: !ySQL 3.23.49—nt [Mark Matthews' !7SQL Driver2.0. (DatabaseExplorer] Selecting Database: eRule (DatabaseExplorer][SeleCtiOnPath] —>localhost—>Databases—>eule—>eule_servet_ (DatabaseExplorer][QUtRT] (Connection]: localhost (Action]: SELECT * FROMeule_serv [AbstractDatabase] Note: Statelent contains 2 SQL—92 keywords (AbstractDatabase] Note: Statelent contains the following SQL-92 keywords:(SELECT,I (AbstractDatabase] Note: Query detected: (SELECT ' FROM'elule_servet_statsi (DatabaseExplorer] Selecting Database: ewule (DatabaseExplorer](SeleCtiOflPath] —>locaihost—>Databases—>ewule->eule_server_1 (DatabaseExplorer](QUERY](Connection]: locaihost [Action]: SELECT* FRON elule_serve [AbstractDatabase] Note: Stateaent contains 2 SQL—92 keywords (AbstractDatabase] Note:'-T?Th Statelent contains the following SQL-92keywords: [SELECT,
rivu ;LC.,IrrwMl Figure3: an example of the message window
Page 76 of 92 Development of a Database Abstraction Layer —UserGuide
More information
More information can be found on the World Wide Web at: http://www .fmf.nl/—eelco/afstuderenI
You can contact me by e-mail at: [email protected]
You can download the Database Explorer application at: http://www.fmf.nI/—eelco/afstuderen/files/sources.zip
Note: The downloadable package includes the Database Abstraction Layer and all the Java sources.
Page 77 of 92 Development of a Database Abstraction Layer —SourceCode
Appendix C: Source Code This appendix shows an overview of the most important Java classes that are implemented. A full description of these classes can be found in the generated Java documentation (JavaDoc) [1]. The full source code can be downloaded from the project page on the WW\V [2,3].
AbstractDatabase.java
packagerug. edu .database; import Java.util.*; import javax .swing.table. import java.sql*;
,** Implementsthe abstract database : * - handlesthe connection with the database * - handlesqueries and transactions * - buffersresuitsets * - storesits properties *1 publicabstract class AbstractDatabase { / final static variables *1 /**Auto-detectmode, tries different retrieve methods until a correct one has been found *1 finalstatic mtbyTrial=0; /**Retrievetable data using catalog information *1 finalstatic mtbyCatalog=1; /**Retrievetable data using schema information *1 finalstatic mtbySchema=2; /**Retrievetable data by comparing catalog and database names *1 finalstatic mtbyComparison=3; / Compatability mode: only use entry level SQL-92 grammar final public static mtuseEntrySQL92=0; /**Fullaccess mode: allow users full SQL access tothe DBMS / final public static mtuseFullAccess=0; /**SQLstatement is erroneous *1 finalpublic static mtisError=-1; /**SQLstatement is a query *1 finalpublic static mtisQuery=0; /**SQLstatement is an update *1 finalpublic static mtisUpdate=1; /**Howerror messages should be announced *1 finalstatic String ERROR ="Error:"; / How warnings should be announced *1 finalstatic String WARNING ="Warning:"; /**Hownotes should be announced *1 finalstatic String NOTE ="Note:"; / connection variables *1 /'howto retrieve/search the databases; default: autodetect */ protectedmtretrieve_method=byTrial; / how to access the DBMS; default: compatibility mode protected mtaccess_method=useEntrySQL92;
Page 78 of 92 I
Development of a Database Abstraction Layer —SourceCode
/**Theconnection to database
@see java.sql.Connection *1 protectedConnection con =null; /**Theresultset returned from the last query *1 privateResultSet rset =null; /**Thenumber of columns in the current resultset *1 privatemtcolumns=0; /**Setto true when last querywassuccessful *1 privateboolean succes =false; 1*configurationvariables *1 /**Theaddress of the database server. This can eitherbe a hostname or an IP-address *1 privateString server = /** Theport number of database server *1 privateString port = The initial selected database *1 /privateString database = /** Theusernaxne used to identify to the DBMS *1 privateString user = /** Thepassword used for authorization *1 privateString password = type of database server *1 private/ String type = follow number, if more connections with the same properties /coexist *1 private mtnumber=0; store password in configuration file ? default=false *1 privateboolean remember_password =false; /** * AbstractDatabaseconstructor. *1
public AbstractDatabase() { super0;
*Thisis the default implementation for SQL-statement checking *Thismethod can be overridden by the actual driver because not * allDBMSsprovide the same level of functionality. * NOTE:very basic checking. Canbeimplemented much better! *NOTE:the most important task is to gather information for the * userin case it goes wrong. * @return(isQuery, isupdate, isError) * paramString (the SQL statement to be checked) *1
public mtcheck(Stringstatement) {
/** * Closesthe connection with the database *1 public abstract void close();
*Executea SQL statement. * Thisis the default implementation for * executinga SQL statement. *Thecheck() method checks if the given * stringis a valid SQL statement compliant * withthe used access method. * @returnboolean
Page 79 of 92 Development of a Database Abstraction Layer —SourceCode
* @paramstatement java.lang.String *1 publicboolean execute(String statement) { mtstatementType=this.check(statement); if (statementType ==isQuery) { returnquery(statement); if (statementType ==isupdate) return update(statement); II not an update or query statement return false;
/** * Returnsthe database access method. * Currentlyavailable access methods are: * {useEntrySQL92,useFullAccess) * @returnmtaccess_method *1 public mtgetAccessMethod() { returnaccess_method;
/** * Deliversthe number of columns in the current table. * @returnmtcolumns *1 public mtgetColuxnnCount() return columns;
/** * Returnthe current connection object of this database * See:java.sql.Connection * @returnConnection (The connection with the database) *1
public Connection getConnection() { returncon;
/** * Returnsthe database name. * @returnString database *1 public String getDatabase() { return database;
* Retrievesthe fetch size in rows for the current ResultSet * object.Returns -l if the fetch size cannot be retrieved. * @returnmtthecurrent fetch size *1 public mtgetFetchSize() { try returnrset.getFetchSize();
) catch (Exception e) return -1;
Page80 of 92 Development of a Database Abstraction Layer — Source Code
* Deliversall the headers from the current table. * @returnan array of strings with headers * *1 publicString[] getHeaders() { mti; if (columns ==0)return null; String(] result =newString[columns]; try for (i =0;i ) catch (SQLException e) { I/do nothing return result; /** * Returnsthe DatabaseMetaData object belonging to the current * connection.The DatabaseMetaData object contains comprehensive * informationabout the database as a whole. * Formore information see Interface java.sql.DatabaseMetaData * @returnjava.sql.DatabaseMetaData The MetaDataObject belonging * tothe current connection *1 public DatabaseMetaData getMetaData() { try return this. getConnection() .getMetaData 0; ) catch (Exception e) { return null; /** * Returnsthe followup number * @returnmt_number *1 public mtgetNumber() { return number; * Returnsthe password used for connecting with this database *Thismethod will return the password ONLYif * remember_passwordis set to TRUE (using setProperties0) * @returnString password *1 public String getPassword() if (remeinber...password==true) return password; return Nil; I /** * Returnsthe connection port number. * @returnString port number. *1 public String getPort() { return port; Page 81 of 92 Development of a Database Abstraction Layer —SourceCode /** * Returnsthe retrieval method used /** * Returnsthe server on which this database is located. * Thiscan either be a hostname or an I? address. * @returnserver_string *1 public String getServer() return server; 1* *Returnsa tablemodel of the currently available ResultSet. * @seerug. edu .database.DatabaseTableModel * @returnrug. edu .database.DatabaseTableModel *1 publicTableModel getTableModel() { try return new DatabaseTableModel(rset); ) catch (Exception e) C //Creation of table model failed. Possible causes: old /1 JDBC 1.0 or buggy driver. Try creating a failsafe II table model based on the default table model return null; /** * Returnsthe type of DBMS used. * @returntype_string *1 publicString getType() C returntype; * Returnsthe usernaine used for the current connection. * @returnString user *1 public String getUser() C return user; /** * Returnsa new row from the ResultSet in an array of Strings. * Ifthere are no more results available 'null' is returned. * @returnAn array with all the values from a single row or * nullif empty. *1 public String[) nextRow() { mti; String[] result =null; try it(rset.nextO) ( result =newString[columns]; for (i =0;i Page82 of 92 Development of a Database Abstraction Layer—SourceCode result[iJ=rset.getString(i+ 1); ) catch (Exception e) { return null; } return result; * Openthe connection with the database *1 publicabstract boolean open(String connection_string); public abstract boolean open(String server, String database, String user, String password); * Printsa message to the standard output * @paramString tp; type of message. *Predefinedmessage types are: (ERRORIWARNINGINOTE) *1 publicvoid printMessage (String tp, String msg) ( N System.out.println((AbstractDatabase] +tp + " N +msg); * Performsa queryviathe given connection andstores * theResuitset and the number of columns. *Thisonly works for SQL NselectN statements. *Update,delete andinsertstatements have to * bemade with updateO. * @paramqueryStr The querytobe executed. * Triesto give the database driver a hint how manyrowsshould * befetched from the database when more rows are needed for this * ResultSetobject. If the fetch size specified is zero, the * databasedriver ignores the value andisfree to make its own *bestguess as to what the fetch size should be. *1 public void setFetchSize(int rows) { } /** *Setsthe basic properties for this AbstractDatabase. *1 publicvoid setProperties (String s, String o, String t, String d, String u, String p, boolean r, mtn) { server= port=0; type = database= user=u; password =p; remember_password =r; number =n; Page 830192 Development of a Database Abstraction Layer —SourceCode /** * Setsthe ResuitSet for this AbstractDatabase * treturnBoolean. TRUE if successful. *1 publicboolean setResultSet(ResultSet r) { * Checksif the last querywassuccessful. * @returnTRUE if the last querywassuccessful. *1 public boolean succesfull() { returnsucces; /** * Returnsa hwnan readable description of the database * connection. *1 publicString toString() { * Performsan update. * @paramthe SQL statement to be executed * @returntrue if statement was executed successfully *1 public boolean update(String updateStr) { IIEnd Of AbstractDatabase.java DatabaseAbstractionLayer.java packagerug. edu .database; import javax.swing. DefaultListModel; public class DatabaseAbstractionLayer { / current selected database *1 private AbstractDatabase adb=null; /'listof current connections *1 private DefaultListModel currentConnections =null; list of stored connections *1 private DefaultListModel storedConnections =null; / list of registered database drivers *1 private DefaultListMOdel registeredDrivers =null; default server type *1 private String defaultServerType =NAutoDetectN; Page 84 of 92 U Development of a Database Abstraction Layer —SourceCode /** * AbstractDatabaseconstructor * initializesthe database abstraction layer, * createsthe lists with the connections and drivers *1 publicDatabase1bstractionLayer() { super 0; currentConnections =newDefaultListModel0; storedConnections =newDefaultListModel0; registeredDrivers =newDefaultListModel0; /** * ClosesALLopenconnections * Thisis important because most DBMSs' canhandle * onlya limited amountofopen connections. *1 publicvoid close() { /** * Codeto perform when this object is garbage collected. *Anyexceptionthrownbya finalize method causes the * finalizationto halt. But otherwise, it is ignored. *1 protectedvoid finalize() throws Throwable { * Returnsthe list of all database connections currently * available. *1 publicDefaultListModel getCurrentConnections() { returncurrentConnections; /** * Returnsthe selected 'default' database. i.e.: the database *onwhich operations will be performed by default. *1 public AbstractDatabase getDatabase() { return adb; /** * Returnsthe default server type * The'default server type' specifies the driver to be used by *default. *1 public String getDefaultServerFype() C returndefaultServerType; /** * Returnsa list with the currently registered database drivers *1 public DefaultListModel getDrivers() C return registeredDrivers; — Development of a Database Abstraction Layer —SourceCode /** * Returnsthe list of stored connections. *1 publicDefaultListModel getStoredConnections() ( returnstoredConnectioflS; /** * Opensa connections with a database * Theconnection string specifies the server, database, username * andpasswordto be used. The connection string is driver *dependent,see the documentation of the driver youwant to use. *NOTE:normally users would use: open(String server, *Stringdatabase, String user, String password)which * isan driver independent implementationof the open() method. * @returnboolean, true if a connection canbeestablished *1 public boolean open(String connection_string) { /** * Opensa connection with a database. *Usersmay supply the server, database, usernaineand password to defaults will * use.When these parameters are not supplied the * beused. *NOTE:java >=1.4.1requires the use of a username/password * combination. *1 publicboolean open(String server, String database,String user, String password) { return this.open(server, database, user, password, defaultServerTyPe); } *Opensa connections with a database password and * Usersmay supply the server, database, username, * to use. driver defaults will be * Whenthese paraneters are not supplied the *used.NOTE: normally users would use:open(String server, * database, String user, String password)which is an String method. * driverindependent implementation of the open() *1 public boolean open(String server, Stringdatabase, String user, String password, String driver) { Page 86 of 92 Development of a Database Abstraction Layer —SourceCode *Tryto find and preload a driver (or any other class) publicboolean registerDriver(String driver) { try String package_name =this.getClass().getName() .substring(O, this.getClass() .getName() .lastlndexOf(' ClassLoader loader =this.getClass().getClassLoaderO; if (loader. loadClass (package_name +driver)! =null) { /*NOTE:method findClass not visible, using loadClass / /*instead.Alternative method: 1*Class.forName(package+driver).newlnstanceO; *1 registeredDrivers.addElement(new String(pkg_naine +driver)); System.err.println("Registering driver: +driver); return true; catch (Exception e) { System.err.println(1jnable to register driver: N + driver); return false; ) *Selectsthe default database to use. *1 public boolean selectDatabase(int index) { } *makea database object and store its properties for later * usage. * @param properties; see AbstractDatabase.setproperties() * @param mtmode:defines the mode for storage * 0:only add when a similar entry does not exist * 1:add a new entry, even when a similar entry exists * 2:replace entry, when a similar entry exists * @returnboolean false if entry could not be added or updated. *1 public boolean store(String server, String port, String type, String database, String user, String password, boolean remember....password, mtmode) { } II End Of DatabaseAbstractionLayer.java DatabaseTableModeLjava packagerug. edu .database; import java.io.*; import java.sql.*; import java.util.*; import javax.swing.table. *; import javax.swing. event. *; Page870192 Development of a Database Abstraction Layer —SourceCode /** * Thisclass takes a ResultSet object and implements the TableModel interface in terms of it so that a Swing JTable component candisplay * thecontents of the ResultSet. * Note1: This class requires a scrollable JDBC 2.0 ResuitSet. * Incase such a ResultSet is not available, the Database- * AbstractionLayerwill automatically create a compatible table model. * Note2: This class only provides read-only access to the created * tablemodel. *1 publicclass DatabaseTableModel implements TableModel { /**TheResuitSet to interpret *1 ResultSetresults; /**Additionalinformation about the results ResuitSetMetaData metadata; /**Howmany rows and columns in the table mtnuincols,nuxnrows; public final static mtCELL_MODE_PLAIN=1; public final static mtCELL_MODE_MAILING=3; public final static mtCELL_MODE_CUSTOMERS=2; private mtcell_mode=CELL_MODE_PLAIN; /** * Thisconstructor creates a TableModel from a ResultSet. *1 DatabaseTableModel(ReSUltSetresults) throws SQLException { this.results=results; IISavethe results metadata =results.getMetaDataO; IIGetmetadata on them numcols =metadata.getColumnCountU;IIHowmany columns? results.lastO; IIMoveto last row numrows =results.getRowU; II How many rows? public void addTableModelListener(TableMOdelListener 1) C) /** * Callthis when done with the table model. * closes the ResuitSet andtheStatement object used * tocreate it. *1 public void close() C try C results.getStatement().close(); catch( SQLException e ) Automaticallyclose when we are garbage collecting *1 protected void finalize() { close 0; /** * returncurrent cell mode *1 public mtgetCellMode() C return cell_mode; Page88 of 92 Development of a Database Abstraction Layer —SourceCode /** * ThisTableModel method specifies the data type for each column. *Wecould map SQL types to Java types, but there's no need for * that.All the returned data is converted to strings. *1 public Class getColuninClass(int column){ return String.class; } /** * Thesetwo TableModel methods return the size of the table *1 public mtgetColuxnnCount(){ return nuxncols; } /** * ThisTableModel method returns columns names from the *ResuitSetMetaData *1 public String getColumnName(int column) { try returnmetadata.getColumnLabel(column+l); ) catch (SQLException e){ return e.toStringO; } /** * Returnsthe number of rows *1 public mtgetRowCount(){ return numrows; *Thisis the key method of TableModel: it returns the value at *eachcell of the table. We use strings in this case. If *anythinggoes wrong, we return the exception as a string, so it *willbe displayed in the table. Note that SQL row and column * numbersstart at 1, but TableModel column numbers start at 0. * getValueAtcanprocessand return results/cells in different * waysusing setCellModeO. This allows getValue to return nice * Stringrepresentations of the data. public Object getValueAt(int row, mtcolumn) { try results.absolute(row+l); /1 Go to the specified row Object o =results.getObject(column+].); if (o ==null) { // No data return null; } else { II Convert data to a nice string representation if (cell_mode ==CELL_MODE_PLAIN) ( /1 Default cell mode, simple data conversion return o.toString() .trimQ; } else { ...1/other cell modes } } catch (SQLException e) { System. err .println (e) return e.toStringO; /** * Ourtableis not editable *Returnsfalse *1 public boolean isCel].Editable(int row, mtcolumn) { returnfalse; Page 89 of 92 Development of a Database Abstraction Layer —SourceCode publicvoid removeTableModelL.istener(TableModelListefler 1) () /** * setthe cell mode for this table model *see:getValueAt() *1 publicvoid setCellMode(int value) { cell_mode=value; *Setsthe contents of the specified cell. *Sinceits not editable, we don't need to implement this method *1 public void setValueAt(Object value, mtrow,mtcolumn)U // End Of DatabaseTableModel.java DatabaseTreeModel.java packagerug. edu .database; import java.sql.*; import java.util.*; import java.awt.*; import java.io.*; import javax.swing.*; import javax.swing. tree. *; import javax.swing. event. *; * Createsand manages a tree model. * TheDatabaseTreeModel enables a user to see a representation * ofall available connections including its properties, databases, * tablesand column names. *1 publicclass DatabaseTreeModel extends javax . swing.tree. Defaul tTreeModel II description of root public final static String ROOT_DESCR =Connections; II description of information node public final static String INFO_DESCR =Properties" II description of table node public final static String SCHEMA_DESCR =Databases; private DatabaseAbstractionLayer dbl =null; private DefaultMutableTreeNode root =null; * DatabaseTreeModelconstructor *1 public DatabaseTreeModel (javax. swing. tree .TreeNode root_node) { super(root_node); root =(DefaultMutableTreeNode)this .getRoot U; this.buildTreeO; Page 90 of 92 Development of a Database Abstraction Layer —SourceCode /** * DatabaseTreeModelconstructor *1 publicDatabaseTreeModel (javax .swing.tree .TreeNode root_node, boolean asksAllowsChildren) { super(root_node, asksAllowsChildren); root =(DefaultMutableTreeNode)this. getRoot 0; this .buildTree0; /** * DatabaseTreeModelconstructor * /** * Addsa single database connection to the tree *1 public void addConnection(int index) { /** * Buildsan entire tree based on all current connections *1 publicvoid buildTree() { if (dbl!=null) for (mti0;i'zdbl.getCurrentConnections0.getSize0; i÷+) addConnection(i); *Cleansan entire tree by removing all its children *1 public void cleanTree() { root .removeAllChildren0; this.reload0; /** * Createa new custom tree renderer for displaying *alternativeicons. Use: tree.setCellRenderer *toset the cell renderer. * @return javax.swing. tree. DefaultTreeCellRenderer *1 public DefaultTreeCellRenderer getCellRenderer() { DefaultTreeCellRenderer renderer =new DefaultTreeCellRenderer 0; renderer. setOpenlcon (null); renderer. setClosedlcon (null); renderer. setLeaf Icon (null); return renderer; Page91 of 92 Development of a Database Abstraction Layer —SourceCode Removesa single database connection from the tree *1 public void removeConnection(int index) { root.remove (index); this.reloadQ; ) II EndOf DatabaseTreeModel.java References [1]ThegeneratedJava documentation (JavaDoc): hup://www.fmf.nl/-eelco/afstuderen/files/iavadoc/index .html [2]The project page: httD: / /wwi. fmf .nl/—eelco/afstuderenl [3] The source code can be downloaded directly from: http://www.fmf.nl/—eelco/afstuderen/files/source.ziP II I I ji I Page92 of 92
* @return[byCatalog byComparison bySchema IbyTrial] *1 publicmtgetRetrieveMethod() { returnretrieve_method;
* @returnTrueif querywasexecuted successfully. *1 publicboolean query(String queryStr) {
* witha given classname. * @returnboolean false on failure, true if succesful * @paramString driver (classname) *1
*1 public DatabaseTreeModel (DatabaseAbstractionLayer d) super(new DefaultMutableTreeNode(ROOT_DESCR)); root =(DefaultMutableTreeNode)this .getRoot 0; dbl =d; this .buildTree0;