Programming the Perl DBI Alligator Descartes & Tim Bunce First Edition February 2000 ISBN: 1-56592-699-4, 350 pages The primary interface for database programming in Perl is DBI. Programming the Perl DBI is coauthored by Alligator Descartes, one of the most active members of the DBI community, and by Tim Bunce, the inventor of DBI. The book explains the architecture of DBI, shows you how to write DBI- based programs and explains both DBI's nuances and the peculiarities of each individual DBD. This is the definitive book for database programming in Perl. Table of Contents Preface 1 1. Introduction 5 From Mainframes to Workstations Perl DBI in the Real World A Historical Interlude and Standing Stones 2. Basic Non-DBI Databases 9 Storage Managers and Layers Query Languages and Data Functions Standing Stones and the Sample Database Flat-File Databases Putting Complex Data into Flat Files Concurrent Database Access and Locking DBM Files and the Berkeley Database Manager The MLDBM Module Summary 3. SQL and Relational Databases 41 The Relational Database Methodology Datatypes and NULL Values Querying Data Modifying Data Within Tables Creating and Destroying Tables 4. Programming with the DBI 57 DBI Architecture Handles Data Source Names Connection and Disconnection Error Handling Utility Methods and Functions 5. Interacting with the Database 76 Issuing Simple Queries Executing Non-SELECT Statements Binding Parameters to Statements Binding Output Columns do( ) Versus prepare( ) Atomic and Batch Fetching 6. Advanced DBI 97 Handle Attributes and Metadata Handling LONG/LOB Data Transactions, Locking, and Isolation Table of Contents (cont...) 7. ODBC and the DBI 116 ODBC-Embraced and Extended DBI-Thrashed and Mutated The Nuts and Bolts of ODBC ODBC from Perl The Marriage of DBI and ODBC Questions and Choices Moving Between Win32::ODBC and the DBI And What About ADO? 8. DBI Shell and Database Proxying 122 dbish-The DBI Shell Database Proxying A. DBI Specification 131 B. Driver and Database Characteristics 171 C. ASLaN Sacred Site Charter 249 Colophon 250 Author Interview 251 Description One of the greatest strengths of the Perl programming language is its ability to manipulate large amounts of data. Database programming is therefore a natural fit for Perl, not only for business applications but also for CGI-based web and intranet applications. The primary interface for database programming in Perl is DBI. DBI is a database-independent package that provides a consistent set of routines regardless of what database product you use - Oracle, Sybase, Ingres, Informix, you name it. The design of DBI is to separate the actual database drivers (DBDs) from the programmer's API, so any DBI program can work with any database, or even with multiple databases by different vendors simultaneously. Programming the Perl DBI is coauthored by Alligator Descartes, one of the most active members of the DBI community, and by Tim Bunce, the inventor of DBI. For the uninitiated, the book explains the architecture of DBI and shows you how to write DBI-based programs. For the experienced DBI dabbler, this book reveals DBI's nuances and the peculiarities of each individual DBD. The book includes: • An introduction to DBI and its design • How to construct queries and bind parameters • Working with database, driver, and statement handles • Debugging techniques • Coverage of each existing DBD • A complete reference to DBI This is the definitive book for database programming in Perl. Programming the Perl DBI Preface The DBI is the standard database interface for the Perl programming language. The DBI is database- independent, which means that it can work with just about any database, such as Oracle, Sybase, Informix, Access, MySQL, etc. While we assume that readers of this book have some experience with Perl, we don't assume much familiarity with databases themselves. The book starts out slowly, describing different types of databases and introducing the reader to common terminology. This book is not solely about the DBI - it also concerns the more general subject of storing data in and retrieving data from databases of various forms. As such, this book is split into two related, but standalone, parts. The first part covers techniques for storing and retrieving data without the DBI, and the second, much larger part, covers the use of the DBI and related technologies. Throughout the book, we assume that you have a basic grounding in programming with Perl and can put together simple scripts without instruction. If you don't have this level of Perl awareness, we suggest that you read some of the Perl books listed in Section P.1. Once you're ready to read this book, there are some shortcuts that you can take depending on what you're most interested in reading about. If you are interested solely in the DBI, you can skip Chapter 2 without too much of a problem. On the other hand, if you're a wizard with SQL, then you should probably skip Chapter 3 to avoid the pain of us glossing over many fine details. Chapter 7 is a comparison between the DBI and ODBC and is mainly of interest to database geeks, design aficionados, and those people who have Win32::ODBC applications and are desperately trying to port them to DBI. Here's a rundown of the book, chapter by chapter: Chapter 1 This introduction sets up the general feel for the book. Chapter 2 This chapter covers the basics of storing and retrieving data either with core Perl functions through the use of delimited or fixed-width flat-file databases, or via non-DBI modules such as AnyDBM_File, Storable, Data::Dumper and friends. Although the DBI isn't used in this chapter, the way the Storable and Data::Dumper modules are used to pack Perl data structures into strings can easily be applied to the DBI. Chapter 3 This chapter is a basic overview of SQL and relational databases and how you can write simple but powerful SQL statements to query and manipulate your database. If you already know some SQL, you can skip this chapter. If you don't know SQL, we advise you to read this chapter since the later chapters assume you have a basic knowledge of SQL and relational databases. Chapter 4 This chapter introduces the DBI to you by discussing the architecture of the DBI and basic DBI operations such as connecting to databases and handling errors. This chapter is essential reading and describes the framework that the DBI provides to let you write simple, powerful, and robust programs. Chapter 5 This chapter is the meat of the DBI topic and discusses manipulating the data within your database - that is, retrieving data already stored in your database, inserting new data, and deleting and updating existing data. We discuss the various ways in which you can perform these operations from the simple "get it working" stage to more advanced and optimized techniques for manipulating data. page 1 Programming the Perl DBI Chapter 6 This chapter covers more advanced topics within the sphere of the DBI such as specifying attributes to fine-tune the operation of DBI within your applications, working with LONG/LOB datatypes, statement and database metadata, and finally transaction handling. Chapter 7 This chapter discusses the differences in design between DBI and ODBC, the other portable database API. And, of course, this chapter highlights why DBI is easier to program with. Chapter 8 This chapter covers two topics that aren't exactly part of the core DBI, per se, but are extremely useful to know about. First, we discuss the DBI shell, a command-line tool that allows you to connect to databases and issue arbitrary queries. Second, we discuss the proxy architecture that the DBI can use, which, among other things, allows you to connect scripts on one machine to databases on another machine without needing to install any database networking software. For example, you can connect a script running on a Unix box to a Microsoft Access database running on a Microsoft Windows box. Appendix A This appendix contains the DBI specification, which is distributed with DBI.pm. Appendix B This appendix contains useful extra information on each of the commonly used DBDs and their corresponding databases. Appendix C This appendix contains the charter for the Ancient Sacred Landscape Network, which focuses on preserving sites such as the megalithic sites used for examples in this book. Resources To help you navigate some of the topics in this book, here are some resources that you might want to check out before, during, and after reading this book: http://www.symbolstone.org/technology/perl/DBI The DBI home page. This site contains lots of useful information about DBI and where to get the various modules from. It also has links to the very active dbi-users mailing list and archives. http://www.perl.com/CPAN This site includes the Comprehensive Perl Archive Network multiplexer, upon which you find a whole host of useful modules including the DBI. An Introduction to Database Systems, by C. J. Date This book is the standard textbook on database systems and is highly recommended reading. A Guide to the SQL Standard, by C. J. Date and Hugh Darwen An excellent book that's detailed but small and very readable. http://w3.one.net/~jhoffman/sqltut.htm http://www.jcc.com/SQLPages/jccs_sql.htm http://www.contrib.andrew.cmu.edu/~shadow/sql.html These web sites contain information, specifications, and links on the SQL query language, of which we present a primer in Chapter 3. Further information can be found by entering "SQL tutorial" or similar expressions into your favorite web search engine. Learning Perl, by Randal Schwartz and Tom Christiansen A hands-on tutorial designed to get you writing useful Perl scripts as quickly as possible.