Enterprise Integration with Ruby A Pragmatic Guide Maik Schmidt The Pragmatic Bookshelf Raleigh, North Carolina Dallas, Texas P r a g m a t i c B o o k s h e l f Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer, Pragmatic Programming, Pragmatic Bookshelf and the linking g device are trademarks of The Pragmatic Programmers, LLC. Every precaution was taken in the preparation of this book. However, the publisher assumes no responsibility for errors or omissions, or for damages that may result from the use of information (including program listings) contained herein. Our Pragmatic courses, workshops, and other products can help you and your team create better software and have more fun. For more information, as well as the latest Pragmatic titles, please visit us at http://www.pragmaticprogrammer.com Copyright © 2006 The Pragmatic Programmers LLC. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmit- ted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher. Printed in the United States of America. ISBN 0-9766940-6-9 Printed on acid-free paper with 85% recycled, 30% post-consumer content. First printing, March 2006 Version: 2006-5-4 Für meine Eltern. Ihr seid die Giganten, auf deren Schultern ich stehe! For my parents. You are the giants on whose shoulders I stand! Contents Foreword viii 1 Introduction 1 1.1 What Is Enterprise Software? ................ 2 1.2 What Is Enterprise Integration? .............. 3 1.3 Why Ruby? .......................... 3 1.4 Who Should Read This Book? ............... 5 1.5 PragBouquet ......................... 5 1.6 Acknowledgments ...................... 6 2 Databases 8 2.1 The Coupon Application ................... 9 2.2 Database Interface (DBI) .................. 25 2.3 Object-Relational Mappers ................. 28 2.4 Lightweight Directory Access Protocol (LDAP) . 51 3 Processing XML 79 3.1 A Short XML Reminder ................... 81 3.2 Generating XML Documents ................ 83 3.3 Processing XML Documents ................ 95 3.4 Validating XML Documents ................. 127 3.5 Are There Alternatives to XML? .............. 132 4 Low-Ceremony Distributed Applications 145 4.1 “I’d Rather Use a Socket” .................. 146 4.2 Remote Procedure Calls Using HTTP ........... 159 5 Distributed Applications with RPC 179 5.1 Another Day, Another Protocol ............... 179 5.2 We Will Take No REST, Will We? .............. 189 5.3 SOAP .............................. 200 5.4 CORBA, RMI, and Friends ................. 221 CONTENTS vii 6 Tools and Techniques 240 6.1 Internationalization and Localization ........... 240 6.2 Logging ............................ 261 6.3 Creating Daemons and Services .............. 279 6.4 Build and Deployment Process ............... 286 6.5 Project Automation with Rake ............... 303 6.6 Testing Legacy Applications ................. 314 A Resources 321 A.1 Bibliography ......................... 321 Foreword A few years ago, I came across the Ruby programming language, and I fell in love. Somehow, it just seemed to work the way my brain works—I can express myself in Ruby more naturally and with less intervening fluff than in any other language I know. I liked it so much I persuaded Andy Hunt to coauthor a book about it. That was back in 1999. Since then, a lot has happened in the Ruby world. The language went from release 1.6 to 1.8, and the standard library matured into something world class. It gained a standardized documentation system, a standard library distribution mechanism, and a fine build tool. I produced a second edition of Programming Ruby to celebrate. And now, for the first time, I can seriously say that Ruby is ready for the enterprise. The language is stable, the libraries are great, and there is a growing pool of talented and enthusiastic Ruby developers, all rising to the challenge. We see companies such as Amazon and EarthLink using Ruby for both internal- and external-facing projects. The problem is that—until now—there wasn’t much documentation on using Ruby in the enterprise. Sure, you can always find the API documentation for a library, but that doesn’t really explain the how and the why. Now the situation has changed. With Enterprise Integration with Ruby, Maik has done something I would have thought impossible. Not only has he doc- umented just how to use Ruby to create new enterprise solutions and to knit together existing applications, but he has also documented the backgrounds to all the technologies, along with how and when to use each. I consider this book a worthy partner to Programming Ruby. With it, you’ll exploit the power and flexibility of Ruby to create new solutions for your com- pany in record time. And, just as importantly, you’ll have fun. Dave Thomas The Pragmatic Programmers There are two types of complex systems: those that have grown out of simpler systems and those that do not work. Unknown Chapter 1 Introduction Have you ever worked for a big enterprise? Do you remember your expectations as you walked into work on that first day? Whistling as the sun shone brightly, you might have been thinking, “It will be great to work for <company name here>. They will have a professional envi- ronment where coffee is free and where every system has been specified accurately, implemented carefully, and tested thoroughly. Hmmmm... I wonder which database and programming language they use.” After your fifth cup of free coffee (around 9:07) you came to realize that the real world looks completely different from your expectations. Typi- cal enterprises use dozens, hundreds, and sometimes even thousands of applications, components, services, and databases. Many of them were custom-built in-house or by third parties, some were bought, oth- ers are based on open source projects, and the origin of a few—usually the most critical ones—is completely unknown. A lot of applications are very old, some are fairly new, and seemingly no two of them were written using the same tools. They run on heterogeneous operating sys- tems and hardware, they use databases and messaging systems from various vendors, and they were written in completely different program- ming languages. The reasons for this are manifold. You can find countless books that explain why the situation is so bad. You can even find books claim- ing that they help you prevent such chaos. This book uses another approach. We will not help you clean up this mess, but we will help you deal with the problems pragmatically. Instead of complaining that valuable data is spread across different database schemas or across databases from several vendors, we will write code that integrates it. We will take it even a step further and write new applications that aggregate WHAT IS ENTERPRISE SOFTWARE? 2 all your existing resources. It doesn’t matter if we have to use relational databases, LDAP repositories, XML files, or web services based on dif- ferent protocol standards. We will blend data from multiple, disparate databases to create new business knowledge. Along the way we’ll show you how to solve all the small day-to-day problems. These are the issues that occur over and over again, espe- cially when developing enterprise software. We will access relational databases such as Oracle and MySQL, and we will work with LDAP repositories. We’ll show you how to do application logging, how to deploy your software, how to automate tedious and error-prone tasks, and how to survive in an international environment. Oh, and as you might have guessed already from the book’s title, we will use Ruby to accomplish all these feats. 1.1 What Is Enterprise Software? In Patterns of Enterprise Application Architecture [Fow03], Martin Fowler writes, “Enterprise applications are about the display, manipulation, and storage of large amounts of often complex data and the support or automation of business processes with that data.” That’s a concise but nevertheless abstract definition, because every nontrivial piece of software has to store, manipulate, and display data. Video games do nothing else (and modern video games also need huge amounts of data that often can get complex). The key point in the pre- vious definition is the second part: the data in enterprise applications is used for business processes and not for rendering alien spaceships. Unsurprisingly, there are more differences between enterprise applica- tions and other types of software. For example, enterprise applications are often created only for a small user group that is in close contact with the development team, implying the developers know their cus- tomers very well. In extreme cases programs are written for only a single person (special report generators for the CEO, for example). Enterprise software demands a certain set of tools. Large amounts of data—complex or not—have to be stored somehow and somewhere. Often it is stored in relational databases, but it can also be in plain-text files or LDAP repositories. In addition, modern enterprise software is often based on distributed architectures consisting of many small to midsize components that perform specialized tasks and that are con- WHAT IS ENTERPRISE INTEGRATION? 3 nected by some kind of middleware such as CORBA, RMI, SOAP, and XML-RPC. Obviously, as an enterprise software developer, you’re better off if you know how to deal with such technologies. You shouldn’t be troubled by the details of reading from a relational database or accessing an LDAP repository. Mastering skills such as these help you concentrate on the fun stuff—the application itself.