Patterns of System Integration with Enterprise Messaging Bobby Woolf Kyle Brown Independent Consultant Sr. Technical Staff Member, IBM [email protected] [email protected] The Pioneer Plaque by Dr. Carl Sagan, a message to extraterrestrial life forms 06/21/02 10:49 AM i Copyright © 2002, Bobby Woolf and International Business Machines Corp. Permission given to PLoP 2002 conference for reproduction. Patterns of System Integration with Enterprise Messaging Bobby Woolf and Kyle Brown Table of Contents 1. Introduction............................................................................................................... 1 1.1 Program-to-Program Communication................................................................. 1 1.2 Messaging Systems (MOMs).............................................................................. 2 1.3 Patterns for Messaging........................................................................................ 3 2. Message Channel Patterns ....................................................................................... 4 Messaging........................................................................................................................ 4 Point-to-Point.................................................................................................................. 7 Publish-Subscribe............................................................................................................ 8 Data Type Channel........................................................................................................ 10 Malformed Message Channel........................................................................................ 12 3. Message Patterns ..................................................................................................... 15 Command Message ....................................................................................................... 15 Document Message ....................................................................................................... 16 Event Message............................................................................................................... 17 Reply Message .............................................................................................................. 18 Reply Specifier.............................................................................................................. 20 Correlation Identifier..................................................................................................... 23 Message Sequence......................................................................................................... 26 Message Expiration....................................................................................................... 28 4. Message Client Patterns ......................................................................................... 30 Polling Consumer.......................................................................................................... 30 Event-Driven Consumer................................................................................................ 31 Message Throttle........................................................................................................... 32 Transactional Client ...................................................................................................... 33 Competing Consumers.................................................................................................. 35 Message Dispatcher....................................................................................................... 37 Message Selector........................................................................................................... 38 5. Messaging Application Patterns ............................................................................ 41 Pipes and Filters Messaging.......................................................................................... 41 Message Translator....................................................................................................... 42 Canonical Message Data Model.................................................................................... 44 Data Format Flexibility................................................................................................. 46 Message Router............................................................................................................. 47 Message Bridge............................................................................................................. 50 Message Bus.................................................................................................................. 51 6. Conclusions .............................................................................................................. 55 7. Bibliography ............................................................................................................ 56 06/21/02 10:49 AM ii Copyright © 2002, Bobby Woolf and International Business Machines Corp. Permission given to PLoP 2002 conference for reproduction. Patterns of System Integration with Enterprise Messaging Bobby Woolf and Kyle Brown 1. Introduction For almost as long as developers have been writing computer programs, they’ve needed a way for multiple programs to exchange information and work together. As computer systems have become more sophisticated and widespread, the need to be able to integrate individual applications into enterprise systems has grown. For example, it’s no longer enough for an accounts-receivable program to do its job well, it must also serve the broader enterprise by working with other applications that manage payroll, order management, shipping and receiving, and so on. This is a paper about how to integrate enterprise systems using a technology called messaging. Messaging is a key technology that helps make separate applications work together as one to serve the enterprise. 1.1 Program-to-Program Communication The general problem of program-to-program communication has been a constant and ongoing source of research, development and heartache almost since the dawn of the computer age. In particular, five different ways of program integration have risen to the forefront as the most commonly implemented solutions to this problem. 1. File Transfer – This was the original means of program-to-program communication, and is still the basis of many mainframe systems today. In it each program works on a physical file (stored perhaps on a hard disk, a tape drive, or a stack of punched cards) and then another program will take that file as its input and produce a new file, or modify the file if the storage medium allows it. While this has proven effective for solving many problems, issues of latency and resource contention have usually made it unattractive for today’s high-speed, high-volume applications. 2. Shared Database – Another popular mechanism derived from the File transfer mechanism is the shared database system. In this solution database software handles some of the resource contention issues by providing mechanisms for locking and unlocking the data appropriately, and also provides standard mechanisms for creating, deleting, searching and updating information. However, the latency issue remains even in this solution – before one program can use information it must be written to a database (a physical file) by the other program. 3. Raw Data Transfer – In this scheme different programs use a mechanism like a network data transfer protocol (like TCP/IP sockets) or a physical transfer mechanism like Shared Memory to communicate information between different programs. The drawback of this solution (which is again still used in millions of programs) is that it requires synchronous communication – each program must wait on the other to complete its request before processing a response. While it is possible to temporally disconnect the systems, this involves adding significant complexity to the overall system, and involves programming issues that few programmers are competent to deal with – for instance, it is up to the programmer to decide how to guarantee that a message is properly sent and received; the developer must provide retry logic to handle all the cases where the network link is severed, or the request or response was lost in transmission. 06/21/02 10:49 AM 1 / 59 Copyright © 2002, Bobby Woolf and International Business Machines Corp. Permission given to PLoP 2002 conference for reproduction. Patterns of System Integration with Enterprise Messaging Bobby Woolf and Kyle Brown 4. RPC – The RPC (Remote Procedure Call) mechanism is a way of reducing the complexity of the Raw Data Transfer approach by wrapping the network protocols within a layer of code libraries such that it appears to the calling and called programs that a normal procedure call had taken place. Again, RPC is extremely popular, and is the basis of modern systems like CORBA, RMI and EJB. However, the basic issues of synchronicity and guaranteed delivery still remain. 5. Messaging – Messaging provides high-speed, asynchronous, program-to-program communication with guaranteed delivery. This particular solution is often implemented as a layer of software called Message Oriented Middleware (MOM). The design of systems to use messaging is the subject of this set of patterns. As compared to the other four communication mechanisms, relatively few developers