DISTRIBUTED COMPUTING Managing Complexity: Middleware Explained Andrew T. Campbell, Geoff Coulson, and Michael E. Kounavis

sk a network designer what middle- in a distributed environment can be a nightmare. ware is, and he’ll characterize it as part For instance, to provide basic communication of the application.Ask an application services, programming languages support sock- designer what it is, and she’ll say it’s ets, which are end points in two-way communica- Apart of the OS.Which one is it? tion links between two programs running on a Traditionally, most definitions seeking to char- network. Sockets require client and server to acterize middleware suggest that it is the software engage in application-level protocols to encode that facilitates remote database access and systems and decode messages. Designing such protocols transactions. More recently, the term has come is cumbersome and can be prone to error. to be associated—somewhat Middleware is basically an alternative to sock- Middleware lets limitingly—with distributed ets; it abstracts the communication interface to platforms like the Open the level of method invocations. Instead of work- you manage the Software Foundation’s Dis- ing directly with sockets, you call a method— complexity and tributed Computing Environ- rather, you have the illusion of calling a ment (DCE) and the Object method—on a local object.The arguments of the heterogeneity of Management Group’s Com- call are in fact packaged up by your middleware mon and shipped off to the call’s remote target. In this distributed- Architecture (CORBA). And case, as in others, middleware provides the isolat- computing some have loosely applied it ing layer of software that shields you from the to systems as diverse as work- complexities of a heterogeneous environment. environments. flow support environments Here’s an and even to the Web itself. MIDDLEWARE FOR THE MASSES We believe the essential Several platforms have re- overview of the role of middleware is to man- cently become quite popular. These platforms middleware age the complexity and het- extend earlier distributed systems technologies— erogeneity of distributed like the Open Group’s DCE, Sun’s remote pro- world. infrastructures and thereby cedure call (RPC) interface, and ’s provide a simpler program- ming environment for dis- tributed-application developers. It is therefore Inside most useful to define middleware as any software layer that is placed above the distributed system’s A Technical Look at infrastructure—the network OS and APIs—and below the application layer. CORBA in Action In the distributed-computing model, your enter- Middleware Market Will Grow prise moves applications and data to where they can operate most efficiently: to desktop worksta- A Glossary of tions and to LAN, Web, or remote servers. But Middleware Terms dealing with the different protocols and interfaces

22 IT Pro September ❘ October 1999 1520-9202/99/$10.00 © 1999 IEEE NetWare—that weren’t object based. The characteristics of the new platforms include A Technical Look at CORBA in Action • masking heterogeneity in the underlying infrastructure by cloaking system specifics; We can best see how CORBA operates by exam- • permitting heterogeneity at the application level by ining the sequence of actions that take place when a allowing the various components of the distributed client application invokes a method in a remote application to be written in any suitable language; object.The sequence of actions begins when • providing structure for distributed components the client application somehow obtains an by adopting object-oriented principles; Interoperable Object Reference (IOR)— • offering invisible, behind-the-scenes distribution perhaps from a name server or as the return as well as the ability to know what’s happening value of a previous remote invocation. The behind the scenes; and client binds to this IOR and, as a result, is • providing general-purpose distributed services given access to a stub—a small software rou- that aid application development and deploy- tine—through which it can invoke the remote ment. object associated with the IOR. The client application sees the stubs as Prime examples of such platforms include OMG’s proxies for remote objects. In other words, CORBA, Microsoft’s Distributed Component they have a language-specific interface that Object Model (DCOM), and Sun’s Java remote corresponds to the IDL from which they were method invocation (RMI) system. generated. The function of stubs is to map— the term in CORBA is marshal—the argu- OMG’s CORBA ments and return values of method calls into Of the three major middleware technologies, CORBA and out of communications buffers, which can be sent is the most comprehensive in scope, largely because OMG across the network by the Generic InterORB Proto- explicitly addresses both the general- and vertical-market col (GIOP). aspects of middleware’s potential uses. Simply stated, On the server side, the implementer of an IDL pro- CORBA allows applications to communicate with one vides a language-specific implementation of the inter- another no matter where they are located or who designed face and registers instances of this implementation them. with the ORB so that their presence can be adver- The Object Request Broker (ORB) is the CORBA mid- tised. A standard interface—called the Portable dleware that establishes the client-server relationships. Object Adapter (POA)—provides object imple- Using an ORB, a client can transparently invoke a method mentations with a standard environment. on a server object, which can be on the same machine or The server implementation is interfaced to the across a network. The ORB intercepts the call and is underlying ORB by skeletons in much the same way responsible for finding an object that can implement the as clients are interfaced to the ORB by stubs.That is, request.The client does not have to be aware of where the skeletons are responsible for unmarshalling requests, object is located, what language it’s written in, or what OS passing the resulting arguments to the target object it uses. By intercepting these requests, ORBs can provide implementation and marshalling any results into interoperability between applications on different ma- GIOP reply messages. chines in heterogeneous distributed environments and can This description assumes that applications have seamlessly connect multiple object systems. their stubs and skeletons prelinked with their exe- In fielding typical client-server applications, developers cutables, which is the usual implementation provided use their own design or a recognized standard to define by most ORBs. There is, however, an alternative in the protocol to be used between devices. How they define the shape of dynamic invocation and dynamic skele- the protocol depends on the implementation language, ton interfaces. network transport, and a dozen other factors. ORBs sim- Dynamic interfaces are useful for applications such plify this process. With an ORB, the protocol is defined as browsers and databases in which it would be impos- through the application interfaces via a single implemen- sible to link all possible stub and skeleton classes stat- tation of a language-independent specification called the ically. The downside of this feature is that each Interface Definition Language (IDL). (See the sidebar “A invocation must be preceded by long and laborious Technical Look at CORBA in Action.”) sequences of code that are required to build the ORBs let you choose the most appropriate operating appropriate request. system, execution environment, and even language to use for each component under construction. And they allow

September ❘ October 1999 IT Pro 23 DISTRIBUTED COMPUTING

from the client and forwarding them Figure 1. OMG’s Object Management to the component in another process. Functionally and technologically, Architecture (OMA) reference model, in DCOM is similar to CORBA. For which CORBA figures prominently as example, both define an IDL and both ORB implementations. provide support services such as per- sistence,security,and transactions.But there are significant differences, the Application Domain Common foremost of which is that CORBA is an interfaces interfaces facilities open specification,while DCOM is not. Being a closed specification main- tained by only one company has one major advantage: DCOM has the ORB potential to evolve at a much faster rate than CORBA because there are no time-consuming politics involved in generating the next version of the Object services specification. But having many inter- ested parties working on CORBA Object services are domain-independent interfaces used by distrib- means it can be deployed far more uted-object applications. For example, a naming service would allow widely than DCOM. clients to find objects based on names. Currently, CORBA runs in most Application interfaces are developed specifically for a given applica- current OS environments, while tion. Because they are application-specific, and because OMG does not DCOM is deployed almost exclusively develop applications, these interfaces are not standardized. in the Windows environment. Fur- Domain interfaces play roles similar to object services and common thermore, porting DCOM to a wider facilities but are oriented toward specific application domains. One of range of environments is problematic the first domain interfaces was for the manufacturing industry. precisely due to the fact that there is Common facilities are oriented toward end-user applications. An no real specification other than the example of such a facility is one that links a spreadsheet object into a Windows implementation itself. report document. At the present time, it is still difficult to say which of these two significant standards will prevail. CORBA en- tered the arena first and has had a sig- nificant head start. But because it is you to integrate existing components. In an ORB-based included with every copy of Windows NT, DCOM is rap- middleware system, developers simply model the legacy idly catching up and—if and when it becomes widely avail- component using the same IDL they use for creating new able on a range of platforms—should prove a tough objects, then write wrapper code that translates between competitor. the standardized bus and the legacy interfaces. CORBA itself is merely one component in OMG’s Sun’s Java RMI Object Management Architecture (OMA). As Figure 1 Distributed systems require that computations running illustrates, the OMA specifies a range of architectural enti- in different address spaces,potentially on different hosts,be ties surrounding the core ORB, which is CORBA proper. able to communicate. Sun’s Java RMI offers a solution that enables communication between different program-level Microsoft’s DCOM objects residing in different address spaces.In such systems, DCOM is Microsoft’s proprietary distributed-object tech- a local surrogate object—a stub—manages the invocation nology. DCOM builds on the earlier Component Object on a remote object. Model (COM) architecture that provides a framework for Sun designed RMI to operate in Java.While other RMI application interoperation within a Windows environment. systems can be adapted to handle Java objects, these sys- A client that needs to communicate with a component in tems fall short of seamless integration because of their another process cannot call the component directly, but interoperability requirements with other languages. For has to use some form of interprocess communication pro- example, CORBA presumes a heterogeneous, multilan- vided by the OS. As Figure 2 illustrates, DCOM provides guage environment and thus must have a language-neu- this communication transparently by intercepting calls tral object model. In contrast, the Java RMI system

24 IT Pro September ❘ October 1999 assumes the homogeneous environment of the Java virtual Figure 2. Microsoft’s DCOM architecture, machine, which means the sys- a proprietary middleware technology tem can take advantage of the Java object model whenever similar to CORBA. possible, but also means you’re largely confined to Java. COM Client COM Component As Figure 3 illustrates, RMI runtime runtime consists of three layers: Security DCE RPC Security DCE RPC provider provider • the stub/skeleton layer, Protocol stack Protocol stack • the remote reference layer, and • the transport layer.

DCOM The boundary at each layer is network protocol defined by a specific interface and protocol; each layer, there- fore, is independent of the next and can be replaced by an alter- In the DCOM model, the Component Object Model (COM) provides object- nate implementation without oriented services to clients and components. COM uses a distributed comput- affecting the other layers in the ing environment (DCE) (RPC)—and whatever security system. For example, the cur- provider is being used—to generate standard network packets that conform to rent transport implementation the DCOM standard. is TCP-based (using Java sock- ets), but a transport based on User Datagram Protocol could be substi- tuted. Figure 3. Sun’s RMI system architecture, Despite this kind of flexibility, RMI remains a language-specific middleware sys- a Java-based middleware technology. tem.This can be a serious drawback because it requires writing all parts of an RMI-based distributed application in a single language. Application Client Server This means that non-Java legacy components cannot be included in an RMI-based appli- cation, nor can real-time components that Stubs Skeletons need to be written in a specialized language. Because of this, it seems unlikely that RMI RMI system Remote reference layer could ever be a major contender in the mid- dleware wars against CORBA and DCOM. Transport

OPEN IMPLEMENTATIONS The success of the middleware concept has The application layer sits on top of the RMI system, which con- naturally led to the desire to deploy the tech- sists of the stub/skeleton layer, the remote reference layer, and the nology in more diverse and demanding appli- transport layer. A remote method invocation from a client to a cation areas than in traditional, networked remote server object travels through the layers of the RMI sys- environments. For example, there are a num- tem to the client-side transport, then up through the server-side ber of middleware implementations in areas transport to the server. A client invoking a method on a remote as diverse as multimedia, real-time, and server object actually uses a stub or proxy for the remote object embedded systems, handheld devices, and as a conduit to the remote object. The remote reference layer is even mobile networking environments. responsible for carrying out the semantics of the invocation, and The key to supporting such a wide range of the transport layer is responsible for connection setup, connec- application areas is configurability, which tion management, and keeping track of remote objects residing in means breaking with the traditional view of the transport’s address space. middleware as an unalterable black box that

September ❘ October 1999 IT Pro 25 new types of multimedia encoding. It can also be used to Middleware Market load a new thread-scheduling algorithm or instantiate a Will Grow new buffer-management policy. To ensure that this high degree of configurability can be properly structured and managed, we are employing two Growth in the worldwide middleware market is on key technologies in our CORBA-compatible ORB: com- the fast track. Revenues will increase an astounding ponent technology and reflective programming. Tech- 438 percent from $2.2 billion in 1998 to $11.6 billion nology is already being used to structure by 2003. This data comes from Inter- applications but has not yet been employed national Data Corporation’s report in building middleware itself. Reflective “Middleware and Businessware: 1999 programming—explained in detail at the Worldwide Markets and Trends” (http:// URL mentioned earlier—complements www.idc.com). component technology by allowing better According to IDC, throughout 2003, access to the internals of a system. Basically, customer demand for middleware tech- reflective programming offers metalevel nology will shift among market segments. interfaces that are kept strictly separate Therefore, the growth rate of the overall from the usual base-level interfaces pro- market and of each segment are both vided by present-day middleware plat- important indicators of the overall mar- forms. ket’s potential. IDC believes the fastest- growing segment will be “businessware” systems— NETWORK PROGRAMMING middleware specifically designed for business trans- Middleware can be used in network programming to actions—which will earn an annual growth rate of 76.5 enable third-party software to run on top of open telecom- percent from 1998 to 2003, which compares with a 40 munications hardware.To program networks, you need to percent annual growth rate for the entire market. have the ability to access the internal controllers and From 1999 to 2003, IDC expects that event-driven resources in routers, switches, or base stations. processing and business process automation will in- For example, programmable IP routers need to support creasingly kick in to drive the growth rate. In 1998, open interfaces for managing the router state, including middleware and businessware use was highest on the accessing routing tables and controlling forwarding behav- Unix environment, which accounted for $884.1 mil- iors. In this way, new protocols could be quickly deployed lion, 41 percent of the total market’s revenues. Unix’s over large, heterogeneous networks consisting of multi- numbers are interesting because they indicate the vendor switching and routing hardware. platform is not losing out to NT. Existing network nodes run proprietary OSs that bun- US vendors dominated the overall market, captur- dle forwarding functions with the algorithms that control ing 74 percent of 1998 revenues.The region also con- and manage networks. Network programmability suggests sumed almost 49 percent of middleware software. an alternative approach to open routers, switches, and Western Europe was the next largest spender, base stations—abstracting their state, resources, and accounting for more than 32 percent of revenues. behavior—so that new network services and architectures can be realized. So how does network programmability benefit from has a fixed set of ORB services and a fixed per-platform middleware standards like CORBA? The answer is simple: implementation. Instead, these environments require tai- Middleware standards separate object interfaces from lored middleware profiles to run on different sorts of plat- their implementations. Middleware can run on top of forms for different sorts of applications. routers, switches, and base stations, which allows them to And sometimes the middleware has to be reconfigured interoperate at the highest possible level. Programmable while it is running, particularly in the case of mobile envi- nodes can expose IDL interfaces, allowing distributed ronments. In mobile environments, the middleware needs objects to run inside the network. IDL interfaces can be to be highly adaptive internally if it is to shield applications used for binding network algorithms to network resources. effectively from the effects of unpredictable disconnections Distributed objects can realize a wide range of diverse and rapid fluctuations in connection quality. network algorithms that include routing in the Internet, It was problems like this that impelled us to take a look connection management in ATM networks, hand-off in at developing our own open, modular ORB that can be wireless networks, and signaling for resource reservation. flexibly configured and reconfigured by applications. The Good interfaces at the service level are the macroscopic ORB we’re developing (http://www.comp.lancs.ac.uk) can counterparts of good coding practices at the implementa- be augmented with new protocols at runtime to handle tion level.

26 IT Pro September ❘ October 1999 A Glossary of Middleware Terms

Asynchronous message delivery: Sending a message Message Oriented Middleware Association (MOMA): regardless of the recipient’s readiness to receive it. An international consortium dedicated to the devel- opment and promotion of message-oriented middle- Callbacks: Programming functions that are executed ware to provide multiplatform, multitier message- when a specific type of event occurs. passing and message-queuing services for distributed Common Object Request Broker Architecture (CORBA): computing architectures. Specification from the Object Management Group that Middleware: Software that is used to move information describes how object-oriented networked applications from one program to one or more other programs in a communicate with one another. distributed environment, shielding the developer from Distributed Computing Environment (DCE): Open dependencies on communication protocols, operating Software Foundation’s specifications for a compre- systems, and hardware platforms. hensive integrated set of open services that support the Multicast: A communication service that allows a single development, use, and maintenance of distributed message to be delivered to a subset of the service’s applications. users. Distributed Component Object Model (DCOM): Object Request Broker (ORB): Software that provides Microsoft’s specification/model that allows developers language-independent, object-oriented remote proce- to create objects and have other programs and objects dure calls where a member function of an object can operate on them in a distributed environment. be invoked remotely using familiar object notations. Dynamic routing: Determining the best route from client Publish-subscribe: A method of in which to client in a constantly changing network topology. processes can subscribe to (register interest in) a sub- Fault tolerance: The ability of a middleware solution to ject and/or publish (send) messages to the subject. operate or to continue functioning in an environment Once a subject has been subscribed to by a process, the where elements (like processes or machines) have failed. process will receive any messages published to that sub- ject in the distributed application. Hierarchical namespace: Distributed data spaces of object information with dynamic interfaces that can be Remote method invocation (RMI): A specification from specified hierarchically, which is important when scal- that enables the programmer to ability is an issue. create distributed Java-to-Java applications in which the methods of remote Java objects can be invoked Load balancing: The ability of distributed applications from other Java virtual machines, possibly on differ- to disseminate their workloads as needed by the given ent hosts. application requirements. Remote procedure call (RPC): A call sent from one Message: A structure that contains the data being passed machine or process to another machine or process for between processes. some service. An RPC is synchronous, beginning with Message passing: Type of message-oriented middleware a request from a local calling program to use a remote that provides a transport for message exchange across procedure and ending when the calling program the components of a distributed application. Unlike receives the results from the procedure. message queuing, information is pushed out to inter- Scalability: The ability of middleware to perform within ested parties. applications of various size. Message queuing: Type of message-oriented middleware Stub: A client-side proxy for the remote object that imple- that shares with message passing the characteristics of ments all the interfaces supported by the remote object. namespace and delivery guarantees, but differs in that Thread: A single, sequential flow of control in a program. message delivery and processing is time deferred. Within a single thread, there is a single point of execu- Message-oriented middleware (MOM): Comprised of tion. In a multithreaded program, there are several message-passing and message-queuing middleware in threads within the program. The threads execute con- which information is passed in the form of a message currently, and there are multiple points of execution at from one program to one or more other programs. any time.

September ❘ October 1999 IT Pro 27 DISTRIBUTED COMPUTING

We have developed open programmable networks over like CORBA. Currently, enabling technologies for com- broadband and mobile telecommunications infrastructures ponents—such as scripting languages and multiple object (http://www.comet.columbia.edu) and are currently inves- interfaces—are undergoing development in the CORBA tigating how to automate the deployment and manage- world. CORBA 3.0—the next version of the specifica- ment of network architectures by dynamically dispatching tion—should make it easier to incorporate component and controlling distributed objects inside the network. technologies into your middleware environments. Another issue that will be important to anyone dealing CURRENT DEVELOPMENTS with middleware in the next few years is system integration. Middleware is now firmly established as an architectural For example,Sun is shipping Java 2 with a bundled CORBA concept in distributed computing. But the field of distrib- implementation, which will permit Java applets running in uted computing is far from static, which means that mid- a Web browser to access network services like legacy data- dleware will itself continue to change. bases that have been wrapped as CORBA objects. Perhaps the most significant recent development is com- ponents. Component-based middleware evolves beyond object-oriented software:You develop applications by glu- oday, perhaps the single most important issue in the ing together off-the-shelf components that may be supplied middleware world is performance. The current gen- in binary form from a range of vendors.This type of devel- T eration of middleware platforms has a deserved rep- opment has been strongly influenced by Sun’s JavaBeans utation for running slowly. However, significant progress is and by Microsoft’s COM and ActiveX technologies. being made in understanding why this is and what we can While component technology appears to be making do about it. Expect a great deal of performance improve- headway in the middleware community, it is still not alto- ments in the next few years. gether easy to apply component concepts to a vendor-neu- Related to performance is of course quality of service. tral and language-independent middleware environment Many commercially available ORBs lack interfaces for QoS specification and enforcement. For example, distrib- uted clients cannot explicitly specify priorities character- izing the way their requests are served.And conventional ORB-based systems incur significant throughput and latency overhead. In response to these kinds of limitations, a number of projects—like the Adaptive Communication Environment project (http://www.cs.wustl.edu/~schmidt/ TAO.html)—have been initiated to offer high-perform- ance, real-time middleware environments to application developers. But there are other issues as well. In the near future, watch for improvements in reliability, availability, and scal- ability. It is likely that such properties will be more effec- tively supported so that middleware platforms can be profitably deployed in hostile environments involving mil- lions of objects that have to adhere to strict performance requirements. ■

Andrew T. Campbell is assistant professor in the Electrical Engineering Department at Columbia University. Contact him at [email protected].

Geoff Coulson is a lecturer in the Computing Department at Lancaster University. Contact him at [email protected]. ac.uk.

Michael E. Kounavis is a PhD candidate at Columbia Uni- versity. Contact him at [email protected].

28 IT Pro September ❘ October 1999