The following paper was originally published in the Proceedings of the Third USENIX Conference on Object-Oriented Technologies and Systems Portland, Oregon, June 1997 Exploiting the Internet Inter-ORB Protocol Interface to Provide CORBA with Fault Tolerance P. Narasimhan, L. E. Moser, P. M. Melliar-Smith Department of Electrical and Computer Engineering University of California Santa Barbara, CA For more information about USENIX Association contact: 1. Phone: 510 528-8649 2. FAX: 510 548-5738 3. Email: [email protected] 4. WWW URL: http://www.usenix.org Exploiting the Internet Inter-ORB Protocol Interface to Provide CORBA with Fault Tolerance P. Narasimhan, L. E. Moser, P. M. Melliar-Smith Department of Electrical and Computer Engineering University of California, Santa Barbara, CA 93106 [email protected], [email protected], [email protected] Abstract Unfortunately, the current CORBA standard makes no provision for fault tolerance, which has led to research The Eternal system is a CORBA 2.0-compliant system that aimed at making CORBA-based applications reliable. One provides, in addition to the location transparency and the approach has been to build the fault-tolerance capabili- interoperability inherent in the CORBA standard, support ties into the ORB itself [16], as in Electra [8, 9] and in for replicated objects and thus fault tolerance. Eternal Orbix+Isis [5]. Another approach, adopted in the Open- exploits the Internet Inter-ORB Protocol (IIOP) interface DREAMS project [3], advocates that reliability be provided to ``attach'' itself transparently to objects operating over as part of the suite of object services available to the a commercial CORBA Object Request Broker (ORB). The ORB. While the former approach makes the fault tolerance Eternal Interceptor captures the IIOP system calls of the transparent to the application, it also involves considerable objects, and the Eternal Replication Manager maps these modification to the CORBA implementation to enable the system calls onto a reliable totally ordered multicast group ORB to take advantage of a multicast group communica- communication system. No modification to the internal tion system underneath it. On the other hand, the latter structure of the ORB is necessary, and fault tolerance approach simply adds an object group service on top of an is provided in a manner that is transparent to both the unmodified ORB, and uses no underlying multicast group application and the ORB. communication system, thereby making the system interop- erable and portable, but with the fault tolerance visible to the application programmer. 1 Introduction The Eternal system that we are developing provides fault tolerance transparently to the application using CORBA, Distributed systems consist of clusters of computers that are without modification to the ORB. The mechanisms for capable of both functioning autonomously and cooperating achieving reliability are hidden from the application pro- harmoniously to achieve a particular task. The integration grammmer, and concern only the system developer. The of an object-oriented paradigm with a distributed computing Eternal system can utilize any commercial implementation platform yields a framework in which objects are distributed of the CORBA 2.0 standard. Although Eternal is layered across the system. Objects invoke other objects, or are over a multicast group communication system, the vendor's themselves invoked, to provide services to the application. ORB does not need to be altered to utilize the fault tolerance The Object Management Group (OMG) has established that Eternal provides. Furthermore, the system is designed the Common Object Request Broker Architecture (CORBA) to enable objects running over different ORBs to interact [12, 14, 15, 17, 18], which is a standard for communications with each other. middleware that defines interfaces to distributed objects and The Eternal system exploits the services provided by the that provides mechanisms for communicating operations to Totem multicast group communication system [1, 6, 11] to objects by means of messages. The key component of this maintain the consistency of the replicas that are employed architecture is the Object Request Broker (ORB), which for fault tolerance. However, since Eternal only deals with handles requests to, and responses from, the objects in the interfaces of objects and of the ORB, any multicast group distributed system. communication system, with an interface and guarantees similar to those of Totem, can alternatively be used. Research supported in part by DARPA grant N00174-95-K-0083 and by Sun Microsystems and Rockwell International Science Center through the The structure of the Eternal system is shown in Figure 1. State of California MICRO Program grants 96-051 and 96-052. In this paper, we focus on the Interceptor, which ``catches'' Client Server Server Skeleton Object Object Method Method Client Stub Eternal Eternal Evolution Evolution Manager CORBA CORBA Manager ORB ORB Eternal Eternal Resource IIOP Interface IIOP Interface Resource Manager Manager Eternal Eternal Eternal Eternal Replication Interceptor Interceptor Replication Manager Manager Totem Multicast Totem Messages Platform Platform Figure 1: Structure of the Eternal system. the system calls made by the ORB to TCP/IP, and also on the CORBA provides location transparency, meaning that relevant part of the Replication Manager, which diverts the the client objects convey their requests only to their calls to Totem. In addition, Eternal supports the evolution of ORBs, which then undertake the task of locating a suit- a system by exploiting the replication of objects to perform able server object and then dispatching the request to live upgrades of objects and their interfaces. Resource it. Thus, a client object need not be aware of the lo- management is also provided for the creation, placement, cation of a server object since the ORB has access to and distribution of objects. this information. Every CORBA object is identified by an object reference, which is assigned to it by the ORB at the time the object is created. Client objects asso- 2 CORBA and the IIOP Interface ciate object references with their requests to enable the ORB to route their requests to the appropriate destina- The CORBA standard specifies an interface for each dis- tions. tributed object. This interface is written in the declarative The interoperability of CORBA arises in the context syntax of the OMG Interface Definition Language (IDL). of communication between heterogeneous ORBs. Every The language-specific implementation of a server object is CORBA 2.0-compliant ORB is equipped with the ability to hidden from client objects that require the service provided communicate using the Internet Inter-ORB Protocol (IIOP) by the server object; the server object can be invoked only [10, 12], which ensures that objects running over different through its interface. ORBs can interwork when they use the IIOP interface. Only Invocations of objects and responses from invoked ob- the ORB hosting an object needs to know the details of the jects are handled through the ORB, which acts as the inter- object, whileother ORBs that wish to interact withthe object mediary or ``communication bus'' for all of the interactions need only be able to address it. Every object is assigned an between the distributed objects in the system. At a client Interoperable Object Reference (IOR) for this purpose. object, a stub, generated by the IDL compiler, receives the The General Inter-ORB Protocol (GIOP) is a general set request, marshals the call into the format appropriate to the of specifications that enable the messages of the ORB to be request, and passes it to the ORB. At the server object, mapped onto any connection-oriented medium that meets a a language-specific mapping of the IDL specification, a minimal set of assumptions (reliable, byte stream-oriented, skeleton, unmarshals the parameters of the call and per- loss-of-connection notification). The Internet Inter-Orb forms any additional processing to invoke the appropriate Protocol (IIOP) is GIOP with the messages transported by method. The results of the operation are returned to the TCP/IP. By sending IIOP messages over TCP/IP, the ORBs client object via the ORB. can use the Internet as the backbone for their communi- cation. Server objects, that use IIOP to interact with their The services of a process group can be invoked trans- client objects in an environment of heterogeneous ORBs, parently, with no knowledge of its exact membership or the publish their references in the form of IIOP IOR profiles. location of its member processes. Thus, a process in the The primary motivation for the use of IIOP is that system can address all of the members of a process group all CORBA 2.0-compliant implementations can use this (including its own) as a whole, using a multicast group simple generic interface, irrespective of the internal details communication system, such as Totem. A process can send of the vendor's ORB, and the platform on which the ORB messages to one or more process groups, of which it may or operates. A number of commercial ORBs now provide may not be a member. These messages are totally ordered IIOP as their native protocol, since an increasing number of within and across all receiving process groups. CORBA applications require interoperability over different The Totem system provides reliable totally ordered mul- platforms and the ability to operate over the Internet. ticasting of messages to processes in process groups. Each message is assigned a unique timestamp, and these times- 3 The Eternal System tamps establish the total order of delivery of messages to the application. For messages multicast and delivered within the same configuration of processors, Totem provides these The Eternal system is designed to work with any com- message delivery guarantees despite communication and mercial off-the-shelf CORBA 2.0-compliant ORB with no processor faults, message loss, and network partitioning. modification whatsoever to the ORB.