Data Manager for Evolvable Real-Time Command and Control Systems

Eric Hughes, Roman Ginis, Bhavani Thuraisingham, Peter Krupp, John Maurer The MITRE Corporation, 202 Burlington Rd., Bedford MA, 01730-1420 {hughes,ginis,thura,pck,johnm}@mitre.org

from research to implementation. We also discuss current implementation efforts for the USAF’s AWACS Abstract (Airborne Warning and Control System). Moderniza- tion of the AWACS mission computing system involves We describe the design and implementation roughly a 40 staff-year effort over the next three years. of real-time data management services. These As with many such efforts, there is not enough money services combine technologies developed in available to replace the system (AWACS) in one block the context of real-time distributed object upgrade, so the approach is to use partial upgrades as management, object DBMSs, and scheduling. an opportunity to migrate the system gradually. This combination simplifies many of the services, and produces a result which is greater We use the following terminology to properly char- than the sum of its parts, because it can be acterize and give attribution for this work. Our RTDM used to improve the portability and flexibility design is a product of MITRE research in coopera- of real-time applications. tion with Victor Fay-Wolfe at the University of Rhode Island (URI). A descriptive vlhite paper on a real- 1 Introduction time Common Object Request Broker Architecture (CORBA) [COR96] is being developed by the real- Many real-time command and control systems have time PSIG of the Object Management Group (OMG); been deployed which use custom-written real-time a standards effort. We have prototyped some of the key data management (RTDM) software that is highly elements of our design. MITRE AWACS personnel, tuned to particular requirements. This custom soft- USAF, Lockheed-Martin, and Boeing have provided ware is not only expensive to develop, but because valuable feedback on the design and prototypes. Fi- it is entwined with the applications, must be rewrit- nally, we relate our work to the implementation being ten when requirements change. We have developed done by Lockheed-Martin and Boeing. an approach that makes real-time command and con- We began by examining the problem of building trol software more evolvable by using shared, flexible distributed, fault-tolerant, hard real-time command RTDM services. and control systems, in collaboration with the Soft- This paper surveys past and ongoing RTDM and ware Engineering Institute (SEI) at Carnegie Mellon distributed object systems work by the authors and University (CMU) [SLB+92]. We then identified and several related groups. At MITRE, this work involves evaluated the flexibility of several architectures, with mission-oriented research as well as a prototyping lab, feedback from the AWACS program, and chose a de- both funded by the US Air Force (USAF). These two sign based on distributed objects [BFG+95]. We also groups participate in the transfer of RTDM technology worked with the URI group to extend CORBA to sup-

Permission to copy without fee all OT part of this material is port real-time systems [TKH+96]. One of the results granted provided that the copies are not made OT distributed for of this work was the identification of requirements for direct commercial advantage, the VLDB copyright notice and the use of real-time CORBA in command and con- the title of the publication and its date appear, and notice is trol systems. We prototyped real-time CORBA by given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, OT to republish, requires a fee porting Xerox’s Inter-Language Unification (ILU) Ob- and/or special permission from the Endowment. ject Request Broker (ORB) [PAR971 to the Lynx real- Proceedings of the 23rd VLDB Conference time operating system [Lyn97] and by providing a Athens, Greece, 1997 distributed scheduler supporting rate-monotonic and

558 deadline-monotonic techniques. maining AWACS applications in the legacy environment. The ported surveillance and tracking applica- 2 Approach tion uses our prototype data manager to store, update, and retrieve information on tracks, which represent Here we outline our approach for RTDM services for sensor information about aircraft. command and control applications. Our design con- ORBS are not optimized for efficient object transfer sists of services that form a framework for real-time and caching, which are better handled by an ODBMS systems, as shown in Figure 1. Our infrastructure in- [CB97] that provides additional RTDM services. In sulates applications from the underlying real-time op- our design, the ODBMS uses the ORB for communi- erating system. The ODBMS, Event Manager, and cation that does not involve transfer of objects, such scheduler also use this infrastructure. It is relatively as messages describing triggered events. In the Lock- easy to migrate a legacy system to use our infrastruc- heed implementation, the ODBMS uses the ORB to ture [HR97]. As with ordinary ODBMS and CORBA allow a local thread to connect to a remote database. systems, migrated applications can then evolve with In addition, in our design, the ORB uses the ODBMS minimal changes to RTDM services. Many technolo- for transactions and persistence of objects. For ex- gies are important to this work but here we discuss ample, the Naming Service of the ORB could use the only the critical subset. ODBMS to store its mapping from object references to implementations. Other approaches have been taken to integrate ORB and ODBMS. The OMG Persistent Object Ser- vice (POS) allows CORBA objects to be stored and retrieved using many possible means (files, databases, etc.) [OMG94]. H owever, this is limited as a means Real-Time Infrastructure for integration of real-time ORBS and ODBMSs. For I (ultimately a real-time ORB1 I example, the POS ignores the role of the ODBMS in efficient transfer and caching of fine-grained objects. Real-Time Operating SyStem I I Reverbel has investigated ORB and ODBMS integration in the context of medical information systems, Figure 1: Evolvable Real-Time System. but his efforts are geared to commercial ORB/ODBMS adapters [Rev96]. It remains to be seen whether these We have chosen an object-oriented model for many adapters could be modified for use in real-time sys- of the same reasons given for non-real-time applica- tems. tions . We also have found objects to be convenient In our design, both ORB and ODBMS are further for the specification of real-time constraints (e.g., how simplified by the use of a scheduler, which completes often an object should be updated) and of seman- the set of RTDM services. This approach is detailed in tic concurrency-control metadata (e.g., which meth- the real-time CORBA white paper and is being imple- ods of an object can be executed concurrently). This mented by Lockheed. The scheduler allows threads to approach in effect extends the object model to sup- request resources for computation based on deadlines port real-time constraints, as is done in RTSORAC and priorities assigned. Many of these computations [FWCPP94]. In most cases, this involves the creation may involve ODBMS transactions. The concurrency of (new) object definitions for data in legacy systems, control service of the ODBMS may need to change and these object definitions are then annotated with the priority of a computation (as described below), real-time metadata. For an existing object-oriented which in our design can be achieved by requests to the system, we believe that annotation with real-time in- scheduler. Our design also isolates applications from formation and enforcement of real-time constraints im- the operating system by providing scheduling, com- proves the system. munication, and memory management services, which At the heart of our RTDM services is a real-time improves the portability of applications (even beyond ORB. The use of an ORB greatly simplifies communi- the portability afforded by POSIX-compliant operat- cation between distributed objects, resulting in a more ing systems). This approach is being carried out in natural and flexible integration of systems. We have the current implementation. used the Lynx operating system environment and ILU In our design, the ODBMS provides constraint en- to prototype a real-time ORB (as described in the real- forcement, concurrency control, querying, and persis- time CORBA white paper). We have also ported an tence services. We prototyped an event manager for experimental surveillance and tracking application of constraint enforcement, and found that this approach AWACS to the prototype ORB while leaving the re- is viable for command and control applications. How-

559 ever, we used a real-time relational DBMS (RDBMS) ory as the primary store and our design includes persis- [THK+96] with this prototype, and found that the tence to store database objects. The Lockheed imple- poor performance of the RDBMS and the impedance mentation uses in-memory databases but will provide mismatch between the event manager and underlying persistence for logging, checkpointing, and application databases makes the RDBMS a poor choice. Our cur- needs. rent design adapts the event manager to an ODBMS We believe that a temporal object model is ex- [THK+96]. Lockh eed ‘s implementation supports the tremely important for real-time applications, and this CORBA Event Service, which in turn supports the model should be supported by RTDM services. In specification and delivery of events. Our constraint our design, objects can be defined to have temporal enforcement design uses this Event Service to create constraints (as described above), timestamps, tempo- events caused by updates to ODBMS objects and to ral checkpoint requirements, etc. As with concurrency deliver these events to the appropriate handlers, which control, our design leaves the choice of temporal object can enforce constraints. For example, a tracking ap- model to the needs of the system. plication might specify a constraint on how often a track must be updated, and violation of this con- 3 Future Work straint should cause an event which eventually results in an update to the track. This approach improves Commercial support is emerging for several key RTDM the evolvability of applications by making such con- services. We expect that commercial implementations straints explicit and easier to change as the system of real-time ORBS (including the Lockheed implemen- requirements change. tation) will be available within the next two years. We are pursuing several possibilities for concur- One of the important challenges for these products is rency control. We designed the affected set priority the ability to adapt to the needs of the system, such ceiling (ASPC) protocol which integrates priority ceil- as choice of concurrency control protocol or temporal ing with semantic locking, and we have implemented a object model. We suspect that real-time ORBS will prototype data manager with this protocol [TKH+96]. require other forms of adaptability, such as choice of In ASPC, the concurrency control service may need to communication protocol, selectable based on the sys- change the priority of a transaction, and this can be tem state and quality of service requirements. We are done by a request to the scheduler. Lockheed is also currently investigating ways to achieve this adaptabil- investigating a multi-version concurrency control ap- ity [GHK+97]. proach. Soparkar et al. discuss other options, including There are many possible applications of the RTDM optimistic concurrency control [SKS96]. Our design al- services we have described. We have discussed lows for a choice of concurrency control protocol based AWACS, a system that combines the avionics and com- on the needs of the application. We also prototyped mand and control domains. Another domain is in- a metadata manager which stores and controls access formation survivability, in which a user could bring to metadata (e.g., real-time constraints) for transac- down a system by over-using its resources [KTM97]. tions and database objects. In the prototype, object Real-time ORBS have application in many domains definitions and transactions are pre-analyzed, and the [COR96], and real-time ODBMSs have similar poten- resulting metadata is used to determine potential con- tial [Sop96]. flicts between executing transactions. A Query Service was not part of our original design, 4 Acknowledgments but feedback from AWACS groups and current implementation suggest that a real-time ODBMS should The MITRE mission-oriented research was performed support an object query language (O&L). Simple real- under contracts with the US Air Force via Rome Labs. time queries can be compiled into efficient ODBMS ac- We wish to thank the members of the cited groups for cesses, which allow end users to interact with an object their consideration of and feedback on new technolo database in a familiar fashion. Queries are also use- gies. MITRE AWACS personnel, including Jonathan ful for non-real-time access to databases (e.g., main- Bernhardt, have been willing users of new technology tenance and browsing). Lockheed is implementing a and new project management approaches. We wish to simple O&L for AWACS based on ODMG [CB97]. thank Maj. Michael Milligan and the AWACS project In our design, the ODBMS is primarily an in- leadership of the US Air Force for their support. Our memory database, which agrees with the consensus of work has benefited from collaboration with Victor Fay- the real-time DBMS community [Sop96]. However, ob- Wolfe at URI and with the real-time CORBA PSIG. jects still must be made persistent, either to support We thank Lockheed-Martin for their willingness to ex- recovery of ODBMS transactions or other application change implementation ideas and Boeing for their sup- needs. Our data manager prototype used shared mem- port.

560 References [SKS96] Nandit Soparkar, Henry Korth, and Abraham Silberschatz. [BFG+95] Ed Bensley, Larry Fisher, Michael Gates, Time-Constrained Transaction Manage- James Houchens, Arkady Kanevsky, ment. Kluwer Academic, 1996. S. Kim, Peter Krupp, Alice Schafer, and Bhavani Thuraisingham. Evolvable Sys- [SLB+92] L. Sha, J. P. Lehoczky, M. Bodson, tems Initiative for Real-Time C3: In- P. Krupp, and C. Nowacki. Position frastructure Requirements. In ICECCS, Paper: Responsive Airborne Radar Sys- Nov. 1995. tems. In IEEE International Workshop on Responsive Systems, 1992. [CB97] R. G. G. Cattell and Douglas K. Barry. The Object Database Standard: ODMG [Sop961 Nandit Soparkar. Post-Workshop Re- 2.0. Morgan-Kaufmann, 1997. port for Workshop on Databases: Active and Real-Time (DART). In http://www. [COR96] CORBA/RT PSIG. CORBA/RT White eecs.umich. edu/-soparkar/Dart96/ Paper. OMG, 1996. report.ps, 1996.

[FWCPP94] Victor Fay-Wolfe, Lisa Cingiser, Joan [THK+96] Bhavani Thuraisingham, Eric Hughes, Peckham, and Janet Prichard. A Peter Krupp, Gary Gengo, Alice Schafer, Model for Real-Time Object-Oriented and Mike Squadrito. Active Real-Time Databases. In Conference on Database Database Management for Command and Expert Systems Applicaitons, 1994. and Control Applications. In Databases: Active and Real-Time (DART), 1996. [GHK+97] Roman Ginis, Eric Hughes, Peter Krupp, John Maurer, and Bhavani Thuraising- [TKH+96] Bhavani Thuraisingham, Peter Krupp, ham. Object-Oriented Data Management Eric Hughes, Mike Squadrito, and Vic- and Distribution for Real-Time Com- tor Fay-Wolfe. Transaction Model and mand and Control Systems. To appear Concurrency Control for Evolvable Real- in RTDB, 1997. Time Command and Control Systems. In ATMA 96, 1996. [HR97] Eric Hughes and Arnon Rosenthal. Mi- gration of Legacy Applications to Use Enterprise Services. In preparation, 1997.

[KTM97] Peter Krupp, Bhavani Thuraisingham, and John Maurer. Survivability Issues for Evolvable Real-Time Command and Control Systems. In DARPA Informa- tion Survivability Workshop, 1997.

[LwW Lynx. The Lynx Operating Sys- tem. In http://www.lynx.com/products/ lynxoshtml, 1997.

[OMG94] OMG. Common Object Services Specifi- cation, Vol. 1. Dot. No. 94-1-1, March 1994.

[PAR971 Xerox PARC. Inter-Language Unifica- tion (ILU). In ftp://parcftp.parc.xerox. com/pub/ilu/ilu.html, 1997.

[Rev961 Francisco Reverbel. Persistence in Dis- tributed Object Systems: ORB/ODBMS Integration. PhD thesis, University of New Mexico, April 1996.

561