PORTABILITY GAINS REALIZED WITH METAH AND ADA95

Bruce Lewis, US Army Aviation and Missile Command, Redstone Arsenal, Alabama

In each of these cases the high Abstract portability and supported functionality of MetaH is an Architecture Description Ada95 was a significant enabler. Current Language (ADL) developed to express and trends say that future avionics systems and evaluate the software architecture of avionics perhaps other safety critical applications will and flight control systems. It is intended for require space and time partitioning. Should not only description and analysis, but also for Ada95 and the Ravenscar profile support these integration of the software components on the capabilities? specified embedded hardware. This automated composition to specification with glue code Problem Statement generation allows rapid development and Due to significant dependency of evolution of real-time embedded mission and software on the execution environment safety critical systems. It also provides very (, operating systems, processors, high portability in these complex systems for buses, I/O devices), it is often very expensive software application code across various to re-host software as execution capacity is execution environments. This is accomplished exceeded. Embedded real-time software is by leveraging language and O/S standards, particularly difficult to re-host because of minimizing component dependencies, and by timing dependencies, performance constructing component timing relationships to requirements, complex processor architectures, specification across differing execution synchronous concurrent processes and platforms. specialized device interfaces. Avionics and This paper describes MetaH and flight control software adds to the complexity provides the results of a highly time by requiring multilevel safety, fault tolerance, sensitive application across significantly modular multiprocessor architectures, and very different embedded hardware/software complex multi-mode system behavior. execution environments. We initially developed Because of the complexity of upgrading a reusable MetaH specification for missile the software for a new processing environment, architectures, populated it with software one of the most significant risks in system components reengineered from a production development of large real-time systems, missile system, and executed it on single and especially avionics and flight control systems, dual i80960MC target configurations. We then is the problem of exceeding the processor retargeted this application to single and dual resources during the software development Pentium target configurations; and to a single process. Program after program has had to PowerPC configuration. We compare the costs scale back system requirements to fit on the of these exercises with estimated costs to do the hardware. Integration, maintenance and same tasks using traditional methods. upgrade costs are driven up since software must

1 be shoe-horned into the available resources for from impacting development. MetaH leverages as long as possible. the rapid construction of systems further by In addition, the execution capacity of adding the automatic integration of hardware many systems is not well understood. Avionics and software in accord with modeling used to systems have long lives with periodic upgrades. analyze the system. DSSA emphasized the use The software system control techniques often of domain specific application generators or used provide no quantitative indication of reusable component libraries in concert with schedulability bounds or the impact of changes ADLs to build systems. The second, the on the application. Even small changes can Evolutionary Design of Complex Systems result in unexpected and difficult to resolve (EDCS), was focused on our ability to build failures. Eventually, these changes exceed the systems that could be rapidly evolved and to capability of the system. predict the impact of change. EDCS started with multiple approaches but ended with a In this age of Commercial Off The strong focus on ADLs as a foundation for Shelf (COTS) processors, and with the very building highly evolvable complex systems. rapid increase in power of those processors, The DARPA program Dynamic Assembly for finding a higher performing processor is often System , Dependability and not the problem. Again, the greater difficulty is Assurance (DASADA) is extending the impact in moving the software onto a new execution of MetaH by addressing efficient dynamic and platform. static scheduling required for efficient The software portability problem also integration of soft real time dynamic manifests itself in fielded systems. Military applications (tactical internet) with hard real mission critical weapons and aircraft systems time (safety critical) applications to build more typically have very long lives and must be dynamic systems with retained dependability upgraded throughout their lifecycle. Capacity and assurance qualities. This also includes on the original processors is soon exhausted if dependable adaptation through architectural its not already exhausted when fielded. constraint based dynamic reconfiguration and Multiple processors become obsolete within the design time automated verification technology. development phase of these systems with Dr. Steve Vestal of the Honeywell Technology millions of dollars and years of effort spent to Center has been the principal investigator. upgrade or re-develop the software each time. Bruce Lewis has served as technical POC on Many more processors and buses will become both DARPA programs and has led the US obsolete over the system lifetime costing many Army Aviation and Missile Command, millions more and significantly delaying Research Development and Engineering system capabilities. A much more evolvable Center, Software Engineering Directorate approach that meets system requirements is (SED) laboratory demonstrations and needed. technology integration with MetaH since 1993. The US Navy, US Air Force, the Ada Joint History Program Office, and the US Army Space and Missile Defense Command have also funded The technology behind the MetaH ADL was MetaH related projects. The Open Systems - developed over several DARPA programs. The Joint Task Force (OS-JTF) has funded projects first, Domain Specific Software Architectures using MetaH’s advanced system building (DSSA), was concerned with using domain capabilities for modular avionics to evaluate specific system engineering knowledge to build the POSIX API and to impact GOA and SAE languages (ADLs) that could specify software OS API standards efforts. OS-JTF is currently architectures and analyze architectural supporting the standardization of an Avionics properties to prevent architectural problems 2 Architecture Description Language (AADL) fielded commercial avionics systems. Space based on MetaH. The synergistic integration of and time partitioning is also becoming advanced DARPA technology for evolvability, important in military systems since military SED lab resources and OS-JTF open systems avionics are now required to be FAA certified and standardization investigations has resulted if they fly in commercial airspace. With Glass in the advanced portability demonstrated in this Cockpits, avionics software becomes much paper. more flight critical. Space and time (S&T) partitioning is also desirable for avionics and An Overview of MetaH space systems to reduce the amount of hardware that must fly. These applications are also among those most likely to use Ada and In this DARPA research context, MetaH was where processing environment upgrades are developed for building missile and aircraft likely to be extremely expensive. avionics and flight control systems. It was MetaH was also developed to provide rapid designed to integrate the multiple domains of development and evolution of the system. One application software in avionics on a generated aspect of evolution is the ability to rapidly architectural backplane based on formal reconfigure these complex real-time systems to scheduling and implementation methods. The new hardware environments. The MetaH MetaH language provides a system designer a process for system development offers low risk simple but precise language for specification of rapid changes in the execution environment. architectural requirements from which it This fundamentally changes the high risk, tied- extracts the formal modeling parameters for to-the-hardware development approach we multiple analyses. It comprehends both currently use. Now, with MetaH, from a hardware and software components. It will software/execution environment perspective, generate the architecture integrating the programs can evolve the hardware multiple hardware and software components into a times during system development and field system complaint with the modeled behavior. with plenty of capacity using modern Current architectural analyses include processors. schedulability, reliability and safety/security. The language and toolset were The MetaH Process developed to meet the requirements for building state-of-the-art modular multiprocessor systems with multilevel safety The combination of MetaH language and security and fault management. It provides and tools that analyze and implement the for the specification and generation of dynamic system to the architecture specified provides a multi-mode behavior across multiple new paradigm for the development of processors under the real-time constraints of embedded real-time systems. The software flight control. It also can build from architecture becomes a specified, analyzable specification advanced space and time entity in the design process and a generated partitioned systems enabling very significant layer in the implementation. The system reductions (estimated at 80%) in re-validation becomes architecture based or architecture and re-testing costs when changes are made to driven. Since the architecture of the final an avionics or mission critical system. The system is generated to the specification, it approach of using a combination of time and results in a system with known architectural space partitioning is an emerging commercial properties and system execution behavior avionics technology first fielded by Honeywell which can be rapidly evolved with predictable on the Boeing 777 and now part of several impact.

3 Figure 1 shows the current software architecture specification is used to model and development paradigm with its specification of analyze schedulability, reliability (fault requirements and design on paper. Integrated handling), and safety/security dependencies. Project Teams alleviate some of the These issues need to be understood early in communication problems in this “Over-The- safety and time critical systems. Once the Wall” approach but the basic approach is systems engineer is satisfied with the specification for human interpretation and architecture, the components can be developed, system construction. Evaluations of reused from another project, or generated in architecture may occur with requirements parallel with incremental automated integration modeling tools and simulations but the results of the system with MetaH. The system is are reduced again to paper for impact on the easily re-integrated through re-generation from final system software. Modeling results tend to the specification. Early integrations may be on be disconnected from the next phase and from a workstation where behavior and system each other. Multiple complex modeling output can be validated. The final system is languages are required, one for each system automatically integrated from the specification analysis area. Integration of components into a and components, hardware and software, on the system is manual, often difficult, complex and target platform where execution behavior and very expensive. Code generation for system or results can again be validated. component analysis is for prototyping and requirements are again specified for human MetaH Process development of a traceable, testable integrated system. Architecture Based, Spec Integrated System

Current Software Process Paper Dependent, Error Prone

Requirements

No Walls ! System Integration Early Execution Rapid Change Integration to Spec Design and Implementation

Requirements Integration Analysis Design Implementation Figure 2

A major benefit is that the architecture Figure 1 and execution behavior specified is captured, not in paper or the heads of the designers, or in Figure 2 shows the new paradigm based on the scattered databases but in one specification that ability to specify an architecture (consisting of integrates the final system and generates the software and hardware components, their executive that drives its execution. Also a interfaces and the system execution behavior), single architectural specification is used for analyze its properties and then automatically multiple formal analyses and therefore the build the system to the specification. First the

4 system is generated compliant with each of the 6000 models used for analysis. 5000 Changes can be quickly made at the

specification level for load balancing, scaling, M 4000 a n

timing, message passing, shared data, new H

o 3000 u

components, adding fault response modes etc. r Current s Approach Since the processor, buses, or other hardware 2000 devices are part of the architecture they can quickly be changed to any of a predefined set. 1000 Using The defined set is user expandable as 0 MetaH demonstrated in this experiment. Execution Review 3-DOF Trans- 6-DOF RT-6DOF late Trans- environment dependencies reside in the toolset Test form RT- 6DOF Build rather than the application code allowing rapid MetaH Current Missile Debug Debug ports to new environments based on toolset ports. Figure 3. Cost in Manhours

The Demonstration This first effort provided a MetaH application that flew correctly on a specific The MetaH Application Ported embedded platform. The missile flight control The SED developed a generic missile application is quite sensitive to timing errors architecture and used it to re-engineer the on and delays since it flies at an angle of attack board software of an Army missile system. that causes it to overcorrect or tumble easily. It MetaH was used to specify the architecture took about a man month with an expert systems interfaces and timing and to integrate re- engineer to debug and tune the system to fly. engineered components and the dual 80960 For the onboard missile, we had 12,000 embedded hardware together to create an source lines of application code (non comment, system. In addition, a 6 Degree of non blank) in 11 concurrent processes and 800 Freedom (6DOF) Software-In-The-Loop lines of textual MetaH generated from a missile flight environment simulation was graphical specification. Developing the developed to allow us to evaluate flight graphical specification is significantly easier characteristics. It was also re-engineered into than typing textual code for most people. The MetaH so it could be executed in real-time. 800 lines of MetaH are used to build the MetaH The 6DOF took one and a half processors and executive which for the missile is 6100 source the missile on-board code ran on the remaining lines of code. Of those about 3000 are half processor. During this first development, generated based on the MetaH Missile MetaH reduced the total effort in man-hours by specification and the rest are standard services more than 40 % as shown in Figure 3. MetaH always uses. The number of lines generated depends on the complexity of the specification. While converting the application to the reference architecture we had designed and to MetaH required an investment, it was cost effective. The cost was less that would have been required to port a typical real-time OS solution on our first application of the

5 technology. Our data for the 6DOF simulation application flew correctly on the new (our first port) shows that MetaH saved us environment the first time it ran. No time was more in embedded system integration time spent modifying or tuning application code to (1200 man hours expected from prior get the timing right. simulation ports to real-time execution) than it We also changed our host environment took to learn (240 mh for training and from the Sun to a PC running NT, another development of the first MetaH process) and to optional upgrade to our environment. MetaH specify the remaining MetaH processes (180 toolset environments existed for both host mh). environments. It took us 31 hours to move the MetaH spec from the file system to the Honeywell experience porting the MetaH NT toolset Our next port was to the dual Pentium Honeywell developed their initial target using the Aonix with the MetaH toolset port to the i80960MC which Pharlap O/S. Here our application, when took 9 man months to develop but used no generated, broke the new dual processor MetaH standards in the interface layer. It was custom target. The Tundra chip commonly used on developed for the Tartan multi-program run- Pentium VME boards had a bug. We time and i80960MC hardware. In contrast, the developed our own dual processor MetaH single processor Pentium retarget using Target HW spec and integrated it into MetaH in standard Ada95 features took 90 hours 48 hours. We developed a workaround for the including some bug workarounds and Tundra chip in 80 hours. After the hardware performance tuning. Creating the multi- problems were solved, we re-specified the Pentium toolset port took another 75 hours. To system for processing on both processors, add execution monitoring (optional) to the regenerated it and had it flying in 1 hour. No targets took over 300 hours due to problems in time was spent modifying application code or the O/S. twicking application timing. Our third port was to the PowerPC. In Army experience porting the missile this case we did the toolset port independently application using MetaH and then regenerated the system from the The SED developed a number of specification and application components on missile application configurations on single and the new execution environment. We did not dual 80960s using MetaH’s specification develop the MetaH execution timing feature on capability and moved the 6DOF to a PC NT this port since it was not required to build or fly environment. Our Pentium port moved the the missile application. Our total time required 6DOF back onto the embedded environment. was 36 hours. This consisted of 10 hours to The first port to a totally new learn the Green Hills Compiler and VxWorks environment was to the Pentium Processor O/S sufficiently to get execution, 8 hours to add VMIC-VMIVME-7589 processor card using byte swapping between the PC with launch Aonix Real Time ObjectAda and Pharlap ETS control and the embedded environment, 16 O/S. Honeywell produced the toolset port. It hours to modify the MetaH HW spec for the took us 24 hrs to come up to speed with the Pentium to the PowerPC, and 2 hours to change Aonix tools and get the application code the MetaH Target package for the locations of running. Reintegrating the 6DOF into MetaH hardware ports. The missile application built and porting the MetaH reflective memory by MetaH flew correctly without changing interface took another 19 hours, which was a timing in the specification or changing system upgrade rather than porting time. The application code.

6 The result is that the simulation (executive generated by MetaH) gets the same numerical Why MetaH Reduces Porting Cost result as the embedded real-time (executive generated by MetaH for a different platform). This capability is provided in a multitasking MetaH improves portability of real-time environment which is auto-constructed by applications for two reasons. specification and is easy to modify and scale up or down. A second impact is that tactical code

in the simulation can be easily ported to the First, MetaH uses Ada 95 interfaces to embedded multiprocessor platform by re- provide functional portability between different specifying the target and allocating the execution environments (compilers, run-times, processes to processors. This provides rapid operating systems and processors). The use of porting from the simulation environment to standard language and OS interfaces is not embedded execution environment with correct new, but our work indicates they can be scheduling in both environments. successfully applied in real-time systems. Use of these interfaces makes it significantly easier to port the toolset to a new execution environment. We estimate an 8 to 1 reduction Analyzing the Cost Impact in porting costs using standard Ada95. A standard POSIX interface has also been developed but early indications say that more or porting a MetaH work is needed in the toolset to demonstrate application is basically retargeting the MetaH significant reductions with real-time POSIX. toolset if the target does not already exist. Low Standard configurations of POSIX, profiles, cost retargeting means more pre-existing and conformance testing will also be important. targets and far less risk for targets not yet available. The application itself will be rebuilt Second, and more novel, MetaH by MetaH on the target once it exists. This provides a target-independent tasking and assumes that the application code will execute message passing timing semantics. The properly in the new environment. For instance behavior of a real-time system is as dependent its important to use portable features of on the timing behavior as on the functional standard languages including precision of data behavior, and in order to achieve portability it types and functions. MetaH should also is necessary to provide standard timing the O/S calls rather than risking potentially interfaces as well as standard functional non-portable direct O/S calls. interfaces. MetaH insures that variations in execution time have no effect on Retargeting the MetaH toolset requires the values output by a system or the times at three steps. First, an interface layer that maps which those values are output as long as the MetaH executive calls onto the underlying overall system remains schedulable (which can micro-kernel, run-time or RTOS must be be assessed using the MetaH schedulability developed. The current toolset comes with analysis tool). standard interface layers that map MetaH executive calls onto either Ada 95 tasking and What this means is that the X (example real-time annex services, or onto real-time th 100 ) execution of process A always inputs POSIX services. Obtaining the interface layer th into the Y execution of process B (say 400 ), is usually a matter of working around target- on any system, whether a workstation or specific bugs and doing target-specific embedded or even embedded multiprocessor performance tuning. Second, a MetaH systems with A and B on different processors. specification of the new type of processor must 7 be developed. This specification identifies the 1 week. The benefit for a small application like interface layer code and also includes overhead the missile flight software for the case where a timing measurements obtained by running MetaH target did not already exist would still some benchmark programs developed for use be about 10 to 1 over the conventional with MetaH. Finally, a set of make scripts may approach. The benefit becomes much higher as need to be modified to invoke the specific the application size grows while the MetaH compiler, linker and other cross-development port time remains relatively the same. tools used for a specific target. In the best situation of predefined MetaH toolset targets and 8000 components, the port is extremely easy. Port 7000 6000

time would be the time needed to change the M

a 5000 target in the MetaH spec, run MetaH and the n

H compiler, download and execute. It takes about o 4000 u r 10 minutes to make a MetaH processor change s 3000 Current and rebuild on the new hardware. Most of the 2000 Approch 1000 time would be spent getting the new execution Using environment in place. However, if we assume 0 MetaH Review 3-DOF Trans- Current three days to get the execution environment 6-DOF RT- late Trans- MetaH 6DOF Test working and 10 minutes for MetaH to change form RT- Build MetaH Current 6DOF Missile Debug the specification, versus 9 months to port and Debug Port retune normally, then the ratio is 60 to 1. Since this ratio is dependent on the size and complexity of the system, the savings ratio Figure 4. Cost in Man-Hours after Port could be much higher. Also, many a program has run into significant re-tuning risk and cost at the transition between simulation and Figure 4 above adds man-hours for one embedded code execution. port to the missile project. One way to quantify the savings is by expert estimation. Our engineer, Ken Stachelbeck from Coleman Research, has over Additional lessons learned during the 18 years in hardware/software integration experience and has developed and ported many ports: hardware-in-the-loop simulation environments (missiles, mortars, smart bombs, RPV’s). He The MetaH implementation on the personally accomplished both the port from the 80960 interfaced at a low level to the Tartan 80960 to the Pentium, and from the Pentium to runtime system. Protected address space the PowerPC and so knows each of the targets (i80960MC, real-time Solaris, Lynx) environments involved. His estimate was that need a trap mechanism. The overheads on to accomplish the port and tune by hand these targets can be significantly reduced (unassisted by MetaH) for the dual Pentiums through the use of an eXtended Operating would have required 8 + 3 man months. The System (XOS) interface. The SAE GOA port and retune to the single PowerPC would subcommittee has evaluated the possibility of a have required 5 + 2 man months. In contrast, standard XOS but this seems quite difficult. given that we were porting to a new target Our current estimate is that the cost of context based on POSIX or Ada95, we probably could switches can be reduced by 80% if a low level have it running in less than 4 weeks, maybe in 8 XOS interface is used. A standard would currently covered in the Ada95. These services benefit all high performance applications with must be currently implemented in a target- greater portability. dependent manner, are optional, but are necessary to support certain MetaH capabilities The single address space targets may such as time partitioning, multi-criticality suffer increased overheads due to multiple scheduling, and slack scheduling of aperiodic software layers, an Ada run-time on top of an and incremental processes. The Execution RTOS. Performance tuning of the MetaH Timers under POSIX 1003.1d will provide toolset port can improve this at the expense of portable services if they are supported by real stepping outside the standard for a few critical time POSIX O/S suppliers. services. For example, the context switching time using the Aonix run-time layered on the Ada95 compilers currently use layered Pharlap ETS RTOS on the 233Mhz Pentium is operating systems to provide these capabilities about 10 us after some target-specific and the MetaH middleware generated calls performance tuning. The context switching POSIX rather than use Ada runtime features. time using the fully standard Ada 95 interface However, applications that would choose layer was originally about 50us. This was Ravenscar are very performance sensitive as reduced by substituting calls to ETS services well as safety critical. High performance for a few of the Ada 95 run-time services. avionics applications may need go under the POSIX API to achieve performance. Now we On the 20Mhz i80960MC with are back to a non-portable approach and more protected address spaces, we were getting 60 expensive retargets. Could a higher microsecond context switches using low level performance approach be available through traps (80960 architecture was optimized for fast Ada that would also provide the high level of multi-register set context switches). portability Ada95 has demonstrated? One of the benefits of MetaH is that performance tuning of the MetaH toolset port does not affect the portability of the MetaH specified system as demonstrated by the ports between the 80960, Pentium and PowerPC. Execution behavior will be correct if schedulable. We were very impressed with the portability of the MetaH toolset and missile components across the Ada95 environments. We were also impressed by the (non-standard) performance of the Tartan target port for space and time partitioned targets. However, to support space and time partitioning and advanced scheduling approaches to provide efficient mixed soft and hard real time requirements for modern avionics architectures with an Ada95 runtime, we also need POSIX like partitions as well as process and thread CPU timers. Accurate task timing requires services from the underlying micro-kernel, run-time or RTOS that are not

9

References:

1. Pam Binns, Matt Englehart, Mike Jackson and Steve Vestal, “Domain-Specific Software Architectures for Guidance, Navigation and control,” Honeywell Technology Center, Minneapolis, MN, International Journal of Software Engineering and Knowledge Engineering, Vol6, No. 2, 1996, pages 201-227. 2. Mark H. Klein, John P. Lehoczky and Ragunathan Rajkumar, “rate-Monotonic Analysis for Real-Time Industrial Computing,” IEEE Computer, January 1994. 3. David J. McConnell, Bruce Lewis and Lisa Gray, “Reengineering a Single Threaded Embedded Missile application onto a Parallel Processing Platform using MetaH,” $th Workshop on Parallel and Distributed Real- Time Systems, 1996. 4. Farnam Jahanian and Aloysius K. Mok, “Modechart: A Specification Language fore Real-Time systems,” IEEE Transactions on Software Engineering, v20, n12, December 1994. 5. Andrew L. Reibman and Malathi Veeraraghavan, “Reliability Modeling: An Overview for Systems Engineers,” IEEE Computer, April 1991. 6. Steve Vestal, “Fixed Priority Sensitivity Analysis for Linear Compute Time Models, “ IEEE Transactions on Software Engineering, April 1994 7. Design Guidance for Integrated Modular Avionics, AEEC/ARINC 651, Airlines Electronic Engineering Committee/ Aeronautical Radio Inc. 1991 8. Software Considerations in Airborne Systems and Equipment Certification, RTCA/DO-178B, RTCA, Inc., Washington D.. , December 1992 9. Steve Vestal, “Mode Changes in a Real-Time Architecture Description Language,” Second International Workshop on Configurable Distributed Systems, March 1994. 10. S. Ramos-Thuel and J.P Lehoczky, “Algoithms for Scheduling Hard Aperiodic Tasks in fixed-Priority Systems Using Slack Stealing,” Real-Time Systems Symposium, December 1994. 11. Pam Binns, “Scheduling slack in MetaH,” Real-Time Systems Symposium session on work in progress, December 1996, http://www.cs.bu.edu/techreports/96- 027-ieee-rtss96-wip/Home.html. 12. Steve Vestal, Laurent Guerby, Robert Dewar, David McConnell, Bruce Lewis, “Reimplementing a Multiprocess Distributed Paradigm for Real-Time Systems in Ada 95,” International Real-Time Ada Workshop, 1997. 13. Jonathan W. Kruger, Steve Vestal, Bruce Lewis, “Fitting The Pieces Together: System/Software Analysis and Code Integration Using MetaH,” Digital Avionics Systems Conference, 1998.

10