Multimed Tools Appl (2017) 76:8195–8226 DOI 10.1007/s11042-016-3448-5 Synthesis of simulation and implementation code for OpenMAX multimedia heterogeneous systems from UML/MARTE models D. de la Fuente1 & J. Barba1 & J. C. López1 & P. Peñil1 & H. Posadas1 & P. Sánchez 1 Received: 21 January 2015 /Revised: 31 January 2016 /Accepted: 15 March 2016 / Published online: 30 March 2016 # Springer Science+Business Media New York 2016 Abstract The design of multimedia systems is becoming a more and more challenging task due to the combination of growing functionalities and strict performance requirements along with reduced time-to-market. In this context, the OpenMAX initiative defines a standard interface for the development and interconnection of HW and SW multimedia components. However, the simulation and implementation steps required to obtain the final prototypes of such complex systems are still a challenge. To solve these problems, this paper presents a framework which enables automatic code generation from high-level UML/MARTE models. SystemC and VHDL codes are synthesized according to the OpenMAX specification require- ments and they are integrated with the application SW, derived from task-based systems models. The generation of the SystemC executable specification enables easy simulation and verification of multimedia systems. After this verification stage, the framework automatically provides the VHDL code which feeds the final implementation and synthesis stage for the * D. de la Fuente [email protected] J. Barba [email protected] J. C. López [email protected] P. Peñil [email protected] H. Posadas [email protected] P. Sánchez [email protected] 1 University of Castilla-La Mancha, Ciudad Real, Spain 8196 Multimed Tools Appl (2017) 76:8195–8226 target platform. To demonstrate this approach, a SOBEL-based use case has been implemented with the developed framework. Keywords OpenMAX . UML/MARTE . SystemC . VHDL . Automatic code generation 1 Introduction The design of multimedia embedded systems is a highly competitive context. New multimedia devices typically include a wide set of applications, with an increasing number of intensive data processing operations in order to fulfil the new standards of audio, video or image quality. Furthermore, the development of a successful product greatly depends on being the first product on the market providing these new complex functionalities. However, these intensive data processing operations require an increment in the computing power of the system. To cope with this complexity, high performance features are demanded. For this purpose, some parts of the system can require implementations on heterogeneous platforms. Therefore, HW/SW integration is necessary. However, common HW/SW design flows are far from being easy and quick to be applied. The lack of standardization in HW/SW integration has led to ad- hoc implementations, requiring in-depth knowledge and great effort from the developers. A consequence of large, complex systems is the multiple design variations that can be considered during the design process. The elements that compose the system can have different properties with implications in their behaviour. These properties can be examined, enabling a design exploration process (DSE) to obtain the best configuration for each system element so as to optimize them and achieve the performance requirements. Furthermore, current chips integrate multiprocessor systems, usually combining a growing number of general-purpose processors (GPPs), different types of processing units (Digital Signal Processors and Graphic Processor Units) with configurable devices. In this way, application elements can be implemented either as SW or HW components during the design process. Therefore, new design approaches should establish a DSE process to find the best config- uration of the system elements, according to the specific characteristics which determine the correctness of the global system behaviour. In addition, these new design approaches should enable exploration of the system element mapping, taking advantage of the heterogeneous nature of current platforms in order to achieve the best system implementation according to the available resources. Different approaches have appeared in order to manage the design of large, complex systems. The adoption of standards supports the development of portable, flexible and reusable designs for embedded systems. At the same time, high-level methodologies also provide powerful solutions for system development and component reuse. Development of electronic system-level (ESL) design methodologies [17] provides a strategy for designing complex systems, in which the initial key activity is specification. SystemC [19, 38] is the most popular language, widely accepted by the ESL community. SystemC is a specification language to model system at functional level. Model-driven development (MDD) methodologies can simplify specifications and make them more understandable, which are major requirements for tackling the design challenge [14]. As an example, the use of standard languages, such as Unified Modelling Language (UML, [42]), provides easy to read and portable specifications. Multimed Tools Appl (2017) 76:8195–8226 8197 MDD methodologies are commonly adopted to handle the design of complex large functionalities. The latest design methodologies start from high-level UML models combined with algorithmic codes (e.g. C, C++, Matlab, etc.) of the different system components [40]. In these models, the user defines the system functionality and the target platform where this functionality is executed. The combination of both approaches takes advantage of the potential synergies in order to obtain an improved result. The approach presented here combines the benefits of the OpenMAX standard with a UML-based synthesis solution. The OpenMAX standard [27] is an initiative promoted by the Khronos Group and supported by many important companies such as AMD, Intel, ARM, Sony, Nokia, NVIDIA, Samsung, etc. OpenMAX is based on components and defines a standardized media compo- nent interface for audio, video, image and others (as defined in the standard itself). The OpenMAX middleware allows developers and platform providers to integrate and communi- cate with multimedia codecs implemented in hardware or software. Through the use of OpenMAX in embedded systems, developers can reduce the effort required to design multimedia HW/SW systems because: (a) the whole core logic can be reused when targeting a new platform (standardized interfaces), (b) it is not necessary to hand write the drivers or code that depends on these components and, (c) communication issues are separated from the processing primitives (the same synchronization protocols, standardized communication mechanism, etc.). In order to provide UML with specific semantics to support complex system design, a set of profiles have been developed. In the specific context of embedded systems, the Modelling and Analysis of Real-Time and Embedded Systems profile (MARTE [25]) adds to the UML language the concepts and semantics needed to describe real-time features at high abstraction levels. The UML Testing Profile [26] enables the definition of models that capture scenarios for system testing. Following this combined approach, this paper presents an infrastructure for automatic code generation for OpenMAX multimedia systems simulation and for the later implementation from UML/MARTE. This infrastructure enables executable specifications to be automatically obtained. Also, this executable models are used to simulate the OpenMAX designs captured in UML/MARTE and decide which system configuration is the most suitable for a latter validation of the system requirements. For the simulation, SystemC [35] has been used in this work since it is a modelling language that is applied to system-level modelling, architectural exploration, performance modelling, software development, functional verification and high-level synthesis. In addition to that, the SystemC executable specification automatic generated corresponds to a HW OpenMAX Integration Layer infrastructure [10]. This HW OpenMAX IL infrastructure can be executed and explored, considering different automatically generated test-benches as well. Finally, the UML/MARTE methodology captures enough information about the target platform to enable automatic VHDL code generation for the implementation of the SW/HW component interconnections, which are allocated in a Field Programmable Gate Array (FPGA). The paper is organized as follows; in Section 2, a study of the state-of-the-art is presented. Section 3 provides the context in which this work is developed. In Section 4, the complete design flow is described. All the aspects of the UML/MARTE methodology, which are the main goal of this paper, are presented in Section 5. In Section 6, the OpenMAX-SystemC simulation process is explained. Later, in Section 7 the OpenMAX synthesis process for 8198 Multimed Tools Appl (2017) 76:8195–8226 generation of VHDL code is defined. A study case is proposed in Section 8 and finally, some conclusions are presented in Section 9. 2 State-of-the-art In order to facilitate understanding of this section, three main groups of related works will be described. These groups have a correspondence with the three main pillars used in this proposal: OpenMax as the standard of reference for integration and multimedia platform modelling, SystemC for executable specifications and UML language