Evaluation of MDE Tools from a Metamodeling Perspective

50 Journal of Database Management, 19(4), 50-75, October-December 2008

Joãão de Sousa Saraiva, INESC-ID/Instituto Superior T´ecnico, Portugal Alberto Rodrigues da Silva, INESC-ID/Instituto Superior T´ecnico, Portugal

Abstract

150 words or less

Keywords: evaluation framework; metamodel; MDE; tool

Introduction (Schmidt, 2006), abstracting over the solution Ever since the appearance of computers, re- domain (computing technologies) instead of searchers have been trying to raise the abstrac- the problem domain. tion level at which software developers write Currently, the abstraction level is being computer programs. Looking at the history of raised into the model-driven engineering (MDE) programming languages, we have witnessed paradigm (Schmidt, 2006). In this abstrac- this fact, with languages evolving from raw tion level, models are considered first-class machine code to machine-level languages, af- entities and become the backbone of the entire terward to procedural programming languages, MDE-oriented software development process; and finally to object-oriented languages, which other important artifacts, such as code and allow developers to write software by mapping documentation, can be produced automatically real-world concepts into modular segments from these models, relieving developers from of code (called objects). Still, object-ori- issues such as underlying platform complexity ented languages are too “computing-oriented”

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Journal of Database Management, 19(4), 50-75, October-December 2008 51

or the inability of third-generation languages is independent of language or technology, and to express domain concepts. is addressed by these initiatives. MDE is not a new idea. Already in the 1980s All these approaches share the same ba- and 1990s, computer-aided software engineer- sic concepts. A model is an interpretation of ing (CASE) tools were focused on supplying a certain problem domain, a fragment of the developers with methods and tools to express real world over which modeling and system software systems using graphical general-pur- development tasks are focused, according to pose language representations. The developer a determined structure of concepts (Silva & would then be able to perform different tasks Videira, 2005). This structure of concepts is over those representations, such as correction provided by a metamodel, which is an attempt analysis or transformations to and from code. at describing the world around us for a par- However, these CASE tools failed due to issues ticular purpose through the precise definition such as (a) poor mapping of general-purpose of the constructs and rules needed for creating languages onto the underlying platforms, which models (Metamodel.com, n.d.). These basic made generated code much harder to understand concepts are the core of metamodeling, the and maintain, (b) the inability to scale because activity of specifying a metamodel that will the tools did not support concurrent engineer- be used to create models, which is the foun- ing, and (c) code was still the first-class entity dation of MDE. in the development process while models were From the developer’s point of view, a key seen as only being suited for documentation issue for acceptance of any approach is good (Schmidt, 2006). Currently, there are better tool support so that software programs can be conditions for such modeling tools to appear. created in an easy and efficient manner. There Software systems today are reaching such a is a wide variety of modeling tools available high degree of complexity that third-generation today, covering most modeling standards and languages simply are not sufficient anymore; approaches in existence. For example, Rational another abstraction level over those languages Rose and Enterprise Architect (EA; SparxSys- is needed. This need, combined with the choices tems, n.d.) are only two examples of a very of IT development platforms currently avail- long list of tools that support UML model- able (Java, .NET, etc.), to which models can ing. DSM has recently become popular with be somewhat easily mapped, is the motivation the developer community, with tools such as for the adoption of MDE. There are already a Microsoft’s DSL Tools (MSDSLTools, n.d.) few MDE-related case studies available, such or MetaCase’s (n.d.) MetaEdit+. as Zhu et al. (2004) and Fong (2007), but since The aim of this article is to present our most MDE work is still in the research phase, evaluation framework for tool support of the there is still a lack of validation through a metamodeling activity, and to evaluate a small variety of real business case studies. set of tools according to this framework; There are already multiple MDE initiatives, although these tools do not reflect everything languages, and approaches, such as the unified that is currently available in MDE tools, they modeling language (UML), the MetaObject address the MDE-based approaches presented Facility (MOF), the model-driven architecture in this article by providing the features typi- (MDA), and domain-specific modeling (DSM; cally found in tools of their corresponding ap- Kelly & Tolvanen, 2008). There are also proach. The evaluation framework used in this other derivative approaches, such as software article focuses on the following issues: (a) factories (http://msdn2.microsoft.com/en-us/ supported exchange formats, (b) support for teamsystem/aa718951.aspx) that follow the model transformation and code generation, (c) MDE paradigm. Nevertheless, it is important tool extensibility techniques, (d) the logical to note that these initiatives are not a part of levels that can be manipulated, (e) support for MDE; rather, MDE itself is a paradigm that specifying metamodel syntax and semantics,

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 52 Journal of Database Management, 19(4), 50-75, October-December 2008 and (f) complexity of the meta-metamodel The OMG’s Approach to MDE hard-coded into the tool. The final purpose The Object Management Group (OMG) has of this evaluation is to determine the strengths created its own MDE initiative based on a set of and weaknesses of the support that each of these OMG standards that make use of techniques for MDE tools offer to the developer’s tasks. metamodeling and model transformation. This article is divided as follows. The second section presents a brief overview of Unified Modeling Language MDE and some related concepts, standards, UML (http://www.omg.org/cgi-bin/apps/ and approaches. Then the article describes the doc?formal/05-07-04.pdf), currently in Version evaluation framework, the selected modeling 2.1.1, is a general-purpose modeling language tools, and the results of their evaluation. Next originally designed to specify, visualize, con- it discusses the current status of MDE-based struct, and document information systems. software tools and some open research issues UML is traditionally used as a metamodel (i.e., for metamodeling. The final section presents developers create models using the language the conclusions of this work. established by UML). However, the UML specification also defines the profile mechanism, Model-Driven Engineering which allows for new notations or terminolo- Software systems are reaching such a high gies, providing a way to extend metaclasses degree of complexity that the current third- to adapt them for different purposes. Profiles generation programming languages (like Java are collections of stereotypes, tagged values, or C#) are not sufficiently adequate to create and constraints (Silva & Videira, 2005). A ste- such systems in an easy and efficient manner. reotype defines additional element properties, One of the problems with current program- but these properties must not contradict the ming languages is that they are still too oriented properties that are already associated with the toward specifying how the solution should work model element; thus, a profile does not allow instead of what the solution should be. This leads the user to edit the metamodel. to a need for mechanisms and techniques that Although UML was definitely a step for- allow the developer to abstract over current ward in setting a standard understood by the programming languages and focus on creating whole software engineering community and a good solution to a certain problem. aligning it toward MDE, it is still criticized for Model-driven engineering (sometimes reasons such as (a) being easy to use in soft- called model-driven development, or MDD) is ware-specific domains (such as IT or telecom- an emerging paradigm based on the systematic style systems) but not for other substantially use of models as first-class entities of the solu- different domains, such as biology or finance tion specification (Schmidt, 2006). Unlike pre- (Thomas, 2004), (b) not being oriented to how vious software development paradigms based it would be used in practice (Henderson-Sell- on source code as a first-class entity, models ers, 2005), and (c) being too complex (Siau become first-class entities, and artifacts such & Cao, 2001). Nevertheless, UML is often the as source code or documentation can then be target of overzealous promotion, which raises obtained from those models. user expectations to an unattainable level; the It is very important to note that, although criticisms that follow afterward are usually MDE is often mentioned alongside MDA (which influenced by this (France, Ghosh, Dinh-Trong, is explained further later), MDE does not depend & Solberg, 2006). An example of such a criti- on MDA, nor is MDA a subset of MDE. In fact, cism is the one regarding the difficulty in using MDA is one of several initiatives that intend to UML to model non-software-related domains: address the MDE paradigm. Although UML is a general-purpose modeling language, it is oriented toward the modeling of

software systems and is not intended to model information when exchanging models between each and every domain. different tools. The QVT specification defines a standard way of transforming source models MetaObject Facility into target models by allowing the definition of MOF (http://www.omg.org/cgi-bin/apps/ the following operations: (a) queries on models, doc?formal/06-01-01.pdf), currently in Ver- (b) views on metamodels, and (c) transforma- sion 2.0, is the foundation of OMG’s approach tions of models. One of the most interesting to MDE. UML and MOF were designed to ideas about QVT is that the transformation be themselves instances of MOF. This was should itself be considered an MOF-based accomplished by defining the UML Infra- model, which means that QVT’s syntax should structure Library (http://www.omg.org/cgi- conform to MOF. Figure 2 presents OMG’s bin/apps/doc?formal/05-07-05.pdf), which typical four-layer architecture: (a) MOF is the provides the modeling framework and notation meta-metamodel in the M3 layer, (b) UML, for UML and MOF, and can also be used for an instance of MOF, is the metamodel in the other metamodels. Figure 1 illustrates the de- M2 layer, (c) the user model contains model pendencies between UML and MOF; note that elements and snapshots of instances of these MOF can be described using itself, making it model elements in the M1 layer, and (d) the reflexive (Nobrega, Nunes, & Coelho, 2006). M0 layer contains the runtime instances of the Besides UML, the OMG has also defined some model elements defined in the M1 layer. other MOF-based standards, such as the XML (extensible markup language) metadata Model-Driven Architecture interchange (XMI) and query-views-transfor- MDA is OMG’s framework for the software mations (QVT). development life cycle driven by the activity of XMI allows the exchange of metadata in- modeling the software system (Kleppe, Warmer, formation by using XML, and it can be used for & Bast, 2003). It is based on other OMG stan- any metadata whose metamodel can be specified dards such as UML, MOF, QVT, and XMI, in MOF. This allows the mapping of any MOF- and places a greater emphasis on UML model based metamodel to XML, providing a portable transformation techniques (through QVT) than way to serialize and exchange models between on metamodeling itself; however, it should be tools. Nevertheless, users often regard XMI as noted that QVT model transformations are made a last resort for exchanging models between possible only because of the model-metamodel tools because tools frequently use their own relationship between UML and MOF. vendor-specific XMI extensions; thus they lose

Figure 1. The dependencies between UML and MOF

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 54 Journal of Database Management, 19(4), 50-75, October-December 2008

Figure 2. An example of OMG’s four-layer metamodel architecture

MDA defines two types of models: (a) the MDA still faces some criticism in the soft- platform-independent model (PIM) and (b) the ware engineering community because of issues platform-specific model (PSM; Kleppe et al., such as its usage of UML (Thomas, 2004) and 2003). A PIM is a model with a high level of the view that while current MDA generators abstraction that makes it independent of any are able to generate a significant portion of an implementation technology, making it suitable application, they are not particularly good at to describe a software system that supports building code that works within an existing a certain business without paying attention code base. to implementation details (like specific -re lational databases or application servers). A Domain-Specific Modeling PSM also specifies the system, but in terms DSM (Kelly & Tolvanen, 2008) uses problem- of the implementation technology. A PIM can domain concepts as the basic building blocks be transformed into one or more PSMs, each of models unlike traditional CASE, which of those PSMs targeting a specific technology uses programming-language concepts. From a because it is very common for software systems technological perspective, DSM is supported today to make use of several technologies. Figure by a DSM system, which can be considered 3 presents an overview of MDA; the solid lines as an application for making domain-specific connecting the boxes are transformations, which CASE tools (or as a tool-building environ- are defined by transformation rules. MDA pre- ment to create CASE tools that can be used scribes the existence of transformation rules, but to produce applications). Thus, DSM adds an it does not define what those rules are; in some abstraction layer over traditional CASE, en- cases, the vendor may provide rules as part of abling the domain-specific configuration of the a standard set of models and profiles. resulting modeling application as illustrated in

Figure 3. An overview of MDA

Figure 4. Because of this, DSM systems are as source code; a DSL uses concepts from the also called meta-CASE tools. DSM is closely problem domain, which means developers do related to the concept of domain-specific lan- not need to worry about how those concepts guage (DSL). A DSL is a language designed to will map to code. be useful for a specific task (or a specific set of tasks) in a certain problem domain unlike a general-purpose language (Kelly & Tolvanen). As Figure 5 illustrates, due to a DSL’s highly Figure 4. How CASE and DSM systems are specialized nature, DSLs and corresponding related generators are usually specified by experts (i.e., experienced developers) in the problem domain; other developers, less experienced with the mapping between domain concepts and source code, will invoke the DSL in their own code. A well-known example of a DSL is the standard query language (SQL), which is a standard computer language for accessing and manipulating databases (so, SQL’s problem domain is the domain of database querying and manipulation). Developers usually prefer DSLs to UML because of the set of used concepts: The latter uses programming concepts directly, which places models at the same abstraction level

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 56 Journal of Database Management, 19(4), 50-75, October-December 2008

Figure 5. Using the expertise of some developers to orient other developers toward the problem domain

UML itself can be seen as a set of DSLs with other definitions that can be found in (corresponding to use-case diagrams, class dia- literature, such as the ones in Kleppe et al. grams, activity diagrams, etc.); however, these (2003) and Söderstrom, Andersso, Johannes- would be dependent on each other in a “DSL son, Perjons, & Wangler (2002). This means spaghetti” manner. UML can also be used to de- that a metamodel provides a language used to fine DSLs using the profile mechanism, although create a model, as Figure 6 illustrates; simi- this does bring some limitations that DSLs do larly, a metamodel that defines the language not, such as the ability to ignore the semantic in which another metamodel is specified is constraints already defined in UML. called a meta-metamodel. Similar in concept to DSM, metamodeling Metamodeling is about developing a language (a metamodel) The approaches presented lead us to the point adapted to the problem domain; for example, where we can see that all concepts presented here MOF is a language adapted to the domain of are deeply related among themselves. We have object-oriented approaches to modeling (Atkin- a recurring pattern—the usage of metamodels son & Ku¨hne, 2005), while UML is a language and their instances of models—and the only adapted to the domain of object-oriented real difference (in modeling terms) between programming languages (OOPLs). A possible all these approaches is in the number of lay- example, in the context of an organization, of ers each one uses. So, aside from a question what could be done with metamodeling can be of vocabulary, all these MDE-based variants the following: (a) the specification of a new have their foundation on the same topic: language or metamodel (with an existing lan- metamodeling. guage as its metamodel, e.g., MOF or UML) But what is metamodeling? Metamodel. that reflects the concepts, syntax, and semantics com (Metamodel.com, n.d.) provides the of the corresponding problem domain, which following definitions: “metamodeling is the is the organization, (b) after creating a tool activity that produces, among other things, that supports the metamodel, the modeling of metamodels” and “a metamodel is a precise a solution using the organization’s terms (e.g., definition of the constructs and rules needed the organization specifies a certain role R1 for creating models.” These definitions agree that can perform activities A1 and A2), and (c)

Figure 6. A metamodel defines a language used to create a model

depending on the features provided by the tool, a certain metamodel would probably have to an application that implements the designed either (a) create a new modeling tool, which is solution could be generated (either by model not reasonable at all (Nobrega et al., 2006) or transformations, or by direct generation of (b) settle on a CASE tool (with a hard-coded source code). In fact, the PSMs for the MDA metamodel) that allows the developer to approach (oriented toward the implementa- perform the desired task with the least pos- tion domain) can be obtained by using UML sible hassle. However, adding metamodeling profiles tailored to an OOPL’s concepts (such support to a tool does bring some practical as C#’s class, struct, etc.). This would pres- issues that should be mentioned, such as (a) ent an advantage over traditional development separating the OOPL class- instance rela- approaches as the solution would be created tion from the metamodel-model relation, (b) using the organization’s terms instead of us- deciding whether the number of logical levels ing implementation terms; we later present a should be limited or potentially unbounded, more detailed view of how software develop- and (c) deciding whether the tool should ment can be done combining metamodeling support model transformation and/or code and model transformations. An example of the generation. need of using metamodeling and metamodels In addition to these issues, it is also neces- can be found in Zhao and Siau (2007), which sary to consider how to change a metamodel, uses metamodels to handle the mediation of which should be considered a very high-risk information sources. activity because models, consistent in the The main difference (in modeling terms) context of a certain metamodel, can become between the presented modeling approaches is inconsistent with only some changes to that their number of modeling layers (i.e., model- metamodel. Obviously, this introduces a metamodel relationships). Theoretically, the potential element of disruption that should number of layers could be infinite, but any be avoided at all costs. One possible way particular approach should have a specific of ensuring the validity of existing models number of layers; otherwise, its implementation when changing their metamodels is through would be impractical, if not impossible. the specification and application of model It is still rare to find a development tool transformations (e.g., UML transformations, that has explicit support for metamodel creation such as those presented in Selonen, Koskim- and/or configuration, which can be surprising ies, & Sakkinen, 2003): For any change to a if we consider that metamodeling is one of the metamodel, a corresponding transformation founding principles of MDE. This means that, must be defined that receives the previously until recently, a developer who wanted to use

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 58 Journal of Database Management, 19(4), 50-75, October-December 2008 consistent models and produces new models of layers in the tool’s architecture should be consistent with the new metamodel. restricted or potentially unbounded. When using However, in our research we have found no the language metaphor, the basic elements of tool that addresses all of these metamodeling each layer (e.g., object, class, metaclass, etc.) issues (although there are tools that address are contained in the hard-coded metamodel some of the presented issues). itself; if the user wanted to add other basic ele- Implementing a modeling tool with just ments, necessary for additional layers, it would one logical level (i.e., user model editing be necessary to alter the hard-coded metamodel. and a hard-coded metamodel) is easily done This metaphor helps in supporting a standard with current OOPLs using the class-instance (such as OMG’s), but at the cost of not being able relation: The logical level is implemented to edit the metamodel. On the other hand, in the by the instance level. Metamodeling adds library metaphor, the hard-coded metamodel one (or more) logical level to the modeling consists only of a minimal core language, and tool, complicating the implementation as the the basic elements of each layer are available instance level now has to hold two or more as predefined types in libraries to which the logical levels (Atkinson & Ku¨hne, 2003). user can add elements (or remove them, if the Level compaction (Atkinson & Ku¨hne, 2005), tool allows it). With this metaphor, users can an example of which is illustrated in Figure 7, is experiment with all metamodel layers because a technique that addresses this problem. Instead only the minimal core is hard-coded; the burden of the representation format for a level being of syntax checking and language semantics is defined by the level above, the format for a placed on the remaining metamodel layers. level is supplied by the modeling tool. Note that if a tool does not use level compaction, Although level compaction is essential for then it obviously uses the language metaphor supporting multiple modeling levels, it is also because the supported modeling levels are important to determine whether the metamodel limited by the class-instance relation, which hard-coded into the tool allows such a number only allows one modeling level (in the instance of levels. Atkinson and Ku¨hne (2005) present level) besides the hard-coded metamodel (in the language and library metaphors, which al- the class level). low tool creators to choose whether the number

Figure 7. An example of using level compaction to compact three logical levels

Another important aspect to consider in framework focuses on a tool’s support for metamodeling tools are model-to-model trans- metamodeling and involves the following formations. It would be natural that, after some dimensions, as illustrated in Figure 8: time using such a tool, a developer has created or adopted some languages adjusted to relevant 1. supported exchange formats, problem domains. However, after modeling a 2. model transformation support, solution using the problem-domain language, 3. usage of the level-compaction technique the developer would then need to re-create the (Atkinson & Ku¨hne, 2005), model in the language of the target domain. 4. usage of the language and library meta- Obviously this would render the first model phors (Atkinson & Ku¨hne, 2005), useless. So, if the tool also provided some 5. the logical levels that the user can ma- kind of framework or language for specifying nipulate, transformations between model languages, this 6. support for specifying metamodel syntax would certainly benefit the developer. and semantics, and 7. the size of the hard-coded meta-metamod- Evaluation of MDE Tools el. One of the key issues for the success of MDE is appropriate tool support as developers will The third and fourth dimensions were only use a certain approach if it is supported directly based on the conceptual framework by available tools. This section first presents defined in Atkinson and Ku¨hne (2005); the the evaluation framework used through the rest other dimensions are derived from the issues of this article. Afterward, we present the tools described in the previous section (“Model- that are evaluated. Finally, the evaluation’s Driven Engineering”) since this evaluation also results are presented. tries to focus on the practical usage of these tools instead of exclusively considering architectural Evaluation Framework details. Note that we do not define a ranking This subsection presents the proposed evalu- system because the ultimate objective of this ation framework used in this article. This evaluation is not to determine the best tool but

Figure 8. An overview of the proposed evaluation framework

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 60 Journal of Database Management, 19(4), 50-75, October-December 2008 rather if (and how) the industry is currently ad- imported or exported, and (b) models, which dressing metamodeling. In addition, we believe involves determining whether user models it is up to each developer to determine what can be imported or exported. This division is approach and tool characteristics are required useful because the formats used by a tool to for development. However, we do believe that import, or export metamodels and models may this framework provides a practical contribution be different; also, a tool may only allow the through its generic set of guidelines that help import or export of models but not metamodels. determine whether a tool can appropriately The values for both dimensions are the set of address metamodeling (both as an activity in standards used (possibly none). itself and as an activity in the context of software development). Moreover, metamodeling is still Model Transformation Framework an active research topic that is not addressed by This dimension measures whether the tool many tools, and we believed that ranking these supports model transformations, and only tools would ultimately yield unfair results (as allows a single value from its measurement some of the tools were not created to address range: yes, meaning that the tool additionally this issue in the first place). provides a framework or language based on the We also highlight the fact that, although metamodel or the meta-metamodel for specify- this evaluation framework has been empirically ing transformations between user models (such validated (in the context of our experience as QVT), and no, meaning that the tool does with various MDE-based tools), some of not provide such a framework. these criteria and measurement metrics are still subjective and can be refined by performing an Level Compaction explicit validation of the existing criteria and This dimension measures whether the tool uses their measurements metrics, according to ap- the level-compaction technique (Atkinson & proaches such as Moore and Benbasat (1991), Ku¨hne, 2005) and only allows a single value from and by adding further (and more objective) its measurement range: yes, meaning that the criteria that address other issues regarding tool uses level compaction and can therefore metamodeling. easily be adjusted to support additional logical levels, and no, meaning that the tool does not Supported Standard Exchange employ level compaction. Formats With all the modeling tools now available, the Language and Library Metaphors ability to exchange models between tools is This dimension measures which of the two meta- becoming a very important requirement; the phors (language or library; Atkinson & Ku¨hne, lack of this ability can easily lead to a situation 2005) are used in the tool, and only allows a in which a developer is stuck with a certain single value from its measurement range: lan- tool. This would require that tools be able guage metaphor or library metaphor, according to export and import models to and from a to the metaphor used. Note that if the dimension standard format, such as XMI. Although each level compaction evaluates as no, then the value tool creator is free to create or choose his or her of this dimension will obviously be the lan- own exchange format, it should be taken into guage metaphor, as presented in the previous account that developers usually choose tools section (“Model-Driven Engineering”). that can import or export to standard formats, Number of Logical Levels the User allowing models to be independent of the tools in which they are manipulated. can Manipulate This dimension is divided into two sub- Despite what architectural options are present dimensions: (a) metamodels, which involves in a tool, one of the aspects that directly affects determining whether metamodels can be a tool’s user is the number of logical levels

that can actually be manipulated in the tool and no, meaning that the tool does not (by creating, editing, or deleting elements) support this. as a limited number may force the user to • Languages used. This dimension deter- compact two or more metamodel levels into a mines the set of languages used by the single layer (i.e., the user places elements from tool to specify the metamodel’s syntax (in- several logical levels in a single level). cluding proprietary or standard languages). This dimension measures how many Note that this dimension can only have a metamodel-model relationships can be handled meaningful value when the specification- by the tool, and it only allows the usage of a support dimension’s value is yes. single natural number (i.e., 1, 2, etc.). For ex- • Semantics. A metamodel’s semantics can be ample, a typical UML CASE tool only allows seen from two perspectives: the semantic the manipulation of one logical level (M1) as domain and the semantics of each model the creation of instances is still performed element. The semantic domain consists of in M1. the whole set of domain elements that the metamodel is supposed to represent (i.e., Support for Metamodel Specification the concepts that were captured during In the evaluation of the support that a tool the analysis of the problem domain). On provides for specifying metamodels, it is im- the other hand, the semantics of a cer- portant to analyze what a tool supports. tain model element is determined by the This dimension is divided into two other relation(s) between that model element dimensions, syntax and semantics, evaluating and one or more domain elements. the support that the selected tools provide to the specification of the syntax and semantics This dimension is divided into two subdi- of metamodels, respectively. The definitions of mensions, specification support and languages metamodel syntax and metamodel semantics used, which evaluate some aspects of the are similar to the ones found at http://www. mechanisms provided for defining metamodel klasse.nl/research/uml-semantics.html and are semantics. described next. • Specification support. This dimension • Syntax. A metamodel’s syntax consists of measures whether the tool supports the the set of model elements (i.e., graphical specification of the semantic component representations of domain elements) and of a metamodel. It only allows a single the relationships between those model ele- value from its measurement range: yes, ments; this is very similar to the definition meaning that the tool allows specification of syntax in the context of linguistics, in of a metamodel’s semantic constraints, and which syntax is the study of the way words no, meaning that the tool does not support are combined together to form sentences. this. The syntax dimension is divided into two • Languages used. This dimension, like subdimensions: specification support and the languages-used dimension of syntax, languages used. determines the set of languages used by • Specification support. This dimension the tool to define a metamodel’s semantic evaluates whether the tool supports the constraints (such as OCL for MOF-based specification of the syntactic component models, available at http://www.omg.org/ of a metamodel (i.e., the graphical repre- cgi-bin/apps/doc?formal/06-05-01.pdf). sentation of its elements). It only allows a Note that this dimension can only have single value from its measurement range: a meaningful value if the specification- yes, meaning that the tool allows the support value is yes. specification of the metamodel’s syntax,

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 62 Journal of Database Management, 19(4), 50-75, October-December 2008

Hard-Coded Meta-Metamodel Size definition of how large a meta-metamodel must An important aspect to consider is the size of be before it can be considered large. the meta-metamodel hard-coded into the tool (or metamodel if the tool only allows creating user MDE Tools models) because it reflects how wide the range Figure 9 presents an overview of the small of metamodel primitives is. In this evaluation, set of MDE tools used in this evaluation: we consider the size of a model (or a meta- Enterprise Architect (SparxSystems, n.d.), metamodel, in this case) to be defined by the MetaSketch (Nobrega et al., 2006), MetaEdit+ quantity of information involved in the formal (MetaCase, n.d.), and Microsoft’s DSL Tools specification of the model (i.e., how many (MSDSLTools, n.d.). objects, relationships, and constraints are used The initial criteria used for the selection to specify the model); the explanation for this of MDE tools to evaluate were the following: lies in the amount of information that the user (a) The tool must be recent (or still be under should be aware of when creating a metamodel development) to ensure it addresses current in order to take full advantage of the language MDE approaches, (b) each tool must address provided by the meta-metamodel. one of the MDE initiatives presented in the This dimension only allows a single value previous section, and (c) the tool must have a from its measurement range: (a) small, meaning relatively smooth learning curve as developers that the tool’s hard-coded meta-metamodel are usually more inclined to choose tools that consists of 15 elements or less (in this article, they find to be user friendly and that facilitate we consider an element to be either an object, their activities. We searched the Internet for a relationship between objects, or a constraint), candidate tools that fit these criteria; however, (b) average, meaning that it consists of 16 to 30 we found many candidate tools, so we limited elements, and (c) large, meaning that it consists this evaluation to popular tools in order to of more than 30 elements. It is important to note keep the evaluation (and this article) simple. that this measurement is highly subjective since We also included MetaSketch in this evalua- we know of no framework to objectively clas- tion because, although it is not yet popular, it sify a model’s size or complexity; ultimately, explicitly addresses the metamodeling activity, it is up to the reader to make his or her own so we believed that including it in the evalu-

Figure 9. The selected MDE tools

ation could yield some interesting results. We Traditional CASE Tools did not consider any of our own tools (i.e., Although traditional CASE tools may be ad- developed in-house) as candidates for this equate for the development of small and simple evaluation in order to maintain an indepen- software systems, they clearly do not support dent perspective over this tool evaluation the development tasks that come with larger, and prevent us from inadvertently specifying complex systems. One of the main problems of dimensions that would favor any one of the such tools is that they only support a specific tools being evaluated. metamodel, usually UML, and do not offer These tools were chosen because we support for altering that metamodel (although consider that this set is a good representative UML does provide the profile mechanism, sup- of the current status of MDE-supporting tools ported by some UML modeling tools). currently available (e.g., Enterprise Architect This type of tools is included in this can do most of what can be done with ArgoUML, evaluation to determine whether current typical http://argouml.tigris.org; Poseidon for UML, CASE tools could easily be adapted to allow http://www.gentleware.com; Rational Rose the creation of models based on a user-specified 2003, http://www-306.ibm.com/software/awd- language. For the evaluation purposes of this tools/developer/datamodeler; or other UML work, we chose Enterprise Architect (Sparx- modeling tools); they also presented enough Systems, n.d.) to represent traditional CASE differences amongst themselves to justify their tools as it is quite easy to use, provides good inclusion in this evaluation. Although these support for UML and its profile mechanism (in tools do not reflect everything that is currently fact, EA makes the definition of a UML profile available in MDE tools, they address the MDE- a simple and easy task), and seems to be one based approaches defined earlier by providing of the best representatives of the current status the features that can often be found in typical of CASE tools. tools of their corresponding approach. (For this evaluation, we used Enterprise The reason we evaluate only a small Architect 6.5, which was the latest version of number of tools is article simplicity and size. this tool at the time this work was written.) However, it is important to reiterate that there are a great number of other tools available, such MetaSketch as the Generic Modeling Environment (GME; MetaSketch (Nobrega et al., 2006) is a MOF- http://www.isis.vanderbilt.edu/projects/gme) based editor, unlike most editors, which are or the Eclipse Graphical Modeling Framework usually based on UML. It is based on the (GMF; http://www.eclipse.org/gmf). Although following ideas: (a) A metamodel is a model in this article we only evaluate this small set that conforms to MOF 2.0, not to UML 2.0, of tools, we believe that an evaluation of a (b) the UML profile mechanism is not powerful greater number of tools, including a wider enough to support the definition of new model- range of areas such as ontology modeling or ing languages, (c) a metamodel should be the enterprise architecture modeling, would yield primary artifact of a modeling language defi- some very interesting results to complement nition and developers should not need to code those obtained here. An added advantage of metamodels, and (d) a metamodel is not the final such an evaluation would also be the diverse goal but the means used to produce models, so set of metamodels used by the evaluated tools it is not reasonable to create another modeling (e.g., enterprise modeling tools tend to use tool each time another metamodel is specified. enterprise-oriented metamodels, such as the These ideas lead to MetaSketch, an editor that is TOGAF or Zachman framework). MOF compliant, allowing the definition of any language that can be specified using MOF (i.e., a MOF-based metamodel). Thus, MetaSketch is best defined as a metamodeling tool.

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 64 Journal of Database Management, 19(4), 50-75, October-December 2008

MetaSketch does not offer code generation erator. Once the expert creates the modeling capabilities by itself, but it can import or export method (or even a prototype), the development defined models and metamodels to XMI; the team can start using it in MetaEdit+ to define tool adheres strictly (with no vendor-specific models, and the corresponding code will be extensions) to XMI 2.1 (Nobrega et al., 2006), automatically generated from those models. so code generation could easily be handled by The code generator itself uses a DSL that al- any code generator that can understand XMI. lows the developer to specify how to navigate The tool also supports the definition of models through models and output its contents along conforming to metamodels specified in XMI with additional text. The tool also provides a (e.g., MOF or UML). Three metalevels, M3, repository for all modeling method informa- M2, and M1, are supported by using level tion, allowing the storage and modification of compaction. Figure 10 illustrates two interesting modeling method definitions; any modifications scenarios that are made possible by MetaSketch: to definitions are also reflected in their- cor the definition of a MOF metamodel by using responding tools, models, and generators. itself (top), and the definition of the UML (For this evaluation, we used MetaEdit+ and CWM metamodels (bottom). In the first 4.5, which was the latest version of this tool scenario, the user takes advantage of MOF’s at the time this work was written.) reflexive property in order to define a metamodel consisting of MOF itself (note the hard-coded Microsoft DSL Tools MOF and the user-defined MOF); UML and Microsoft’s DSL Tools, available at http://msdn. CWM can then be defined as user models microsoft.com/vstudio/dsltools, is a suite of from that metamodel. In the second scenario, tools for creating, editing, visualizing, and the user defines the UML metamodel by using using domain-specific data for automating the the hard-coded MOF meta-metamodel. UML enterprise software development process. DSL user models can then be created based on that Tools allow developers to design graphical metamodel (note that this second scenario is modeling languages and to generate artifacts very similar to the typical OMG architecture, (such as code or documentation) from those illustrated in Figure 2). languages; the visual language tools are based on Microsoft Visual Studio. MetaEdit+ The process of creating a new DSL begins MetaEdit+, available at http://www.metacase. with the DSL Designer Wizard, which provides com, is a DSM-oriented environment (i.e., a some metamodel templates (such as class dia- meta-CASE tool) that allows the creation of grams or use-case diagrams) and guides the modeling tools and generators fitting to - ap developer through specifying the features of the plication domains without having to write desired DSL. As a result of executing the wizard, code (Tolvanen & Rossi, 2003). It uses a a Visual Studio solution is created, containing a meta-metamodel called GOPPRR (graph, DSL project with the language’s domain model object, property, port, relationship, and role), (classes and relationships), its visual represen- named after the metatypes that are used when tation (diagram elements), and the mappings specifying the metamodel. between domain elements and visual elements. In MetaEdit+, an expert creates a model- The source code that will support the DSL tool ing method by (a) defining a domain-specific is generated by using text templates, which language containing the problem domain’s process the DSL’s specification and output the concepts and rules (in this article, we will treat corresponding code. Developers can provide a DSL in MetaEdit+ as a metamodel since the additional code to refine aspects of the model tool does treat DSLs as metamodels), and (b) designer, define constraints over the language, specifying the mapping from that language to and/or even alter the text templates (which source code in a domain-specific code gen- can have substantial effects on the generated

Figure 10. MOF is used as a meta-metamodel A Small Case Study: Social Network and as a metamodel Metamodel An essential part of the evaluation of a tool is determining how that tool actually supports the activities necessary toward the resolution of a certain problem. Thus, we use the facilities provided by each tool to specify and imple- ment (when possible) a simple metamodel that supports the specification for simple social networks. This metamodel can be textually described by the following statements:

• A social network is composed of people and relationships between people. • A person’s participation in a relationship is defined by the role they play in it. • A role must have a corresponding relationship. • A role must have a corresponding person. • A social relationship must involve at least two different people.

Figure 11 presents this metamodel (and two user models, for illustrative purposes) modeled in Enterprise Architect. Note that this case study, because of its simplicity, could also be addressed with typical source code). Testing is done within Visual CASE tools (in fact, this is done in Enterprise Studio by launching another instance of the Architect). However, the main objective of environment with the specified DSL tool. After this article is to evaluate how the selected ensuring that the tool is working correctly, the tools behave in specifying the Social Net- final step is creating a deployment package work metamodel and afterward producing and that allows its distribution. adapting a tool that can be used to create user (For this evaluation, we used the DSL models (i.e., with types and instances) using Tools’ Version 1 release, which was the latest the language defined by that metamodel. version of this tool at the time this article was written.) Evaluating the Tools The evaluation framework’s application to Applying the Framework the presented tools was performed by us, so This subsection describes the small case study we did not need to resort to agreement mea- used to support this evaluation and the results sures, such as Cohen’s Kappa coefficient. To obtained by applying the evaluation framework compensate for the lack of a greater number to each of the selected tools. of test participants, we tried not to define any dimensions that depended on the user’s previous familiarity with one (or more) of the tools. Thus, the usage of each tool to define the Social Networks metamodel case study was

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 66 Journal of Database Management, 19(4), 50-75, October-December 2008

Figure 11. The Social Network metamodel and two user models

accompanied by thorough reading of avail- (i.e., editing or removing elements and able tool documentation and previous tests of constraints). the tool in order to gain a reasonable amount of experience with each of the selected tools. The definition of a profile in EA is limited Nevertheless, we acknowledge that such dimen- to specifying the generic syntax of the profile sions are important to measure usability and (i.e., defining stereotypes and what metaclasses the tool’s learning curve (and can be a good they extend, enumerations, etc.). Other semantic indicator of whether the tool will be accepted and syntactic relationships and constraints en- by the community). tered in the profile definition (using a text-based notation such as OCL) are not enforced when • Enterprise Architect. Enterprise Architect is the user creates a model using that profile; an easy-to-use tool with a minimal learning the only validation that EA does enforce is the curve. However, its traditional CASE-tool application of a stereotype to an instance of a roots make it extremely limited when it metaclass (e.g., a stereotype that extends the comes to metamodeling. Since EA is a metaclass Association cannot be applied to an UML modeling tool, the only mechanism instance of the metaclass Class). EA does pres- that it provides for metamodeling support ent the advantage of not requiring the creation is the UML profile mechanism, which of a new tool adapted to the problem domain as only allows adding elements and seman- it supports both the definition and application tics to the metamodel, but not altering it

of a UML profile (as is typically the case with • MetaSketch. From the set of evaluated tools, profile-supporting CASE tools). only MetaSketch supported metamodel- Like other CASE tools, EA does not appear ing based on the MOF standard. The tool to use level compaction or any similar technique supports the XMI import and export of because modeling is limited to one logical models and metamodels, so a user-defined level; in this case, adapting the tool to support model can become a metamodel simply more logical levels (by using level compaction) by exporting it to XMI and then import- would require an extra effort in order to sepa- ing it from XMI as a metamodel. In fact, rate the metamodel-model and class-instance the tool can easily handle the XMI-based relationships. The tool offers code generation specifications of MOF and UML available capabilities and some predefined basic model on the OMG Web site. transformations to support MDA, such as PIM to PSM. However, they require that PIMs and MetaSketch uses the language metaphor PSMs be specified in UML as it is the tool’s (Nobrega et al., 2006), which in this case limits hard-coded metamodel. the user to manipulating two logical levels: Figure 12 shows the definition of a profile the metamodel and the user model. However, representing the Social Networks metamodel MetaSketch uses level compaction, so it could previously presented; additionally, Figure 11 be adapted to use the library metaphor with presents two user models (obtained through the relatively little effort. Although MetaSketch application of the profile) modeled in EA. does not support model transformations (to It is important to reiterate that the reason either source code or other models), this can why EA is used in this evaluation is to show be remedied because of the tool’s XMI import that typical CASE tools are not adequate for the and export capabilities; the user could export metamodeling needs that are currently surfac- the model to XMI, and then process it with a ing, even though EA (as other CASE tools) is code generator (such as the Eclipse Modeling not designed to support metamodeling; this Framework, available at http://www.eclipse. evaluation is not meant in any way to diminish org/emf) or a model transformation tool (likely EA as a tool, and these results should not be based on QVT). interpreted as such. The syntax of the metamodel is specified in XML (outside the tool’s environment) by composing simple shapes (rectangles, ellipses,

Figure 12. A screenshot of Enterprise Architect with a profile definition

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 68 Journal of Database Management, 19(4), 50-75, October-December 2008 etc.) and using the tool’s geometry manage- does not include behavioral features (only ment mechanism (Nobrega et al., 2006), which structural features), which can impact the dynamically adjusts the spatial arrangement of possible set of metamodels that can be those shapes. The semantics of the metamodel defined by the tool. is specified in the tool itself when modeling the user model that later becomes the metamodel; MetaEdit+ apparently uses the language however, there is no support yet for constraint metaphor, limiting the number of logical levels specification. Nevertheless, it is important to the user can edit to the metamodel and the note that the tool is still a prototype under active user model. However, this metaphor is used not development, so it can be expected that such because of programming-language restrictions, issues will be corrected in the future. Thus, but by choice of the tool creators, so the tool the results obtained in this evaluation do not could be adapted to use the library metaphor reflect the full potential of this tool. with relatively little effort. Although the tool does not offer support for model transforma- • MetaEdit+. MetaEdit+ is based on a very tions, it does provide a report mechanism that simple and flexible meta-metamodel, allows the generation of text-based artifacts GOPPRR; however, this meta-metamodel (such as source code, HTML [hypertext markup

Figure 13. Social Network metamodel and user model in MetaSketch

language], or XML) based on the information following elements: (a) class, (b) domain available in the model’s repository. property, (c) embedding, (d) reference, and Syntax specification is done by creating (d) inheritance. Like MetaEdit+, this meta- instances of the meta-metamodel’s elements metamodel is highly object oriented but and, eventually, creating vectorial images to does not include behavioral features. represent those instances. Semantic specification is done when creating an instance of a The tool’s architecture is based on the lan- graph (which corresponds to a type of model, guage metaphor and limits the possible logical like UML’s class diagram or use-case diagram); levels editable by the user to the metamodel constraints are then entered in the graph’s cor- and the user model. However, this limitation responding form (e.g., “Objects of a certain is because DSL Tools are based on the class- type may be in, at most, a certain number of instance relation, so the adaptation of DSL relationships”), which is designed to avoid as Tools to use the library metaphor would likely much manual text entering as possible (since require a great deal of effort. it is prone to errors). The DSL designer itself is divided into two This tool did present a few important us- panes: Domain Model and Diagram Elements. ability problems, such as the fact that it does not In Domain Model, the developer identifies the allow the altering of the superclass-subclass relevant concepts of the problem domain and relationships between object types: Once the expresses them in the domain-model section user chooses an object type’s superclass (when of the designer along with model details like creating the object type), it cannot be changed; cardinality and source-code-specific details the user should first draw the metamodel on such as whether an association end should a piece of paper or another modeling tool in generate a property. Validations and constraints order to obtain the definitive metamodel, and can also be specified by typing source code in then re-create it within MetaEdit+. additional validation classes. In Diagram Ele- Figure 14 presents a model derived from ments, aspects relating to the graphical layout the Social Network metamodel presented of the model elements are specified, such as earlier. shapes used, association line styles, the shapes that can be on either end of an association, and • Microsoft DSL Tools for Visual Studio. how value properties are graphically displayed. This tool’s meta-metamodel consists of the Thus, the specification of the syntax and -se

Figure 14. Models of the Social Network metamodel in MetaEdit+

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 70 Journal of Database Management, 19(4), 50-75, October-December 2008 mantics of the metamodel is done entirely in Enterprise Architect provides a framework for the DSL designer (except for validations and model-to-model or model-to-code transforma- constraints, which are expressed in source tions in the context of the MDA initiative), likely code) as the DSL Tools are highly focused on because of the immature state of the area. the graphical specification of user models and Another interesting conclusion is that subsequent generation of text-based artifacts. each of the evaluated tools uses the language Figure 15 presents the Social Network user metaphor, although most of them support the model specified in DSL Tools. specification of two logical levels. This is likely due to the fact that a tool that supports more Results than two logical levels is likely to reveal itself The results of this evaluation are shown in as confusing since it can usually be assumed Figure 16. From these results, we can see that that developers will not need more than two some tools already treat metamodel exchange logical levels (one to specify a language that as an important issue as only Enterprise represents the problem domain—and perhaps Architect and MetaEdit+ do not export their another language that represents the solution metamodel definition. However, in Enterprise domain—and another to specify a solution to Architect’s case, this is understandable since the problem). However, both MetaSketch and the metamodel (UML) is hard-coded into the MetaEdit+ use level compaction, and thus tool and never changes. We find noteworthy could be adapted to use the library metaphor the fact that MetaSketch is the only tool al- (therefore supporting additional logical levels) lowing metamodel import and export using with little effort; DSL Tools would require a a well-defined standard (XMI). User-model much more extensive effort to support such exchange, however, is supported by all tools, additional logical levels. using either XMI or XML. Most tools support the specification of Model transformation does not currently a metamodel’s syntax to some degree (either seem to be a major concern as most tools do by associating elements with external image not provide any kind of support for it (only sources or by providing internal facilities to

Figure 15. Models of the Social Network metamodel in Microsoft’s DSL Tools

create such graphical representations). However, to the area of meta-CASE systems, which allow the specification of a metamodel’s semantic the automatic creation of development tools tai- constraints seems to be sketchy at best, with lored to specific design processes, determined only MetaEdit+ and DSL Tools supporting by organizational requirements. such constraint specifications. (MetaEdit+ The core problem with traditional CASE uses its meta-metamodel concepts to establish tools is that they only support specifying constraints in the metamodel, while DSL Tools the solution; the identification of the -prob requires that developers use source code to lem-domain requirements is often done apart specify constraints.) from these tools (usually in a word processor Finally, the tools that do not follow a or similar). Hence, developers do not have a standard meta-metamodel (e.g., MetaEdit+ problem-domain-oriented language in which and MS DSL Tools) seem to prefer using a they can express the solution to the problem, meta-metamodel that is as simple as possible: forcing them to think of the solution in compu- MetaEdit+’s consists of six elements, while tational terms (toward which traditional CASE DSL Tools’ consists of five. tool metamodels are especially oriented) rather than in problem-domain terms. The solution in- Discussion evitably becomes misaligned with the problem Although CASE tools failed on their first ap- domain and, therefore, with the problem itself. pearance some years ago (Booch, Brown, Iy- The consequences of this can be seen over the engar, Rumbaugh, & Selic, 2004), they brought entire development process, but become espe- the idea that development processes could be cially critical during the product maintenance supported by such tools as long as those tools phase, when the product must be adapted to ad- were adjusted to the development process. ditional problem conditions and requirements, Early CASE tools were too inflexible, usually usually requiring extensive developer effort forcing the development process to be adjusted because of the difficulty of assuring that the to the CASE tool instead of having the CASE product still solves the old problems while tool support the development process. This led also solving additional problems.

Figure 16. The evaluation results

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 72 Journal of Database Management, 19(4), 50-75, October-December 2008

However, when considering metamodeling adequate for requirements specification. Using and meta-CASE tools, we need to be careful a DSM system, a developer experienced in the because of possible meta-metamodel frag- problem domain creates a metamodel reflecting mentation: In this evaluation, we can see an that domain and specifies how domain concepts example of this as Microsoft DSL Tools uses are mapped to code (or any other artifact type). its own meta-metamodel and so does Me- Requirements are then specified as models taEdit+. This could lead to a panorama much (oriented toward not the implementation but like the one from a few years ago, in which what the client wants the system to do) using there was a myriad of modeling languages the defined metamodel. These models are then (i.e., metamodels) all doing the same and yet mapped into code using the mappings initially all different among themselves. Now that the defined. However, DSM as it is used today community (and the industry) is beginning to has a weak point: the transformation between focus on metamodeling and meta-metamodels models and code (or even between models of (i.e., metamodel languages), we need to start different languages). If the DSM system user considering meta-metamodel standards as they wishes to switch target platforms (for example, help eliminate gratuitous diversity (Booch et from Java to .NET), the mappings will have to al., 2004). Otherwise, the diversity of languages be re-created by the expert developer, unlike that would be defined would very likely lead what happens with MDA, as PIMs and PSMs to the fragmentation that UML was designed provide the ability to exchange target platforms to eliminate in the first place. with minimal extra effort. This is not unlike All this is theory that must be put into what is said in Schmidt (2006), which states practice in tools that developers can use. For that MDE is evolving toward DSLs combined such tools to be of help to the developer, they with transformation engines and generators; in must support the whole software development other words, MDE seems to be evolving toward life cycle, from requirements specification MDA and DSM working together. to deployment and maintenance. This also This is why we consider tools such as requires that tools allow developers to specify MetaSketch to be of utter importance to the solutions in problem-domain terms, which of industry, as MetaSketch reveals a genuine con- course requires that tools support some form cern with adhering to OMG standards (which of metamodeling. However, as the results of opens the door for its usage in MDA-oriented this evaluation show, the current tool support development scenarios) while also trying to is primarily directed toward DSM, and issues address the metamodeling problem that we such as model-to-model transformations (upon are facing today. which MDA is based) are being left out in all Another issue that we consider important but a few tools (such as the Eclipse Model- to the success of metamodeling is complexity. ing Project, available at http://www.eclipse. The usage of standards is always conditioned org/modeling). by their complexity and how well adapted they We believe MDA and UML have the are to the domain of interest. These points potential to adequately cover the development can be decisive factors over the difficulty of phases more directly related to software itself, creating a model that correctly represents the like implementation design and coding. How- problem (from the perspectives of syntax and ever, MDA does not address the requirements semantics), which is where DSM differentiates phase, leading to the known gap between what itself. The fundamental issue is that developers the client wants the system to do and what the and clients need to identify themselves with system actually does; in part, this is because the metamodels they use; otherwise, they will UML is not adequate for requirements model- look upon those metamodels as nuisances. ing. On the other hand, DSM’s strength over An example can be seen in MOF, sometimes MDA comes from the fact that it is more than considered too complex for defining user

metamodels, because it includes concepts that that can establish a higher degree of consensus would only be useful in the context of OMG- regarding metamodeling issues. defined metamodels. This is why tools such as After presenting the framework, we applied MetaEdit+ (with simple meta-metamodels) are it to a small set of current modeling tools that gaining popularity throughout the developer we believe to be representative of the status community, and MOF/UML CASE tools (with of the mainstream MDE area. Finally, this complex meta-metamodels) are typically con- article discussed some open research issues sidered as only good for documentation and for metamodeling-based software develop- a last resort for code generation. ment tools. Finally, we consider that the evaluation framework defined in this article is quite relevant Acknowledgment because it provides a good insight into the main We would like to thank Leonel Nobrega for his problems that metamodeling tools would face: promptness in supplying the latest version of his Its dimensions include support for language MetaSketch tool as well as all the documenta- specification (syntax and semantics) and model tion that was available at the time. We would transformations, which are essential to the also like to thank the reviewers of this article creation of metamodels and models, as well for all their excellent constructive suggestions as to obtaining new models in an automatic, to improve its quality. MDE-oriented fashion. References Conclusion Atkinson, C., & Ku¨hne, T. (2003, September-Octo- Just as development paradigms changed and ber). Model-driven development: A metamodeling evolved over the last decades from assembly foundation. IEEE Software, 20(5), 36-41. Retrieved code to subsequent generations of program- June 5, 2006, from http://doi.ieeecomputersociety. ming languages, the development paradigm org/10.1109/MS.2003.1231149 is changing from our current third-generation programming languages to a higher abstraction Atkinson, C., & Ku¨hne, T. (2005, October). Con- cepts for comparing modeling tool architectures. level. This shift is gradually happening as MDE In L. Briand & C. Williams (Eds.), Model Driven is gaining importance as an abstraction mecha- Engineering Languages and Systems: Eighth Inter- nism over traditional programming activity. national Conference, MoDELS 2005 (pp. 398-413). However, tools need to follow and sup- Springer. Retrieved June 23, 2006, from http://dx.doi. port this paradigm change. The only way that org/10.1007/11557432 30 a modeling tool can effectively support the soft- Booch, G., Brown, A., Iyengar, S., Rumbaugh, J., & ware developer’s complex tasks is by providing Selic, B. (2004, May). An MDA manifesto. Business metamodeling support: Such a tool should allow Process Trends/MDA Journal. Retrieved June 15, a software developer or architect to specify a 2006, from http://www.bptrends.com/publication- language or metamodel and be able to auto- files/05-04COLIBMManifesto-Frankel-3.pdf matically create tools that enable the creation of models based on that metamodel. Fong, C. K. (2007, June). Successful implementation of model driven architecture: A case study of This article presented a framework for how Borland Together MDA technologies were suc- evaluating a tool’s adequacy in the metamodel- cessfully implemented in a large commercial bank. ing activity. This framework defines some cri- Retrieved November 23, 2007, from http://www. teria that address both theoretical and practical borland.com/resources/en/pdf/products/together/to- issues in metamodeling and in modeling tools; gether-successful-implementation-mda.pdf nevertheless, it is still subjective and open to France, R. B., Ghosh, S., Dinh-Trong, T., & Solberg, further refinement by adding more important A. (2006, February). Model-driven development criteria and by defining measurement metrics using UML 2.0: Promises and pitfalls. Computer,

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. 74 Journal of Database Management, 19(4), 50-75, October-December 2008

39(2), 59-66. Retrieved June 5, 2006, from http://doi. Silva, A., & Videira, C. (2005). UML, metodologias e ieeecomputersociety.org/10.1109/MC.2006.65 ferramentas CASE (Vol. 2, 2nd ed.). Portugal: Centro Atlântico. Henderson-Sellers, B. (2005, February). UML the good, the bad or the ugly? Perspectives from a panel S¨oderstrom, E., Andersso, B., Johannesson, P., of experts. Software and Systems Modeling, 4(1), Perjons, E., & Wangler, B. (2002, May). Towards a framework for comparing process modelling 4-13. Retrieved June 5, 2006, from http://dx.doi. languages. In CAiSE ’02: Proceedings of the 14th org/10.1007/s10270-004-0076-8 International Conference on Advanced Informa- Kelly, S., & Tolvanen, J.-P. (2008). Domain-specific tion Systems Engineering (pp. 600-611). London:

modeling. Hoboken, NJ: John Wiley & Sons. Springer-Verlag. Retrieved June 21, 2006, from http://portal.acm.org/citation.cfm?coll=GUIDE&d Kleppe, A., Warmer, J., & Bast, W. (2003). MDA l=GUIDE&id=680389# explained: The model driven architecture. Practice and promise. Reading, MA: Addison-Wesley. SparxSystems. (n.d.). Enterprise architect: UML design tools and UML CASE tools for software devel- MetaCase. (n.d.). MetaCase: Domain-specific model- opment. Retrieved June 5, 2006, from http://www. ing with MetaEdit+. Retrieved June 5, 2006, from sparxsystems.com/ products/ea.html http://www.metacase.com Thomas, D. (2004, May-June). MDA: Revenge of the Metamodel.com: Community site for meta-modeling modelers or UML utopia? IEEE Software, 21(3), 15- and semantic modeling. (n.d.). Retrieved June 5, 17. Retrieved June 5, 2006, from http://doi.ieeecom- 2006, from http://www.metamodel.com putersociety.org/10.1109/MS.2004.1293067 Moore, G. C., & Benbasat, I. (1991, September). De- Tolvanen, J.-P., & Rossi, M. (2003, October). Me- velopment of an instrument to measure the perceptions taEdit+: Defining and using domain-specific mod- of adopting an information technology innovation. eling languages and code generators. In OOPSLA Information Systems Research , 2(3), 192-222. ’03: Companion of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Sys- Nobrega, L., Nunes, N. J., & Coelho, H. (2006, tems, Languages, and Applications (pp. 92-93). New June). The meta sketch editor: A reflexive modeling York: ACM Press. Retrieved June 5, 2006, from editor. In G. Calvary, C. Pribeanu, G. Santucci, & http://doi.acm.org/10.1145/949344.949365 J. Vanderdonckt (Eds.), Computer-Aided Design of User Interfaces V: Proceedings of the Sixth Inter- Visual Studio 2005: Domain-specific language tools. national Conference on Computer-Aided Design of (n.d.). Retrieved June 5, 2006, from http://msdn. User Interfaces (CADUI 2006) (pp. 199-212). Berlin, microsoft.com/vstudio/dsltools Germany: Springer-Verlag. Zhao, L., & Siau, K. (2007, November). Information Schmidt, D. C. (2006, February). Guest editor’s mediation using metamodels: An approach using introduction: Model-driven engineering. Computer, XML and common warehouse metamodel. Journal 39(2), 25-31. Retrieved June 5, 2006, from http://doi. of Database Management , 18(3), 69-82. ieeecomputersociety.org/10.1109/MC.2006.58 Zhu, J., Tian, Z., Li, T., Sun, W., Ye, S., Ding, W., et Selonen, P., Koskimies, K., & Sakkinen, M. (2003). al. (2004). Model-driven business process integra- Transformations between UML diagrams. Journal of tion and management: A case study with the Bank Database Management , 14(3), 37-55. SinoPac regional service platform. IBM Journal of Research and Development, 48(5/6), 649-669. Siau, K., & Cao, Q. (2001). Unified modeling lan- Retrieved November 23, 2007, from http://www. guage: A complexity analysis. Journal of Database research.ibm.com/journal/rd/485/zhu.pdf Management, 12(1), 26-34.

Jo˜ao de Sousa Saraiva was born in Macau, in September 1982. In September 2004 he joined the Information Systems Group (GSI) of INESC-ID, where he has since been developing skills in web application development and graphical specification of information software systems. In 2005, he graduated in Computer Science at Instituto Superior T´ecnico (IST), in Lisbon, Portugal. In 2007, he obtained an MSc in Computer Science from the same institution. He is currently a PhD student at IST, and a researcher at INESC-ID.

Alberto Rodrigues da Silva is professor of information systems at the Department of Computer Science and Engineering at Technical University of Lisbon (IST/UTL) Portugal. He is also a senior researcher at INESC-ID and director at the SIQuant Company. Rodrigues da Silva’s professional and research inter- ests are in modeling and metamodeling, model-driven engineering, requirement engineering, enterprise knowledge-based platforms, and CSCW and CASE Tools