JetBrains MPS as a Tool for Extending Java Vaclav Pech Alex Shatalin Markus Voelter JetBrains JetBrains independent/Itemis [email protected] [email protected] [email protected] Abstract bench [8], MetaEdit+1, Eclipse Xtext 2 and Spoofax 3 (for a JetBrains MPS is an integrated environment for language brief comparison of some of these tools see Related Work in Section 4). For a general overview of language workbenches, engineering. It allows language designers to define new pro- 4 gramming languages, both general-purpose and domain- please refer to the Language Workbench Competition . specific, either as standalone entities or as modular exten- Real-world use of MPS As a real-world proof of MPS, sions of already existing ones. Since MPS leverages the con- JetBrains has built YouTrack 5, a web-based bug tracking cept of projectional editing, non-textual and non-parseable application. It heavily relies on an MPS-based set of lan- syntactic forms are possible, including tables or mathemat- guages for web application development and database map- ical symbols. This tool paper introduces MPS and shows ping { Webr-DNQ. In less than three years YouTrack became how its novel approach can be applied to Java development. an important part of JetBrains' product portfolio, validating Special attention will be paid to the ability to modularize the LOP idea as well as its implementation in MPS. More and compose languages. on how YouTrack benefits from LOP can be found in [9]. Categories and Subject Descriptors D.2.6 [Software The MPS team cooperated with several universities to Engineering]: Programming Environments - Programmer help them adopt MPS as a tool for language design research. workbench For example, a student at the Faculty of Mathematics and Physics of Charles University in Prague implemented a real- General Terms D.3.2 Extensible languages, D.3.4 Code time Java development environment as part of his master generation, D.3.4 Translator writing systems and compiler thesis in 2012 [3]. generators Although MPS is historically bound to Java and the JVM, its principles are universal and can be applied to any Keywords language extension, DSLs, development envi- technological domain. The mbeddr project [11] successfully ronments, formal methods built a powerful development stack for embedded C develop- 1. Introduction ment. They primarily target programmers who write C code for micro-controllers in robots, avionics or car appliances. JetBrains MPS is an open-source language workbench based on a projectional editor. It started as an experimental Further plans Inspired by the success of YouTrack and project about ten years ago with the aim to test and vali- mbeddr, JetBrains has been gradually turning MPS into a date the ideas of Language Oriented Programming (LOP), solid open-source language workbench that is ready for the as summarized in [2]. In essence, MPS attempted to bring tasks in the mainstream of software development. The re- code generation and projectional editing (also known as cent 2.5 version can be considered a fully functional, stable structured editing, see [12] for a good definition) to the production-ready tool, targeting language and DSL design- programming mainstream, enabling easy language modular- ers as well as programmers. The open source license facili- ization and composition. The approach blurs the distinction tates adoption for open-source and academic use. between general-purpose (GPL) and domain-specific (DSL) programming languages with an intended positive impact on developer productivity. This paper introduces MPS, fo- cusing on extensibility of languages and IDEs in general and 2. Language Engineering in MPS for Java, specifically. Eating its own dog food, MPS, like many other language The term Language Workbench was coined by Mar- workbenches, offers a set of DSLs for defining various aspects tin Fowler [4]. Besides MPS, there are several other lan- of languages. These include the structure, editor, type sys- guage workbenches including the Intentional Domain Work- tem, the generator as well as support for sophisticated IDE functionality, such as code completion, intentions, refactor- ings, debugger and dataflow analysis. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. 1 Copyrights for third-party components of this work must be honored. http://metacase.com For all other uses, contact the owner/author(s). 2 http://eclipse.org/Xtext PPPJ'13,, September 11{13, 2013, Stuttgart, Germany.. 3 http://spoofax.org Copyright is held by the owner/author(s). 4 ACM 978-1-4503-2111-2/13/09. http://languageworkbenches.net http://dx.doi.org/10.1145/2500828.2500846 5 http://youtrack.jetbrains.com You may also check out two introductory screen-casts on inspect values in the high-level language (DSL), avoiding projectional editing in MPS 6 and on language definition7, exposure to the actual execution technology used. to get a concrete example of the discussed content. Type System The next step is the definition of the Abstract Syntax As a first step when defining a new type system. For example, the type of a condition of an language we specify the structure (aka abstract syntax or IfStatement must be boolean, or a type of a return state- meta model). This resembles object-oriented programming, ment must be compatible with the return type of the sur- language concepts are similar to classes. Concepts in MPS rounding method. MPS comes with a type system engine represent types of AST nodes and define the properties, that is capable of evaluating type system rules to assign methods, children and references that instance nodes may types to language elements (type inference) and to verify have. MPS offers three language aspects to define abstract correctness of types. Typing rules are specified as a set of syntax: the structure aspect defines the concepts, their prop- equations, such as the following: erties and relationships, constraints restrict the allowed set of values for properties and references, and the behavior as- infer typeof(ifStatement.condition) :<=: <boolean>; sociates methods with concepts. where the :<=: symbol specifies equal type or a subtype. The Concrete Syntax The second step is defining the editor engine tries to assign values to the type variables so that all for the concepts. This reflects the projectional nature of equations are satisfied. The declarative approach (based on MPS. Since code in MPS is never represented as plain unification8) enables easy extension of the type system by text (neither on the screen nor on disk), MPS languages adding additional equations and types. are never parsed and thus no grammar is required. This MPS also support checks that are not directly related enables the use of non-parseable notations, such as tables or to typing rules. A dedicated type of rules, called checking mathematical symbols. Instead of a parser, we define editors rules, may analyze the model and report warnings or errors for language concepts { a visual representation for AST whenever an incorrect construct is detected. Examples for nodes. MPS' editor definition capabilities let the language checking rules in Java include detecting the repetitive use designer define editors for her new language constructs as of the same string literal, assignment to a final variable, well as override editors of concepts from existing languages. ignoring a return value and many others. Language designers Editors can be modularized to support reuse of visual syntax may associate automated fixes with checking rules, which elements and multiple editors can be defined for a concept, users can choose to execute in order to fix the problem. allowing the programmer to choose the notation that fits Code generation Languages in MPS also define transfor- best the task at hands. mation rules to lower-level languages or to plain text. The The default MPS editor needs some getting used to generation process in MPS consists of two phases. Phase for programmers familiar with text editing. Building code one uses a template-based model-to-model transformation from predefined language concepts instead of from individual engine to reduce the program code into the target language, characters definitely feels different. However, MPS goes a based on reduction rules specified in the generator. The tar- long way towards the editing experience of text editors. The get language may be further reduced based on its own re- editor's behavior can be tuned by specifying many of its duction rules, and so on. When no further model-to-model dynamic characteristics using, for example, node factories, transformation is applicable to a model, the second phase side transformations, or replacement rules. uses text generators to convert that final model into regular IDE Functionality Developers today expect the editor program text that can be fed into a compiler. to instantly assist them with code completion, smart navi- A language generator aspect consists of two main build- gation, intentions, refactorings, hints and code analysis. Be- ing blocks. Mapping Configurations define which concepts ing able to quickly build an IDE for a language is one of the are processed with which templates and in which context. most noteworthy advantages of MPS. The editor offers many Templates define the code in the target language that will re- of its essential features out-of-the-box, the remainder can place the specified piece of the source concept(s). Templates be implemented by the language designer through appropri- work differently from normal text generation templates such ate language aspects. For example, whenever the developer as for example Xpand9, Jet10 or StringTemplate11, since in hits Ctrl-Space, the code-completion pop-up menu is auto- MPS they do not define text output, but instead model-to- matically populated with all type-compatible nodes that are model transformations.