Open Comput. Sci. 2019; 9:52–79
Research Article Open Access
Lorenzo Bettini* Type errors for the IDE with Xtext and Xsemantics https://doi.org/10.1515/comp-2019-0003 perience and contributes to the adoption of that language. Received June 29, 2018; accepted February 11, 2019 However, developing a compiler and an IDE for a language from scratch is usually time consuming. Language work- Abstract: Providing IDE support for a programming lan- benches, a term coined by Martin Fowler in 2005 [1], are guage or a DSL (Domain Specific Language) helps the users software development tools designed to easily implement of the language to be more productive and to have an im- languages together with their IDE support. Xtext [2] is one mediate feedback on possible errors in a program. Static of the most popular language workbench. Starting from a types can drive IDE mechanisms such as the content as- grammar definition Xtext generates a parser, an abstract sist to propose sensible completions in a given program syntax tree (AST), and typical IDE mechanisms. While the context. Types can also be used to enrich other typical IDE main target of Xtext is Eclipse, a language developed with parts such as the Outline and the Hovering pop-ups. In this Xtext can be easily ported to other IDEs and editors. Xtext paper, we focus on statically typed imperative languages, comes with good defaults for all the above artifacts, di- adopting some form of type inference. We present a few rectly inferred from the grammar definition, and the lan- general patterns for implementing efficient type systems, guage developer can easily customize them all. focusing on type error recovery. This way, the type system Statically typed languages provide good IDE support. is able to type as many parts of the program as possible, Indeed, given an expression and its static type, the editor keeping a good IDE experience. Type error messages will can provide useful completion proposals that make sense be placed on the important parts of the program, avoid- in that program context. The same holds for other typical ing cascading errors that can confuse the user. We show IDE features, e.g., navigation to declaration, hovering and two case studies: we apply the presented patterns to imple- outline. In spite of the ease of implementing a language ment the type system of two statically typed DSLs, a sim- and its IDE support with a language workbench like Xtext, ple expression language and a reduced Java-like language, the implementation of the type system requires some effort with OOP features. We use Xtext as the language work- and careful tweaking: bench for implementing the compiler and the IDE sup- – The type system should type as many parts of a pro- port and Xsemantics, a DSL for implementing type systems gram as possible, so that the IDE can provide mean- using a syntax that mimics formal systems. The patterns ingful code completions even in the presence of an in- shown in the paper can be reused also for implementing complete and invalid program. languages with other language frameworks. – Type error messages should be placed on the impor- Keywords: language implementation, type system, IDE tant parts of the program, helping the user to quickly understand and fix the errors, avoiding cascading er- rors that would only confuse the user. 1 Introduction – Type computation should be performed quickly, in or- der to keep the IDE responsive. Integrated Development Environments (IDEs) help pro- In this paper we concentrate on mechanisms related to the grammers with mechanisms like syntax aware editor (syn- implementation of type systems addressing the above is- tax highlighting), immediate error reporting, code comple- sues, focusing on type errors and IDE support. The contri- tion (also known as content assist) and easy navigation butions of the paper are: to declarations. Providing IDE support for a language or – We describe some general implementation patterns a DSL (Domain Specific Language) enhances the user ex- for implementing type systems that aim at reporting only the important errors, with useful error messages, keeping the IDE responsive. In particular, we concen- *Corresponding Author: Lorenzo Bettini: Dipartimento di Statis- trate on imperative languages, adopting some form of tica, Informatica, Applicazioni, Università di Firenze, Italy; type inference. E-mail: [email protected]
Open Access. © 2019 Lorenzo Bettini, published by De Gruyter. This work is licensed under the Creative Commons Attribution alone 4.0 License Type errors for the IDE with Xtext and Xsemantics Ë 53
– We then demonstrate the benefits of such patterns first case study, showing the implementation of thetype by implementing two example DSLs in Xtext: a sim- system for the Expressions DSL. Section 5 shows our sec- ple “Expressions DSL” (with arithmetic, string and ond case study, that is, the implementation of the type boolean expressions) and a reduced version of Java, system of an OO language, Featherweight Java. Section 6 based on Featherweight Java (FJ) [3], a well-known for- discusses further improvements to the type system imple- mal framework for studying properties of the Java lan- mentations of the two DSLs. Section 7 discusses a few re- guage and for introducing extensions to Java. lated works. Section 8 concludes the paper. – For both examples, we show the benefits of applying the described patterns, in terms of the user experi- ence. 2 General implementation patterns Both languages¹ are simple languages but their type sys- A crucial issue to keep in mind when implementing type tems are complex enough to help us investigate best prac- systems is error recovery. This is similar to error recovery in tices in type system implementation. We implement the parser implementations [5]: a parser should be able to han- type system of the first DSL manually, while for FJ we use dle possible parsing errors and carry on parsing as much Xsemantics [4], a DSL for implementing type systems for of the rest of the input as possible, avoiding or minimizing Xtext languages. Xsemantics provides many features for additional cascading errors. The same holds for the type implementing complex type systems, like the one of FJ (in system: it should be able to type and check as many parts spite of its simplicity, the type system of FJ formalizes the of a program as possible, even in the presence of type er- main parts of the Java type system). Moreover, Xsemantics rors. Thus, the type system should avoid throwing cascad- provides a syntax that is close to formal type systems and ing errors due to previously failed type computations. This this makes it suitable to implement type systems that have is important in a command line compiler, but in the IDE it been formalized, like FJ’s type system [3]. The syntax of is even more crucial: we should minimize the portions of the two example DSLs is partly inspired by our previous the edited file marked with errors. This way, the user can work [2, 4], but the implementations shown in this paper easily understand the errors and quickly fix them. In par- are new. ticular, the type system will be continuously used by the The patterns presented in the paper are not strictly de- IDE while the user is editing the program. Thus, the pro- pendent on Xtext, Xsemantics and Eclipse: these are the gram that is examined by the type system will be an in- frameworks that are used in the paper to present our case complete program most of the time. studies. The same patterns could be applied to other lan- In this section we present some general implementa- guage workbenches. The basic idea of the implementation tion patterns for implementing type systems that aim at patterns will be given independently from these frame- reporting only the important errors, with useful messages works. and at providing a good user experience in the IDE. The intended audience of the paper is developers of We consider the implementation of a type system as statically typed languages and DSLs with IDE support. In the synergy of two main components: the type inference, particular, we will use Xtext as the language workbench that is, the mechanism responsible of computing the types and Eclipse as the target IDE. A basic knowledge of lan- of the program elements, and the type checker, that is, the guage implementation and type systems is advisable. We mechanism responsible of validating the program with re- will provide enough details on Xtext in order to make the spect to types. These two components will also have to co- examples understandable even by readers who are not fa- operate with the mechanism for cross reference resolution. miliar with Xtext. The paper is structured as follows. Section 2 motivates Infer a type as quickly as possible. Given a compos- our work and presents our general implementation pat- ite expression, if its type can be computed independently terns for type systems. Section 3 provides a brief introduc- from its subexpressions, the subexpressions should not tion to Xtext and its main mechanisms using the Expres- be inspected recursively. This has the benefit of improv- sions DSL as a running example. Section 4 illustrates our ing error recovery and the type inference performance. For example, let us consider an arithmetic expression of the shape e1 * e2². We can infer that the type of the expres-
1 The source code of the implemented DSLs presented in this pa- per can be found at: https://github.com/LorenzoBettini/xtext-type- errors-examples. 2 Where * is assumed to be the standard multiplication operator. 54 Ë Lorenzo Bettini sion is numeric, independently from the types of subex- pression will not be marked with an error. If we mixed type pressions. If one of the subexpressions has a non-numeric inference and type checking, we would issue an error on type, the whole expression is invalid. However, the validity the whole expression stating that it cannot be typed. This check should be performed by the type checker, in a sepa- would not be helpful. We refer to Section 7 for a review of rate component, as detailed in the next point. Sometimes, related works on type error reporting. depending on the semantics of the language, it might not We will see a few examples in Sections 4.5, 5.4 and 5.5. be possible to avoid inspecting subexpressions to compute Error recovery and cross references. In a lan- the type for the whole expression. For example, let us as- guage implementation, cross reference resolution might sume that in the language the operator + is overloaded and be strictly related to the type system. For example, in an represents both the arithmetic sum and the string concate- Object-Oriented language, in order to resolve a member nation (like in the first case study, Section 4). The type in- reference in a member selection expression (e.g., a method ference needs to first infer the types of subexpressions be- invocation on an object), we first need to compute the type fore computing the resulting type of e1 + e2. of the receiver expression. If the type inference is sepa- We will see a few examples in Sections 4.2 and 5.5. rate from type checking, we will be able to type a receiver Keep type inference and type checking separate. expression even in the presence of errors. This allows us It is also crucial to keep type inference and type checking to resolve the member. Otherwise, besides an error on the separate in the type system implementation. Mixing the whole receiver expression, there would be an error also on two mechanisms could allow the language developer to the member reference that cannot be resolved. An example implement a prototype of the type system quickly, but in will be shown in Section 5.5. the presence of type errors, the type inference could not Visibility and validity. When computing the possible compute types for the rest of the program. Going back to candidates for cross reference resolution, we should avoid the example above, e1 * e2, if type inference and type filtering out candidates that would not be valid because of checking are implemented as a single mechanism, a type some semantic rules of the language. This means that we error in one of the subexpressions would prevent the type should not mix “visibility” and “validity”: a declared ele- system from assigning a type to the whole expression. This ment can be visible in a program context even if its use is would also lead to cascading errors in many other parts not valid in that context. Cross reference resolution should of the program. Instead, from the type inference point of only deal with visibility, and validity should be checked view, we can still assign a numeric type to the expression. by another component of the language implementation. Even if one of the subexpressions has a non-numeric type For example, if we tried to refer to a local variable that and thus the whole expression is invalid, we can still in- is never declared, cross reference resolution should actu- fer that the type of the expression is numeric. Being able ally fail. On the contrary, if we tried to refer to a local vari- to assign a type to such an expression, even in the pres- able that is declared after the reference, we could still re- ence of a type error, will allow the type system to type other solve the reference; then, another component will check if parts of the program. Besides avoiding cascading errors, the reference is valid. In this example, the error reported a type system with error recovery can also detect type er- by the validation component would be more informative, rors in other parts of the program which could be useful for marking the reference as an invalid “forward reference”, the user. This pattern was already applied in openArchi- rather than a more general “unresolved reference” (an ex- tectureWare [6], and inspired applications in other frame- ample is shown in Section 4.5). Another typical example, works such as Spoofax [7], see for example [8]. in an Object-Oriented language, is trying to access a pri- We will see a few examples in Sections 4.5 and 5.5. vate field from another class. We should not mark theref- Useful type error reports. Separating type inference erence as unresolvable: it is more useful to report that in from type checking allows us to spot the crucial type er- that program context that field is not accessible (an exam- rors in the program. This way, we are able to create useful ple is shown in Section 5.4). type error messages and to report them on the important Cache computed types. In the type system imple- parts of the program. The user is then able to fix the er- mentation, the type of the same expression could be used rors easily. For example, if in the expression e1 * e2 the in many places, especially in our context, where the com- type of e2 is not conformant with the one expected by * ponents of the type system are kept separate. For this rea- we can mark only e2 with an error. The error message will son, the results of type computations could be cached to clearly state that the actual type of e2 does not match with improve the performance so that the IDE is kept respon- the type expected by *. Since the type inference is still able sive. We will see a few examples in Section 6. to give a type to the whole expression, then the whole ex- Type errors for the IDE with Xtext and Xsemantics Ë 55
Content assist and valid proposals. The content as- For FJ (Section 5), instead, we first implement the type sist should help the user by proposing completions that system with Xsemantics following the formalization of FJ are sensible in a given program context. A proposal is sen- of [3]. Unfortunately, formal type systems typically deal sible if it does not make the program invalid. Thus, the con- with type inference and type checking at the same time. tent assist should rely on the type system for filtering out Thus, this first implementation would not benefit from the proposals that would generate a type error in the program. advantages of the patterns described in this section. We In particular, it could perform the filtering by relying on the then apply such patterns to improve the type system im- type checker. We will see a few examples in Sections 4.6 plementation in order to have a better error recovery mech- and 5.4. anism and to provide better error messages. Show types in the IDE. Especially when the language The patterns presented in this section are meant to supports a form of type inference, where types are not de- be general enough to be applied also when using other clared by the user, the IDE should show the inferred types, language workbenches, and are not strictly dependent on for example, in the editor itself, in the outline, when the Xtext, Xsemantics and Eclipse. In this paper, we apply user hovers on a program part, etc. This way, the user can them using these frameworks and we adapt them to the understand how the type system computes the types, and rules and lifecycles of these frameworks. Xtext makes such can then easily debug a type error. A few examples are an adaption easy because it hides most of the internal de- shown in Section 4.6. If the type system enjoys error recov- tails of Eclipse components. An example is error marker ery, the IDE can then show information about types even generation: in Xtext it is enough to call an API method, and in a program with type errors. the Eclipse error markers will be created automatically by Xtext in all the Eclipse parts (Section 3.3). Moreover, Xtext The patterns described in this section should lead to lifecycle automatically executes most of the compilation an efficient type system with error recovery aiming at help- mechanisms (Section 3.4) applying good defaults and del- ing the user to easily understand possible type errors and egating to possible custom implementations (the typical to quickly fix them. Moreover, such an implementation of example is cross reference resolution in Xtext, Section 3.2). the type system should also allow the language developer Summarizing, the code of the examples implemented to implement useful IDE mechanisms, like, the content as- in the next sections might not be used as it is in other sist, code navigation (e.g., from a reference to the resolved frameworks. The implementation patterns can however declaration) and other features. still be applied. How easy this can be done highly depends Finally, keeping the mechanisms of a type system sep- on the features of the language workbench and it is out of arate allows us to have a modular implementation consist- the scope of the paper. ing of several components, loosely coupled, which are eas- ily testable in isolation. The implementation of the DSLs presented in this paper (available at https://github.com/ LorenzoBettini/xtext-type-errors-examples) contains all 3 Small introduction to Xtext the JUnit tests for all the aspects (from the type system up to the UI mechanisms, like, e.g., the content assist). In- In this section we provide a brief introduction to Xtext³, deed, the examples presented in this paper have been im- using a simple DSL for expressions that we call “Expres- plemented with a test-driven development approach. Such sions DSL”. This will also be the example of our first case a modular architecture also leaves the door open to an easy study. In the Expressions DSL, programs consist of variable replacement of each single component. declarations with an initialization expression and of eval- In the next sections, we will apply the patterns de- uations statements. The syntax of variable declarations is scribed above to the implementation of the type systems var name = exp. The syntax of evaluation statements is of our DSL case studies. eval exp. Expressions can perform standard arithmetic For the Expressions DSL (Section 4) we will first infor- operations, compare expressions, use logical connectors, mally define its type system and then we will proceed with and concatenate strings. Expressions can also refer to vari- its implementation keeping the aspects of the type system ables. We will use + both for representing arithmetic addi- separate, showing how we can easily deal with error recov- tion and for string concatenation (in this case, subexpres- ery thanks to our modular implementation. This allows us sions are meant to be automatically converted to strings). to show useful error messages, avoiding cascading errors, and to enrich the Eclipse tooling with type information, even in the presence of type errors. 3 https://www.eclipse.org/Xtext/ 56 Ë Lorenzo Bettini
The types used in this DSL are only integer, string and grammar boolean types. In order to make the example more inter- org.example.expressions.Expressions esting, types of variables are not declared explicitly and with org.eclipse. xtext .common.Terminals they are automatically inferred from the initialization ex- generate pression. expressions "http://www.example.org/expressions/Expressions" We will provide enough details on Xtext in order to make the examples understandable even by readers who ExpressionsModel: are not familiar with Xtext. We then refer to [2] for full de- elements+=AbstractElement*; tails on Xtext. AbstractElement: Xtext is a language workbench: the developer writes Variable | EvalExpression; a grammar definition and starting from this definition Variable: Xtext generates a parser, an abstract syntax tree (AST), "var" name=ID ’=’ expression=Expression; and a fully-fledged Eclipse editor with syntax highlight- EvalExpression: ing, navigation, content assist and outline. Xtext also gen- "eval" expression=Expression; erates an automatic Eclipse builder. Xtext relies on EMF (Eclipse Modeling Framework) for the generation of the Expression: Or; AST. EMF [9] is a modeling framework for representing Or returns Expression: and manipulating structured data models. From a model And ( current "||" specification (the metamodel) EMF provides tools for code {Or. left = } right=And )*; generation mechanisms and runtime support to produce a set of Java types (interfaces and classes), factories for And returns Expression: Equality ( the model, and a set of classes with observable/observer {And.left=current} "&&" right=Equality mechanisms on the model. Xtext takes care of creating the )*; metamodel and generate the Java code. The DSL developer Equality returns Expression: only needs to know the EMF conventions about the gener- Comparison ( ated Java code and the EMF general API for traversing an {Equality. left =current} op=("==" | "!=") right=Comparison EMF model (i.e., in this context, the AST). We partly intro- )*; duce the main EMF features in the rest of this section to make the code snippets understandable. Comparison returns Expression: PlusOrMinus ( {Comparison.left=current} op=(">=" | "<=" | ">" | "<") right=PlusOrMinus 3.1 Grammar )*; PlusOrMinus returns Expression: An Xtext grammar is specified using an EBNF-like syntax. MulOrDiv ( ({ Plus. left =current} ’+’ | {Minus.left=current} ’-’) The Xtext grammar for the Expressions DSL is shown in right=MulOrDiv Figure 1. Before getting into the details of an Xtext gram- )*; mar, we show in Figure 2 the editor generated by Xtext MulOrDiv returns Expression: starting only from the grammar in Figure 1. Besides the Primary ( syntax highlighting, a default content assist is also gener- ({ MulOrDiv.left=current} op=(’*’ | ’/’)) right=Primary ated based on the grammar: after the “+” it proposes also )*; the variables defined in the program. We will later cus- returns tomize this content assist (Section 4.6). Primary Expression: ’(’ Expression ’)’ | In an Xtext grammar, a rule is defined using a se- {Not} "!" expression=Primary | quence of terminals (that is, quoted strings) and non- Atomic; terminals (that is, names of other rules). For example, in Atomic returns Expression: the rule Variable, "var" and ’=’ are terminals while ID {IntConstant} value=INT | and Expression are non-terminals: {StringConstant} value=STRING | {BoolConstant} value=(’true’ | ’false’) | Variable: {VariableRef} variable=[Variable ]; "var" name=ID ’=’ expression=Expression; Figure 1: The Xtext grammar of the Expressions DSL. Type errors for the IDE with Xtext and Xsemantics Ë 57
public interface AbstractElement { Expression getExpression(); void setExpression(Expression value); }
public interface Variable extends AbstractElement { String getName(); void setName(String value); }
public interface EvalExpression extends AbstractElement { }
public interface VariableRef extends Expression { Figure 2: The editor generated for the Expressions DSL. Variable getVariable (); void setVariable(Variable value); } Alternatives are marked by a pipe |. For example, an AbstractElement can be either a Variable or an Figure 3: The EMF Java interfaces generated for the rules EvalExpression: AbstractElement, Variable, EvalExpression and VariableRef.
AbstractElement: Variable | EvalExpression; ... In the grammar rules, assignment operators define {VariableRef} variable=[Variable ]; how the AST is to be constructed during parsing. The iden- By default, cross references are parsed as identifiers. Cross tifier on the left-hand side refers to a property of theAST reference resolution is handled automatically by Xtext and class (which is also generated by Xtext). Since the AST is can be customized by the developer (as described in Sec- generated in the shape of an EMF model [9], we will use tion 3.2). the EMF terminology for “property”, that is, feature. The An Xtext grammar cannot be left-recursive since Xtext EMF model representing the parsed AST, is contained in relies on LL-parsing algorithms, which are known to be memory in an EMF resource. EMF resources are in turn very good at error recovering but cannot handle left recur- contained in a resource set. The right-hand side is a ter- sion [5]. For this reason, rules that would be left recursive minal or non-terminal. When a feature in the AST is a list, by nature, have to be written avoiding left recursion, ac- the operator += is used in assignments, see, for example cording to the patterns prescribed by Xtext, known as left elements in ExpressionsModel: factoring; we refer the interested reader to [2] for all the de- ExpressionsModel: tails. We just mention here that rules with a lower prece- elements+=AbstractElement*; dence (like Or) must be defined in terms of rules witha The standard operators can be used for the cardinality: ? higher precedence (like And). for zero or one, * for zero or more and + for one or more. During parsing, the AST is automatically generated by The with specification at the beginning of the gram- Xtext as an EMF model. Thus, we can manipulate the AST mar represents the language inheritance support provided using all mechanisms provided by EMF itself. There is a by Xtext: similar to Object-Oriented inheritance, the rules direct correspondence between the rules of the grammar of the parent grammar will be available in the inheriting and the generated EMF model Java classes. For instance, grammar. For example the rule for a standard identifier, in Figure 3 we show the EMF generated interfaces for ID, used in the rule Variable, is defined in the parent the rules AbstractElement, Variable, EvalExpression grammar Terminals, one of the base grammar provided and VariableRef. Note also the inferred inheritance by Xtext. relation: Variable and EvalExpression inherit from Square brackets on the right-hand side of an assign- AbstractElement and the common structure is pulled up ment define cross references to another element inthe in the supertype. Finally, cross references are evident also parsed model; the name in the brackets is the type of the in the structure of the generated Java interfaces: the type of referred element, which, by default, corresponds to the the variable feature in VariableRef is Variable. Inter- name of a rule. For example, variable=[Variable], in nally, Xtext automatically keeps the EMF model represent- the Atomic rule, represents in the AST a reference to an ing the AST and the editor’s contents in synchronization. element of type Variable: The code generated by Xtext comes with good and sensible defaults, thus, it can be used as it is for several Atomic returns Expression: 58 Ë Lorenzo Bettini aspects of the implemented language. However, mecha- public class ExpressionsExampleValidator nisms like the type system of the language cannot be ex- extends AbstractExpressionsValidator { pressed in the grammar itself, and have to be implemented @Check public void checkVariableNameLowercase(Variable v) { by the developer by customizing some classes used in the if (! Character.isLowerCase(v.getName().charAt(0))) framework. The custom code is “injected” in the classes warning("Variable name should start with a lowercase", null of the Xtext framework using Google-Guice [10], a depen- v, ); } dency injection framework. The Java annotation @Inject, } which we will use in the code snippets from now on, comes from Google-Guice. Figure 4: An example of a @Check method. The next subsections describe the two complementary mechanisms of Xtext, scoping and validation, respectively, Once cross references are resolved, the editor gen- which the developer typically has to customize. We will see erated by Xtext automatically provides navigation mech- how they relate to the mechanisms of a type system. Scop- anisms (“Navigate to symbol”). Moreover, the scope ing and validation together implement the mechanism for provider is also automatically used by the content assist. checking the correctness of a program. These two compo- Thus, the developer customizes only the single concept nents are kept separate to facilitate a modular implemen- of scope and the IDE mechanisms automatically work ac- tation of a language. This separation can be found in other cordingly. approaches as well, such as, e.g., [11–14], where the con- For FJ (Section 5), we implement the scope provider cept of scoping is referred to as binding. We will also de- for field and method references by using the type ofthe scribe the stages that Xtext executes and when the above receiver expression. Thus, the scope provider is imple- mechanisms come into play. mented using the type inference.
3.2 Scoping 3.3 Validation Xtext automatically deals with resolving cross references, Apart from cross reference resolution, which is part of the that is, binding references to the original definitions. validation of a program, all the other semantic checks that When defining an Xtext grammar, as shown in the previ- are not expressable in the grammar, are delegated by Xtext ous section, rules specify cross references in the AST by to a validator. using []. The programmer provides a custom AbstractDecla- By default, Xtext binds references based on contain- rativeValidator and Xtext automatically calls the meth- ment relations [9] in the EMF model of the AST. This strat- ods in this class annotated with @Check for validating the egy works as expected in some simple cases: a parame- model according to the runtime type of the AST node. The ter reference occurring in a function body is bound to the validation automatically takes place in the background, parameter declaration in the signature of the containing while the user of the DSL is writing in the editor; an im- function. In other situations this will not suffice. For ex- mediate feedback is provided to the user. ample, in FJ (Section 5) field and method references must The DSL developer can call the methods error and take into account the inheritance relation among classes, warning of the validator base class, providing messages which is not modeled as a containment relation. about an issue and the offending element in the AST. Xtext In order to customize the binding of cross references, then automatically generates the appropriate error mark- in Xtext it is enough to customize the concept of scope, ers in the Eclipse workbench (e.g., in the editor, in the that is, the collection of all the elements that are “visible” “Problems” view and in the “Project Explorer” view). In in the current context of a reference. This is achieved by Figure 4 we show a possible validation for the Expressions providing a custom ScopeProvider. When Xtext needs to DSL: if a variable’s name does not start with a lowercase resolve a symbol, it will use the scope returned by such we issue a warning. a ScopeProvider. Using the returned scope, Xtext auto- In a statically typed DSL, validation is strictly con- matically resolves cross references or issue an error in case nected to the type system implementation. This implies the reference cannot be resolved. Note that Xtext automat- that first types have to be computed, especially when the ically handles cross references among different files and DSL allows types to be inferred, as in the Expressions DSL. import mechanisms, which, however, are out of the scope If a type cannot be computed for a given program term, an of the current paper. Type errors for the IDE with Xtext and Xsemantics Ë 59 error should be issued. If types can be computed, then type unresolvable cross references are reported by Xtext au- conformance must be checked. Type conformance deals tomatically as errors. with checking whether an expression has a type that con- forms to the type expected in a given program context. Xtext keeps track of proxies that are currently being re- The visit of the AST is handled by Xtext, which tra- solved and possible cycles during this resolution, due to verses the tree and call the appropriate @Check methods, bugs in the DSL implementation, are caught by Xtext, according to the actual type of the AST node. Finding the avoiding infinite loops. right program context to validate a term depends on the se- The type system can be used both during scoping and mantics of the DSL. Choosing the right context for valida- validation, depending on how the language developer im- tion also allows the DSL developer to provide useful infor- plements these two mechanisms. This implies, due to the mation on type errors and to report errors on the relevant above stages, that the type system itself can transparently parts of the program. trigger further proxy resolutions. Note that Xtext provides some predefined validation Finally, recall that the language developer does not checks that can be enabled for the DSL, like checking that customize the actual cross reference resolution: she cus- names are unique in the program. For the DSLs presented tomizes the scoping. Thus, the developer provides the can- in this paper, this validation mechanism can be used out didates that are then used by Xtext to actually perform of the box and we do not have to manually check for dupli- cross reference resolution. That is why it is hard to intro- cate names. duce bugs that lead to the above mentioned cycles.
3.4 Xtext stages 3.5 Xtend
In this section we briefly describe the stages executed Xtext fosters the use of Xtend for implementing all the cus- internally by Xtext, in order to clarify when the scope tom mechanisms of a DSL implementation. Xtend is a DSL provider and the validator are used. for the JVM (implemented in Xtext itself). Xtend code is 1. The textual program is parsed and the AST is created compiled directly into Java code. Xtend has a Java-like syn- as an EMF model, stored in a resource (in turn, con- tax removing most of the “verbosity” of Java. For exam- tained in a resource set). During this stage, cross refer- ple, terminating semicolons are optional and so are paren- ences are not resolved. Instead, EMF proxies are used thesis in method invocations without arguments. Even to represent cross references. EMF proxies are place- return is optional in methods: the last expression is the holders where Xtext stores information for later cross returned expression (this applies also to branch instruc- reference resolution, e.g., the “name” of the referred tions). Xtend is completely interoperable with the Java type element. system, thus any Java library can be reused from within 2. Before the validation, the resource is traversed to re- Xtend. Xtend has a powerful type inference mechanism: solve cross references (resolution of proxies). If the types in declarations can be omitted when they can be in- proxy resolution logic traverses cross references of the ferred from the context (as in variable declarations with model, that are yet unresolved, it transparently trig- the syntax val or var, for final and non final variables, re- gers resolution there too. As mentioned before, Xtext spectively). Xtend also includes enhanced switch expres- automatically handles cross references among differ- sions (i.e., with type guards and automatic casting) to ent files. During this stage possible cross references avoid writing lengthy instanceof statements. Moreover, to external resources’ contents are resolved as well, it provides extension methods that simulate adding new traversing other resources in the resource set. As de- methods to existing types without modifying them: in- scribed in Section 3.2, when Xtext resolves a cross ref- stead of passing the first argument inside the parentheses erence, it queries the scope provider for possible can- of a method invocation, the method can be called with the didates. first argument as its receiver. 3. Finally, the validator is executed. Since unresolvable Xtend lambda expressions have the shape proxies are cached by Xtext, if there are yet unre- [ param1, param2, ... | body ] solved cross references, the scope provider will not be where the types of parameters can be omitted if they can used again. If the validation traverses other resources, be inferred from the context. proxy resolution is triggered on demand on the other resources. After validation has completed, possible 60 Ë Lorenzo Bettini
In Xtend the equality operator == maps to the equals catenation expression, are meant to be implicitly con- Java methods for objects. The triple operator must be used verted to strings; === for comparing object references. 4. boolean expressions require the subexpressions to Xtend itself is an Object-Oriented programming lan- have type boolean and the result has type boolean; guage, mimicking Java structures such as methods, fields, 5. an Equality expression requires the two subexpres- classes and interfaces, with the same semantics. Also in sions to have the same type and the result has type this case, Xtend removes “syntactic noise” adopting de- boolean; faults: classes and methods are public by default and 6. the same holds for a Comparison, but boolean expres- fields are private by default. Any declaration marked with sions cannot be used in a Comparison expression; the keyword extension allows the programmer to use its 7. the type of a variable is inferred from the initialization methods as extension methods. expression; We will use Xtend in this paper. Java programmers 8. the type of a variable reference is the type of the re- should have no problems in understanding the Xtend code ferred variable, if the reference is valid (i.e., it refers presented in this paper. to an actually declared variable and it is not a forward reference), or null otherwise.
4 Typing expressions The Expressions DSL comes with an interpreter and a code generator (that simply generates a textual file with the in- terpreted expressions); as usual, the interpreter must re- In this section we will sketch the type system implemen- spect the static semantics, i.e., the type system, and must tation for the Expressions DSL that we introduced in the perform conversions accordingly. This is out of the scope previous section. We apply the patterns described in Sec- of the current paper, and we refer the interested reader to tion 2. In spite of the simplicity of the Expressions DSL, the complete implementation of the example. we chose it as the first running example because typing The shape of the AST, which reflects operator prece- expressions might be tricky to implement efficiently, espe- dence, associativity and possible grouping by explicit cially when errors have to be reported in an IDE. Moreover, parenthesis, is taken into consideration when computing reporting relevant errors on the relevant parts of the pro- types of subexpressions. However, by the rules informally gram requires some additional effort. defined above, the final computed type of an expression will not depend on the shape of the AST. For example, both 4.1 Expressions DSL type system "a" + 1 + 2 and "a" + (1 + 2) will have type string. In particular, in the latter case, (1 + 2) has type inte- ger but then the presence of the string "a" will assign the First, we informally define the type system of this DSL. This whole expression type string. On the contrary, the result of informal presentation defines both the type computation the evaluation will be different: "a" + 1 + 2 evaluates to rules (i.e., the type of each kind of expression) and the type "a12", while "a" + (1 + 2) evaluates to "a3". Both eval- checking rules (i.e., the expected types of the subexpres- uations respect the static semantics. Java string concate- sions): nation behaves the same way. 1. constant expressions have the corresponding types as In the next section we implement this type system in expected; the Xtend class ExpressionsTypeSystem. In particular, 2. arithmetic expressions have type integer and require we separate the mechanisms for type computation (i.e., the subexpressions to have type integer, with the ex- type inference), type conformance and type checking, as ception of Plus, as detailed in the next point; advised in Section 2. 3. a Plus requires the two subexpressions to have type integer. The only exception is the case when one of the subexpressions has type string: the whole expression 4.2 Type inference is considered to have type string (since it is to be inter- preted as the string concatenation operation); if they The code related to type inference is shown in Figure 5. have both type integer, then the whole expression has From now on, we will use Xtend features extensively type integer. Note that the semantics of Plus implies (Section 3.5). For example, in Figure 5, the expression that any expression can be concatenated with a string, e.left.inferredType uses the syntactic sugar for get- and integer and boolean expressions, in a string con- ters, the method inferredType is used as an extension Type errors for the IDE with Xtext and Xsemantics Ë 61
class ExpressionsTypeSystem { // ... continues from the previous listing public static val STRING_TYPE = new StringType public static val INT_TYPE = new IntType /** public static val BOOL_TYPE = new BoolType * Is type1 conformant to type2: is type1 subtype of type2? */ @Inject extension ExpressionsModelUtil def boolean isConformantTo(ExpressionsType type1, ExpressionsType type2) { def isStringType(ExpressionsType type) { return type2.isStringType || type1 === type2 type === STRING_TYPE } } def isIntType (ExpressionsType type) { // ... similar // ... continues
def ExpressionsType inferredType(Expression e) { switch (e) { Figure 6: Type conformance for the Expressions DSL. // trivial cases StringConstant: STRING_TYPE IntConstant | MulOrDiv | Minus: INT_TYPE We would like to stress that the type inference is sim- BoolConstant | Not | Comparison | Equality | And | Or: BOOL_TYPE ple and compact also because it does not perform any form Plus: { // recursive case of type checking. It is also straightforward to verify that the val leftType = e. left .inferredType code in Figure 5 implements the type computation part of val rightType = e. right .inferredType if ( leftType .isStringType || rightType.isStringType) the type system informally described in Section 4.1. In par- STRING_TYPE ticular, the only cases when a type cannot be inferred (and else it is null) are forward references and references to unde- INT_TYPE } fined variables. VariableRef: { if (e.isVariableDefinedBefore) // avoid infinite recursion e.variable .expression.inferredType } 4.3 Type conformance } // else the result is null } // ... continues Even if the Expressions DSL is a simple language, it has the notion of type conformance: since the DSL provides Figure 5: Type inference for the Expressions DSL. implicit string conversion (with the string concatenation operation), then any expression can be used in a context where a string is expected. The implementation of type method, and the parenthesis for method call without ar- conformance is shown in Figure 6. Since we implemented guments are omitted. In Java that would correspond to the types as singleton instances, we compare them directly expression inferredType(e.getLeft()). with === (see Section 3.5). Since in this DSL types are not explicit and new types cannot be defined, the types are modeled by sin- gleton instances of simple classes (not shown here). 4.4 Type expectations The classes representing types all extend the base class ExpressionsType. The last mechanism we need in our type system is the one For most cases type computation is trivial, since we defining the expected type of an expression in a given con- do not need to inspect subexpressions. For computing text. In this DSL only non-atomic expressions have expec- the type of a variable reference, we use the utility method tations on the subexpressions. ExpressionsModelUtil.isVariableDefinedBefore The computation of expected types is shown in Fig- (used here as an extension method) that detects possible ure 7, where the eContainer method is part of the stan- forward references. Since this is not related to typing dard EMF API [9]. Note that this implementation only lists concepts we are not showing the code here. We need the cases when there are expectations, and in all other to check that a variable reference does not refer to a cases the returned expectation is null. Again, the case for forward declaration because this could trigger an infinite the sum is the interesting one, since if none of the subex- recursion. The other interesting case is the one of type pressions are strings, then both subexpressions are ex- inference of an additive expression: the inferred type pected to have integer type. depends on the fact that one of the two subexpression The implementation of Figure 7 covers all the cases of has type string. This case actually requires inspecting the type system informally described in Section 4.1. subexpressions. 62 Ë Lorenzo Bettini
// ... continues from the previous listing class ExpressionsValidator extends AbstractExpressionsValidator { /** * The expected type or null if there’s no expectation protected static val ISSUE_CODE_PREFIX = */ "org.example.expressions." def ExpressionsType expectedType(Expression exp) { public static val TYPE_MISMATCH = val container = exp.eContainer ISSUE_CODE_PREFIX + "TypeMismatch" switch (container) { MulOrDiv | Minus: INT_TYPE @Inject extension ExpressionsTypeSystem Not | And | Or: BOOL_TYPE Plus: { @Check val leftType = container. left .inferredType; def void checkForwardReference(VariableRef varRef) { val rightType = container. right .inferredType; // not shown here if (! leftType .isStringType && !rightType.isStringType) { } INT_TYPE } else { @Check STRING_TYPE def void checkTypeConformance(Expression exp) { } val actualType = exp.inferredType } val expectedType = exp.expectedType } } if (expectedType !== null && actualType !== null && // ... continues !actualType.isConformantTo(expectedType)) { error ("expected " + expectedType + ", but was " + Figure 7: Type expectations for the Expressions DSL. actualType, exp.eContainer, exp.eContainingFeature, 4.5 Type checking TYPE_MISMATCH) } } Now that we implemented all the mechanisms for type in- // ... continues ference, conformance and expectations, we are ready to implement type checking in the validator of the Expres- Figure 8: Checking type conformance. sions DSL, ExpressionsValidator. Since we separated type inference and type expecta- correct information and the error markers will be automat- tions, it is straightforward to verify type conformance with ically generated. In Figure 8, we create a string message of a single @Check method, as shown in Figure 8 (see Sec- the shape “expected
It is crucial to issue an informative error and to place it in the relevant part of the program. In Xtext, as sketched in 4 Just like eContainer, eContainingFeature is part of the standard Section 3.3, it is enough to call the method error with the EMF API, which all the Java code generated by EMF inherits. Type errors for the IDE with Xtext and Xsemantics Ë 63
Figure 10: Type inference is not affected by other type errors. Figure 9: The errors about type conformance. conformant with the expected type boolean. The error is issued on the containing node And in the part that corre- sponds to the right feature containing "a" (refer to the And rule in Figure 1). Note that once we provide the correct arguments to the error method, all the markers are gener- ated automatically by Xtext in the “Problems” view, in the editor rulers and in hovering pop-ups. We believe that this strategy effectively produces in- formative errors in the relevant part of the program. The user of the DSL should have no problem in understanding what is going wrong in the program. Moreover, since we kept type inference separate from type checking, a type er- Figure 11: Type inference is not affected by forward reference errors. ror in a part of the program does not prevent our type sys- tem from typing the other parts. For example, as shown The validator should also implement the type check- in Figure 10, although the initialization expression of the ing concerning requirements 5 and 6; since their imple- variable k contains a type conformance error, our type sys- mentation consists of two further @Check methods follow- tem can still infer the type boolean for k so that other ex- ing a similar strategy to the one described so far, we will pressions using k are not marked with errors. Of course, not show them in the paper. the whole program is not valid, but generating cascading errors would only add “noise”, making it harder to fix the errors. 4.6 Using types in the IDE Note also that type inference is not affected by other errors like forward references. Also the validator shown in Xtext allows the DSL developer to customize many Eclipse Figure 8 does not perform conformance checking when the parts concerning the DSL, for example, the “Outline” type of an expression cannot be computed, like in the case (which is automatically synchronized with the editor) and of a forward reference. That problem is issued by the other the “Hovering” for single parts of the current editor textual @Check method (not shown in the paper). Reporting an- elements. It is quite standard for statically typed languages other error would not add useful information. In Figure 11 to use types in such IDE parts. For example, Eclipse JDT an example of such a situation is shown, where only the (Java Development Tools) provides nice representations of useful error is reported. It is worth mentioning that also the the current Java program in several views of Eclipse. scope provider implementation of the Expressions DSL al- In the Expressions DSL we can use types in the “Out- lows Xtext to resolve a forward reference: the figure shows line” view: we can show all the variable declarations with that the forward reference i in the variable declaration k their inferred types. That is another reason why it is crucial is still correctly bound to the declaration of i. to be able to compute as many types as possible, disregard- 64 Ë Lorenzo Bettini ing of type errors. We can also show type information when the user hovers on an expression. The results can be seen in Figure 12, where types are correctly shown even in the presence of errors in the program (the popup appears since we hover on && in the editor). The implementation of such Eclipse mechanisms is straightforward in Xtext and in this example it is just a matter of using the ExpressionsType- System, thus we do not show it here.
Figure 13: Content Assist suggests only valid proposals.
class ExpressionsProposalProvider extends AbstractExpressionsProposalProvider { @Inject extension ExpressionsModelUtil @Inject extension ExpressionsTypeSystem
override completeAtomic_Variable(EObject elem, ...) { if (elem instanceof Expression) { val elemType = elem.inferredType elem.variablesDefinedBefore . filter [ variable | Figure 12: Types used in the Outline and for Hovering. variable .expression .inferredType.isConformantTo(elemType) ] As mentioned in Section 2, the content assist should .forEach[ avoid suggesting proposals that would make the program variable | // ... create proposal invalid. After all, one of the advantages of statically typed ] languages is that the IDE can propose sensible comple- } tions by using the types. For the Expressions DSL, for ex- } } ample, the content assist should not propose completions for variable references that refer to a forward reference. Figure 14: The Content Assist for variable reference completion, Moreover, it should not propose variables whose type is relying on types. not conformant to the type expected by the current expres- sion. The desired result is shown in Figure 13, where only represents content (for example, labels and icons) that are the variables of type integer (and that are not forward ref- shown within the text editor along with the source text, but erences) are proposed in the context of the multiplication are not part of the program itself (thus, that additional text expression. The interesting parts of the implementation is ignored by the compiler). Some examples in Java are the of these filtered proposals in the content assist are shown number of references to a given declaration or the name of in Figure 14 (Xtext will reflectively invoke methods in this the parameters in a method invocation. class based on the name of the method and on the specific Xtext supports code mining and we can use this mech- program context where content assist is requested). anism to show the inferred types of variables also in the Once again, having a type system that is able to com- editor itself, while the user is writing the program. The ad- pute types in spite of errors in other parts of the program is ditional text of the code mining is not part of the program, crucial for implementing sensible proposals in the content it does not have to be part of the DSL syntax and it is not assist: most of the time, when the user asks for proposals, even modifiable by the programmer. the program is not complete and probably not valid. An example of the code mining in action is shown in Finally, we can use the inferred type for implementing the screenshots of Figure 15, where we see how the code code mining for the Expressions DSL editor. A code mining⁵ mining changes while the user is editing the program. Re- call that the “ :
class Object { } class A extends Object {} class B extends Object {}
class Pair extends Object { private Object fst ; private Object snd;
public Object getFst () { return this . fst ; } public Object getSnd() { return this .snd; } public Pair setFst(Object newFst) { return new Pair(newFst, this .snd); } public Pair setSnd(Object newSnd) { return new Pair(this . fst , newSnd); Figure 15: Code Mining in action for inferred types of expressions. } } class ExpressionsCodeMiningProvider new Pair(new A(), new B()). setFst(new B()) extends AbstractXtextCodeMiningProvider {
@Inject extension ExpressionsTypeSystem Figure 17: An example in FJ.
override void createCodeMinings(XtextResource resource, ...) { // get all variable declarations centrate on typing issues, focusing on the main mecha- val variables = EcoreUtil2.eAllOfType(resource.contents.get(0), nisms of OOP,namely, inheritance and method invocation. Variable) We implement the type system of FJ using Xsemantics. Since FJ provides the formalization of the type system, us- for ( variable : variables) { val type = variable .expression.inferredType ing Xsemantics has the benefit that the implementation, if (type !== null ){ that is, the specification in Xsemantics, is quite similar to // ... create the code mining } the formal type system of FJ. Note that Xsemantics could be } used also for implementing the type system of the Expres- } sions DSL (Section 4). However, for the first case study we } preferred to provide a hand written implementation of the type system to focus on the design choices and implemen- Figure 16: The code mining for variable declarations, relying on types. tation patterns, without introducing another framework. In general, the use Xsemantics for implementing the type system could be preferred to a hand written implementa- error recovery capabilities of our type system. The interest- tion for languages where the type system has been already ing parts of the implementation of code mining, using the formalized. However, the mechanisms provided by Xse- type system, are shown in Figure 16. mantics, like caching and tracing (described in Section 6), could be helpful also when the formalization of the type system is not already available. 5 Typing an OO language
In this section, following the patterns described in Sec- 5.1 FJ in Xtext tion 2, we implement the type system of Featherweight Java (FJ) [3, 15], a lightweight functional version of Java, An example of an FJ program, borrowed from [3] is shown which focuses on the main basic features of an Object- in Figure 17. Note that in FJ a method’s body consists of Oriented language. FJ has been proposed to formally study a single return statement (since FJ is a functional version the properties of the Java type system and to design Java of Java where block of statements are not supported). The extensions. It is not meant to be used to implement fully- receiver in a member selection expression must be always featured Java programs. However, it will allow us to con- specified; super is not considered in FJ. In FJ the class con- structor has a fixed shape, thus, for simplicity, we consider 66 Ë Lorenzo Bettini
FJProgram: (classes += FJClass)* (main = FJExpression)? ; judgments { inferType |− FJExpression expression : output FJClass FJClass: error "cannot type " + expression ’class’ name=ID (’extends’ superclass=[FJClass])? ’{’ source expression (members += FJMember)* subtype |− FJClass left <: FJClass right ’}’ ; error left + " is not a subtype of " + right subtypesequence |− FJMember: FJField | FJMethod; List
FJMethod: In order to make the example more interesting for the access=FJAccessLevel? type=[FJClass] name=ID goals of the paper, we added to FJ the standard access-level ’(’ (params+=FJParameter modifiers for fields and methods. Since there is no“pack- (’,’ params+=FJParameter)*)? ’)’ ’{’ age” concept in FJ the default level is always private. ’return’ expression=FJExpression ’;’ ’}’ ;
FJParameter: type=[FJClass] name=ID ; 5.2 Typing FJ FJTypedElement: FJMember | FJParameter ; The implementation of the type system of FJ in Xsemantics FJExpression: FJTerminalExpression =>( shown here is tailored to the goals of the paper itself, thus, {FJMemberSelection.receiver=current} ’.’ it is quite different from the one presented in [4]. member=[FJMember] We will provide a brief description of the syntax of Xse- (methodinvocation?=’(’ (args+=FJExpression (’,’ args+=FJExpression)*)? ’)’)? mantics in order to make the examples shown in the pa- )* ; per understandable. We refer the interested reader to [4]
FJTerminalExpression returns FJExpression: for more details on Xsemantics. { FJThis } ’this’ | Xsemantics uses a syntax that resembles rules in a {FJParamRef} parameter=[FJParameter] | formal setting [15–17]. A system definition in Xsemantics {FJNew} ’new’ type=[FJClass] ’(’ (args+=FJExpression (’,’ args+=FJExpression)*)? ’)’ | is a set of judgments, i.e., assertions about some proper- {FJCast} ’(’ type=[FJClass] ’)’ expression=FJExpression | ties of programs, and a set of rules, which assert the va- ’(’ FJExpression ’)’ ; lidity of certain judgments, possibly on the basis of other judgments. Rules have a conclusion and a set of premises. Figure 18: The Xtext grammar of FJ. The preamble of the grammar is Rules act on the EMF objects representing the AST of the not shown. program. The Xsemantics compiler will then generate Java code that can be used in the DSL implemented in Xtext for the constructors as implicit. For this reason, a new expres- scoping, validation and other mechanisms. sion must pass an argument for each field in the class, including inherited fields (arguments for inherited fields must come first). The class Object is not implicitly defined 5.2.1 Judgments in this version of FJ. A type in FJ is a reference to an FJ class definition, thus there are no basic types. After class defini- The judgments in Xsemantics for FJ are shown in Figure 19. tions, an FJ program can contain an expression that repre- In general, the developers can choose the symbols that sents the main expression. they see fit. We chose symbols such as |-, : and <: because The Xtext grammar for FJ is shown in Figure 18. Note these are the same symbols that are used in FJ formaliza- that the grammar rule for member selection handles both tion [3] and are typical of formal systems [15–17]. In fact, field selection and method invocation. A method invoca- in a typical formal type system, you find judgments of the tion differs from a field selection for the opening parenthe- shape Γ ⊢ e : T, to be read as “in the environment Γ, e sis. The presence of parentheses is stored in the boolean has type T” and Γ ⊢ T1 <: T2, to be read as “in the envi- feature methodinvocation. ronment Γ, T1 is a subtype of T2”. We will get back to the concept of “environment” later in this section. Type errors for the IDE with Xtext and Xsemantics Ë 67
The judgment inferType takes an FJExpression as rule ClassSubtyping input parameter and provides an FJClass as output pa- G |− FJClass left <: FJClass right rameter. The judgment subtype does not have output pa- from { left == right rameters (thus its output result is implicitly boolean, stat- or ing whether the judgment succeeded). superclasses(left ). contains(right ) Judgment definitions can include error specifications } that are useful for generating custom error information. An rule SubtypeSequence error specification, besides the error message, specifies the G |− List
axiom TThis checkrule CheckMethodBody for G |− FJThis t : env(G, ’this’, FJClass) FJMethod method error "’this’ cannot be used in the current context" from { source t // pass an environment for " this " ’this’ <− method.eContainer |− rule TCast method.expression : var FJClass bodyType G |− FJCast cast : cast.type from { empty |− bodyType <: method.type G |− cast.expression : var FJClass expType or fail G |− cast.type <: expType error "Type mismatch: cannot convert from " + or bodyType.name + " to " + method.type.name G |− expType <: cast.type source method.expression } }
rule TNew checkrule CheckMain for G |− FJNew newExp : newExp.type FJProgram program from { from { var fields = fields (newExp.type) program.main === null // nothing to check G |− newExp.args << fields or } empty |− program.main : var FJClass mainType }
Figure 21: Some typing rules of the judgment inferType. Figure 22: Two checkrules for FJ. ments will have the values assigned in the invoked rule. This is what happens in the loop of the second rule: the corresponding error will be issued. Note that Xsemantics type of each expression is computed using the judgment allows the developer to specify an error also at the single inferType and then the subtyping is checked using the rule/axiom level, as in the axiom TThis. The rule for typ- judgment subtype. ing cast expressions can be read as: “the type of a cast ex- If one of the premises fails, then the whole rule will pression is the type we cast to, provided that, if the type of fail, and the whole stack of rule invocations will fail as the casted expression is expType, then either the type we well. In particular, if the premise is a boolean expression, cast to is a subtype of expType or the other way round”. it will fail if it evaluates to false. If the premise is a rule This corresponds to the fact that in Java the two types in- invocation, it will fail if the invoked rule fails. volved in a cast must be related in the class hierarchy. The As shown in the rule SubtypeSequence, Xsemantics rule for FJNew uses another auxiliary function (not shown allows the developer to use an explicit failure statement here) that computes the list of all the fields in the class hi- in an or branch in order to provide errors that could be erarchy. We also use the subtypesequence judgment for more useful than the default ones. In the above rule, with- checking that the arguments of the FJNew expression are out the explicit fail with a detailed error, in case of lists conformant to the types of the fields. The other typing rules of different size, the generic failure message “failed: follow the same pattern, especially the one for method in- expressions.size == typedElements.size” would be vocation, which uses subtypesequence similarly to the reported, which contains too many internal details to be rule for FJNew, and we omit them here. effectively useful. In Figure 21 we show other typing rules, belonging to the judgment inferType. The implementation of these 5.2.3 Checkrules rules is based on the rules formalized in [3]. The first one is the axiom for typing this. For typing this we access the Xsemantics provides some special rules, checkrules, environment with the predefined function env, by specify- which do not belong to any judgment. A checkrule has ing the key and the expected Java type of the correspond- a single parameter, which is the AST node to be checked ing value. If no key is found in the environment or if the by the validator, and the premises (but no rule envi- value cannot be assigned to the specified Java type then ronment). Xsemantics generates a Java validator with a the rule will fail. This axiom assumes that the passed en- @Check method (Section 3.3) for each checkrule. vironment contains a mapping for this. We will see later In Figure 22 we show two checkrules: the one for when such a mapping is passed. Thus, if this is used out- checking that the body of a method conforms to the de- side a method’s body, its type will not be computed and a clared return type and the one for checking that the main expression can be typed in an empty environment. In the Type errors for the IDE with Xtext and Xsemantics Ë 69
class FJScopeProvider extends AbstractFJScopeProvider {
@Inject FJTypeSystem typeSystem
override getScope(EObject context, EReference reference) { switch (context) { FJMemberSelection: { val receiverType = typeSystem.inferType (environmentForThis(context), context.receiver ) if (receiverType !== null ) Scopes.scopeFor( typeSystem.fields(receiverType) + typeSystem.methods(receiverType) ) else Figure 23: Useful error message placed on the correct part. IScope.NULLSCOPE } default: super.getScope(context, reference) } }
def private environmentForThis(EObject context) { val env = new RuleEnvironment val containingClass = getContainerOfType(context, FJClass) if (containingClass !== null ) env.add("this", containingClass) return env } }
Figure 25: Scoping implementation for FJ member selection.
Figure 24: The error is not useful and it spans too much contents in ber selection expression, since we first have to compute the the program. type of the receiver expression and then inspect the hier- archy of that type. Note that the receiver expression might first rule we explicitly pass a mapping for this in the en- be in turn another member selection expression. Thus, in vironment for computing the type of the method body, the implementation of the scope provider for FJ we use so that occurrences of this can be successfully typed the Java code generated by Xsemantics from the type sys- with the axiom shown in Figure 21. In particular, this is tem specification we sketched in the previous section. Note mapped to the container of the method, i.e., the contain- that Xsemantics generates a Java method for each Xseman- ing FJClass. Then, we verify that such a type is a subtype tics judgment; such methods accept also an explicit en- of the declared return type. Note that we explicitly gen- vironment (further details about Xsemantics code gener- erate an error in case of failure with a useful description ation can be found in [4]). and with the source of the error, i.e., the node in the AST The implementation of the scope provider is shown that will be marked with the error. The result in Eclipse is in Figure 25. Xtext calls the method getScope passing shown in Figure 23. Had we not made the error explicit, the the context expression and the reference that must be re- default generated error would not be useful enough and solved. In our case, we are only interested in the case when it would be placed on the whole method declaration, as the context is an FJMemberSelection and the reference to shown in Figure 24. Other checkrules, e.g., checking the be resolved is a reference to an FJMember. We have to com- class hierarchy is not cyclic, that method overrides is cor- pute the type of the receiver expression, providing an en- rect, etc., are not shown in the paper. vironment with a mapping for this; this is constructed by retrieving the container of type FJClass (using the stan- dard EMF utility API, getContainerOfType). Remember 5.3 Scoping for member selection that in case this was used in the “main” expression of an FJ expression, such a container would be null and occur- Implementing cross reference resolution in an OO lan- rences of this could not be typed. If the receiver expres- guage is strictly related to typing when it concerns a mem- 70 Ë Lorenzo Bettini
override getScope(EObject context, EReference reference) { // ... as before if (receiverType !== null ){ if (context.methodinvocation) Scopes.scopeFor( typeSystem.methods(receiverType) ) else Scopes.scopeFor( typeSystem.fields(receiverType) ) } // ... as before }
Figure 26: Alternative scoping implementation for FJ member selec- tion, which is too strict.
sion can be given a type, the scope will consist of all the Figure 27: Too generic errors about member selection and cascad- fields and methods, including the inherited ones, of such ing error on the method body due to unresolved member. a type (computed through the auxiliary functions fields and methods, respectively). erence” errors, another error is issued on the body of the Further details about the implementation of this scope method: since the field f is not resolved, the body of the provider are described in the next section, where we ad- method cannot be given a type and the check for confor- dress crucial issues concerning useful error reporting. mance w.r.t. the method return type fails as well. As described in Section 2, we should not mix visibility 5.4 Scoping vs. validity and validity. We then should be less strict on the validity of the scope, as we did in the original scope provider imple- mentation of Figure 25. This way, even if a member is used In the scope provider implementation of Figure 25 we in the wrong way, the type system can still type the rest never check whether the current member selection must of the program. We then implement a separate check rule, refer to a field or to a method. As we said inSec- where we check whether the (already resolved) member is tion 5.1, an FJMemberSelection deals with both field se- valid in the member selection expression. This allows us lection and method invocation, using the boolean feature to make the error clear: the member can be referred, since methodinvocation to distinguish them. it is visible in the current context, but it is being used in the It might be tempting to filter the returned scope ac- wrong way. The check rule is shown in Figure 28. Note that cording to the kind of member selection, so that in a Xsemantics error specification also allows the developer to method invocation expression fields cannot be referred at specify the part of the node to mark with the error (mim- all and the other way round for field selection. For exam- icking the arguments of the standard Xtext validator error ple, an alternative implementation is shown in Figure 26, method that we used in Figure 8). The feature is specified where the returned scope takes into consideration whether using the constants in the AST classes, automatically gen- the member selection must refer to a field or to a method. erated by Xtext starting from the grammar definition. This implementation is certainly sound, but it has a The result can be seen in Figure 29. Note how the errors few drawbacks. First of all, if the user selects a field and are much clearer than the ones in Figure 27, not to mention uses it as a method (in a method invocation expression) that cascading errors are also avoided (indeed, the mem- the resulting error will be a generic “Couldn’t resolve refer- bers are correctly resolved and linked—see the occurrences ence to...” (similarly for the case when a method is used of m correctly marked in the editor). as a field). Instead, if a member is used in a member se- Mixing visibility and validity in an OO language typ- lection expression in the wrong way, but it is still a visible ically leads to mixing visibility and accessibility. An ex- member of the receiver’s class, we should generate a more ample of an accessibility error is a subclass trying to ac- useful error message. Moreover, since the member cannot cess a private field in a superclass. The private field is“vis- be resolved, other parts of the type system will fail as well, ible” in that context, but it is not “accessible”. In fact, losing the error recovery property. An example is shown private, protected, etc., are called access level modifiers in Figure 27, where, besides the generic “unresolved ref- Type errors for the IDE with Xtext and Xsemantics Ë 71
checkrule CheckMemberSelection for auxiliary isAccessible(FJMember member, EObject context) { FJMemberSelection sel val receiverClass = getContainerOfType(context, FJClass) from { val memberClass = getContainerOfType(member, FJClass) val member = sel.member return member.access == FJAccessLevel.PUBLIC || if (member instanceof FJField && sel.methodinvocation) { receiverClass === memberClass || fail { error "Method invocation on a field" empty |− receiverClass <: memberClass source sel member.access != FJAccessLevel.PRIVATE feature FJ_MEMBER_SELECTION__MEMBER } } else if ... // similar for the other way round } } checkrule CheckAccessibility for FJMemberSelection sel Figure 28: Checking member selection. from { val member = sel.member
isAccessible(member, sel.receiver) or fail error "The " + member.access + " member " + member.name + " is not accessible here" source sel feature FJ_MEMBER_SELECTION__MEMBER }
Figure 30: Checking member accessibility.
Figure 29: Errors concerning wrong member selections. in Java [21], not visibility modifiers. The Java compiler is- sues an informative error of the shape “the field has pri- vate access” (accessibility error), instead of a generic “the symbol cannot be found” (visibility error). The Eclipse JDT editor follows the same strategy. Moreover, the Java editor still allows the user to navigate to the definition of a mem- ber even if it is not accessible in that part of the program. For the above reason, we did not implement any filter- ing based on the access level in our scope provider imple- mentation (Figure 25) and we have a specific checkrule Figure 31: Errors due to invalid member access. implementing this check, shown in Figure 30. The check rule relies on the auxiliary function isAccessible. The not accessible in that program context. We do that by using reason why we have a dedicated auxiliary function will be the auxiliary function isAccessible defined in the FJ type clear in the following. system (Figure 30). The customization of the content assist Keeping visibility and validity separate allows us to is shown in Figure 32, in a simplified form, not showing provide useful errors, without preventing cross reference internal details of Xtext. resolution, as shown in Figure 31. The customized content assist in action is shown in As done in the previous DSL, we customize the content Figure 33: the private members of the superclass are not assist in order to filter out completions that would make shown as completions since in that method they are not the program invalid. For FJ, the proposals for a member accessible, while public and protected members are pro- selection expression must not include members that are posed. 72 Ë Lorenzo Bettini
resolved members. Moreover, since the type system does class FJProposalProvider extends AbstractFJProposalProvider { not recover from errors, possible actual errors in the sub- sequent expressions are not detected at all. For example, @Inject FJTypeSystem typeSystem the problem with the invocation of the method m is that we override completeFJExpression_Member(EObject model, ...) { do not pass any argument, while one is expected. This er- lookupCrossReference(...) [ ror is not detected, since m cannot be resolved at all. exp | typeSystem.isAccessible(exp as FJMember, model) ] } }
Figure 32: The content assist for FJ filters members that would not be accessible in the current context.
Figure 33: Filtered proposals.
Figure 34: Too many cascading errors. It is worth mentioning that the default implementa- tion of the Xtext proposal provider simply uses the scope provider. Thus, if we implemented filtering at the scope provider level we would not need to customize the pro- axiom posal provider at all. On the contrary, since we do not fil- TNew G |− FJNew newExp : newExp.type ter the elements in our scope provider implementation we must customize the content assist by filtering out invalid axiom TCast G |− FJCast cast : cast.type proposals. This is the price to pay if we want to achieve a better DSL user experience, as we saw in this section. checkrule CheckNew for FJNew newExp from { // ... type checking moved here } checkrule CheckCast for FJCast cast from { 5.5 Type computation vs. type checking // ... type checking moved here } As anticipated in Section 2, formal type systems typically Figure 35: deal with type computation and type checking at the same Improved type system implementation separating type computation from type checking. time. Since the typing rules for FJ that we showed in Sec- tion 5.2 aimed at mimicking the formal type system pre- sented in [3], type computation and type checking are In this subsection, we improve the implementation of mixed together in our implementation. For example, con- the type system by separating type computation from type sidering the typing rule TNew in Figure 21, the type system checking. We turn type computation rules into axioms, will not compute the type of a FJNew expression if the argu- since the type can be computed without checking subex- ments passed to the constructor are not valid. Member ref- pressions. Then, for each FJ expression we write an addi- erences on the FJNew expression will then be unresolved. tional checkrule: the premises of the rules of the original An example is shown in Figure 34. In the figure, the er- implementation are moved into the new checkrules. Some rors are not helpful, especially the ones related to the un- modifications are sketched in Figure 35. Type errors for the IDE with Xtext and Xsemantics Ë 73
tests. Further benchmarks on the importance of caching have already been shown in [4, 22]. An Xsemantics specification can enable caching at several levels (e.g., judgments, auxiliary functions, etc.) so that Xsemantics generates Java code that automatically uses the Xtext caching. The implementation of FJ (see the source code of the examples) uses the caching mecha- nisms provided by Xsemantics. Note that it is up to the developer to enable caching only on specific judgements. Indeed, caching should be enabled with care, otherwise it could decrease the perfor- mance. In fact, the caching is based on the Java hashing features, thus it makes sense only when used with actual object instances, not with references. Indeed, in the AST of a program there might be many different references to the same object. In such a scenario references as cache keys will only lead to many cache misses. Figure 36: Only useful errors are reported. Xsemantics automatically generates errors through the standard Xtext error mechanisms described in Sec- tion 3.3 by using the error information generated during With this modified type system, only the useful errors the application of the rules and checkrules. By default, are reported, and the scope provider can resolve member Xsemantics will use the inner most error information re- references even in the presence of errors, as shown in Fig- ferring to a node in the AST. This is usually the interest- ure 36. Indeed, and can be resolved in spite of the errors m n ing error. However, this behavior can be customized by the in the receiver expressions. Moreover, the error concerning developer, for example, in order to show all the rules that invoking without arguments is now detected. m failed. This can be seen in the “Lambda DSL” example that is part of Xsemantics distribution; in that example DSL we implement the type system with unification for a simple 6 Other improvements λ-calculus (also briefly described in [4]), and when a term cannot be typed it is useful to show the whole trace of ap- As we said in Section 2, caching type computations can plied rules that made the unification fail. increase the performance of the type system implementa- Thus, Xsemantics internally keeps track of the trace of tion. The type system is used by many Eclipse parts, for the applied rules when invoking a judgment implemented continuously checking the program while the user is edit- in Xsemantics. This trace could be useful also for debug- ing it, and for populating several views. Caching would ging purposes or for testing the type system itself (in [4] then keep the IDE responsive. In particular, typing rules an example of use of the trace for proving formal proper- might be used many times for computing the type of a ties is also shown). Most of all, as we saw above, it con- given expression. Some subtyping checks and some aux- tains all the information that leads to a possible failure in iliary functions are also used more than once from within the type system. Besides the customization of the strategy the type system implementation. In the FJ example the for generating errors, the trace of failures is also directly subtyping relation between the same two classes can be available in the Xsemantics specification itself, for creat- checked by many checkrules during the validation. ing useful error information in the rules and checkrules. In order to avoid the typical problems when dealing The access is provided by the implicit variable previous- with caching, that is, keeping track of changes that should Failure. This is automatically handled by Xsemantics at invalidate the cached values, Xtext provides some caching run-time: in case of a rule failure, this variable contains all mechanisms that automatically discard the cached values the problems that took place when applying the rule. as soon as the contents of the program changes. An example of this can be seen in the checkrule for a The sources of the “Expressions DSL” make use of method invocation expression, show in Figure 37, which such caching mechanisms and a few benchmarks about is the one generating the error on the invocation of m in the impact of caching are available in the example’s unit Figure 36. When checking whether the passed arguments respect the method parameters, in case of a failure the 74 Ë Lorenzo Bettini
guage. The crucial problem with such DSLs is that possi- checkrule CheckSelection for FJMemberSelection selection ble type errors usually leak details of the DSL implemen- from { tation and are less informative for the user [30]. For this // check message if it ’s a method call val member = selection.member reason, several mechanisms have been proposed for func- if (member instanceof FJMethod) { tional languages to customize the type errors generated ’this’ − < member.eContainer by the compiler [31–37], in order to make such errors ex- |− selection.args << member.params or fail pressed in terms of the domain. error previousFailure.message For an insightful and more complete study on the cur- source selection feature FJ_MEMBER_SELECTION__MEMBER rent research status of support for debugging type errors } we refer the interested reader to [38]. } The context of this paper is slightly different: we focus on imperative external DSLs with a limited form of type in- Figure 37: Checking method invocation with involved error han- ference. Thus, the above mentioned problems with type er- dling. ror reporting are less prominent. In fact, the domain is part of the DSL, and the type system can already provide type whole expression would be marked with error, that is, the error messages that include domain specific information. receiver and the invoked method. This would be distract- However, in a General Purpose Language, improving type ing for the user. For this reason, in the rule of Figure 37, we error messages, and allowing the developer to customize create an explicit error by specifying the part of the expres- them for specific contexts, is still useful. For example, C++ sion to be marked with error (with the feature specifica- and Java can sometime generate obscure error messages tion) but we reuse the original error message, retrieved by (see, e.g., [39, 40], respectively). means of previousFailure. Thus, we plan to investigate if and how some of the This mechanism is useful when implementing a com- approaches presented in the above cited papers could be plex type system, where errors have to be really informa- merged into the patterns presented in this paper. In partic- tive to help the user fix type errors. An example is shown ular, we plan to study how the implementation patterns in [22], where error messages in the presence of Java-like proposed in the paper can be applied to type inference generics are created by inspecting errors in the trace of ap- systems based on unification like the Hindley-Milner type plied rules by means of previousFailure. systems [41], where, as mentioned above, it is challenging to generate informative error messages. As stated in Sec- tion 6, we have an implementation of the type system with 7 Related work unification for a simple λ-calculus (also briefly described in [4]) and when a term cannot be typed we show the whole trace of applied rules that made the unification fail. Thus, In this section we discuss some related work, concern- the error information shown to the user try to cover several ing type errors, language workbenches and type systems program positions that contribute to the type error (along frameworks. the lines of other implementations [25, 42, 43]). However, this strategy is not optimal, since error messages are not 7.1 Type errors filtered, and they could overwhelm the user with internal details of the unification algorithm. Since in this paper we target the IDE, it also becomes Type inference in a language relieves programmers from interesting to investigate on possible mechanisms, espe- the burden of explicitly specifying types of declared el- cially applied to Xsemantics, to automatically generate ements. However, when a type cannot be inferred, error suggestions to the user in order to fix a type error. With messages might become obscure, making it hard to fix that respect, existing works would be used as inspiration, the problem in the program. This is especially true for like, e.g., [26, 44–47]. In particular, such approaches use functional languages with almost full type inference, like heuristics for detecting type errors and helping debugging Haskell and OCaml (see, for example, the papers [23–28], type errors. Other approaches, like [48], also use machine just to mention a few). learning to provide useful type error messages. We plan to In particular, in functional languages, it is common to investigate in that direction, even because machine learn- implement embedded DSLs [29], that is, DSLs developed ing is employed in other parts of IDE mechanisms, like the as a particular form of API in the host general purpose lan- content assist [49] and the code formatter [50]. Type errors for the IDE with Xtext and Xsemantics Ë 75
7.2 Language workbenches mentation and it aims at being complementary to Xtext concerning the type system implementation. Moreover, we Concerning the implementation framework, we chose are currently working on extending Xsemantics with scop- Xtext since it is the de-facto standard framework for im- ing rule definitions in order to have also the Xtext scope plementing DSLs in the Eclipse ecosystem, it is supported provider automatically generated. with a wide community and it also has many applications Xtext only provides single inheritance mechanisms for in the industry. As shown in Section 8, Xtext targets other grammars, so different grammars can be composed only development environment besides Eclipse. linearly. The same holds for Xsemantics system extension, There are many tools for implementing DSLs and IDE illustrated in [4]. These extensibility and compositionality tooling and we refer to [51–53] for a wider comparison. features are not as powerful as frameworks like [7, 13, 61], Tools like IMP (The IDE Meta-Tooling Platform) [54] only where language specifications are fully modular. deal with IDE features. TCS (Textual Concrete Syntax) [55] Since Xsemantics targets mainly Xtext language im- is similar to Xtext, but TCS assumes that a metamodel for plementations, the work that is closest to Xsemantics is the AST is already implemented and requires the devel- XTS [62], which shares with Xsemantics the main goals. oper to associate syntactical elements to metamodel ele- However, XTS aims at expression based languages, not at ments. On the contrary, Xtext only requires the grammar general purpose languages, and it would not be straight- specification and derives automatically the metamodel forward to write the type system for FJ in XTS. Moreover, from such a specification. EMFText [56] is also similar to XTS targets type systems only, while Xsemantics can be Xtext, but, similarly to TCS, it requires the language to be used also for writing reduction rules (e.g., for compilers implemented to be defined in an abstract way using an and interpreters) as sketched in [4]. EMF metamodel. Systems like Silver [63], JastAdd [13] (and its integra- Other language workbenches with type system sup- tion with EMF,JastEMF [64]) and LISA [65], allow the devel- port are described in the next subsection. oper to specify type systems using attribute grammars [66], that is, associating attributes with AST elements. These attributes can capture arbitrary data about the element 7.3 Type systems frameworks including its type. Xsemantics shares something with at- tribute grammars: it associates a type attribute with pro- MPS (Meta Programming System) [57] is another tool for gram elements. Since Xsemantics aims at defining the se- developing a DSL. MPS targets projectional editing for mantics of a language, it provides a more concise nota- building DSL editors with tables and diagrams. MPS has tion and a more specific error reporting mechanism. How- its own DSL for specifying the type system. The developer ever, in contrast to general-purpose attribute grammar sys- specifies type system equations in this DSL and a solver tems, Xsemantics only aims at rules for semantics specifi- tries to solve all the equations relevant to a given program cation. Moreover, since Xsemantics specifications are not in order to compute and check types. Spoofax [7] is an- part of the grammar specifications, Xsemantics type sys- other language workbench that targets Eclipse and it relies tems could be “injected” in existing implementations, or on Stratego [58] for rule-based specifications to analyze different implementations of type systems can be enabled the programs. In [18], Spoofax is extended with a collec- in a DSL. Indeed, Xsemantics is not bound to an Xtext tion of declarative meta-languages in order to create a lan- grammar: it refers to Java types representing nodes of the guage designer’s workbench that supports all the aspects AST. of language implementation including verification infras- In fact, Xsemantics might also be used to validate tructure and interpreters: NaBL [14] for name binding and any model, independently from Xtext itself, and possibly scope rules, TS for the type system and DynSem [19] for the be used also with other language frameworks like EMF- operational semantics. More recently, the NaBL2 language Text [56]. This is an advantage with respect to other ap- has been introduced, which is based on the constraint lan- proaches (e.g., [57, 61, 67–71]), which instead require the guage described in [59]. The approach presented in [59] programmer to use the framework also for defining the has been further generalized to structural and generic syntax of the language. types in [60]: Statix is introduced, a domain-specific meta- Xsemantics is the successor of Xtypes [72] and it pro- language for the specification of static semantics, based vides a much richer syntax for rules, thanks to Xbase. on scope graphs and constraints. With that respect, Xse- Moreover, Xtypes targets type systems only, while Xse- mantics shares with the mentioned systems the goal of re- mantics deals with any kind of rules. The very first pro- ducing the gap between the formalization and the imple- totype of Xsemantics was introduced in [73], where it 76 Ë Lorenzo Bettini was used, together with other frameworks, in an analysis of several approaches for implementing type systems for Xtext DSLs. Then, in [74], the first release of Xsemantics was presented. At that time, Xsemantics was still a proof of concept for studying prototype language implementa- tions starting from existing language formalizations. Since then, Xsemantics has started to be adopted also in in- dustry [75]. Moreover, it has been used to implement a typed Javascript dialect N4JS [22, 76], that is, a super set of JavaScript with modules and classes with a static type system on top of it, which combines the type systems pro- vided by Java, TypeScript and Dart. N4JS, besides primi- tive types, declared types such as classes, interfaces and roles, supports union types [77], generic types and generic Figure 38: The web editor of the Expressions DSL, generated by methods, including wildcards, requiring the notion of ex- Xtext. istential types [78]. Thus its type system implementation represents a nice case study of the usability of Xseman- lowing a formal specification, like we did for FJ (Section 5), tics for real world languages. The adoption of Xseman- and then concentrate on the improvements applying the tics in the industry required to rewrite many of its parts patterns presented in the paper. In both cases, we believe so that complex type systems could be effectively and ef- that a good test suite, with good code coverage, is really ficiently implemented [4]. This also led Xsemantics tobe- crucial. This is what we actually did for the actual im- come part of the Eclipse eco-system as an official Eclipse plementations (https://github.com/LorenzoBettini/xtext- project https://github.com/eclipse/xsemantics. type-errors-examples). Finally, we would like to stress that thanks to the com- Statically computed types are used also to enrich sev- plete integration of Xsemantics with Java, the developers eral parts of the IDE. The most important one is the con- are not forced to implement all the aspects of the seman- tent assist that relies on the static types to filter propos- tics of a language with Xsemantics. Some specific tasks can als, avoiding completions that would make the program be implemented directly in Java. Xsemantics and Java code invalid. The fact that the type system enjoys error recovery can then co-exist and from Xsemantics one can delegate to is crucial for implementing a useful content assist, since Java code (as done, for example, in the implementation of when the content assist is invoked, the program will surely N4JS). be incomplete (the user is still writing the program). Other parts of the IDE can benefit from computed types, aswe have shown throughout the paper, like Outline views, Hov- 8 Conclusions ering pop-ups and code mining. Although the main target of Xtext is Eclipse, Xtext tar- In this paper we presented a few general patterns to im- gets also other platforms. For example, a web editor for plement a type system that is able to type as many parts of the DSL can be automatically generated by Xtext reusing the program as possible, detecting only the most impor- the implementation of the DSL. Figure 38 shows the web tant type errors and avoiding cascading errors. We then editor generated by Xtext for the Expressions DSL, reusing presented two case studies related to the implementation all the mechanisms we implemented for the DSL (compare of type systems, using such patterns, aiming at mean- this figure with Figure 9). ingful error reporting and useful IDE features. We also The importance of IDE support for languages also re- showed how the type system can be used for resolving cently led to the creation of the Language Server Pro- cross references. We believe that the type error messages tocol (LSP)⁶, that is, a protocol to be used between an produced by such implementations have the typical prop- editor or IDE and a language server that provides the erties found in the literature (see, e.g., [40, 79]). typical IDE features. The idea is to avoid repeating the One could start using such patterns from the very be- effort to implement IDE mechanisms targeting different ginning of the language implementation, like we did for the Expressions DSL (Section 4). Alternatively, one could start implementing a first prototype of the type system fol- 6 https://microsoft.github.io/language-server-protocol/ Type errors for the IDE with Xtext and Xsemantics Ë 77
[4] Bettini L., Implementing type systems for the IDE with Xseman- tics, Journal of Logical and Algebraic Methods in Programming, 2016, 85(5, Part 1), 655–680 [5] Aho A. V., Lam M. S., Sethi R., Ullman J. D. (Eds.), Compilers: principles, techniques, and tools, Addison Wesley, 2nd ed., 2007 [6] Haase A., Völter M., Efftinge S., Kolb B., Introduction to open Ar- chitecture Ware 4.1.2, In: MDD Tool Implementers Forum, 2007 [7] Kats L. C. L., Visser E., The Spoofax language workbench, Rules for declarative specification of languages and IDEs, In: OOPSLA, Figure 39: The Visual Studio Code editor of the Expressions DSL ACM, 2010, 444–463 using LSP support by Xtext. [8] Hemel Z., Groenewegen D. M., Kats L. C. L., Visser E., Static con- sistency checking of Web applications with WebDSL, Journal of IDEs. A Language Server provides the language-specific Symbolic Computation, 2011, 46(2), 150–182 features and communicates with the IDE over a proto- [9] Steinberg D., Budinsky F., Paternostro M., Merks E., EMF: Eclipse Modeling Framework, Addison-Wesley, 2nd ed., 2008 col that enables inter-process communication. The same [10] Prasanna D. R., Dependency Injection: Design patterns using Language Server can be re-used in multiple IDEs, which, Spring and Guice, Manning, 1st ed., 2009 in turn, can support multiple languages with minimal ef- [11] Reps T., Teitelbaum T., The synthesizer generator, In: Software fort. Eclipse and Xtext fully support LSP.Languages imple- Engineering Symposium on Practical Software Development En- mented with Xtext can automatically provide IDE mecha- vironments, ACM, 1984, 42–48 [12] Sewell P., Nardelli F. Z., Owens S., Peskine G., Ridge T., Sarkar nisms to Eclipse and to all the IDEs and editors that sup- S., Strnisa R., Ott: Effective tool support for the working seman- port LSP. Figure 39 shows the Expressions DSL editor in ticist, Journal of Functional Programming, 2010, 20(1), 71–122 Visual Studio Code, relying on the LSP support provided [13] Ekman T., Hedin G., The JastAdd system – modular extensi- by Xtext (compare this figure with Figure 13). ble compiler construction, Science of Computer Programming, While we focused on language implementation target- 2007, 69(1-3), 14–26 ing the IDE, Xtext can also automatically generate a com- [14] Konat G., Kats L., Wachsmuth G., Visser E., Declarative name binding and scope rules, In: SLE, LNCS, Springer, 2012, 7745, mand line compiler for the DSL, reusing all the imple- 311–331 mented mechanisms (apart from the ones related to the [15] Pierce B. C., Types and Programming Languages, Cambridge, UI). This is out of the scope of the paper. We only stress MA: The MIT Press, 2002 that the same useful error messages generated by the im- [16] Cardelli L., Type systems, ACM Computing Surveys, 1996, 28(1), plementation shown in the paper would be reported on the 263–264 [17] Hindley J. R., Basic Simple Type Theory, Cambridge University console. Press, 1987 In the implementations shown in the paper we relied [18] Visser E., Wachsmuth G., Tolmach A. P., Neron P., Vergu V. A., on Xtext and Xsemantics, exploiting their features for the Passalaqua A., Konat G., A language designer’s workbench: a IDE support and for mimicking formal systems, respec- one-stop-shop for implementation and verification of language tively. However, the patterns presented in Section 2 are designs, In: Onward!, ACM, 2014, 95–111 general and could be applied in other language implemen- [19] Vergu V. A., Neron P., Visser E., DynSem: A DSL for dynamic se- mantics specification, In: RTA, LIPIcs, Schloss Dagstuhl-Leibniz- tation frameworks. Zentrum fuer Informatik, 2015, 3, 365–378 [20] Efftinge S., Eysholdt M., Köhnlein J., Zarnekow S., Hasselbring Acknowledgements: I would like to thank the anonymous W., von Massow R., Xbase: implementing domain-specific lan- reviewers for their suggestions for improving the paper. guages for Java, In: GPCE, ACM, 2012, 112–121 [21] Gosling J., Joy B., Steele G., Bracha G., Buckley A., The Java lan- guage specification, Java SE 7 Edition, Addison-Wesley, 2013 [22] Bettini L., von Pilgrim J., Reiser M.-O., Implementing a typed References Java script and its IDE: a case-study with Xsemantics, Interna- tional Journal on Advances in Software, 2016, 9(3-4), 283–303 [23] Johnson G. F. , Walz J. A., A maximum-flow approach to anomaly [1] Fowler M., Language Workbenches: The killer-app for do- isolation in unification-based incremental type inference, In: main specific languages?, 2005, https://www.martinfowler. POPL, ACM, 1986, 44–57 com/articles/languageWorkbench.html [24] Wand M., Finding the source of type errors, In: POPL, ACM, 1986, [2] Bettini L., Implementing domain-specific languages with Xtext 38–43 and Xtend, Packt Publishing, 2nd ed., 2016 [25] Heeren B., Top Quality Type Error Messages, PhD thesis, Utrecht [3] Igarashi A., Pierce B., Wadler P., Featherweight Java: a mini- University, Netherlands, 2005 mal core calculus for Java and GJ, ACM TOPLAS, 2001, 23(3), [26] Lerner B. S., Flower M., Grossman D., Chambers C., Searching 396–450 for type-error messages, In: PLDI, ACM, 2007, 425–434 78 Ë Lorenzo Bettini
[27] Chambers C., Chen S., Le D., Scafldi C., The function, and dys- [51] Voelter M., Benz S., Dietrich C., Engelmann B., Helander M., function, of information sources in learning functional program- Kats L. C. L., Visser E., Wachsmuth G., DSL Engineering - Design- ming, Journal of Computing Sciences in Colleges, 2012, 28(1), ing, Implementing and Using Domain-Specific Languages, 2013 220–226 [52] Pfeiffer M., Pichler J., A comparison of tool support for textual [28] Tirronen V., Uusi-Mäkelä S., Isomöttönen V., Understanding be- domain-specific languages, In: OOPSLA Workshop on Domain ginners’ mistakes with Haskell, Journal of Functional Program- Specific Modeling, 2008, 1–7 ming, 2015, 25 [53] Erdweg S., van der Storm T., Völter M., Tratt L., Bosman R., Cook [29] Hudak P., Building domain-specific embedded languages, ACM W. R., et al., Evaluating and comparing language workbenches: Computing Surveys, 1996, 28(4es), 196 Existing results and benchmarks for the future, Computer Lan- [30] Hage J., DOMain Specific Type Error Diagnosis (DOMSTED), guages, Systems & Structures, 2015, 44(A) Tech. Rep. UU-CS-2014-019, Department of Information and [54] Charles P., Fuhrer R., Sutton Jr. S., Duesterwald E., Vinju J., Ac- Computing Sciences, Utrecht University, 2014 celerating the creation of customized, language-Specific IDEs in [31] Heeren B., Hage J., Swierstra S. D., Scripting the type inference Eclipse, In: OOPSLA, ACM, 2009, 191–206 process, In: ICFP, ACM, 2003, 3–13 [55] Jouault F., Bézivin J., Kurtev I., TCS: a DSL for the specification of [32] Stuckey P. J., Sulzmann M., Wazny J., Improving type error diag- textual concrete syntaxes in model engineering, In: GPCE, ACM, nosis, In: SIGPLAN Workshop on Haskell, ACM, 2004, 80–91 2006, 249–254 [33] Wazny J., Type inference and type error diagnosis for Hind- [56] Heidenreich F., Johannes J., Karol S., Seifert M., Wende C., ley/Milner with extensions, PhD thesis, University of Mel- Derivation and Refinement of Textual Syntax for Models, In: bourne, Australia, 2006 ECMDA-FA, LNCS, Springer, 2009, 5562, 114–129 [34] Plociniczak H., Miller H., Odersky M., Improving human- [57] Voelter M., Language and IDE modularization and composition compiler interaction through customizable type feedback, with MPS, In: GTTSE, LNCS, Springer, 2011, 7680, 383–430 Tech. Rep. 197948, EPFL, 2014 [58] Bravenboer M., Kalleberg K. T., Vermaas R., Visser E., Strate- [35] Charguéraud A., Improving type error messages in OCaml, In: go/XT 0.17. A language and toolset for program transformation, ML/OCaml, EPTCS, 2014, 198, 80–97 Science of Computer Programming, 2008, 72(1-2), 52–70 [36] Serrano A., Hage J., Type error diagnosis for embedded DSLs [59] Van Antwerpen H., Néron P., Tolmach A., Visser E., Wachsmuth by two-stage specialized type rules, In: ESOP, LNCS, Springer, G., A constraint language for static semantic analysis based on 2016, 9632, 672–698 scope graphs, In: PEPM, ACM, 2016, 49–60 [37] Serrano A., Hage J., Type error customization in GHC: Controlling [60] Van Antwerpen H., Bach Poulsen C., Rouvoet A., Visser E., expression-level type errors by type-level programming, In: IFL, Scopes as types, In: Proceedings of the ACM on Programming ACM, 2017, 2:1–2:15 Languages, 2018, 2(OOPSLA), 114:1–114:30 [38] Wu B., Chen S., How type errors were fixed and what students [61] Vacchi E., Cazzola W., Neverlang: A Framework for Feature- did?, In: OOPSLA, ACM, 2017, 1, 105:1–105:27 Oriented Language Development, Computer Languages, Sys- [39] Chen S., Erwig M., Early detection of type errors in C++ tem- tems & Structures, 2015, 43(3), 1–40 plates, In: PEPM, ACM, 2014, 133–144 [62] Völter M., Xtext/TS - a type system framework for [40] el Boustani N., Hage J., Improving type error messages for Xtext, 2011, http://code.google.com/a/eclipselabs.org/p/xtext- generic Java, Higher-Order and Symbolic Computation, 2011, typesystem/ 24(1), 3–39 [63] Wyk E. V., Bodin D., Gao J., Krishnan L., Silver: an extensible [41] Damas L., Milner R., Principal type schemes for functional pro- attribute grammar system, Science of Computer Programming, grams, In: POPL, ACM, 1982, 207–212 2010, 75(1), 39–54 [42] Stuckey P. J., Sulzmann M., Wazny J., Interactive type debug- [64] Bürger C., Karol S., Wende C., Aßmann U., Reference attribute ging in Haskell, In: SIGPLAN Workshop on Haskell, ACM, 2003, grammars for metamodel semantics, In: SLE, LNCS, Springer, 72–83 2011, 6563, 22–41 [43] Haack C., Wells J., Type error slicing in implicitly typed higher- [65] Mernik M., Lenic M., Avdicausevic E., Zumer V., LISA: an interac- order languages, Science of Computer Programming, 2004, tive environment for programming language development, In: 50(1), 189–224 Compiler Construction, LNCS, Springer, 2002, 2304, 1–4 [44] Hage J., Heeren B., Heuristics for type error discovery and recov- [66] Knuth D. E., Semantics of context-free languages, Mathematical ery, In: IFL, LNCS, Springer, 2007, 4449, 199–216 Systems Theory, 1968, 2(2), 127–145 [45] el Boustani N., Hage J., Corrective hints for type incorrect [67] Borras P., Clement D., Despeyroux T., Incerpi J., Kahn G., Lang generic Java programs, In: PEPM, ACM, 2010, 5–14 B., Pascual V., CENTAUR: the system, In: Software Engineering [46] Chen S., Erwig M., Counter-factual typing for debugging type er- Symposium on Practical Software Development Environments, rors, In: POPL, ACM, 2014, 583–594 SIGPLAN, ACM, 1988, 24, 14–24 [47] Zhang D., Myers A. C., Vytiniotis D., Jones S. L. P., Diagnosing [68] Van den Brand M., Heering J., Klint P., Olivier P. A., Compil- type errors with class, In: PLDI, ACM, 2015, 12–21 ing language definitions: the ASF+SDF compiler, ACM TOPLAS, [48] Wu B., Campora III J. P., Chen S., Learning user friendly type- 2002, 24(4), 334–368 error messages, In: OOPSLA, ACM, 2017, 1, 106:1–106:29 [69] Dijkstra A., Swierstra S. D., Ruler: programming type rules, In: [49] Bruch M., Monperrus M., Mezini M., Learning from examples to FLOPS, LNCS, Springer, 2006, 3945, 30–46 improve code completion systems, In: ESEC/SIGSOFT FSE, ACM, [70] Felleisen M., Findler R. B., Flatt M., Semantics Engineering with 2009, 213–222 PLT Redex, Cambridge, Mass.: The MIT Press, 2009 [50] Parr T., Vinju J. J., Towards a universal code formatter through [71] Xu H., EriLex: An Embedded Domain Specific Language Genera- machine learning, In: SLE, ACM, 2016, 137–151 tor, In: TOOLS, LNCS, Springer, 2010, 6141, 192–212 Type errors for the IDE with Xtext and Xsemantics Ë 79
[72] Bettini L., A DSL for writing type systems for Xtext languages, [76] Bettini L., von Pilgrim J., Reiser M.-O., Implementing the Type In: PPPJ, ACM, 2011, 31–40 System for a Typed Javascript and its IDE, In: COMPUTATION [73] Bettini L., Stoll D., Völter M., Colameo S., Approaches and tools TOOLS, IARIA, 2016, 6–11 for implementing type systems in Xtext, In: SLE, LNCS, Springer, [77] Igarashi A., Nagira H., Union types for object-oriented program- 2012, 7745, 392–412 ming, Journal of Object Technology, 2007, 6(2), 31–45 [74] Bettini L., Implementing Java-like languages in Xtext with Xse- [78] Cameron N., Ernst E., Drossopoulou S., Towards an Existential mantics, In: OOPS (SAC), ACM, 2013, 1559–1564 Types Model for Java Wildcards, In: Formal Techniques for Java- [75] Heiduk A., Skatulla S., From Spaghetti to Xsemantics - Prac- like Programs (FTfJP), 2007 tical experiences migrating type systems for 12 languages, [79] Yang J., Michaelson G., Trinder P., Wells J. B., Improved Type Er- XtextCon, 2015 ror Reporting, In: Workshop on Implementation of Functional Languages, Aachner Informatik-Berichte, 2000, 71–86