An Extensible Programrning Environment for Modula-3% Mick Jordan’ Abstract 2 Goals This paper describes the design and implementation of The long term objective was to build a practical envi- a practical programming environment for the Modula-3 ronment to support large scale software development in programming language. The environment is organised Modula-3. Initially, the environment would contain only a around an extensible intermediate representation of pro- basic set of well integrated tools such as a compiler, linker grams and makes extensive use of reusable components. and debugger, but would be capable of supporting real The environment is implemented in Modula-3 and exploits projects on a variety of systems. In the longer term ad- some of the novel features of the language. ditional tools such as code browsers, code analysers, and improvements like smart recompilation would be added, building on the basic environment. In this paper, we 1 Introduction are more concerned with how tools are constructed and combined than with the exact details of their functionality The successof a programming language owes much to its or interface to users of the environment. The idea is that, associatedprogramming environment, especially to the set of tools which enable the construction and modification of if the construction of new tools is made easy enough, programs. W5th a new language there is trade-off between new and powerful tools will appear routinely. Important producing an implementation quickly and in providing a sub-goals for the environment were as follows: rich and powerful set of tools. Choosing the first approach typically results in low quality, poorly integrated tools. The second approach can result in an unacceptable delay A Framework for Program Analysis and before the language can be used. This paper describes Wansformation the design and implementation of a toolset for M,odula-3 [G&39], a new programming language that adds threads, There is a large class of tools which engage in program objects, exceptions and garbage collection to Msodula-2 analysis or transformation, for example, browsers, com- mr83]. Our aim was to steer between the two extremes pilers, pretty-printers. These tools require access to an noted above and produce a basic set of tools designed intermediate representation of program source, and sup- around an open, extensible framework, to which new port mechanisms to facilitate the analysis. It is important tools could be added incrementally. not to prejudice the addition of such tools by choosing an overly restrictive framework. *This work was carried out at Olivetti Software Technolog:y Labora- tory, Menlo Park, CA 94025, USA. +Author’s current address: Digital Systems Research Center, 130 Lytton Avc, Palo Alto, CA 9430 1, USA Extensibility Permission to copy without fee all or part of this material is In order to make effective use of the framework, adding a granted provided that the copies are not made or distributed for new tool must be straightforward. Detailed knowledge of direct commercial advantage, the ACM copyright notic’s and the other tools shouIdnot be necessary. New tools should have title of the publication and its date appear, and notice is given equal status with the initial set. Note that the extensibility that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee is aimed at tool developers rather than at users of the and/or specific permission. environment, although the design does not preclude the @ 1990 ACM O-89791-41 8-X/90/001 2-0066...$1.50 latter as a subsequent enhancement. 66 Portability and Availability this paper. The DIANA reference manual [Goo83], which defines an AST for Ada1 POD83], contains an excellent The system must be portable to a wide range of systems analysis of the fundamental issues and has many useful and execute on stock hardware, typically 3 Mip worksta- recommendations for AST designers. Indeed the design tions with 8-16 Mb of main memory. This placed some of the Modula-3 AST (M3AST) was strongly influenced constraints on the system design and on the degree to by DIANA and several techniques and conventions were which we could incorporate external tools. adopted directly. Where M3AST differs from DIANA is in the explicit use of an object-type hierarchy to capture Implernent in Modula-3 commonality and in the separation of the spetication into pieces that correspond to individual tool fragments. The implementation of a programming environment ex- Also the notion of M3AST as the output from a com- tends from low-level (unsafe) programming through to piler front-end is de-emphasised in favour of support for application-level programming. This is precisely the a wider collection of tools. These techniques contribute area to which Modula-3 is targeted and we believe that to the readability and maintainability of the specification Modula-3 offers a better set of facilities for this area than and aid in supporting extensible tool development. To its competitors. Implementing the compiler for a (sys- illustrate the confusion that can arise with the monolithic tems) language in itself is also a good baseline test of an approach consider our previous experience with an AST implementation, and lends credibility to a new language. for Modula-2+. This AST was defined in a single iuter- face as a collection of variant record types. It was rather 3 Design complex, and it was not always clear when a given node or attribute was set, by which phase (or tool) and whether The two key features needed to achieve the goals are: an attribute was temporary or (usefully) persistent. This confusion did not facilitate extensibility, which demands l A well defined and extensible intermediate represen- some form of incremental&y in the specfication. tation of program text. l A system design that encourages the reuse of com- 4.2 AST Structure ponents. The AST is modeled as an attributed tree. Following The foremost requirement of the intermediate representa- the terminology used in DlANA, the tree contains named tion (IR) is that it be information preserving. That is, with nodes that correspond to non-terminals in an abstract the possible exception of lexical details, all information version of the Modula-3 grammar. Members of the right- present in the source text must be represented in the IR. hand side of the production rule for a non-terminal are The second requirement is that the IR be capable of record- denoted by typed attributes of the node. Non-terminals ing additional information, perhaps implicit in the source with alternative productions, for example, Expression, text, that is generated by other tools in the environment. are modeled as classes whose members are the associated In short the IR must be extensible. One such iR that sat- set of nodes. In other systems, for example the Synthesiser isfies these requirements is an attributed Abstract Syntax Generator [Rep89], classes are referred to as phlyu and Tree (AST), which is well established in the literature as the subtree rooted at a node is called a term. Attribute a sound basis for compiler development [Aho86, Bor88, types may be basic types such as INTEGER,user-specified Don84, Goo83, Rep89]. An extensible AST provides an abstract types, such as might be used to denote an identifier excellent integrationmechanism for the toolset. However, name, or a node or class type. The repetitive constructs for extensibility to really work well, new tool developers in grammar productions are represented by an attribute of must be able to compose existing tools in order to generate type sequence. The set of nodes, classes and attributes suitable AST instances for their task. They must also be corresponding to the language syntax form the backbone of provided with a rich set of packages or tool fragments that the extensible AST and comprise the lexical and syntactic can be invoked to simplify their task. views. A new view is created by adding extra attributes to existing nodes and perhaps by defining new node 4 The Modula-3 Abstract Syntax and class types. An example is the semantic view, where attributes to denote identifier bindings are introduced. The Tree views form a hierarchy that initially reflects the traditional sequence of phases in a compiler. An important kind 4.1 Background of view is one which contains nodes and classes that A detailed discussion of the merits of an AST as the inter- represent an embedded language with its own syntax and mediate representation for programs is beyond the scope of ‘Ada is a registered trademark of the U.S. Government 67 semantics, for example a specification language such as Pre-Linker This view supports the pre-linker tool, de- Larch [Gut85]. The ability to share a single formalism scribed in section 5.4, and detines attributes of a more and to establish interconnects between the two ASTs is global nature, for example the depends-on relation very useful. In the basic toolset there are the following for a module, which is used in determinin g Program AST views: initialisation order. Lexical This view is very simple, defining only the basic types needed to denote identifiers, numbers, com- ments etc. These are used as attribute types in the 4.3 Mapping the AST into a Programming syntactic view. Language Syntactic The syntactic view defines the backbone of the AST, which other views decorate with additional The above discussion treats the AST in an abstract manner, attributes. There is a very close relationship between but ultimately this abstraction must be mapped into some the syntactic view and the Modula-3 grammar. programming language. This can be achieved in many ways and involves several classic space/time/flexibility Pragmas The revised language provides a pragma mech- trade-offs.