Scala Macros: Let Our Powers Combine! On How Rich Syntax and Static Types Work with Metaprogramming Eugene Burmako EPFL, Switzerland eugene.burmako@epfl.ch ABSTRACT Compile-time metaprogramming has been proven immensely Compile-time metaprogramming can be thought of as the useful enabling programming techniques such as language algorithmic construction of programs at compile-time. It’s virtualization, embedding of external domain-specific lan- often used with the intent of allowing programmers to gen- guages, self-optimization, and boilerplate generation among erate parts of their programs rather than having to write many others. these program portions themselves. Thus, metaprograms In the recent production release of Scala 2.10 we have in- are programs who have a knowledge of other programs, and troduced macros, an experimental facility which gives its which can manipulate them. users compile-time metaprogramming powers. Alongside of Across languages and paradigms, this sort of metapro- the mainline release of Scala Macros, we have also intro- gramming has been proven immensely useful, acting as an duced other macro flavors, which provide their users with enabling force behind a number of programming techniques, different interfaces and capabilities for interacting with the such as: language virtualization (overloading/overriding se- Scala compiler. mantics of the original programming language) [12], em- In this paper, we show how the rich syntax and static bedding of external domain-specific languages (tight inte- types of Scala synergize with macros, through a number gration of external DSLs into the host language) [40, 48], of real case studies using our macros (some of which are self-optimization (self-application of optimizations based on production systems) such as language virtualization, type analysis of the program’s own code) [35], and boilerplate providers, materialization of type class instances, type-level generation (automatizing repetitive patterns which cannot programming, and embedding of external DSLs. We explore be readily abstracted away by underlying language) [36, 30]. how macros enable new and unique ways to use pre-existing In the recent production release of Scala 2.10 we have in- language features such as implicits, dynamics, annotations, troduced Scala Macros [6] as a new experimental language string interpolation and others, showing along the way how feature– Scala’s realization of compile-time metaprogram- these synergies open up new ways of dealing with software ming. This new feature enables the compiler to recognize development challenges. certain methods in Scala programs as metaprograms, or macros, which are then themselves invoked at certain points of compilation. When invoked, macros are provided with a Categories and Subject Descriptors compiler context, which exposes the compiler’s representa- D.3.3 [Programming Languages]: Language Constructs tion of the program being compiled along with an API pro- and Features viding certain compiler functionality such as parsing, type- checking and error reporting. Using the API available in the General Terms context, macros can influence compilation by, for example, changing the code being compiled or affecting type inference Languages performed by the typechecker. The most basic form of compile-time metaprogramming in Keywords our system is achieved by def macros, plain methods whose Compile-Time Metaprogramming, Type Classes, Domain- invocations are expanded during compilation. In addition to Specific Languages, Scala these def macros, we have identified, implemented, and ex- perimented with different macro flavors: dynamic macros, string interpolation macros, implicit macros, type macros, 1. INTRODUCTION and macro annotations. Each of these flavors encompasses some different way in which macros are presented to, and can be used by users. We will go on to explore a number Permission to make digital or hard copies of all or part of this work for of applications, which have proven markedly difficult or im- personal or classroom use is granted without fee provided that copies are possible to achieve via other means, each of which exercise not made or distributed for profit or commercial advantage and that copies one of these macro flavors. bear this notice and the full citation on the first page. To copy otherwise, to Our contributions are as follows: republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. • Scala ’13, Montpellier, France We describe a number of macro flavors, which are in- Copyright 2013 ACM 978-1-4503-2064-1 ...$15.00. tegrated in a principled way alongside of Scala’s rich syntax and strong static type system. Even in this simple form, the macro is arguably more use- ful than a corresponding function in an eager language such • We provide a comprehensive validation of the utility of as Scala, because it does not calculate the message unless the these macro flavors through a number of real case stud- asserted condition is violated. The necessity to shield the ies. We show that macros (a) enable language virtu- evaluation of the message for performance reasons usually alization, (b) can implement a form of type providers, produces noticeable amounts of boilerplate, which cannot (c) can be used to automatically generate type class be easily abstracted away. Scala does support lazy eval- instances, (d) simplify type-level programming, and uation with by-name parameters, but the inner workings (e) enable embedding of external domain-specific lan- of their internal representation might also degrade perfor- guages. We additionally go on to show that macros mance. Macros are able to address the performance problem can re-implement non-trivial language features such as without downsides. code lifting and materialization of type class instances. In addition to def macros, introduced in Scala 2.10, we The rest of the paper is organized as follows. Section 2 have created a fork of Scala [5], where we have conceived, im- provides a basic introduction to Scala macros. Section 3 in- plemented, and experimented with a number of other macro troduces the macro flavors we have experimented with, set- flavors– macros which provide different interfaces and capa- ting the stage for Section 4, which outlines some of the use bilities for interacting with the Scala compiler. cases that these flavors enable and discusses alternative ways of achieving similar functionality. Throughout the paper we 3. HAMMERS: THE MACRO FLAVORS deliberately avoid going into the details of our macro sys- Macros realize the notion of textual abstraction [17], which tem (expansion semantics, integration with the typechecker, consists of recognizing pieces of text that match a specifica- handling of hygiene, interplay between macros, etc) in or- tion and replacing them according to a procedure. In Lisp, der to focus specifically on how macros work together with the origin of macros, programs are represented in a homo- Scala’s rich syntax and static type system. geneous way with S-expressions. Therefore recognition and replacement of program fragments can be done uniformly, 2. INTUITION regardless of whether a transformed fragment represents e.g. To get acquainted with metaprogramming in Scala, let an arithmetic expression or a function definition. us explore the simplest flavor of Scala macros, def macros, In Scala, a language with rich syntax and static types, which were inspired by macros in Lisp [17, 13] and Nemerle compile-time transformations of code naturally distinguish [36]. terms and types, expressions and definitions, following the Def macros are methods, whose calls are expanded at com- architecture of scalac, the Scala compiler. Therefore it pile time. Here, expansion means transformation into a code makes sense to recognize the following three realizations of snippet derived from the method being called and its argu- textual abstraction in Scala: term macros, which expand ments. When such macros are expanded, they operate with terms, type macros, which expand types, and macro anno- a context, which exposes the code to be expanded and rou- tations, which expand definitions. In this section we will tines to manipulate code snippets. highlight these three kinds of macros along with their fla- The def macro context provides the opaque type Code vors, which appear on the intersection with other language representing untyped code snippets, exposes the macroAp- features. plication method, which returns the call being expanded, and defines the q string interpolator, which makes it possible 3.1 Def macros to create and pattern match snippets using the convenient The most natural flavor of term macros are def macros, string literal syntax. For example, q"$x + $y" creates a briefly covered in the Section 2. To the programmer, def snippet which represents addition of two arguments speci- macros look like regular Scala methods with an unusual fied by snippets x and y, and val q"$x + $y" = z pattern property– when a method call in a Scala program is resolved matches z as addition and binds x and y to summands. to represent an application of a def macro, that macro defini- Asserts are the canonical example of familiar function- tion is expanded by invoking a corresponding metaprogram, ality, which can be enhanced with def macros. The as- called macro implementation. As a convenience, the macro sert function evaluates the provided boolean expression and engine automatically destructures the method call being ex- raises an error if the result of evaluation is false.