Practical Reflection and Metaprogramming for Dependent Types David Raymond Christiansen Advisor: Peter Sestoft Submitted: November 2, 2015 i Abstract Embedded domain-specific languages are special-purpose pro- gramming languages that are implemented within existing general- purpose programming languages. Dependent type systems allow strong invariants to be encoded in representations of domain-specific languages, but it can also make it difficult to program in these em- bedded languages. Interpreters and compilers must always take these invariants into account at each stage, and authors of embedded languages must work hard to relieve users of the burden of proving these properties. Idris is a dependently typed functional programming language whose semantics are given by elaboration to a core dependent type theory through a tactic language. This dissertation introduces elabo- rator reflection, in which the core operators of the elaborator are real- ized as a type of computations that are executed during the elab- oration process of Idris itself, along with a rich API for reflection. Elaborator reflection allows domain-specific languages to be imple- mented using the same elaboration technology as Idris itself, and it gives them additional means of interacting with native Idris code. It also allows Idris to be used as its own metalanguage, making it into a programmable programming language and allowing code re-use across all three stages: elaboration, type checking, and execution. Beyond elaborator reflection, other forms of compile-time reflec- tion have proven useful for embedded languages. This dissertation also describes error reflection, in which Idris code can rewrite DSL er- ror messages before presenting domain-specific messages to users, as well as a means for integrating quasiquotation into a tactic-based elaborator so that high-level syntax can be used for low-level reflected terms. ii Resumé Indlejrede domænespecifikke sprog er skræddersyede program- meringssprog, som er implementeret inde i eksisterende alment an- vendelige programmeringssprog. Afhængige typer tillader, at stærke invarianter indkodes i datarepræsentationer, herunder repræsenta- tioner af programmeringssprog. Dog kan denne kontrol også be- sværliggøre programmeringsprocessen, da brugeren altid skal de- monstrere, at invarianterne er overholdt. Fortolkere og oversættere skal tage disse invarianter i betragtning i hvert trin. Når indlejrede sprog bliver designet, skal der arbejdes hårdt for at brugerne ikke bebyrdes med tungt bevisarbejde. Idris er et funktionsprogrammeringssprog med afhængige ty- per, hvis semantik er givet ved elaborering til en afhængigt-typet kerneteori ved brug af et taktiksprog. Denne afhandling introdu- cerer elaboratorrefleksion, hvor elaboratorens grundlæggende opera- tioner er realiseret som en type, hvis bereginger bliver afviklet un- der Idris’s egen elaborering. Dertil findes et rigt API for refleksion om termer, datatyper, og andre definitioner. De reflekterede elabore- ringsoperationer muliggør, at domænespecifikke sprog kan imple- menteres med Idris’s elaboreringsteknologi, og således får de elabo- rerede sprog flere muligheder for interaktion med Idris. Desuden bliver Idris anvendelig som sit eget metasprog, hvilket gør Idris til et programmerbart programmeringssprog og muliggør genbrug af kode i alle tre stadier: elaborering, typetjek og afvikling. Udover elaboreringsrefleksion har andre former for statisk re- fleksion vist sig at være nyttige for indlejring af sprog. Denneaf- handling præsenterer desuden refleksion af fejlmeddelelser, hvor kode skrevet i Idris kan omskrive fejlmeddelelser med oprindelse i do- mænespecifikke sprog til domænespecifikke fejlmeddelelser. Deru- dover præsenteres en teknik for integrering af kvasicitering i en tak- tikbaseret elaborator, så højtniveausyntaks kan anvendes for termer i kernesproget. Acknowledgments This dissertation represents the culmination of a long process of learn- ing and hard work that was made possible largely by the generosity and caring of other people. I would like to thank my supervisor, Peter Sestoft, for being a won- derful example of how to communicate clearly and how to let theory and practice inform one another. Even when my work diverged somewhat from our original plans, he has been supportive and helpful, always ready to discuss what I’ve been working on. Our partners at Edlund A/S, especially Henning Niss and Klaus Grue, got me off to a good start and kept me on the right track. Their feedback and good example was immensely valuable. I would also like to thank Edwin Brady for being so helpful as I’ve extended and modified his language, and the #idris channel on Freenode for being such a great community. I would especially like to thank my fellow Ph.D. students Hannes Meh- nert and Ahmad Salim Al-Sibahi. Hannes’s hacker spirit was always an inspiration, and we had many good discussions about both programming languages and wider themes. Ahmad’s encyclopedic knowledge helped me find many papers that I didn’t even know existed, and his feedback on an earlier draft of this dissertation made it much better. If not for the Danish welfare society, I may have never had the chance to go to graduate school. The Danish public sector paid my tuition and stipend while I was working towards my master’s degree, and my work was funded as part of the Actulus project, Danish National Advanced Technology Foundation (Højteknologifonden) grant 017-2010-3. Højteknologi- fonden has since become part of Innovationsfonden. Tusind tak! And thanks, Lisbet, for support both moral and material, and for your patience during the last few months. Contents Acknowledgments iii Contents v 1 Introduction 1 I Background 7 2 Embedding Languages 9 2.1 Embeddings, Deep and Shallow .................... 9 2.2 Representing Binders ......................... 10 2.3 Syntactic Re-Use ........................... 11 2.4 Elaborating Embedded Languages ................... 12 3 Reflection and Metaprogramming 15 3.1 Quasiquotations in Programming Languages .............. 15 3.2 Static Reflection ........................... 17 3.3 Reflection and Dependent Types .................... 18 4 The Idris Elaborator 21 4.1 The Core Language .......................... 23 4.2 Pattern-Matching Definitions and Data Types .............. 24 4.3 The Development Calculus ...................... 25 4.4 Elaboration Example ......................... 26 4.5 Binders in the Elaborator ....................... 34 5 Reflection in Idris 37 vi Contents II Metaprogramming Idris 45 6 A Pretty Printer that Says What it Means 49 6.1 Presentations ............................ 50 6.2 The Idris IDE Protocol ......................... 56 6.3 Annotated Pretty Printing ....................... 57 6.4 Conclusions ............................. 60 7 Quasiquotation 61 7.1 Example ............................... 62 7.2 Idris Quasiquotations ......................... 64 7.3 Elaborating Quasiquotations ..................... 65 7.4 Polymorphic Quotations ........................ 72 7.5 Quoted Names ............................ 73 7.6 Future Extensions .......................... 74 8 Error Reflection 77 8.1 Introduction ............................. 77 8.2 Error Reflection ........................... 78 8.3 Applications ............................. 85 8.4 Argument Error Handlers ....................... 90 8.5 Implementation Considerations .................... 91 8.6 Related Work ............................ 93 8.7 Conclusion and Future Work ...................... 93 9 Elaborator Reflection 95 9.1 Introductory Examples ........................ 96 9.2 Elaborator Reflection, Defined ..................... 103 9.3 Implementation Considerations .................... 111 9.4 The Pruviloj Library .......................... 113 9.5 Other Applications .......................... 117 9.6 Agda-Style Reflection ......................... 135 9.7 Reflections on Elaborator Reflection .................. 140 10 Conclusions 147 Bibliography 149 A Elaborator Tactics 161 Contents vii B The Idris Language 165 Glossary 171 Chapter 1 Introduction Static type systems in programming languages are becoming more and more expressive over time. As type systems get more and more expres- sive, they offer greater abilities to express complex invariants, ruling out more classes of bugs. Additionally, an expressive static type language can enable better programming tools. If the system knows more about what the programmer is trying to build, then it is in a better position to help her get there! As type systems become more and more expressive, the job of the pro- grammer when writing a type begins to resemble the job of the program- mer when writing a program. In other words, types begin to need to do computation. Unfortunately for programmers, most type systems were not designed as programming languages, and systems in which the ability to write programs is accidental tend to not be very pleasant to write pro- grams in. Enter dependent types. Dependently typed languages use the same language for type-level and program-level computation. Because this lan- guage is designed explicitly for programming, type-level programming is no longer a difficult subject reserved for the most advanced users. Ad- ditionally, it becomes possible to re-use code in both stages: the same list datatype that is used to contain run-time values from a database can be used to represent a statically-checked typing context for an embedded domain-specific language. Martin-Löf’s type theory and its descendants have been recognized