First-Class Support for Resugaring in Rascal Master's Thesis Wouter Nederhof Supervisor: dr. Tijs van der Storm Centrum voor Wiskunde en Informatica Faculty of Science University of Amsterdam The Netherlands January 5, 2016 Abstract In this Master's thesis, we aimed to investigate how capable resugaring techniques are for real-world use. As part of our research, we created a prototype based on a literature study and improved this artefact using observations from different case studies. We started our research by studying two papers of J. Pombrio and S. Krishnamurthi on resugaring to build our initial prototype[1][2]. During our research, we observed that the techniques described in "Lifting Evaluation Sequences through Syntactic Sugar" were neither efficient nor expressive enough for our case studies. The techniques described in the paper "Hygienic resugaring of compositional desugaring" had sufficient performance capacities, but were too constrained for resugaring terms. Following the literature study, we integrated the techniques from these papers into Rascal. During the implementation, we met two notable problems. First, we observed that patterns containing ellipses can break symmetry. As a solution for this problem, we devised a technique in which the lengths of ellipses are fixed after a transformation. Second, we had to work around a constraint in the techniques of the latter paper, which required transformation functions to consume and produce patterns. We found this to be a major restriction, since Rascal did not have a pattern data type. This meant that we had to devise new techniques to obtain a similar level of expressivity. Although our prototype was based on techniques by J. Pombrio and S. Krishnamurthi, adapt- ing and extending their techniques to Rascal ultimately led to a significantly different design. In order to sustain our design decisions, we sketched how we could formally address most of them. Finally, we demonstrated that our finished prototype was capable of desugaring and resug- aring multiple different cases. Some of these cases could not be addressed using the original techniques and required a change in our design. We showed that our prototype was able to desugar and resugar different categories of syntactic sugar in ES6. Additionally, we were able to build an evaluation stepper that desugars a functional programming language into the Lambda Calculus, step through the code and resugar intermediate results, including Church numerals. Lastly, we demonstrated that our prototype is capable of desugaring and resugar- ing terms over 50.000 characters in size in just over a second, and how the different techniques perform using a performance benchmark. Contents 1 Introduction 5 1.1 Context . .6 1.2 Motivation . .6 1.3 Research Questions . .6 1.4 Research Method . .7 1.5 Contributions . .7 1.6 Related Work . .7 1.7 Outline . .9 2 Background 10 2.1 Concrete and Abstract Syntax . 10 2.2 Pattern Matching and Substitution . 10 2.3 Rascal . 11 2.3.1 Static Typing . 11 2.3.2 Syntax Declaration and ADTs . 11 2.3.3 Functions . 12 2.3.4 Annotations . 13 2.4 Lambda Calculus . 13 3 Desugaring and Resugaring 15 3.1 Desugaring . 15 3.2 Benefits of Desugaring . 16 3.3 The Need for Resugaring . 17 4 Resugaring Techniques 19 4.1 Resugaring: Lifting Evaluation Sequences through Syntactic Sugar . 19 4.2 Hygienic resugaring of compositional desugaring . 21 5 Resugaring in Rascal 24 5.1 Desugaring and Resugaring . 24 5.2 Sugar Function Declarations . 24 5.2.1 Intermediate Sugar Function Declarations . 25 5.2.2 Compositional Sugar Function Declarations . 25 5.2.3 Custom Sugar Function Declarations . 26 5.3 Fixing the Lengths of Ellipses . 27 5.4 When-conditions . 27 1 5.5 Resugaring Fallback Mechanism . 28 5.6 Break-out functions . 29 5.7 Usage Examples . 29 5.7.1 Basic Usage . 30 5.7.2 Intermediate Versus Compositional Desugaring . 30 5.7.3 Desugaring Using Different Types . 31 5.7.4 When-Conditions . 32 6 Implementation and Observations 34 6.1 Initial Design and Implementation . 34 6.1.1 Evaluation . 35 6.1.2 Retrospect . 35 6.2 Compositional Desugaring . 37 6.2.1 Compositional Desugaring Transformation . 37 6.2.2 Limitations . 38 6.2.3 Desugaring Intermediate Core Terms . 39 6.3 Matching Ambiguity . 40 6.4 Sugar Contexts . 41 6.5 Beyond Substitution-Based Desugaring . 42 6.6 Fallback Resugaring Functions . 42 6.7 Annotations . 43 6.8 Node Identity . 44 7 Formal Justification Sketch 45 7.1 Properties . 45 7.2 Outline . 46 7.3 Formal Definition . 46 7.4 Emulation . 48 7.5 Abstraction . 52 7.6 Coverage . 53 8 Evaluation 54 8.1 Expressiveness Evaluation: Fun to Lambda Calculus . 54 8.1.1 Syntax Definition . 54 8.1.2 Church Numerals . 55 8.1.3 Matching-And-Substitution-Based Transformations . 56 8.1.4 Evaluator . 57 8.1.5 Usage Examples . 57 8.1.6 Conclusion . 60 8.2 Expressiveness Evaluation: ES6 to ES5 . 61 8.2.1 Arrows to Functions . 61 8.2.2 Binary and Octal Numbers to Decimal Numbers . 63 8.2.3 Rest and Keyword Parameters . 64 8.2.4 Conclusion . 66 8.3 Performance Benchmark . 66 8.3.1 Setup . 66 8.3.2 Results and Conclusion . 67 2 8.4 Performance Evaluation Case Study . 68 8.4.1 Banana Algebra Fun Language . 69 8.4.2 Setup . 69 8.4.3 Results and Conclusion . 69 9 Discussion 71 9.1 Research Questions . 71 9.1.1 Efficiency . 71 9.1.2 Expressivity . 72 9.1.3 Integration . 72 9.2 Prototype Limitations . 74 9.2.1 Layouts . 74 9.2.2 Fixing Ellipses' Lengths . 74 9.2.3 When-Conditions . 75 9.2.4 Typing-Related Resugaring Failure . 75 9.2.5 Layering Syntactic Sugar . 75 9.3 Recommendations . 76 9.4 Threats to Validity . 76 9.4.1 Performance . 76 9.4.2 Formal Justification . 77 9.4.3 Expressiveness . 77 10 Conclusion 79 11 References 80 A Prototype Implementation 82 A.1 Altered Files . 82 A.1.1 Operations on patterns and terms . 82 A.1.2 Environments . 84 A.1.3 Desugaring and Resugaring . 84 A.1.4 Bootstrapping and Syntax . 84 A.1.5 Semantics . 85 A.2 Installation Instructions . 85 B Unit.