A Principled Approach to REPL Interpreters L
Total Page:16
File Type:pdf, Size:1020Kb
A principled approach to REPL interpreters L. Thomas Mauricio Verano Merino Pierre Jeanjean van Binsbergen Eindhoven University of Inria, University of Rennes, CRNS, Centrum Wiskunde & Informatica Technology IRISA Amsterdam, The Netherlands Eindhoven, The Netherlands Rennes, France [email protected] [email protected] [email protected] Tijs van der Storm Benoit Combemale Olivier Barais Centrum Wiskunde & Informatica University of Rennes, Inria, CNRS, University of Rennes, Inria, CNRS, Amsterdam, The Netherlands IRISA IRISA University of Groningen Rennes, France Rennes, France Groningen, The Netherlands [email protected] [email protected] [email protected] ABSTRACT CCS CONCEPTS Read-eval-print-loops (REPLs) allow programmers to test • Software and its engineering ! Interpreters; Domain out snippets of code, explore APIs, or even incrementally specific languages; Integrated and visual development en- construct code, and get immediate feedback on their actions. vironments; • Human-centered computing ! Command However, even though many languages provide a REPL, the line interfaces. relation between the language as is and what is accepted at the REPL prompt is not always well-defined. Furthermore, KEYWORDS implementing a REPL for new languages, such as DSLs, may interpreters, REPLs, software language engineering, note- incur significant language engineering cost. books, meta-languages, language workbenches In this paper we survey the domain of REPLs and investi- ACM Reference Format: gate the (formal) principles underlying REPLs. We identify L. Thomas van Binsbergen, Mauricio Verano Merino, Pierre Jean- and define the class of sequential languages, which admita jean, Tijs van der Storm, Benoit Combemale, and Olivier Barais. sound REPL implementation based on a definitional inter- 2020. A principled approach to REPL interpreters. In Proceedings of preter, and present design guidelines for extending existing the 2020 ACM SIGPLAN International Symposium on New Ideas, New language implementations to support REPL-style interfaces Paradigms, and Reflections on Programming and Software (Onward! (including computational notebooks). The obtained REPLs ’20), November 18–20, 2020, Virtual, USA. ACM, New York, NY, USA, can then be generically turned into an exploring interpreter, 17 pages. https://doi.org/10.1145/3426428.3426917 to allow exploration of the user’s interaction. The approach is illustrated using three case studies, based 1 INTRODUCTION on MiniJava, QL (a DSL for questionnaires), and eFLINT (a “The top level is hopeless”, Matthew Flatt1 DSL for normative rules). We expect sequential languages, Read-eval-print-loops (REPLs, also known as command-line and the consequent design principles, to be stepping stones interfaces, or interactive shells) are a popular way for pro- towards a better understanding of the essence of REPLs. grammers to interact with programming languages. They allow incremental definition of abstractions, testing out snip- Permission to make digital or hard copies of all or part of this work for pets of code with immediate feedback, debugging executions, personal or classroom use is granted without fee provided that copies are not and explore APIs. made or distributed for profit or commercial advantage and that copies bear Some languages, such as scripting languages or interpreted this notice and the full citation on the first page. Copyrights for components languages, are more naturally compatible with the REPL of this work owned by others than ACM must be honored. Abstracting with mode of interaction and the styles of programming that it credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request enables (and that programmers have come to expect). For permissions from [email protected]. example, a sequence of valid code snippets written in the Onward! ’20, November 18–20, 2020, Virtual, USA REPL of Python is itself a valid Python program. On the © 2020 Association for Computing Machinery. other hand, JShell, for instance, allows programmers to write ACM ISBN 978-1-4503-8178-9/20/11...$15.00 https://doi.org/10.1145/3426428.3426917 1https://gist.github.com/samth/3083053 Onward! ’20, November 18–20, 2020, Virtual, USA L.T. van Binsbergen, M. Verano Merino, P. Jeanjean, T. van der Storm, B. Combemale, O. Barais expressions, statements, variable declarations and method in the Rascal language workbench [15], to make it sequen- declarations as code snippets, even though these constructs tial. This extended MiniJava is then the base interpreter for are not allowed at the top-level in Java programs. a computational notebook interface through Bacatá, Ras- Consider the following example JShell interaction (every cal’s bridge to Jupyter [22]. The second case study involves line is a code snippet sent separately): QL, a DSL for defining spreadsheet-like interactive question- naires [8, 9]. This case study show that it is feasible to obtain class Example {} REPLs for languages that are not statement- or expression- Example obj = new Example(); oriented. The third case-study applies the methodology to class Example { public int meth() { return var; } } obtain interactive services for eFLINT, a DSL for executable int var = 1; normative specifications [43]. The resulting services allow users and policy-aware software to navigate choices and This example raises the questions whether classes can be decisions in the realm of law and regulation. redefined, whether obj can be accessed after Example is rede- To summarize, the contributions of this paper are: fined or if obj is migrated, and, if so, what methods it has and, if meth is available, whether a call obj.meth() returns 1. With- • A feature-based analysis of the landscape of REPLs out giving answers here, the example shows that the relation for a selection of the most popular programming lan- between a programming language and the behavior of its guages (Section2). REPL is not immediately obvious. Matthew Flatt’s ceterum • A formalization of the notion of sequential language censeo quoted above bears witness to the fact that the relation as the underlying principle of REPLs (Section3) can actually be strenuous and cause a lot of confusion. The • A language-parametric exploring interpreter algorithm above questions are fundamentally about language design: on top of existing interpreters, allowing users to navi- several sensible answers are possible and the answers have gate user interaction history (Section4) a significant impact on programmer experience. • A methodology for developing REPL interpreters by se- In some sense, JShell can be seen to implement its own quentializing languages with a definitional interpreter language, which, even though strongly reminiscent of Java, (Section5). is markedly different. In this paper, we take this observation • Three case studies to illustrate the feasibility of the and run with it: we assume that a REPL interpreter for L approach (Section6). effectively defines its own language R, often as an extension The paper is concluded with a discussion of limitations, re- or modification of L, whose programs are sequences of valid lated work, and directions for further research. code snippets according to the REPL. To this end we identify and define the class of languages 2 REPL DOMAIN ANALYSIS that underlie REPL interpreters as sequential languages. The This section provides a study of existing REPL interpreters essence of sequential languages is that the concatenation of and their main features. We have studied freely available two programs is again a program. Or, to put it more precisely, a REPL implementations, listed in Table1, for the 15 most pop- language is sequential if it features an associative sequencing ular languages from the TIOBE index2, with the exception operator o, such that the following equation holds: 9 of Visual Basic, for which we could not find an freely avail- o able implementation. For MATLAB we have selected GNU p1 9 p2 = p2 ◦ p1 J K J K J K Octave as a substitute. We performed a feature-oriented do- The meaning of a sequence of program fragments is defined main analysis [14], resulting in the feature model of Figure1. by composing the meanings of the individual fragments, Below we briefly describe the main mandatory and optional including any impure effects of these fragments. features. The notion of sequential language informs a methodology to make a language sequential, and hence suitable for sound Mandatory Features. An interpreter must have certain fea- REPL interpreters. The methodology enforces certain design tures to be considered a REPL. In particular, a REPL has the principles on the REPL engineer to ensure that questions ability to execute multiple code snippets across multiple in- like the ones asked about the JShell interaction are answered teractions in a single session (as opposed to executing one precisely and are explicitly addressed as matters of language full program per session). In most of the investigated REPL design, instead of an implementation concern. Furthermore, implementations, the REPL maintains execution context and sequential languages are amenable to interfaces which allow executes snippets incrementally (the “Incremental” alterna- exploring execution traces resulting from REPL interactions. tive of the “Snippet Execution” feature). Optionally, a REPL We have applied this methodology in three case studies. The first extends an existing implementation of