Fexprs As the Basis of Lisp Function Application Or $Vau : the Ultimate Abstraction by John N

Fexprs as the basis of Lisp function application or $vau : the ultimate abstraction by John N. Shutt A Dissertation Submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in Computer Science August 23, 2010 Approved: Prof. Michael A. Gennert, Advisor Head of Department Prof. Daniel J. Dougherty Prof. Carolina Ruiz Prof. Shriram Krishnamurthi Computer Science Department, Brown University Abstract Abstraction creates custom programming languages that facilitate programming for specific problem domains. It is traditionally partitioned according to a two-phase model of program evaluation, into syntactic abstraction enacted at translation time, and semantic abstraction enacted at run time. Abstractions pigeon-holed into one phase cannot interact freely with those in the other, since they are required to occur at logically distinct times. Fexprs are a Lisp device that subsumes the capabilities of syntactic abstraction, but is enacted at run-time, thus eliminating the phase barrier between abstractions. Lisps of recent decades have avoided fexprs because of semantic ill-behavedness that accompanied fexprs in the dynamically scoped Lisps of the 1960s and 70s. This dissertation contends that the severe difficulties attendant on fexprs in the past are not essential, and can be overcome by judicious coordination with other elements of language design. In particular, fexprs can form the basis for a simple, well- behaved Scheme-like language, subsuming traditional abstractions without a multi- phase model of evaluation. The thesis is supported by a new Scheme-like language called Kernel, created for this work, in which each Scheme-style procedure consists of a wrapper that in- duces evaluation of operands, around a fexpr that acts on the resulting arguments. This arrangement enables Kernel to use a simple direct style of selectively evaluat- ing subexpressions, in place of most Lisps’ indirect quasiquotation style of selectively suppressing subexpression evaluation. The semantics of Kernel are treated through a new family of formal calculi, introduced here, called vau calculi. Vau calculi use direct subexpression-evaluation style to extend lambda calculus, eliminating a long- standing incompatibility between lambda calculus and fexprs that would otherwise trivialize their equational theories. The impure vau calculi introduce non-functional binding constructs and unconven- tional forms of substitution. This strategy avoids a difficulty of Felleisen’s lambda-v- CS calculus, which modeled impure control and state using a partially non-compatible reduction relation, and therefore only approximated the Church–Rosser and Plotkin’s Correspondence Theorems. The strategy here is supported by an abstract class of Regular Substitutive Reduction Systems, generalizing Klop’s Regular Combinatory Reduction Systems. Preface The concept of first-class object is due to Christopher Strachey (1916–1975). A first-class object in any given programming language is an object that can be employed freely in all the ways that one would ordinarily expect of a general value in that language. The “rights and privileges” of first-class objects —the ways one expects they may be freely employed— depend on the language. Although one might draw up a partial list of first-class rights and privileges for a given language, e.g. [AbSu96, §1.3.4], a complete list is never quite possible, because (as in human societies) the rights and privileges of first-class-ness are difficult to recognize until they are missed. Strachey’s prime example, then, of second-class objects was procedures in Algol. Procedures in Algol can be called, or passed as arguments to procedure calls, but —unlike numbers— cannot be stored as the values of variables, nor returned as the results of procedure calls. Thus Algol procedures cannot be denoted by compound expressions, nor by aliases; as [Stra67, §3.5] put it, they “need to appear in person under their own names.” The Scheme programming language pioneered first-class treatment of several types of objects, notably procedures (whose promotion to first-class status will be discussed in this dissertation in §3.3.2). However, for those cases in which the operands in an expression are not to be evaluated automatically, special reserved operator symbols are used; and the evaluation rules denoted by these reserved symbols cannot be evoked in any other way — they always have to appear in person under their own names.1 The Kernel programming language is a redesign of Scheme for the current work2 that grants first-class status to the objects named by Scheme’s special-form operators (both built-in and user-defined). Why, how, and with what consequences it does so are the concerns of this dissertation. In place of Scheme’s single type of first-class procedures, Kernel has two types of first-class combiners: applicatives, which are analogous to Scheme procedures; and operatives (known historically in the Lisp community as fexprs), which take their operands unevaluated and correspond to Scheme’s special-form combiners. To avoid gratuitous confusion between the two, Kernel conventionally prefixes the names of its 1A more detailed discussion of first-class-ness, both in general and in the particular case of Scheme, appears in [Shu09, App. B (First-class objects)]. 2That is, Kernel and this dissertation are both part of the same work, by the same author. See §3.5. iii operatives with “$” (a convention that enhances the lucidity of even mundane Kernel code, independent of any exotic use of first-class operatives). The elegance of Kernel’s support for first-class operatives arises from the synergism of two distinct innovations. The flashier innovation is the $vau operative, which behaves nearly identically to $lambda except that its result is operative rather than applicative. $vau also differs from $lambda by having an extra parameter, appearing after the usual parameter tree, which locally binds the dynamic environment from which the constructed operative is called. One could, for example, write ($define! $if ($vau (x y z) env (0.1) ($cond ((eval x env) (eval y env)) (#t (eval z env))))) , deriving a compound operative $if from pre-existing operative $cond. As an alter- native to hygienic macro declarations, $vau gains clarity by using ordinary Scheme/ Kernel tools rather than a separate macro sublanguage, and by specifying its evalu- ations —and its uses of the dynamic environment— explicitly; but the more elegant, and subtler, innovation of Kernel’s operative support lies in its treatment of first-class applicatives. A Kernel applicative is simply a shell, or wrapper, to induce argument evaluation, around some other underlying combiner (which may even be another applicative). The constructor of applicatives is an applicative wrap. The introduction of wrap as a primitive, evidently orthogonal to $vau, obviates the need for a primitive $lambda, since $lambda can then be constructed as a compound operative: ($define! $lambda ($vau (ptree . body) static-env (0.2) (wrap (eval (list* $vau ptree #ignore body) static-env)))) . The introduction of an inverse operation unwrap completes the orthogonalization of Kernel combiner semantics, by allowing apply to be constructed as well: ($define! apply ($lambda (applicative argument-tree) (0.3) (eval (cons (unwrap applicative) argument-tree) (make-environment)))) . Content of the dissertation The main content of the dissertation is divided into two parts: Part I addresses Kernel (the practical instrument of the thesis, designed by the author for the extended work), while Part II addresses vau calculi (its theoretical instrument, designed as part of the iv dissertation). Each part begins with a chapter of background materials preliminary to that part. Chapter 1 contains background materials that motivate and clarify the thesis, and therefore logically precede both parts. Chapter 2 contains background materials on the use of language in the dissertation that cross-cut the division between theory and practice. Much of the background material is historical. Historical background serves three functions: it clarifies where we are, by explaining how we got here; it explains why we got here, laying motivational foundation for where to go next; and it reviews related work. The background chapters also concern themselves not only with how programming languages specify computations (semantics), but with how programming languages influence the way programmers think (psychological bias). This introduces a sub- jective element, occasionally passing within hailing distance of philosophy; and the thesis, since it is meant to be defended, mostly avoids this element by concerning itself with semantics. However, the background chapters are also meant to discuss language design motivations, and psychology is integral to that discussion because it often dominates programmer productivity. Semantics may make some intended computations difficult to specify, but rarely makes them impossible — especially in Lisp, which doesn’t practice strong typing; whereas psychological bias may prevent en- tire classes of computations outright, just by preventing programmers from intending them. Chapter 16 contains concluding remarks on the dissertation as a whole. Several appendices contain material either tediously uninsightful (and therefore deferred from the dissertation proper) but technically necessary to the thesis defense; or not directly addressing the thesis, but of immediate interest in relation

Load more