THEORY & PRACTICE

Mutant languages from the LISP lab

By Brian Hayes APL is like a diamond. It has a beau are a bunch of malcontents who will not Planning and conniving tiful crystal structure; all of its parts leave well enough alone. They all think Some of the most influential variations are related in a uniform and elegant they know better than anyone else how on LISP were conceived in the late 1960s way. But if you try to extend this struc the language ought to work. and early 1970s by of the ture in any way—even by adding an Rolling-mudball-gathers-lots-of-moss Massachusetts Institute of Technology. other diamond—you get an ugly kludge. theory. LISP is so easily extended that it The first of his languages, called PLAN LISP, on the other hand, is like a ball of sprouts new languages willy-nilly. Every NER,3 added to LISP a facility for goal- mud. You can add any amount of mud LISP program is a new version of LISP. directed programming. to it and it still looks like a ball of mud. There may be truth in all of these pro A PLANNER program had two parts: —Joel Moses, as paraphrased positions, but more substantial factors a data base of assertions and a set of by Guy Lewis Steele Jr. are also at work. One important point is procedures for proving theorems about and Gerald Jay Sussman1 that LISP has a simple syntax. Most of the assertions. For example, an assertion the work of computation in a LISP pro might state that Socrates is human, and I f you ask the aver gram is done by a single kind of expres a theorem might hold that anyone who is age programmer- sion, the function call, which takes the human is mortal. Given the goal of prov I in-the-street what following form: (function-name ing that Socrates was mortal, PLAN is important or distinctive about LISP, argument, argument2 arguments...). NER would attempt to apply every the answer will probably have something There is not much more to know about theorem it knew to every assertion in the to do with artificial intelligence. That the syntax of LISP programs. data base until it either satisfied the goal judgment is certainly understandable: Other computer languages, in contrast, or exhausted the possibilities. LISP and Al were born at the same time have given rise to strident debates over The idea of computing by proving and place to the same parents, which syntactic issues. (Is it better to bracket theorems has now become familiar makes them twin siblings. They remain statements with begin...end or with {...}? through the language PROLOG. PLAN constant companions. Nevertheless, I Should the semicolon be a terminator or NER definitely influenced the develop would argue that the real importance of a separator of statements? What is the ment of PROLOG,4 but the connection LISP lies elsewhere. best delimiter for comments?) In the should not be overemphasized; there are What sets LISP apart, in my view, is LISP community, questions like these fundamental differences between the two its role as a language laboratory. LISP is have largely been ignored. Most of the languages. In the first place, PLANNER a culture medium for computer lan languages that have grown out of LISP theorems were expressed in procedural guages: a nutrient broth where new ideas have adopted the syntax of the parent form, whereas PROLOG is a purely de emerge from the ooze, then mutate, language with little change. clarative language. Furthermore, PRO evolve, and cross-breed. Here I will re Dismissing the problem of syntax in LOG uses a more sophisticated theorem- view a few of the more interesting ex this offhand way has had two important proving strategy, called unification. periments to come out of the LISP effects. First, it has focused attention on (Soon after PROLOG appeared, of laboratory. semantics—on the meaning of expres course, it was implemented in LISP.5) sions rather than their form. Second, it The most conspicuous innovation in Why LISP has made the implementation of new lan PLANNER was its control structure: Why has LISP been the focus of all this guages within LISP much easier. Be from the programmer's point of view, linguistic experimentation? A number of cause parenthesized lists serve as a there was none. Procedures were invoked hypotheses have been offered: notation for both programs and data, the automatically by a pattern-matching Ivory-tower theory. Until recently, the same expression can be interpreted as process.6 In proving the mortality of Soc economic significance of LISP program data by the underlying LISP system and rates, any procedures matching the pat ming was nil. There was little commer as a program by the embedded language. terns (mortal ?x) or (human ?x) would cial pressure to standardize the language. Parsing a program-which is a major be triggered. The sequence of procedure Thus LISP evolved freely while other undertaking for a language such as Ada calls was not under the programmer's languages of the same generation, such or PL/I-becomes almost trivial. Most of control. PLANNER always made a as FORTRAN and COBOL, were fro the work is done by the built-in function depth-first search of the data base and zen in an early stage of development. ready which digests entire expressions in backtracked when it came to a dead end. Mother-of-invention theory. LISP has a single gulp. There is a certain irony in The full PLANNER language was been used to attack hard problems, the longevity and stability of LISP syn never implemented, but a subset called which demand powerful tools. LISP itself tax. The parenthesized prefix notation MicroPLANNER was written in Mac- has not been adequate so there has been was introduced in 1958 only as a tempo LISP (the MIT dialect of LISP) by Ger no choice but to build new and better rary measure to get the first LISP inter ald Jay Sussman, , and languages. preter running.2 The plan was to replace Hotshot theory. LISP programmers it as soon as something better could be devised. Evidently, nothing better has turned up yet.

23 Eugene Charniak.7 Its most celebrated has little to do with Hewitt's work in compound data structure. In most lan use was in Winograd's SHRDLU pro LISP; ACTOR is implemented in a guages only simple values, such as num gram, which answered natural-language Forth-like language.) bers, have all of these rights and queries about a world of toy blocks. privileges, and in earlier LISPs proce Eliminating all control structures took Scheming dures were unquestionably second-class a lot of the bother out of programming, One small feature of the PLASMA lan citizens. Scheme emancipates them. but a search based on blind backtracking guage, seemingly nothing more than a The combination of lexical scope and did not make for dazzling efficiency. technical nicety, has had far-reaching first-class procedures encourages a dis CONNIVER,8 developed by Sussman consequences. To make the Actors model tinctive style of programming in Scheme, and Drew V. McDermott, was a succes work properly, PLASMA was based on emphasizing modularity and data ab sor to PLANNER with provisions for ex lexical scope rules, in which the value of straction. For example, a procedure plicit control of execution. CONNIVER a variable is determined by the textual named make-db might set up a private was PLANNER with a manual trans context in which it is defined. Tradition data base and then return another proce mission. Using the control facilities pro ally, LISP has employed dynamic scope dure as its value. When the latter proce vided by CONNIVER, the system could rules, in which values are determined at dure is called it has exclusive access to be instructed to examine all assertions the point where a variable is referenced. the data base. Moreover, each invocation about Socrates before looking at those Lexical scope was essential in PLASMA of make-db creates a new instance of the having to do with humans and mortals. because the acquaintances of an Actor data base and a new access procedure. are listed in the Actor definition and Acting must not be altered by any rebinding of Naming Hewitt's second linguistic experiment is the same names elsewhere in the Just as Scheme liberates procedures from called the Actors model of computa program. their second-class status, a language tion.9'10 Actors are independent entities In 1975 Sussman and Guy Lewis called Symmetric LISP is built on the that encapsulate both data and proce Steele Jr., while studying the Actors concept of first-class environments. Sym dures. A program (or a system of pro- model (out of "morbid curiosity," as they metric LISP is currently being developed grams-the distinction is blurred) put it), made a surprising discovery.13 by David Gelernter of Yale University, constitutes a society of Actors that com They had written a small Actors inter Suresh Jagannathan of MIT, and Thom municate with one another by sending preter with lexical scope rules and had as London of AT&T Bell Laboratories.17 messages. Each Actor is defined by two found that the constructs representing An environment is essentially a diction components: the set of messages it recog Actors took a familiar form: they were ary in which one can look up any vari nizes and the set of acquaintances with simply lambda expressions. able name and find the corresponding whom it can communicate. Lambda is the mechanism of proce value or definition. A common way of Just as PLANNER is reminiscent of dural abstraction in LISP, the means by implementing an environment is by using PROLOG, the concepts underlying the which an expression or a series of expres the data structure known in LISP as an Actors model will be familiar to those sions is wrapped.up to form a unit of ex association list, or a-list. The environ who know object-oriented languages such ecutable code. For example, whereas ment ((x 3)(y 4)), represented here as an as . Actors correspond to ob (+ x x) evaluates immediately to twice a-list, records two facts: that the variable jects in Smalltalk; actions are methods; the value of x, (lambda (x) (+ x x)) x is bound to the value 3 and that y is messages have the same name in both yields a procedure that doubles the value bound to 4. If the procedure represented languages. Once again, however, similar it is given as an argument. Like an Ac by (lambda () (+ x y)) is executed in ity is no guide to heritage. Actors and tor, a lambda expression has two parts: this environment it returns the value 7. Smalltalk were invented independently the code to be executed—(+ x x) in this Gelernter and his colleagues point out and at about the same time; they both case—and the environment in which that that associating names with values is a owe much to Simula, which was devel code is to be evaluated. The code corre central activity in many areas of comput oped a decade earlier. sponds to the actions taken by an Actor ing. Structures analogous to environ Hewitt and his colleagues implement in response to messages. The environ ments are needed to support the global ed the Actors model in a language called ment, which is a list of the variable bind and local variables of any language with PLASMA (for PLANNER-like System ings in effect when the lambda form is lexical scope. Packages or modules for Modeled on Actors). PLASMA is built defined, corresponds to an Actor's list of separate compilation also establish name within LISP, although, as will be dis acquaintances. spaces. Records with named fields, as in cussed later, it differs significantly from Out of this small observation Sussman Pascal, are merely a variation on the most versions of LISP prevalent at the and Steele (with later contributions by same theme. Even a directory of disk time. Studies of the Actors model have many others) constructed an entire new files is organized as a dictionary associat continued, including a project called the dialect of LISP called Scheme.16 Among ing file names with file contents. In Sym Apiary," which proposes a parallel com the experiments discussed here, Scheme metric LISP, all of these forms of puter architecture in which each Actor is the only one yet to have escaped from naming (and others) are handled by a has its own processor. In recent years, the laboratory and proved its mettle in single data structure—the first-class others in the LISP community have the wild. At last count some nine imple environment. adopted object-oriented programming mentations were available, including The programmer's basic activity in with enthusiasm, and several dialects of some for microcomputers. Symmetric LISP is to build up nested LISP now have object-oriented exten Like PLASMA, Scheme has lexical environments in which names are bound sions or sublanguages. The best known of scope rules, but its most remarkable de to the values of arbitrarily complex ex these are LOOPS12 (LISP Object-Ori parture from established custom is the pressions. Even lambda expressions are ented Programming System), Flavors'3, first-class status it accords to procedures. represented as environments, where the and the public domain XLISP.14 (To add A first-class object in a programming bindings are those of the formal param to the confusion, the recently introduced language is one that has no arbitrary re eters. Evaluating an environment yields a language ACTOR is object-oriented but strictions on where it can go and what it new environment whose bindings have can do. A first-class object can be as been updated. The ultimate result of a signed as the value of a variable, passed as an argument to a procedure, returned as the result of a function, or stored in a

24 COMPUTER LANGUAGE ■ APRIL 1987 computation is an environment in which Add Data Security to Your C Programs variables are bound to answers. A major motivation for the develop ment of Symmetric LISP is parallel pro 1 cessing. As a rule, all the name-value pairs in an environment can be evaluated concurrently. Where one binding depends The SECURITYmUBRARY, on a value calculated in another binding, the conflict can be detected automatical Add Fast or Thorough Encryption You Choose the Security Level ly; no special control structures are need or Compression to Your Programs Algorithms provided include The National ed to synchronize parallel execution. WITHOUT Royalties Bureau of Standards' Data Encryption Standard Implementation of Symmetric LISP on a Build the safest and most popular methods of (DES) and the Vernam and Vegenere ciphers. multiprocessor system is currently under protecting your data directly into your program Encrypt a 10K file in 3 seconds with one method way; at the moment a Symmetric LISP without paying royalties. Don't worry if you're or 50 seconds with another. interpreter runs (without parallelism) on not sure what to use. The discussion of security Six algorithms are provided, along with a sequential LISP system. methods in the manual (with demos on disk) password and non-password encryption will help you make a choice based on your schemes. Valuable Extras Xapping application. Parallel processing has become a major With Security Library you can: The Huffman coding routine can reduce the size of a file by 25 to 50%. The routine to change preoccupation of the LISP community in • Keep files secure on a multi-user network • Speed up data transmission and the attribute bytes of a file can make that file recent years, and Symmetric LISP is by invulnerable to casual browsing or accidental no means the only result. Several other communications through data compression • Control access based on privileges given to a deletion. There's even a program to change language proposals take the control- hard disk or LAN user every byte of a file to a null character - not structure approach to parallelism. An ex even un-erase programs can recover it then! ample is Multilisp, created by Robert H. "I create custom software for business Halstead Jr. of MIT.18 applications. I'm using the Security Library to Call 800-821-2492 to order Security encrypt so that certain information will not be Multilisp is a dialect of Scheme aug readily available. The documentation is Library risk-free for only $125. Source mented with constructs for the control of excellent. It gives a thorough explanation about & Object is $250. parallel execution. The most important how you can secure a file." of these constructs is called future. An — Bruce Philips, Custom Software Design ^Solution expression of the form (future x) has two Virginia Beach, CA effects: it starts the calculation of x, <5ystems Requires MSDOS 2.0 +. Wivki with Microsoft C md Computer Inmnaliora' 335-L Washington St. which can be any expression, and it also C86. Ptcax specify compiler when you order. Norwell, MA 02061 immediately returns a placeholder value, (617) 659-1571 which will eventually be replaced by the actual value of jc. Hence, (cons (future CIRCLE 15 ON READER SERVICE CARD x) (future y)) would launch the calcula tion of both x and y while simultaneously allowing cons to return a placeholder value to its caller. Personalize Still another form of parallelism is be ing explored by Steele and W. Daniel your computing Hillis of Thinking Machines Corp.19 In Connection Machine LISP they intro environment. duce a new data structure called a xap The MKS Toolkit now contains ping that serves to organize fine-grained parallelism (the concurrent execution of the Korn shell command interpreter. many small operations rather than a few The MKS version of Bell Labs' Korn shell has this and more large blocks). • the full power of the UNIX System V.2 • command aliases A xapping is a collection of pairs, Bourne shell • interactive command-line facilities where the two items making up each pair • the most requested features of • previous command history and editing are called an index and a value. Thus a Berkeley's C shell • a powerful • the full-UNIX utility of executable shell • shell variable expansion xapping maps (or rather xaps) indices files • arithmetic evaluation onto values. In the xapping { x-*2 y-+3 All this has been fine-tuned to create the optimum environment under DOS. The Kom z-»4}the indices are x, y, and z and the shell is just one of over 100 commands — fully compatible with UNIX System V.2 — values are 2, 3, and 4. The mapping from now contained in the MKS Toolkit, including the following: a w k c a t c h m o d indices to values suggests that a xapping emp cp cpio ctags cut date is somewhat like an association list, but d d d f d i f f du echo ed egrep ex fgrep in other respects it is quite different. The fi l e fi n d h e a d help join 1c Is more mv n m o d p a s t e pairs of a xapping have no intrinsic or Pg prof rm sed size sort split strings tail time touch tr uniq vi wc dering and, most importantly, they can and much, much more... be operated on concurrently. These programs run from the shell orcommand.com under DOS on machines such as Parallel operations on xappings are in the IBM PC, XT, and AT, the AT&T 6300, and most PC compatibles. Full documentation troduced by an operator, a, that carries is included. Phone support is available 9-6 EST. Not copy protected. the meaning "apply to all." For example, a square {x-+2y-+3z—4} would apply the Everything for only $139. function square to the value of each pair Mortice Kern Systems Inc. in the xapping, yielding the new xapping 43 Bridgeport Road East, Waterloo, Ontario, Canada N2J 2J4 {x-*fy-*9z-»-i6'}.Assuming that enough For information or ordering call collect: (519) 884-2251 Prices quoted in U:S. funds. MasterCard and VISA orders accepted. OEM and dealer inquiries processors are available, all of the squar- invited. UNIX is a trademark of Bell Labs. MS-DOS is a trademark of Microsoft Corp. ings can be done simultaneously. CIRCLE 16 ON READER SERVICE CARD 25 Reflecting so, I feel perfectly safe in predicting that Perspective, vol. 2:435-465, The MIT Press, In this resume of LISP daydreams, the most of them will not survive, simply be 1979. prize for unfettered imagination goes to cause there is not room enough in the 10. Pugh, John R. "Actors—The Stage is Brian Cantwell Smith of the Xerox Palo world for dozens of LISP-like languages. Set." SIGPLAN Notices, vol. 19, no. 3 (Mar. Alto Research Center and Stanford Uni PLANNER and CONNIVER are al 1984):61-65. 11. Hewitt, Carl. "The Apiary Network Ar versity. Smith has proposed an "intro ready extinct. The Actors model, on the chitecture for Knowledgeable Systems." Con spective" dialect of LISP. Its most other hand, has children and grandchil dren in abundance—so many, in fact, ference Record of the 1980 LISP distinctive feature is that it requires an Conference. Association for Computing Ma infinite number of interpreters, all run that they may suffocate their parent. An chinery (reprinted). (1985):107-117. ning simultaneously.20 Furthermore, other LISP derivative, Logo, seems to 12. Bobrow, Daniel G., and M. J. Stefik. The Smith and others have shown that such have the opposite problem: Logo itself is LOOPS Manual. Xerox Corp. 1983. an ungainly monster can actually be thriving, but it may prove to be sterile. If 13. Weinreb, Daniel and David Moon. "Fla vors: Message Passing in the Lisp Machine." built and made to work efficiently. parallel hardware ever catches on, at When an ordinary LISP program is least one of the parallel LISPs will surely Artificial Intelligence Memo No. 602, MIT, Cambridge, Mass., Nov. 1980. being run by an interpreter there are succeed. As for 3-LISP, I would not dare 14. Betz, David. "An XLISP Tutorial." Byte. three levels of active computation: the to guess its fate. The one sure winner (Mar. 1985):221-236. user program is running, the interpreter among the new LISPs is Scheme, which 15. Sussman, Gerald Jay, and Guy Lewis is running, and the machine that serves already has a secure place in the aca Steele Jr. "Scheme: An Interpreter for Ex as host to the interpreter is running. But demic world and bright prospects tended Lambda Calculus." Artificial Intelli suppose the interpreter itself is written in elsewhere. gence Memo No. 349, MIT, Cambridge, LISP. Then the interpreter can start an Meanwhile, LISP itself lumbers on, al Mass., Dec. 1975. other copy of itself, which can start an 16. Rees, Jonathan, and William Clinger, ed. though its days of carefree oat-sowing "Revised3 Report on the Algorithmic Lan other copy, which can start....In this way are probably over. LISP programs are no guage Scheme." Artificial Intelligence Memo we build an infinite tower of interpreters. longer without economic value and the No. 848a, MIT, Cambridge, Mass., Sept. Why would anyone want to do such a demand today is for portable, produc 1986. thing? Smith points out that an inter tion-quality, industrial-strength compil 17. Gelernter, David, Suresh Jagannathan, preter often has information about the ers. The urge to standardize seems and Thomas London. "Environments as First state of a computation that is not direct irresistable. Presumably, the standard Class Objects." Proceedings of the Fifteenth ly available to the interpreted program. will be Common LISP,23 although not ev ACM Symposium on Principles of Program A debugger uses such information when eryone is happy with that choice.24 Per ming Languages. Jan. 1987 (in press). 18. Halstead, Robert H. Jr. "Multilisp: A it displays a trace of procedure calls or haps the eventual outcome will be lists the contents of a stack. Language for Concurrent Symbolic Computa something like this: Common LISP will tion." ACM Transactions on Programming become the FORTRAN of Al, while an Many languages (including versions of Languages and Systems, vol. 7, no. 4 (Oct. LISP) provide ad hoc mechanisms for other language—probably Scheme or a 1985):501-538. gaining access to the interpreter's inter descendant of Scheme—becomes the new 19. Steele, Guy L. Jr. and W. Daniel Hillis. nal knowledge. Smith has created a new culture medium, the new breeding "Connection Machine Lisp: Fine-Grained language, 3-LISP, incorporating a sys ground for linguistic innovation. H Parallel Symbolic Processing." Proceedings tematic and well-structured method for of the 1986 ACM Conference on Lisp and allowing programs to reflect on their own References . (Aug. 1986):279- execution. 3-LISP is a descendant of 1. Steele, Guy Lewis Jr., and Gerald Jay Suss 297. man. "The Revised Report on Scheme: A 20. Des Rivieres, Jim, and Brian Cantwell Scheme, with certain additional features; Dialect of LISP." Artifical Intelligence Memo Smith. "The Implementation of Procedurally in particular, a procedure can be desig No. 452, MIT, Cambridge, Mass., Jan. 1978. Reflective Languages." Proceedings of the nated either simple or reflect. A simple 2. Wand, Mitchell. "What Is LISP?" Ameri 1984 ACM Symposium on LISP and Func procedure executes at its own level, but a can Mathematical Monthly. (Jan. 1984):32- tional Programming. (Aug. 1984):331-347. reflective one passes its code up to the 42. 21. Friedman, Daniel P., and Mitchell Wand. next higher level in the tower to be 3. Hewitt, Carl. "PLANNER: A Language "Reification: Reflection without Metaphys executed. for Proving Theorems in Robots." Proceed ics." Proceedings of the 1984 ACM Sympo The trick is to avoid creating an infi ings of the International Joint Conference on sium on LISP and Functional nite tower of interpreters. In fact, a finite Artificial Intelligence. (1969):295-301. Programming.(Aug. 1984):348-355. 4. Kowalski, Robert. "The Origins of Logic 22. Wand, Mitchell, and Daniel P. Friedman. tower will suffice, provided there is some "The Mystery of the Tower Revealed: A Non- level in the hierarchy above, which only Programming." Byte. (Aug. 1985):192-193. 5. Robinson, J. A., and E. E. Sibert. "LOG- Reflective Description of the Reflective nonreflective procedures are being ex LISP: Motivation, Design and Implementa Tower." Proceedings of the 1986 ACM Con ecuted; the interpreters above this level tion." . Academic Press, ference on LISP and Functional Program will never be called on explicitly, so they 1982, pp. 299-313. ming. (Aug. 1986):298-307. can be omitted. Daniel P. Friedman of 6. Kornfeld, William A. "Pattern-Directed In 23. Steele, Guy L. Jr., et al. Common LISP: Indiana University and Mitchell Wand vocation Languages." Byte. (Aug. 1979):34- The Language. Digital Press, 1984. of Northeastern University have since 48. 24. Allen, John R. 'The Death of Creativity: shown that all the properties of the infi 7. Sussman, Gerald Jay, Terry Winograd, and Is Common LISP a LISP-like Language?" Al Expert, vol. 2, no. 2. (Feb. 1987):48-61. nite reflective tower can be emulated by Eugene Charniak. "MicroPLANNER Refer ence Manual." Artificial Intelligence Memo a single interpreter written in Scheme.21,22 No. 203A, MIT, Cambridge, Mass., 1971. 8. Sussman, Gerald Jay, and Drew V. McDer- Predicting mott. "Why Conniving Is Better Than Plan In nature most mutations are unhelpful, ning." Artificial Intelligence Memo No. if not disastrous. However, in nature mu 255A, MIT, Cambridge, Mass., 1972. tation is a random process, whereas the 9. Hewitt, Carl. "Control Structure as Pat mutant LISPs described here have been terns of Passing Messages." Artificial Intelli carefully and deliberately designed. Even gence, vol. 8, no. 3 (June 1977):323-363. Also published in Artificial Intelligence: An MIT

26 COMPUTER LANGUAGE ■ APRIL 1987