Metaobject protocols: Why we want them and what else they can do Gregor Kiczales, J.Michael Ashley, Luis Rodriguez, Amin Vahdat, and Daniel G. Bobrow Published in A. Paepcke, editor, Object-Oriented Programming: The CLOS Perspective, pages 101 ¾ 118. The MIT Press, Cambridge, MA, 1993. © Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. Metaob ject Proto cols WhyWeWant Them and What Else They Can Do App ears in Object OrientedProgramming: The CLOS Perspective c Copyright 1993 MIT Press Gregor Kiczales, J. Michael Ashley, Luis Ro driguez, Amin Vahdat and Daniel G. Bobrow Original ly conceivedasaneat idea that could help solve problems in the design and implementation of CLOS, the metaobject protocol framework now appears to have applicability to a wide range of problems that come up in high-level languages. This chapter sketches this wider potential, by drawing an analogy to ordinary language design, by presenting some early design principles, and by presenting an overview of three new metaobject protcols we have designed that, respectively, control the semantics of Scheme, the compilation of Scheme, and the static paral lelization of Scheme programs. Intro duction The CLOS Metaob ject Proto col MOP was motivated by the tension b etween, what at the time, seemed liketwo con icting desires. The rst was to have a relatively small but p owerful language for doing ob ject-oriented programming in Lisp. The second was to satisfy what seemed to b e a large numb er of user demands, including: compatibility with previous languages, p erformance compara- ble to or b etter than previous implementations and extensibility to allow further exp erimentation with ob ject-oriented concepts see Chapter 2 for examples of directions in which ob ject-oriented techniques might b e pushed. The goal in developing the MOP was to allow a simple language to 1 b e extensible enough that all these demands could b e met. Traditionally, languages have b een designed to b e viewed as blackbox abstractions; end pro- grammers have no control over the semantics or implementatation of these abstractions. The CLOS 1 Some might argue that the base CLOS language is not so simple. What is more interesting is to consider how much of what is in that language could b e dropp ed given the current or an improved metaob ject proto col, now that wehave a b etter handle on that technology. Among those features that would b e prime candidates for elision would b e: metho d combination, the :argument-precedence-order option, metho ds on individual s eql sp ecializers and class variables. More aggressive prop osals might drop asp ects of the initiali zatio n proto col like slot- lling initargs and default initargs. Even more aggressive prop osals might drop multiple inheritance and multi-metho ds. 1 MOP on the other hand, \op ens up" the CLOS abstraction, and its implementation to the pro- grammer. The programmer can, for example, adjust asp ects of the implementation strategy suchas instance representation, or asp ects of the language semantics suchasmultiple inheritance b ehavior. The design of the CLOS MOP is such that this op ening up do es not exp ose the programmer to arbitrary details of the implementation, nor do es it tie the implementor's hand unecessarily | only the essential structure of the implementation is exp osed. In more recentwork, wehave pushed the metaob ject proto col idea in new directions, and wenow b elieve that idea is not limited to its sp eci c incarnation in CLOS, or to ob ject-oriented languages, or to languages with a \large runtime" or even to Lisp-like languages. Instead, we b elieve that providing an op en implementation can b e advantageous in a wide range of high-level languages and that metaob ject proto col technology is a p owerful to ol for providing that p ower to the programmer. The purp ose of this chapter is to summarize what has b een learned to date ab out MOP tech- nology, and suggest directions it might b e pursued in the near future. In the rst two sections, we start with the CLOS MOP, rst summarizing the motivation for it, and then giving a brief overview of how one might think ab out its development. We then present a general framework for thinking ab out MOP design, together with some early principles for design cleanliness in a MOP. Finally, we presentanoverview of three new MOPS wehave designed for Scheme, showing that a MOP can b e used to handle a range of issues other than those in the CLOS case. Motivating Examples In this section, we presenttwo simple problems that arise in languages like CLOS without a MOP,asaway of summarizing the motivation for the CLOS MOP.We also generalize from these examples, as a way of suggesting what other kinds of problems an op en language implementation might b e used to solve. The rst example has to do with p erformance. In it, the programmer has written a program which CLOS expresses quite well, but nds p erhaps to their surprise that the underlying language implementation do esn't p erform adequately. In the next section we return to this example to showhow the CLOS MOP addresses this problem by op ening up part of the implementation strategy to the programmer. The second example, is a case where the programmer can b e well-served by b eing able to adjust some part of the language semantics to b etter suit their needs. APerformance Problem Consider the two CLOS class de nitions shown in Figure 1. The class position might b e part of a graphics application, where the instances are used to represent the p osition of the mouse as it moves. The class de nes two slots x and y. In this case, the b ehavior of the program is such that there will b e a very large numb er of instances, b oth slots will b e used in every instance and access to those slots should b e as fast as p ossible. The second de nition, person, might come from a knowledge representation system, where the instances are b eing used as frames to represent di erent individuals. The thousand slots de ned in the class corresp ond to a thousand prop erties of a p erson that mightbeknown. In this application, the b ehavior is such that although there will b e a very large numb er of instances, in any given instance only a few slots will actually b e used. Furthermore, access to these prop erties will rarely b e in the inner lo op of a computation. 2 (defclass position () (defclass person () (x y)) (name age address ...)) many instances, many instances, both slots always used only a few slots used in any one instance array-like representation hash-table like representation Figure 1: Two sample CLOS class de nitions. Ideally, each class would like a di erent underlying instance implementation strategy. In a traditional sans MOP CLOS implementation, one or the other of these classes probably person will have bad p erformance. Clearly, the ideal instance implementation strategy is di erent for the two classes. For position, an array-like strategy would b e ideal; it provides compact storage of instances, and rapid access to the x and y slots. For person, a hash-table like strategy would b e more appropriate, since it isn't worth allo cating space for a slot until it is known that it will b e used. This makes access slower, but it is a worthwhile tradeo given a large numb er of instances. The likely default implementation in most ob ject-oriented languages is the array-like strategy. This serves the author of the position class quite well, but author of the person will not b e so happy.Even though the CLOS language abstraction serves to express their program quite clearly, supp osedly hidden prop erties of the implementation will impair p erformance. This particular problem is not an isolated incident, it is an instance of a common, and much more general problem. As a programming language b ecomes higher and higher level, its imple- mentation in terms of the underlying machine involves more and more tradeo s, on the part of the implementor, ab out what cases to optimize at the exp ense of what other cases. That is, the fact that a typical ob ject-oriented language implementation will do well for position and p o orly for person is no accident | the designer of the implementation made what we hop e was a concious decision to tune their implementation this way. What a prop erly designed op en language imple- mentation do es is allow the end-programmer to go back, and \re-make" one of those tradeo s to b etter suit their needs. A more complete discussion of this kind of p erformance problem, and the ways in which op en language implementations based on MOPs can help address it is given in [1]. The Desire to Customize Behavior As an example of how a programmer can derive b ene t from b eing able to adjust the language's semantics, consider the case of a programmer who is p orting a large b o dy of Flavors [2] or Lo ops [3] co de to CLOS.