sign in sign up
<<
Home , Subtyping, Type system

A Core Calculus of

Sam Tobin-Hochstadt Eric Allen

Sun Microsystems Laboratories Burlington MA 01803 .@sun.com

Abstract of class Dimension, then Length would not be a class and so 3 feet could not be an instance of Length. Al- Metaclasses provide a useful method of abstraction for ternatively, we might define Length to be a subclass of programmers working with object-oriented languages, Dimension. But then we still cannot define 3 feet to but they have not seen the formal exploration applied be an instance of Length because anything that is an to more conventional object-oriented programming fea- instance of Length would also be an instance of class tures. In order to elucidate the structure of metaclasses Dimension and 3 feet is obviously not a dimension. and their relationship with static typing, we present a The fundamental limitation of conventional object- core calculus for a nominally-typed object-oriented lan- oriented languages is that they do not allow a single guage with metaclasses and prove type soundness over concept to serve both as a class and as an instance. In this core. This calculus is presented as an adaptation of [18], Welty and Ferrucci lay out a comprehensive case Featherweight GJ [13], and is powerful enough to cap- that this limitation prevents conventional modeling lan- ture relationships beyond what are express- guages from capturing many important aspects of the ible in common object-oriented languages, including ar- world. In their primary example, they show the difficul- bitrary metaclass and classes as values. We ties in modeling a simple ontology including the notions also explore the integration of object-oriented design Species and Eagle, while also capturing the relation- patterns with metaclasses, and show how our calculus ship between Eagle and Harry, a particular eagle. One integrates several such patterns cleanly. solution they consider and reject is to separate the no- tion of “eagle” into two concepts: the Eagle and Eagle, 1 Introduction with the former an instance of Species and the latter a class whose instances are specific birds. This approach One of the stated benefits of object-oriented languages is inadequate because it fails to express a relationship is their ability to model aspects of the world in the ob- between the notions the Eagle and Eagle. ject . However, most such languages are un- In order to overcome these limitations in expres- able to model many simple and common relationships. siveness, several object-oriented languages that are not For example, consider the relationship between physical statically typed, such as , Python, and Self, quantities, units of measurement, and physical dimen- allow for more flexible class relationships [11], [14], sions. It is obvious that “length” is a dimension, and [17]. In the case of Smalltalk and Python, classes are that “3 feet” is a length. However, modeling this rela- instances of metaclasses. Static members of a class tionship in a conventional language such as the JavaTM are modeled as ordinary members of the correspond- is impossible without attribut- ing metaclass. But traditional metaclass systems have ing to some of these concepts properties which do not placed important limitations on expressiveness. For ex- properly belong to them. It is natural to model 3 feet ample, in Smalltalk, the metaclass hierarchy is only two as an instance of class Length. But if we were to define levels deep. If we want to represent the relationships “A a class Dimension and define Length to be an instance is an instance of ”, “B is an instance of ”, and “C is an instance of ”, we can only do this in Smalltalk if D is the special Smalltalk class Metaclass. This does Submission for FOOL 2005 not allow the multi-level hierarchies required to model the examples discussed above [11]. Self, on the other hand, is a prototype-based object-oriented language, where there are no classes at all; object instantiation choices. The motivating examples from the introduc- consists of cloning an existing instance. The members tion are presented in MCJ in section 2.3, demonstrating of the clone can be modified and added to at will, mak- the benefits of metaclasses. We discuss in section 3 how ing static checking difficult. the need for many object-oriented “design patterns” is An extremely expressive system of metaclasses, very obviated through the use of metaclasses. We then de- similar to ours, was presented by Cointe in [6]. In his scribe its formal properties in section 4. In section 4.2 paper, he makes a number of persuasive arguments for we provide a formal semantics for the language, and in the benefits of metaclasses. In response to complaints section 4.3 we prove a type soundness result. In section that the Smalltalk80 metaclass system was too com- 5 we consider related work and in section 6 we describe plex, he suggested that a more powerful and flexible conclusions and future directions. system, as he and we present, has the potential to be much simpler. He claims “The ObjVlisp model’ pri- 2 MCJ mary advantage is uniformity.” We agree, and believe that the treatment of metaclasses presented here unifies 2.1 Overview both disparate metaclass systems, and different ad-hoc mechanisms describing the properties of classes as ob- MCJ is a generically-typed object-oriented calculus jects in languages without metaclasses. based on Featherweight GJ that includes metaclasses. Cointe also quotes from [4]: “With respect to , An MCJ program consists of a sequence of class defi- Smalltalk also abandons ... strong typing, allowing it nitions followed by a trailing expression. This trailing to ... introduce the notion of meta-classes”. In this expression is evaluated in the context of the class table paper, we show that such a compromise is not necessary, induced by the class definitions. Before discussing the and that static typing can coexist with the flexibility of formal specification of MCJ, we first give a high-level metaclasses. overview of the nature of MCJ class definitions. We present and formalize a statically typed calcu- Each class is an instance of a metaclass. A metaclass lus with nominal for metaclasses where every is either a user-defined class or class Object. We say that class can serve as both an instance of a class and as if a class C is an instance of a class D, then C is an instance a class of instances, and there is no limitation on the class of D, and that D is the immediate containing class, levels of nesting of metaclasses. Our work is motivated or , of C. If class is a superclass of D then we by [2], which presents a system for integrating the static also say that C is an instance class of E and that E is a checking of dimensions of physical quantities with a con- containing class of C. ventional object-oriented . In that system, There are two key distinguishing features of MCJ. metaclasses are used extensively to model the type re- First, a class (or an instantiation of a generic class) can lationships of dimensions. The notion of metaclass in be used as an expression. Second, all classes have both that system is more general than that of languages such a superclass and a kind. Instances of a class inherit as Smalltalk, where each class has a unique metaclass behavior from the superclass. The class itself, when that defines its class members. In this paper, we formal- used in an expression context, inherits behavior from ize the core of that system related to metaclasses and its kind. prove it sound. In the process, we discover several in- A class definition consists of a header plus a collec- teresting properties about the structure that a nominal tion of fields and methods associated with the class. type system for metaclasses must take. The header names the class and specifies the type pa- In the interests of economy and clarity of presen- rameters, as well as the superclass and the kind. For tation, in the following discussion we restrict ourselves example, the following header declares a class C with to simple examples, and do not delve in the the com- superclass Object and kind D and no type parameters: plexities of object-oriented analysis, or of dimension class C kind D extends Object {...} types. This is certainly not because we believe that metaclasses are most useful for the development of toy Each class may have both class members and in- object-oriented programs. On the contrary, metaclasses stance members. Class members define the behavior are likely to show their value in large systems, both in of the class when used as an instance. Instance mem- terms of modeling and in the treatment of design pat- bers define the behavior of instances of a class. In the terns in section 3. abstract syntax presented in this paper, class members The remainder of this paper is organized as follows. are distinguished from instance members by order, and In section 2 we present a core calculus for metaclasses with the keyword class 1. Members of a class definition we dub MCJ. We introduce the language and describe 1This syntax is not ambiguous since MCJ has no inner classes. its key features, as well as some of the important design

2 are laid out as follows: first, class fields are defined, fol- may be a . Unlike the system presented in lowed by class methods, then instance fields and finally [1], MCJ does not support first-class genericity because instance methods. For example, the following class defi- a naked type variable must not appear in the extends nition includes a class method m and an instance method clause or kind clause of a class definition and new ex- n: pressions on naked type variables are not supported. Also, polymorphic methods (which are orthogonal to class C kind Object extends Object { the features we explore) are not supported. class Object m() {...} Because classes can be used as expressions, we need Object n() {...} a bound on the behavior of classes when used in ex- } pression contexts. Therefore, we place two bounds on every type variable in the header of the class defini- Class fields and methods bear a resemblance to static tion. The first bound on T is a bound on the kind of an fields and methods, but there are important differences. instantiation of T; the second bound is a bound on the For a given class C, instance members of C’s kind are superclass of T. For example, the following class header inherited as class members of C. For example, a class declares a class C with one type parameter T that must method m in C with the same name as an instance be instantiated with a type of kind D that is a subtype method of C’s kind overrides the definition of m in the of E: kind (and it must have the same signature as the over- ridden method). Note that if an instance class C of class C type T is assigned to a variable v of type T, references kind Object to the instance members of v refer to the class mem- extends Object {...} bers of C. As in [2], a class is allowed to define both a static method and an instance method of the same 2.2 Key Design Points name. We prevent ambiguity in member references by requiring that all references to class members in MCJ Private Constructors One benefit of a metaclass must explicitly denote the receiver. system is that constructors no longer need to play such Superclasses behave as in Featherweight GJ, pro- a central role in the language. Class instances exist at viding implementation inheritance of instance behavior the beginning of a program execution; they need not and subtyping. A class, when used as an expression, be constructed. Non-class instances are naturally con- is an instance of its kind. If a method is invoked on structed with class (factory) methods [10]; in this way a class C, first the class methods of C are examined. If a class instance can be passed as a type parameter and the method is found there, it is invoked. Otherwise, the factory methods can be called on the class instance to instance methods of the kind D of C are searched, and construct new instances without stipulating the class of if it is found there, the method is invoked. Otherwise, the constructed instance. This flexibility is similar to the superclass hierarchy of D is examined as it is for an that provided with first-class genericity [5, 1], but obvi- ordinary instance of D. ates the need for with clauses because the bound on the For example, consider the following class definitions: kind of a type variable stipulates what factory methods can be called on it. A class can define class methods class A kind Object extends Object {...} with arbitrary signatures that return new instances of class B kind Object extends A {...} the class, but each such method must ultimately in- class C kind B extends Object {...} cludes a new expression, which can occur only within the scope of the class whose instance it returns. A new To resolve the method call C.m() first the class meth- expression looks like a call to method named new, that ods of C are examined, and if a match is found, it is takes a single argument for each type parameter and invoked. Then the instance methods of B and then A each instance field of the syntactically enclosing class. are examined. The result of a new expression is a new instance of the If the method call were instead b.m(), where b de- enclosing class whose type parameters are instantiated notes an instance of B, then the instance methods of with, and whose fields are initialized with, the given ar- first B then A would be examined. guments. In our formal semantics, new expressions are The generic type system of MCJ is similar to the annotated with the name of the enclosing class. These generic type system in Featherweight GJ. However, as annotations need not be added by a programmer; they in [5, 2], we allow for type variables to occur in type- could be added easily by a straightforward syntactic dependent contexts such as casts, preventing the use of preprocessing over the program, since they can be de- type erasure as an implementation technique. In ad- termined merely by the lexical scopes of the new expres- dition, the receiver of a reference to a class member sions.

3 All uses of constructors in Featherweight GJ [13] are namic type of a object. However, despite the name sim- -expressible [9] in MCJ, that is, they can be ex- ilarity, these functions bear no relationship to the oper- pressed via local transformations. Of course, not all ator described here. In MCJ, an application of Featherweight GJ programs can be expressed in MCJ, is a static type reference. because of the differences in the type system. Namely, MCJ does not include polymorphic methods. Non-transitivity The standard subclassing relation- Metaclasses allow other kinds of flexibility in object ship, like subtyping, is transitive. That is, if B is sub- creation, as discussed in section 3. class of C and C is a subclass of D, then B is a sub- class of D. However, this relationship does not hold for Fields and Initialization To ensure , we instanceof relationships. Therefore, if if B is an instance must have an initial value for every field, or prevent of C and C is an instance of D, then no judgment about fields from being used before they are initialized. For the relationship of B to D can be inferred. instance fields of objects created with a constructor, this is achieved by requiring that the constructor have an Cyclic class hierarchies One property of meta- argument for every instance field. Combined with the classes that we have not yet discussed is that cycles can solution for flexibility in object creation outlined above, occur among instance class relationships. The simplest this allows us the simplicity in semantics of FGJ, and such cycle is: the flexibility of more general constructors. However, classes which are themselves instances of class C<> : C<> / Object other classes have fields as well, and these fields must {} also be initialized before they are used. These fields Although this pathology appears to be dangerously include those instance fields of the kind which become close to Russell’s paradox, it does not lead to inconsis- class fields of the new class. Our solution is to require tencies. Class C is an instance of itself. It can be used that all class fields, including those obtained by inheri- in any context where one of its instances can be used. C tance, must be redeclared and provided with values. can be thought of as a self-replicating value. This class hierarchy causes no problems for method lookup be- cause method resolution never proceeds through more typeOf Given that class references can be used as ex- than a single containing class to an instance class. pressions, it is natural to ask: what is the type of a Therefore, we see no reason to disallow it. class reference? In the world of the λ-calculus, such types are known as kinds. In MCJ, however, the kind of a class does not capture all of the properties of the 2.3 Examples, Reprised class as a value. For example, a class may add new Having seen the outlines of MCJ, we return to the mo- class fields or class methods which are not present in tivational examples discussed earlier. First, the rela- the kind. Therefore, each class freely generates a new tionships between dimensions is easy to capture. 2 We type, which is the type of the class considered as a value. define We represent this type with the typeOf operator, which is produces a type that is not also a class. Since this class Dimension kind Object extends Object { ... } type is not a class, it cannot be the superclass or kind class Length kind Dimension extends Object { ... } of another class, and it cannot serve as the instantiation class Meter kind Length extends Object { ... } of a generic type parameter. These new types can create complex relationships Here Meter is a singleton class (there is only one Me- between classes and types. For example, the following ter, in the Platonic sense). We have easily expressed class headers: the desired type relationships, and can statically check program invariants that rely on dimensional relation- class A() kind Object extends Object {...} ships. class B() kind A extends Object {...} The example from [18] is also easily expressed:

induce the following type relationships: class Species kind Object extends Object { ... } class Eagle kind Species extends Object { B : typeOf[B] class Eagle make(String name) { ... } typeOf[B] <: A } B instanceof A Eagle harry = Eagle.make("Harry") 2 The name typeOf has been used in other contexts Of course, dimensions have many subtleties, not captured here. All we present is the essence of the type relationships. to refer to a runtime operation that determines the dy-

4 Here we simply create an object to represent our con- that the concept of a Factory method is easily (and crete instance of an Eagle. The naive implementation necessarily) expressed in MCJ. However, a number of of the above code in a conventional OO language would other patterns also find easy expression. The Abstract look something like this: Factory pattern is a simple application of inheritance in the metaclass hierarchy. The Prototype pattern is, class Species extends Object { ... } as mentioned in the Design Patterns book, supported class TheEagle extends Species { ... } in languages with metaclasses. Finally, there is no need class Eagle extends Object { ... } for the Singleton pattern, since the class itself can serve Eagle harry = new Eagle("Harry") as a singleton. While this is possible in the Java Pro- gramming Language and in C++, it is undesirable since There are at least two substantial problems with this the static behavior that would need to be used cannot code. First, TheEagle and Eagle have no relationship inherit from another class. In fact, the inflexibility of to each other in the type system. One is a singleton class or static operations is one of the rationales cited class, representing a particular species, and one is a for the Singleton pattern. name for a of objects, being the members of that Seeing that a number of patterns can be expressed species. The fundamental nature of species has a two- quite simply in MCJ, the question arises: is the simple level containment relationship, which the type system expression of these patterns a benefit? We argue that is failing to express. This means that our type checker it is. The existence of design patterns is a sign of short- cannot determine what we want, and cannot help us comings in language design. Fundamentally, a design avoid mistakes. pattern is an abstraction that cannot be expressed in Another problem, which is in some ways just a symp- the language. For example, the Singleton and Visitor tom of the first problem, but which bears special atten- patterns are abstractions that have obvious invariants, tion, is the use of generic types. Imagine the following but they cannot be expressed directly and they require method, where T is a type parameter in scope:3 complex cooperation from many parts of the system. foo(T x) { ... } Sophisticated macro systems, such as those found in [16] or Scheme [8] allow for expression If we want to perform the call foo(harry), then the of many forms of abstraction, as do extremely flexible bound on T must be either Object, in which case foo object systems such as CLOS. However, these solutions must have no knowledge of the object, or Eagle, in add significantly to what a programmer must under- which case the function cannot handle multiple Species. stand in order to use the language productively. The “solution” in conventional languages is to use a An alternative solution explored in this paper is bound of Object and insert a downcast, which fails at to provide additional flexibility in the language, which runtime if the wrong argument is passed. In MCJ T can does not allow for arbitrary additional abstraction, but be bound by kind Species, and instantiated with Eagle, instead makes the abstractions commonly needed easy allowing the original call to be type-checked. This gives to express. One does not need the full power of macros the programmer both safety and expressiveness. to encode the Factory pattern. In MCJ, we have pre- sented a language where several kinds of patterns are 3 Design Patterns as Language Features easy to express, without adding an additional language on top of the original. Design Patterns, as exemplified in [10], have had a very substantial impact on the world of object-oriented pro- 4 Formal Specification of MCJ gramming and beyond. Design patterns allow program- mers to increase the flexibility and abstraction of their Having outlined the motivation for metaclasses, and the software designs. However, few design patterns have of their use, we now turn to a formal exposition been integrated into programming languages. of the syntax and semantics of MCJ, followed by an One of the advantages of our metaclass framework outline of the proof of soundness for the calculus. is that it allows clean expression of several patterns in the language, rather than adding additional abstrac- 4.1 Syntax tion techniques on top of the language. For example, a number of the classic design patterns deal with object The syntax of MCJ is given in Figure 1. When describ- creation, as discussed above in section 2.2. It is clear ing the formal semantics of MCJ, we use the following metavariables: 3Generic methods would be straightforward to add to MCJ, but we excluded them to simplify the presentation of the type Expressions or mappings: e, d, system, and since they do not add complexity to the proofs. •

5 CL : := class C : A / A class T e; M; T f; N; class declaration { } N : := T m (T x) return e; method declaration M : := class T m{ (T x) return} e; class method declaration { } e : := x variable reference e.f field access | e.m(e) method invocation | newC(e) instance creation | (T)e cast | R type reference | Φ mapping | G T : := R non-typeOf type typeOf[R] typeOf application | R : := X type variable A | A : := C type application Object | Figure 1: MCJ Syntax

Field names: f, g sequences of methods and fields are free of duplicates. • Second, there is an implicit well-formedness constraint Variables: x • on programs that no class be a superclass of itself, ei- Method names: m ther directly or indirectly (however, as discussed below, • cycles in the kind hierarchy are allowed). Third, we Values: v assume that is never used as the name of a vari- • this Method declarations: M, N able, method or field. We also take Object to be a • distinguished member of the hierarchy, with no specific Type names: C, D definition, which does not have a superclass or a kind. • For example, the simplest MCJ class is: Types: I, J, , T, U, V, W • class C<> : Object / Object Non-typeOf Types: O, P, Q, R, S {} • This is a class named C, with no type arguments, Types that are either Object or a type application: with kind Object and superclass Object, which has no • A, B fields or methods whatsoever. In examples that follow, we omit empty <>. Note that both the kind and the su- Ground types (contain no type variables): G perclass are Object. MCJ does not have a distinguished • class Class. Interestingly, we determined that a class Type variables: X, Y, Z • Class would need no consideration from the type sys- Mappings (field name value): Φ tem. Since both Class and Object would sit atop the • 7→ hierarchy with no contents, we determined there was As in Featherweight Java and many other works, x no need to include both of them. In a more substantial stands for a possibly-empty sequence of x.[X S] language, where Object might have methods or fields, 7→ denotes a substitution of the S for the X, which can be there might be a use for a distinguished Class class. applied to either an expression or a type, and which can However, it would not need any special treatment by substitute either type or expression variables. the type system. A number of symbols are used to abbreviate key- A more complicated MCJ class is:4 words: / stands for extends and : stands for kind. Also @ represents concatenation of sequences of syntactic class Pair for the definition of T. kind Object extends Object A number of restrictions on MCJ programs are im- 4Here we use extends for / and kind for :. plicit in the formal rules. First, we assume that all

6 { each new expression evaluates to an instance of the syn- class Pair make(A a, B b){ new(A,B) (a,b) } ;tactically enclosing class.5 Had there been additional A fst; fields in the superclass of Pair, it would have been nec- B snd; essary to initialize them in the new expression as well. Pair setfst(B b){ new expressions are explained in detail in section 2.2. return new (b, this.snd); }; 4.2 Semantics } There are two forms that a value can take in MCJ: that This class specifies a polymorphic pair data struc- of an instance class, and that of a conventional (non- ture. It also demonstrates a number of important MCJ class) value. If we were to distinguish these two forms features. First, we see two different kinds of methods. of value, we would significantly increase the number of The make method creates a new instance of Pair. This rules necessary to describe our semantics because every method contains a new expression, which initializes the rule referring to a value would have to be written twice. instance fields of the class positionally, so that the first To avoid this complexity, we introduce a special map- argument to make becomes fst, and the second becomes ping construct (similar to a record value) to denote the snd. Also, note that we must mention the type param- results of computations.6 A mapping takes field names eters of the newly created instance. to expressions, and is annotated with a ground type. Second, we have an instance method, setfst, which Mappings from a sequence of fields f to a sequence of creates a new pair with the old second element, and a expressions e with type G are written f e . A map- { 7→ }G new first element which has the same type as the second ping denoting an instance class C consists of a sequence element. of class fields mapped to a sequence of expressions and Finally, adding the following to the above definition is annotated with the type typeOf(C). Note that the of Pair gives a full MCJ program: right hand sides of such maps need not always be val- ues, and thus computation can take place inside of a class O kind Object extends Object mappings. Mappings are not available to the program- { mer, and thus can only be created by operation of the class O make(){return new () ()}; reduction rules. Therefore, unlike Featherweight Java, } our reductions do not operate entirely in the expressible syntax of the language. We use the metavariable Φ to Pair.make(O.make(), O.make()).fst range over mappings. When we need to refer to the type annotation of a mapping explicitly, the metavariable is This program, which creates two instances of O, then written ΦG. creates a Pair to hold them both, and finally selects one The semantics are given in figures 2, 3 ,4, and 5 with of the two to become the value of the program, demon- auxiliary functions given in figure 6. strates three of the expression forms in MCJ. Pair is a type, which, by itself, can be a value. O.make() is a 4.2.1 Typing method invocation (here with a class, not an instance, as the receiver). The entire expression is a field access, Rules governing the typing of MCJ programs are given of the fst instance field. in figures 2, 3 and 4. The bound∆ function is defined as Other kinds of expressions are seen in the body of follows: the Pair class. new (a,b) is a new expression, which creates an instance of the enclosing class (here bound∆(X) = ∆(X) Pair). In the body of make is a reference to the variable bound∆(typeOf[X]) = ∆(typeOf[X]) b, which has the obvious value. The final directly ex- bound∆(S) = S pressible expression is the cast, which works as follows: This function maps type variables and typeOf ap- (Object)O.make() is an expression of type Objectthat plied to type variables to their bounds, and leaves oth- evaluates to an instance of O. ers unchanged. With an understanding of expressions, we can exam- The metavariables ∆ and Γ range over bounds envi- ine the rest of the Pair class. There are two methods: a ronments, written X / S and type environments, written class method (make) and an instance method (setfst). There are also two fields, both of which are instance 5In the grammar of figure 1, new expressions are annotated with this enclosing class. In the examples, this redundant infor- fields and initialized the new expressions. In MCJ, a new mation is elided. expression is different from those in Featherweight GJ; 6This simplification was suggested by Jan-Willem Maessen in response to an earlier draft of the MCJ semantics.

7 x : T respectively. A bounds environment contains two 4.2.3 Evaluation bounds for each type variable, so that if ∆ = X <:(A, B), The evaluation rules for MCJ are given in figure 5. The then ∆(X) = A and ∆(typeOf[X]) = B. evaluation relation, written e e , states that e tran- The notation A <: B means A is a subtype of B. Sub- 0 sitions to e in one step. The reflexive→ transitive closure typing judgments are made in the context of a bounds 0 of this relation is written e ∗ e0. environment, which relates a type to the declared bound → of that type from the class header. Type judgments are of the form ∆; Γ e : T, which Bad Casts There is one way that evaluation in MCJ states that in bounds environment ∆ and` type envi- can get stuck: the bad cast. This problem occurs for all ronment Γ, expression e has type T. Type judgments languages that allow static downcasts. A expression e are not transitive in the way that instanceof relation- is a bad cast iff e = (S)ΦT where S is not a supertype ships are with respect to subtyping: if ∆; Γ e : T, of T. and ∆ T <: S, it is not necessarily the case` that The evaluation relation given here is non- ∆; Γ e`: S. This is important, since our proofs de- deterministic. For example, no order is prescribed pend` on having unique derivations of a given typing for evaluating the arguments to a new expression judgment. We represent empty environments with or method call, or for evaluating the initializers for and we abbreviate judgments of the form ; e : T∅ class fields. There are also a number of places where and T <: S as e : T and T <: S respectively.∅ ∅ ` either congruence or reduction rules can be applied. We∅ ` now discuss several non-obvious aspects of our Therefore, in the presence of non-termination or bad type system resulting from the need to statically check casts, the results may differ depending on evaluation uses of metaclasses. order. However, confluence can be regained simply by requiring that [C-Mapping] is applied whenever possible. This restriction, combined with the require- Casts and Mappings We follow [1] in typing all ment in the premise of [R-New] that the arguments casts as statically correct, so as to avoid the compli- be values, ensures that every program with an error cations of “stupid casts”. Mappings, which are not ex- or non-termination will either cause an error, or fail to pressible in the source language, are typed merely by a terminate. well-formedness constraint.

4.3 Type Soundness 4.2.2 Well-Formedness With the above definitions, we are now able to turn There are three kinds of well-formedness constraints on to a proof of type soundness. Given the simplicity we MCJ programs. First, type well-formedness, written are able to achieve in the definitions, the proof is not “∆ T ok”, states that type T refers to a defined class, significantly more complicated than the soundness proof and` that if it is a type application, the arguments satisfy given for Featherweight GJ [13]. However, there are a the bounds. number of significant lemmas, of which we list the most More important are the class and method well- important: formedness constraints, which together determine if a class table is well-formed, and thus part of a legal MCJ Lemma 1 (Fields are Preserved by Subtypes) program. Method well-formedness is a judgment of the If fields(U) = F and V <: U then G where form “M ok in G”, where M is a method. We make use fields(V) = F @ G. ∅ ` ∃ of the latter portion (G) of this judgment as a second argument, allowing us to use this rule to check both Proof We prove this by induction over the derivation instance and class methods. Method well-formedness for fields(V). consists mostly in fitting the body to the , and checking for invalid overrides. Case V = Object: Then U = Object and = @ . Class well-formedness involves all of the above ∅ ∅ ∅ checks. All methods must be well-formed, and all class Case V = C: Then CT (C) = fields must have correct initialization expressions. Fur- class C : B / A ...H... { } ther, class fields must contain the instance fields of the Continue by induction on the derivation of kind. Finally, all new expressions must have the correct V <: U. The only interesting cases are [S-Super∅ `] class name annotation. and [S-Trans]. Subcase [S-Super]: [X R]A = U. Thus G = H. 7→

8 X / T ∆ ∆ V <: T ∆ T <: U ∆ T <: T [S-Reflex] ∈ [S-Bound] ` ` [S-Trans] ` ∆ X <: T ∆ V <: U ` ` CT (C) = class C : B / A ... CT (C) = class C : B / A ... { } [S-Kind] { } [S-Super] ∆ typeOf[C] <:[X S]B ∆ C <:[X S]A ` 7→ ` 7→ ∆ X <: T ∆ T ok ∆ Object ok [WF-Object] ` [WF-Var] ` [WF-Typeof] ` ∆ X ok ∆ typeOf[T] ok ` ` CT (C) = class C : B / A ... ∆ S <:[X S]I ∆ typeOf[S] <:[X S{]J } ∆ S ok ` 7→ ` 7→ ` [WF-Class] ∆ C ok ` Figure 2: Subtyping and Well-Formed Types

∆ T ok ∆; Γ e : U ∆ S ok ∆; Γ x : Γ(x) [T-Var] ` ` [T-Cast] ` [T-Class] ` ∆; Γ (T)e : T ∆; Γ S : typeOf[S] ` ` ∆; Γ e : U ∆ C ok fields(C) = T f ` ` fields(bound∆(U)) = T f ∆ U <: T ∆; Γ e : U [T-Field] ` ` [T-New] ∆; Γ e.fi : Ti ∆; Γ new (e) : C ` ` C fields(G) = T f mtype(m, bound∆(W)) = U T ∆; Γ e : S ∆ S <: T ∆; Γ r : W ∆; Γ e : V ∆ →V <: U ` ` [T-Mapping] ` ` ` [T-Invk] ∆; Γ f e : G ∆; Γ r.m(e) : T ` { 7→ }G `

Figure 3: Expression Typing

params(G) = X @ (I, J) ∆ = X / I, typeOf[X] / J Γ = x : T, this : G ∆ {T ok ∆ U ok override} (m,{super(G), T U)} ` ∆;` Γ e : W ∆ W <: U → ` ` [WF-Method] U m (T x) return e; ok in G M ok in typeOf[C] N ok{ in C } ∆ = X / I, typeOf[X] / J ∆ B ok ∆ A ok ∆ I ok ∆ J ok ∆{ T ok ∆ W ok } ` ` ` ` ` ` fields(B) T f ∆; e : T0 T0 <: T ⊆new (d)∅ `e, M, N. D =∅ `C ∀ D ∈ [WF-ClassDef] class C : B / A T f e; M; W g; N; ok { }

Figure 4: Well-Formed Constructs

Subcase [S-Trans]: Let T be the intermediate type. fields(B) W h. Then [X S]fields(B) [X ⊆ 7→ ⊆ 7→ Then fields(V) = fields(T)@ H and fields(V) = S]W h, and fields([X S]B) [X S]W h by lemma 0 7→ ⊆ 7→ fields(T)@ H00 by the induction hypothesis. Thus Substitution Distributes over fields. G = H @ H . 0 00 Lemma 2 (Subtyping Preserves Method Typing) Case V = typeOf[C]: Then CT (C) = If mtype(m, U) = T S and V <: U then class C : B / A W h e... . We continue mtype(m, V) = T S. → ∅ ` by induction over the derivation{ of} V <: U. The → ∅ ` only complex cases are [S-Kind] and [S-Trans]. Proof By induction over the derivation of V <: U. ∅ ` Subcase [S-Trans]: As above. Case [S-Reflex]: Trivial. Subcase [S-Kind]: [X S]A = U Then fields(V) = [X S]W h. By well-formedness7→ of C, we know that Case [S-Bound]: Not applicable. 7→

9 fields(C) = T f Object Object [R-Object] [R-New] → {} new (v) f v C → { 7→ }C G <: T field-vals(typeOf[C]) = T f e ∅ ` [R-Cast] [R-Class] (T)Φ Φ C f e G → G → { 7→ }typeOf[C] mbody(m, G) = (x, e0) ΦG.f ΦG(f) [R-Field] [R-Invk] → Φ .m(d) [x d, this Φ ]e0 G → 7→ 7→ G e e0 e e0 e e0 → [C-Field] → [C-Rcvr] → [C-New] e.f e0.f e.m(d) e0.m(d) new (e) new (e0) → → C → C e e0 e e0 e e0 → [C-Arg] → [C-Cast] → [C-Mapping] d.m(e) d.m(e0) (T)e (T)e0 f e f e0 → → { 7→ }G → { 7→ }G Figure 5: Computation Rules

fields(Object) = [F-Object] field-vals(Object) = [FV-Object] field-vals(C) = [FV-Class] ∅ ∅ ∅ CT (C) = class C : B / A T f e; M; W g; N; fields([X R]A) = U{h } 7→ [F-Class] fields(C) = U h [X R]W g CT (C) = class C : B∪/ A 7→T f e; M; W g; N; { } [F-Typeof] fields(typeOf[C]) = [X R]T f CT (C) = class C : B / A T7→f e; M; W g; N; { } [FV-Typeof] field-vals(typeOf[C]) = [X R]T f e U m (T x) return e; methods(G) 7→U m (T x) return e; methods(G) { } ∈ [MType] { } ∈ [MBody] mtype(m, G) = T U mbody(m, G) = (x, e) → methods(Object) = [MethodsObject] ∅ CT (C) = class C : B / A T f e; M; W g; N; { } [MethodsClass] methods(C) = [X R]N methods([X R]A) CT (C) = class C : B∪/ A T f e; M;7→W g; N; { } [MethodsTypeof] methods(typeOf[C]) = [X R]M methods([X R]B) mtype(m, G) = U V7→implies∪W = U and T =7→V → [Override] override(m, G, W T) CT (C) = class C : B / A ... CT→(C) = class C : B / A ... { } [Params] { } [ParamsTypeof] params(C) = X @ (I, J) params(typeOf[C]) = X @ (I, J) CT (C) = class C : B / A ... CT (C) = class C : B / A ... { } [Super] { } [SuperTypeof] super(C) = A super(typeOf[C]) = B

Figure 6: Auxiliary Functions

Case [S-Super]: In this case, V = C, where Subcase U = D: Then, U = [X R]A. By CT (C) = class C : B / A ...M and U = [MethodsClass], methods(V) = methods7→ (C) = [X R]A. We need to show that mtype{ (m}, C) = ([X R]M) methods([X R]A) = ([X R]M) T 7→S. By lemma methods is well-defined, it methods7→ (U).∪ Therefore, any7→ method in methods7→ (U∪) suffices→ to show that U m (T x) return e; must be in methods(V). methods(C). { } ∈ Case [S-Typeof]: Analagous to [S-Super]. We know that U m (T x) return e; methods(U). So, we proceed{ by induction} on ∈ Case [S-Trans]: Let the intermediate type be T. Then the derivation of methods(U). V <: T and T <: U. Thus, by the in- duction∅ ` hypothesis,∅ `mtype(m, V) = mtype(m, T) = Subcase U = Object: Impossible, since Object has no mtype(m, U). methods. Lemma 3 (Substitution Preserves Typing)

10 If ∆; Γ e : T and Γ = x : S and ∆; d : U and Case [C-Rcvr]: We know that e = e0.m(d) and e0 = ` ∅ ` ∆ U <: S then ∆; [x d]e : T0 where ∆ T0 <: T. e0 .m(d). Further, e must have been typed by [T- ` ∅ ` 7→ ` 0 Invk], which means that ; e0 : W for some ∅ ∅ ` Proof With the above two lemmas, this one follows by ground type W, and that ; d : V and ∅ ∅ ` ∅ ` straightforward structural induction over the derivation V <: U, and also mtype(m, W) = U T. By the → of ∆; Γ e : T. induction hypothesis, ; e0 : W0 where ∅ ∅ ` ∅ ` ` W0 <: W. Therefore, by lemma Subtyping Preserves Given the above lemmas, the following subject re- Method Typing, mtype(m, W0) = U T and thus → duction proof is a simple structural induction with a ; e0.m(d) : T by [T-Invk]. case analysis on the typing rule used to derive ; ∅ ∅ ` ∅ ∅ ` Case [C-Field]: If e.fi e0.fi, then e e0. Fur- e : S. → → ther, e.fi must have been typed by [T-Field] to Theorem 1 (Subject Reduction) If ; e : S and have type Ti. Therefore, by the induction hypoth- ∅ ∅ ` e e0 then ; e0 : T where T <: S. esis, ; e : S and ; e0 : S0 where S <: S0. → ∅ ∅ ` ∅ ` Then,∅ by∅ ` the lemma∅ Fields∅ ` are Preserved∅ ` by Sub- Proof We prove this by structural induction on the types, fields(S0) = fields(S)@ F for some F, and by [T-Field], ; : . derivation of e e0. e0.fi Ti → ∅ ∅ ` Case [R-Object]: Immediate. The proof of progress presents no additional compli- cations, and requires only one new lemma. Case [R-New]: Immediate from the premises of [T- New] and [R-New]. Lemma 4 (Agreement of fields and field-vals) If fields(G) = T f and field-vals(G) = T f e then T = T , Case [R-Class]: Immediate from the premises of 0 0 0 f = f0 and ; e : S where S <: T. [WF-ClassDef] and [R-Class]. ∅ ∅ ` ∅ ` Theorem 2 (Progress) If ; e : S then one of the Case [R-Cast]: By [T-Cast], e = (T)ΦG must have following holds: ∅ ∅ ` type T. By [T-Mapping], e0 = ΦG must have type G. By hypothesis of the reduction rule, G <: T. e = f v ∅ ` • { 7→ }G Case [R-Field]: We know that e = f d .f and { 7→ }G i e e0 that e0 = di. Since e must have been typed by [T- • → Field], we know that fields(G) = T f and ; e = (S)e0 and ; e0 : T and T <: S. ∅ ∅ ` • ∅ ∅ ` ∅ ` 6 e : Ti. Further, since f d must have been { 7→ }G typed by [T-Mapping], we know that ; di : S Proof By induction over the derivation of ; e : S. where S <: T. ∅ ∅ ` ∅ ∅ ` ∅ ` Case [T-Var]: This is a contradiction, since e is Case [R-Invk]: We know that e = ΦG.m(d) and that ground. e0 = [x d, this Φ ]e0. Further, e was typed 7→ 7→ G by [T-Invk] to have type U where mtype(m, G) = Case [T-Class]: In this case e = C where C is ground. Then by the lemma Agreement of field- T U and ; d : T0 and T0 <: T. As a premise → ∅ ∅ ` ∅ ` vals and fields, params(C) = X (I, J) for of [R-Invk], we know that mbody(m, G) = (x, e0). @ some Further, field-vals(typeOf[C]) = T f e By the lemma mtype and mbody agree, 0 for some T, f, and e. Therefore, [R-Class] applies ; x : T, this : G e0 : U0 where U0 <: U. ∅ ` ∅ ` and S f [X S0]C . Then by the lemma Substitution Preserves Typ- → { 7→ 7→ }typeOf[C] ing, ; e0 = [x d, this Φ ]e0 : U00 where ∅ ∅ ` 7→ 7→ G Case [T-Mapping]: Either e is already a value, or U00 <: U0. ∅ ` e = f e G where not all of the e are val- Case [C-Cast]: Trivial, since (T)e : T for any e. ues.{ Then7→ by} the induction hypothesis, there is ∅ ` some i such that either e e , in which case [C- Case [C-Map]: Immediate from the induction - i 0i Mapping] applies, or e contains→ a bad cast, and pothesis and the transitivity of subtyping. i the case is complete. Case [C-New]: Immediate from the induction hy- Case [T-Cast]: Here there are three cases: pothesis and the transitivity of subtyping. e = (S)e where e is not a mapping. Then Case [C-Arg]: Immediate from the induction hypoth- 0 0 • [C-Cast] applies. esis and the transitivity of subtyping.

11 e = (S)ΦT where T <: S. Then [R-Cast] dynamically-typed, and many of the uses to which • applies. ∅ ` Python metaclasses are put are not possible in a stati- cally typed language. The work on static type systems e = (S)ΦT where T <: S. Then e is a bad • cast. ` 6 for Python has not included metaclasses [15]. Several type systems have also been proposed for Case [T-New]: From the antecedent of [T-New] the languages with metaclasses, including the premise of [R-New] applies. language [3], which turns Smalltalk into a structurally- typed language with extensive static checking. How- Case [T-Invk]: Here there are two cases. ever, because the Smalltalk metaclass system is so dif- e = r.m(d) where r is not a mapping. Then ferent from the one in MCJ, many of the interesting • by the induction hypothesis, either r contains aspects of the type system do not carry over. Further- a bad cast or r r0 and [C-Rcvr] applies. more, the Strongtalk papers do not provide a formal → semantics and analysis of the system. A formal analy- e = ΦT.m(d) We know from the antecedent • that mtype(m, boundT) = U V and therefore sis of inheritance in Smalltalk is provided in [7] but this mtype(m, T) = U V since T →is ground. There- again does not consider the hierarchy of metaclasses fore, since mbody→is defined everywhere mtype presented here. Graver and Johnson [12] present another type sys- is defined, mbody(m, T) = (x, e0) for some x tem for Smalltalk, with a formalism and sketch of a and e0. Thus [R-Invk] applies. safety proof. Again, the metaclass type system is sub- Case [T-Field]: Here there are two cases, either the stantially different, reflecting the underlying Smalltalk reciever is a mapping or not. In the first, by the system. Additionally, the paper is concerned primarily antecedent of the typing rule, we can lookup the with optimization as opposed to static checking. field successfully and apply [R-Field]. Otherwise, Cointe [6] presents a model of metaclasses extremely we can apply [C-Field]. similar to that presented here. In it, he provides several overlapping motivations to our own. One is to regular- From these, we can conclude the desired type safety ize the metaclass system of Smalltalk, and another is to result. enable additional programming flexibility. To this we Theorem 3 (Soundness) If e : S then either add modeling freedom, and a relation to static methods in more recent OO languages. Cointe’s work, however, e ∗ f v does not provide a formal model, so it is difficult to • → { 7→ }G determine the exact relationship between the systems. e ∗ e0 where e0 is an invalid cast • → Additionally, his work is in a untyped setting (Lisp and e reduces infinitely. Smalltalk) and thus the safety theorems proved here are • not possible. Proof Immediate from Subject Reduction and When viewed as “instance generators”, our meta- Progress. classes are similar to prototypes in untyped languages such as Self. Prototypes generate new instances, which themselves can generate new instances. Our language 5 Related Work is more restrictive than prototype-based languages in A number of object-oriented languages have included the sense that all metaclasses and instance classes must some form of metaclass system. Most notable among be declared statically (i.e., by writing down class def- these is Smalltalk [11], but others include Common Lisp initions). But our language is more expressive in the with CLOS [14]. sense that we include classes and subclassing relation- All of these systems share a common architecture ships. Also, unlike typical prototype-based languages, of the metaclass system in which each class has its own our language is statically typed. freely generated metaclass, defined by the class methods Formalized calculi for object-oriented languages are and fields of the class. In contrast, MCJ provides a abundant in the programming languages community hierarchy for structuring metaclass relationships, which today, including Featherweight Java [13], upon which provides significantly more modeling and abstraction MCJ is based. However, none of them have considered flexibility. static methods, which are the closest analogue of the The metaclass system present in Python [17] is metaclass functionality in MCJ. quite similar to that provided here, with inheritance Finally, the motivation for this work comes from the of instance methods as class methods from arbitrary language MetaGen, introduced in [2]. MetaGen can be other classes as metaclasses. However, Python is seen as an extension of MCJ, which provides numerous

12 other advanced type features. Here we restrict ourselves References to metaclasses and analyze the properties of the system formally. [1] Eric Allen, Jonathan Bannet, and Robert Cartwright. A first-class approach to genericity. In Proceedings of the 2003 ACM SIGPLAN con- 6 Conclusions and Future Work ference on Object-Oriented Programming, Systems, With MCJ, we have devised a core calculus for meta- Languages, and Applications, pages 96–114. ACM classes that is more flexible than that available in more Press, 2003. traditional metaclass systems such as Smalltalk and [2] Eric Allen, David Chase, Victor Luchangco, Jan- that allows clean expression of many common design Willem Maessen, and Guy Steele. Object- patterns. In doing so, we have demonstrated that meta- Oriented Units of Measurement. In Proceed- classes can be added to a nominally-typed, statically- ings of the 2004 ACM SIGPLAN Conference checked language without either significant complica- on Object-oriented Programing, Systems, Lan- tion of the semantics or difficulty in the proof of sound- guages, and Applications, 2004. Available at ness. http://research.sun.com/projects, under Program- In addition to this contribution, we have elucidated ming Languages. several other important points about the integration of metaclasses into an object-oriented system. The typeOf [3] Gilad Bracha and David Griswold. Strongtalk: type operator is to our knowledge unique, and plays a Typechecking Smalltalk in a Production Envi- key role in the soundness of the system. The discovery ronment. In Proceedings of the OOPSLA ’93 that the Class class played no special role, and that thus Conference on Object-Oriented Programming Sys- the class and metaclass hierarchies could both be rooted tems, Languages and Applications, pages 215–230, at Object is also novel. Finally, we have shown how an September 1993. expressive framework of metaclasses has positive effects [4] . A semantics of multiple inheritance. in other areas of the language, such as mechanisms for In Proc. of the international symposium on Se- object construction. mantics of data types, pages 51–67. Springer-Verlag A natural extension of the work in this paper is to New York, Inc., 1984. expand MCJ to include more of the features presented in [2], so as to allow inclusion of the system for checking [5] Robert Cartwright and Guy L. Steele, Jr. Com- dimensions of physical quantities. Such an extension patible genericity with run-time types for the java would allow for a proof of “dimensional soundness” in programming language. In Proceedings of the 1998 the resulting system. Further, any realistic system will ACM SIGPLAN Conference on Object-Oriented have imperative features, and while such features do Programming, Systems, Languages, and Applica- not seem likely to interact badly with metaclasses, no tions, pages 201–215. ACM Press, 1998. sound system is built on such assumptions. [6] Pierre Cointe. Metaclasses are first class: the Ob- Another interesting extension would be to expand jVLisp model. In Proceedings of the OOPSLA ’87 our calculus to include either multiple inheritance (as Conference on Object-Oriented Programming Sys- does Python) or some alternative such as or tems, Languages and Applications, pages 156–162. traits, as there may be interesting interactions between ACM Press, 1987. metaclasses and these features that have yet to be dis- covered. [7] William R. Cook. A Denotational Semantics of Inheritance. PhD thesis, Brown University, May 15 Acknowledgments 1989.

We would like to thank Jan-Willem Maessen for his ex- [8] R. Kent Dybvig. Writing Hygienic Macros in tremely helpful feedback on the formal rules presented Scheme with Syntax-Case. Technical report, In- in this paper. We also thank David Chase for not be- diana University Science Dept., July 03 lieving in us unless we were right, Victor Luchangco for 1992. many valuable discussions, and Guy Steele for his many [9] Matthias Felleisen. On the expressive power of helpful comments. programming languages. In Neil D. Jones, edi- tor, ESOP’90, 3rd European Symposium on Pro- gramming, volume 432 of Lecture Notes in Com- puter Science, pages 134–151, Copenhagen, Den- mark, 15–18 May 1990. Springer.

13 [10] Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison Wes- ley, Reading, Massachusetts, 1994. [11] Adele Goldberg and David Robson. Smalltalk-80: The Language and Its Implementation. Addison- Wesley, Reading, Massachusetts, 1989. [12] Justin O. Graver and Ralph E. Johnson. A type system for Smalltalk. In Conference Record of the Seventeenth Annual ACM Symposium on Princi- ples of Programming Languages, pages 136–150, San Francisco, California, January 1990. [13] Atshushi Igarashi, Benjamin Pierce, and Philip Wadler. Featherweight Java: A minimal core cal- culus for Java and GJ. In Proceedings of the 1999 ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applica- tions, pages 132–146. ACM Press, 1999. [14] Gregor Kiczales, Jim des Rivi`eres,and Daniel G. Bobrow. The Art of the Protocol. MIT Press, Cambridge, MA, 1991. [15] Michael Salib. Static with Starkiller. In PyCon DC, 2004. http://www.python.org/pycon/dc2004/papers/1/paper.pdf. [16] Guy L. Steele Jr. Common Lisp: The Language, Second Edition. Digital Press, Bedford, Mas- sachusetts, 1990. [17] Guido van Rossum. Unifying types and classes in Python 2.2. http://www.python.org/2.2.2/descrintro.html, 2002. [18] Christopher A. Welty and David A. Ferrucci. What’s in an instance? Technical report, Rochester Polytechnic Institute Dept., 1994.

14