Object-Oriented Programming with Flavors

David A. Moon Symbelics, Inc.

programs. This newly-designed Flavors is the version Abstract described in this paper. It will be released with the next Symholics software release. (5) This paper describes ' newly redesigned object- oriented programming system, Flavors. Ravors encourages What is obJect-orlentod programm/ng? program modularity, eases the development of large, complex programs, and provides high efficiency at run We view object-oriented programming as a techn/que for time. Flavors is integrated into Lisp and the Symbelics organizing very large programs. This technique makes it program development environment. This paper describes practical to deal with programs that would otherwise be the philosophy and soma of the msjor characteristics of impossibly complex. Symbollcs' Flavors and shows how the above goals are addressed. Full details of Flavors are left to the An object-oriented program consists of a set of objects and programmers' manual, Re, fence Guile to 8ymboli~ a set of operations on those objects. These entities are . (5) not de/'med in a monolithic way. Instead, the dei'mitions of the operations are distributed among the various objects that they can operate upon. At the same time, History of F/avora the definitions of the objects are distributed among the The origulal Flavors system was developed by the /,fiT various facets of their behavior. An object-oriented Lisp Machine group in 1979. (I, 2, 3) It was used to build programming system is an organizations] framework for a window system and later applied to other system combining these distributed definitions and managing the programming. In 1981, Symbel/cs designed a more interactions among them. efficient implementation of Flavo~ Since that time, we Object-oriented programming is also an abstract/on have made increasingly heavy use of Flavors in nearly mechanism. A program that manipulates an object uses every aspect of the Symbelics 3600 software, such as I/O certain defined operations to manipulate it. These streams, network control programs, the debugger, the operations serve as an interface, and the program does editor, and user interface facilities. Flavors permeate not need to know how the object implements the beth the operating system and higher-level utilities. The operations. The implementation of one operation can be same Flavors tools used in-house are fully documented and different for different kinds of objects. At the same time, are used by most SymbeLics customers to build their an object's behavior can be divided into several facets, application programs. which need not know each other's internal detaiLs. Five years' experience with Flavors has pointed out the strengths and weaknesses of the original design. In 1985 Goals of Flavors we undertook a thorough redesign of Flavors to solve the problems that we had identified. The result is a new There are many possible and useful styles of object- Flavo~ system that has been implemented at Symbelics oriented programming. Flavors adopts an approach aimed and has been used in-houas to develop several complex at these goals: * Encourage program modularity. By this we mean that Flavors should make it easier to construct programs out of existing parts, to modify the behavior of Permission to copy without fee all or pen of this material is granted provided that the copies are not made or distributed for direct commercial advantage. existing programs without massively rewriting them, the ACM copyright notice and the title of the publication and its date appear, to understand programs one piece at a time, and to and notice is given that copyingis by permission of the Association for identify interfaces between modules. Computin$ Machinery. To copy otherwise, or to republish, requires a fee and/ or specificpermission. • Ease development of large, complex programs. In addition to encouraging modularity, Ravors should © 1986 ACM 0-89791-204-7/86/0900-0001 75¢

September 1986 OOPSLA '86 Proceedings 1 allow programs to be constructed incrementally, make it possible to change data representations and program --> 14 organization while the program is running, and provide tools for analyzing programs. Similarly, the :writ able- inst ance-veriable s option allows us to alter the value of an instance • Provide high efficiency at run-time. The CPU time and variable: page fault rate associated with object-oriented operations should not be very much larger than that (setf (ship-mass my-ship) i00) associated with ordinary function calling and data --> 100 manipulation. This must be accomplished without compromising other goals. Generic functions The operations that are performed on objects are . Be compatible with the previous Flavors system. By this known as generic functions. we mean tools and backward-compatibility features to ease conversion of programs written with the old Methods Flavors to run with the new Flavors. Compatibility The Lisp function that performs a generic function on features are not discussed in this paper. instances of a certain flavor is called a method. The instance variables are accessible by name inside the Later sections discuss how these goals are met. We first body of a method. Often, one generic function has present the concepts of Flavors. several methods defined for it, attached to different flavors. An example method: Basic Flavors Concepts (defmethod (ship-speed ship) () It is often convenient to organize programs around objects, (sqrt (+ (expt x-velocity 2) which model real-world things. Each real-world object is (expt y-velocity 2)))) modelled by a single Lisp object. Each object has some state and a set of operations that can be performed on it. Generic Functions A Flavors program is built around: A generic function operates on an object by selecting one Flavors or more methods that implement the generic operation in Each kind of object is implemented as a flavor. A a specialized way for that object. One of the arguments flavor is an abstraction of the characteristics that all (usually the first) to a generic function is the object: the objects of this flavor have in common. It is a new available methods are those attached to its flavor. Generic aggregate data type. For example, this form defines a functions are the interface between objects. They provide flavor that represents ships: abstraction and isolation between modules.

(defflavor ship Figure I shows the differences between ordinary Lisp (x-velocity y-velocity mass) functions and generic functions. () : readable-instance-variables :writ able-instance-variables Ordinary Func~ons Gener/~ Func~ons : init able-inst ance-variables) Have a single Have a distributed definition. Instances of a flavor definition. Each object is implemented as an instance of a flavor. Interface is specified Interface can be specified For example, here we create and initialize an instance by defun. by defgenerlc or defmethod. of the ship flavor: Do not treat flavor First argument is usually (setq my-ship instances specially. an instance of a flavor. (make-instance 'ship :mass 14 :x-velocity 24 Implementation is the Implementation varies £rom :y-velocity 2) ) same whenever the ceil to call, depending on function is called. the flavor of the first Instance variables argument. This is a set of named variables with separate values Figure I for each instance. The values of the instance Differences between ordinary and generic Lisp functions. variables represent the state of each object. The

instance variables of the chip flavor are x-velocity, ...... y-velocity, and mass. The :readable-lnstance- variables option generates accessor functions for Generic functions are smoothly integrated into the Lisp reading the values of instance variables; for example: environment. Ordinary functions and generic functions are called with the same syntax. Making generic functions (ship-mass my-ship) syntactically and semantically compatible with ordinary functions has the following advantages:

2 OOPSLA '86 Proceedings September 1986 • The caller of a function need not know whether it is The goal of ordering flavor components is to preserve the generic. modularity of programs. A favor should be treated as an intact unit, with well-defined characteristics and behavior; • The Common Lisp package system (4) can be used to it is essential that mixing flavors together does not alter isolate modules and to distinguish between public and the internal details of any of the component flavors. This private interfaces by exporting the names of public makes it easier to assemble a program from pieces by generic functions. combining pre-existing flavors. The rules for ordering • Debugging tools such as trace can be used on generic components support this by ensuring that a flavor's components will be in the same order when that flavor is functions. part of another as they are when it stands alone. • Program-development tools that get the argument list or documentation of a function work equally well on Here is an example of ordering components. The third subform of d~fflavor is the list of direct components. generic functions. defflavor pie () (apple cinnamon)) Mixing Flavors defflavor apple () (fruit)) defflavor cinnamon () (spice)) A typical flavor is defu~d by combining several other defflavor fruit () (food)) flavors, called its components. The new flavor inherits defflavor spice () (food)) instance variables, methods, and additional component defflavor food () (.)) flavors from its components. In a well-organized program, each component flavor is a module that defines a single The resulting ordering of flavor components for pie is: facet of behavior. When two types of objects have some behavior in common, they each inherit it from the same (pie apple fruit: cinnamon spice food vanilla) flavor, rather than duplicating the code. When flavors are vanAlla is the flavor that is always included, to provide mixed together, Flavors organizes and manages the default behavior. interactions between them. This multiple inheritance is a key aspect of the design of Flavors;, the mechanism is Programmers are not allowed to mix together flavors with described in the following sections. incompatible constraints. When no ordering of components can satisfy all of the constraints, Flavors lists the conflicting constraints and requires the programmer to Ordering Flavor Components take corrective action. For example: When a flavor is defined with one or more component flavors, Flavors chooses an ordering of its components, (defflavor splce-cake () (spice pie)) including both the direct components and the inherited produces the error message: components. Components at the beginning of the ordering are the most specific and those at the end are Cannot order the components of SPICE-CAKE: the most general. The ordering is important because it No ordering of SPICE, PIE, CINNAMON works: controls how a flavor's methods are inherited from SPICE-CAKE has SPICE and PIE as components and how components earlier in the ordering direct components in that order. specialize the behavior of those later in the ordering. PIE depends directly on CINNAMON. The details of method inheritance are explained in a later CINNAMON depends directly on SPICE. section, but first the way the ordering is chosen must be However SPICE, PIE, CINNAMON have understood. no conflicting methods.

Each flavor defines certain constraints on the ordering of Instance Variable Inheritance itself and its direct components. Taken together, these constraints determine a partial ordering of all of the The instance variables of a flavor are the union of the components of s favor. Flavors computes a total ordering instance variables of its components. If several that is consistent with the partial ordering. Three rules components define instance variables with the same name, control the ordering of flavor components: they are combined into one instance variable. Thus • A flavor alwoys precedes its own components. instance variables can be used for communication between component flavors ff the programmer so chooses. A • The local ordering of components of a flavor is method can access inherited instance variables by name. preserved. This is the order of components given in The variable self is scoped like an instance variable but the clef flavor form. its value is the instance itself.

• Duplicate flavors are eliminated from the ordering. If a Programmers often employ a convention that only certain flavor appears more than once, it is placed as close to methods access a given instance variable. These methods the beginning of the ordering as possible, while still are considered to be inside the module that owns that obeying the other rules. instance variable. All other usage of that instance variable is by applying a generic function to self, which

September 1986 OOPSLA '86 Proceedings 3 provides a more abstract interface. This convention is • Collect the values of the methods into a list. allowed but not enforced by Flairs. The particular conventions for each instance variable in a program • Compute the arithmetic sum of the values of the depend on the design of that program. This is similar to methods. the way Lisp allows but does not enforce a convention that private functions of a module should only be called • Divide the methods into three categories: primary from inside that module. methods, before-daemons, and after-daemons. Call all the before-daemons, then call the most specific Initialization of an instance variable is controlled by the primary method, then call all the after-daemons. most specific flavor that specifies an initialization. Often an instance variable is initialized by the flavor that • Use the second argument to the generic function to defines it, but sometimes initializing an inherited instance select one of the methods. variable is a useful way to customize inherited behavior. Declarative Control of Method Combination

Method Inheritance Programmers control method combination separately from When a generic function is applied to an object of a the definition of the methods themselves. They control it particular flavor, methods for that generic function by declaring a method-combination type and constraints attached to that flavor or to one of its components are on the ordering of component flavors. The details of how available. From this set of available methods, one or this declarative specification is implemented as executable more are selected to be called. If more than one is code in the combined method can be ignored most of the selected, they must be called in some particular order and time. the values they return must be combined somehow. When defining a method, the programmer only thinks The simplest form of method inheritance is to use the about what that method must do itself, and not about method of the most specific flavor that provides one, and details of its interaction with other methods that aren't to ignore methods of more general flavors. This is often part of a defined interface. When specifying a methed- useful, but is not sufficient for all cases. Sometimes good combination type, the programmer only thinks about how program modularity requires that different parts of the the methods will interact, and not about the internal implementation be specified by multiple methods, which details of each method, nor about how the method- must then be combined. combination type is implemented. Programming an individual method and programming the combination of methods are two different levels of abstractio~ Keeping Method Combination them separate promotes modularity. The selection and combination of methods for a (generic function, flavor) pair is controlled by a method- Defining New Method-Combination Types combination type. The programmer can specify the name of a type and optional parameters when defining a flavor, Programmers can easily define new method-combination allow it to be inherited from a component flavor, specify types. Flavors provides macros that accept a largely it when defining a generic function, or simply allow the declarative specification of method sorting, filtration, and default type to be used. Several method-combination combination, and automatically produce the detailed cede types are built in and programmers are encouraged to to combine the methods. The full details are beyond the define additional types of their own. scope of this paper, but the following examples of built-in types should convey the philosophy. The method-combination type sorts the available methods This defines a methed-combination type named :mum that according to the component ordering, thus identifying calls all the methods and passes their values as more specific and less specific methods. It then chooses a arguments to the + function. The generic function subset of the methods (possibly all of them). It controls returns the sum of the numbers returned by the methods: how the methods are called and what is done with the values they return by constructing Lisp code that calls (define-simple-method-combination :sum +) the methods. Any of the functions and special forms of the language may be used. The resulting function is This defines the method-combination type mentioned called a combined method. earlier that calls daemon methods before and after the primary method: Some examples of butt-in method-combination types are: (define-method-combination :daemon () * Call only the most specific method. ;; Select three subsets of methods • Call all the methods, most-specific in-st or in the ; ; by pattern-matchlng reverse order. ((before "before" :every :most-speciflc-first (:before)) * Try each method, starting with the most specific, until (primary "primary" :first one is found that returns a value other than nil :most-specific-first ())

4 OOPSLA '86 Proceedings September 1986 the old. When a flavor is changed, the system propagates {after "after" :every the changes to any flavors of which it is a direct or :most-specific-last (:after))) indirect component. It is also possible to erase a ;; A Lisp form that calls the methods definition without replacing it with a new one. • (mult ipl e-value-progl (progn , (call-component-methods before) Changing the data representation is just as easy. If you , (call-component-method primary) ) redefine a flavor, to add or remove instance variables, old , (call-component-methods after))) instances of the flavor automatically convert themselves to the new format the next time they are accessed. The pattern-matohing specifications bind the variables before and afte~ to lists of all the daemon methods You can redefine a generic function to be an ordinary and bind the variable pr~ffiary to the most specific function, or an ordinary function to be a generic function, primary method. The backquoto expression then without having to recompile its callers. generates the Lisp code that calls all the methods in the desired order and returns the values of the primary Program Development TooLs method. The Symbolics programming environment offers a variety Encouraging Program Modularity of tools for analyzing Flavors.based programs. These tools can be invoked through the command processor or by No programming system can guarantee program pointing the mouse at displayed instances, flavor names, modularity or eliminate the need for careful design of a or method names. program's structure. However, a programming system can make it easier to build modular programs. Flavors Show Flavor Components provides organizational techniques for writing programs in Answers: What is the order of flavor components, and a modular way and keeping them modular as they evolve. why did Flavors pick that order?

Inheritance of methods encourages modularity by allowing Show Flavor Dependents objects that have similar behavior to share code. Objects Answers: What flavors inherit from this one? that have somewhat different behavior can combine the Show Flavor Instance Variables generalized behavior with code that specializes it. Answers: What stats is maintained by instances of Multiple inheritance further encourages modularity by this flavor? allowing object types to be built up from a toolkit of Show Flavor Operations component parts. Answers: What generic functions are supported by this Interfaces between modules are typically defined as flavor? generic functions. Using any kind of function as an Show Flavor Methods interface lends some abstraction. Using a generic Answers: What methods are defined for this flavor or function has the additional advantage that there can be inherited from its component flavors? several modules conforming to the same interface but each implementing it in a different way. Show Flavor Initializations Answers: How are new instances of this flavor A common technique is to mix flavors so that a single initialized? object is composed of several modules. The modules communicate through generic functions applied to self, Show Flavor Differences combination of methods (such as before- and after- Answers: What are the differences between two daemons) belonging to different modules, and shared flavors? instance variables. Show Generic Function Answers: What flavors provide a method for this Easing Development of Large, Complex Programs generic function?

The encouragement of modularity outlined above is a key List/Edit Methods feature when developing large programs. In addition, View or edit the source code of all methods for this Flavors makes it easy to change design decisions at any generic function. time and provides tools to assist the programmer in understanding and modifying the structure of the Show Flavor Handler program. Answers: When a given generic function is applied to an instance of a given flavor, what methods implement The flexibility to change parts of a program quickly and the operation? What is the actual Lisp code produced easily is useful in the development stage. Flavors enables to combine these methods? you to redefine flavors, methods, and generic functions at any time, even while the program is running. To do so, List/Edit Combined Methods you need only evaluate a new definition, which replaces View or edit the source code of the methods that

September 1986 OOPSLA '86 Proceedings 5 implement a given generic function on an instance of program development operations, especially when a given flavor. modifying flavors that have hundreds of dependents. This tradeoff is acceptable, since development operations need Show Effect of Definition not be faster than human speeds (several seconds), while Answers: If I evaluate this definition (text in the run-time operations must operate at computer speeds editor), or erase it, what changes would take place? (microseconds). Speeding up the run-time operations also For example, changing the order of a flavor's speeds up the development tools built on them. components might change which inherited method it uses for a particular generic function. The key areas that are important to optimize in an object- oriented programming system are: By pointing the mouse at a displayed object, the programmer can view: • Selection of one or more methods when a generic function is called • Names of the arguments and return values of a generic function. If not explicitly declared, these are • Instance variable access from a method deduced from the methods. • Instance creation • Documentation for a flavor or a generic function. The following sections discuss how Flavors implements • Source code of a flavor, a genetic function, or a these operations. method. Implementation of Method Selection Efficiency Considerationm The first time a flavor is instantiated, or during Efficiency can be defined in many ways. We are compilation of the program ff so directed, a handler concerned with four dimensions of efficiency: function is precomputod for each generic function that the flavor supports. The method-combination procedure • Efficient use of the programmer's time. selects a set of methods and produces Lisp code that combines them. If the code can be optimized into calling • CPU time. a single method, that method is the handler. Otherwise a • Page fault rate. combined method is generated, compiled to machine code, and used as the handler. The combined method calls the • Virtual memory occupied. methods with ordinary Lisp function calls.

We optimize the programmer's time through declarative The results of this precomputation are saved in a handler mechanisms, powerful tools, the flexibility to redefine tab/e associated with the flavor, keyed by the generic pieces of the program, and a complex implementation that functiorL Thus when a generic function is called, the allows the programmer's own programs to remain simple. method selection process consists of finding the instance's flavor, looking in the handler table, and calling the The efficiency philosophy of Flavors is to optimize run- handler. (See figure 2.) The handler table is a hash time speed to the maximum extent that does not table whose structure is optimized to exploit the pipelined compromise other goals, such as the flexibility to redefine anything while the program is running. In addition to characteristics of the 3600's memory bus. Flavors.related goals, general Symbelics system goals, such Subsequent changes to the program such as adding as full run-time error checking, avoiding widespread use methods, removing methods, declaring a different method of declarations, and providing the best functionality, must combination type, or changing a flavor's components not be compromised for the sake of efficiency, incrementally update all affected handler tables, compiling new combined methods when necessary. For this purpose Run.time speed is important because applications, each flavor is linked to the flavors that depend on it and including the development tools themselves, are typically built on several layers of Flavors.based substrate. The each combined method records how it was generated. Flavors approach is to use a complex, machine-dependent implementation of relatively simple-appearing features, Implementation of Instance Variable Access such as generic functions and instance variables. An instance is a represented as a block of storage whose Programmers enjoy the efficiency benefit of this first word references the flavor and whose remaining implementation without having to write their own words contain values of instance variables. Methods that programs in a complex machine-dependent way. access instance variables cannot contain constant offsets CPU time and page fault rate determine response time, so of instance variables within the instance. These offsets they are more important than virtual memory size, which are variable at run time, depending on the flavor of the only consumes inexpensive disk storage. Consequently instance, which can be any flavor that has the method's F/st,ors maintains multiple copies of information when that flavor as a component. Multiple inheritance makes it improves virtual memory locality or execution speed. impossible to allocate fixed offsets to instance variables, Keeping those multiple copies consistent slows down because two flavors using the same offset might later be

6 OOPSLA '86 Proceedings September1986 mixed together and one of them would have to change. Implementation of Instance Creation The solution is to use indirect addressing. Each entry in a handler table includes a mapping table, which contains The fu-st time a flavor is instantiated, the initialization instance variable offsets. (See figure 2.) A method information from all its components is combined and receives a mapping table as an argument. Offsets into the saved in a convenient form. Subsequent instantiations mapping table are compiled into methods. These offsets consist of allocating storage, copying a template instance, don't change, because when two flavors are mixed initializing any instance variables whose initial values are together each has its own mapping table. Accessing an not constant, and invoking initialization methods if any instance variable fetches the offset from the mapping have been defined. table, adds it to the address of the instance, and references that memory location. Tlminf Measurements

When a combined method calls another method, it These measurements were performed on a Symbolics 3640 supplies a mapping table fetched from its own mapping using the bets.test version of Release 7.0. Absolute table. Thus mapping tables actually form a tree structure timing measurements vary depending on hardware parallel to the tree structure of flavor components. configuration, software version, and measurement methodology, so the most meaningful information here is In principle every flavor needs a separate mapping table the ratios. for each component, and the total number of mapping tables could be proportional to the square of the number Calling a generic function with two arlFiments takes of flavors. In practice the average number of components twice as long as calling an ordinary function with two of a flavor is small and only components that have arguments; 13 microseconds versus 7, Times include instance variables need mapping tables. Thus the average computing trivial arguments, the actual call, executing a number of mapping tables per flavor is only 4.1 and the trivial function body, and the retur~ total memory occupied by mapping tables is negligible. The first instance variable accessed by a method takes about 5 microseconds; succeeding instance variables take Fleztble Representation of lswtanees 2.2 microseconds. For comparison, accessing a lexical Redefining a flavor in a way that changes the variable in a closure takes 1.1 m/craseconds and accessing representation of instances, such as adding or deleting an a =Im¢£al variable takes 1.7 microseconds. instance variable, arranges for existing instances to be Creating an instance with two instance variables using updated automatically. It makes a new flavor (with the i~aki-lnatllnce, the most general mechanism, takes 353 same name) and changes the old flavor's handler table so microseconds, 6.1 times as long as creating a dlsfatruct: that all generic functions rearrange the instance, change structure with two slots. Using a Flavors constructor its flavor reference to the new flavor, and retry the function, analogous to a d~fatcuc¢ constructor, takes operation. If the new instance representation is larger, 165 microseconds if keyword arguments are used or 68 rearranging the instance allocates new storage, copies the microseconds with positional arguments, reducing the instance variable values into it, and deposits forwarding ratio to 2.8 or 1.2. addresses with a special tag into the old storage. The instance variable accessing instructions and the garbage These timing measurements suggest that programming collector recognize this special tag. with Flavors is not substantially less efficient than now object-oriented programming. In fact, it can be more efficient:

• A Flavors instance is smeller than a dsfstzmct

Stick FTirl~I InltlltKe Fllvot

- ~. ,vo,,,,,j t t I c,~,1 L i.v. o-,- j 'J vo,,ob~. L ,v o.o,. j I I-~t =.r.I-- ~ - J'w"'"'JC., T~'~ ""'"~b"t ~L ,.v. o,,.., j " 1 | I.v. oe./~t ] ~cy J : Gc,~ic Fu,ction Val~e ~ Com#inedMe¢~ VSluet ~ lUfLIppi~ Teblc FhM,.e 2 Flavors ]mplemenl~tlon

September 1986 OOPSLA '86 Proceedings 7 structure with the same number of slots, by one word.

• Dispatching on object types with typeca=,, takes about 15 times as long for a typical case as a generic function call.

Directions for Future Research

The following less wall-understood aspects of object. oriented programming are good directions for future research:

• Protecois--Formallzing the notion of a generic interface, and further separating the contract of an object from the implementation of the object.

• Flavors for Non.lnstances-.Integrating the built-in data types of the Lisp language into the Flavors framework.

• Methods for Primitive Functions--Making car or + applied to an instance turn into a generic function and invoke a method, without slowing down the common non-object-oriented case.

• Higher.level Tools..Programmere of very large programs need all the help they can get. Programming tools that incorporate a model of the programming process, rather than just answering one question at a time, work better when the program is structured around a framework they understand. F/a~ could be one such framework.

• Database..lntegrating object-oriented programming with the persistent, reliable, data.independent structure of a database.

• Multiadic Operations-Generic functions that use more than one of their ar~zrnents to select methods are easy to implement, but a coherent and useful framework for organizing programs that work this way needs to be developed.

References 1. D. Weinrab, D. Moon, Lisp Machine Manual, MIT AI Lab, 1981, Chapter 20.

2. I-L I. Cannon, "Flavors: A non.hierarch/eal approach to object-oriented programming", 1982.

3. It D. Greenblatt, et el., "The LISP Machine', lateroetive I~umm~ Enviroame~, D.K Barstew, H.E. Shrobo, E. Sandewal], eds. McGraw-H/l], 1984.

4. G. L. Steele, Common Lisp tl~ Langu~e, Digital Press, 1984.

5. Reference Guide to 8ymbolics Common L~p: Language Concepts, Symbolics Release 7 Document Set, 1986.

8 OOPSLA ~6 Proceedings September1986