What is a Recursive Module?* Karl Gary Robert Harper Sidd Puri Carnegie Mellon University Carnegie Mellon University Microsoft Corporation Abstract also support parameterized, or generic, modules to bet- ter support code reuse. A hierarchical module system is an effective tool for There is no question that hierarchical design is an structuring large programs. Strictly hierarchical mod- important tool for structuring large systems. It has of- ule systems impose an acyclic ordering on import depen- ten been noted, however, that strict adherence to a hi- dencies among program units. This can impede modu- erarchical architecture can preclude the decomposition lar programming by forcing mutually-dependent compo- of a system into “mind-sized” components. In some nents to be consolidated into a single module. Recently situations the natural decomposition of a system into there have been several proposals for module systems modules introduces cyclic dependencies, which cannot that admit cyclic dependencies, but it is not clear how be expressed in a purely hierarchical formalism. The these proposals relate to one another, nor how one might only solution is to consolidate mutually-dependent frag- integrate them into an expressive module system such ments into a single module, which partially undermines as that of ML. the very idea of modular organization. To address this question we provide a type-theoretic In response several authors have proposed linguis- analysis of the notion of a recursive module in the con- tic mechanisms to support non-hierarchical modular text of a “phase-distinction” formalism for higher-order decomposition. Recent examples include: Sirer, et module systems. We extend this calculus with a recur- al.‘s extension of Modula-3 with a “cross-linking” sive module mechanism and a new form of signature, mechanism [21]; Flatt and Felleisen’s extension of called a recursively dependent signature, to support the their MzScheme language with cyclically-dependent definition of recursive modules. These extensions are “units” [8]; Duggan and Sourelis’s “mixin modules” that justified by an interpretation in terms of more primitive extend the Standard ML module system with a special language constructs. This interpretation may also serve “mixlink” construct for integrating mutually-dependent as a guide for implementation. structures [6, 71; and Ancona and Zucca’s algebraic for- malism for mixin modules [2]. Each of these proposals 1 Introduction seeks to address the problem of cyclic dependencies in a module system, but each does so in a slightly. different Hierarchical decomposition is a fundamental design way. For example, Flatt and Felleisen’s formalism does principle for controlling the complexity of large pro- not address the critical issue of controlling propagation grams. According to this principle a software sys- of type information across module boundaries. Duggan tem is to be decomposed into a collection of modules and Sourelis’s framework relies on a syntactic transfor- whose dependency relationships form a directed, acyclic mation that, in effect, coalesces the code of mutually- graph. Most modern programming languages include dependent modules into a single module. It is not clear module systems that support hierarchical decomposi- what are the fundamental ideas, nor is it clear how to tion. Many, such as Standard ML [16] and O’Caml[14], integrate the various aspects of these proposals into a full-featured module system. ‘This research was sponsored by the Advanced Research It is natural to ask: what is a recursive module? We Projects Agency CSTO under the title “The Fox Project: Ad- vanced Languages for Systems Software”, ARPA Order No. propose to address this question in the framework of C533, issued by ESC/ENS under Contract No. F19628-95-C- type theory, which has proved to be a powerful tool for 0050. The views and conclusions contained in this document are both the design and implementation of module systems. those of the authors and should not be interpreted as represent- We conduct our analysis in the context of the “phase ing official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the U.S. Government. distinction” module formalism introduced by Harper, Mitchell, and Moggi [ll] (hereafter, HMM), augmented Permission to make digital or hard copies of all or part of this work for to support recursive types and functions, and to support personal or classroom uss is granted without fee provided that type definitions in signatures [9, 131. The phase distinc- copies are not made or distributed for profit or commercial advan- tion tage and that copies bear this notice and the full citation On the first Page. calculus provides a rigorous account of higher-order To copy otherwise, to republish, to post on servers or to modules (supporting hierarchy and parameterization) in redistribute to lists, requires prior specific permission and/or a fee. a framework that makes explicit the critical distinction SIGPLAN ‘99 (PLDI) 5/99 Atlanta, GA., USA between the static, or compile-time, part of a module 0 1999 ACM l-581 13-033.3/99/001 l-85.00 50 and the dynamic, or run-time, part. This calculus has structure calculus, extending the core language with proved to be of fundamental importance to the imple- a primitive module construct without explicit mecha- mentation of higher-order modules, as evidenced by its nisms for hierarchy (e.g., substructures) or parameter- use in Shao’s FLINT formalism used in the SML/NJ ization (e.g., functors). Primitive modules consist of a compiler [18, 201 and in the TIL/ML compiler [23]. static, or compile-time, part containing the type con- Our analysis proceeds in two stages. First we con- structors of the module, together with a dynamic, or sider a straightforward extension of the phase distinc- run-time, part containing the executable code of the tion calculus with a notion of recursive (self-referential) module. This separation is known as the phase distinc- module. An interpretation of this new construct is pro- tion. An important property of the formalism is that vided by an interpretation of it into the primitive mod- the phase distinction is maintained, even in the pres- ule formalism of the phase distinction calculus. This ence of higher-order (and, as we shall see, recursive) interpretation renders the compile-time part as a recur- module constructs. sive type and the run-time part as a recursive function, The main result of HMM is that higher-order mod- as might be expected. In essence a recursive module is ule constructs are a definitional extension of the prim- just a convenient way of introducing recursive types and itive structure calculus. In other words higher-order functions. constructs are already present in the primitive struc- Unfortunately this simple-minded extension does ture calculus in the sense that they may be defined in not go far enough to be of much practical use. As Dug- terms of existing constructs. (This interpretation may gan and Sourelis have observed [7], it is of critical impor- be thought of as a compilation strategy for higher-order tance for most practical examples that the type equa- modules, and indeed this fact has been exploited in the tions that hold of a recursive module be propagated into FLINT [20] and TIL [23] compilers.) This means that the definition of the recursive module itself. In essence we need not explicitly discuss higher-order module con- the definitions of the type components of a recursive structs in this paper, but rather appeal to HMM for a module must be taken to be the types that they will detailed discussion of their implicit presence. eventually turn out to be once the recursive declara- To support the extension with recursive modules we tion has been processed. Accounting for this “forward enrich the core phase distinction calculus with these ad- reference” is the core contribution of our work. We in- ditional constructs: troduce a new form of signature (interface) for recursive modules, called a recursively dependent signature, that Singleton and dependent kinds to allow expression allows us to capture the required type identities during of type sharing information in signatures. Related type checking of a recursive module binding. This sig- formalisms for expressing type sharing informa- nificantly increases the expressive power of the recursive tion are given by Harper and Lillibridge [9] and module formalism, and is, we assert, of fundamental im- Leroy [13]. portance to the very idea of recursive modules. A fixed point operation for building collections In this paper we aim to focus on the core issues lying of mutually-recursive type constructors. These at the center of a recursive module system, so we study recursive constructors are definitionally equal to recursive modules in the framework of a small internal their unrollings. We term such constructors equi- language that is sufficient to bring out the main issues recursive, to distinguish them from the more con- and that could be used by a type-directed compiler to ventional iso-recursive constructors, for which con- implement recursive modules. Therefore, we make no versions between the constructors and their un- specific proposals as to what form an external language rollings must be mediated by the explicit use of supporting recursive modules should take, although we an isomorphism. We discuss the interplay of equi- do present most of our examples in a hypothetical ex- and iso-recursive constructors in Section 5.3. ternal language. Indeed, some important questions re- garding the design of an external language remain open, A fixed point operation for building collections of such as the practicality of typechecking. In Section 5 we mutually-recursive functions. As will become ap- make some observations and preliminary proposals re- parent later on, we cannot (as in SML) limit this garding the design of an external language.