University of Massachusetts Amherst ScholarWorks@UMass Amherst Computer Science Department Faculty Publication Computer Science Series 1996 Using Reflection to Support Type-Safe Evolution in Persistent Systems G.N.C. Kirby University of St Andrews Follow this and additional works at: https://scholarworks.umass.edu/cs_faculty_pubs Part of the Computer Sciences Commons Recommended Citation Kirby, G.N.C., "Using Reflection to Support Type-Safe Evolution in Persistent Systems" (1996). Computer Science Department Faculty Publication Series. 93. Retrieved from https://scholarworks.umass.edu/cs_faculty_pubs/93 This Article is brought to you for free and open access by the Computer Science at ScholarWorks@UMass Amherst. It has been accepted for inclusion in Computer Science Department Faculty Publication Series by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. This paper should be referenced as: Kirby, G.N.C., Connor, R.C.H., Morrison, R. & Stemple, D. “Using Reflection to Support Type-Safe Evolution in Persistent Systems”. University of St Andrews Technical Report CS/96/10 (1996). Using Reflection to Support Type-Safe Evolution in Persistent Systems G.N.C. Kirby, R.C.H. Connor, R. Morrison and D. Stemple* Department of Mathematical and Computational Sciences, University of St Andrews, North Haugh, St Andrews KY16 9SS, Scotland. *Department of Computer and Information Science, University of Massachusetts, Amherst, MA 01038, USA. Abstract Reflection has been used to address many different problem areas, and the term reflection has itself been used to describe several distinct processes. This paper identifies three simple operations, generation, raising and dynamic rebinding, which may be composed to yield several varieties of reflection. These can be used to allow a self-contained programming system to evolve, through the incorporation of new behaviour into either the application programs or the interpreter which controls their execution. Reflection is a powerful mechanism and potentially dangerous. Used in the context of persistent programming systems, safety is an important consideration: the integrity of large amounts of data may be at stake. This has led to the use of type checking in conjunction with reflection in such systems to provide some guarantees of safety. The paper describes the nature of reflection in persistent systems and identifies some example applications. 1 Introduction Reflection may be defined loosely as the ability for a programming system to change its own behaviour. In general, the behaviour of any system may be captured by a definition of that system with respect to a framework for explaining the definition. For example the behaviour of a program may be captured by the definition of the program itself, and a semantics for the language in which it is written. There are two corresponding mechanisms by which a system can support its own evolution: by changing its own definition, or by changing the interpretation of its definition. The former may be termed linguistic reflection and the latter behavioural reflection. Linguistic reflection is of particular interest in large, long-lived systems and applications. The importance of the mechanism is that it allows evolutionary change within such systems; it is distinguished from other mechanisms by the fact that no lower-level or external semantics are required to effect the change. The ultimate goal of linguistic reflection is to allow self- supporting evolving systems: no matter how the system is required to evolve in the future, it may always be achieved from within, without stepping out of the system into a different semantic domain. Linguistic and behavioural reflection are both potentially dangerous mechanisms, and should be treated with great care. This is particularly important in strongly typed persistent programming systems which may contain large amounts of highly structured and valuable data. One approach to achieving the flexibility provided by reflection in such systems, while imposing sufficient restrictions for safety, is to combine linguistic reflection with type checking. In this way all changes to the system definition are verified by the type checker before use. This is in line with the already extended role of the type checker in such persistent systems, where it is used to control the use of data throughout the lifetime of the data. 1 The remainder of the paper contains a brief description of the distinguishing features of persistent programming systems, a description of the linguistic and behavioural reflective processes in such systems, some examples of their use, and finally a summary of related work. An appendix contains a more detailed description of the reflective processes. 2 Persistent Programming Systems 2.1 Persistence The persistence of data is the length of time for which it exists; this may range from the very short to the very long. Some examples at the extremes of the spectrum are the temporary data created during the evaluation of an expression in a program and the long-term data stored in a company’s customer database. In the first case the data persists for a brief fraction of a single program execution, while in the second the data may outlive the programs that operate on it. Persistence technology is at the core of database programming languages. In an orthogonally persistent programming system, the manner in which data is manipulated is independent of its persistence. The same mechanisms operate on both short-term and long-term data, avoiding the traditional need for separate systems to control access to data of different degrees of longevity. Thus data may remain under the control of a single persistent programming system for its entire lifetime [ABC+83, AM85, Con90]. Since translation between storage formats is automatic and hidden from the programmer, a conceptually simpler programming system results. The principal gains provided by orthogonal persistence are: • improved programming productivity from simpler semantics; • avoidance of the need for ad hoc arrangements for long term data storage, and for data format translations; and • type checking protection mechanisms operate over the whole environment. 2.2 Typing in Persistent Systems Type systems are historically viewed as mechanisms which impose static safety constraints upon a program. Within a persistent environment, however, the type system takes on a wider role. Data manipulated by a programming language is governed by that language’s type system. In non-persistent languages, however, data which persists for longer than the invocation of a program may only be achieved by the use of an operating system or database interface which is shared by all applications. As a consequence of this, such data passes beyond the jurisdiction of the type system of any one language. Mechanisms which govern long term data, such as protection and module binding, must be dealt with at the level of this interface. Historically this has the consequence that the type system may not be enforced, and knowledge of the typed structure of data may not be taken advantage of. In a persistent system, the storage of data beyond a single program invocation is handled by programming language mechanisms, and no common operating system interface is necessary. The only route by which data may be accessed is through the programming language, and so the type system of a single language may be used to enforce protection upon both transient and permanent data. High-level modelling may be relied upon for the entire lifetime of the data, as it never passes outside the language system. The universality of the persistent type system has consequences in terms of both the modelling and protection provided by the type system itself, and also presents some new challenges in terms of implementation. With respect to modelling, the persistent type system must be sufficiently flexible to allow the modelling of activities normally provided by untyped support systems. Such activities include, for example, the linking of separately prepared program units, and file system access protection. 2 With respect to protection, the increased role of a type system means that any protection mechanisms programmed at a high level may be fully relied upon to protect the data for its lifetime, as access from outside the constraints of the type system is not possible. In particular, various high level information hiding techniques may be used to restrict data access, instead of relying upon the normally coarse grain and typeless control provided by outside technologies. 2.3 Longevity and Evolution One of the goals of persistent language research is to develop self-contained programming systems in which all aspects of the software process are represented within a uniform and consistent framework. A programming system will however become obsolete if it can no longer meet the changing needs of the applications that it supports. Evolution is inevitable in a long running system as the people who use the data, the data and the uses to which the data is put all change. This is reflected within the programming system by changes to the data, the programs which use the data and the meta-data (the types of the programs and data). Changes to program and data with the invariant of fixed meta-data may be handled relatively simply by updates to programs and data respectively. In fact an orthogonally persistent system will support persistent first-class procedures, so that programs and other data need not be differentiated—a program may be stored in the persistent system as a procedure value. A particularly difficult problem of evolution is to change the meta-data while keeping all the existing programs and data consistent with the semantics of the change. Here the constraints of the type system, introduced in order to improve safety, act to hinder the required evolution. For example if the definition of type A has been evolved to type B, it may be desirable to make all persistent values of type A now have type B.