University of London Imperial College of Science, Technology and Medicine Department of Computing Automatic Methods for Program Transformation by CHIN Wei Ngan This thesis is submitted in part fulfilment of the requirements for the degree of Doctor of Philosophy (PhD) and the Diploma of the Imperial College (DIC) Submitted - March 1990 Examined - May 1990 Abstract The transformational approach to software development is recognised as an important formal route for software construction. One major benefit of this approach is the possibility of providing machine assistance to this development process. This thesis works towards this goal by systematizing four major classes of transformations into automated methods (or tactics). The first class of transformations involves the fusion of composed expressions to eliminate unnecessary intermediate data structures and function calls. A producer-consumer model of functions is introduced to explain this transformation. With this model, we show how the deforestation algorithm of Wadler can be extended to all first-order programs. The extended transformation algorithm is presented, termination proof given and further improvements suggested. A similar consideration is also developed for compositions in set abstractions. The second class of transformations involves the removal of certain higher-order features from well-typed programs. Three transformation algorithms are developed. Two of them are directly concerned with the removal of higher-order features. Termination proofs for these algorithms are given. A third algorithm preserves full laziness in a manner compatible with higher-order features removal. With these transformations, the extension of deforestation to higher-order programs is also developed. The third class of transformations concerns the removal of redundant computation through tup ling. A survey of past techniques is presented, followed by the development of a new analysis technique to discover eureka tuples to remove redundancy. This new technique makes novel use of selection orderings to search for eureka tuples. We give some methods to determine appropriate orderings and provide extensions to the basic analysis technique. The last class of transformations involves the use of constraints (or invariants) to help improve recursive programs. Both the finite differencing tactic and a new base-case filter promotion tactic are part of this category of transformations. The necessary laws and semantic conditions for these two tactics are systematized. Acknowledgements The author would like to acknowledge the support received from the following parties: National University of Singapore for generously providing the financial support and study leave necessary to pursue this work. John Darlington for the supervision, insight and guidance necessary to bring my research to fruition. Phil Wadler for inspiring much of the work done in Chapter 3 and 4. Chris Hankin for clarifying my thoughts on abstract interpretation David Lillie, Guido Jouret and David Sharp for acting as sounding boards and verifiers for some of my late ideas. D Lillie and G Jouret also provided invaluable feedback on the thesis. Peter Harrison, Tony Field, David Sands, Hessam and Paul Kelly for carefully reading and commenting on various parts of this thesis Helen, Keith, Nigel and Lyndon for bailing me out of the technical intrigues of programming H o p e + and the transformation-based environment The Rest of Functional Programming Section for contributing to a most stimulating place for research. Bee Hoon for her understanding and encouragement. Dedications To Mum and Dad Table of Contents 1. Introduction 1 1.1. The Software Problem 1 1.2. A Suitable Language for Specification and Transformation 2 1.2.1. An Extended Functional Language 3 1.2.2. Types of Abstractions Available 4 1.2.2.1. Data Abstraction 4 1.2.2.2. Program Abstraction 5 1.2.2.3. Specification Abstraction 5 1.2.2.4. System Abstraction 7 1.3. A Suitable Program Transformation Methodology 7 1.3.1. Catalogue versus Generative Set Systems 8 1.3.2. Desirable Criteria for Transformation Systems 9 1.3.3. The Unfold/Fold Methodology 10 1.3.4. Correctness 12 1.3.5. Completeness 13 1.4. Contribution of Thesis 14 2. Automatic Methods 18 2.1. A Pitfall to Avoid 18 2.2. A Comprehensive Transformation System 19 2.2.1. Transformation Processor and the Script Language 20 2.2.2. Program Repository 22 2.2.3. LemmaLibraiy 22 2.2.4. Inference Sub-System 23 2.2.5. Methods or Tactics Bank. 24 2.3. Common Structure of Tactics 25 2.4. An Algorithmic Tactic: Removing Useless Parameters 26 2.4.1. A Simple Functional Language 26 2.4.2. Phase 1: Analysis for Useless Parameters 27 2.4.3. Phase 2: Transformation 30 2.4.3.1. Transformation Algorithm 30 2.4.3.2. An Example 32 2.4.3.3. A Version of t/ which Carries the Definition Set 33 2.4.3.4. A Version of V which Returns Scripts 34 2.5. A Schematic Tactic: Conversion-to-Iteration 35 2.5.1. Constructing a Schematic Rule 36 2.5.2. Applying the Schematic Rule 3 8 2.5.3. The Stages for Rule Applications 41 2.5.4. Wider Applicability of our Approach 42 2.5.5. Benefits of Schematic Rules 43 2.6. Summary of Chapter 2 44 3. Fusion 45 3.1. Introduction 45 3.1.1. Two Forms of Composition 45 3.1.1.1. Functional Composition 45 3.1.1.2. Set Abstraction 47 3.1.2. Consumer-Producer Model 49 (i) 3.2. W adler’s Deforestation 50 3.2.1. Pure Treeless Form 51 3.2.2. Deforestation Theorem 5 2 3.2.3. T ransformation Algorithm 5 3 3.2.4. Making all Steps Explicit 54 3.2.5. Proof for Deforestation Theorem 55 3.2.6. Blazed Treeless Form 59 3.3. Our Extension to Deforestation 61 3.3.1. Parameter-Based Annotation Scheme 62 3.3.1.1. V ariable-Only Criterion 62 3.3.1.2. Linearity Criterion 63 3.3.1.3. Annotation Scheme 64 3.3.2. Extended Treeless Form 65 3.3.3. Extended Deforestation Theorem 66 3.3.4. Transformation Algorithm 67 3.3.5. An Example 71 3.3.6. Proof for Extended Deforestation Theorem 7 2 3.3.7. A Minor Improvement to Extended Deforestation 82 3.4. Deforestation for First-Order Expressions 83 3.4.1. Universal Treelessness 84 3.4.2. Proof of Universal Deforestation Theorem 86 3.4.3. Converting Functions to Universal Treeless Form 87 3.4.3.1. Bottom-Up Conversion Strategy 88 3.4.3.2. Conversion for Non-Recursive Functions 8 8 3.4.3.3. Conversion for Recursive Functions 89 3.4.4. Example of Universal Treeless Conversion 90 3.5. Further Improvements to Deforestation 92 3.5.1. Eliminating Unnecessary Mutual Recursive Functions 93 3.5.2. Linearising Pattern-Matching Parameters 94 3.5.3. Laws to Improve Deforestation 95 3.6. Supercompilation - A Related Work < 96 3.7. Fusion in Set Abstractions 97 3.7.1. Filter Promotion 9 8 3.7.1.1. Desirable Filter Form 98 3.7.1.2. Desirable Generator Form 99 3.7.1.3. Transformation Algorithm 100 3.7.1.4. An Example 102 3.7.1.5. Undesirable Expression Forms 104 3.7.1.5.1. Undesirable Generators 104 3.7.1.5.2. Undesirable Filters 105 3.7.1.6. An Extension (Patterns instead of Variables) 107 3.7.2. Fusions between Generators and Processors 109 3.7.2.1. Desirable Processor Form 109 3.7.2.2. Transformation Algorithm 110 3.7.2.3. An Example 111 3.8. Summary of Chapter 3 112 4. Higher-Order Removal 114 4.1. Introduction 114 4.1.1. A Simple Higher Order Language 115 4.1.2. Overview of General Application Elimination 116 4.1.3. Overview of Higher-Order Argument Elimination 117 4.1.4. Overview of Full Laziness Technique 119 4.1.5. Overview of Fully Lazy Higher-Order Argument Elimination 121 4.1.6. Outline of Stages for the Higher-Order Removal Tactic 122 (ii) 4.2. Elimination of General Applications 124 4.2.1. Transformation Algorithm, A 124 4.2.2. An Example 127 4.2.3. Proof of Theorem 4.2 128 4.2.3.1. Proof for Lemma 4.3 (on applyless form) 128 4.2.3.2. Proof for Lemma 4.4 (on efficiency) 128 4.2.3.3. Proof for Lemma 4.5 (on termination) 128 4.3. Elimination of Higher-Order Arguments 130 4.3.1. Higher-Order Specialised Form 131 4.3.2. Transformation Algorithm, 131 4.3.3. A Simple Example 135 4.3.4. An Optimisation for the Nested Rules 135 4.3.5. Further Examples 136 4.3.5.1. mapreduce 136 4.3.5.2. twice_map 137 4.3.5.3. fix-point operator and % 12 137 4.3.6. Proof of Theorem 4.7 139 4.4. Full Laziness Concern 145 4.4.1. Various Techniques for Full Laziness 146 4.4.1.1. L(a) Extract out GT-MFEs from lambda abstractions. 146 4.4.1.2. L(b) Convert grounded GT-MFEs to CFs 146 4.4.1.3. £(c) Direct Unfolds for non-recursive function calls 147 4.4.1.4. L(d) Indirect Unfolds by Re-Defining non-recursive functions. 147 4.4.1.5. Recursive Functions and the Space Leak Problem. 148 4.4.2. Full Laziness Transformation Algorithm 149 4.4.2.1. The L rules 149 4.4.2.2. The Ernies 151 4.4.2.3. The rules 153 4.4.3. Fully Lazy Elimination of Higher-Order Arguments 155 4.4.4. An Example.-. - • 156 4.4.5. Possible Losses of Efficiency 157 4.4.5.1.