DOCSLIB.ORG

By Joseph Ray Davidson

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
By Joseph Ray Davidson AN INFORMATION THEORETIC APPROACH TO THE EXPRESSIVENESS OF PROGRAMMING LANGUAGES by Joseph Ray Davidson Submitted in fulfilment of the requirements for the degree of Doctor of Philosophy University of Glasgow College of Science and Engineering School of Computing Science July 2015 The copyright in this thesis is owned by the author. Any quotation from the thesis or use of any of the information contained in it must acknowledge this thesis as the source of the quotation or information. Abstract The conciseness conjecture is a longstanding notion in computer science that programming languages with more built-in operators, that is more expressive languages with larger semantics, produce smaller programs on average. Chaitin defines the related concept of an elegant program such that there is no smaller program in some language which, when run, produces the same output. This thesis investigates the conciseness conjecture in an empirical manner. Influenced by the concept of elegant programs, we investigate several models of computation, and implement a set of functions in each programming model. The programming models are Turing Machines, λ-Calculus, SKI, RASP, RASP2, and RASP3. The information content of the programs and models are measured as characters. They are compared to investigate hypotheses relating to how the mean program size changes as the size of the semantics change, and how the relationship of mean program sizes between two models compares to that between the sizes of their semantics. We show that the amount of information present in models of the same paradigm, or model family, is a good indication of relative expressivity and aver- age program size. Models that contain more information in their semantics have smaller average programs for the set of tested functions. In contrast, the rela- tive expressiveness of models from differing paradigms, is not indicated by their relative information contents. RASP and Turing Machines have been implemented as Field Programmable Gate Array (FPGA) circuits to investigate hardware analogues of the hypotheses above. Namely that the amount of information in the semantics for a model directly influences the size of the corresponding FPGA circuit, and that the rela- tionship of mean circuit sizes between models is comparable to the relationship of mean program sizes. We show that the number of components in the circuits that realise the se- mantics and programs of the models correlates with the information required to implement the semantics and program of a model. However, the number of com- ponents to implement a program in a circuit for one model does not relate to the number of components implementing the same program in another model. This is in contrast to the more abstract implementations of the programs. Information is a computational resource and therefore follows the rules of Blum’s axioms. These axioms and the speedup theorem are used to obtain an alternate proof of the undecidability of elegance. This work is a step towards unifying the formal notion of expressiveness with the notion of algorithmic information theory and exposes a number of interesting research directions. A start has been made on integrating the results of the thesis with the formal framework for the expressiveness of programming languages. 2 Contents 1 Introduction 13 1.1 Motivation............................... 15 1.2 InvestigationOverview . 15 1.3 Hypotheses .............................. 16 1.4 Contributions ............................. 18 1.5 Structure ............................... 21 1.6 Publications.............................. 21 2 Literature Review 23 2.1 Computability............................. 23 2.1.1 HilbertandGödel. 23 2.1.2 Church and Turing . 26 2.2 Information and Algorithmic Theories . 29 2.2.1 Kolmogorov-Chaitin Complexity . 30 2.2.2 Elegance............................ 31 2.2.3 Other Measures of Complexity . 32 2.3 ModelsofComputation. 33 2.3.1 Imperative/Procedural Languages . 33 2.3.2 Functional Language . 41 2.4 Semantics ............................... 51 2.4.1 Structured Operational Semantics . 52 2.5 Expressiveness............................. 54 2.5.1 Formalisations. 55 2.5.2 Formalising Expressiveness . 56 2.5.3 The Conciseness Conjecture . 58 3 2.6 Conclusion............................... 58 3 Preliminaries 60 3.1 HypothesesRevisited. 60 3.1.1 BlumsAxioms......................... 60 3.1.2 The Semantic Information and Total Information Hypotheses 63 3.1.3 The Semantic Circuit and Total Circuit Hypotheses . 70 3.1.4 HypothesesSummary. 72 3.2 ComparisonMetrics. 73 3.3 Formats ................................ 74 3.3.1 Semantics ........................... 74 3.3.2 Turing Machines . 75 3.3.3 RASPmachines........................ 76 3.3.4 λ-calculus ........................... 76 3.3.5 SKIcombinators ....................... 78 3.4 Semantics ............................... 79 3.4.1 Turing Machines . 80 3.4.2 RASPMachines........................ 84 3.4.3 λ-calculus ........................... 92 3.4.4 SKI combinator calculus . 98 3.5 SemanticSizes............................. 100 4 Arithmetic, List and Universal Programs 103 4.1 PrimitiveandPartialRecursion . 103 4.2 TheArithmeticFunctions . 105 4.2.1 Addition............................ 106 4.2.2 Subtraction .......................... 108 4.2.3 Equality ............................ 110 4.2.4 Multiplication . 112 4.2.5 Division ............................ 114 4.2.6 Exponentiation . 117 4.3 FunctionsonaList .......................... 119 4.3.1 ListMembership . 119 4 4.3.2 LinearSearch ......................... 122 4.3.3 ReversingaList. 123 4.3.4 Statefully Reversing a List . 127 4.3.5 BubbleSort .......................... 130 4.4 UniversalMachines . 134 4.4.1 Universal Turing Machines . 135 4.4.2 Universal RASP Machines . 139 4.5 Results................................. 143 4.6 Conclusion............................... 144 5 Circuit Information 146 5.1 InfiniteRegress ............................ 146 5.2 Background .............................. 148 5.2.1 Architecture and Components . 149 5.3 Implementations ........................... 150 5.3.1 RASP ............................. 153 5.3.2 TM............................... 154 5.4 Results................................. 159 6 Analysis 163 6.1 OverallTrends ............................ 164 6.1.1 Arithmetic........................... 164 6.1.2 List .............................. 166 6.1.3 Universal ........................... 166 6.2 GroupedAnalysis........................... 167 6.2.1 RASPMachines........................ 169 6.2.2 RASPvsTM ......................... 171 6.2.3 SKI vs λ-calculus ....................... 172 6.2.4 RASPvsSKI ......................... 174 6.2.5 RASP vs λ-calculus...................... 175 6.2.6 TMvsSKI .......................... 176 6.2.7 TM vs λ-calculus ....................... 177 6.2.8 The SI and TI Hypotheses . 178 5 6.3 FPGAAnalysis ............................ 180 6.3.1 RASPsonFPGAs ...................... 187 6.3.2 RASPvsTM ......................... 189 6.3.3 The SC and TC Hypotheses . 190 6.4 FurtherObservations . 193 6.4.1 Phase Transitions . 193 6.4.2 Interpretation vs Evaluation Semantics . 195 6.5 Inputs ................................. 199 6.5.1 RASPs............................. 199 6.5.2 TM............................... 201 6.5.3 λ-Calculus........................... 202 6.5.4 SKI .............................. 203 6.5.5 Comparison .......................... 205 6.6 TheUTM ............................... 206 6.6.1 Neary’sUTM ......................... 206 6.6.2 Encodings ........................... 208 6.6.3 InputGrowth ......................... 213 6.7 Conclusions .............................. 214 7 Discussion and Conclusion 217 7.1 ThisWork............................... 217 7.1.1 Aims.............................. 217 7.1.2 Method ............................ 218 7.1.3 Results............................. 219 7.2 RelatedAspects............................ 223 7.2.1 Conservative Extensions . 223 7.2.2 Compilation . 228 7.2.3 Types ............................. 229 7.2.4 SemanticSchemes. 230 7.2.5 Related Minimalism . 232 7.3 FurtherInvestigations . 235 7.3.1 Formalism........................... 236 7.3.2 Program Equivalences . 238 6 7.3.3 InputSizes .......................... 239 7.3.4 Phase Transitions . 241 7.3.5 Symbol Grounding . 242 7.3.6 OtherWork .......................... 244 Bibliography 245 A The Busy Beaver Problem 254 A.1 TuringMachineBusyBeavers . 254 A.2 RASPBusyBeavers ......................... 255 A.3 Finding the Champions . 256 A.3.1 BruteForceMethods . 256 A.3.2 GeneticAlgorithms . 257 A.4 Reflection ............................... 259 A.4.1 LandscapeandFitness . 259 A.4.2 ArchitectureandSeeding. 260 B Full Programs 261 B.1 RASP ................................. 261 B.1.1 Addition............................ 261 B.1.2 Subtraction .......................... 262 B.1.3 Equality ............................ 262 B.1.4 Multiplication. 263 B.1.5 Division ............................ 264 B.1.6 Exponentiation . 265 B.1.7 ListMembership . 266 B.1.8 LinearSearch ......................... 267 B.1.9 ListReversal ......................... 268 B.1.10 StatefulListReversal. 269 B.1.11BubbleSort .......................... 270 B.1.12UniversalTM . 272 B.1.13UniversalRASP. 274 B.2 RASP2................................. 278 B.2.1 Addition............................ 279 7 B.2.2 Subtraction .......................... 279 B.2.3 Equality ............................ 279 B.2.4 Multiplication. 280 B.2.5 Division ...........................
Load more

Recommended publications
  • Enriched Lawvere Theories for Operational Semantics
    Enriched Lawvere Theories for Operational Semantics John C. Baez and Christian Williams Department of Mathematics U. C. Riverside Riverside, CA 92521 USA [email protected], [email protected] Enriched Lawvere theories are a generalization of Lawvere theories that allow us to describe the operational semantics of formal systems. For example, a graph-enriched Lawvere theory describes structures that have a graph of operations of each arity, where the vertices are operations and the edges are rewrites between operations. Enriched theories can be used to equip systems with oper- ational semantics, and maps between enriching categories can serve to translate between different forms of operational and denotational semantics. The Grothendieck construction lets us study all models of all enriched theories in all contexts in a single category. We illustrate these ideas with the SKI-combinator calculus, a variable-free version of the lambda calculus. 1 Introduction Formal systems are not always explicitly connected to how they operate in practice. Lawvere theories [20] are an excellent formalism for describing algebraic structures obeying equational laws, but they do not specify how to compute in such a structure, for example taking a complex expression and simplify- ing it using rewrite rules. Recall that a Lawvere theory is a category with finite products T generated by a single object t, for “type”, and morphisms tn ! t representing n-ary operations, with commutative diagrams specifying equations. There is a theory for groups, a theory for rings, and so on. We can spec- ify algebraic structures of a given kind in some category C with finite products by a product-preserving functor m : T ! C.
    [Show full text]
  • Proofs As Cryptography: a New Interpretation of the Curry-Howard Isomorphism for Software Certificates Amrit Kumar, Pierre-Alain Fouque, Thomas Genet, Mehdi Tibouchi
    Proofs as Cryptography: a new interpretation of the Curry-Howard isomorphism for software certificates Amrit Kumar, Pierre-Alain Fouque, Thomas Genet, Mehdi Tibouchi To cite this version: Amrit Kumar, Pierre-Alain Fouque, Thomas Genet, Mehdi Tibouchi. Proofs as Cryptography: a new interpretation of the Curry-Howard isomorphism for software certificates. 2012. hal-00715726 HAL Id: hal-00715726 https://hal.archives-ouvertes.fr/hal-00715726 Preprint submitted on 9 Jul 2012 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Proofs as Cryptography: a new interpretation of the Curry-Howard isomorphism for software certificates K. Amrit 1, P.-A. Fouque 2, T. Genet 3 and M. Tibouchi 4 Abstract The objective of the study is to provide a way to delegate a proof of a property to a possibly untrusted agent and have a small certificate guaranteeing that the proof has been done by this (untrusted) agent. The key principle is to see a property as an encryption key and its proof as the related decryption key. The protocol then only consists of sending a nonce ciphered by the property. If the untrusted agent can prove the property then he has the corresponding proof term (λ-term) and is thus able to decrypt the nonce in clear.
    [Show full text]
  • Download (2MB)
    Urlea, Cristian (2021) Optimal program variant generation for hybrid manycore systems. PhD thesis. http://theses.gla.ac.uk/81928/ Copyright and moral rights for this work are retained by the author A copy can be downloaded for personal non-commercial research or study, without prior permission or charge This work cannot be reproduced or quoted extensively from without first obtaining permission in writing from the author The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the author When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given Enlighten: Theses https://theses.gla.ac.uk/ [email protected] Optimal program variant generation for hybrid manycore systems Cristian Urlea Submitted in fulfilment of the requirements for the Degree of Doctor of Philosophy School of Engineering College of Science and Engineering University of Glasgow January 2021 Abstract Field Programmable Gate Arrays (FPGAs) promise to deliver superior energy efficiency in het- erogeneous high performance computing (HPC), as compared to multicore CPUs and GPUs. The rate of adoption is however hampered by the relative difficulty of programming FPGAs. High-level synthesis tools such as Xilinx Vivado, Altera OpenCL or Intel’s HLS address a large part of the programmability issue by synthesizing a Hardware Description Languages (HDL) representation from a high-level specification of the application, given in programming lan- guages such as OpenCL C, typically used to program CPUs and GPUs. Although HLS solutions make programming easier, they fail to also lighten the burden of optimization.
    [Show full text]
  • Visualizing the Turing Tarpit
    Visualizing the Turing Tarpit Jason Hemann Eric Holk Indiana University {jhemann,eholk}@cs.indiana.edu Abstract In this paper we begin by refreshing some of the the- Minimalist programming languages like Jot have limited in- ory behind this work, including the λ-calculus and its var- terest outside of the community of languages enthusiasts. ious reduction strategies, universal combinators and their This is a shame, as the simplicity of these languages as- application to languages like Iota and Jot (Section 2). Using cribes to them an inherent beauty and provides deep in- this, we devise several strategies for plotting the behavior of sight into the nature of computation. We present a fun way large numbers of Jot programs (Section 3). We make several of visualizing the behavior of many Jot programs at once, observations about the resulting visualizations (Section 4), providing interesting images and also hinting at somewhat describe related work in the field (Section 5), and suggest non-obvious relationships between programs. In the same possibilities for future work (Section 6). way that research into fractals yielded new mathematical insights, visualizations such as those presented here could yield new insights into the structure and nature of compu- tation. Categories and Subject Descriptors F.4.1 [Mathemat- ical Logic]: Lambda calculus and related systems; D.2.3 [Coding Tools and Techniques]: Pretty printers General Terms language, visualization Keywords Jot, reduction, tarpit, Iota 1. Introduction A Turing tarpit is a programming language that is Turing complete, but so bereft of features that it is impractical for use in actual programming tasks, often even for those con- sidered trivial in more conventional languages.
    [Show full text]
  • Arxiv:1610.02247V3 [Cs.LO] 16 Oct 2016 Aeifrain O Xml,Cnie Hsad Ento Famon a of Definition Agda “Interface This an Consider Or Most Example, “Module” [7]
    Logic as a distributive law Michael Stay1 and L.G. Meredith2 1 Pyrofex Corp. [email protected] 2 Synereo, Ltd [email protected] Abstract. We present an algorithm for deriving a spatial-behavioral type system from a formal presentation of a computational calculus. Given a 2-monad Calc : Cat → Cat for the free calculus on a category of terms and rewrites and a 2-monad BoolAlg for the free Boolean algebra on a category, we get a 2-monad Form = BoolAlg + Calc for the free category of formulae and proofs. We also get the 2-monad BoolAlg ◦Calc for subsets of terms. The interpretation of formulae is a natural transformation J−K: Form ⇒ BoolAlg◦Calc defined by the units and multiplications of the monads and a distributive law transformation δ : Calc ◦ BoolAlg ⇒ BoolAlg ◦ Calc. This interpretation is consistent both with the Curry-Howard isomor- phism and with realizability. We give an implementation of the “possibly” modal operator parametrized by a two-hole term context and show that, surprisingly, the arrow type constructor in the λ-calculus is a specific case. We also exhibit nontrivial formulae encoding confinement and liveness properties for a reflective higher-order variant of the π-calculus. 1 Introduction Sylvester coined the term “universal algebra” to describe the idea of expressing a mathematical structure as set equipped with functions satisfying equations; the idea itself was first developed by Hamilton and de Morgan [7]. Most modern programming languages have a notion of a “module” or an “interface” with this arXiv:1610.02247v3 [cs.LO] 16 Oct 2016 same information; for example, consider this Agda definition of a monoid.
    [Show full text]
  • Download/Pdf/81105779.Pdf
    UC Riverside UC Riverside Previously Published Works Title Enriched Lawvere Theories for Operational Semantics Permalink https://escholarship.org/uc/item/3fz1k4k1 Authors Baez, John C Williams, Christian Publication Date 2020-09-15 DOI 10.4204/eptcs.323.8 Peer reviewed eScholarship.org Powered by the California Digital Library University of California Enriched Lawvere Theories for Operational Semantics John C. Baez and Christian Williams Department of Mathematics U. C. Riverside Riverside, CA 92521 USA [email protected], [email protected] Enriched Lawvere theories are a generalization of Lawvere theories that allow us to describe the operational semantics of formal systems. For example, a graph-enriched Lawvere theory describes structures that have a graph of operations of each arity, where the vertices are operations and the edges are rewrites between operations. Enriched theories can be used to equip systems with oper- ational semantics, and maps between enriching categories can serve to translate between different forms of operational and denotational semantics. The Grothendieck construction lets us study all models of all enriched theories in all contexts in a single category. We illustrate these ideas with the SKI-combinator calculus, a variable-free version of the lambda calculus. 1 Introduction Formal systems are not always explicitly connected to how they operate in practice. Lawvere theories [20] are an excellent formalism for describing algebraic structures obeying equational laws, but they do not specify how to compute in such a structure, for example taking a complex expression and simplify- ing it using rewrite rules. Recall that a Lawvere theory is a category with finite products T generated by a single object t, for “type”, and morphisms tn ! t representing n-ary operations, with commutative diagrams specifying equations.
    [Show full text]
  • Application of Category-Theory Methods to the Design of a System of Modules for a Functional Programming Language
    Warsaw University Faculty of Mathematics, Informatics and Mechanics Mikołaj Konarski Application of Category-Theory Methods to the Design of a System of Modules for a Functional Programming Language Supervisor Prof. dr hab. Andrzej Tarlecki Institute of Informatics Warsaw University °c Mikołaj Konarski 2006 Author’s Declaration I declare that I have composed this dissertation myself. Supervisor’s Declaration The dissertation is ready for review. Abstract Module systems embed elements of methodologies, formalisms and tools for programming-in-the-large into programming languages. In our thesis we design a module system, its categorical model and a constructive semantics of the system in the model. The categorical model is simple and general, built using elementary categorical notions, admitting many variants and ex- tensions of the module system. Our module system features programming mechanisms inspired by category theory and enables fine-grained modular programming, as witnessed by the case studies presented. We show how to extend our categorical model with data types express- ible as adjunctions, as well as with (co)inductive constructions and with fully parameterized exponents. We analyze and compare alternative axiom- atizations of the extensions and demonstrate a series of properties concern- ing rewriting of terms denoting morphisms of 2-categories; in particular, the terms expressing general adjunctions. On the basis of the extended model and without introducing recursive types we define the mechanism of mutual dependency among modules, in the form of (co)inductive module construction. The extended module sys- tem contains the essential mechanisms of modular programming, such as type sharing, transparent as well as non-transparent functor application and grouping of related modules.
    [Show full text]
  • Enriched Lawvere Theories for Operational Semantics
    ENRICHED LAWVERE THEORIES FOR OPERATIONAL SEMANTICS John C. Baez Department of Mathematics University of California Riverside CA, USA 92521 and Centre for Quantum Technologies National University of Singapore Singapore 117543 Christian Williams Department of Mathematics University of California Riverside CA, USA 92521 email: [email protected], [email protected] September 4, 2019 Abstract. Enriched Lawvere theories are a generalization of Lawvere theories that allow us to describe the operational semantics of formal systems. For example, a graph-enriched Lawvere theory describes structures that have a graph of operations of each arity, where the vertices are operations and the edges are rewrites between operations. Enriched theories can be used to equip systems with operational semantics, and maps between enriching categories can serve to translate between different forms of operational and denotational semantics. The Grothendieck construction lets us study all models of all enriched theories in all contexts in a single category. We illustrate these ideas with the SKI-combinator calculus, a variable-free version of the lambda calculus, and with Milner's calculus of communicating processes. 1. Introduction Formal systems are not always explicitly connected to how they operate in practice. Lawvere theories [19] are an excellent formalism for describing algebraic structures obeying equational laws, but they do not specify how to compute in such a structure, for example taking a complex expression and simplifying it using rewrite rules. Recall that a Lawvere theory is a category with finite products T generated by a single object t, for \type", and morphisms tn ! t representing n-ary operations, with commutative diagrams specifying equations.
    [Show full text]
  • Verified Translation of a Strongly Typed Functional Language with Variables to a Language of Indexed Gates
    Verified Translation of a Strongly Typed Functional Language with Variables to a Language of Indexed Gates Rob Spoel 2019 Contents 1 Abstract 1 2 Research goal 2 2.1 Preamble .............................. 2 2.2 Research statement ......................... 2 2.3 Contribution ............................. 2 2.4 Scientific relevance ......................... 3 3 Background 4 3.1 Dependently typed programming: Agda .............. 4 3.2 The Curry-Howard isomorphism .................. 4 3.3 Hardware design .......................... 5 3.4 Verified translation ......................... 6 3.5 Embeddings and EDSLs ...................... 7 3.5.1 Variable binding in embedded domain specific languages . 7 3.5.2 Nameless .......................... 7 3.5.3 De Bruijn .......................... 8 3.5.4 HOAS/PHOAS ....................... 8 4 SKI transpiler 10 4.1 Simply-typed 휆-calculus ...................... 10 4.2 SKI combinators .......................... 12 4.3 Translation ............................. 13 4.4 Correctness ............................. 15 5 Π-Ware and Λ1 16 5.1 Π-Ware ............................... 16 5.2 Plugs versus named variables .................... 18 5.3 Λ1 .................................. 18 5.3.1 Type universe ........................ 20 5.3.2 Variable bindings ...................... 22 5.3.3 Gates ............................ 23 6 Translation 24 6.1 Intermediate language ........................ 24 6.2 Atomization of polytypes ...................... 26 6.3 Stage 1 ............................... 28 6.3.1 Translation ........................
    [Show full text]