DOCSLIB.ORG

Parallel Functional Programming with Skeletons: the Ocamlp3l Experiment

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Parallel Functional Programming with Skeletons: the Ocamlp3l Experiment Parallel Functional Programming with Skeletons the ocamlpl exp eriment ? Marco Danelutto Rob erto Di Cosmo Xavier Leroy Susanna Pelagatti ? Dipartimento di Informatica LIENSDMI INRIA Ro cquencourt Universita di Pisa Ecole Normale Superieure Domaine de Voluceau PISA ITALY PARIS FRANCE ROCQUENCOURT FRANCE parallel the evaluation of all the parameters of a strict func Abstract tion call Both the p ossibility of automatically exploiting Writing parallel programs is not easy and debugging them this kind of implicit parallelism through parallel graph re is usually a nightmare To cop e with these diculties a duction or other compiler techniques and structured approach to parallel programs using skeletons the p ossibility of providing the user with ways to annotate and template based compiler techniques has b een developed somehow functional programs in order to drive parallelism over the past years by several researchers including the PL exploitation have b een explored with dierent results group in Pisa After Coles work on skeletons a new research track has This approach is based on the use of a set of primitive b een initiated concerning parallel functional programming forms that are just functionals implemented via templates with skeletons As an example Bratvold showed how skele exploiting the underlying parallelism so it is natural to ask tons can b e lo oked for and exploited within an ML dialect whether marrying a real functional language like Ocaml with and Darlingtons group at the Imp erial College in Lon the PL skeletons can b e the basis of a p owerful parallel pro don started considering plain functional skeletons and gramming environment We show that this is the case our came up with the denition of a skeleton functional co ordi prototype written entirely in Ocaml using a limited form of nation language closure passing allows a very simple and clean programming Some of the authors of this pap er developed at the Uni style shows real sp eedup over a network of workstations versity of Pisa a skeleton parallel programming language and as an added fundamental b onus allows logical debug pl pl is actually an imp erative programming lan ging of parallel programs in a sequential framework without guage although the skeleton framework used is completely changing the user co de functional The pl compiler uses an original template Key words skeletons functional languages closures based parallelism exploitation technique achieving very go o d parallelism p erformance as well as programmability and p ortability fea tures The authors started lo oking at the p ossibility of exp orting the templatebased skeleton parallelism exploita Introduction tion techniques to the functional world In the meanwhile the Ob jective Caml functional pro Functional programming languages have greatly improved gramming language has b een developed at INRIA Ro cquen since their app earance in the sixties The p erformance court in France Ob jective Caml ocaml in the se achieved in functional program execution is now compara quel is a functional language of the ML family It ble to the p erformance achieved using classical imp erative supp orts b oth functions as rstclass values and full imp er programming languages In addition mo dern func ative features in particular arrays mo diable inplace This tional programming languages have advanced features like combination of features makes it well adapted to skeleton strong typing and advanced mo dule systems that improve based programming higherorder functions functions tak b oth programmer pro ductivity and co de quality and main ing userprovided functions as arguments can b e suitably tainability used to mo delimplement skeletons while the imp erative But what ab out parallel functional programming Since features can b e exploited to provide a parallel implementa the very b eginning it has b een claimed that functional pro tion of skeletons Other useful features of ocaml include a gramming languages are implicitly parallel mainly due to p owerful mo dule system allowing several implementations the p ossibility of using eager evaluation strategies for all the of the skeletons to b e substituted for one another without strict functions app earing in a program Eager evaluation changing the user co de and a builtin marshaler allowing strategies in conjunction with the referential transparency transmission of arbitrary data structures over byte streams prop erty sp orted by pure functional languages allow func based on the same structural information used by the ocaml tional languages compilers or interpreters to schedule in garbage collector 0 The ocamlpl pro ject has b een partially funded by the The goal of the authors at the b eginning of the ocamlpl Galileo bilateral FranceItaly pro ject n Further in pro ject was to assess the merits and the feasibility of the formation on the ocamlpl Ocaml and pl pro jects can integration of the pl language inside ocaml This gave b e found at the URLs httpwwwdiunipiitmarcod ocamlplocamlplhtml httppauillacinriafrocaml and ocamlpl a programming environment that allows to write httpwwwdiunipiitsusannaplhtml parallel programs in ocaml according to the skeleton mo del supp orted by the parallel language pl It provides seamless pip eline stages integration of parallel programming and functional program In ocamlpl we allow skeletons to b e arbitrarily com ming and advanced features like sequential logical debugging p osed and we provide the user with b oth stream parallel of parallel programs and strong typing useful b oth in teach and data parallel skeletons data parallelism b eing the one ing parallel programming and in building fullscale applica coming from p erforming in parallel computations relative to tions In addition we wanted the skeleton implementation a single input data item to run on widely available computer systems The skeleton set of ocamlpl contains most of the skele During the implementation of the system it turned out tons app earing in pl although some of the ocamlpl skele that we could get more than that In our implementation the tons are actually simplied versions of those app earing in user co de containing the skeleton expressions can b e linked pl here we will denote by f the function computed by with dierent mo dules in order to get either a parallel exe skeleton F on a single data item of its input data stream cutable running on a network of workstation or a sequen Farm skeleton The farm skeleton written farm computes tial executable running on a single workstation Therefore in parallel a function f over dierent data items app ear users are enabled to p erform logical program debugging us ing in its input stream From a functional viewp oint ing the sequential version of the program and then to move given a stream of data items x x farmF k to the parallel version by just changing a linker option 1 n computes f x f x F b eing the skeleton pa The high level of abstraction provided by functional pro 1 n rameter Parallelism is exploited by having k inde gramming coupled with the ability to send closures over an p endent parallel pro cesses computing f on dierent IO channel provided the key to an elementary and robust items of the input stream runtime system that consists of a very limited number of lines of co de Pip eline skeleton The pip eline skeleton denoted by the inx op erator p erforms in parallel the computa Skeletons tions relative to dierent stages of a function over dif ferent data items of the input stream Functionally The skeleton parallel programming mo del supp orts struc F F F computes f f f x over 1 2 N n 2 1 i tured parallel programming Using this mo del all the data items x b elonging to the input stream i the parallel structureb ehavior of an application has to b e Parallelism is exploited by having n indep endent par expressed by using skeletons picked up from of a set of prede allel pro cesses Each pro cess computes a function f i ned ones p ossibly in a nested way Each skeleton mo dels a over the data items pro duced by pro cess computing typical pattern of parallel computation and it is parametric f and delivers results to pro cess computing f i1 i+1 in the computation p erformed in parallel Skeletons can b e understo o d as second order functionals that mo del the par Map skeleton The map skeleton written mapvector allel computation coming out from the application of a given computes in parallel a function over all the data items parallelism exploitation pattern to the parameter functions of a vector generating a new vector Therefore for As an example a common skeleton is the pipeline each vector X app earing in the input data stream one Pip eline is a stream parallel skeleton The paral mapvector F n computes the function f over all the lelism exploited comes from p erforming in parallel compu items of the vector using n dierent parallel pro cesses tations relative to dierent input data sets app earing on computing f over distinct vector items the input data stream In particular the pip eline skele Reduce skeleton The reduce skeleton written ton mo dels the parallelism coming from the parallel com reducevector folds a binary asso ciative func putation of dierent stages of a function onto dierent tion over all the data items of a vector Therefore input data items app earing on the input data stream reducevector F n computes x f x f f x out of 1 2 n In other words pipeline f f computes the func 1 n the vector x x for each one of the vectors 1 n
Load more

Recommended publications
  • Functional Languages
    Functional Programming Languages (FPL) 1. Definitions................................................................... 2 2. Applications ................................................................ 2 3. Examples..................................................................... 3 4. FPL Characteristics:.................................................... 3 5. Lambda calculus (LC)................................................. 4 6. Functions in FPLs ....................................................... 7 7. Modern functional languages...................................... 9 8. Scheme overview...................................................... 11 8.1. Get your own Scheme from MIT...................... 11 8.2. General overview.............................................. 11 8.3. Data Typing ...................................................... 12 8.4. Comments ......................................................... 12 8.5. Recursion Instead of Iteration........................... 13 8.6. Evaluation ......................................................... 14 8.7. Storing and using Scheme code ........................ 14 8.8. Variables ........................................................... 15 8.9. Data types.......................................................... 16 8.10. Arithmetic functions ......................................... 17 8.11. Selection functions............................................ 18 8.12. Iteration............................................................. 23 8.13. Defining functions ...........................................
    [Show full text]
  • The Machine That Builds Itself: How the Strengths of Lisp Family
    Khomtchouk et al. OPINION NOTE The Machine that Builds Itself: How the Strengths of Lisp Family Languages Facilitate Building Complex and Flexible Bioinformatic Models Bohdan B. Khomtchouk1*, Edmund Weitz2 and Claes Wahlestedt1 *Correspondence: [email protected] Abstract 1Center for Therapeutic Innovation and Department of We address the need for expanding the presence of the Lisp family of Psychiatry and Behavioral programming languages in bioinformatics and computational biology research. Sciences, University of Miami Languages of this family, like Common Lisp, Scheme, or Clojure, facilitate the Miller School of Medicine, 1120 NW 14th ST, Miami, FL, USA creation of powerful and flexible software models that are required for complex 33136 and rapidly evolving domains like biology. We will point out several important key Full list of author information is features that distinguish languages of the Lisp family from other programming available at the end of the article languages and we will explain how these features can aid researchers in becoming more productive and creating better code. We will also show how these features make these languages ideal tools for artificial intelligence and machine learning applications. We will specifically stress the advantages of domain-specific languages (DSL): languages which are specialized to a particular area and thus not only facilitate easier research problem formulation, but also aid in the establishment of standards and best programming practices as applied to the specific research field at hand. DSLs are particularly easy to build in Common Lisp, the most comprehensive Lisp dialect, which is commonly referred to as the “programmable programming language.” We are convinced that Lisp grants programmers unprecedented power to build increasingly sophisticated artificial intelligence systems that may ultimately transform machine learning and AI research in bioinformatics and computational biology.
    [Show full text]
  • Functional Programming Laboratory 1 Programming Paradigms
    Intro 1 Functional Programming Laboratory C. Beeri 1 Programming Paradigms A programming paradigm is an approach to programming: what is a program, how are programs designed and written, and how are the goals of programs achieved by their execution. A paradigm is an idea in pure form; it is realized in a language by a set of constructs and by methodologies for using them. Languages sometimes combine several paradigms. The notion of paradigm provides a useful guide to understanding programming languages. The imperative paradigm underlies languages such as Pascal and C. The object-oriented paradigm is embodied in Smalltalk, C++ and Java. These are probably the paradigms known to the students taking this lab. We introduce functional programming by comparing it to them. 1.1 Imperative and object-oriented programming Imperative programming is based on a simple construct: A cell is a container of data, whose contents can be changed. A cell is an abstraction of a memory location. It can store data, such as integers or real numbers, or addresses of other cells. The abstraction hides the size bounds that apply to real memory, as well as the physical address space details. The variables of Pascal and C denote cells. The programming construct that allows to change the contents of a cell is assignment. In the imperative paradigm a program has, at each point of its execution, a state represented by a collection of cells and their contents. This state changes as the program executes; assignment statements change the contents of cells, other statements create and destroy cells. Yet others, such as conditional and loop statements allow the programmer to direct the control flow of the program.
    [Show full text]
  • Functional and Imperative Object-Oriented Programming in Theory and Practice
    Uppsala universitet Inst. för informatik och media Functional and Imperative Object-Oriented Programming in Theory and Practice A Study of Online Discussions in the Programming Community Per Jernlund & Martin Stenberg Kurs: Examensarbete Nivå: C Termin: VT-19 Datum: 14-06-2019 Abstract Functional programming (FP) has progressively become more prevalent and techniques from the FP paradigm has been implemented in many different Imperative object-oriented programming (OOP) languages. However, there is no indication that OOP is going out of style. Nevertheless the increased popularity in FP has sparked new discussions across the Internet between the FP and OOP communities regarding a multitude of related aspects. These discussions could provide insights into the questions and challenges faced by programmers today. This thesis investigates these online discussions in a small and contemporary scale in order to identify the most discussed aspect of FP and OOP. Once identified the statements and claims made by various discussion participants were selected and compared to literature relating to the aspects and the theory behind the paradigms in order to determine whether there was any discrepancies between practitioners and theory. It was done in order to investigate whether the practitioners had different ideas in the form of best practices that could influence theories. The most discussed aspect within FP and OOP was immutability and state relating primarily to the aspects of concurrency and performance. ​ ​ ​ ​ ​ ​ This thesis presents a selection of representative quotes that illustrate the different points of view held by groups in the community and then addresses those claims by investigating what is said in literature.
    [Show full text]
  • Functional Javascript
    www.it-ebooks.info www.it-ebooks.info Functional JavaScript Michael Fogus www.it-ebooks.info Functional JavaScript by Michael Fogus Copyright © 2013 Michael Fogus. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/ institutional sales department: 800-998-9938 or [email protected]. Editor: Mary Treseler Indexer: Judith McConville Production Editor: Melanie Yarbrough Cover Designer: Karen Montgomery Copyeditor: Jasmine Kwityn Interior Designer: David Futato Proofreader: Jilly Gagnon Illustrator: Robert Romano May 2013: First Edition Revision History for the First Edition: 2013-05-24: First release See http://oreilly.com/catalog/errata.csp?isbn=9781449360726 for release details. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. Functional JavaScript, the image of an eider duck, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trade‐ mark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.
    [Show full text]
  • Functional Programming Lecture 1: Introduction
    Functional Programming Lecture 13: FP in the Real World Viliam Lisý Artificial Intelligence Center Department of Computer Science FEE, Czech Technical University in Prague [email protected] 1 Mixed paradigm languages Functional programming is great easy parallelism and concurrency referential transparency, encapsulation compact declarative code Imperative programming is great more convenient I/O better performance in certain tasks There is no reason not to combine paradigms 2 3 Source: Wikipedia 4 Scala Quite popular with industry Multi-paradigm language • simple parallelism/concurrency • able to build enterprise solutions Runs on JVM 5 Scala vs. Haskell • Adam Szlachta's slides 6 Is Java 8 a Functional Language? Based on: https://jlordiales.me/2014/11/01/overview-java-8/ Functional language first class functions higher order functions pure functions (referential transparency) recursion closures currying and partial application 7 First class functions Previously, you could pass only classes in Java File[] directories = new File(".").listFiles(new FileFilter() { @Override public boolean accept(File pathname) { return pathname.isDirectory(); } }); Java 8 has the concept of method reference File[] directories = new File(".").listFiles(File::isDirectory); 8 Lambdas Sometimes we want a single-purpose function File[] csvFiles = new File(".").listFiles(new FileFilter() { @Override public boolean accept(File pathname) { return pathname.getAbsolutePath().endsWith("csv"); } }); Java 8 has lambda functions for that File[] csvFiles = new File(".")
    [Show full text]
  • Notes on Functional Programming with Haskell
    Notes on Functional Programming with Haskell H. Conrad Cunningham [email protected] Multiparadigm Software Architecture Group Department of Computer and Information Science University of Mississippi 201 Weir Hall University, Mississippi 38677 USA Fall Semester 2014 Copyright c 1994, 1995, 1997, 2003, 2007, 2010, 2014 by H. Conrad Cunningham Permission to copy and use this document for educational or research purposes of a non-commercial nature is hereby granted provided that this copyright notice is retained on all copies. All other rights are reserved by the author. H. Conrad Cunningham, D.Sc. Professor and Chair Department of Computer and Information Science University of Mississippi 201 Weir Hall University, Mississippi 38677 USA [email protected] PREFACE TO 1995 EDITION I wrote this set of lecture notes for use in the course Functional Programming (CSCI 555) that I teach in the Department of Computer and Information Science at the Uni- versity of Mississippi. The course is open to advanced undergraduates and beginning graduate students. The first version of these notes were written as a part of my preparation for the fall semester 1993 offering of the course. This version reflects some restructuring and revision done for the fall 1994 offering of the course|or after completion of the class. For these classes, I used the following resources: Textbook { Richard Bird and Philip Wadler. Introduction to Functional Program- ming, Prentice Hall International, 1988 [2]. These notes more or less cover the material from chapters 1 through 6 plus selected material from chapters 7 through 9. Software { Gofer interpreter version 2.30 (2.28 in 1993) written by Mark P.
    [Show full text]
  • Purely Functional Data Structures
    Purely Functional Data Structures Chris Okasaki September 1996 CMU-CS-96-177 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Thesis Committee: Peter Lee, Chair Robert Harper Daniel Sleator Robert Tarjan, Princeton University Copyright c 1996 Chris Okasaki This research was sponsored by the Advanced Research Projects Agency (ARPA) under Contract No. F19628- 95-C-0050. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of ARPA or the U.S. Government. Keywords: functional programming, data structures, lazy evaluation, amortization For Maria Abstract When a C programmer needs an efficient data structure for a particular prob- lem, he or she can often simply look one up in any of a number of good text- books or handbooks. Unfortunately, programmers in functional languages such as Standard ML or Haskell do not have this luxury. Although some data struc- tures designed for imperative languages such as C can be quite easily adapted to a functional setting, most cannot, usually because they depend in crucial ways on as- signments, which are disallowed, or at least discouraged, in functional languages. To address this imbalance, we describe several techniques for designing functional data structures, and numerous original data structures based on these techniques, including multiple variations of lists, queues, double-ended queues, and heaps, many supporting more exotic features such as random access or efficient catena- tion. In addition, we expose the fundamental role of lazy evaluation in amortized functional data structures.
    [Show full text]
  • Functional Programming with C++ Template Metaprograms
    Functional Programming with C++ Template Metaprograms Zolt´an Porkol´ab E¨otv¨os Lor´and University, Faculty of Informatics Dept. of Programming Languages and Compilers P´azm´any P´eter s´et´any 1/C H-1117 Budapest, Hungary [email protected] Abstract. Template metaprogramming is an emerging new direction of generative programming: with the clever definitions of templates we can enforce the C++ compiler to execute algorithms at compilation time. Among the application areas of template metaprograms are the expres- sion templates, static interface checking, code optimization with adap- tion, language embedding and active libraries. However, as this feature of C++ was not an original design goal, the language is not capable of elegant expression of template metaprograms. The complicated syn- tax leads to the creation of code that is hard to write, understand and maintain. Despite that template metaprogramming has a strong rela- tionship with functional programming paradigm, language syntax and the existing libraries do not follow these requirements. In this paper we give a short and incomplete introduction to C++ templates and the ba- sics of template metaprogramming. We will enlight the role of template metaprograms, some important and widely used idioms and techniques. 1 Introduction Templates are key elements of the C++ programming language [3, 32]. They enable data structures and algorithms to be parameterized by types, thus cap- turing commonalities of abstractions at compilation time without performance penalties at runtime [37]. Generic programming [28, 27, 17] is a recently popu- lar programming paradigm, which enables the developer to implement reusable codes easily. Reusable components { in most cases data structures and algo- rithms { are implemented in C++ with the heavy use of templates.
    [Show full text]
  • Functional Programming at Work in Object-Oriented Programming
    Functional Programming at Work in Object-Oriented Programming Narbel version 2010 Narbel Functional Programming at Work in Object-Oriented Programming 1 A Claim about Programming Styles Claim: Adding functional programming capabilities to an object-oriented language leads to benefits in object-oriented programming design. Narbel Functional Programming at Work in Object-Oriented Programming 2 Existing Languages with a FP-OOP Mix Some old and less old languages with FP+OOP: For instance, Smalltalk, Common Lisp (CLOS). More recently, Python or Ruby. Notations: FP, Functional programming; OOP, Object-oriented programming, Narbel Functional Programming at Work in Object-Oriented Programming 3 FP techniques emulated in OOP Practices in OOP languages include emulations of FP techniques: C++ programmers: function pointers and overloadings of the () operator, i.e. “object-functions” or functors. Java programmers: anonymous classes and introspection/reflexion. Narbel Functional Programming at Work in Object-Oriented Programming 4 Existence of FP-OOP Comparison Points The idea of using FP to enrich OOP is old, see e.g. the discussions about the problem of the user-defined datatype extension: User-defined types and procedural data structures as complementary approaches to data abstraction. Reynolds. 1975. The Expression Problem. Wadler. 1998. Narbel Functional Programming at Work in Object-Oriented Programming 5 A Trend: FP Extensions for OO Languages A recent trend: to propose and include typed FP extensions in mainstream static OO languages. Extensions for C++ (see e.g. Laufer, Striegnitz, McNamara, Smaragdakis), and work in progress in the C++ standard committees. Java 7 expected to include FP constructs. C# offers FP constructs (even more in its 3.0 version).
    [Show full text]
  • Concurrency and Parallelism in Functional Programming Languages
    Concurrency and Parallelism in Functional Programming Languages John Reppy [email protected] University of Chicago April 21 & 24, 2017 Introduction Outline I Programming models I Concurrent I Concurrent ML I Multithreading via continuations (if there is time) CML 2 Introduction Outline I Programming models I Concurrent I Concurrent ML I Multithreading via continuations (if there is time) CML 2 Introduction Outline I Programming models I Concurrent I Concurrent ML I Multithreading via continuations (if there is time) CML 2 Introduction Outline I Programming models I Concurrent I Concurrent ML I Multithreading via continuations (if there is time) CML 2 Introduction Outline I Programming models I Concurrent I Concurrent ML I Multithreading via continuations (if there is time) CML 2 Programming models Different language-design axes I Parallel vs. concurrent vs. distributed. I Implicitly parallel vs. implicitly threaded vs. explicitly threaded. I Deterministic vs. non-deterministic. I Shared state vs. shared-nothing. CML 3 Programming models Different language-design axes I Parallel vs. concurrent vs. distributed. I Implicitly parallel vs. implicitly threaded vs. explicitly threaded. I Deterministic vs. non-deterministic. I Shared state vs. shared-nothing. CML 3 Programming models Different language-design axes I Parallel vs. concurrent vs. distributed. I Implicitly parallel vs. implicitly threaded vs. explicitly threaded. I Deterministic vs. non-deterministic. I Shared state vs. shared-nothing. CML 3 Programming models Different language-design axes I Parallel vs. concurrent vs. distributed. I Implicitly parallel vs. implicitly threaded vs. explicitly threaded. I Deterministic vs. non-deterministic. I Shared state vs. shared-nothing. CML 3 Programming models Different language-design axes I Parallel vs.
    [Show full text]
  • Functional Programming Patterns in Scala and Clojure Write Lean Programs for the JVM
    Early Praise for Functional Programming Patterns This book is an absolute gem and should be required reading for anybody looking to transition from OO to FP. It is an extremely well-built safety rope for those crossing the bridge between two very different worlds. Consider this mandatory reading. ➤ Colin Yates, technical team leader at QFI Consulting, LLP This book sticks to the meat and potatoes of what functional programming can do for the object-oriented JVM programmer. The functional patterns are sectioned in the back of the book separate from the functional replacements of the object-oriented patterns, making the book handy reference material. As a Scala programmer, I even picked up some new tricks along the read. ➤ Justin James, developer with Full Stack Apps This book is good for those who have dabbled a bit in Clojure or Scala but are not really comfortable with it; the ideal audience is seasoned OO programmers looking to adopt a functional style, as it gives those programmers a guide for transitioning away from the patterns they are comfortable with. ➤ Rod Hilton, Java developer and PhD candidate at the University of Colorado Functional Programming Patterns in Scala and Clojure Write Lean Programs for the JVM Michael Bevilacqua-Linn The Pragmatic Bookshelf Dallas, Texas • Raleigh, North Carolina Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals.
    [Show full text]