Literate Programming, a Practioner's View

Total Page:16

File Type:pdf, Size:1020Kb

Literate Programming, a Practioner's View Literate Programming, A Practioner’s View Bart Childs Texas A&M University Department of Computer Science College Station, TX 77843-3112 Phone (409) 845-5470; FAX (409) 847-8578 Internet: [email protected] Abstract I have been using the WEB style of Literate Programming since my first efforts to port TEX to the Data General AOS system. When I looked back at those efforts, the work in porting drivers that were not written in WEB and the writing of drivers in WEB (based upon DVITYPE, of course), the value of this method of programming became evident. I have concentrated my research (and some teaching) efforts upon this style of programming. I will relate my insights and opinions of the following: some quantitative and qualitative measures of the value of WEB programming; a description of some tools that are part of an environment for writing and maintaining literate programs; literate programming environments that are alternatives to the WEB style; an annotated list of some literate programming systems; and I will conclude with my perception of the future of literate programming. Introduction A Definition Donald Knuth created the WEB system of literate I use the following list of requirements to imply a programming when he wrote the TEX typesetting definition of a literate program and the minimum system a second time (see “The WEB system set of tools which are needed to prepare, use, and of structured documentation”, 1983; “Literate study the resulting code. programming”, 1984; TEX: The Program, 1986; • The high-level language code and the system METAFONT WEB and :TheProgram). The system documentation of the program come from the can be described as the merging of documentation, same set of source files. code, and presenting the listings in a typeset format • The documentation and high-level language with aids of table of contents, cross-referencing, and code are complementary and should address indices. the same elements of the algorithms being I have used Knuth’s original WEB system and written. several descendants for a number of years. It is • The literate program should have logical my opinion that the training necessary to learn how subdivisions. Knuth called these modules or to program in a literate style is relatively small. I sections. believe the benefits of literate programming make • The system should be presented in an order it worthwhile. The benefits are better and more based upon logical considerations rather than maintainable code (it can be argued that this is not syntactic constraints. proven.) • The documentation should include an examina- In this paper I will also report on available tion of alternative solutions and should suggest literate programming systems, some of my successes future maintenance problems and extensions. and failures as a literate programmer, creation of • The documentation should include a descrip- some tools to aid the literate programmer, the tion of the problem and its solution. This community of literate programmers that I have should include all aids such as mathematics been able to identify, alternatives to the WEB system, and graphics that enhance communication of and suggested directions for use and research in the problem statement and the understanding literate programming. of its challenge. TUGboat, Volume 13(1992), No. 3—Proceedings of the 1992 Annual Meeting 261 Bart Childs • Cross references, indices, and different fonts for by today’s systems programmers but sometimes it is text, high-level language keywords, variable a convenient way to represent certain ideas. (Knuth names, and literals should be reasonably used Pascal because “it was everybody’s second automatic and obvious in the source and the best language” at the time (Knuth, TUGboat, 7(2), documentation. 1986). It was before C was widely available.) These requirements have been adapted from (Knuth, 1992), and (VanWyk, 1989 and 1990). Some of My Successes and Failures My adaptations of the list were affected by my My introduction to the WEB system of literate experience as a WEB user — first in a maintenance programming was rather abrupt — I was porting mode, then as an author, and finally using T X82. I had not written a Pascal program or a WEB in undergraduate and graduate education E T X document at the time. I had written a number environments. The last has involved the creation E of systems programs in PL/1. of some tools to enhance the use of literate Version 0.6 of the T Xsystemwasmade programming in all environments. E available to me in the fall semester of 1982. Knuth posed this thought to introduce literate Distribution tapes contained about 300 files. Most programming: “Instead of imagining that our main of these had to do with the fonts in a binary form. task is to instruct a computer what to do, let us Binary files were written in a format of twenty bytes concentrate rather on explaining to human beings per record. Each byte was converted to four ASCII what we want a computer to do” (Knuth, 1984). characters; for example, ‘255’ for the byte with all His thesis was that it should be just as important to bits being one. Over a two or three week period I communicate with the other persons who read the ported the TANGLE processor and began work on the program as it is to communicate with the computer WEAVE processor. I also had the ‘report version’ of which executes it (Knuth, 1992). David Ness said volume B of the C&Tseries (Knuth, 1986). “it is the most important task.” Theendofthesemestercausedmetostopthe work and I did not resume until the spring with Knuth’s WEB System version 0.9 of TEX. I spent some time reviewing the When Don Knuth wrote TEX the second time, changes in the necessary parts of the TEXsystem: he gave thought to making it portable to many TANGLE, WEAVE,andTEX. different systems. WEB was created as a superset of The WEB sources were usually complemented Unix TEXandPascal. with the change files for TOPS, VMS, and WEB’s design encourages writing programs in systems. These were valuable, but I recall the small chunks which Knuth called modules (he also feeling that the “system dependencies” entries in used the term sections). Modules have three parts: the index were even more valuable. It helped to documentation, definitions, and code.Atleastone see what was changed for other systems, but often of these three parts must be non-null. those changes were in terms of “system calls” whose The documentation portion is often a verbal documentation I did not have. description of the algorithm. It may be any The necessary codes are approximately 2,700, textual information that aids the understanding of 7,000, and 25,000 lines of Pascal each of the the problem. It is not uncommon for a WEB to INITeX and TEX processors. I was able to port have a number of ‘documentation only’ modules. approximately 60,000 lines of Pascal code to a These usually describe the problem independent system on which only 2,700 lines had previously of the chosen language for implementation. For been ported. I did this in three days (probably 15 example, a WEB for a subprogram that solves the hours of work). I regret that I did not keep a diary linear equation, Ax = b, could have discussion of or log errors like Don Knuth did, (Knuth, TUGboat, singularity, condition numbers, partial pivoting, the 7(2), 1986; and Knuth, chapters 10 and 11, 1992). banded nature of the expected coefficient matrices, The experience of porting the system convinced etc. It should be an unusual but not exceptional me that the literate programming style had case when a module contains no documentation. significant merit. Most of the programs I The definition part of a module was often subsequently wrote for the TEXsystemwereWEBs. used by Knuth to offset shortcomings of Pascal The existence of dvitype asamodelfordrivers when used in systems programming. (Wirth created helped this decision. I created a family of drivers for Pascal to be a language for pedagogy, not systems the AOS system for QMS, HP, and Canon printers. programming.) This is rarely the language of choice These used a common source and adaptations for 262 TUGboat, Volume 13(1992), No. 3—Proceedings of the 1992 Annual Meeting Literate Programming, A Practioner’s View the different printers were accomplished by change • WEB systems that have been adapted to a files. different high-level language by pre-processors This further experience led to the logical and/or post-processors. These include support conclusion that an “environment” could help for Fortran,andthefirstWEB for Reduce. significantly. We created several environments • WEB systems that have a different basis for to automate the steps of creation of code and their creation but generally follow the same WEB documentation for the AOS system. The emergence concepts. Ramsey’s Spider enables reasonably of Unix and availability of workstations has easy creation of WEBs for several different relegated these early works to simply being a languages (Ramsey, 1989). His example include pleasant memory that in some sense could be called WEBs for Ada, C, awk, and SPL. Gragert and a failure. Roelofs created the second WEB for Reduce with Marcus Brown built part of an “Interactive Spider (Gragert, 1991). They are now creating Environment for Literate Programming” as part of a WEB for Maple (Gragert, 1992). his dissertation under my direction (Brown, 1988 • WEB systems using a different language or and 1990). This was essentially the output side of formatting system. Three will be mentioned. an editor which also allowed the navigation of the Thimbelby did his CWEB with the Unix source with views of the typeset output, a graphical standards of C and troff (Thimbelby, 1986).
Recommended publications
  • Donald Knuth Fletcher Jones Professor of Computer Science, Emeritus Curriculum Vitae Available Online
    Donald Knuth Fletcher Jones Professor of Computer Science, Emeritus Curriculum Vitae available Online Bio BIO Donald Ervin Knuth is an American computer scientist, mathematician, and Professor Emeritus at Stanford University. He is the author of the multi-volume work The Art of Computer Programming and has been called the "father" of the analysis of algorithms. He contributed to the development of the rigorous analysis of the computational complexity of algorithms and systematized formal mathematical techniques for it. In the process he also popularized the asymptotic notation. In addition to fundamental contributions in several branches of theoretical computer science, Knuth is the creator of the TeX computer typesetting system, the related METAFONT font definition language and rendering system, and the Computer Modern family of typefaces. As a writer and scholar,[4] Knuth created the WEB and CWEB computer programming systems designed to encourage and facilitate literate programming, and designed the MIX/MMIX instruction set architectures. As a member of the academic and scientific community, Knuth is strongly opposed to the policy of granting software patents. He has expressed his disagreement directly to the patent offices of the United States and Europe. (via Wikipedia) ACADEMIC APPOINTMENTS • Professor Emeritus, Computer Science HONORS AND AWARDS • Grace Murray Hopper Award, ACM (1971) • Member, American Academy of Arts and Sciences (1973) • Turing Award, ACM (1974) • Lester R Ford Award, Mathematical Association of America (1975) • Member, National Academy of Sciences (1975) 5 OF 44 PROFESSIONAL EDUCATION • PhD, California Institute of Technology , Mathematics (1963) PATENTS • Donald Knuth, Stephen N Schiller. "United States Patent 5,305,118 Methods of controlling dot size in digital half toning with multi-cell threshold arrays", Adobe Systems, Apr 19, 1994 • Donald Knuth, LeRoy R Guck, Lawrence G Hanson.
    [Show full text]
  • Knuthweb.Pdf
    Literate Programming Donald E. Knuth Computer Science Department, Stanford University, Stanford, CA 94305, USA The author and his associates have been experimenting for the past several years with a program- ming language and documentation system called WEB. This paper presents WEB by example, and discusses why the new system appears to be an improvement over previous ones. I would ordinarily have assigned to student research A. INTRODUCTION assistants; and why? Because it seems to me that at last I’m able to write programs as they should be written. The past ten years have witnessed substantial improve- My programs are not only explained better than ever ments in programming methodology. This advance, before; they also are better programs, because the new carried out under the banner of “structured program- methodology encourages me to do a better job. For ming,” has led to programs that are more reliable and these reasons I am compelled to write this paper, in easier to comprehend; yet the results are not entirely hopes that my experiences will prove to be relevant to satisfactory. My purpose in the present paper is to others. propose another motto that may be appropriate for the I must confess that there may also be a bit of mal- next decade, as we attempt to make further progress ice in my choice of a title. During the 1970s I was in the state of the art. I believe that the time is ripe coerced like everybody else into adopting the ideas of for significantly better documentation of programs, and structured programming, because I couldn’t bear to be that we can best achieve this by considering programs found guilty of writing unstructured programs.
    [Show full text]
  • Programming Pearls: a Literate Program
    by jot1 Be~~tley programming with Special Guest Oysters Don Knuth and Doug McIlroy pearls A LITERATE PROGRAM Last month‘s column introduced Don Knuth’s style of frequency; or there might not even be as many as k “Literate Programming” and his WEB system for building words. Let’s be more precise: The most common programs that are works of literature. This column pre- words are to be printed in order of decreasing fre- sents a literate program by Knuth (its origins are sketched quency, with words of equal frequency listed in al- in last month‘s column) and, as befits literature, a review. phabetic order. Printing should stop after k words So without further ado, here is Knuth’s program, have been output, if more than k words are present. retypeset in Communications style. -Jon Bentley 2. The input file is assumed to contain the given Common Words Section text. If it begins with a positive decimal number Introduction.. , . , , . , . , . , , . 1 (preceded by optional blanks), that number will be Strategic considerations . , a the value of k; otherwise we shall assume that Basic input routines . , , . 9 k = 100. Answers will be sent to the output file. Dictionary lookup . , . , . , .17 define default-k = 100 (use this value if k isn’t The frequency counts . .32 otherwise specified) Sortingatrie . ...36 Theendgame................................41 3. Besides solving the given problem, this program is Index . ...42 supposed to be an example of the WEB system, for people who know some Pascal but who have never 1. Introduction. The purpose of this program is to seen WEB before.
    [Show full text]
  • Teaching Foundations of Computation and Deduction Through Literate Functional Programming and Type Theory Formalization
    Teaching Foundations of Computation and Deduction Through Literate Functional Programming and Type Theory Formalization Gérard Huet Inria Paris Laboratory, France Abstract We describe experiments in teaching fundamental informatics notions around mathematical struc- tures for formal concepts, and effective algorithms to manipulate them. The major themes of lambda-calculus and type theory served as guides for the effective implementation of functional programming languages and higher-order proof assistants, appropriate for reflecting the theor- etical material into effective tools to represent constructively the concepts and formally certify the proofs of their properties. Progressively, a literate programming and proving style replaced informal mathematics in the presentation of the material as executable course notes. The talk will evoke the various stages of (in)completion of the corresponding set of notes along the years (see [1, 2, 14, 5, 6, 10, 12, 16, 7, 15, 17, 9, 3, 13, 11, 4, 8]), and tell how their elaboration proved to be essential to the discovery of fundamental results. 1998 ACM Subject Classification F.3.1 Specifying and Verifying Programs, F.4.1 Mathematical Logic and G.4 Mathematical Software Keywords and phrases Foundations, Computation, Deduction, Programming, Proofs Digital Object Identifier 10.4230/LIPIcs.FSCD.2016.3 Category Invited Talk References 1 Gérard Huet. Initiation à la logique mathématique. Notes de cours du DEA d’Informatique, Université Paris-Sud, 1977. 2 Gérard Huet. Démonstration automatique en logique de premier ordre et programmation en clauses de horn. Notes de cours du DEA d’Informatique, Université Paris-Sud, 1978. 3 Gérard Huet. Confluent reductions: Abstract properties and applications to term rewriting systems.
    [Show full text]
  • Typeset MMIX Programs with TEX Udo Wermuth Abstract a TEX Macro
    TUGboat, Volume 35 (2014), No. 3 297 Typeset MMIX programs with TEX Example: In section 9 the lines \See also sec- tion 10." and \This code is used in section 24." are given. Udo Wermuth No such line appears in section 10 as it only ex- tends the replacement code of section 9. (Note that Abstract section 10 has in its headline the number 9.) In section 24 the reference to section 9 stands for all of ATEX macro package is presented as a literate pro- the eight code lines stated in sections 9 and 10. gram. It can be included in programs written in the If a section is not used in any other section then languages MMIX or MMIXAL without affecting the it is a root and during the extraction of the code a assembler. Such an instrumented file can be pro- file is created that has the name of the root. This file cessed by TEX to get nicely formatted output. Only collects all the code in the sequence of the referenced a new first line and a new last line must be entered. sections from the code part. The collection process And for each end-of-line comment a flag is set to for all root sections is called tangle. A second pro- indicate that the comment is written in TEX. cess is called weave. It outputs the documentation and the code parts as a TEX document. How to read the following program Example: The following program has only one The text that starts in the next chapter is a literate root that is defined in section 4 with the headline program [2, 1] written in a style similar to noweb [7].
    [Show full text]
  • Miktex Manual Revision 2.0 (Miktex 2.0) December 2000
    MiKTEX Manual Revision 2.0 (MiKTEX 2.0) December 2000 Christian Schenk <[email protected]> Copyright c 2000 Christian Schenk Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the con- ditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another lan- guage, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. Chapter 1: What is MiKTEX? 1 1 What is MiKTEX? 1.1 MiKTEX Features MiKTEX is a TEX distribution for Windows (95/98/NT/2000). Its main features include: • Native Windows implementation with support for long file names. • On-the-fly generation of missing fonts. • TDS (TEX directory structure) compliant. • Open Source. • Advanced TEX compiler features: -TEX can insert source file information (aka source specials) into the DVI file. This feature improves Editor/Previewer interaction. -TEX is able to read compressed (gzipped) input files. - The input encoding can be changed via TCX tables. • Previewer features: - Supports graphics (PostScript, BMP, WMF, TPIC, . .) - Supports colored text (through color specials) - Supports PostScript fonts - Supports TrueType fonts - Understands HyperTEX(html:) specials - Understands source (src:) specials - Customizable magnifying glasses • MiKTEX is network friendly: - integrates into a heterogeneous TEX environment - supports UNC file names - supports multiple TEXMF directory trees - uses a file name database for efficient file access - Setup Wizard can be run unattended The MiKTEX distribution consists of the following components: • TEX: The traditional TEX compiler.
    [Show full text]
  • Math Symbol Tables
    APPENDIX A I Math symbol tables A.I Hebrew letters Type: Print: Type: Print: \aleph ~ \beth :J \daleth l \gimel J All symbols but \aleph need the amssymb package. 346 Appendix A A.2 Greek characters Type: Print: Type: Print: Type: Print: \alpha a \beta f3 \gamma 'Y \digamma F \delta b \epsilon E \varepsilon E \zeta ( \eta 'f/ \theta () \vartheta {) \iota ~ \kappa ,.. \varkappa x \lambda ,\ \mu /-l \nu v \xi ~ \pi 7r \varpi tv \rho p \varrho (} \sigma (J \varsigma <; \tau T \upsilon v \phi ¢ \varphi 'P \chi X \psi 'ljJ \omega w \digamma and \ varkappa require the amssymb package. Type: Print: Type: Print: \Gamma r \varGamma r \Delta L\ \varDelta L.\ \Theta e \varTheta e \Lambda A \varLambda A \Xi ~ \varXi ~ \Pi II \varPi II \Sigma 2: \varSigma E \Upsilon T \varUpsilon Y \Phi <I> \varPhi t[> \Psi III \varPsi 1ft \Omega n \varOmega D All symbols whose name begins with var need the amsmath package. Math symbol tables 347 A.3 Y1EX binary relations Type: Print: Type: Print: \in E \ni \leq < \geq >'" \11 « \gg \prec \succ \preceq \succeq \sim \cong \simeq \approx \equiv \doteq \subset c \supset \subseteq c \supseteq \sqsubseteq c \sqsupseteq \smile \frown \perp 1- \models F \mid I \parallel II \vdash f- \dashv -1 \propto <X \asymp \bowtie [Xl \sqsubset \sqsupset \Join The latter three symbols need the latexsym package. 348 Appendix A A.4 AMS binary relations Type: Print: Type: Print: \leqs1ant :::::; \geqs1ant ): \eqs1ant1ess :< \eqs1antgtr :::> \lesssim < \gtrsim > '" '" < > \lessapprox ~ \gtrapprox \approxeq ~ \lessdot <:: \gtrdot Y \111 «< \ggg »> \lessgtr S \gtr1ess Z < >- \lesseqgtr > \gtreq1ess < > < \gtreqq1ess \lesseqqgtr > < ~ \doteqdot --;- \eqcirc = .2..
    [Show full text]
  • Programming Technologies for the Development of Web-Based Platform for Digital Psychological Tools
    (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 9, No. 8, 2018 Programming Technologies for the Development of Web-Based Platform for Digital Psychological Tools Evgeny Nikulchev1, Dmitry Ilin2 Pavel Kolyasnikov3 Ilya Zakharov5, Sergey Malykh6 4 MIREA – Russian Technological Vladimir Belov Psychological Institute of Russian University & Russian Academy Russian Academy Science Academy of Education Science, Moscow, Russia Moscow, Russia Moscow, Russia Abstract—The choice of the tools and programming In addition, large accumulated data sets can become the technologies for information systems creation is relevant. For basis for machine learning mechanisms and other approaches every projected system, it is necessary to define a number of using artificial intelligence. Accumulation of data from criteria for development environment, used libraries and population studies into a single system can allow a technologies. The paper describes the choice of technological breakthrough in the development of systems for automated solutions using the example of the developed web-based platform intellectual analysis of behavior data. of the Russian Academy of Education. This platform is used to provide information support for the activities of psychologists in The issue of selecting methodological tools for online and their research (including population and longitudinal offline research includes several items. researches). There are following system features: large scale and significant amount of developing time that needs implementation First, any selection presupposes the existence of generally and ensuring the guaranteed computing reliability of a wide well-defined criteria, on the basis of which a decision can be range of digital tools used in psychological research; ensuring made to include or not to include techniques in the final functioning in different environments when conducting mass toolkit.
    [Show full text]
  • The LATEX Web Companion
    The LATEX Web Companion Integrating TEX, HTML, and XML Michel Goossens CERN Geneva, Switzerland Sebastian Rahtz Elsevier Science Ltd., Oxford, United Kingdom with Eitan M. Gurari, Ross Moore, and Robert S. Sutor Ä yv ADDISON—WESLEY Boston • San Francisco • New York • Toronto • Montreal London • Munich • Paris • Madrid Capetown • Sydney • Tokyo • Singapore • Mexico City Contents List of Figures xi List of Tables xv Preface xvii 1 The Web, its documents, and D-ItX 1 1.1 The Web, a window an die Internet 3 1.1.1 The Hypertext Transport Protocol 4 1.1.2 Universal Resource Locators and Identifiers 5 1.1.3 The Hypertext Markup Language 6 1.2 BTEX in die Web environment 11 1.2.1 Overview of document formats and strategies 12 1.2.2 Staying with DVI 14 1.2.3 PDF for typographic quality 15 1.2.4 Down-translation to HTML 16 1.2.5 Java and browser plug-ins 20 1.2.6 Other L4TEX-related approaches to the Web 21 1.3 Is there an optimal approach? 23 1.4 Conclusion 24 2 Portable Document Format 25 2.1 What is PDF? 26 2.2 Generating PDF from TEX 27 2.2.1 Creating and manipulating PDF 28 vi Contents 2.2.2 Setting up fonts 29 2.2.3 Adding value to your PDF 33 2.3 Rich PDF with I4TEX: The hyperref package 35 2.3.1 Implicit behavior of hyperref 36 2.3.2 Configuring hyperref 38 2.3.3 Additional user macros for hyperlinks 45 2.3.4 Acrobat-specific commands 47 2.3.5 Special support for other packages 49 2.3.6 Creating PDF and HTML forms 50 2.3.7 Designing PDF documents for the screen 59 2.3.8 Catalog of package options 62 2.4 Generating
    [Show full text]
  • Mmixware a RISC Computer for the Third Millennium Reihe: Lecture Notes in Computer Science
    springer.com Donald E. Knuth MMIXware A RISC Computer for the Third Millennium Reihe: Lecture Notes in Computer Science MMIX is a RISC computer designed by Don Knuth to illustrate machine-level aspects of programming. In the author's book series "The Art of Computer Programming", MMIX replaces the 1960s-style machine MIX. A particular goal in the design of MMIX was to keep its machine language simple, elegant, and easy to learn. At the same time, all of the complexities needed to achieve high performance in practice are taken into account. This book constitutes a collection of programs written in CWEB that make MMIX a virtual reality. Among other utilities, an assembler converting MMIX symbolic files to MMIX objects and two simulators executing the programs in given object files are provided. The latest version of all programs can be downloaded from MMIX's home page. The book provides a complete documentation of the MMIX computer and its assembly language. It also presents mini-indexes, which make the programs much easier to understand. A corrected reprint of the book has been published in 1999, X, 550 p. August 2014, replacing the version of 1999. Gedrucktes Buch Softcover 79,99 € | £69.99 | $99.99 [1]85,59 € (D) | 87,99 € (A) | CHF 94,50 eBook 67,40 € | £55.99 | $79.99 [2]67,40 € (D) | 67,40 € (A) | CHF 75,50 Erhältlich bei Ihrer Bibliothek oder springer.com/shop MyCopy [3] Printed eBook for just € | $ 24.99 springer.com/mycopy Erhältlich bei Ihrem Buchhändler oder – Springer Nature Customer Service Center GmbH, Haberstrasse 7, 69126 Heidelberg, Germany / Call: + 49 (0) 6221-345-4301 / Fax: +49 (0)6221-345-4229 / Email: [email protected] / Web: springer.com [1] € (D) sind gebundene Ladenpreise in Deutschland und enthalten 7% MwSt; € (A) sind gebundene Ladenpreise in Österreich und enthalten 10% MwSt.
    [Show full text]
  • Statistical Software
    Statistical Software A. Grant Schissler1;2;∗ Hung Nguyen1;3 Tin Nguyen1;3 Juli Petereit1;4 Vincent Gardeux5 Keywords: statistical software, open source, Big Data, visualization, parallel computing, machine learning, Bayesian modeling Abstract Abstract: This article discusses selected statistical software, aiming to help readers find the right tool for their needs. We categorize software into three classes: Statisti- cal Packages, Analysis Packages with Statistical Libraries, and General Programming Languages with Statistical Libraries. Statistical and analysis packages often provide interactive, easy-to-use workflows while general programming languages are built for speed and optimization. We emphasize each software's defining characteristics and discuss trends in popularity. The concluding sections muse on the future of statistical software including the impact of Big Data and the advantages of open-source languages. This article discusses selected statistical software, aiming to help readers find the right tool for their needs (not provide an exhaustive list). Also, we acknowledge our experiences bias the discussion towards software employed in scholarly work. Throughout, we emphasize the software's capacity to analyze large, complex data sets (\Big Data"). The concluding sections muse on the future of statistical software. To aid in the discussion, we classify software into three groups: (1) Statistical Packages, (2) Analysis Packages with Statistical Libraries, and (3) General Programming Languages with Statistical Libraries. This structure
    [Show full text]
  • Exploratory Data Science Using a Literate Programming Tool Mary Beth Kery1 Marissa Radensky2 Mahima Arya1 Bonnie E
    The Story in the Notebook: Exploratory Data Science using a Literate Programming Tool Mary Beth Kery1 Marissa Radensky2 Mahima Arya1 Bonnie E. John3 Brad A. Myers1 1Human-Computer Interaction Institute 2Amherst College 3Bloomberg L. P. Carnegie Mellon University Amherst, MA New York City, NY Pittsburgh, PA [email protected] [email protected] mkery, mahimaa, bam @cs.cmu.edu ABSTRACT engineers, many of whom never receive formal training in Literate programming tools are used by millions of software engineering [30]. programmers today, and are intended to facilitate presenting data analyses in the form of a narrative. We interviewed 21 As even more technical novices engage with code and data data scientists to study coding behaviors in a literate manipulations, it is key to have end-user programming programming environment and how data scientists kept track tools that address barriers to doing effective data science. For of variants they explored. For participants who tried to keep instance, programming with data often requires heavy a detailed history of their experimentation, both informal and exploration with different ways to manipulate the data formal versioning attempts led to problems, such as reduced [14,23]. Currently even experts struggle to keep track of the notebook readability. During iteration, participants actively experimentation they do, leading to lost work, confusion curated their notebooks into narratives, although primarily over how a result was achieved, and difficulties effectively through cell structure rather than markdown explanations. ideating [11]. Literate programming has recently arisen as a Next, we surveyed 45 data scientists and asked them to promising direction to address some of these problems [17].
    [Show full text]