DOCSLIB.ORG

Statistics and Econometrics Using Xlispstat

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Statistics and Econometrics Using Xlispstat STATISTICS AND ECONOMETRICS USING XLISPSTAT John E. Floyd University of Toronto May 27, 2009 Contents 1 Introduction 1 2 Working with XLispStat 5 2.1 Using XLispStat as a Calculator . 6 2.2 De¯ning Objects and Working with Lists . 8 2.3 Writing Lisp Functions . 16 2.4 Working with Matrices . 19 2.5 Reading and Writing Data Files . 26 2.6 Transforming Data . 32 2.7 Error Messages . 46 3 Descriptive Statistics 49 4 Hypothesis Tests 63 4.1 Probability Densities and Quantiles . 63 4.2 Plotting Probability Distributions . 68 4.3 Generating Random Data . 71 4.4 Tests of the Mean and Standard Deviation . 73 4.5 Tests of the Di®erence Between Two Means . 75 4.6 Tests of Goodness of Fit . 80 5 Linear Regression Analysis 85 5.1 Using Matrix Calculations . 86 5.2 Using the Regression-Model Function . 90 5.3 Heteroskedasticity . 93 5.4 Time Series: Autocorrelated Residuals . 95 5.5 Multicollinearity . 102 5.6 Some Improved Linear Regression Functions . 107 5.6.1 A Basic OLS-Regression Function . 108 5.6.2 Regressions on Cross-Sectional Data . 113 i ii CONTENTS 5.6.3 Time-Series Regressions . 114 5.6.4 Adjusting the Lengths of Time-Series and Setting up Lagged Values . 118 6 Regression Analysis of Panel Data 119 6.1 Di®erences Estimation . 122 6.2 Entity Demeaned Fixed E®ects Regression . 125 6.3 Using Fixed-E®ects Dummy Variables . 126 6.4 Reorganisation of Panel Data Sets . 137 7 Instrumental Variables Regression 141 7.1 Two-Stage Least Squares . 142 7.2 Estimation Using Ordinary Least Squares . 144 7.3 First Stage TSLS Estimation . 146 7.4 Second Stage TSLS Estimation . 147 7.5 An Application to Panel Data . 152 8 Probit, Logit and Nonlinear Regression 159 8.1 The Linear Probability Model . 159 8.2 Probit and Logit Models . 161 8.3 Nonlinear Least Squares Estimation . 163 8.4 Maximum Likelihood Estimation . 167 9 Spurious Regression and Cointegration 181 9.1 Checking for Stationarity . 181 9.1.1 Dickey-Fuller Tests . 182 9.1.2 Phillips-Perron Tests . 187 9.1.3 The Problem of Low Power . 192 9.2 Testing for Cointegration . 192 9.2.1 Tests of Regression Residuals for Cointegration . 193 9.2.2 Johansen Cointegration Tests . 199 10 Further Topics in Regression Analysis 215 10.1 Joint Hypotheses Tests . 215 10.2 Non-Nested Hypotheses Tests . 237 10.2.1 F-Tests . 237 10.2.2 J-Tests . 239 10.2.3 Complete Parameter Encompassing Tests . 242 10.3 Generalised Least Squares . 245 10.3.1 The Nature of GLS . 245 CONTENTS iii 10.3.2 Quasi-Di®erencing . 246 10.3.3 Seemingly Unrelated Regression Techniques . 263 11 Vector Autoregression Analysis 271 11.1 Standard-Form Estimation . 274 11.2 Moving Average Representation . 285 11.3 Identi¯cation . 289 11.3.1 Choleski Decompositions . 290 11.3.2 Structural Decompositions . 306 11.3.3 Blanchard-Quah Decompositions . 336 11.4 Bootstrapping Con¯dence Intervals . 343 12 Forecasting 361 12.1 Trend Projections . 361 12.2 ARIMA Forecasts . 365 12.3 OLS Forecasts . 386 12.4 Near-VAR Forecasts . 391 iv CONTENTS Chapter 1 Introduction The purpose of this manual is to show the reader how to use the free program XLispStat to do basic statistical and econometric analysis. It has evolved into somewhat of a tutorial for those interested in learning basic statistics and econometrics. A small amount of Lisp programming, which a diligent reader will learn how to do in a few hours, will be required. For students and other beginners, this will provide a good background for subsequently learning how to cope with commercial statistical and computer programs that one often eventually needs to use. For day-to-day work, and even to learn the basics, the reader can work through the small manual I have written and the exercises and examples there referred to, consulting this big manual for details and deeper and more sophisticated issues. XLispStat is a wonderful rich platform for statistical computing written by Luke Tierney at the University of Minnesota. Its depth far exceeds what is utilised here. Those who, having worked through this manual, want to really get serious about XLispStat are advised to get Luke Tierney's book.1 I believe that one could program in XLispStat routines equivalent in purpose and result to virtually anything commonly found in commercial econometrics software. Functions are already present for non-linear least squares, maximisation and maximum likelihood estimation and Bayesian computations along with object-oriented methods for data handling and graphics. So most of the work required would involve adapting these existing resources to the needs at hand. And one of the best ways to develop an understanding of statistical and econometric techniques is to program the 1Luke Tierney, Lisp-Stat: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, 1990. 1 2 CHAPTER 1. INTRODUCTION routines oneself. Indeed, what follows would not have been written but for my inclination to try to ¯nd out what is really happening when my favourite commercial program, RATS, is performing its calculations. Nearly all of the actual econometrics functions used here, as well as many data handling routines, were written by me and are available from my web-site in the ¯le addfuncs.lsp. That ¯le must be loaded into the workspace after loading XLispStat before working through any material presented in this manual. Readers are free to work through and modify any of the materials in that ¯le and add new ones as desired, thereby making the program their own. An MS-Windows version of XLispStat can be obtained by following the appropriate links on my web-site www.economics.utoronto.ca/floyd. You will need the self-extracting zip ¯les wxls32zp.exe, which contains the pro- gram itself, and xlispdf.exe, which contains the data referred to in this manual and exexamp.exe which contains some exercises and example pro- grams. And the addfuncs.lsp must, of course, also be obtained along with the XLispStat ¯le maximize.lsp that will be needed for maximum likelihood estimation. A version of XLispStat for Apple computers can be obtained by searching the Web as can Linux versions for most distributions. The data used here are made available for Linux versions in the ¯les xlispdf.tar.gz and the exercise and example ¯les are in the ¯le exeamp.tar.gz. The next chapter provides a simple guide to working with and writing programs in XLispStat. Everything you would need to program all functions created here is explained. Chapter 3 sets out the procedures that enable us to describe properly the data we are working with and Chapter 4 focuses on hypothesis testing, beginning with a discussion of probability distributions. Chapter 5 introduces regression analysis, starting with a discussion of how to run regressions using matrix calculations. The XLispStat function called regression-model is explored next. This function is important be- cause it is used repeatedly in all functions I wrote that use OLS regression calculations although, used alone, its output presentation is too crude for day-to-day work. Heteroskedastic and autocorrelated residuals are then dis- cussed followed by the problem of multicollinearity. Finally, three new func- tions to perform OLS regressions on cross-sectional and time-series data are then presented, along with some additional functions to simplify the process of adjusting the lengths of time-series and setting up lagged values. Chapters 6, 7, and 8 deal respectively with panel-data analysis, instru- mental variables, and logit and probit estimation. Of these issues, only instrumental variables estimation has been used in my own research, so the other two chapters contain only very rudimentary analyses. All three chap- ters are based on the introductory textbook by Stock and Watson and data 3 sets there referred to.2 More sophisticated extensions dealing with panel- data and logit and probit analysis await my ¯nding a joint author whose main research uses these techniques. My own focus on time-series analysis accounts for the intensive exami- nation in Chapter 9 of how to test for stationarity and cope with problems of spurious regression. I ¯nd my functions dealing with these issues, partic- ularly the tests for stationarity and cointegration, more useful for my pur- poses than those in most commercial programs. Students working through this chapter, and the references to the textbooks by Enders and Hamilton, should ¯nd the e®ort helpful in understanding the basics of cointegration analysis.3 Chapter 10 deals with a number of additional topics in regression analysis that have been important in my own work. These are joint hypothesis tests, non-nested hypothesis tests and generalised least squares estimation of seemingly-unrelated regressions. An extensive treatment of vector autoregression analysis is the subject of Chapter 11, again reflecting the importance of this area in my own empirical work. Students should ¯nd these materials, and the references on which they are based, useful in developing an understanding of the basics in this area. And the functions I present can do all types of VARs, along with bootstrapped con¯dence intervals, though admittedly not as elegantly as RATS. The ¯nal chapter deals in a rudimentary way with forecasting time series. There is no pretension of competing with business software, but the func- tions provided are useful in making pseudo (in-sample) forecasts of agents' expected levels of variables from which unanticipated shocks to these vari- ables can be calculated and used in econometric analysis. The forecasting of variables beyond the period for which data are available is also briefly discussed.
Load more

Recommended publications
  • Emacs Speaks Statistics (ESS): a Multi-Platform, Multi-Package Intelligent Environment for Statistical Analysis
    Emacs Speaks Statistics (ESS): A multi-platform, multi-package intelligent environment for statistical analysis A.J. Rossini Richard M. Heiberger Rodney A. Sparapani Martin Machler¨ Kurt Hornik ∗ Date: 2003/10/22 17:34:04 Revision: 1.255 Abstract Computer programming is an important component of statistics research and data analysis. This skill is necessary for using sophisticated statistical packages as well as for writing custom software for data analysis. Emacs Speaks Statistics (ESS) provides an intelligent and consistent interface between the user and software. ESS interfaces with SAS, S-PLUS, R, and other statistics packages under the Unix, Microsoft Windows, and Apple Mac operating systems. ESS extends the Emacs text editor and uses its many features to streamline the creation and use of statistical software. ESS understands the syntax for each data analysis language it works with and provides consistent display and editing features across packages. ESS assists in the interactive or batch execution by the statistics packages of statements written in their languages. Some statistics packages can be run as a subprocess of Emacs, allowing the user to work directly from the editor and thereby retain a consistent and constant look- and-feel. We discuss how ESS works and how it increases statistical programming efficiency. Keywords: Data Analysis, Programming, S, SAS, S-PLUS, R, XLISPSTAT,STATA, BUGS, Open Source Software, Cross-platform User Interface. ∗A.J. Rossini is Research Assistant Professor in the Department of Biostatistics, University of Washington and Joint Assis- tant Member at the Fred Hutchinson Cancer Research Center, Seattle, WA, USA mailto:[email protected]; Richard M.
    [Show full text]
  • XLISP-STAT a Statistical Environment Based on the XLISP Language (Version 2.0)
    I XLISP-STAT A Statistical Environment Based on the XLISP Language (Version 2.0) by Luke Tierney l.5i1 University of Minnesota School of Statistics Technical Report Number 528 July 1989 Contents Preface .. 3 1 Starting and Finishing 6 2 Introduction to Basics 8 2.1 Data ........ 8 2.2 The Listener and the Evaluator . 8 3 Elementary Statistical Operations 11 3.1 First Steps ......... 11 3.2 Summary Statistics and Plots 12 3.3 Two Dimensional Plots 16 3.4 Plotting Functions ..... 19 4 More on Generating and Modifying Data 20 4.1 Generating Random Data . 20 4.2 Generating Systematic Data . 20 4.3 Forming Subsets and .Deleting Cases 21 4.4 Combining Several Lists 22 4.5 Modifying Data . 22 5 Some Useful Shortcuts 24 5.1 Getting Help . 24 5.2 Listing and Undefining Variables .. 26 5.3 More on the XLISP-STAT Listener .. 26 5 .4 Loading Files . 28 5.5 Saving Your Work ..... 28 5.6 The XLISP-STAT Editor 29 5.7 Reading Data Files .. 29 5.8 User Initialization File 29 6 More Elaborate Plots 30 6.1 Spinning Plots . ..... 30 6.2 Scatterplot Matrices • It ••••• 32 6.3 Interacting with Individual Plots 35 6.4 Linked Plots ....... 35 6.5 Modifying a Scatter Plot . 36 6.6 Dynamic Simulations . 39 7 Regression 42 8 Defining Your Own Functions and Methods 47 8.1 Defining Functions .... 47 8.2 Anonymous Functions .. 48 8.3 Some Dynamic Simulations 48 8.4 Defining Methods . 51 8.5 Plot Methods . 52 9 Matrices and Arrays 53 10 Nonlinear Regression 54 1 11 One Way ANOVA 57 12 Maximization and Maximum Likeliho~d Estimation 58 13 Approximate Bayesian Computations 61 A XLISP-STAT on UNIX Systems 68 A.1 XLISP-STAT Under the X11 Window System.
    [Show full text]
  • Spatial Tools for Econometric and Exploratory Analysis
    Spatial Tools for Econometric and Exploratory Analysis Michael F. Goodchild University of California, Santa Barbara Luc Anselin University of Illinois at Urbana-Champaign http://csiss.org Outline ¾A Quick Tour of a GIS ¾Spatial Data Analysis ¾CSISS Tools Spatial Data Analysis Principles: 1. Integration ¾Linking data through common location the layer cake ¾Linking processes across disciplines spatially explicit processes e.g. economic and social processes interact at common locations 2. Spatial analysis ¾Social data collected in cross- section longitudinal data are difficult to construct ¾Cross-sectional perspectives are rich in context can never confirm process though they can perhaps falsify useful source of hypotheses, insights 3. Spatially explicit theory ¾Theory that is not invariant under relocation ¾Spatial concepts (location, distance, adjacency) appear explicitly ¾Can spatial concepts ever explain, or are they always surrogates for something else? 4. Place-based analysis ¾Nomothetic - search for general principles ¾Idiographic - description of unique properties of places ¾An old debate in Geography The Earth's surface ¾Uncontrolled variance ¾There is no average place ¾Results depend explicitly on bounds ¾Places as samples ¾Consider the model: y = a + bx Tract Pop Location Shape 1 3786 x,y 2 2966 x,y 3 5001 x,y 4 4983 x,y 5 4130 x,y 6 3229 x,y 7 4086 x,y 8 3979 x,y Iij = EiAjf (dij) / ΣkAkf (dik) Aj d Ei ij Types of Spatial Data Analysis ¾ Exploratory Spatial Data Analysis • exploring the structure of spatial data • determining
    [Show full text]
  • Paquetes Estadísticos Con Licencia Libre (I) Free Statistical
    2013, Vol. 18 No 2, pp. 12-33 http://www.uni oviedo.es/reunido /index.php/Rema Paquetes estadísticos con licencia libre (I) Free statistical software (I) Carlos Carleos Artime y Norberto Corral Blanco Departamento de Estadística e I. O. y D.M.. Universidad de Oviedo RESUMEN El presente artículo, el primero de una serie de trabajos, presenta un panorama de los principales paquetes estadísticos disponibles en la actualidad con licencia libre, entendiendo este concepto a partir de los grandes proyectos informáticos así autodefinidos que han cristalizado en los últimos treinta años. En esta primera entrega se presta atención fundamentalmente al programa R. Palabras clave : paquetes estadísticos, software libre, R. ABSTRACT This article, the first in a series of works, presents an overview of the major statistical packages available today with free license, understanding this concept from the large computer and self-defined projects that have crystallized in the last thirty years. In this first paper we pay attention mainly to the program R. Keywords : statistical software, free software, R. Contacto: Norberto Corral Blanco Email: [email protected] . 12 Revista Electrónica de Metodología Aplicada (2013), Vol. 18 No 2, pp. 12-33 1.- Introducción La palabra “libre” es usada, y no siempre de manera adecuada, en múltiples campos de conocimiento así que la informática no iba a ser una excepción. La palabra es especialmente problemática porque su traducción al inglés, “free”, es aun más polisémica, e incluye el concepto “gratis”. En 1985 se fundó la Free Software Foundation (FSF) para defender una definición rigurosa del sófguar libre. La propia palabra “sófguar” (del inglés “software”, por oposición a “hardware”, quincalla, aplicado al soporte físico de la informática) es en sí misma problemática.
    [Show full text]
  • High Dimensional Data Analysis Via the SIR/PHD Approach
    High dimensional data analysis via the SIR/PHD approach April 6, 2000 Preface Dimensionality is an issue that can arise in every scientific field. Generally speaking, the difficulty lies on how to visualize a high dimensional function or data set. This is an area which has become increasingly more important due to the advent of computer and graphics technology. People often ask : “How do they look?”, “What structures are there?”, “What model should be used?” Aside from the differences that underly the various scientific con- texts, such kind of questions do have a common root in Statistics. This should be the driving force for the study of high dimensional data analysis. Sliced inverse regression(SIR) and principal Hessian direction(PHD) are two basic di- mension reduction methods. They are useful for the extraction of geometric information underlying noisy data of several dimensions - a crucial step in empirical model building which has been overlooked in the literature. In this Lecture Notes, I will review the theory of SIR/PHD and describe some ongoing research in various application areas. There are two parts. The first part is based on materials that have already appeared in the literature. The second part is just a collection of some manuscripts which are not yet published. They are included here for completeness. Needless to say, there are many other high dimensional data analysis techniques (and we may encounter some of them later on) that should deserve more detailed treatment. In this complex field, it would not be wise to anticipate the existence of any single tool that can outperform all others in every practical situation.
    [Show full text]
  • Pl a N a Ss E Ss De V Elo P E X P Lo R E R E S E a Rc H B U Ild C O N Ta C T P
    PLAN ACTION VERBS Use consistent verb tense (generally past tense). Start phrases with descriptive action verbs. Supply quantitative data whenever possible. Adapt terminology to include key words. Incorporate action verbs with keywords and current “hot” topics, programs, tools, testing terms, and instrumentation to develop concise, yet highly descriptive phrases. Remember that résumés are scanned for such words, so do everything possible to incorporate important phraseology and current keywords into your résumé. ASSESS From What Color is Your Parachute, Richard Bolles, 2005 achieved delivered founded motivated resolved adapted detailed gathered navigated responded analyzed detected generated operated restored arbitrated determined guided perceived retrieved DEVELOP ascertained devised hypothesized persuaded reviewed assessed diagnosed identified piloted risked attained discovered illustrated predicted scheduled audited displayed implemented problem-solved selected built dissected improvised proofread served collected distributed influenced projected shaped conceptualized diverted initiated promoted summarized EXPLORE compiled eliminated innovated publicized supplied computed enforced inspired purchased surveyed conducted established installed reasoned synthesized conserved evaluated integrated recommended taught consolidated examined investigated referred tested constructed expanded maintained rehabilitated transcribed consulted experimented mediated rendered trouble-shot controlled expressed mentored reported tutored RESEARCH counseled
    [Show full text]
  • Emacs Speaks Statistics: One Interface — Many Programs
    DSC 2001 Proceedings of the 2nd International Workshop on Distributed Statistical Computing March 15-17, Vienna, Austria http://www.ci.tuwien.ac.at/Conferences/DSC-2001 K. Hornik & F. Leisch (eds.) ISSN 1609-395X Emacs Speaks Statistics: One Interface | Many Programs Richard M. Heiberger∗ Abstract Emacs Speaks Statistics (ESS) is a user interface for developing statisti- cal applications and performing data analysis using any of several powerful statistical programming languages: currently, S, S-Plus, R, SAS, XLispStat, Stata. ESS provides a common editing environment, tailored to the grammar of each statistical language, and similar execution environments, in which the statistical program is run under the control of Emacs. The paper discusses the capabilities and advantages of using ESS as the primary interface for sta- tistical programming, and the design issues addressed in providing a common interface for differently structured languages on different computational plat- forms. 1 Introduction Complex statistical analyses require multiple computational tools, each targeted for a particular statistical analysis. The tools are usually designed with the assump- tion of a specific style of interaction with the software. When tools with conflicting styles are chosen, the interference can partially negate the efficiency gain of using appropriate tools. ESS (Emacs Speaks Statistics) [7] provides a single interface to most of the tools that a statistician is likely to use and therefore mitigates many of the incompatibility issues. ESS provides a
    [Show full text]
  • Statistical Inference for Apparent Populations Author(S): Richard A
    Statistical Inference for Apparent Populations Author(s): Richard A. Berk, Bruce Western and Robert E. Weiss Source: Sociological Methodology, Vol. 25 (1995), pp. 421-458 Published by: American Sociological Association Stable URL: https://www.jstor.org/stable/271073 Accessed: 24-08-2019 15:58 UTC JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms American Sociological Association is collaborating with JSTOR to digitize, preserve and extend access to Sociological Methodology This content downloaded from 160.39.33.163 on Sat, 24 Aug 2019 15:58:49 UTC All use subject to https://about.jstor.org/terms STATISTICAL INFERENCE FOR APPARENT POPULATIONS Richard A. Berk* Bruce Westernt Robert E. Weiss* In this paper we consider statistical inference for datasets that are not replicable. We call these datasets, which are common in sociology, apparent populations. We review how such data are usually analyzed by sociologists and then suggest that perhaps a Bayesian approach has merit as an alternative. We illustrate our views with an empirical example. 1. INTRODUCTION It is common in sociological publications to find statistical inference applied to datasets that are not samples in the usual sense. For the substantive issues being addressed, the data on hand are all the data there are.
    [Show full text]
  • Quantian: a Single-System Image Scientific Cluster Computing
    Quantian: A single-system image scientific cluster computing environment Dirk Eddelbuettel, Ph.D. B of A, and Debian [email protected] Presentation at the Extreme Linux SIG at USENIX 2004 in Boston, July 2, 2004 Quantian: A single-system image scientific cluster computing environment – p. 1 Introduction Quantian is a directly bootable and self-configuring Linux sytem that runs from a compressed dvd image. Quantian offers zero-configuration cluster computing using openMosix. Quantian can boot ’thin clients’ directly via PXE in an ’openmosixterminalserver’ setting. Quantian contains around 1gb of additional ’quantitative’ software: scientific, numerical, statistical, engineering, ... Quantian also contains tools of general usefulness such as editors, programming languages, a very complete latex suite, two ’office’ suites, networking tools and multimedia apps. Quantian: A single-system image scientific cluster computing environment – p. 2 Family tree overview Quantian is based on clusterKnoppix, which extends Knoppix with an openMosix-enabled kernel and applications (chpox, gomd, tyd, ....), kernel modules and security patches. ClusterKnoppix extends Knoppix, an impressive ’linux on a cdrom’ system which puts 2.3gb of software onto a cdrom along with the very best auto-detection and configuration. Knoppix is based on Debian, a Linux distribution containing over 6000 source packages available for 10 architectures (such as i386, alpha, ia64, amd64, sparc or s390) produced by hundreds of individuals from across the globe. Quantian: A single-system image scientific cluster computing environment – p. 3 Family tree: Debian ’Linux the Linux way’: made by volunteers (some now with full-time backing) from across the globe. Focus on very high technical standards with rigorous policy and reference documents.
    [Show full text]
  • Quantian As an Environment for Distributed Statistical Computing
    Quantian for distributed statistical computing Dirk Eddelbuettel Background Quantian as an environment for Introduction Timeline distributed statistical computing Quantian Motivation Content Distributed Computing Dirk Eddelbuettel Overview Preparation Beowulf Debian Project openMosix [email protected] R Examples Snow SnowFT papply DSC 2005 – Directions in Statistical Computing 2005 Others University of Washington, Seattle, August 13-14, 2005 Summary Quantian for distributed statistical computing Outline Dirk Eddelbuettel Background Introduction Timeline 1 Background Quantian Motivation Content 2 Quantian Distributed Computing Overview Preparation 3 Distributed Computing Beowulf openMosix R Examples Snow 4 R Examples SnowFT papply Others Summary 5 Summary Quantian for distributed statistical computing What is Quantian? Dirk A live-dvd for numbers geeks Eddelbuettel Background Introduction Timeline • Quantian is a directly bootable and self-configuring Quantian Motivation Linux sytem that runs from a compressed dvd image. Content • Quantian offers zero-configuration cluster computing Distributed Computing using openMosix, including ’openMosix Terminalserver’ Overview Preparation via PXE. Beowulf openMosix • Quantian contains over 6gb of software, including an R Examples Snow additional 4gb of ’quantitative’ software: scientific, SnowFT papply numerical, statistical, engineering, ... Others • Summary Quantian also contains editors, programming languages, complete latex support, two ’office’ suites, networking tools and more. Quantian for distributed
    [Show full text]
  • Literate Statistical Practice
    New URL: http://www.R-project.org/conferences/DSC-2001/ DSC 2001 Proceedings of the 2nd International Workshop on Distributed Statistical Computing March 15-17, Vienna, Austria http://www.ci.tuwien.ac.at/Conferences/DSC-2001 K. Hornik & F. Leisch (eds.) ISSN 1609-395X Literate Statistical Practice A.J. Rossini∗ Abstract Literate Statistical Practice (LSP) is an method for statistical practice which suggests that documentation and specification occur at the same time as statistical coding. It applies literate programming Knuth (1992) to the practice of statistics. We discuss 2 different approaches for LSP, one currently implemented using Emacs with Noweb and Emacs Speaks Statistics (ESS), and the other developed based on eXtensible Markup Language (XML) tools. The interference needed to change an individual's habits comes at a high cost, and good tools are critical for encouraging people to switch to a more literate style. We discuss why LSP can help, and suggest how ESS can ease the burden placed on the analyst. Keywords: Data Analysis, Programming Tools, User Interfaces, Literate Pro- gramming, Reproducible Research. 1 Introduction In statistics, the ability to document both programming language coding as well as mathematical thought is critical to understandable, explainable, and reproducible data analysis. We will refer to the activities involved in statistical research and data analysis as statistical practice. These activities often involve computing, ranging from finding reference and background material to programming and computation. Literate Statistical Practice is a programming methodology, derived from Lit- erate Programming, which encourages the construction of documentation for data management and statistical analysis as the code for it is produced.
    [Show full text]
  • Computer Intensive Statistics STAT:7400
    Computer Intensive Statistics STAT:7400 Luke Tierney Spring 2019 Introduction Syllabus and Background Basics • Review the course syllabus http://www.stat.uiowa.edu/˜luke/classes/STAT7400/syllabus.pdf • Fill out info sheets. – name – field – statistics background – computing background Homework • some problems will cover ideas not covered in class • working together is OK • try to work on your own • write-up must be your own • do not use solutions from previous years • submission by GitHub at http://github.uiowa.edu or by Icon at http://icon.uiowa.edu/. 1 Computer Intensive Statistics STAT:7400, Spring 2019 Tierney Project • Find a topic you are interested in. • Written report plus possibly some form of presentation. Ask Questions • Ask questions if you are confused or think a point needs more discussion. • Questions can lead to interesting discussions. 2 Computer Intensive Statistics STAT:7400, Spring 2019 Tierney Computational Tools Computers and Operating Systems • We will use software available on the Linux workstations in the Mathe- matical Sciences labs (Schaeffer 346 in particular). • Most things we will do can be done remotely by using ssh to log into one of the machines in Schaeffer 346 using ssh. These machines are l-lnx2xy.stat.uiowa.edu with xy = 00, 01, 02, . , 19. • You can also access the CLAS Linux systems using a browser at http://fastx.divms.uiowa.edu/ – This connects you to one of several servers. – It is OK to run small jobs on these servers. – For larger jobs you should log into one of the lab machines. • Most of the software we will use is available free for installing on any Linux, Mac OS X, or Windows computer.
    [Show full text]