Data Analysis in R

Total Page：16

File Type：pdf, Size：1020Kb

Data Analysis in R Course at a Glance This course will provide an introduction to reproducible data analysis with R (see Syllabus). Instructor Gabriel Baud-Bovy ([email protected] ) Credits: 5 Synopsis This course aims at giving to the student a methodology to analyze experimental results, from how to organize data to the writing of a report. It includes: an introduction to R an introduction to reproducible research with R examples of statistical analysis with R During this course, the student will have to analyze his own data and is expected to read before each course the material that will be made available on this page. The final grade will consist in the evaluation of a report demonstrating familiarity with the concepts and methods presented in the course. As an editor, the instructor will use Notepad++ (together with NppToR) on a Windows Machines but the student might use other ones (e.g., R studio, EMACS+ESS, Lyx, TexWork). The course will use Mardown as typesetting language. For those desirous to work with Latex and/or generate pdf, you will need to install also MikeTex. Syllabus Total of 15 hours Class 1: Case study, Reproducible research Class 2: R fundamental Class 3: Exploratory data analysis and graphical methods in R Class 4: Basic statistics Class 5: To be determined There will be a final examination decided by the instructor. Prerequisites The course assumes some familiarity with programming concepts and data structures (MATLAB, C/C++, Java or any other programming language). Contact the instructor if you have never programmed anything. Install R on your laptop. Bring a data set that you want to analyze. Reading List Baud-Bovy (2014) Notes on reproducible research with R R Core Development Team “An introduction to R". References Baud-Bovy (2014) Notes on reproducible research with R. Draft R Core Development Team “An introduction to R". Onlines (optional) The R Project website: http://cran.r-project.org/ (to download R, R packages knitr and R documentation) pandoc: http://johnmacfarlane.net/pandoc/index.html Notepad++: http://notepad-plus-plus.org/ NppToR: http://sourceforge.net/projects/npptor/ MikeTex: http://miktex.org/ R studio: see http://www.rstudio.com/. knitr website: http://yihui.name/knitr/ References books (optional) Yihui Xie Dynamic (2013) Documents with R and knitr. CRC Press. Christopher Gandrud (2014) Reproducible Research with R and RStudio. CRC Press Peter Dalgaard. Introductory Statistics with R. Robert Kabacoff . R in Action. Brian S. Everitt, Torsten Hothorn. A Handbook of Statistical Analyses Using R. Norman Matloff. The Art of R Programming: A Tour of Statistical Software Design . Venue Istituto Italiano di Tecnologia, Via Morego 30, Bolzaneto, Genova Course dates March/April 2014 .

Copy Link

Recommended publications

Advanced R Second Edition Chapman & Hall/CRC the R Series Series Editors John M
Advanced R Second Edition Chapman & Hall/CRC The R Series Series Editors John M. Chambers, Department of Statistics, Stanford University, California, USA Torsten Hothorn, Division of Biostatistics, University of Zurich, Switzerland Duncan Temple Lang, Department of Statistics, University of California, Davis, USA Hadley Wickham, RStudio, Boston, Massachusetts, USA Recently Published Titles Testing R Code Richard Cotton R Primer, Second Edition Claus Thorn Ekstrøm Flexible Regression and Smoothing: Using GAMLSS in R Mikis D. Stasinopoulos, Robert A. Rigby, Gillian Z. Heller, Vlasios Voudouris, and Fernanda De Bastiani The Essentials of Data Science: Knowledge Discovery Using R Graham J. Williams blogdown: Creating Websites with R Markdown Yihui Xie, Alison Presmanes Hill, Amber Thomas Handbook of Educational Measurement and Psychometrics Using R Christopher D. Desjardins, Okan Bulut Displaying Time Series, Spatial, and Space-Time Data with R, Second Edition Oscar Perpinan Lamigueiro Reproducible Finance with R Jonathan K. Regenstein, Jr. R Markdown The Definitive Guide Yihui Xie, J.J. Allaire, Garrett Grolemund Practical R for Mass Communication and Journalism Sharon Machlis Analyzing Baseball Data with R, Second Edition Max Marchi, Jim Albert, Benjamin S. Baumer Spatio-Temporal Statistics with R Christopher K. Wikle, Andrew Zammit-Mangion, and Noel Cressie Statistical Computing with R, Second Edition Maria L. Rizzo Geocomputation with R Robin Lovelace, Jakub Nowosad, Jannes Münchow Dose-Response Analysis with R Christian Ritz, Signe M. Jensen, Daniel Gerhard, Jens C. Streibig Advanced R, Second Edition Hadley Wickham For more information about this series, please visit: https://www.crcpress.com/go/ the-r-series Advanced R Second Edition Hadley Wickham CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2019 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S.
[Show full text]
R Generation [1] 25
IN DETAIL > y <- 25 > y R generation [1] 25 14 SIGNIFICANCE August 2018 The story of a statistical programming they shared an interest in what Ihaka calls “playing academic fun language that became a subcultural and games” with statistical computing languages. phenomenon. By Nick Thieme Each had questions about programming languages they wanted to answer. In particular, both Ihaka and Gentleman shared a common knowledge of the language called eyond the age of 5, very few people would profess “Scheme”, and both found the language useful in a variety to have a favourite letter. But if you have ever been of ways. Scheme, however, was unwieldy to type and lacked to a statistics or data science conference, you may desired functionality. Again, convenience brought good have seen more than a few grown adults wearing fortune. Each was familiar with another language, called “S”, Bbadges or stickers with the phrase “I love R!”. and S provided the kind of syntax they wanted. With no blend To these proud badge-wearers, R is much more than the of the two languages commercially available, Gentleman eighteenth letter of the modern English alphabet. The R suggested building something themselves. they love is a programming language that provides a robust Around that time, the University of Auckland needed environment for tabulating, analysing and visualising data, one a programming language to use in its undergraduate statistics powered by a community of millions of users collaborating courses as the school’s current tool had reached the end of its in ways large and small to make statistical computing more useful life.
[Show full text]
A History of R (In 15 Minutes… and Mostly in Pictures)
A history of R (in 15 minutes… and mostly in pictures) JULY 23, 2020 Andrew Zief!ler Lunch & Learn Department of Educational Psychology RMCC University of Minnesota LATIS Who am I and Some Caveats Andy Zie!ler • I teach statistics courses in the Department of Educational Psychology • I have been using R since 2005, when I couldn’t put Me (on the EPSY faculty board) SAS on my computer (it didn’t run natively on a Me Mac), and even if I could have gotten it to run, I (everywhere else) couldn’t afford it. Some caveats • Although I was alive during much of the era I will be talking about, I was not working as a statistician at that time (not even as an elementary student for some of it). • My knowledge is second-hand, from other people and sources. Statistical Computing in the 1970s Bell Labs In 1976, scientists from the Statistics Research Group were actively discussing how to design a language for statistical computing that allowed interactive access to routines in their FORTRAN library. John Chambers John Tukey Home to Statistics Research Group Rick Becker Jean Mc Rae Judy Schilling Doug Dunn Introducing…`S` An Interactive Language for Data Analysis and Graphics Chambers sketch of the interface made on May 5, 1976. The GE-635, a 36-bit system that ran at a 0.5MIPS, starting at $2M in 1964 dollars or leasing at $45K/month. ’S’ was introduced to Bell Labs in November, but at the time it did not actually have a name. The Impact of UNIX on ’S' Tom London Ken Thompson and Dennis Ritchie, creators of John Reiser the UNIX operating system at a PDP-11.
[Show full text]
August 12-14, Dortmund, Germany
AASC Austrian Association for Statistical Computing 2008 Program August 12-14, Dortmund, Germany useR! 2008, Dortmund, Germany 1 Contents Greetings and Miscellaneous 2 Maps 5 Social Program 8 Program 9 Tutorials 9 Schedule 10 List of Talks 14 2 useR! 2008, Dortmund, Germany Dear useRs, the following pages provide you with some useful information about useR! 2008, the R user con- ference, taking place at the Fakultät Statistik, Technische Universität Dortmund, Germany from 2008-08-12 to 2008-08-14. Pre-conference tutorials will take place on August 11. The confer- ence is organized by the Fakultät Statistik, Technische Universität Dortmund and the Austrian Association for Statistical Computing (AASC). Apart from challenging and likewise exciting scientiﬁc contributions we hope to offer you an attractive conference site and a pleasant social program. With best regards from the organizing team: Uwe Ligges (conference), Achim Zeileis (program), Claus Weihs, Gerd Kopp (local organiza- tion), Friedrich Leisch, and Torsten Hothorn Address / Contact Address: Uwe Ligges Fakultät Statistik Technische Universität Dortmund 44221 Dortmund Germany Phone: +49 231 755 4353 Fax: +49 231 755 4387 e-mail: [email protected] URL: http://www.R-Project.org/useR-2008/ Program Committee Micah Altman, Roger Bivand, Peter Dalgaard, Jan de Leeuw, Ramón Díaz-Uriarte, Spencer Graves, Leonhard Held, Torsten Hothorn, François Husson, Christian Kleiber, Friedrich Leisch, Andy Liaw, Martin Mächler, Kate Mullen, Ei-ji Nakama, Thomas Petzoldt, Martin Theus, and Heather Turner Conference Location Technische Universität Dortmund Campus Nord Mathematikgebäude / Audimax Vogelpothsweg 87 44227 Dortmund Conference Ofﬁce Opening hours: Monday, August 11: 08:30–19:30 Tuesday, August 12: 08:30–18:30 Wednesday, August 13: 08:30–18:30 Thursday, August 14: 08:30–15:30 useR! 2008, Dortmund, Germany 3 Public Transport The conference site at the university campus and the city hall are best to be reached by public transport.
[Show full text]
Language-Agnostic Data Analysis Workflows and Reproducible Research
Language-agnostic data analysis workflows and reproducible research Andrew John Lowe 28 April 2017 Abstract This is the abstract for a document that briefly describes methods for interacting with different programming languages within the same data analysis workflow. 0.1 Overview • This talk: language-agnostic (or polyglot) analysis workflows • I’ll show how it is possible to work in multiple languages and switch between them without leaving the workflow you started • Additionally, I’ll demonstrate how an entire workflow can be encapsulated in a markdown file that is rendered to a publishable paper with cross-references and a bibliography (and with raw a LATEX file produced as a by-product) in a simple process, making the whole analysis workflow reproducible 0.2 Which tool/language is best? • TMVA, scikit-learn, h2o, caret, mlr, WEKA, Shogun, . • C++, Python, R, Java, MATLAB/Octave, Julia, . • Many languages, environments and tools • People may have strong opinions • A false polychotomy? • Be a polyglot! 0.3 Approaches • Save-and-load • Language-specific bindings • Notebook • knitr 1 Save-and-load approach 1.1 Flat files • Language agnostic • Formats like text, CSV, and JSON are well-supported • Breaks workflow • Data types may not be preserved (e.g., datetime, NULL) • New binary format Feather solves this 1 1.2 Feather Feather: A fast on-disk format for data frames for R and Python, powered by Apache Arrow, developed by Wes McKinney and Hadley Wickham # In R: library(feather) path <- "my_data.feather" write_feather(df, path) # In Python: import feather
[Show full text]
Dynamic Documents with R and Knitr, by Yihui Xie 116 Tugboat, Volume 35 (2014), No
TUGboat, Volume 35 (2014), No. 1 115 ● ● ● Book review: Dynamic Documents with R ● ● ● and knitr, by Yihui Xie ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Boris Veytsman ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Yihui Xie, Dynamic Documents with R and knitr. ● ● ● ● ● ● ● ● ● ● ● ● ● ● Chapman & Hall/CRC Press, 2013, 190+xxvi pp. ● ● ● ● ● ● ● ● ● US$ ISBN ● Paperback, 59.95. 978-1482203530. ● ● ● ● Petal Length, cm Petal ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 1 2 3 4 5 6 7 0.5 1.0 1.5 2.0 2.5 Petal Width, cm This plot shows an almost linear dependence between the parameters. We can try a linear fit for these data: model <- lm(Petal.Length ˜ Petal.Width, data = iris) model$coefficients ## (Intercept) Petal.Width ## 1.084 2.230 summary(model)$r.squared ## [1] 0.9271 The large value of R2 = 0.9271 indicates the good quality of the fit. Of course we can replot the data There are several reasons why this book might be of together with the prediction of the linear model: interest to a TEX user. First, LATEX has a prominent place in the book. Second, the book describes a very plot(Petal.Length ˜ Petal.Width, interesting offshoot of literate programming, a topic data = iris, xlab = "Petal Width, cm", traditionally popular in the TEX community. Third, ylab = "Petal Length, cm") abline(model) since a number of TEX users work with data analysis and statistics, R could be a useful tool for them. Since some TUGboat readers are likely not fa- ● ● ● miliar with R, I would like to start this review with ● ● a short description of the software. R [1] is a free ● ● ● ● ● ● ● ● ● ● ● implementation of the S language (sometimes R is ● ● ● ● ● ● ● ● ● ● ● ● ● ● called GNU S).
[Show full text]
LYX and Knitr
knitr.1 LYX and knitr The knitr package allows one to embed R code within LYX and LATEX documents. When a document is compiled into a PDF, LYX/LATEX connects to R to run the R code and the code/output is automatically put into the PDF. In addition to this being a convenient way to use both LYX/LATEX with R, it also provides an important component to the reproducibility of research (RR). For example, one can include the code for a data analysis de- scribed in a paper. This ensures that there would be no “copying and pasting errors” and also provide readers of the paper an im- mediate way to reproduce the research. RR continues to become more important and fortunately more tools are being developed to make it possible. Below are some discussions on the topic: • AMSTAT News column on RR at http://magazine. amstat.org/blog/2011/01/01/scipolicyjan11. • CRAN task view for RR and R at http://cran.r-project. org/web/views/ReproducibleResearch.html. • Yihui Xie: Author of knitr – First and second editions of his Dynamic Documents with R and knitr book. Note that this book was typed in LYX. – Website for knitr at http://yihui.name/knitr The Sweave environment is another way to include R code inside of LYX/LATEX. This was developed prior to knitr, but it is more diﬃcult to use. The purpose of this section is to examine the main compo- nents of knitr so that you will be able to complete the rest of the semester using LYX and knitr together for all assign- ments in our course! Also, a very important purpose is to give you the tools needed to complete all assignments in other R- based courses by using knitr and LYX together! The ﬁles used knitr.2 here are intro_example_cereal.lyx, intro_example_cereal.pdf, cereal.csv, ExternalCode.R, FirstBeamer-knitr.zip, JSM2015.zip, and RMarkdown.zip.
[Show full text]
The R Journal Volume 3/2, December 2011
The Journal Volume 3/2, December 2011 A peer-reviewed, open-access publication of the R Foundation for Statistical Computing Contents Editorial..................................................3 Contributed Research Articles Creating and Deploying an Application with (R)Excel and R...................5 glm2: Fitting Generalized Linear Models with Convergence Problems.............. 12 Implementing the Compendium Concept with Sweave and DOCSTRIP............. 16 Watch Your Spelling!........................................... 22 Ckmeans.1d.dp: Optimal k-means Clustering in One Dimension by Dynamic Programming 29 Nonparametric Goodness-of-Fit Tests for Discrete Null Distributions............... 34 Using the Google Visualisation API with R.............................. 40 GrapheR: a Multiplatform GUI for Drawing Customizable Graphs in R............. 45 rainbow: An R Package for Visualizing Functional Time Series.................. 54 Programmer’s Niche Portable C++ for R Packages...................................... 60 News and Notes R’s Participation in the Google Summer of Code 2011....................... 64 Conference Report: useR! 2011..................................... 68 Forthcoming Events: useR! 2012.................................... 70 Changes in R............................................... 72 Changes on CRAN............................................ 84 News from the Bioconductor Project.................................. 86 R Foundation News........................................... 87 2 The Journal is a peer-reviewed publication
[Show full text]
The R Journal, June 2012
The Journal Volume 2/1, June 2010 A peer-reviewed, open-access publication of the R Foundation for Statistical Computing Contents Editorial..................................................3 Contributed Research Articles IsoGene: An R Package for Analyzing Dose-response Studies in Microarray Experiments..5 MCMC for Generalized Linear Mixed Models with glmmBUGS ................. 13 Mapping and Measuring Country Shapes............................... 18 tmvtnorm: A Package for the Truncated Multivariate Normal Distribution........... 25 neuralnet: Training of Neural Networks............................... 30 glmperm: A Permutation of Regressor Residuals Test for Inference in Generalized Linear Models.................................................. 39 Online Reproducible Research: An Application to Multivariate Analysis of Bacterial DNA Fingerprint Data............................................ 44 Two-sided Exact Tests and Matching Conﬁdence Intervals for Discrete Data.......... 53 Book Reviews A Beginner’s Guide to R......................................... 59 News and Notes Conference Review: The 2nd Chinese R Conference........................ 60 Introducing NppToR: R Interaction for Notepad++......................... 62 Changes in R 2.10.1–2.11.1........................................ 64 Changes on CRAN............................................ 72 News from the Bioconductor Project.................................. 85 R Foundation News........................................... 86 2 The Journal is a peer-reviewed publication of
[Show full text]
R Markdown: the Definitive Guide
Yihui Xie, J. J. Allaire, Garrett Grolemund R Markdown: The Deﬁnitive Guide To Jung Jae-sung (1982 – 2018), a remarkably hard-working badminton player with a remarkably simple playing style Contents List of Tables xvii List of Figures xix Preface xxiii About the Authors xxxv I GetStarted 1 1 Installation 5 2 Basics 7 2.1 Example applications .................. 7 2.1.1 Airbnb’s knowledge repository ......... 7 2.1.2 Homework assignments on RPubs ....... 7 2.1.3 Personalized mail ................. 8 2.1.4 2017 Employer Health Beneﬁts Survey ..... 8 2.1.5 Journal articles .................. 8 2.1.6 Dashboards at eelloo ............... 8 2.1.7 Books ....................... 8 2.1.8 Websites ...................... 8 2.2 Compile an R Markdown document .......... 8 2.3 Cheat sheets ....................... 9 2.4 Output formats ...................... 9 2.5 Markdown syntax .................... 9 2.5.1 Inline formatting ................. 9 2.5.2 Block-level elements ............... 9 2.5.3 Math expressions ................. 9 2.6 R code chunks and inline R code ............ 10 2.6.1 Figures ....................... 10 2.6.2 Tables ....................... 10 vii viii Contents 2.7 Other language engines ................. 10 2.7.1 Python ....................... 10 2.7.2 Shell scripts .................... 10 2.7.3 SQL ........................ 11 2.7.4 Rcpp ........................ 11 2.7.5 Stan ........................ 11 2.7.6 JavaScript and CSS ................ 11 2.7.7 Julia ........................ 11 2.7.8 CandFortran ................... 11 2.8 Interactive documents .................. 11 2.8.1 HTML widgets .................. 11 2.8.2 Shiny documents ................. 12 II Output Formats 13 3 Documents 17 3.1 HTML document ..................... 17 3.1.1 Table of contents ................
[Show full text]
Stat 302 Statistical Software and Its Applications the Knitr Package and Rstudio
Stat 302 Statistical Software and Its Applications The knitr Package and RStudio Fritz Scholz Department of Statistics, University of Washington Winter Quarter 2015 January 8, 2015 1 The knitr Package The knitr package integrates code, output, and narrative. It was created by Yihui Xie who was inspired by Sweave. Right click on the R icon and start an Administrator R session. Install knitr via install.packages("knitr", dependencies=TRUE) It is assumed that LATEX is already installed on your system. In the initial use of knitr in R or RStudio you may be prompted to install further LATEX packages. That happens just once and is automatic. 2 First Usage of knitr Create a working directory, say EX2. Inside EX2 and using a text editor create a le named minimal.Rnw with content shown on the next two slides. Note that it looks very much like a LATEX le in structure, except for the chunk delimited by <<plot .... >>= R code commands @ Also note the use of \Sexpr{...} to access contents of R objects created with the R commands. You can use ’right’ or ’left’ in place of ’center’ in fig.align. The plot after << acts as plot le name handle, see the gure subfolder in EX2. Dierent plots require dierent handles. 3 Content of minimal.Rnw (Slide 1) \documentclass{article} \setlength{\topmargin}{-1in} \setlength{\oddsidemargin}{-.25in} \setlength{\evensidemargin}{-.25in} \setlength{\textwidth}{7in} \setlength{\textheight}{10in} \begin{document} \begin{flushright} \parbox{1in}{Fritz Scholz\\ ID: 1234567\vspace{1.25in} } \includegraphics[width=.5in]{./figure/fritzbuzz.jpg} \end{flushright} \begin{center} {\large A Minimal Example}\vspace{.1in}\\ Fritz Scholz \end{center} We examine the relationship between speed and stopping distance using a linear regression model: $Y=\beta_0+\beta_1 x+\epsilon$.
[Show full text]
Reproducible Research Tools for R
Reproducible Research Tools for R January 2017 Boriana Pratt Princeton University 1 Literate programming Literate programming (1984) “ I believe that the time is right for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: ‘Literate Programming’. “ Donald E. Knuth. Literate Programming. The Computer Journal, 27(2):97-111, May1984. http://comjnl.oxfordjournals.org/content/27/2/97.full.pdf+html WEB “… is a combination of two other languages: (1) a document formatting language and (2) a programming language.” 2 terms • Tangle extract the code parts (code chunks), then run them sequentially • Weave extract the text part (documentation chunks) and weave back in the code and code output 3 Noweb Norman Ramsey (1989) – Noweb – simple literate programming tool… https://www.cs.tufts.edu/~nr/noweb/ “Literate programming is the art of preparing programs for human readers. “ Noweb syntax includes two parts: Code chunk: <<chink name>>= - section that starts with <<name>>= Documentation chunk: @ - line that starts with @ followed by a space; default for the first chunk 4 Tools for R 5 Tools for R • Sweave https://stat.ethz.ch/R-manual/R-devel/library/utils/doc/Sweave.pdf What is Sweave? A tool that allows to embed the R code in LaTeX documents. The purpose is to create dynamic reports, which can be updated automatically if data or analysis change. How do I cite Sweave? To cite Sweave please use the paper describing the first version: Friedrich Leisch. Sweave: Dynamic generation of statistical reports using literate data analysis.
[Show full text]