R Programming for Data Science
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Introduction to IDL®
Introduction to IDL® Revised for Print March, 2016 ©2016 Exelis Visual Information Solutions, Inc., a subsidiary of Harris Corporation. All rights reserved. ENVI and IDL are registered trademarks of Harris Corporation. All other marks are the property of their respective owners. This document is not subject to the controls of the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Contents 1 Introduction To IDL 5 1.1 Introduction . .5 1.1.1 What is ENVI? . .5 1.1.2 ENVI + IDL, ENVI, and IDL . .6 1.1.3 ENVI Resources . .6 1.1.4 Contacting Harris Geospatial Solutions . .6 1.1.5 Tutorials . .6 1.1.6 Training . .7 1.1.7 ENVI Support . .7 1.1.8 Contacting Technical Support . .7 1.1.9 Website . .7 1.1.10 IDL Newsgroup . .7 2 About This Course 9 2.1 Manual Organization . .9 2.1.1 Programming Style . .9 2.2 The Course Files . 11 2.2.1 Installing the Course Files . 11 2.3 Starting IDL . 11 2.3.1 Windows . 11 2.3.2 Max OS X . 11 2.3.3 Linux . 12 3 A Tour of IDL 13 3.1 Overview . 13 3.2 Scalars and Arrays . 13 3.3 Reading Data from Files . 15 3.4 Line Plots . 15 3.5 Surface Plots . 17 3.6 Contour Plots . 18 3.7 Displaying Images . 19 3.8 Exercises . 21 3.9 References . 21 4 IDL Basics 23 4.1 IDL Directory Structure . 23 4.2 The IDL Workbench . 24 4.3 Exploring the IDL Workbench . -
A Comparative Evaluation of Matlab, Octave, R, and Julia on Maya 1 Introduction
A Comparative Evaluation of Matlab, Octave, R, and Julia on Maya Sai K. Popuri and Matthias K. Gobbert* Department of Mathematics and Statistics, University of Maryland, Baltimore County *Corresponding author: [email protected], www.umbc.edu/~gobbert Technical Report HPCF{2017{3, hpcf.umbc.edu > Publications Abstract Matlab is the most popular commercial package for numerical computations in mathematics, statistics, the sciences, engineering, and other fields. Octave is a freely available software used for numerical computing. R is a popular open source freely available software often used for statistical analysis and computing. Julia is a recent open source freely available high-level programming language with a sophisticated com- piler for high-performance numerical and statistical computing. They are all available to download on the Linux, Windows, and Mac OS X operating systems. We investigate whether the three freely available software are viable alternatives to Matlab for uses in research and teaching. We compare the results on part of the equipment of the cluster maya in the UMBC High Performance Computing Facility. The equipment has 72 nodes, each with two Intel E5-2650v2 Ivy Bridge (2.6 GHz, 20 MB cache) proces- sors with 8 cores per CPU, for a total of 16 cores per node. All nodes have 64 GB of main memory and are connected by a quad-data rate InfiniBand interconnect. The tests focused on usability lead us to conclude that Octave is the most compatible with Matlab, since it uses the same syntax and has the native capability of running m-files. R was hampered by somewhat different syntax or function names and some missing functions. -
Sage 9.4 Reference Manual: Finite Rings Release 9.4
Sage 9.4 Reference Manual: Finite Rings Release 9.4 The Sage Development Team Aug 24, 2021 CONTENTS 1 Finite Rings 1 1.1 Ring Z=nZ of integers modulo n ....................................1 1.2 Elements of Z=nZ ............................................ 15 2 Finite Fields 39 2.1 Finite Fields............................................... 39 2.2 Base Classes for Finite Fields...................................... 47 2.3 Base class for finite field elements.................................... 61 2.4 Homset for Finite Fields......................................... 69 2.5 Finite field morphisms.......................................... 71 3 Prime Fields 77 3.1 Finite Prime Fields............................................ 77 3.2 Finite field morphisms for prime fields................................. 79 4 Finite Fields Using Pari 81 4.1 Finite fields implemented via PARI’s FFELT type............................ 81 4.2 Finite field elements implemented via PARI’s FFELT type....................... 83 5 Finite Fields Using Givaro 89 5.1 Givaro Finite Field............................................ 89 5.2 Givaro Field Elements.......................................... 94 5.3 Finite field morphisms using Givaro................................... 102 6 Finite Fields of Characteristic 2 Using NTL 105 6.1 Finite Fields of Characteristic 2..................................... 105 6.2 Finite Fields of characteristic 2...................................... 107 7 Miscellaneous 113 7.1 Finite residue fields........................................... -
Navigating the R Package Universe by Julia Silge, John C
CONTRIBUTED RESEARCH ARTICLES 558 Navigating the R Package Universe by Julia Silge, John C. Nash, and Spencer Graves Abstract Today, the enormous number of contributed packages available to R users outstrips any given user’s ability to understand how these packages work, their relative merits, or how they are related to each other. We organized a plenary session at useR!2017 in Brussels for the R community to think through these issues and ways forward. This session considered three key points of discussion. Users can navigate the universe of R packages with (1) capabilities for directly searching for R packages, (2) guidance for which packages to use, e.g., from CRAN Task Views and other sources, and (3) access to common interfaces for alternative approaches to essentially the same problem. Introduction As of our writing, there are over 13,000 packages on CRAN. R users must approach this abundance of packages with effective strategies to find what they need and choose which packages to invest time in learning how to use. At useR!2017 in Brussels, we organized a plenary session on this issue, with three themes: search, guidance, and unification. Here, we summarize these important themes, the discussion in our community both at useR!2017 and in the intervening months, and where we can go from here. Users need options to search R packages, perhaps the content of DESCRIPTION files, documenta- tion files, or other components of R packages. One author (SG) has worked on the issue of searching for R functions from within R itself in the sos package (Graves et al., 2017). -
Installing R
Installing R Russell Almond August 29, 2020 Objectives When you finish this lesson, you will be able to 1) Start and Stop R and R Studio 2) Download, install and run the tidyverse package. 3) Get help on R functions. What you need to Download R is a programming language for statistics. Generally, the way that you will work with R code is you will write scripts—small programs—that do the analysis you want to do. You will also need a development environment which will allow you to edit and run the scripts. I recommend RStudio, which is pretty easy to learn. In general, you will need three things for an analysis job: • R itself. R can be downloaded from https://cloud.r-project.org. If you use a package manager on your computer, R is likely available there. The most common package managers are homebrew on Mac OS, apt-get on Debian Linux, yum on Red hat Linux, or chocolatey on Windows. You may need to search for ‘cran’ to find the name of the right package. For Debian Linux, it is called r-base. • R Studio development environment. R Studio https://rstudio.com/products/rstudio/download/. The free version is fine for what we are doing. 1 There are other choices for development environments. I use Emacs and ESS, Emacs Speaks Statistics, but that is mostly because I’ve been using Emacs for 20 years. • A number of R packages for specific analyses. These can be downloaded from the Comprehensive R Archive Network, or CRAN. Go to https://cloud.r-project.org and click on the ‘Packages’ tab. -
BUGS Code for Item Response Theory
JSS Journal of Statistical Software August 2010, Volume 36, Code Snippet 1. http://www.jstatsoft.org/ BUGS Code for Item Response Theory S. McKay Curtis University of Washington Abstract I present BUGS code to fit common models from item response theory (IRT), such as the two parameter logistic model, three parameter logisitic model, graded response model, generalized partial credit model, testlet model, and generalized testlet models. I demonstrate how the code in this article can easily be extended to fit more complicated IRT models, when the data at hand require a more sophisticated approach. Specifically, I describe modifications to the BUGS code that accommodate longitudinal item response data. Keywords: education, psychometrics, latent variable model, measurement model, Bayesian inference, Markov chain Monte Carlo, longitudinal data. 1. Introduction In this paper, I present BUGS (Gilks, Thomas, and Spiegelhalter 1994) code to fit several models from item response theory (IRT). Several different software packages are available for fitting IRT models. These programs include packages from Scientific Software International (du Toit 2003), such as PARSCALE (Muraki and Bock 2005), BILOG-MG (Zimowski, Mu- raki, Mislevy, and Bock 2005), MULTILOG (Thissen, Chen, and Bock 2003), and TESTFACT (Wood, Wilson, Gibbons, Schilling, Muraki, and Bock 2003). The Comprehensive R Archive Network (CRAN) task view \Psychometric Models and Methods" (Mair and Hatzinger 2010) contains a description of many different R packages that can be used to fit IRT models in the R computing environment (R Development Core Team 2010). Among these R packages are ltm (Rizopoulos 2006) and gpcm (Johnson 2007), which contain several functions to fit IRT models using marginal maximum likelihood methods, and eRm (Mair and Hatzinger 2007), which contains functions to fit several variations of the Rasch model (Fischer and Molenaar 1995). -
Supplementary Materials
Tomic et al, SIMON, an automated machine learning system reveals immune signatures of influenza vaccine responses 1 Supplementary Materials: 2 3 Figure S1. Staining profiles and gating scheme of immune cell subsets analyzed using mass 4 cytometry. Representative gating strategy for phenotype analysis of different blood- 5 derived immune cell subsets analyzed using mass cytometry in the sample from one donor 6 acquired before vaccination. In total PBMC from healthy 187 donors were analyzed using 7 same gating scheme. Text above plots indicates parent population, while arrows show 8 gating strategy defining major immune cell subsets (CD4+ T cells, CD8+ T cells, B cells, 9 NK cells, Tregs, NKT cells, etc.). 10 2 11 12 Figure S2. Distribution of high and low responders included in the initial dataset. Distribution 13 of individuals in groups of high (red, n=64) and low (grey, n=123) responders regarding the 14 (A) CMV status, gender and study year. (B) Age distribution between high and low 15 responders. Age is indicated in years. 16 3 17 18 Figure S3. Assays performed across different clinical studies and study years. Data from 5 19 different clinical studies (Study 15, 17, 18, 21 and 29) were included in the analysis. Flow 20 cytometry was performed only in year 2009, in other years phenotype of immune cells was 21 determined by mass cytometry. Luminex (either 51/63-plex) was performed from 2008 to 22 2014. Finally, signaling capacity of immune cells was analyzed by phosphorylation 23 cytometry (PhosphoFlow) on mass cytometer in 2013 and flow cytometer in all other years. -
A Practical Guide for Improving Transparency and Reproducibility in Neuroimaging Research Krzysztof J
bioRxiv preprint first posted online Feb. 12, 2016; doi: http://dx.doi.org/10.1101/039354. The copyright holder for this preprint (which was not peer-reviewed) is the author/funder. It is made available under a CC-BY 4.0 International license. A practical guide for improving transparency and reproducibility in neuroimaging research Krzysztof J. Gorgolewski and Russell A. Poldrack Department of Psychology, Stanford University Abstract Recent years have seen an increase in alarming signals regarding the lack of replicability in neuroscience, psychology, and other related fields. To avoid a widespread crisis in neuroimaging research and consequent loss of credibility in the public eye, we need to improve how we do science. This article aims to be a practical guide for researchers at any stage of their careers that will help them make their research more reproducible and transparent while minimizing the additional effort that this might require. The guide covers three major topics in open science (data, code, and publications) and offers practical advice as well as highlighting advantages of adopting more open research practices that go beyond improved transparency and reproducibility. Introduction The question of how the brain creates the mind has captivated humankind for thousands of years. With recent advances in human in vivo brain imaging, we how have effective tools to peek into biological underpinnings of mind and behavior. Even though we are no longer constrained just to philosophical thought experiments and behavioral observations (which undoubtedly are extremely useful), the question at hand has not gotten any easier. These powerful new tools have largely demonstrated just how complex the biological bases of behavior actually are. -
Introduction to Ggplot2
Introduction to ggplot2 Dawn Koffman Office of Population Research Princeton University January 2014 1 Part 1: Concepts and Terminology 2 R Package: ggplot2 Used to produce statistical graphics, author = Hadley Wickham "attempt to take the good things about base and lattice graphics and improve on them with a strong, underlying model " based on The Grammar of Graphics by Leland Wilkinson, 2005 "... describes the meaning of what we do when we construct statistical graphics ... More than a taxonomy ... Computational system based on the underlying mathematics of representing statistical functions of data." - does not limit developer to a set of pre-specified graphics adds some concepts to grammar which allow it to work well with R 3 qplot() ggplot2 provides two ways to produce plot objects: qplot() # quick plot – not covered in this workshop uses some concepts of The Grammar of Graphics, but doesn’t provide full capability and designed to be very similar to plot() and simple to use may make it easy to produce basic graphs but may delay understanding philosophy of ggplot2 ggplot() # grammar of graphics plot – focus of this workshop provides fuller implementation of The Grammar of Graphics may have steeper learning curve but allows much more flexibility when building graphs 4 Grammar Defines Components of Graphics data: in ggplot2, data must be stored as an R data frame coordinate system: describes 2-D space that data is projected onto - for example, Cartesian coordinates, polar coordinates, map projections, ... geoms: describe type of geometric objects that represent data - for example, points, lines, polygons, ... aesthetics: describe visual characteristics that represent data - for example, position, size, color, shape, transparency, fill scales: for each aesthetic, describe how visual characteristic is converted to display values - for example, log scales, color scales, size scales, shape scales, .. -
Sharing and Organizing Research Products As R Packages Matti Vuorre1 & Matthew J
Sharing and organizing research products as R packages Matti Vuorre1 & Matthew J. C. Crump2 1 Oxford Internet Institute, University of Oxford, United Kingdom 2 Department of Psychology, Brooklyn College of CUNY, New York USA A consensus on the importance of open data and reproducible code is emerging. How should data and code be shared to maximize the key desiderata of reproducibility, permanence, and accessibility? Research assets should be stored persistently in formats that are not software restrictive, and documented so that others can reproduce and extend the required computations. The sharing method should be easy to adopt by already busy researchers. We suggest the R package standard as a solution for creating, curating, and communicating research assets. The R package standard, with extensions discussed herein, provides a format for assets and metadata that satisfies the above desiderata, facilitates reproducibility, open access, and sharing of materials through online platforms like GitHub and Open Science Framework. We discuss a stack of R resources that help users create reproducible collections of research assets, from experiments to manuscripts, in the RStudio interface. We created an R package, vertical, to help researchers incorporate these tools into their workflows, and discuss its functionality at length in an online supplement. Together, these tools may increase the reproducibility and openness of psychological science. Keywords: reproducibility; research methods; R; open data; open science Word count: 5155 Introduction package standard, with additional R authoring tools, provides a robust framework for organizing and sharing reproducible Research projects produce experiments, data, analyses, research products. manuscripts, posters, slides, stimuli and materials, computa- Some advances in data-sharing standards have emerged: It tional models, and more. -
Reproducible Reports with Knitr and R Markdown
Reproducible Reports with knitr and R Markdown https://dl.dropboxusercontent.com/u/15335397/slides/2014-UPe... Reproducible Reports with knitr and R Markdown Yihui Xie, RStudio 11/22/2014 @ UPenn, The Warren Center 1 of 46 1/15/15 2:18 PM Reproducible Reports with knitr and R Markdown https://dl.dropboxusercontent.com/u/15335397/slides/2014-UPe... An appetizer Run the app below (your web browser may request access to your microphone). http://bit.ly/upenn-r-voice install.packages("shiny") Or just use this: https://yihui.shinyapps.io/voice/ 2/46 2 of 46 1/15/15 2:18 PM Reproducible Reports with knitr and R Markdown https://dl.dropboxusercontent.com/u/15335397/slides/2014-UPe... Overview and Introduction 3 of 46 1/15/15 2:18 PM Reproducible Reports with knitr and R Markdown https://dl.dropboxusercontent.com/u/15335397/slides/2014-UPe... I know you click, click, Ctrl+C and Ctrl+V 4/46 4 of 46 1/15/15 2:18 PM Reproducible Reports with knitr and R Markdown https://dl.dropboxusercontent.com/u/15335397/slides/2014-UPe... But imagine you hear these words after you finished a project Please do that again! (sorry we made a mistake in the data, want to change a parameter, and yada yada) http://nooooooooooooooo.com 5/46 5 of 46 1/15/15 2:18 PM Reproducible Reports with knitr and R Markdown https://dl.dropboxusercontent.com/u/15335397/slides/2014-UPe... Basic ideas of dynamic documents · code + narratives = report · i.e. computing languages + authoring languages We built a linear regression model. -
The Split-Apply-Combine Strategy for Data Analysis
JSS Journal of Statistical Software April 2011, Volume 40, Issue 1. http://www.jstatsoft.org/ The Split-Apply-Combine Strategy for Data Analysis Hadley Wickham Rice University Abstract Many data analysis problems involve the application of a split-apply-combine strategy, where you break up a big problem into manageable pieces, operate on each piece inde- pendently and then put all the pieces back together. This insight gives rise to a new R package that allows you to smoothly apply this strategy, without having to worry about the type of structure in which your data is stored. The paper includes two case studies showing how these insights make it easier to work with batting records for veteran baseball players and a large 3d array of spatio-temporal ozone measurements. Keywords: R, apply, split, data analysis. 1. Introduction What do we do when we analyze data? What are common actions and what are common mistakes? Given the importance of this activity in statistics, there is remarkably little research on how data analysis happens. This paper attempts to remedy a very small part of that lack by describing one common data analysis pattern: Split-apply-combine. You see the split-apply- combine strategy whenever you break up a big problem into manageable pieces, operate on each piece independently and then put all the pieces back together. This crops up in all stages of an analysis: During data preparation, when performing group-wise ranking, standardization, or nor- malization, or in general when creating new variables that are most easily calculated on a per-group basis.