DOCSLIB.ORG

Overview of Statistical Software SPSS, Stata, SAS, R

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Overview of Statistical Software SPSS, Stata, SAS, R Overview of Statistical Software SPSS, Stata, SAS, R Debby Kermer Data Services George Mason University Software v 25 v 15 spss.com stata.com v 9.4 3.5.1 sas.com r-project.org 2 Pros and Cons SPSS Stata SAS R Use High Low High Growing Jobs Some Academic Many More Cost Expensive Depends Expensive Free Learning Easy Middle Hard Very Hard Extensible Scripts Users Built-in Users 3 What can it do well? SPSS Stata ANOVA, Factor Analysis, Regression, diagnostics, and Discriminant Analysis robust regression; Analysis of License modules separately Survey Data, Time Series, SEM Trends, Missing Data, Tables Freely downloadable packages SAS R Data Management; Complex Anything, if you can find a models; Mixed Model Analysis, [well written] package License components separately Download additional packages SAS/GIS, SAS/STAT, SAS/ACCESS from CRAN for free 4 Who Uses it? SPSS Stata Academic: Social Scientists Academic: Economics, Public (the “SS”), and non-scientists Policy, Biomedical Researchers Non-Academic: Companies Non-Academic: Groups that that just want to do neat things often work with academics SAS R Academic: Statistics, Medicine Academic: Statistics, various Non-Academic: Government, Non-Academic: Small and corporations who are companies with big plans, and serious about data others serious about data 5 Which to Pick? SPSS Stata Easy to start, limited capability Easy syntax, highly extensible Best for those with infrequent Best for academics doing and/or minimal needs cutting-edge research SAS R Hard to learn, highly capable Hard to learn, highly extensible Best for managing huge and/or Best for those who program complex datasets and know what they are doing 6 Job Prospects R vs SAS vs Python Survey of selected “quantitative professionals”, 2016 http://www.burtchworks.com/2016/07/13/sas-r-python-survey-2016-tool-analytics-pros-prefer/ 9 Use in Academia # of Scholarly Articles on Google Scholar 2015 http://r4stats.com/articles/popularity/ 12 Use in Industry # of Analytics Jobs on Indeed.com February 2014 http://r4stats.com/articles/popularity/ 13 Companies using it http://blog.datacamp.com/statistical-language-wars-the-infograph/ 14 Use Interface SPSS Stata SAS R 16 GUI SPSS Stata SAS Studio Deducer & R Cmdr 17 Syntax Contingency Table for variable q1 and q2; with only n, row %, and χ2 test SPSS Stata CROSSTABS tabulate q1 q2, obs row chi2 /TABLES= q1 BY q2 /STATISTICS=CHISQ /CELLS=COUNT ROW. SAS R PROC FREQ data=test; mytable <- table(q1, q2) table q1*q2 mytable / NOCOL NOPERCENT CHISQ; prop.table(mytable, 1) RUN; chisq.test(mytable) 19 Learning Curve http://guides.nyu.edu/quant/statsoft#s-lib-ctab-6295863-7 20 Important Differences Working with multiple files SPSS Multiple datasets allowed, active data can be specified Stata One dataset at a time, allows multiple instances SAS Data always specified, no datasets in memory R Data always specified, multiple objects in memory 22 Directories & Data Files SPSS cd "directory" filename.sav Stata cd "directory" filename SAS libname name "directory" name.filename R setwd("directory") use / or \\ filename.RData 23 Labeled/Categorical Variables SPSS separate LABEL VALUES assigns labels to levels Stata shared label define creates a 'label' SAS shared PROC step creates label 'formats' R separate defining a 'factor' creates labels for levels 24 Missing Values > # < # SPSS . no value or user defined FALSE FALSE Stata . highest possible value TRUE FALSE SAS . lowest possible value FALSE TRUE R NA no value, comparable TRUE TRUE 25 Code Characteristics Code Code Command Case Code File Prompt End Sensitive Comment SPSS Syntax File [nothing] . No * Stata Do file . [line break] Yes * SAS Program [line #] ; No * R R Script > or + [interpreted] Yes # 26 Data Files Files Data Syntax Output Others SPSS .sav .sps .spo / .spv .por Stata .dta .do .smcl / .log .dct SAS .sas7bdat .sas .lst / .log .sas7??? R .RData / .rda .R / .txt .txt .R?? 28 Opening other File Types in… Can open Stata and SAS directly Use usespss, R, or Stat/Transfer (commercial) Can import SPSS and Stata directly Use packages foreign or haven to convert 29 Resources Help transitioning, links to help for each software http://dataservices.gmu.edu/resources/software Single Statistical Software Initiative https://wikis.uit.tufts.edu/confluence/display/SSSI/Home 31.
Load more

Recommended publications
  • Python for Economists Alex Bell
    Python for Economists Alex Bell [email protected] http://xkcd.com/353 This version: October 2016. If you have not already done so, download the files for the exercises here. Contents 1 Introduction to Python 3 1.1 Getting Set-Up................................................. 3 1.2 Syntax and Basic Data Structures...................................... 3 1.2.1 Variables: What Stata Calls Macros ................................ 4 1.2.2 Lists.................................................. 5 1.2.3 Functions ............................................... 6 1.2.4 Statements............................................... 7 1.2.5 Truth Value Testing ......................................... 8 1.3 Advanced Data Structures .......................................... 10 1.3.1 Tuples................................................. 10 1.3.2 Sets .................................................. 11 1.3.3 Dictionaries (also known as hash maps) .............................. 11 1.3.4 Casting and a Recap of Data Types................................. 12 1.4 String Operators and Regular Expressions ................................. 13 1.4.1 Regular Expression Syntax...................................... 14 1.4.2 Regular Expression Methods..................................... 16 1.4.3 Grouping RE's ............................................ 18 1.4.4 Assertions: Non-Capturing Groups................................. 19 1.4.5 Portability of REs (REs in Stata).................................. 20 1.5 Working with the Operating System....................................
    [Show full text]
  • TSP 5.0 Reference Manual
    TSP 5.0 Reference Manual Bronwyn H. Hall and Clint Cummins TSP International 2005 Copyright 2005 by TSP International First edition (Version 4.0) published 1980. TSP is a software product of TSP International. The information in this document is subject to change without notice. TSP International assumes no responsibility for any errors that may appear in this document or in TSP. The software described in this document is protected by copyright. Copying of software for the use of anyone other than the original purchaser is a violation of federal law. Time Series Processor and TSP are trademarks of TSP International. ALL RIGHTS RESERVED Table Of Contents 1. Introduction_______________________________________________1 1. Welcome to the TSP 5.0 Help System _______________________1 2. Introduction to TSP ______________________________________2 3. Examples of TSP Programs _______________________________3 4. Composing Names in TSP_________________________________4 5. Composing Numbers in TSP _______________________________5 6. Composing Text Strings in TSP_____________________________6 7. Composing TSP Commands _______________________________7 8. Composing Algebraic Expressions in TSP ____________________8 9. TSP Functions _________________________________________10 10. Character Set for TSP ___________________________________11 11. Missing Values in TSP Procedures _________________________13 12. LOGIN.TSP file ________________________________________14 2. Command summary_______________________________________15 13. Display Commands _____________________________________15
    [Show full text]
  • Towards a Fully Automated Extraction and Interpretation of Tabular Data Using Machine Learning
    UPTEC F 19050 Examensarbete 30 hp August 2019 Towards a fully automated extraction and interpretation of tabular data using machine learning Per Hedbrant Per Hedbrant Master Thesis in Engineering Physics Department of Engineering Sciences Uppsala University Sweden Abstract Towards a fully automated extraction and interpretation of tabular data using machine learning Per Hedbrant Teknisk- naturvetenskaplig fakultet UTH-enheten Motivation A challenge for researchers at CBCS is the ability to efficiently manage the Besöksadress: different data formats that frequently are changed. Significant amount of time is Ångströmlaboratoriet Lägerhyddsvägen 1 spent on manual pre-processing, converting from one format to another. There are Hus 4, Plan 0 currently no solutions that uses pattern recognition to locate and automatically recognise data structures in a spreadsheet. Postadress: Box 536 751 21 Uppsala Problem Definition The desired solution is to build a self-learning Software as-a-Service (SaaS) for Telefon: automated recognition and loading of data stored in arbitrary formats. The aim of 018 – 471 30 03 this study is three-folded: A) Investigate if unsupervised machine learning Telefax: methods can be used to label different types of cells in spreadsheets. B) 018 – 471 30 00 Investigate if a hypothesis-generating algorithm can be used to label different types of cells in spreadsheets. C) Advise on choices of architecture and Hemsida: technologies for the SaaS solution. http://www.teknat.uu.se/student Method A pre-processing framework is built that can read and pre-process any type of spreadsheet into a feature matrix. Different datasets are read and clustered. An investigation on the usefulness of reducing the dimensionality is also done.
    [Show full text]
  • Why SAS Programmers Should Learn Python Too Michael Stackhouse, Covance, Inc
    PharmaSUG 2018 - Paper AD-12 Why SAS Programmers Should Learn Python Too Michael Stackhouse, Covance, Inc. ABSTRACT Day to day work can often require simple, yet repetitive tasks. All companies have tedious processes that may involve moving and renaming files, making redundant edits to code, and more. Often, one might also want to find or summarize information scattered amongst many different files as well. While SAS programmers are very familiar with the power of SAS, the power of some lesser known but readily available tools may go unnoticed. The programming language Python has many capabilities that can easily automate and facilitate these tasks. Additionally, many companies have adopted Linux and UNIX as their OS of choice. With a small amount of background, and an understanding of Linux/UNIX command line syntax, powerful tools can be developed to eliminate these tedious activities. Better yet, these tools can be written to “talk back” to the user, making them more user friendly for those averse to venturing from their SAS editor. This paper will explore the benefits of how Python can be adopted into a SAS programming world. INTRODUCTION SAS is a great language, and it does what it’s designed to do very well, but other languages are better catered to different tasks. One language in particular that can serve a number of different purposes is Python. This is for a number of reasons. First off, it’s easy. Python has a very spoken-word style of syntax that is both intuitive to use and easy to learn. This makes it a very attractive language for newcomers, but also an attractive language to someone like a SAS programmer.
    [Show full text]
  • Base SAS® Software Flexible and Extensible Fourth-Generation Programming Language Designed for Data Access, Transformation and Reporting
    FACT SHEET Base SAS® Software Flexible and extensible fourth-generation programming language designed for data access, transformation and reporting What does Base SAS® software do? Base SAS is a fourth-generation programming language (4GL) for data access, data transformation, analysis and reporting. It is included with the SAS Platform. Base SAS is designed for foundational data manipulation, information storage and retrieval, descrip- tive statistics and report writing. It also includes a powerful macro facility that reduces programming time and maintenance headaches. Why is Base SAS® software important? Base SAS runs on all major operating systems. It significantly reduces programming and maintenance time, while enabling your IT organization to produce the analyses and reports that decision makers need in the format they prefer. For whom is Base SAS® software designed? Base SAS is used by SAS programming experts and power users who prefer to code to manipulate data, produce and distribute ad hoc queries and reports, and/or interpret the results of descriptive data analysis. Many IT organizations struggle with functionality, including direct access to familiar Python interface using the SAS problems arising from complex and distrib- standardized data sources and advanced pipefitter package, fostering consis- uted data, spending excessive time synchro- statistical analysis. tency of code in the organization. nizing and reformatting data for various • Simplify reporting and delivery to applications. Producing accurate and visually Key Benefits mobile devices. Base SAS provides appealing reports often requires dispropor- maximum reporting flexibility. Easily • Integrate data across environments. tionate programming resources. Addition- create reports in formats such as RTF, Available with the SAS Platform, Base ally, IT often must manage a plethora of PDF, Microsoft PowerPoint, HTML and SAS integrates into any computing software packages that only support a e-books that can be read on many environment infrastructure, unifying specific demand.
    [Show full text]
  • Data Workflows with Stata and Python
    In [1]: from IPython.display import IFrame import ipynb_style from epstata import Stpy import pandas as pd from itertools import combinations from importlib import reload In [2]: reload(ipynb_style) ipynb_style.clean() #ipynb_style.presentation() #ipynb_style.pres2() Out[2]: Data Workflows in Stata and Python (http://www.stata.com) (https://www.python.org) Data Workflows in Stata and Python Dejan Pavlic, Education Policy Research Initiative, University of Ottawa Stephen Childs (presenter), Office of Institutional Analysis, University of Calgary (http://ucalgary.ca) (http://uottawa.ca/en) (http://socialsciences.uottawa.ca/irpe- epri/eng/index.asp) Introduction About this talk Objectives know what Python is and what advantages it has know how Python can work with Stata Please save questions for the end. Or feel free to ask me today or after the conference. Outline Introduction Overall Motivation About Python Building Blocks Running Stata from Python Pandas Python language features Workflows ETL/Data Cleaning Stata code generation Processing Stata output About Me Started using Stata in grad school (2006). Using Python for about 3 years. Post-Secondary Education sector University of Calgary - Institutional Analysis (https://oia.ucalgary.ca/Contact) Education Policy Research Initiative (http://socialsciences.uottawa.ca/irpe-epri/eng/index.asp) - University of Ottawa (a Stata shop) Motivation Python is becoming very popular in the data world. Python skills are widely applicable. Python is powerful and flexible and will help you get more done,
    [Show full text]
  • Clinical Trial Laboratory Data Management Using the SAS ® System
    Clinical Trial Laboratory Data Management using the SAS® System Marianne Hack, Covance, Inc., Princeton NJ ABSTRACT In terms of volume, for many clinical trials laboratory process lab data, these standard agreements are data comprises a very large portion of all data collected. adhered to whenever possible on any studies that Add to that the fact that laboratory data is inherently utilize these laboratories. complex, and the management of the data acquisition (electronic and/or data entry), cleaning (synchronization Given that we know what to expect when receiving data of databases), and reporting (unit conversions, files from a particular laboratory data vendor, we can character to numeric conversions, result flagging, etc.) then build standard SAS modules to import and processes can be overwhelming. This paper discusses reformat the vendor data. The end result is that a SAS an approach to building and using an arsenal of data set (see example B) containing data from Vendor specialized SAS program modules in order to A will be very similar to a SAS data set containing data standardize the various processes involved in from Vendor B. These SAS data sets are then ready managing and reporting clinical laboratory data. for the data cleaning stage. DATA CLEANING INTRODUCTION Data review and cleaning of vendor data requires As a CRO (Contract Research Organization), we are additional steps over and above what one might required to be able to work with data from a multitude of ordinarily do to review data-entered laboratory data. sources. Clinical laboratory data is usually integrated Commonly called ‘header cleaning’, this involves into the clinical database via direct data entry or reconciling the patient identifiers from the vendor through importation of electronic data files.
    [Show full text]
  • Gretl User's Guide
    Gretl User’s Guide Gnu Regression, Econometrics and Time-series Allin Cottrell Department of Economics Wake Forest university Riccardo “Jack” Lucchetti Dipartimento di Economia Università Politecnica delle Marche December, 2008 Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation (see http://www.gnu.org/licenses/fdl.html). Contents 1 Introduction 1 1.1 Features at a glance ......................................... 1 1.2 Acknowledgements ......................................... 1 1.3 Installing the programs ....................................... 2 I Running the program 4 2 Getting started 5 2.1 Let’s run a regression ........................................ 5 2.2 Estimation output .......................................... 7 2.3 The main window menus ...................................... 8 2.4 Keyboard shortcuts ......................................... 11 2.5 The gretl toolbar ........................................... 11 3 Modes of working 13 3.1 Command scripts ........................................... 13 3.2 Saving script objects ......................................... 15 3.3 The gretl console ........................................... 15 3.4 The Session concept ......................................... 16 4 Data files 19 4.1 Native format ............................................. 19 4.2 Other data file formats ....................................... 19 4.3 Binary databases ..........................................
    [Show full text]
  • Introduction to SAS
    1 Introduction to SAS How to Get SAS® The University has a license agreement with SAS® for individual use for $35 a year. You can buy and download the software package at: https://software.ucdavis.edu/index.cfm. If you can have the software installed on your laptop and bring it with you to the seminar, we can work through the examples and exercises together. SAS will run in Windows® Vista, 7, 8, and 10, and on UNIX. To run SAS on a Mac® you can run SAS University, which is a limited version that is free and runs virtually on OS X http://www.sas.com/en_us/software/university-edition/download-software.html#os-x or you will need to log in remotely to a Windows or UNIX system or use a desktop virtualization program such as Parallels®. Your department can purchase a license for Parallels® using the link above for a discount. You can get a free 14 day trial of Parallels at: http://trial.parallels.com/?lang=en&terr=us. What is SAS®? SAS is an integrated software system that enables you to perform a variety of tasks. For the purposes of this seminar series, we will be focusing on: 1) Data entry, retrieval, and management 2) Tables and graphics 3) Statistical analysis For this session held on Sept 14 and 21 we will mainly discuss (1) and (2) as a start. The future seminars will discuss (3) in detail in the context of statistical analysis methods covered. SAS stands for Statistical Analysis Software, but now we just use SAS as the registered trademark for the software package.
    [Show full text]
  • IBM SPSS Modeler Premium
    IBM Software IBM SPSS Modeler Premium IBM SPSS Modeler Premium Improve model accuracy with unstructured data IBM® SPSS® Modeler Premium helps you achieve superior outcomes Highlights by basing your business decisions on predictive intelligence. Predictive intelligence creates more effective strategies because it allows Solve business problems faster with organizations not only to evaluate trends and benchmarking results, analytical techniques that deliver deeper insight. plans and performance, but also to look into the future by evaluating likely outcomes and understanding how the interplay of factors affects • Easily access, prepare and integrate structured data and text, web and survey those outcomes. data. With SPSS Modeler Premium, you can solve any business problem • Quickly identify and extract sentiments from text in more than 30 languages. faster using powerful, proven analytical techniques that deliver deeper insight into your customers or constituents. However, because the • Extend access to text stored in legacy majority of data is trapped in unstructured or textual form – in systems, such as mainframes, through support for IBM Classic Federation Server comments, files or on the web – modeling with structured data alone and IBM® zDB2® data stores. may provide an incomplete view into your business processes and outcomes. • Easily perform in-database ­ analytics and mining for IBM® Netezza®*. SPSS Modeler Premium enables organizations to tap into the • Broaden visibility into the full customer predictive intelligence held in all forms of data. It goes beyond the view with the ability to export to IBM® Cognos® Business Intelligence directly analysis of structured numerical data and includes information from from the interface. unstructured data, such as web activity, blog content, customer feedback, emails, articles and more to discover relationships between • Add more deployment options through zLinux, SuSE Linux Enterprise Server and concepts and sentiments and create the most accurate predictive models inclusion in IBM Smart Analytics System possible.
    [Show full text]
  • Julia: a Modern Language for Modern ML
    Julia: A modern language for modern ML Dr. Viral Shah and Dr. Simon Byrne www.juliacomputing.com What we do: Modernize Technical Computing Today’s technical computing landscape: • Develop new learning algorithms • Run them in parallel on large datasets • Leverage accelerators like GPUs, Xeon Phis • Embed into intelligent products “Business as usual” will simply not do! General Micro-benchmarks: Julia performs almost as fast as C • 10X faster than Python • 100X faster than R & MATLAB Performance benchmark relative to C. A value of 1 means as fast as C. Lower values are better. A real application: Gillespie simulations in systems biology 745x faster than R • Gillespie simulations are used in the field of drug discovery. • Also used for simulations of epidemiological models to study disease propagation • Julia package (Gillespie.jl) is the state of the art in Gillespie simulations • https://github.com/openjournals/joss- papers/blob/master/joss.00042/10.21105.joss.00042.pdf Implementation Time per simulation (ms) R (GillespieSSA) 894.25 R (handcoded) 1087.94 Rcpp (handcoded) 1.31 Julia (Gillespie.jl) 3.99 Julia (Gillespie.jl, passing object) 1.78 Julia (handcoded) 1.2 Those who convert ideas to products fastest will win Computer Quants develop Scientists prepare algorithms The last 25 years for production (Python, R, SAS, DEPLOY (C++, C#, Java) Matlab) Quants and Computer Compress the Scientists DEPLOY innovation cycle collaborate on one platform - JULIA with Julia Julia offers competitive advantages to its users Julia is poised to become one of the Thank you for Julia. Yo u ' v e k i n d l ed leading tools deployed by developers serious excitement.
    [Show full text]
  • SWAT's It All About? SAS Viya for Python Users
    Paper 3610-2019 SWAT’s it all about? SAS Viya® for Python Users Carrie Foreman, Amadeus Software Limited ABSTRACT Python® is described within the community as the “second-best language for everything” (CALLAHAN, 2018). The strength of Python as a language stems from its huge range of uses including as a tool for data science. Python is taught on many university courses and as a result there are a vast number of coders with Python skills in the data science industry. Last year at Amadeus we explored the use of SASpy as an initial tool to bridge the gap between Python and SAS 9.4 (FOREMAN, 2018), this year we move on to look at integration with SAS Viya®. In July 2016, SAS® released Python SWAT (Scripting Wrapper for Analytics Transfer), a Python library. It allows connections to CAS (Cloud Analytic Services) and therefore opens up SAS Viya’s functionality to Python users. SWAT allows users who do not have a SAS background to perform their data manipulation and analytics using more familiar Pythonic syntax whilst still harnessing the power of CAS and SAS Viya. In this paper we will demonstrate the use of Python SWAT to first connect to a CAS session, load data from Python into CAS and use CAS action sets to analyze it. We will then make the data available to other applications such as SAS Visual Analytics to demonstrate some of the functionality this gives. INTRODUCTION Python is an integral part of the data science culture and is used within many organizations worldwide. Many universities teach Python as part of technology courses creating a steady stream of Python enabled programmers who then take those skills into industry.
    [Show full text]