DOCSLIB.ORG

Preparing Data for Analysis in Stata Before You Can Analyse Your Data, You Need to Get Your Data Into an Appropriate Format, to Enable Stata to Work for You

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Preparing Data for Analysis in Stata Before You Can Analyse Your Data, You Need to Get Your Data Into an Appropriate Format, to Enable Stata to Work for You SIDM2 – Preparing data for analysis Preparing Data for Analysis in Stata Before you can analyse your data, you need to get your data into an appropriate format, to enable Stata to work for you. To avoid rubbish results, you need to check your data is sensible and free of nonsense values. Recap on preceding workshop, SIDM1 It is essential to create and save a do file of commands, otherwise you will lose your work. Attention needs to be paid to the presence of missing data, which are treated as infinitely large positive values by Stata. You learnt to run commands from this do file as you go along. You also learn how to open and save datasets, log files and graphs. You learn to distinguish between different types of data. You learnt to use variable labels for a fuller description of what they are, and value labels to define categories (when coded numerically, so we know what the numbers represent). You learn basic graph and tables commands, how to create new variables and amend their values, use of if statement. Learning objectives of this Session, SIDM2 This describes how to read in data Stata in the first place. There is a recap on different types of data, and rationale on need to change some variables between formats before analysis can begin, in many cases. This gives example code, and exercises (with solutions available) for you to see how these commands are used in practice. Learning objectives of further workshops, SIDM3 and SIDM4. Merging datasets in many different ways, reshaping datasets, looping in Stata, and extracting saved results into files. Efficient production of publication quality tables. Further resources complementary to this series This series teaches most of the material contained in Stata Data Management.doc, referenced SDM. The accompanying Stata commands crib sheet.xls, SCCS, acts as a quick reference guide (and also summarises some data analysis commands). Stata manuals (accessed online and via help) and Stata help itself, are both excellent resources. The manuals teach statistics, as well as Stata, and provide statistics references. Contents 1. Reading data into Stata from other files ................................................................................................................... 2 2. Recap on types of data .............................................................................................................................................. 2 3. Converting strings to numeric and categorical data as necessary ............................................................................ 3 4. Dealing with Dates .................................................................................................................................................... 4 5. Checking for errors and missing data ....................................................................................................................... 4 6. When your dataset erroneously has 2 or more lines of data for a few patients ...................................................... 5 7. Recoding numeric data into groups .......................................................................................................................... 5 8. Extracting information from string variables ............................................................................................................ 7 9. Further sources of help ............................................................................................................................................. 7 SDM=Stata Data Management.doc Hilary Watt SIDM=Stata Introduction and Data Management.doc workshops SCCS=Stata Commands Crib Sheet.xls 2.1 SIDM2 – Preparing data for analysis 1. Reading data into Stata from other files Now suppose that the census data we looked at last week was received as an excel file, that we need to read into Stata before we can analyse it. See SDM 2.3 Opening data from an excel file. See SDM chapter 2 for reading in from other sources. cd "H:\_MPHTeaching\stata\dataman\wk2" clear import excel census, firstrow /* reading data in from excel file census. Xls, firstrow indicates that the first row is treated as variable names or labels */ descr // look for string variables, storage type = str## browse // look for string variables which appear in red Student Exercise: Read in nlsw88dates2.xls into Stata. View the data, looking at types of variables and seeing what is contains. Summarise the data. Here is some further information on what is contained in this data set, with commands for labelling appropriately: label var grade "Current grade completed" label var c_city "Lives in central city" label var wage "Hourly Wage" label var south "Lives in South" label var union "Union Worker" label var hours "Usual hours worked" label var ttl_exp "Total Work Experience" label var tenure "Job Tenure (years)" label var quesday "Day of month that questionnaire was filled in (started to be filled in)" label var quesmon "Month that questionnaire was filled in (started to be filled in)" label var quesyr "Year that questionnaire was filled in (started to be filled in)" label var quesfinish "Date that questionnaire was completed" 2. Recap on types of data There are 4 main types of data in Stata: i) numeric (numerical with types int, byte, float, double – black in data browser) ii) string (e.g. str2, str24 – red in data browser) iii) categorical (i.e. numeric with value labels - blue in data browser) iv) dates & times (numeric with format %d or %td or similar – black in data browser). The describe command will detail data types, format, presence of value labels and variable labels. It is usually necessary to have data in numeric format, in order to use it in Stata data analysis and most graph commands; this includes dates & time in Stata format recognised as such (showing up in black in the data editor) and categorical data (showing as blue and looking like text). The main exception to this is patient id variable (or hospital id’s or regions or similar) where it is generally okay to use a string variable. SDM=Stata Data Management.doc Hilary Watt SIDM=Stata Introduction and Data Management.doc workshops SCCS=Stata Commands Crib Sheet.xls 2.2 SIDM2 – Preparing data for analysis Dates and categorical data are often read into Stata as string variables, this is generally the best way to do it. Hence the need to recode into different types of variables. There may also be a desire to recode numeric data into categories, and to recode categorical variables, perhaps by combining categories. 3. Converting strings to numeric and categorical data as necessary See DSM 5.5 and 5.6 Converting strings to numeric data and categorical data *** converting from string variables to numeric variables (when most/ all values already look like strings) destring medage2, replace // converts string to numeric data, keeping the same variable name tab medage2 medage, miss // check new variable against another variable that looks the same scatter medage2 medage // they are identical summ medage medage2 // identical also in number of missing values list if medage2==. // identical also in missing values, they are for the same observation drop medage2 // we don't need 2 identical variables destring marriage, replace // gives error because there is non-numeric data help destring destring marriage, force replace /* converts string var marriage into numeric var, with missing value where there is any non-numeric data */ *** converting from strings to categorical variables descr encode region, gen(region2) // create a categorical (numeric) var (region2) from the string var, region tab region region2 // compare the newly created and original variables tab region region2, nolabel // compare the newly created and original vars without value labels codebook region2 // see correspondence of values and value labels drop region // the string version is nolonger needed encode state2, gen(state_2) // create categorical var (state_2) from string var, state2 tab state2 state_2 // compare - browse state2 state_2 // easier to compare this way codebook state_2 // gives examples of coding label list state_2 // gives the full numeric correspondence between numbers and value labels drop state2 // string version is no-longer needed Student Exercise: a) Look for string variables in nlsw88dates2.xls that look numeric/ as if they should be numeric. Create numeric version of (one or more of) the variables. b) Look for categorical variables and convert (some of) them also to numeric variables. For instance, do this for industry and race. Does encode command work well for both/all? If not, then what approach shall we take? c) Try help string function and decide whether it is a good strategy to use one or a few string functions to tidy up the variables before using the encode command. For instance, could take just the first character and change to lower case. SDM=Stata Data Management.doc Hilary Watt SIDM=Stata Introduction and Data Management.doc workshops SCCS=Stata Commands Crib Sheet.xls 2.3 SIDM2 – Preparing data for analysis 4. Dealing with Dates See SDM chapter 7 on dates. *** convering to Stata dates variables from string variables gen dateofsurvey3=date( dateofsurvey, "DMY") /* converts from string to Stata date variable, string ordered day month year DMY */ browse dateofsurvey dateofsurvey3 // dates are coded as number of days from a fixed date format dateofsurvey3 %d /* display the date variable in a format that we can understand as a date (not a number) */ browse dateofsurvey dateofsurvey3 //
Load more

Recommended publications
  • IBM Cognos Analytics - Reporting Version 11.1
    IBM Cognos Analytics - Reporting Version 11.1 User Guide IBM © Product Information This document applies to IBM Cognos Analytics version 11.1.0 and may also apply to subsequent releases. Copyright Licensed Materials - Property of IBM © Copyright IBM Corp. 2005, 2021. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM, the IBM logo and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at " Copyright and trademark information " at www.ibm.com/legal/copytrade.shtml. The following terms are trademarks or registered trademarks of other companies: • Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. • Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. • Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. • Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. • UNIX is a registered trademark of The Open Group in the United States and other countries. • Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.
    [Show full text]
  • A Dll Required for This Install Could Not Be Run
    A Dll Required For This Install Could Not Be Run Foldable Hannibal saunter anticipatorily and respectably, she reseat her wentletrap overlaid retractively. Which Arne igniting butso cheap berrying that her Dickie instruments triturated flying. her tanists? Analyzed Giovanne still encyst: salt and Circassian Whitby outface quite fervently This product was an option, microsoft distributed dll required dll for could not a install the problem for fixing the table doe Will not a install could be run this dll required for what is solved by multiple rows into boot. How to resolve my case, code and performance cookies and this dll for install a could not be run in. Any solution is required actions, hardware failure and be. If he're running Windows installation as the repair source or sale you're using Windows from a. Thank you for safe prompt response. A DLL required for this installation to complete could not be run. Does this solution from your pc scan with windows installer on target system is this tool in any proposed solutions to use windows updates about how did run a this dll required for could not install be. Reddit on the respective owners in this dll for a required. Set properties are you hate cookies may not a install be run this dll required for instant savings! Could not initialized handler. Your pc and framework, dll could not be able to customize it? Qgis also for this issue, dll required for could not a install it is a time i run. Fix problems installing Chrome Google Chrome Help.
    [Show full text]
  • System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 15 SP1
    SUSE Linux Enterprise Server 15 SP1 System Analysis and Tuning Guide System Analysis and Tuning Guide SUSE Linux Enterprise Server 15 SP1 An administrator's guide for problem detection, resolution and optimization. Find how to inspect and optimize your system by means of monitoring tools and how to eciently manage resources. Also contains an overview of common problems and solutions and of additional help and documentation resources. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006– 2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see https://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xii 1 Available Documentation xiii
    [Show full text]
  • Diskgenius User Guide (PDF)
    www.diskgenius.com DiskGenius® User Guide The information in this document is subject to change without notice. This document is not warranted to be error free. Copyright © 2010-2021 Eassos Ltd. All Rights Reserved 1 / 236 www.diskgenius.com CONTENTS Introduction ................................................................................................................................. 6 Partition Management ............................................................................................................. 6 Create New Partition ........................................................................................................ 6 Active Partition (Mark Partition as Active) .............................................................. 10 Delete Partition ................................................................................................................ 12 Format Partition ............................................................................................................... 14 Hide Partition .................................................................................................................... 15 Modify Partition Parameters ........................................................................................ 17 Resize Partition ................................................................................................................. 20 Split Partition ..................................................................................................................... 23 Extend
    [Show full text]
  • Troubleshooting TCP/IP
    4620-1 ch05.f.qc 10/28/99 12:00 PM Page 157 Chapter 5 Troubleshooting TCP/IP In This Chapter ᮣ Troubleshooting TCP/IP in Windows 2000 Server ᮣ TCP/IP troubleshooting steps ᮣ Defining which is the best TCP/IP troubleshooting tool to solve your problem ᮣ Mastering basic TCP/IP utilities roubleshooting is, it seems, an exercise in matrix mathematics. That is, Twe use a variety of approaches and influences to successfully solve our problems, almost in a mental columns-and-rows format. These approaches may include structured methodologies, inductive and deductive reasoning, common sense, experience, and luck. And this is what troubleshooting is made of. Troubleshooting TCP/IP problems is really no different from troubleshooting other Windows 2000 Server problems mentioned in this book, such as instal- lation failures described in Chapter 2. Needless to say, Windows 2000 Server offers several TCP/IP-related tools and utilities to assist us, but more on the specifics in a moment. TCP/IP Troubleshooting Basics The goal in TCP/IP troubleshooting is very simple: fix the problem. Too often, it is easy to become overly concerned about why something happened instead of just fixing the problem. And by fixing the problem, I mean cost effectively. 1. Be cost effective. Don’t forget that several hours’ worth of MCSE-level consulting could more than pay for the additional bandwidth that may easily solve your TCP/IP WAN-related problem. Don’t overlook such an easy fix when struggling to make a WAN connection between sites utilizing Windows 2000 Server. Too little bandwidth is typically the result of being penny wise and pound foolish.
    [Show full text]
  • Facing the Challenge(S) of Windows Logs Collection to Leverage Valuable Iocs
    Facing the challenge(s) of Windows logs collection to leverage valuable IOCs . Michel de Crevoisier Security Analyst, Radar Cyber Security 15.10.2019, Berne © RadarServices // Classification: Public The five challenges © RadarServices // Classification: Public #1 High diversity of log sources Server Microsoft 3rd party Built-in roles software software Advanced Threat ADFS Application Analytics (ATA) Ivanti software Certification authority Exchange PowerShell Kaspersky DHCP server Skype Security DNS server SQL Server Veeam Backup System IIS web server SYSMON […] […] NPS Radius Defender © RadarServices // Classification: Public 3 #2 Different log extensions EVTX ETL TXT (standard Windows logs (analytical logs, like DNS (IIS, NPS, DHCP, in XML format) Server or PowerShell) PowerShell Transcript, former DNS logs) © RadarServices // Classification: Public 4 #3 Multiple architectural approaches Access method / Protocol (MS-EVEN6, RPC, WMI,…) Push vs Pull Agent vs Agentless Intermediate collector VS Direct sending to receiver Central file store vs Shared folder Managed agent VS Unmanaged agent © RadarServices // Classification: Public 5 #4 Disabled and restrictive event logs • Protected users (if configured, on DCs only) Valuable event • LSA (Local Security Authority) logs disabled • IIS web server • DNS client Event logs with • SMB server restrictive • SMB client access • IIS web server © RadarServices // Classification: Public 6 6 #5 Operational constraints Security Data exchange Performance Configuration Environment • Avoid usage of • Data
    [Show full text]
  • Electric Vehicle Fact Sheet Opens in a New Window
    ⇝ Get CHARGED UP in ROSENDALE! ⇜ SOME QUESTIONS AND ANSWERS ABOUT ELECTRIC VEHICLES (EVs): Do electric vehicles (EVs) cost more than gas-powered cars? All major car manufacturers are now producing either fully electric vehicles or plug-in hybrids, and there is a wide selection of styles and prices to choose from. If you buy now, you can benefit from generous state rebates and federal tax credits. These may reduce the base price of an EV to between $22,000 and $29,000.6,7,8 How do I charge an EV? There is no need for a special charging station at home; just plug into any regular electrical outlet. Rosendale has a public charging station, right in the municipal lot behind the Rosendale Theatre! EV owners can charge up while enjoying Main Street, catching a movie, or hiking up Joppenbergh! Many other towns and highway rest stops around the country now have public charging stations. Popular websites and FREE phone apps, such as Plugshare, allow you to 5,12 easily find the closest public charging station while traveling. Rosendale’s charging station. How far can I drive on electric power? Plug-in hybrids have both electrical ports and gas tanks. They can run 20-80 miles on pure electric power depending on the model, after which they convert to gas and run as a fuel-efficient car. Newer all-electric cars offer faster charging capacity and a higher range, like the Chevy Bolt (238 mi. range) and Nissan Leaf (107 mi. range). How is an electric vehicle better for the environment? Another benefit of EVs: Since it uses less energy, an EV has a lower carbon footprint, No more oil changes! equivalent to that of a 68 MPG gasoline-burning car (if one existed!).
    [Show full text]
  • 1) Data Storage Types 2) Variable Naming 3) Labels and Notes Written
    Stata Community Resource Contents: 1) Data storage types 2) Variable naming 3) Labels and Notes Written on 1/27/2019 (examples shown running version 15). Data storage types: Numeric Unlike some other statistical packages (e.g., SPSS or SAS) Stata stores data as different “types.” Each data storage type has a different level of accuracy and therefore different memory storage requirements. For example, if we have a dataset loaded already, and want to examine our storage types: . describe Here we can see that the storage type is listed as “byte.” Byte indicates that the variable is stored as an integer between -127 and 100. The default data storage type for Stata is “float.” By inquiring with Stata using the help command, we see that the float variable type is much larger relative to byte: . help datatypes Also note that byte, int, and long storage types can only hold integers, whereas float and double are floating point data storage types. 1 Another difference between the various numerical data storage types is in their levels of precision. For example, float variable type data will be stored with a 7-digit level of accuracy. This means that Stata will not round a number stored as a float type, as long as the number is 7 total digits or fewer. If precision is needed past the 7- digit limit, however, the “double” storage type should be used. One potential complication of storing a >7 digit variable as a “float” or smaller is that Stata may round down the integer without advertising that it is doing so.
    [Show full text]
  • HP Openvms Volume Shadowing in Action
    I302 OpenVMS Volume Shadowing Performance Keith Parris, HP Shadowing Performance • Shadowing is primarily an availability tool, but: − Shadowing can often improve performance (e.g. reads), however: − Shadowing can often decrease performance (e.g. writes, and during merges and copies) Shadow Copy/Merge Performance: Why Does It Matter? • Excessive shadow-copy time increases Mean Time To Repair (MTTR) after a disk failure, or a site outage in a disaster-tolerant cluster • Shadow Full Merges can have a significant adverse effect on application performance • Shadow Full Copies or Full Merges can generate a high data rate (e.g. on inter-site links for Disaster Tolerant clusters) − Acceptable shadow full-copy times and link costs are often the major factors in selecting inter-site link(s) for multi-site disaster-tolerant clusters Shadowset State Hierarchy • Mini-Merge state • Copy state −Mini-Copy state Transient states −Full Copy state • Full Merge state • Steady state Copy and Merge • How a Copy or Merge gets started: − A $MOUNT command triggers a Copy This implies a Copy is a Scheduled operation − Departure of a system from the cluster while it has a shadowset mounted triggers a Merge This implies a Merge is an Unscheduled operation I/O Algorithms Used by Shadowing • Variations: − Reads vs. Writes − Steady-State vs. Copy vs. Merge • And for Copies and Merges, − Full- or Mini- operation − Copy or Merge Thread I/Os − Application I/Os: Ahead of or Behind the Copy/Merge “Fence” Application Read during Steady-State • Read from any Application
    [Show full text]
  • The CONVERT Command
    The CONVERT command Use the CONVERT command with the SQL Backup -SQL parameter to convert SQL Backup backup files (.sqb) to Microsoft Tape Format (MTF) files (.bak) using the command line or extended stored procedure. You can use the native SQL Server RESTORE command to restore MTF files. Alternatively, you can use the SQL Backup File Converter (either from the command line, or using the GUI application). This application can also convert SQL Backup v7 or v6 files to SQL Backup v5 files; both file types have extension .sqb. See SQL Backup File Converter for more information. If you convert a backup file that was created with multiple threads (THREADCOUNT > 1), an individual .bak file is created for each thread. For example, if backup file BackUpA.sqb was created using three threads, the following files are created when you convert the backup: BackUpA_01.bak BackUpA_02.bak BackUpA_03.bak You can then restore the MTF (.bak) format backups as a group of backup files. When using the extended stored procedure the parameter or set of parameters (such as -SQL) must be delimited by single quotes. Therefore, wherever a single quote is used for the arguments below, for the extended stored procedure you must use two single quotes so that SQL Server does not interpret it as a string delimiter. See Using the extended stored procedure for more information. Syntax CONVERT { 'p_hysical_backup_device_name' } ] TO { 'physical_file_name' } [ WITH [ [ , ] MAXDATABLOCK = { 65536 | 131072 | ... | 2097152 } ] [ [ , ] PASSWORD = { 'password' } ] [ [ , ] SINGLERESULTSET ] ] Arguments MAXDATABLOCK Specifies the maximum size of data blocks to be used when SQL Backup stores backup data. Valid values are integers in multiples of 65536, up to a maximum value of 2097152.
    [Show full text]
  • Traceroute and BGP AS Path Incongruities
    Traceroute and BGP AS Path Incongruities Young Hyun, Andre Broido, kc claffy CAIDA, San Diego Supercomputer Center University of California, San Diego youngh, broido, kc ¡ @caida.org Internetworking 2003, San Jose, June 2003 Background Motivation Methodology data collection initial processing Analysis exchange point ASes ASes under common ownership remaining causes Conclusions Background AS-level Internet topology is very useful . – for studying growth, performance, resiliency, convergence times – for designing routing protocols complete, up-to-date topology not available – only two practical sources of partial topology: ¡ BGP tables (e.g., at RouteViews and RIPE) ¡ AS paths derived from traceroute paths most analysis/modeling of Internet topology based on BGP AS paths Motivation Are Internet topologies based on BGP AS paths valid? answer by comparing two topology sources: BGP tables and traceroute paths simplistically: BGP AS path = specified (by policy) traceroute AS path = actual (by per-hop forwarding decisions) expect specified and actual paths to agree, but they differ in practice want to know the extent and causes of incongruities Methodology 1. collect data at three sites worldwide BGP table from router near the host performing traceroutes 2. convert traceroute IP paths to AS paths 3. match up traceroute AS paths with BGP AS paths 4. compare pairs of AS paths Source of traceroute paths CAIDA’s skitter monitors – around two dozen deployed worldwide – TTL-based like traceroute but using ICMP ECHO REQUEST – probe predetermined set of
    [Show full text]
  • Partition.Pdf
    Linux Partition HOWTO Anthony Lissot Revision History Revision 3.5 26 Dec 2005 reorganized document page ordering. added page on setting up swap space. added page of partition labels. updated max swap size values in section 4. added instructions on making ext2/3 file systems. broken links identified by Richard Calmbach are fixed. created an XML version. Revision 3.4.4 08 March 2004 synchronized SGML version with HTML version. Updated lilo placement and swap size discussion. Revision 3.3 04 April 2003 synchronized SGML and HTML versions Revision 3.3 10 July 2001 Corrected Section 6, calculation of cylinder numbers Revision 3.2 1 September 2000 Dan Scott provides sgml conversion 2 Oct. 2000. Rewrote Introduction. Rewrote discussion on device names in Logical Devices. Reorganized Partition Types. Edited Partition Requirements. Added Recovering a deleted partition table. Revision 3.1 12 June 2000 Corrected swap size limitation in Partition Requirements, updated various links in Introduction, added submitted example in How to Partition with fdisk, added file system discussion in Partition Requirements. Revision 3.0 1 May 2000 First revision by Anthony Lissot based on Linux Partition HOWTO by Kristian Koehntopp. Revision 2.4 3 November 1997 Last revision by Kristian Koehntopp. This Linux Mini−HOWTO teaches you how to plan and create partitions on IDE and SCSI hard drives. It discusses partitioning terminology and considers size and location issues. Use of the fdisk partitioning utility for creating and recovering of partition tables is covered. The most recent version of this document is here. The Turkish translation is here. Linux Partition HOWTO Table of Contents 1.
    [Show full text]