DOCSLIB.ORG

Confidence Intervals for the Area Under the Receiver Operating Characteristic Curve in the Presence of Ignorable Missing Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Confidence Intervals for the Area Under the Receiver Operating Characteristic Curve in the Presence of Ignorable Missing Data Loyola University Chicago Loyola eCommons Mathematics and Statistics: Faculty Faculty Publications and Other Works by Publications and Other Works Department 4-2019 Confidence Intervals for the Area Under the Receiver Operating Characteristic Curve in the Presence of Ignorable Missing Data Hunyong Cho University of North Carolina at Chapel Hill Gregory J. Matthews Loyola University Chicago, [email protected] Ofer Harel University of Connecticut Follow this and additional works at: https://ecommons.luc.edu/math_facpubs Part of the Mathematics Commons Author Manuscript This is a pre-publication author manuscript of the final, published article. Recommended Citation Cho, Hunyong; Matthews, Gregory J.; and Harel, Ofer. Confidence Intervals for the Area Under the Receiver Operating Characteristic Curve in the Presence of Ignorable Missing Data. International Statistical Review, 87, 1: 152-177, 2019. Retrieved from Loyola eCommons, Mathematics and Statistics: Faculty Publications and Other Works, http://dx.doi.org/10.1111/insr.12277 This Article is brought to you for free and open access by the Faculty Publications and Other Works by Department at Loyola eCommons. It has been accepted for inclusion in Mathematics and Statistics: Faculty Publications and Other Works by an authorized administrator of Loyola eCommons. For more information, please contact [email protected]. This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License. © The Authors, 2019. HHS Public Access Author manuscript Author ManuscriptAuthor Manuscript Author Int Stat Manuscript Author Rev. Author manuscript; Manuscript Author available in PMC 2019 April 18. Published in final edited form as: Int Stat Rev. 2019 April ; 87(1): 152–177. doi:10.1111/insr.12277. Confidence Intervals for the Area Under the Receiver Operating Characteristic Curve in the Presence of Ignorable Missing Data Hunyong Cho1, Gregory J. Matthews2, and Ofer Harel3 1Department of Biostatistics, University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill, NC 27599, USA 2Department of Mathematics and Statistics, Loyola University Chicago, 1032 W. Sheridan Road, Chicago, IL 60660, USA 3Department of Statistics, University of Connecticut, 215 Glenbrook Road U-4120, Storrs, CT 06269, USA Summary Receiver operating characteristic curves are widely used as a measure of accuracy of diagnostic tests and can be summarised using the area under the receiver operating characteristic curve (AUC). Often, it is useful to construct a confidence interval for the AUC; however, because there are a number of different proposed methods to measure variance of the AUC, there are thus many different resulting methods for constructing these intervals. In this article, we compare different methods of constructing Wald-type confidence interval in the presence of missing data where the missingness mechanism is ignorable. We find that constructing confidence intervals using multiple imputation based on logistic regression gives the most robust coverage probability and the choice of confidence interval method is less important. However, when missingness rate is less severe (e.g. less than 70%), we recommend using Newcombe’s Wald method for constructing confidence intervals along with multiple imputation using predictive mean matching. Keywords ROC curve; AUC; Mann–Whitney statistic; confidence interval; multiple imputation; predictive mean matching; missing data; logistic regression 1 Introduction Diagnostic tests are often used to classify subjects as either diseased or non-diseased. Assuming that large values of test results are indicative of diseased status, a subject will be classified as diseased or non-diseased if their test results are above or below a certain threshold, respectively. Then, for a specific threshold value, the performance of the test can be evaluated using measures such as sensitivity and specificity, where sensitivity is the probability of a true positive or true positive rate (TPR) (i.e. given a positive test result, the Supporting Information Additional supporting information may be found online in the Supporting Information section at the end of the article. Cho et al. Page 2 subject actually has the disease) and specificity is the probability of a true negative or true Author ManuscriptAuthor Manuscript Author Manuscript Author Manuscript Author negative rate (TNR) (i.e. given a negative test results, the subject is actually non-diseased). A more comprehensive wayto measure the accuracy of a diagnostic test is to use the receiver operating characteristic (ROC) curve (Pepe, 2003), which graphically compares specificity and sensitivity levels; specifically, the ROC curve is a plot of sensitivity against 1 – specificity across all possible cut-off values. When comparing two diagnostic tests with one another, if the ROC curve for one test is located uniformly higher than the other test across all possible thresholds, the test is said to be strictly more accurate and therefore preferred over the other test. It is, however, not straight forward to compare two ROC curves that intersect at some location other than the end points. One common way to summarise the ROC curve is to compute the area under the ROC curve (AUC). The AUC ranges in value from 0.5 (essentially a meaningless test) to 1 (a perfect test) and can be interpreted as the probability that a randomly chosen diseased subject has a higher test value than that of a randomly chosen healthy subject (given higher test value is indicative of the disease) (Hanley & McNeil, 1982). The AUC is, in most cases, calculated empirically by calculating the area under the sample ROC curve, and it is often of interest to present an interval estimate of the AUC rather than simply a point estimate. There exist a number of methods that have been proposed to compute an interval estimator for the AUC. Many of these have been derived based on the equivalence relation between the AUC and Wilcoxon’s rank-sum test statistic (Hollander et al., 2014) such as Bamber (1975), Hanley & McNeil (1982), DeLong et al. (1988), Mee (1990) and Newcombe (2006). Other proposals, such as Reiser & Guttman (1986) and Newcombe (2006), developed confidence interval (CI) estimators of the AUC by assuming a parametric distribution for the test scores and deriving the interval in that manner. Cortes & Mohri (2005) suggested a CI method using a combinatoric approach, and Obuchowski & Lieber (2002) introduced an exact method, which can be used when the estimated AUC is close to 1. These methods can be easily applied when the true disease status and the test values for all subjects in the sample are known. However, in practice, data may be incomplete. This often happens when the true disease status of subjects is unknown, which commonly occurs when a patient does not go through a so-called gold standard test that, in many cases, is invasive and/or expensive. Missing data can also occur on patients’ test scores for many different reasons including they simply did not have the test performed. When observations with missing values are removed from analysis (i.e. complete case analysis), the estimator is potentially subject to bias, and when bias is caused specifically by missing disease status, this is referred to as verification bias. Many corrected estimators of AUC have been proposed to adjust biases caused by missing values. Begg & Greenes (1983), Alonzo & Pepe (2005), He et al. (2009), along with others, came up with AUC estimators correcting verification bias directly (i.e. ‘direct approach’). In order to construct CIs based on those methods, bootstrap techniques are used. Alternatively, CI methods for complete data settings can be used after applying multiple imputation (MI) techniques to the incomplete data set (i.e. ‘MI approach’) as in Harel & Zhou (2006), which sought to correct the verification bias for measuring sensitivity and specificity, and Harel & Int Stat Rev. Author manuscript; available in PMC 2019 April 18. Cho et al. Page 3 Zhou (2007a). To address the problem of missing biomarker values, Long et al. (2011a) and Author ManuscriptAuthor Manuscript Author Manuscript Author Manuscript Author Long et al. (2011b) came up with a robust approach and an approach using non-parametric MI, respectively. All the estimators correcting for bias introduced here are based on the assumption that the data are missing at random (MAR) (Rubin, 1976) and exploit the covariates to predict the missing values. Although quite a few estimators have been proposed, relatively little work has been done reviewing the performance of the CI methods for the AUC in the presence of missing data. In this article, we compare the performance of several Wald-type interval estimators for the AUC in data sets where missing data exist. In our example, the incomplete data will be true disease statuses, and it will be assumed that the missingness mechanism is MAR. 2 Wald-type Confidence Interval Methods for the Area Under the Receiver Operating Characteristic Curve Consider a test such that higher test scores are associated with a higher probability that the subject is diseased and vice versa. By definition, the AUC (θ) is the integral of the TPR of the theoretical ROC curve by its false positive rate (FPR = 1 – TNR), that is, θ ∞ t d t where TPR(t) and FPR(t) are the TPR and FPR for a threshold = ∫ −∞ TPR ( ) FPR ( ), value t. As the AUC can be interpreted as the probability that a randomly chosen diseased subject has higher biomarker value than a randomly chosen healthy subject’s biomarker value (plus half of the probability of ties is added, if any), it can also be expressed as 1 θ = P[Y > X] + P[Y = X], where X and Y are the biomarkers of the randomly chosen non- 2 diseased and diseased subjects, respectively. So it is straightforward to have an unbiased estimator of the AUC as n n Y X 1 ∑ ∑ I y > x + I y = x i = 1 j = 1 i j 2 i j θ = , nYnX where xj and yi are the test scores for the j -th and i-th individual in the non-diseased and diseased groups, respectively, and j = 1, …, nX and i = …, nY.
Load more

Recommended publications
  • Spatial Duration Data for the Spatial Re- Gression Context
    Missing Data In Spatial Regression 1 Frederick J. Boehmke Emily U. Schilling University of Iowa Washington University Jude C. Hays University of Pittsburgh July 15, 2015 Paper prepared for presentation at the Society for Political Methodology Summer Conference, University of Rochester, July 23-25, 2015. 1Corresponding author: [email protected] or [email protected]. We are grateful for funding provided by the University of Iowa Department of Political Science. Com- ments from Michael Kellermann, Walter Mebane, and Vera Troeger greatly appreciated. Abstract We discuss problems with missing data in the context of spatial regression. Even when data are MCAR or MAR based solely on exogenous factors, estimates from a spatial re- gression with listwise deletion can be biased, unlike for standard estimators with inde- pendent observations. Further, common solutions for missing data such as multiple im- putation appear to exacerbate the problem rather than fix it. We propose a solution for addressing this problem by extending previous work including the EM approach devel- oped in Hays, Schilling and Boehmke (2015). Our EM approach iterates between imputing the unobserved values and estimating the spatial regression model using these imputed values. We explore the performance of these alternatives in the face of varying amounts of missing data via Monte Carlo simulation and compare it to the performance of spatial regression with listwise deletion and a benchmark model with no missing data. 1 Introduction Political science data are messy. Subsets of the data may be missing; survey respondents may not answer all of the questions, economic data may only be available for a subset of countries, and voting records are not always complete.
    [Show full text]
  • Missing Data Module 1: Introduction, Overview
    Missing Data Module 1: Introduction, overview Roderick Little and Trivellore Raghunathan Course Outline 9:00-10:00 Module 1 Introduction and overview 10:00-10:30 Module 2 Complete-case analysis, including weighting methods 10:30-10:45 Break 10:45-12:00 Module 3 Imputation, multiple imputation 1:30-2:00 Module 4 A little likelihood theory 2:00-3:00 Module 5 Computational methods/software 3:00-3:30 Module 6: Nonignorable models Module 1:Introduction and Overview Module 1: Introduction and Overview • Missing data defined, patterns and mechanisms • Analysis strategies – Properties of a good method – complete-case analysis – imputation, and multiple imputation – analysis of the incomplete data matrix – historical overview • Examples of missing data problems Module 1:Introduction and Overview Module 1: Introduction and Overview • Missing data defined, patterns and mechanisms • Analysis strategies – Properties of a good method – complete-case analysis – imputation, and multiple imputation – analysis of the incomplete data matrix – historical overview • Examples of missing data problems Module 1:Introduction and Overview Missing data defined variables If filled in, meaningful for cases analysis? • Always assume missingness hides a meaningful value for analysis • Examples: – Missing data from missed clinical visit( √) – No-show for a preventive intervention (?) – In a longitudinal study of blood pressure medications: • losses to follow-up ( √) • deaths ( x) Module 1:Introduction and Overview Patterns of Missing Data • Some methods work for a general
    [Show full text]
  • Coping with Missing Data in Randomized Controlled Trials
    EVALUATION TECHNICAL ASSISTANCE BRIEF for OAH & ACYF Teenage Pregnancy Prevention Grantees May 2013 • Brief 3 Coping with Missing Data in Randomized Controlled Trials issing data in randomized controlled trials (RCTs) can respondents3 if there are no systematic differences between Mlead to biased impact estimates and a loss of statistical respondents in the treatment group and respondents in the power. In this brief we describe strategies for coping with control group. However, if respondents in the treatment group missing data in RCTs.1 Specifically, we discuss strategies to: are systematically different from respondents in the control group, then the impact we calculate for respondents lacks ● Describe the extent and nature of missing data. We suggest causal validity. We call the difference between the calculated ways to provide a clear and accurate accounting of the extent impact for respondents and the true impact for respondents and nature of missing outcome data. This will increase the “causal validity bias.” This type of bias is the focus of the transparency and credibility of the study, help with interpret- evidence review of teen pregnancy prevention programs ing findings, and also help to select an approach for addressing (Mathematica Policy Research and Child Trends 2012). missing data in impact analyses. We suggest conducting descriptive analyses designed to assess ● Address missing data in impact analyses. Missing outcome the potential for both types of bias in an RCT. or baseline data pose mechanical and substantive problems when conducting impact analyses. The mechanical problem With regard to generalizability, we suggest conducting two is that mathematical operations cannot be performed on analyses.
    [Show full text]
  • Monte Carlo Likelihood Inference for Missing Data Models
    MONTE CARLO LIKELIHOOD INFERENCE FOR MISSING DATA MODELS By Yun Ju Sung and Charles J. Geyer University of Washington and University of Minnesota Abbreviated title: Monte Carlo Likelihood Asymptotics We describe a Monte Carlo method to approximate the maximum likeli- hood estimate (MLE), when there are missing data and the observed data likelihood is not available in closed form. This method uses simulated missing data that are independent and identically distributed and independent of the observed data. Our Monte Carlo approximation to the MLE is a consistent and asymptotically normal estimate of the minimizer ¤ of the Kullback- Leibler information, as both a Monte Carlo sample size and an observed data sample size go to in¯nity simultaneously. Plug-in estimates of the asymptotic variance are provided for constructing con¯dence regions for ¤. We give Logit-Normal generalized linear mixed model examples, calculated using an R package that we wrote. AMS 2000 subject classi¯cations. Primary 62F12; secondary 65C05. Key words and phrases. Asymptotic theory, Monte Carlo, maximum likelihood, generalized linear mixed model, empirical process, model misspeci¯cation 1 1. Introduction. Missing data (Little and Rubin, 2002) either arise naturally|data that might have been observed are missing|or are intentionally chosen|a model includes random variables that are not observable (called latent variables or random e®ects). A mixture of normals or a generalized linear mixed model (GLMM) is an example of the latter. In either case, a model is speci¯ed for the complete data (x; y), where x is missing and y is observed, by their joint den- sity f(x; y), also called the complete data likelihood (when considered as a function of ).
    [Show full text]
  • Missing Data
    4 Missing Data Paul D. Allison INTRODUCTION they have spent a great deal time, money and effort in collecting. And so, many methods for Missing data are ubiquitous in psychological ‘salvaging’ the cases with missing data have research. By missing data, I mean data that become popular. are missing for some (but not all) variables For a very long time, however, missing and for some (but not all) cases. If data are data could be described as the ‘dirty little missing on a variable for all cases, then that secret’ of statistics. Although nearly everyone variable is said to be latent or unobserved. had missing data and needed to deal with On the other hand, if data are missing on the problem in some way, there was almost all variables for some cases, we have what nothing in the textbook literature to pro- is known as unit non-response, as opposed vide either theoretical or practical guidance. to item non-response which is another name Although the reasons for this reticence are for the subject of this chapter. I will not deal unclear, I suspect it was because none of the with methods for latent variables or unit non- popular methods had any solid mathematical response here, although some of the methods foundation. As it turns out, all of the most we will consider can be adapted to those commonly used methods for handling missing situations. data have serious deficiencies. Why are missing data a problem? Because Fortunately, the situation has changed conventional statistical methods and software dramatically in recent years.
    [Show full text]
  • Auditor: an R Package for Model-Agnostic Visual Validation and Diagnostics
    auditor: an R Package for Model-Agnostic Visual Validation and Diagnostics APREPRINT Alicja Gosiewska Przemysław Biecek Faculty of Mathematics and Information Science Faculty of Mathematics and Information Science Warsaw University of Technology Warsaw University of Technology Poland Faculty of Mathematics, Informatics and Mechanics [email protected] University of Warsaw Poland [email protected] May 27, 2020 ABSTRACT Machine learning models have spread to almost every area of life. They are successfully applied in biology, medicine, finance, physics, and other fields. With modern software it is easy to train even a complex model that fits the training data and results in high accuracy on test set. The problem arises when models fail confronted with the real-world data. This paper describes methodology and tools for model-agnostic audit. Introduced tech- niques facilitate assessing and comparing the goodness of fit and performance of models. In addition, they may be used for analysis of the similarity of residuals and for identification of outliers and influential observations. The examination is carried out by diagnostic scores and visual verification. Presented methods were implemented in the auditor package for R. Due to flexible and con- sistent grammar, it is simple to validate models of any classes. K eywords machine learning, R, diagnostics, visualization, modeling 1 Introduction Predictive modeling is a process that uses mathematical and computational methods to forecast outcomes. arXiv:1809.07763v4 [stat.CO] 26 May 2020 Lots of algorithms in this area have been developed and are still being develop. Therefore, there are countless possible models to choose from and a lot of ways to train a new new complex model.
    [Show full text]
  • Parameter Estimation in Stochastic Volatility Models with Missing Data Using Particle Methods and the Em Algorithm
    PARAMETER ESTIMATION IN STOCHASTIC VOLATILITY MODELS WITH MISSING DATA USING PARTICLE METHODS AND THE EM ALGORITHM by Jeongeun Kim BS, Seoul National University, 1998 MS, Seoul National University, 2000 Submitted to the Graduate Faculty of the Department of Statistics in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 2005 UNIVERSITY OF PITTSBURGH DEPARTMENT OF STATISTICS This dissertation was presented by Jeongeun Kim It was defended on April 22, 2005 and approved by David S. Stoffer, Professor Ori Rosen, Professor Wesley Thompson, Professor Anthony Brockwell, Professor Dissertation Director: David S. Stoffer, Professor ii Copyright c by Jeongeun Kim 2005 iii PARAMETER ESTIMATION IN STOCHASTIC VOLATILITY MODELS WITH MISSING DATA USING PARTICLE METHODS AND THE EM ALGORITHM Jeongeun Kim, PhD University of Pittsburgh, 2005 The main concern of financial time series analysis is how to forecast future values of financial variables, based on all available information. One of the special features of financial variables, such as stock prices and exchange rates, is that they show changes in volatility, or variance, over time. Several statistical models have been suggested to explain volatility in data, and among them Stochastic Volatility models or SV models have been commonly and successfully used. Another feature of financial variables I want to consider is the existence of several missing data. For example, there is no stock price data available for regular holidays, such as Christmas, Thanksgiving, and so on. Furthermore, even though the chance is small, stretches of data may not available for many reasons. I believe that if this feature is brought into the model, it will produce more precise results.
    [Show full text]
  • ROC Curve Analysis and Medical Decision Making
    ROC curve analysis and medical decision making: what’s the evidence that matters for evidence-based diagnosis ? Piergiorgio Duca Biometria e Statistica Medica – Dipartimento di Scienze Cliniche Luigi Sacco – Università degli Studi – Via GB Grassi 74 – 20157 MILANO (ITALY) [email protected] 1) The ROC (Receiver Operating Characteristic) curve and the Area Under the Curve (AUC) The ROC curve is the statistical tool used to analyse the accuracy of a diagnostic test with multiple cut-off points. The test could be based on a continuous diagnostic indicant, such as a serum enzyme level, or just on an ordinal one, such as a classification based on radiological imaging. The ROC curve is based on the probability density distributions, in actually diseased and non diseased patients, of the diagnostician’s confidence in a positive diagnosis, and upon a set of cut-off points to separate “positive” and “negative” test results (Egan, 1968; Bamber, 1975; Swets, Pickett, 1982; Metz, 1986; Zhou et al, 2002). The Sensitivity (SE) – the proportion of diseased turned out to be test positive – and the Specificity (SP) – the proportion of non diseased turned out to be test negative – will depend on the particular confidence threshold the observer applies to partition the continuously distributed perceptions of evidence into positive and negative test results. The ROC curve is the plot of all the pairs of True Positive Rates (TPR = SE), as ordinate, and False Positive Rates (FPR = (1 – SP)), as abscissa, related to all the possible cut-off points. An ROC curve represents all of the compromises between SE and SP can be achieved, changing the confidence threshold.
    [Show full text]
  • Estimating the Variance of a Propensity Score Matching Estimator for the Average Treatment Effect
    Observational Studies 4 (2018) 71-96 Submitted 5/15; Published 3/18 Estimating the variance of a propensity score matching estimator for the average treatment effect Ronnie Pingel [email protected] Department of Statistics Uppsala University Uppsala, Sweden Abstract This study considers variance estimation when estimating the asymptotic variance of a propensity score matching estimator for the average treatment effect. We investigate the role of smoothing parameters in a variance estimator based on matching. We also study the properties of estimators using local linear estimation. Simulations demonstrate that large gains can be made in terms of mean squared error, bias and coverage rate by prop- erly selecting smoothing parameters. Alternatively, a residual-based local linear estimator could be used as an estimator of the asymptotic variance. The variance estimators are implemented in analysis to evaluate the effect of right heart catheterisation. Keywords: Average Causal Effect, Causal Inference, Kernel estimator 1. Introduction Matching estimators belong to a class of estimators of average treatment effects (ATEs) in observational studies that seek to balance the distributions of covariates in a treatment and control group (Stuart, 2010). In this study we consider one frequently used matching method, the simple nearest-neighbour matching with replacement (Rubin, 1973; Abadie and Imbens, 2006). Rosenbaum and Rubin (1983) show that instead of matching on covariates directly to remove confounding, it is sufficient to match on the propensity score. In observational studies the propensity score must almost always be estimated. An important contribution is therefore Abadie and Imbens (2016), who derive the large sample distribution of a nearest- neighbour propensity score matching estimator when using the estimated propensity score.
    [Show full text]
  • Statistical Inference in Missing Data by MCMC And
    I Takahashi, M 2017 Statistical Inference in Missing Data by MCMC and CODATA '$7$6&,(1&( S Non-MCMC Multiple Imputation Algorithms: Assessing the Effects of U -2851$/ Between-Imputation Iterations. Data Science Journal, 16: 37, pp. 1–17, DOI: https://doi.org/10.5334/dsj-2017-037 RESEARCH PAPER Statistical Inference in Missing Data by MCMC and Non-MCMC Multiple Imputation Algorithms: Assessing the Effects of Between-Imputation Iterations Masayoshi Takahashi IR Office, Tokyo University of Foreign Studies, Tokyo, JP [email protected] Incomplete data are ubiquitous in social sciences; as a consequence, available data are inefficient (ineffective) and often biased. In the literature, multiple imputation is known to be the standard method to handle missing data. While the theory of multiple imputation has been known for decades, the implementation is difficult due to the complicated nature of random draws from the posterior distribution. Thus, there are several computational algorithms in software: Data Augmentation (DA), Fully Conditional Specification (FCS), and Expectation-Maximization with Bootstrapping (EMB). Although the literature is full of comparisons between joint modeling (DA, EMB) and conditional modeling (FCS), little is known about the relative superiority between the MCMC algorithms (DA, FCS) and the non-MCMC algorithm (EMB), where MCMC stands for Markov chain Monte Carlo. Based on simulation experiments, the current study contends that EMB is a confidence proper (confidence-supporting) multiple imputation algorithm without between-imputation iterations; thus, EMB is more user-friendly than DA and FCS. Keywords: MCMC; Markov chain Monte Carlo; Incomplete data; Nonresponse; Joint modeling; Conditional modeling 1 Introduction Generally, it is quite difficult to obtain complete data in social surveys (King et al.
    [Show full text]
  • The Effects of Missing Data
    1.2 REPLACING MISSING DATA FOR ENSEMBLE SYSTEMS Tyler McCandless*, Sue Ellen Haupt, George Young 1 The Pennsylvania State University A major problem for statistical predictive system 1. INTRODUCTION developers is obtaining enough input data from Numerical Weather Prediction (NWP) models for Missing data presents a problem in many fields, developing, or training, new post-processing methods. including meteorology. The data can be missing at Such post-processing, or calibrating, of NWP forecasts random, in recurring patterns, or in large sections. has been shown to decrease errors in forecasting since Incomplete datasets can lead to misleading conclusions, the introduction of Model Output Statistics (MOS) as demonstrated by Kidson and Trenberth (1988), who (Glahn and Lowry 1972). For these statistical post- assessed the impact of missing data on general processing techniques to make these improvements in circulation statistics by systematically decreasing the forecast skill, however, requires training them on amount of available training data. They determined that archived forecasts and validation data. Enough data the ratio of the Root Mean Square Error (RMSE) in the must be available for the resulting technique to be monthly mean to the daily standard deviation was two to stable. Thus, imputing any missing data will provide three times higher when the missing data was spaced larger training and validation datasets for this process. randomly compared to spaced equally, and RMSE The advent of ensembles of forecasts has led to increased by up to a factor of two when the missing data new methods of post-processing. Post-processing occurred in one block. Therefore, the spacing of the ensemble temperature data can produce an even more missing data can have an impact on statistical analyses.
    [Show full text]
  • An Introduction to Logistic Regression: from Basic Concepts to Interpretation with Particular Attention to Nursing Domain
    J Korean Acad Nurs Vol.43 No.2, 154 -164 J Korean Acad Nurs Vol.43 No.2 April 2013 http://dx.doi.org/10.4040/jkan.2013.43.2.154 An Introduction to Logistic Regression: From Basic Concepts to Interpretation with Particular Attention to Nursing Domain Park, Hyeoun-Ae College of Nursing and System Biomedical Informatics National Core Research Center, Seoul National University, Seoul, Korea Purpose: The purpose of this article is twofold: 1) introducing logistic regression (LR), a multivariable method for modeling the relationship between multiple independent variables and a categorical dependent variable, and 2) examining use and reporting of LR in the nursing literature. Methods: Text books on LR and research articles employing LR as main statistical analysis were reviewed. Twenty-three articles published between 2010 and 2011 in the Journal of Korean Academy of Nursing were analyzed for proper use and reporting of LR models. Results: Logistic regression from basic concepts such as odds, odds ratio, logit transformation and logistic curve, assumption, fitting, reporting and interpreting to cautions were presented. Substantial short- comings were found in both use of LR and reporting of results. For many studies, sample size was not sufficiently large to call into question the accuracy of the regression model. Additionally, only one study reported validation analysis. Conclusion: Nurs- ing researchers need to pay greater attention to guidelines concerning the use and reporting of LR models. Key words: Logit function, Maximum likelihood estimation, Odds, Odds ratio, Wald test INTRODUCTION The model serves two purposes: (1) it can predict the value of the depen- dent variable for new values of the independent variables, and (2) it can Multivariable methods of statistical analysis commonly appear in help describe the relative contribution of each independent variable to general health science literature (Bagley, White, & Golomb, 2001).
    [Show full text]