DOCSLIB.ORG

Error Bars in Experimental Biology • Cumming Et Al

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Error Bars in Experimental Biology • Cumming Et Al JCB: FEATURE Error bars in experimental biology Geoff Cumming,1 Fiona Fidler,1 and David L. Vaux2 1School of Psychological Science and 2Department of Biochemistry, La Trobe University, Melbourne, Victoria, Australia 3086 Error bars commonly appear in fi gures error bars encompass the lowest and high- in publications, but experimental est values. SD is calculated by the formula biologists are often unsure how they ∑()XM− 2 should be used and interpreted. In this SD = − article we illustrate some basic features n 1 of error bars and explain how they can where X refers to the individual data help communicate data and assist points, M is the mean, and Σ (sigma) correct interpretation. Error bars may means add to fi nd the sum, for all the n show confi dence intervals, standard data points. SD is, roughly, the average or errors, standard deviations, or other typical difference between the data points Downloaded from quantities. Different types of error bars and their mean, M. About two thirds of the data points will lie within the region give quite different information, and so of the data %95ف of mean ± 1 SD, and fi gure legends must make clear what points will be within 2 SD of the mean. error bars represent. We suggest eight simple rules to assist with effective use Figure 1. Descriptive error bars. Means with er- www.jcb.org It is highly desirable to ror bars for three cases: n = 3, n = 10, and n = and interpretation of error bars. 30. The small black dots are data points, and the use larger n, to achieve column denotes the data mean M. The bars on the left of each column show range, and the bars What are error bars for? on May 2, 2007 narrower inferential on the right show standard deviation (SD). M and Journals that publish science—knowledge error bars and more SD are the same for every case, but notice how gained through repeated observation or much the range increases with n. Note also that precise estimates of true although the range error bars encompass all of experiment—don’t just present new the experimental results, they do not necessarily conclusions, they also present evidence so population values. cover all the results that could possibly occur. SD readers can verify that the authors’ error bars include about two thirds of the sample, and 2 x SD error bars would encompass roughly reasoning is correct. Figures with error bars 95% of the sample. can, if used properly (1–6), give information Descriptive error bars can also be describing the data (descriptive statistics), used to see whether a single result fi ts within the normal range. For example, if SD of your results will tend to more and THE JOURNAL OF CELL BIOLOGY or information about what conclusions, or inferences, are justifi ed (inferential you wished to see if a red blood cell count more closely approximate the true stan- σ statistics). These two basic categories of was normal, you could see whether it was dard deviation ( ) that you would get if error bars are depicted in exactly the same within 2 SD of the mean of the population the experiment was performed an infi nite way, but are actually fundamentally as a whole. Less than 5% of all red blood number of times, or on the whole popula- different. Our aim is to illustrate basic pro- cell counts are more than 2 SD from the tion. However, the SD of the experimental σ perties of fi gures with any of the common mean, so if the count in question is more results will approximate to , whether n is error bars, as summarized in Table I, and to than 2 SD from the mean, you might con- large or small. Like M, SD does not change explain how they should be used. sider it to be abnormal. systematically as n changes, and we can As you increase the size of your use SD as our best estimate of the un- σ What do error bars tell you? sample, or repeat the experiment more known , whatever the value of n. Descriptive error bars. Range times, the mean of your results (M) will Inferential error bars. In experi- and standard deviation (SD) are used for tend to get closer and closer to the true mental biology it is more common to be descriptive error bars because they show mean, or the mean of the whole popula- interested in comparing samples from two μ how the data are spread (Fig. 1). Range tion, . We can use M as our best estimate groups, to see if they are different. For of the unknown μ. Similarly, as you repeat example, you might be comparing wild- David L. Vaux: [email protected] an experiment more and more times, the type mice with mutant mice, or drug with © The Rockefeller University Press $15.00 The Journal of Cell Biology, Vol. 177, No. 1, April 9, 2007 7–11 http://www.jcb.org/cgi/doi/10.1083/jcb.200611141 JCB 7 (Fig. 2). The interval defi nes the values that are most plausible for μ. Because error bars can be descriptive or inferential, and could be any of the bars listed in Table I or even something else, they are meaningless, or misleading, if the fi gure legend does not state what kind they are. This leads to the fi rst rule. Rule 1: when showing error bars, always describe in the fi gure legends what they are. Statistical signifi cance tests and P values If you carry out a statistical signifi cance Figure 3. Inappropriate use of error bars. En- test, the result is a P value, where P is the zyme activity for MEFs showing mean + SD probability that, if there really is no differ- from duplicate samples from one of three repre- sentative experiments. Values for wild-type vs. ence, you would get, by chance, a differ- −/− MEFs were signifi cant for enzyme activity ence as large as the one you observed, or at the 3-h timepoint (P < 0.0005). This fi gure and Figure 2. Confi dence intervals. Means and even larger. Other things (e.g., sample its legend are typical, but illustrate inappropriate 95% CIs for 20 independent sets of results, each and misleading use of statistics because n = 1. = μ = of size n 10, from a population with mean size, variation) being equal, a larger differ- The very low variation of the duplicate samples 40 (marked by the dotted line). In the long run we ence in results gives a lower P value, implies consistency of pipetting, but says nothing μ Downloaded from expect 95% of such CIs to capture ; here 18 do about whether the differences between the wild- so (large black dots) and 2 do not (open circles). which makes you suspect there is a true − − < type and / MEFs are reproducible. In this Successive CIs vary considerably, not only in po- difference. By convention, if P 0.05 you case, the means and errors of the three experi- μ sition relative to , but also in length. The varia- say the result is statistically signifi cant, ments should have been shown. tion from CI to CI would be less for larger sets of < results, for example n = 30 or more, but varia- and if P 0.01 you say the result is highly tion in position and in CI length would be even signifi cant and you can be more confi dent repeated your experiment often enough to = greater for smaller samples, for example n 3. you have found a true effect. As always reveal it. It is a common and serious error www.jcb.org with statistical inference, you may be to conclude “no effect exists” just because placebo, or experimental results with wrong! Perhaps there really is no effect, P is greater than 0.05. If you measured the controls. To make inferences from the data and you had the bad luck to get one of the heights of three male and three female (i.e., to make a judgment whether the 5% (if P < 0.05) or 1% (if P < 0.01) of Biddelonian basketball players, and did on May 2, 2007 groups are signifi cantly different, or sets of results that suggests a difference not see a signifi cant difference, you could whether the differences might just be due where there is none. Of course, even if re- not conclude that sex has no relationship to random fl uctuation or chance), a differ- sults are statistically highly signifi cant, it with height, as a larger sample size might ent type of error bar can be used. These does not mean they are necessarily bio- reveal one. A big advantage of inferential are standard error (SE) bars and confi - logically important. It is also essential to error bars is that their length gives a dence intervals (CIs). The mean of the note that if P > 0.05, and you therefore graphic signal of how much uncertainty data, M, with SE or CI error bars, gives an cannot conclude there is a statistically sig- there is in the data: The true value of the indication of the region where you can ex- nifi cant effect, you may not conclude that mean μ we are estimating could plausibly pect the mean of the whole possible set of the effect is zero. There may be a real ef- be anywhere in the 95% CI. Wide inferen- results, or the whole population, μ, to lie fect, but it is small, or you may not have tial bars indicate large error; short inferen- tial bars indicate high precision.
Load more

Recommended publications
  • Serrbar — Graph Standard Error Bar Chart
    Title stata.com serrbar — Graph standard error bar chart Description Quick start Menu Syntax Options Remarks and examples Acknowledgment Also see Description serrbar is typically used with a dataset containing means, standard deviations or standard errors, and an xvar. serrbar uses these data to create a standard error bar chart. The means are plotted against xvar, and error bars around the means have a width determined by the standard deviation or standard error. While it is most common to use serrbar with this type of data, serrbar may also be used to create a scatterplot with error bars for other types of data. Quick start Plot of y versus x with error bars representing y ± s serrbar y s x As above, but with error bars for y ± 2 × s serrbar y s x, scale(2) Menu Statistics > Other > Quality control > Standard error bar chart 1 2 serrbar — Graph standard error bar chart Syntax serrbar mvar svar xvar if in , options options Description Main scale(#) scale length of graph bars; default is scale(1) Error bars rcap options affect rendition of capped spikes Plotted points mvopts(scatter options) affect rendition of plotted points Add plots addplot(plot) add other plots to generated graph Y axis, X axis, Titles, Legend, Overall twoway options any options other than by() documented in[ G-3] twoway options Options Main £ # mvar + £scale( ) controls the length of the bars. The upper and lower limits of the bars will be scale() × svar and mvar − scale() × svar. The default is scale(1). Error bars rcap options£ G-2 £ affect the rendition of the plotted error bars (the capped spikes).
    [Show full text]
  • Error Analysis Interludes
    Princeton University Physics 103/105 Lab Physics Department Reference Packet ERROR ANALYSIS INTERLUDES Fall, 2003 NOTE TO STUDENTS: This material closely parallels that found in John R. Taylor's An Introduction to Error Analysis, 2nd Edition (University Science Books, Sausalito, California; 1997). Taylor's text is a standard on the subject, and is well worth acquiring for your long-term use. The Physics Department has in the past also provided Errors Without Tears, by Professor Daniel Marlow, as a handout in Physics 103/105 lab. It covers much the same material as this packet; copies are available on request. Material from Matt Trawick (Physics 103, Fall 2002) Format revised Summer 2003 (S. Smith) 2 "Table of Contents" for Error Analysis Interludes ERROR ANALYSIS INTERLUDE #1 PAGE 5 What is Uncertainty, and Who Cares? How to Report Uncertainties Rule for confidence level of errors Absolute and Relative Uncertainties Rule for relative and absolute uncertainties Significant Figures Rule for significant figures Different Kinds of Error (precision vs accuracy; random vs systematic) Vocabulary Review ERROR ANALYSIS INTERLUDE #2 PAGE 10 What is Error Propagation? Unit Conversion Rule for converting units Easy Propagation of Single Errors Rule for easy propagation of single errors Rule for multiplying by a constant General rule for single small errors ERROR ANALYSIS INTERLUDE #3 PAGE 14 Adding or Subtracting Two Quantities Rule for adding and subtracting quantities with independent errors Rule for adding and subtracting quantities with dependent
    [Show full text]
  • Descriptive Statistics and Graphing
    Draft date: 2019.3.29 Descriptive Statistics and Graphing <Material provided by> AMED “International Journals Project” Unauthorized reproduction prohibited. 1 Contents Introduction Types of Graph and How to Read Them Bar Charts and Error Bars Indicating What the Error Bar Shows Box-and-whisker Plots with a Variety of Information Histograms Scatter Diagrams 2 Introduction In this module, you will learn about ways of using graphs to visually present data. Presenting data in graphs enables you to communicate the point of a study or information that would be hard to understand if summarized using numerical values alone in a way that the reader can understand more easily. Learning Objectives Name each graph and explain its significance. Explain the meaning of error bars and how to use them. Types of Graph and How to Read Them Bar Charts and Error Bars The type of graph that most of you almost certainly have seen at least once is the bar chart. This graph is often used when comparing numerical values between groups. The most commonly used bar chart is a graph in which the height of the bar represents the mean. From the graph below, we can see that mean blood pressure in Group A is higher than in Group B. Mean blood pressure blood Mean Group A Group B However, this graph does not tell us whether the difference in mean blood pressure between Group B and Group A is statistically significant. It is easier to find statistically significant differences when the difference in data that you wish to detect between groups (such as the 3 differences in mean blood pressure between groups) is greater than the variation in the data.
    [Show full text]
  • Curve Fitting Curve Fitting
    Chapter 16: Curve Fitting Curve Fitting Before You Begin Selecting the Active Data Plot When performing linear or nonlinear fitting when the graph window is active, you must make the desired data plot the active data plot. To make a data plot active, select the data plot from the data list at the bottom of the Data menu. The data list includes all the data plots in the active layer. The currently active data plot is checked. Selecting a Range of a Data Plot To select a range of a data plot for fitting, click the Data Selector tool on the Tools toolbar. Data markers display at both ends of the active data plot. Additionally, the Data Display tool opens if it is not already open. To mark the data segment of interest, click and drag the markers with the mouse. You can also use the left and right arrow keys to select a marker. The CTRL + left or right arrow keys move the selected marker to the next data point. Holding both the SHIFT and CTRL keys while depressing the left or right arrow keys moves the data markers in increments of five along the data plot. (Note: If your X data is not sorted, you may need to sort the data before selecting a range. To do this, activate the worksheet and select Analysis:Sort Worksheet:Ascending.) As with the Data Reader tool, you can press the spacebar to increase the cross hair size. After you have defined the range of interest, press ESC or click the Pointer button on the Tools toolbar.
    [Show full text]
  • A Cautionary Note on the Use of Error Bars
    1SEPTEMBER 2005 NOTES AND CORRESPONDENCE 3699 A Cautionary Note on the Use of Error Bars JOHN R. LANZANTE NOAA/Geophysical Fluid Dynamics Laboratory, Princeton University, Princeton, New Jersey (Manuscript received 18 May 2004, in final form 3 March 2005) ABSTRACT Climate studies often involve comparisons between estimates of some parameter derived from different observed and/or model-generated datasets. It is common practice to present estimates of two or more statistical quantities with error bars about each representing a confidence interval. If the error bars do not overlap, it is presumed that there is a statistically significant difference between them. In general, such a procedure is not valid and usually results in declaring statistical significance too infrequently. Simple examples that demonstrate the nature of this pitfall, along with some formulations, are presented. It is recommended that practitioners use standard hypothesis testing techniques that have been derived from statistical theory rather than the ad hoc approach involving error bars. 1. Introduction et al. 2001). Instances of inappropriate use of error bars were found in both the Journal of Climate and the The statistical analysis of climate data typically in- TAR.1 Given that these are two of the most respected volves the estimation of quantities such as the mean publications in the field of climate, some clarification value or the regression coefficient from a trend analy- on the use of error bars seems warranted. sis. Such estimates are viewed as incomplete without Before beginning discussion of the problem, it is the inclusion of an uncertainty estimate that can then worth reviewing some of the types of error bars that are be used to test a hypothesis.
    [Show full text]
  • The Ways of Our Errors
    The Ways of Our Errors Optimal Data Analysis for Beginners and Experts c Keith Horne Universit y of St.Andrews DRAFT June 5, 2009 Contents I Optimal Data Analysis 1 1 Probability Concepts 2 1.1 Fuzzy Numbers and Dancing Data Points . 2 1.2 Systematic and Statistical Errors { Bias and Variance . 3 1.3 Probability Maps . 3 1.4 Mean, Variance, and Standard Deviation . 5 1.5 Median and Quantiles . 6 1.6 Fuzzy Algebra . 7 1.7 Why Gaussians are Special . 8 1.8 Why χ2 is Special . 8 2 A Menagerie of Probability Maps 10 2.1 Boxcar or Uniform (ranu.for) . 10 2.2 Gaussian or Normal (rang.for) . 11 2.3 Lorentzian or Cauchy (ranl.for) . 13 2.4 Exponential (rane.for) . 14 2.5 Power-Law (ranpl.for) . 15 2.6 Schechter Distribution . 17 2.7 Chi-Square (ranchi2.for) . 18 2.8 Poisson . 18 2.9 2-Dimensional Gaussian . 19 3 Optimal Statistics 20 3.1 Optimal Averaging . 20 3.1.1 weighted average . 22 3.1.2 inverse-variance weights . 22 3.1.3 optavg.for . 22 3.2 Optimal Scaling . 23 3.2.1 The Golden Rule of Data Analysis . 25 3.2.2 optscl.for . 25 3.3 Summary . 26 3.4 Problems . 26 i ii The Ways of Our Errors c Keith Horne DRAFT June 5, 2009 4 Straight Line Fit 28 4.1 1 data point { degenerate parameters . 28 4.2 2 data points { correlated vs orthogonal parameters . 29 4.3 N data points . 30 4.4 fitline.for .
    [Show full text]
  • Data and Error Analysis in Science a Beginner’S Guide
    2015 Data and Error Analysis in Science A Beginner’s Guide Paul Strode, PhD Fairview High School Boulder, Colorado Ann Brokaw Rocky River High School Rocky River, Ohio Mathematics and Statistics in Science Pg. 1 August 2015 Mathematics and Statistics in Science Part 1: Statistical Symbols and Basic Equations Part 2: Descriptive Statistics Used in Science Measures of Average: Mean, Median, and Mode Mean Median Mode When to Use Which One Measures of Variability: Range, Standard Deviation, and Variance Range Standard Deviation and Variance Understanding Degrees of Freedom Measures of Confidence: Standard Error of the Mean and 95% Confidence Interval About This Guide Many state science standards encourage the use of mathematics and statistics in science education, including the newly designed AP and IB sciences courses, the Next Generation Science Standards, and the Common Core. Two high school biology teachers, Paul Strode and Ann Brokaw, conceived this guide to give first-year secondary science students a general foundation for performing the data, error, and uncertainty analysis they will need in their science education in high school and beyond. This brief guide is not meant to be a textbook of statistics. However, this guide will function as a starting point for the inferential statistical tests that students may learn as they move into more advanced science courses like biology, chemistry, physics, and earth and space science. The guide is organized in two parts: Part 1 is a “cheat sheet” of most of the statistical symbols and equations students will need to perform descriptive statistics on the data they generate in their lab activities and the inquiry based investigations they perform.
    [Show full text]
  • Data Analysis and Fitting for Physicists
    Data Analysis and Fitting for Physicists Peter Young (Dated: September 17, 2012) Contents I. Introduction 2 II. Averages and error bars 2 A. Basic Analysis 2 B. Advanced Analysis 8 1. Traditional method 9 2. Jackknife 11 3. Bootstrap 14 III. Fitting data to a model 17 A. Fitting to a straight line 19 B. Fitting to a polynomial 20 C. Fitting to a non-linear model 24 D. Confidence limits 25 E. Confidence limits by resampling the data 28 A. Central Limit Theorem 30 B. The number of degrees of freedom 34 C. The χ2 distribution and the goodness of fit parameter Q 35 D. Asymptotic standard error and how to get correct error bars from gnuplot 36 E. Derivation of the expression for the distribution of fitted parameters from simulated datasets 38 Acknowledgments 39 References 40 2 I. INTRODUCTION These notes describe how to average and fit numerical data that you have obtained, presumably by some simulation. Typically you will generate a set of values x , y , , i = 1, N, where N is the number of i i ··· ··· measurements. The first thing you will want to do is to estimate various average values, and determine error bars on those estimates. This is straightforward if one wants to compute a single average, e.g. x , but not quite so easy for more complicated averages such as fluctuations in a h i quantity, x2 x 2, or combinations of measured values such as y / x 2. Averaging of data will h i−h i h i h i be discussed in Sec.
    [Show full text]
  • Error Bars Μ Σ 10 Sample Means with 95% CI 9 8 7
    THIS MONTH abPopulation distribution 100 POINTS OF SIGNIFICANCE 50 s.e.m. 40 30 95% CI n 20 Error bars µ σ 10 Sample means with 95% CI 9 8 7 The meaning of error bars is often misinterpreted, Sample size, 6 5 as is the statistical significance of their overlap. 4 3 Last month in Points of Significance, we showed how samples are 0 σ 2σ used to estimate population statistics. We emphasized that, because Error bar size of chance, our estimates had an uncertainty. This month we focus on Figure 2 | The size and position of confidence intervals depend on the how uncertainty is represented in scientific publications and reveal sample. On average, CI% of intervals are expected to span the mean—about several ways in which it is frequently misinterpreted. 19 in 20 times for 95% CI. (a) Means and 95% CIs of 20 samples (n = 10) drawn from a normal population with mean m and s.d. σ. By chance, two of The uncertainty in estimates is customarily represented using the intervals (red) do not capture the mean. (b) Relationship between s.e.m. error bars. Although most researchers have seen and used error and 95% CI error bars with increasing n. bars, misconceptions persist about how error bars relate to statisti- cal significance. When asked to estimate the required separation does not ensure significance, nor does overlap rule it out—it depends between two points with error bars for a difference at significance on the type of bar. Chances are you were surprised to learn this unin- P = 0.05, only 22% of respondents were within a factor of 2 (ref.
    [Show full text]
  • Principles of Effective Data Visualization
    ll OPEN ACCESS Perspective Principles of Effective Data Visualization Stephen R. Midway1,* 1Department of Oceanography and Coastal Sciences, Louisiana State University, Baton Rouge, LA 70803, USA *Correspondence: [email protected] https://doi.org/10.1016/j.patter.2020.100141 THE BIGGER PICTURE Visuals are an increasingly important form of science communication, yet many sci- entists are not well trained in design principles for effective messaging. Despite challenges, many visuals can be improved by taking some simple steps before, during, and after their creation. This article presents some sequential principles that are designed to improve visual messages created by scientists. Mainstream: Data science output is well understood and (nearly) universally adopted SUMMARY We live in a contemporary society surrounded by visuals, which, along with software options and electronic distribution, has created an increased importance on effective scientific visuals. Unfortunately, across scien- tific disciplines, many figures incorrectly present information or, when not incorrect, still use suboptimal data visualization practices. Presented here are ten principles that serve as guidance for authors who seek to improve their visual message. Some principles are less technical, such as determining the message before starting the visual, while other principles are more technical, such as how different color combinations imply different information. Because figure making is often not formally taught and figure standards are not readily enforced in science, it is incumbent upon scientists to be aware of best practices in order to most effectively tell the story of their data. INTRODUCTION menu of visualization options can sometimes lead to poor fits be- tween information and its presentation.
    [Show full text]
  • An Introduction to Experimental Uncertainties and Error Analysis
    An Introduction to Experimental Uncertainties and Error Analysis for Physics 28 Harvey Mudd College T. W. Lynn TABLE OF CONTENTS No Information without Uncertainty Estimation! …………………….…...…3 What Is an Error Bar? ……………………………………………….….…....4 Random Errors, Systematic Errors, and Mistakes ………………….…….....5 How to Estimate Error Bars in Data ………………………………………..7 Sample Mean, Standard Deviation, and Standard Error …………………....9 How and When to Throw Out Data …………………………………….…12 Combining Unrelated Sources of Error …………………………………...13 Error Propagation in Calculations: Functions of a Single Measured Quantity ………………………...15 Error Propagation in Calculations: Functions of Several Measured Quantities ………………………..17 A Reality Check for Error Propagation: Fractional Uncertainty ………….18 Describing a Data Set with a Function: Graphing and Chi-Square Fitting ………………………………..19 Concluding Remarks ……………………………………………………...22 2 No Information without Uncertainty Estimation! LAX is about 59 minutes from Harvey Mudd by car. You can learn this from the driving directions on Google Maps, and it’s a useful piece of information if you are checking out possible travel bargains. But what if you already have reservations out of LAX and need to know when to leave campus for the airport? Then you’d better know that the drive can easily take as little as 45 minutes or as much as an hour and a half, depending on factors you can’t possibly determine in advance – like an accident in which a truckload of lemons is accidentally scattered across the 105 Freeway. You will wish you had been told (preferably before booking your tickets) that, while it may take you an hour and fifteen minutes in reasonable traffic, the westbound traffic is never reasonable between seven and ten o’clock on a weekday morning.
    [Show full text]
  • Standard Deviation, Standard Error and Confidence Intervals As Error Bars
    “Inference by Eye“ Standard deviation, standard error and confidence intervals as error bars Gabriela Czanner PhD CStat Department of Biostatistics Department of Eye and Vision Science 10 October 2012 MERSEY POSTGRADUATE TRAINING PROGRAMME Workshop Series: Basic Statistics for Eye Researchers and Clinicians Inference by eye “Inference by eye is the interpretation of graphically presented data” - Cumming, Finch 2005 Two goals to draw a picture • To enhance description of data • To visualize results of statistical inference On first seeing the figure on left, what questions should spring to our mind? • What is the centre? • What are the error bars i.e. the antenas in the picture? • 1 Standard deviation, 2 x Standard deviation (or 1.96), • Standard errors, Confidence interval (2 x Standard error), • Range i.e. min and max Outline • Fundamental statistical concepts • Mean • Standard deviation • Normal distribution • Standard error • Confidence interval for the mean • What error bars should we use? • References Fundamental statistical concepts We aim to learn about population. • Population is theoretical concept used to describe the entire group. • Characteristics of a population can be described by quantities called parameters. We choose sample of subjects and collect data on the sample. • Data are used to provide estimates of population parameters. Inference about populations of like • We describe the sample using graphs and individuals (or eyes) numerical summaries. Called descriptive statistics • We use data to make inference about the parameters of the population. Called statistical inference. Example – population and parameter of interest We want to investigate the mean value of mfERG of patients with diabetic maculopathy without CSMO who are 25 to 75 years old.
    [Show full text]