Introductory sta s cs for biological engineers
Module 2, Lecture 7 20.109 Spring 2014 Agi Stachowiak, Shannon Hughes, and Leona Samson Lecture by Zac Nagel, 4/8/14 What is an experiment?
• A test of a hypothesis • A comparison of two condi ons • Seeing what happens when we change a variable • Asking a ques on about the world Typical Experimental Data:
80
75
70
65 Height (inches)
60
Male Female Did my experiment tell me anything?
• Did I get an answer? • How sure am I of the answer? • Can we all agree on how sure of the answer we are? Experimental data speak with limited authority Is this the cure for cancer? Yes
h p://jimbuchan.com/ Experimental data speak with limited authority Is this the cure for cancer? Eh…
h p://tvbythenumbers.zap2it.com/ All Science is an approxima on
• Measure twice, cut once
h p://www.harborfreight.com/
h p://www.neontommy.com/ All Science is an approxima on
• Measure even more than twice? Why? • à We never have all the data! Very limited radar, satelite and blackbox pinger data had been used (as of the date of this lecture) together with mathema cal modeling to obtain a sort of confidence interval delinea ng the most likely loca on of Malasia Airlines flight 370. h p://www.cbsnews.com/news/malaysia-airlines-flight-370-no-new-black-box-like-sounds- heard/ All Science is an approxima on
• Measure even more than twice? Why? • à We almost never have all the data!
Nate Silver has used sta s cs to predict the most likely outcome in elec ons with great success; at le , his last predic on before the 2012 presiden al elec on. He happened to get 50/50 this me: red states to Mi Romney, blue states to Barack Obama. Note the predic ons amounted to probabili es based on a sample of h p://www.laprogressive.com/elec on-2012- polling data. watch/ Can a sample of 12 people be used to draw conclusions about the global human popula on?
80
75
70
65 Height (inches)
60
Male Female Sta s cs is a powerful tool for deciding what can be concluded from data An example: how long does a light bulb last?
h p://www.clipartbest.com/clipart-Kijxd4riq What real data look like: Number of light bulbs
Life me (hours) What real data look like: Number of light bulbs
Life me (hours) Mean is the average life me: Mean
n ∑xi mean = x = i n
Where: Number of light bulbs x = Average value for xi
Life me for a par cular bulb xi = n = Total number of bulbs Life me (hours) Standard devia on measures how closely data cluster around the mean
Smaller Standard Devia on
Larger Standard Devia on Number of light bulbs
Life me (hours) Standard devia on measures how closely data cluster around the mean Standard Devia on n 2 ∑(xi − x) s = i n −1
Where:
Number of light bulbs s = Standard devia on x = Average value for xi Life me for a par cular bulb xi = Life me (hours) n = Total number of bulbs Standard devia on measures how closely data cluster around the mean Standard Devia on n 2 ∑(xi − x) s = i n −1 Number of light bulbs Degrees of n −1= freedom
Life me (hours) If we had all the data
x à True mean, mu s à True Standard devia on, sigma Since we almost never have all the data, how confident are we about our es mate of the mean? Confidence Interval: a range of values within which there is a specified probability of finding the true mean Confidence Interval: a range of values within which there is a specified probability of finding the true mean Number of light bulbs
Life me (hours) Confidence Interval: a range of values within which there is a specified probability of finding the true mean
• For a Gaussian curve, a 68.3% of the data fall within 1 sigma of the mean, mu Number of light bulbs
Life me (hours) Confidence Interval: a range of values within which there is a specified probability of finding the true mean ts µ = x ± n µ = The true mean t = Student’s t s = Standard devia on n = Number of observa ons Where do we get t? Confidence Interval Example: ts 80 µ = x ± 75 n
70 x = 67.9 s = 2.9 65 Height (inches) n = 7 60 DOF = 6
Male Female t50% = 0.718 Confidence Interval Example: ts µ = x ± 80 n
75 µ50% = 67.9 ± 0.8
70
65 x = 67.9 Height (inches) s = 2.9 60 n = 7 Male Female DOF = 6
t50% = 0.718 99.9% Confidence interval is larger: ts µ = x ± 80 n
75 µ99.9% = 67.9 ± 6.5
70 x = 67.9 65 Height (inches) s = 2.9 60 n = 7 Male DOF 6 Female =
t99.9% = 5.959 How do we know whether two means are different from one another?
80
75
70
65 Height (inches)
60
Male Female How do we know whether two means are different from one another? Number of People
Height T-test for comparison of means:
x1 x2 x1 − x2 n1n2 tcalc = spooled n1 + n2
2 2 1 (n1 −1)+ 2(n2 −1) Number'of'light'bulbs' s s spooled = Number of People n1 + n2 − 2
Life1me'(hours)'Height T-test for comparison of means:
x1 x2 x1 − x2 n1n2 tcalc = spooled n1 + n2
74.2 − 67.9 5× 7 tcalc = Number'of'light'bulbs' 7.6 5+ 7 Number of People
Life1me'(hours)'Height T-test for comparison of means:
x1 x2 x1 − x2 n1n2 tcalc = spooled n1 + n2
tcalc =1.41
Number'of'light'bulbs' ttable50% = 0.7 Number of People
ttable90% =1.8
Life1me'(hours)'Height T-test for comparison of means:
tcalc =1.41 x1 x2
ttable50% = 0.7
ttable90% =1.8
à Significant at the 50% confidence level, not at the 90% confidence level
à There is a chance greater than 10% of Number'of'light'bulbs'
Number of People observing a difference this large if the two popula ons were the same
Life1me'(hours)'Height Be aware of assump ons
• The data adopt a normal distribu on • The data correspond to a representa ve sample of the total popula on • The (self-reported) data are accurate What can we do to increase our confidence?
x1 − x2 n1n2 tcalc = spooled n1 + n2
• Seek larger t values by increasing n – For a given total number of observa ons (n1 + n2), t is maximized when n1 = n2 • More precise experimental technique • Avoiding sampling bias, repor ng bias • Perform a second, complementary experiment Tools for compu ng sta s cal significance • Excel TTEST func on (spreadsheet available on course page) • Graphing so ware has built in tools to perform t tests Conclusions
• We are never totally certain of a result • Sta s cs provide a way to tell others how certain we are • Increasing the number of data points can increase certainty • Doing a second independent experiment can be very helpful Lies, Damn Lies, and Sta s cs
• What can go wrong? – Ill-defined ques on – Insufficient data – Over-interpreta on of Data • The following slides show how data mining can result in spurious correla ons; this illustrates a pi all of working with large data sets. We o en call a result significant if there is only a 5% chance of obtaining the result by chance; but if you “roll the dice” 1000 mes, you will find quite a few examples of that 5% chance. Data Fail S & P 500
he Journal of Inves ng Spring 2007, Vol. 16, No. 1: pp. 15-22 Data Fail S & P 500
he Journal of Inves ng Spring 2007, Vol. 16, No. 1: pp. 15-22 Examples of Gaussian Curves forming before your eyes • Exhibit at the Boston Museum of Science h ps://www.youtube.com/watch? v=KtPr7iipUso
• Pa erns of snow accumula on around an obstruc on appear to generate a similar effect