Introduction to Probabilities, Bayesian and Frequentist Statistics
Moutoshi Pal, Lica Iwaki & Carolin Lingl Research Methods in Clinical Psychology October 30, 2019 Outline of the Presentation
1. Quick summary of the paper
2. The Frequentist approach
3. Misconceptions of the p-value
4. The Bayesian approach
5. Benefits of Bayesian Statistics
6. Comparison of the two approaches 1. Quick summary of the paper
● replication crisis ● issues within Frequentist statistics that may have led to the replication crisis ● the alternative—Bayesian statistics ● rather than mere statistical reform, what is needed is a better understanding
➔ each approach provides a different kind of information that is useful for different aspects of scientific research
(Colling & Szűcs, 2018) 2. Frequentist Statistics Null Hypothesis Significance Testing (NHST)
"surely the most bone-headedly misguided procedure ever institutionalized in the rote training of science students" (Rozeboom, 1997, p. 335). Issues with Frequentist Statistics
● p-hacking ● Data dredging ● Significance chasing ● Optional stopping (a.k.a. data peaking) ● QPRs (questionable research practices) 2. Frequentist Statistics p-value: The p value is calculated from the sampling distribution, which describes what is to be expected over the long run when samples are tested
➔ Relates to the idea of a large (infinite) number of repeated experiments (i.e. coin toss) ➔ Given the null, the frequency of finding a significant result (p < 0.05) is 5% for each test
(Colling & Szűcs, 2018) 2. Frequentist Statistics
Definition of the p-value
A p -value is the probability of the observed data, or more extreme data, under the assumption that the null hypothesis is true. 2. Frequentist Statistics
Definition of the p-value
A p -value is the probability of the observed data, or more extreme data, under the assumption that the null hypothesis is true. 2. Frequentist Statistics
Definition of the p-value
A p -value is the probability of the observed data, or more extreme data, under the assumption that the null hypothesis is true. 2. Frequentist Statistics
Definition of the p-value
A p -value is the probability of the observed data, or more extreme data, under the assumption that the null hypothesis is true. 3. Misconceptions of the p-value Now it’s time for… a Kahoot Quiz!! Two approaches
● Two approaches on evidence and inference:
1. Frequentist approach: probability is a limit of the relative frequency of an event after many trials (i.e. coin toss)
- The issue: misconceptions led to replication crisis
2. Bayesian approach: an alternative method The difference
Recall, Frequentist’s definition of p-value:
“The p -value is the probability of the observed data, or more extreme data, under the assumption that the null hypothesis is true.”
**Emphasis**: the p-value is not the probability of a theory or hypothesis, but
the probability of the observed data given the hypothesis
P(D|Ho) The difference
Bayesian approach: probability is an expression of a degree of belief of an event, based on prior knowledge (i.e. previous experiments) or personal beliefs
The p- value is the probability of the hypothesis given the data
P(Ho|D) P(D|Ho) ≠ P(Ho|D)
Frequentist Bayesian The probability of the observed data/result given that some hypothesis is true is NOT equivalent to the probability that a hypothesis is true given that some data/result has been observed. Bayesian Statistics
● As opposed to Frequentist Statistics (which is based on an infinite number of repeatable events), Bayesian Statistics relates with subjective belief
● Thus, Bayesian Statistics uses the idea of updating beliefs with new information when testing a hypothesis
Historical background:
● In the past, Bayesian statistics was often overlooked by social scientists due to its complexity ● More recently, the development of powerful statistical software tools allows for the estimation of complex models using the Bayesian perspective Bayesian Statistics
● Uses the idea of updating beliefs with new information when testing a hypothesis
● Start with a belief about how something works (i.e. “eating sushi is dangerous”)
Prior beliefs x Bayes’ Factor = Posterior belief (= updated, new belief)
Bayes’ Factor: represents the amount of information that we’ve learned about our hypotheses form the data
● Criticism: prior belief can be different from person to person ○ Includes subjective “beliefs” in calculation (subjective, not objective) Bayesian Statistics ● Prior Distribution ● Likelihood gives the function of a parameter given the data ● Data Bayesian Curves
Uninformative Prior belief
훉 = 0.5 Comparison of Frequentist Statistics & Bayesian
Frequentist Bayesian
1. Parameters are 1. Parameters are fixed but unknown random and and data are data are fixed random 2. Probability is a 2. Probability is a degree of measure of certainty about frequency of values repeated events Comparison of Frequentist Statistics & Bayesian
Frequentist Bayesian
Confidence Interval: Credible Interval: “there is a 95% probability “given our observed that when I compute a data (posterior confidence interval from distribution), there is data of this sort, the true a 95% probability that value of θ will fall within it”. the true value of θ falls within the credible region” Benefits of Bayesian Statistics
1. They are dependent only on the observed data (and not the data that might have been collected) Subjective Priors Different Priors will lead 2. They are immune to the unknown to different posteriors & intentions of the researcher conclusions
3. They provide a measure of the strength of evidence that takes into account both the null and the alternative. References
● Colling, L. J., & Szűcs, D. (2018). Statistical Inference and the Replication Crisis. Review of Philosophy and Psychology, 1-27.
● Rozeboom, W. W. (1997). Good science is abductive, not hypothetico-deductive. What if there were no significance tests, 335-391.
● Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p-values: context, process, and purpose. The American Statistician, 70(2), 129-133. T.H.E E.N.D.