Introduction to Bayes factors Jeffrey R. Stevens https://osf.io/h38sx/ 9 May 2019 Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 1 / 39 Disclaimers 1 I am not a Bayesian statistics expert. 2 This introduction is not meant to accurately reflect the data analysis process; it is only meant to illustrate general techniques for calculating Bayes factors. 3 Please read the software documentation and Bayes factor literature to learn how to properly implement these techniques. Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 2 / 39 Questions 1 What is a Bayes factor? 2 Why use Bayes factors? 3 When to use Bayes factors? 4 How to present Bayes factors? 5 How to calculate Bayes factors? Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 3 / 39 What is a Bayes factor? Section 1 What is a Bayes factor? Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 4 / 39 What is a Bayes factor? What are we as scientists trying to do? Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 5 / 39 What is a Bayes factor? What are we as scientists trying to do? Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 6 / 39 What is a Bayes factor? What are we as scientists trying to do? Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 7 / 39 The probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value1 Probability of obtaining results at least as extreme as those observed given that the null hypothesis is true Based on hypothetical outcomes more extreme than the ones observed Gives probability of data given null hypothesis P(data|hypothesis) rather than probability of alternative hypothesis given data P(hypothesis|data) What is a Bayes factor? The p-value What is a p-value? 1 (Wasserstein & Lazar, 2016, American Statistician) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 8 / 39 Probability of obtaining results at least as extreme as those observed given that the null hypothesis is true Based on hypothetical outcomes more extreme than the ones observed Gives probability of data given null hypothesis P(data|hypothesis) rather than probability of alternative hypothesis given data P(hypothesis|data) What is a Bayes factor? The p-value What is a p-value? The probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value1 1 (Wasserstein & Lazar, 2016, American Statistician) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 8 / 39 Based on hypothetical outcomes more extreme than the ones observed Gives probability of data given null hypothesis P(data|hypothesis) rather than probability of alternative hypothesis given data P(hypothesis|data) What is a Bayes factor? The p-value What is a p-value? The probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value1 Probability of obtaining results at least as extreme as those observed given that the null hypothesis is true 1 (Wasserstein & Lazar, 2016, American Statistician) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 8 / 39 Gives probability of data given null hypothesis P(data|hypothesis) rather than probability of alternative hypothesis given data P(hypothesis|data) What is a Bayes factor? The p-value What is a p-value? The probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value1 Probability of obtaining results at least as extreme as those observed given that the null hypothesis is true Based on hypothetical outcomes more extreme than the ones observed 1 (Wasserstein & Lazar, 2016, American Statistician) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 8 / 39 What is a Bayes factor? The p-value What is a p-value? The probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value1 Probability of obtaining results at least as extreme as those observed given that the null hypothesis is true Based on hypothetical outcomes more extreme than the ones observed Gives probability of data given null hypothesis P(data|hypothesis) rather than probability of alternative hypothesis given data P(hypothesis|data) 1 (Wasserstein & Lazar, 2016, American Statistician) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 8 / 39 =6 What is a Bayes factor? The p-value “You might just as well say,” added the March Hare, “that ‘I like what I get’ is the same thing as ‘I get what I like’!” (Lewis Carroll, Alice in Wonderland) P(data|hypothesis) 6= P(hypothesis|data) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 9 / 39 =6 What is a Bayes factor? The p-value “You might just as well say,” added the March Hare, “that ‘I like what I get’ is the same thing as ‘I get what I like’!” (Lewis Carroll, Alice in Wonderland) P(data|hypothesis) 6= P(hypothesis|data) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 9 / 39 What is a Bayes factor? The p-value “You might just as well say,” added the March Hare, “that ‘I like what I get’ is the same thing as ‘I get what I like’!” (Lewis Carroll, Alice in Wonderland) P(data|hypothesis) 6= P(hypothesis|data) =6 Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 9 / 39 Bayesian parameter estimation Bayesian hypothesis testing What is a Bayes factor? Bayesian inference Based on Bayes theorem P(D|H)P(H) P(H|D) = P(D) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 10 / 39 What is a Bayes factor? Bayesian inference Based on Bayes theorem P(D|H)P(H) P(H|D) = P(D) Bayesian parameter estimation Bayesian hypothesis testing Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 10 / 39 Posterior odds are the relative plausibility of models after observing data. The change from prior to posterior odds brought about by the data is the Bayes factor.2 What is a Bayes factor? Bayesian hypothesis testing Assessing the relative plausibilities of competing hypotheses H0 and H1: P(H |D) P(H ) P(D|H ) 1 = 1 × 1 P(H0|D) P(H0) P(D|H0) | {z } | {z } | {z } Posterior odds Prior odds Likelihood ratio (Bayes factor) 2 (Wagenmakers, 2007, Psychonomic Bulletin & Review) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 11 / 39 What is a Bayes factor? Bayesian hypothesis testing Assessing the relative plausibilities of competing hypotheses H0 and H1: P(H |D) P(H ) P(D|H ) 1 = 1 × 1 P(H0|D) P(H0) P(D|H0) | {z } | {z } | {z } Posterior odds Prior odds Likelihood ratio (Bayes factor) Posterior odds are the relative plausibility of models after observing data. The change from prior to posterior odds brought about by the data is the Bayes factor.2 2 (Wagenmakers, 2007, Psychonomic Bulletin & Review) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 11 / 39 Strength of evidence from data about the hypotheses4 Relative predictive accuracy of one hypothesis over another5 What is a Bayes factor? The Bayes factor Extent to which the data sway our relative belief from one hypothesis to the other3 3 (Wagenmakers, 2007, Psychonomic Bulletin & Review; Etz et al., 2018, Psychonomic Bulletin & Review) 4 (Wagenmakers, 2007, Psychonomic Bulletin & Review) 5 (Wagenmakers, 2007, Psychonomic Bulletin & Review; Rouder et al., 2018, Psychonomic Bulletin & Review) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 12 / 39 Relative predictive accuracy of one hypothesis over another5 What is a Bayes factor? The Bayes factor Extent to which the data sway our relative belief from one hypothesis to the other3 Strength of evidence from data about the hypotheses4 3 (Wagenmakers, 2007, Psychonomic Bulletin & Review; Etz et al., 2018, Psychonomic Bulletin & Review) 4 (Wagenmakers, 2007, Psychonomic Bulletin & Review) 5 (Wagenmakers, 2007, Psychonomic Bulletin & Review; Rouder et al., 2018, Psychonomic Bulletin & Review) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 12 / 39 What is a Bayes factor? The Bayes factor Extent to which the data sway our relative belief from one hypothesis to the other3 Strength of evidence from data about the hypotheses4 Relative predictive accuracy of one hypothesis over another5 3 (Wagenmakers, 2007, Psychonomic Bulletin & Review; Etz et al., 2018, Psychonomic Bulletin & Review) 4 (Wagenmakers, 2007, Psychonomic Bulletin & Review) 5 (Wagenmakers, 2007, Psychonomic Bulletin & Review; Rouder et al., 2018, Psychonomic Bulletin & Review) Jeffrey R. Stevens (https://osf.io/h38sx/) Introduction to Bayes factors 9 May 2019 12 / 39 Whereas BF01 is evidence for H0 over H1 P(D|H0) P(D|H1) BF10 = 1/BF01 BF10=8 means that there is 8 times more evidence for H1 than H0 (and is equivalent to a BF01 = 1/8 = 0.125) Bayes factors range from 0 to ∞ What is a Bayes factor? The Bayes factor BF10 is the Bayes factor giving the evidence for H1 over H0 P(D|H1) P(D|H0) Jeffrey R.