Reported Self-Control Is Not Meaningfully Associated With

University of Dundee

Reported self-control is not meaningfully associated with inhibition-related executive function Saunders, Blair; Milyavskaya, Marina; Etz, Alexander; Randles, Daniel; Inzlicht, Michael

Published in: Collabra: Psychology

DOI: 10.1525/collabra.134

Publication date: 2018

Licence: CC BY

Document Version Publisher's PDF, also known as Version of record

Link to publication in Discovery Research Portal

Citation for published version (APA): Saunders, B., Milyavskaya, M., Etz, A., Randles, D., & Inzlicht, M. (2018). Reported self-control is not meaningfully associated with inhibition-related executive function: A Bayesian analysis. Collabra: Psychology, 4(1), [39]. https://doi.org/10.1525/collabra.134

General rights Copyright and moral rights for the publications made accessible in Discovery Research Portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from Discovery Research Portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain. • You may freely distribute the URL identifying the publication in the public portal. Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Download date: 26. Sep. 2021 Saunders, B., et al. (2018). Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function: A Bayesian Analysis. Collabra: Psychology, 4(1): 39. DOI: https://doi.org/10.1525/collabra.134

ORIGINAL RESEARCH REPORT Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function: A Bayesian Analysis Blair Saunders*, Marina Milyavskaya†, Alexander Etz‡, Daniel Randles§ and Michael Inzlicht§,‖

Self-control is assessed using a remarkable array of measures. In a series of five data-sets (overall N = 2,641) and a mini meta-analysis, we explored the association between canonical operationalisations of self-control: The Self-Control Scale and two measures of inhibition-related executive functioning (the Stroop and Flanker paradigms). Overall, Bayesian correlational analyses suggested little-to-no relationship between self-reported self-control and performance on the Stroop and Flanker tasks. The Bayesian meta- analytical summary of all five data-sets further favoured a null relationship between both types of measurement. These results suggest that the field’s most widely used measure of self-reported self-control is uncorrelated with two of the most widely adopted executive functioning measures of self-control. Consequently, theoretical and practical conclusions drawn using one measure (e.g., the Self-Control Scale) cannot be generalised to findings using the other (e.g., the Stroop task). The lack of empirical correlation between measures of self-control do not invalidate either measure, but instead suggest that treatments of the construct of self-control need to pay greater attention to convergent validity among the many measures used to operationalize self-control.

Keywords: self-control; cognitive control; inhibitory executive functioning; Bayesian statistics

Classically defined as the overriding of unwanted impulses but seldom investigated empirically (cf., Duckworth & (Baumeister, 2014; Roberts, Lejuez, Krueger, Richards, Kern, 2011). Yet, the importance of understanding the & Hill, 2014), self-control is among the most celebrated nature of self-control measures cannot be understated. facets of higher cognition. Impulse control enjoys such Poor self-regulation has been identified as “the major theoretic prominence that inhibition has been proposed social pathology of the present time” (Baumeister et al., to underlie 80–90% of self-regulation (Baumeister, 2014; 1994, pp. 3; see also, Mischel, Shoda, & Rodriguez, 1989), Baumeister, Heatherton, & Tice, 1994). Also remarkable with low reported self-control predicting poorer health, is the array of measures used to assess self-control—from finances, and wellbeing, and higher rates of mortality introspective self-report questionnaires to reaction-timed and criminal convictions (Moffitt et al., 2011; Tangney, tests of executive functioning. While such measures are Baumeister, & Boone, 2004). Therefore, understanding undoubtedly diverse, what seems to unite them is the operationalisations of the construct of self-control is idea that they each tap some ability to override unwanted critical. Most people have a lay conceptualisation of what dominant impulses (Baumeister et al., 2014; Baumeister, it feels like to resist temptation and to exert self-control. 2014; Hofmann, Schmeichel, & Baddeley, 2012; Inzlicht, However, are common empirical measures, that each Schmeichel, & Macrae, 2014). putatively assess the ability to resist impulses in their own That all these measures relate to a common construct right, statistically related to each other? Do people who called self-control has been assumed often by face- self-report high levels of self-control on questionnaire validity (e.g., “I am good at resisting temptations”; measures also show improved performance on laboratory Stroop performance requires inhibiting word-reading), tests of self-control? Here, we addressed these questions by examining the relationship between self-report measurements of self- * Psychology, School of Social Sciences, University of Dundee, GB control and performance measures of inhibition-related † Department of Psychology, Carleton University, Ottawa, CA executive functioning through five novel data-sets and a ‡ Department of Cognitive Sciences, University of California, meta-analytical summary (N = 2,641). Irvine, US § Department of Psychology, University of Toronto, CA The Self-Control Scale ‖ Rotman School of Management, University of Toronto, CA Developed by Tangney and colleagues (2004), the Self- Corresponding author: Blair Saunders Control Scale is a self-report questionnaire assessing ([email protected]) “the ability to override or change one’s inner responses, Art. 39, page 2 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function as well as interrupt undesired behavioural tendencies switching have also been identified as dissociable sub- (such as impulses) and refrain from acting on them” components (Miyake et al., 2000). More recent analyses (pp. 274, emphasis added). Consistent with prominent have suggested, however, that the inhibition component is theories (Baumeister et al., 1994), considerable emphasis particularly strongly related to the shared variance among was placed on inhibition in the development of the Self- executive functions (Miyake & Friedman, 2012). This latter Control Scale. Several items in the Self-Control Scale finding is consistent with the idea that inhibition-related include content that is face-valid in relation to inhibition processes play a central role in governing flexible goal- (e.g., “I am good at resisting temptation”; “I refuse things directed actions. that are bad for me, even if they are fun”). Beyond In the present studies, we used the Stroop and flanker inhibition, however, the Self-Control Scale assesses tasks as performance measures of inhibition-related multiple outcomes and processes that are more globally executive functioning. In the Stroop task, people identify reflective of self-discipline and goal-directed actions (e.g., the physical colour of a word while ignoring the lexical “I have trouble concentrating”; “I tend to be disorganised”; meaning of the word that may be either compatible “I say inappropriate things”; “I am lazy”; all reverse-scored). (the word “blue” in blue ink) or incompatible (the word Initial validation work demonstrated that higher “red” written in blue ink). Participants in the flanker Self-Control Scale scores were associated with better task must respond to a central target letter (e.g., “S” or psychological adjustment, reduced problematic food and “H”) while surrounding flanker letters prime either the alcohol consumption, increased relationship satisfaction, correct response on compatible trials (e.g., “SSSSS”), and more adaptive emotional responses (Tangney et al., or the incorrect response on incompatible trials (e.g., 2004). These relationships were largely confirmed in “HHSHH”). In both cases control is required to overcome a subsequent meta-analysis of 102 studies (de Ridder, the dominant but unwanted response tendency primed Lensvelt-Mulders, Finkenauer, Stok, & Baumeister, 2012). by the flankers or automatic word-reading processes on In short, self-control—as operationalised by the Self- incompatible trials. Control Scale—predicts the good-life. The Stroop and flanker tasks were originally conceived to Likely reflecting the wide scope of the Self-Control investigate cognitive interference, and were presented in Scale, this measure is strongly correlated with other broad within-subjects experimental designs (Eriksen & Eriksen, individual differences including conscientiousness (John, 1974; Stroop, 1935). However, these paradigms have been & Srivastava, 1999; Roberts, Jackson, Fayard, Edmonds, widely adopted as self-control measures in social and & Meints, 2009) and grit (Duckworth et al., 2007). personality psychology (Allom et al., 2016; Edmonds et Conscientiousness encompasses needs for achievement, al., 2009; Gailliott et al., 2007; Inzlicht & Gutsell, 2007; organization, restraint, and rule following (Goldberg, Molden et al., 2012). Despite their experimental origins, 1990; John, & Srivastava, 1999; Roberts, Jackson, Fayard, performance measures of inhibition-related executive Edmonds, & Meints, 2009), and reliably predicts health, functioning are frequently used in cross-sectional wealth, and well-being across the lifespan (Bogg & Roberts, designs to assess individual differences in inhibition and 2004; Kern & Friedman, 2008). Similarly, grit is defined impulsivity (Davidson, Amso, Anderson, & Diamond, as perseverance and passion for long-term goals, even in 2006; Hall, 2012; Nigg, 2001; Snyder, 2012). Individual the face of short-term setbacks and obstacles (Duckworth differences in executive functions have been related to et al., 2007; Von Culin, Tsukayama, & Duckworth, 2014). multiple real-world outcomes thought to characterise Relative to self-control, conscientiousness and grit were good self-control, such as relationship fidelity (Pronk, not developed with a central emphasis on inhibition. Karremans, & Wigboldus, 2011), treatment-compliance However, given the strong correlations among grit, among drug users (Streeter et al., 2008), and not snacking conscientiousness, and the self-control scale (cf., Roberts on unhealthy foods (e.g., Allan, Johnston, & Campbell, et al., 2014), our analyses also included a “self-discipline” 2010). Thus, while tasks like the Stroop are often used to composite that combined conscientiousness, grit, and the assess the influence of short-term, state effects on self- Self-Control Scale into a single measure. control (e.g., Inzlicht & Gutsell, 2007; Molden et al., 2012), it is variation in these tasks as an individual difference Impulse control and executive functioning that is of interest in the current work. Whenever self- In addition to self-report measurement, the ability to control on the Stroop task or flanker task is mentioned override impulses is commonly measured using executive in this manuscript as a correlate of other measures (e.g., functioning tasks such as the Stroop, Flanker, and the Self-Control Scale), we are referring to self-control as Go-Nogo paradigms (Miyake et al., 2001). Bearing a close an individual difference rather than a momentary, state resemblance to definitions of self-control (Baumeister effect. et al., 1994; Hofmann et al., 2012), executive functions One controversy regarding conflict control tasks is the reflect a domain-general range of processes that allow specific role of inhibition in successful task performance. individuals to flexibly regulate attention and behaviour For example, interference from incompatible trials could in a goal-directed manner (Banich, 2009; Miyake et al., be overcome either by inhibiting the inappropriate motor 2000). Assessed using a range of tasks, executive functions response to the irrelevant stimulus dimension (e.g., demonstrate both unity and diversity. While a common inhibiting the flanker letters; overriding word-reading factor appears to unite diverse measures of executive processes in the Stroop task) or by focusing attention functioning, the processes of inhibition, updating, and on the task-relevant dimension (e.g., the central letter in Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Art. 39, page 3 of 16 Executive Function the flanker stimulus, or the physical colour of the Stroop variability. Consequently, low between-subject variability target; Cohen, Dunbar, McClelland, 1990; Egner & Hirsch, might limit the extent to which behavioural measures of 2005). No clear consensus has emerged on whether these self-control (e.g., Stroop, flanker) can correlate with other executive functioning tasks rely on selective attention, individual differences, such as the Self-Control Scale. Again, inhibition, or both. Most important for present concerns, this reliability paradox might explain why correlations however, is that these tasks more broadly assess the between self-reported self-control and inhibition-related ability to control the influence of inappropriate impulses executive functions would be rather small. on performance in a goal-directed manner. Hereafter, Finally, common biases in academic publishing might these processes will be referred to as inhibition-related mean that prior meta-analytical effect sizes might executive functions, while acknowledging that the exact overestimate the strength of relationship between scale mechanisms underlying successful performance on these measures of self-control and performance measures, tasks requires clarification. like the Stroop or flanker tasks. The modest correlations between scale and performance measures reported in The convergence between self-reported self- prior meta-analyses (i.e., Duckworth & Kern, 2011) might control and performance measures of self-control overestimate the underlying effect sizes because they did The heterogeneity of tools available to investigate self- not correct for publication bias (i.e., the general tendency control might suggest that a single latent process underlies for significant results to be published more often than impulse control across contexts and modalities—from null results; Rothstein, Sutton, & Borenstein, 2005). reaction time performance at a millisecond resolution to global introspections about one’s ability to self-regulate. The current work This consilient view of self-control assessment is In a series of 5 data-sets and a meta-analysis, we asked if hindered, however, by the modest correlations among the Self-Control Scale (Tangney et al., 2004)—likely the measures of self-control. A recent meta-analysis most common self-control questionnaire—is correlated (Duckworth & Kern, 2011) reported small but statistically with the overriding processes commonly identified by the significant convergence among measures of self-control Stroop and flanker tasks. When formatting our research (i.e., self-report, informant report, choice tasks, and question, it is important to consider the smallest effect inhibition-related executive functioning, r = .27). This size that would still be of practical and theoretical interest. meta-analysis further revealed that while the expected A correlation of at least moderate strength (e.g., r ≥ .4) correlations between self-reported self-control and might be expected if moderate-to-strong convergence is inhibition-related executive functions were present, the anticipated between inhibitory executive functions and effect size of these associations were particularly small in trait-self-control. However, effect sizes in this range seem magnitude (r = .10). particularly unlikely given prior meta-analytical results Multiple factors might contribute to low convergence (Duckworth & Kern, 2011). As such, we anticipated that any among measures of self-control. As mentioned previously, correlation between scale and inhibition-related executive the self-control scale is not solely focused on inhibition, functioning measures of self-control would be small (e.g., but incorporates a wide range of characteristics and rs > .10 < .25). It should be further noted that the smallest behaviours that are indicative of self-discipline. The effect size of theoretical significance differs depending breadth of the Self-Control Scale might mean that on the proposed application, however, it seems that any it assesses a broad spectrum of characteristics, with effect size approaching zero (<.10) would be small enough this content only overlapping to a small extent with to be off little practical or theoretical significance. the inhibitory processes that are assessed in tests of Our work builds on prior investigations in two key ways. inhibition-related executive functioning. It is also true First, working against potential file-drawer problems, that inhibition-related executive functions focus more we present every study that we are aware of from narrowly on a constrained range of cognitive processes that our laboratory (i.e., the Toronto Laboratory for Social allow people to overcome interference that is specific to a Neuroscience) that includes the Stroop or flanker task and single, rather abstract, goal (e.g., “name colours”, “identify the Self-Control Scale. Second, by exclusively employing the central target letter”). As such, this relatively narrow null-hypothesis significance testing, existing studies can scope of the Stroop and flanker tasks might mean that only support convergence among measures. Instead, inhibition-related executive functions can only correlate we used Bayesian methods to obtain a Bayes factor very slightly with the broader definition of inhibition used comparing a null and non-null hypothesis, and computed in the Self-Control Scale, simply because these executive the posterior distribution of the correlation given the functioning measures capture significantly less content. It alternative model. The Bayes factor gives us the evidence is noteworthy that this logic still anticipates a detectable for or against the hypothesis of no correlation, and the (albeit small) relationship between the broader trait posterior tells us how big any non-zero correlation is likely measure and the performance measure. to be (see Etz & Vandekerckhove, in press). The reliability paradox is another factor that might contribute to the relatively low correlations (Hedge, Powell, Study 1 & Sumner, 2017). This paradox refers to a phenomenon The Stroop task is a common laboratory operationalisation where behavioural tasks only become established in the of self-control (e.g., Gaillott et al., 2007; Inzlicht & Gutsell, experimental literature if they show little between-subject 2007; Molden et al., 2012). In study 1, we used the Stroop Art. 39, page 4 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function task to test convergence between reported self-control 1500 ms) followed by a blank screen (550 ms).1 Self- and inhibition-related executive functioning. paced breaks occurred between blocks where participants reported their subjective experience (not analysed here). Method We created mean scores for each dependent measure Participants (Stroop effects [i.e., RT/error-rates on incompatible trials We decided apriori to collect data from at least 200 minus RT/error rates on compatible trials], accuracy and participants. This sample size exceeds that of similar prior reaction times on compatible trials and incompatible investigations (Allom et al., 2016; Edmonds et al., 2009), trials) separately for each experimental block to assess and meets rule-of-thumb guidelines for sample size in the internal consistency of the Stroop task. Consequently, social-personality psychology (Fraley & Vazire, 2014). 224 Cronbach’s α was calculated using 9 different values (i.e., undergraduate students from the University of Toronto one from each experimental block) for each measure in Scarborough participated for course credit (81 females; the experiment (e.g., the Stroop effect in RT). The resulting mean age = 18.8, SD = 2.5 years). Seven participants were internal consistency estimates suggested that reliabilities excluded from these analyses either because of software were good-to-excellent for most measures (compatible malfunction or incorrectly responding to likert-type mean RT, α = .95; incompatible mean RT, α = .93; questions (responding outside the range of the scale). compatible error rates, α = .83; incompatible error rates, For each study in this manuscript, ethical approval was α = .83), but low for the Stroop effect in mean reaction obtained from the Research Ethics Board at the University time (α = .52) and error rates (α = .46; see also Wöstmann of Toronto, Scarborough before data collection started. et al., 2013). In each study informed consent was provided by each Given these reliabilities, it might be suggested that the participant before taking part. subsequently reported null correlations were a result of low measurement reliability, rather than low association Scales among reported self-control and inhibition-related Self-control was assessed using the 13-item Self-Control executive function. However, it should be noted that Scale (Tangney et al., 2004). We additionally measured similar small-to-null correlations were observed when two other self-regulatory individual differences: Bayesian correlations were conducted between reported conscientiousness and grit. Conscientiousness was self-control/self-discipline and non-difference executive assessed using the conscientiousness 9-item subscale of functioning scores (i.e., mean RT and error-rates for the 44-item Big Five Inventory (John & Srivastava, 1999). incompatible and compatible trials) across all studies in Grit was assessed using the 12-item grit scale (Duckworth this manuscript (see supplemental materials). et al., 2007). Please see Table 1 for descriptive and reliability statistics for all scales. Although theoretical Analysis strategy distinctions exist between grit, conscientiousness, and the Bayesian Pearson’s correlations were computed to quantify Self-Control Scale (cf., Duckworth & Gross, 2014), these the association between the Stroop effect and self-report self-regulatory traits tend to correlate highly with each scores. The primary benefit of adopting this Bayesian other and the Self-Control Scale (rs ~ .70–.80; Roberts approach is that in addition to providing evidence in et al., 2013). Consequently, we created a composite self- favour of an alternative hypothesis (e.g., a negative discipline measure aggregating across the Self-Control correlation between the Self-Control Scale and the Stroop Scale, conscientiousness, and grit. Similar composite scores effect) and estimating the size of the correlation, Bayes have previously demonstrated higher utility in predicting factors can also be used to provide evidence in favour of the ability to select between competing impulses than a null relationship (Wagenmakers, Verhagen, & Ly, 2015). single measures (Duckworth & Seligman, 2006). Thus, Bayes factors can be interpreted “as the degree to which by improving signal-to-noise ratios, this composite scale the data sway our belief from one to the other hypothesis” might show a particularly reliable relationship with (Etz & Vandekerckhove, 2016, p. 4). For example, in the inhibition-related executive functioning. Scales were case where we initially have no preferred hypothesis, so computerized and completed by participants immediately that we assign 50% prior probability for each of the null after the Stroop task. and alternative hypotheses, a Bayes factor of 3 in favor of the null brings the probability of the null hypothesis to Stroop paradigm 75% (a Bayes factor of 10 brings the probability of the null The Stroop started with 10 practice trials of 6 object-words hypothesis to 91%). (chair, house, lamp, spoon, table, and window) presented in red or blue. The left arrow-key was pressed to identify Results and Discussion blue words, and the right arrow key for red words. Correlations among self-regulatory constructs The main Stroop task started after these practice trials. As anticipated, grit, conscientiousness, and Self-Control Stimuli were the words “RED” and “BLUE” presented in Scale scores were strongly correlated (all rs > .598, either red or blue to create compatible (e.g., “RED” in red see Table 1). A self-discipline composite measure was font) and incompatible (e.g., “BLUE” in red font) targets. created by z-scoring grit, the Self-Control Scale, and Trials commenced with a fixation cross (250 ms), followed conscientiousness and summing these standardised by a target stimulus until response (min: 150 ms; max: values. Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Art. 39, page 5 of 16 Executive Function

Table 1: Table showing correlations (Pearson’s rho) and descriptive statistics for the Self-Control Scale, Conscientious- ness, and Grit.

Scale Descriptives Self-disc. Self-control Grit Consc. M (SD) α Study 1 Consc. .892 0.695 0.644 – 3.32(0.69) .75 Grit .855 0.598 – 2.96(0.67) .83 Self-Control .875 – 3.19(0.57) .77 Self-disc. – 0.00(2.6) .85 Note: Consc. = Big Five Inventory – Conscientiousness; Self-Control = Self-Control Scale; Self-disc = Self-Discipline Composite; α = Cronbach’s alpha.

Stroop effects Studies 2 a-c Our procedure successfully produced the Stroop effect in The following three studies replicated the above findings reaction time and error rates. Responses were faster on using a large online sample of participants and another compatible (M = 438 ms; SE = 3.80) than incompatible common test of inhibition-related executive functions, trials (M = 455 ms; SE = 4.42) trials, t(216) = –12.579, p < the Flanker paradigm (cf., Eriksen & Eriksen, 1974; Miyake .001, d = –0.947. Error rates were higher on incompatible et al., 2001). Because these studies share an almost (M = 7.5%; SE = 0.35) than compatible trials (M = 4.9% SE identical protocol we present them together noting any = 0.28), t(216) = –11.521, p < .001, d = –0.821. methodological divergence.

Bayesian correlations Method We next tested correlations between the Stroop Participants effect and the self-report measures. Difference values Study 2a. Eight hundred and fifty-six participants (397 were used because they are indicative of overriding females; mean age = 35.23, SD = 11.11 years) were processes while controlling for base rate performance.2 recruited from Mechanical Turk (MTurk) in October 2015. As increasing values on each self-report scale should MTurk is crowd-sourced marketplace in which workers are be associated with reduced Stroop effects (i.e., higher remunerated for completing online tasks. While data from control), we set a prior on the correlation that is skewed MTurk tends to be noisier than lab data, this platform such that most of its mass (78%) falls on negative facilitates the recruitment of lager samples in service of correlation values (see Figure 1).3 Here, a Bayes factor statistical power (Crump, McDonnell, & Gureckis, 2013). in favour of the alternative hypothesis (BF01 < 1) would All recruited MTurk workers were located in the USA and support correlations among performance and self-report had completed a minimum of 5 HITs with >90% success measures. In contrast, a Bayes factor favouring the rate. Participants were compensated $3 USD, and sessions null (BF > 1) would suggest no relationship between lasted 25 minutes. The study consisted of an arrow 01 ~ questionnaire and behavioural measures. We report version of the flanker task, followed by a battery of self- posterior medians and 95% posterior (credible) intervals report questionnaires. This data was initially collected as a in brackets for each correlation, which tell us the most training sample for a machine learning project. However, likely range for the value of the correlation if it is in fact we only analysed variables pertinent to the hypotheses non-zero. addressed in Study 1: The flanker data; reported self- Stroop effect in reaction time. The data provide control, conscientiousness, and grit. evidence that the Stroop effect in reaction times was Two hundred and nine participants were excluded for not correlated with Self-Control Scale scores (r = –.012 making >40% errors on the flanker task or having missing

[–.143, .119] BF01 = 7.73) or the self-discipline composite data from the flanker task (e.g., because of software 5 measure (r = –.027 [–.158, .105], BF01 = 7.26), see malfunction). While this exclusion rate is high (>24%), Figure 2. Moreover, the posterior intervals suggest that the 40% error criteria ensures that responding is above any correlation between reaction time performance and chance. One further participant was excluded for very these scales would likely be small. fast mean reaction time (<100 ms). The final sample Stroop effect in choice error rates. The data included 647 participants, resulting in 80% power to find provide some evidence that the Stroop effect in error a correlation of .11 when α = .05; and >99% power to find rates was not associated with scores on the Self-Control a correlation of .20.

Scale (r = –.067 [–.196, .065], BF01 = 4.81) or the self- Study 2b. 1,621 participants (1227 females, mean discipline composite measure (r = –.102 [–.231, .029], age =39.20, SD = 14.22) recruited by the online survey

BF01 = 2.44), see Figure 2. The posterior intervals and market research company Tellwut (www.tellwut. suggest that any correlation between error rates and com). As with the previous sample, all participants either of these scales would most likely be negative and were recruited from the USA, paid $3 USD for taking small.4 part, and completed the same study protocol as in Art. 39, page 6 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function

Figure 1: Prior distribution used in the Bayesian correlation analysis in Study 1. The vertical line marks the median of this distribution at –.29. This prior was obtained by first placing a N(–.3, .4) prior on Fisher’s Z and then converting back to the correlation scale (see the supplementary materials for details, https://osf.io/8etus/). Correlation coef- ficients unaffected by the prior can be found in the online code on our OSF page.

Figure 2: Scatterplots show the Stroop effect in reaction time (top panels) and error rates (lower panels) as a function of the Self-Control Scale (left) and the self-discipline composite measure (right). Lines of best fit show linear regressions including 95% confidence intervals.

Study 2a. Exclusion criteria were identical to study inspection of the scatterplots one participant was also 2a resulting in 9006 participants being excluded removed whose flanker effect that was >4 SDs below and a final sample of 723 participants. Upon visual the mean. Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Art. 39, page 7 of 16 Executive Function

Study 2c. 1,769 participants (932 females, mean age = suggested that the reliability of the Flanker is similar to 40.21, SD = 12.53) were recruited from the online panel that which we obtained in Study 1 (Wöstmann et al., 2013). company Cint (www.cint.com). These participants were also based in the USA, were paid $3 USD for participation, Results and Discussion and completed a protocol that was identical to studies 2a Scale correlations and reliability and 2b. The same exclusion criteria applied to studies 2a All three studies supported strong positive and 2b identified 888 participants for exclusion resulting associations among self-reported grit, self-control, and in a final sample of 881 individuals. conscientiousness, all rs > .698, all ps < .001, see Table 2. A self-discipline composite measure was created as in Scales Study 1. In each study, grit and conscientiousness were assessed as in Study 1, while we used the extended 36 item version Flanker effects of the Self-Control Scale (cf., Tangney et al., 2004). See These brief online experiments revealed robust Flanker Table 2 for descriptive statistics. Conscientiousness was effects in all three data-sets: Responses were slower and assessed in study 2c using the relevant subscale from the more error-prone on incompatible than compatible trials IPIP-NEO-120 (Johnson, 2014). (see Table 3).

Arrow Flanker task Bayesian correlations All three studies used an identical arrow flanker task. To evaluate the correlations from studies 2a, 2b, and 2c Flanker stimuli consisted of five arrowheads (i.e., we employed the following strategy: For each analysis in compatible: <<<<<, >>>>>; incompatible: <<><<, study 2a, we used the posterior from the corresponding >><>>). Participants responded to the central arrow test in study 1 as the prior. Then for each analysis in using the left or right arrow-key on their keypad. Here, study 2b, we used the posterior from 2a as the prior, and control is required on incompatible trials to override repeated this again for 2c. This Bayesian updating yields the misinformation primed by the flankers. The main sequential Bayes factors (BF0r; see Ly, Etz, Marsman, & experiment consisted of 100 flanker trials (50 compatible, Wagenmakers, 2017), and these can be interpreted as the 50 incompatible) preceded by 10 practice trials. Each trial additional evidence gained from each study beyond the started with a fixation cross for 1000 ms, followed by the evidence we had before. target until response (max. 1000 ms). Flanker effect in mean reaction times. The data Our statistical approach followed that used in Study 1. from studies 2a and 2b provide little additional evidence Due to the short nature of the flanker task in these studies, for discerning whether scores on the Self-Control Scale we were unable to obtain comparable measures of internal are associated with the flanker effect or not (2a: r = .013 consistency in the brief online flanker task as we did in [–.054, .080], BF0r = 1.87; 2b: r = –.009 [–.058, .040], BF0r Study 1 for the Stroop task. However, previous reports have = 1.37), whereas 2c provides some counter evidence in

Table 2: Table showing correlations (Pearson’s rho) and descriptive statistics for the Self-Control Scale, Conscientious- ness, and Grit.

Scale Descriptives Self-disc. Self-Control Grit Consc. M(SD) α Study 2a Consc. .931 .792 .790 – 3.89(0.74) .895 Grit .921 .764 – 3.51(0.74) .897 Self-Control .922 – 3.62(0.62) .936 Self-disc. – 0.01(2.77) .915 Study 2b Consc. .897 .700 .711 – 3.74(0.74) .849 Grit .9 .699 – 3.38(0.62) .791 Self-Control .894 – 3.36(0.59) .905 Self-disc. – 0.01(2.66) .879 Study 2c Consc. .913 .775 .726 – 5.17(0.89) .892 Grit .905 .745 – 3.42(0.57) .765 Self-Control .924 – 3.48(0.58) .913 Self-disc. – 0.001(2.74) .901 Note: Consc. = Big Five Inventory – Conscientiousness; Self-Control = Self-Control Scale; Self-disc. = Self-Discipline Composite; α = Cronbach’s alpha. Art. 39, page 8 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function

Table 3: Table showing descriptive and inferential statistics for the Flanker tasks for studies 2a–c.

Flanker effects Compatible Incompatible M(SD) M(SD) t Cohen’s d Study 2a RT (ms) 452(58) 474(61) –14.940*** –0.563 %Errors 5.25(6.27) 8.19(9.74) –9.159*** –0.389 Study 2b RT (ms) 456(56) 481(56) –20.406*** –0.759 %Errors 9.09(8.74) 13.70(12.97) –12.114*** –0.486 Study 2c RT (ms) 449(61) 478(60) –20.916*** –0.995 %Errors 9.00(8.60) 15.01(14.83) –14.450*** –0.544 Note: RT = mean reaction time; %Errors = mean choice error rate; *** = p < .001. favour of a non-zero correlation (2c: r = –.045 [–.084, approximately two hours, including setting up the EEG

–.006], BF0r = .107), see Figure 3. A similar pattern holds apparatus, completing the computerised tasks, and the for the associations between the flanker effect and Self- subsequent questionnaires. Forty-three participants were discipline composite scores (2a: r = .024 [–.043, .090] BF0r excluded following the same exclusion criteria that were

= 1.68; 2b: r = .005 [–.044, .054], BF0r = .1.69; 2c: r = –.026 applied in Studies 2a–c.

[–.065, .014], BF0r = .568), see Figure 4. Choice error rates. The data from studies 2a–2c Scales provide mixed evidence for whether flanker effect in Self-control and grit were assessed in the same manner error rates are associated with the Self-Control Scale as Study 1 (see Table 4), however, the likert questions

(2a: r = –.009 [–.076, .058], BF0r = 3.12; 2b: r = –.035 for the Self-Control Scale were a range of 7 points rather

[–.084, .014] BF0r = .535; 2c: r = –.061 [–.010, .021], BF0r = than 5. Grit was assessed by the 8-item short grit scale .035). A similar overall pattern holds for the association (Duckworth & Quinn, 2009; α = .76). The questionnaires between flanker effect error rates and the self-discipline were answered by participants after the computerised composite (2a: r = –.014 [–.081, .052], BF0r = 5.81; 2b: r tasks.

= –.024 [–.073, .026], BF0r = 0.949, 2c: r = –.045 [–.085,

–.006], BF0r = .162). Arrow-flanker task As in studies 2a–c, participants performed an arrow Study 3 version of the flanker task in a dimly lit room. Here, we Study 3 was an in-lab version of the flanker task that was only note deviations from the protocol used in these conducted as part of a battery of baseline measures for a previous studies. Trials commenced with the presentation longitudinal goal-striving study that was on-going at the of a fixation cross (250 ms) that was followed by a flanker time of writing this manuscript. The flanker task always target stimulus until response (min: 100 ms, max: 1000 came first in a series of 3 tasks that were undertaken ms), followed by a blank screen for 600–1000 ms before by participants while electroencephalography (EEG) the start of the next trial. Participants performed a total was recorded. The other two tasks in this series were a of 420 trials. Participants were given self-paced breaks time-estimation paradigm and passive picture viewing after blocks of 60 trials, and were instructed to respond as task that were not relevant to the current research quickly and accurately as possible. questions. After these tasks the participants answered a battery of questionnaires that included assessments Results and Discussion of conscientiousness, grit, and trait self-control. As with all previous studies, strong positive correlations The following analyses report only on the executive were observed between conscientiousness, trait self- functioning and self-report measures included in studies control, and grit, all rs > .676, ps < .001. 1 and 2a–c. Flanker effects Method Responses were slower on incompatible trials (M = 488 Participants ms, SD = 44.24) than compatible trials (M = 400 ms, SD = Two hundred and twenty-one participants took part in 34.27), t(177) = –46.08, p < .001, d = –3.481. Error rates the study (61.5% females, mean age = 20.2, SD = 5.6), were also higher for incompatible trials (M = 18.9%, SD and were largely recruited through the undergraduate = 12.55) than compatible trials (M = 2.3%, SD = 4.16), pool at the University of Toronto Scarborough, while t(177) = –20.07, p < .001, d = –1.504. Together, these a smaller number of participants were recruited results confirm that the classic flanker interference effect through community advertisements. The session lasted was present in this data set. As in the previous studies, in Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Art. 39, page 9 of 16 Executive Function

Figure 3: Scatterplots show the Flanker effect in reaction time (top panels) and error rates (lower panels) as a function of the Self-Control Scale. Lines of best fit show linear regressions including 95% confidence intervals.

Figure 4: Scatterplots show the Flanker effect in reaction time (top panels) and error rates (lower panels) as a function of the self-discipline composite measure. Lines of best fit show linear regressions including 95% confidence intervals.

Study 3 we used the difference scores (incompatible minus Bayesian correlations compatible) created from the flanker task separately for Following the strategy of Bayesian updating from the reaction times and error-rates. previous studies, the posterior from Study 2c was used as Art. 39, page 10 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function

Table 4: Table showing correlations (Pearson’s rho) and descriptive statistics for the Self-Control Scale, Conscientious- ness, and Grit.

Scale Descriptives Self-disc. Self-control Grit Consc. M(SD) α Study 3 Consc. .893 .697 .682 – 3.37(0.64) .76 Grit .884 .676 – 3.26(0.62) .83 Self-Control .892 – 4.06(1.00) .76 Self-disc. – 0.01(0.89) .92 Note: Consc. = Big Five Inventory – Conscientiousness; Self-Control = Self-Control Scale; Self-disc. = Self-Discipline Composite; α = Cronbach’s alpha. the prior for study 3 to yield a further sequential Bayes The results of each random-effects meta-analysis are factor (Ly et al., 2017). We can interpret each posterior summarized in Figure 5. The posterior distribution for the from study 3 as the result of a Bayesian fixed-effects meta-analytic correlation (rho) between the Self-Control meta-analysis because they contain all the information Scale predicting conflict effects in reaction time suggests accumulated across all the studies. Furthermore, a fixed any association that might exist is likely negative and effect meta-analytic Bayes factor is given by the product of small (r = –.028 [–.181, .109]), and the Bayes factor favours all individual studies’ sequential Bayes factors. For mean the null model (BF01 = 8.34). Similarly, the posterior for reaction times, the data from Study 3 provided a little the meta-analytic correlation between the self-discipline more evidence in support of a null relationship between composite score and conflict effects in reaction time was the self-control scale and the flanker effect, r = –.039 small (r = –.021 [–.161, .093]), and again the Bayes factor

[–.077, 0], BF0r = 1.8, with the overall evidence favouring favoured the null model (BF01 = 10.33). no association (fixed effect meta-analysis BF01 = 3.81). A A similar pattern of results holds for the meta-analytic similar result was obtained for the flanker effect in reaction correlations between the conflict effects on error rates time for the self-discipline composite, r = –.024 [–.062, and scores the Self-Control Scale (r = –.059 [–.190,

.015], BF0r = 1.12, with the overall evidence again favoring .056]), with a Bayes factor slightly favouring the null no association (fixed effect meta-analysis BF01 = 13.1). The model (BF01 = 3.93). Similar results were found for the data from the flanker effect in error rates provided a little self-discipline composite measure (r = –.051 [–.183, more evidence favoring a small association with the self- .057], BF01 = 4.87). control scale, r = –.060 [–.098, –.022], BF0r = .848, with the overall evidence favoring a small association (fixed effects General Discussion meta-analysis BF01 = .239). The association between error Combining 5 data-sets with over 2,600 participants, we rates and the self-discipline composite showed a similar found a consistent pattern of a small-to-zero relationship pattern in this study, r = –.046 [–.085, –.008], BF0r = .763, between self-reported self-control and two performance but the overall evidence was essentially equivocal (fixed measures of inhibition-related executive functioning (the 7 effects meta-analysis BF01 = 1.67). Stroop and flanker tasks). Most individual studies were consistent with no association between self-reported Bayesian Random Effects Meta-Analysis self-control and inhibition-related executive functions, We found converging lines of evidence for zero to small with only study 2c tending to support a small negative correlations between inhibition-related executive relationship. Further Bayesian meta-analyses—both fixed functions and reported self-control across five data-sets. and random effect—suggested little to no relationship We next conducted a Bayesian random-effects meta- between reported self-control and conflict effects in analysis to estimate the overall size of the correlations reaction time and choice error rates. This conclusion while accounting for potential heterogeneity of is supported by both Bayes factors supporting a null results across the data sets. To this end, four separate association and the corresponding small posterior meta-analyses (reaction time and choice-error rates estimates. separately for the Self-Control Scale and self-discipline composite) were conducted using the Bayesian What do these results mean for the science of self- statistical software Stan (Carpenter, et al., 2016; Stan control? Development Team, 2017), and we computed meta- Both questionnaire and inhibition-related performance analytic Bayes factors using bridge sampling (Gronau measures are established and widely accepted measures et al., 2017; Gronau, Singmann, & Wagenmakers, 2017). of self-control (de Ridder et al., 2012; Duckworth & Kern, These random effect meta-analyses were instantiated 2011; Hofmann et al., 2012; Molden et al., 2012; Inzicht & as hierarchical models where the individual studies can Gutsell, 2007). Indeed, it was once estimated that inhibition be related to each other through shared population- underlies 80%–90% of self-regulation (Baumeister et al., level parameters; see the supplementary materials for 1994; Baumeister, 2014). Given that inhibition is evoked details. as a mechanism underlying executive functions (Miyake Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Art. 39, page 11 of 16 Executive Function

Figure 5: Forest plots depicting the results of the random effects meta-analyses as a function of self-report measure (Self-Control Scale, Self-Discipline Composite) and the difference scores on the executive functioning tasks for both

reaction time and choice error rates. Error bars depict 95% confidence intervals. BF01 in sub-titles show evidence favouring the null for the meta-analytical (rho) effect across all four data-sets. Tau is a measure of heterogeneity of effect size, and was low for each of our meta-analyses.

& Friedman, 2012) and self-reported self-control (Tangney Orzechowski, Nowak, & Wójcik, 2018; Stahl et al., 2013) or et al., 2004), it might be expected that associations among conscientiousness (Flemming, Heintzelman, & Bartholow, these self-control measures should be sizeable and robust. 2016). We further extend these frequentist analyses by However, the current results suggest that questionnaire providing Bayesian support for a null relationship between measures of self-control and canonical performance these measures. The strongest interpretations of these measures of inhibition-related executive function are findings are: a) that theoretical and practical conclusions largely unrelated to each other. Importantly, we do not drawn using one measure (e.g., the Self-Control Scale) claim that our results invalidate one measure or the other. cannot be generalised to findings using the other (e.g., the Our results suggest that the Stroop and flanker tasks do Stroop task); and b) that there is little-to-no relationship not reflect the broader individual difference construct among these measures that are both commonly identified that is reflected in self-report scales, and, equally, that as operationalisations of the psychological construct of scores on the self-control scale are not analogous to the self-control. processes assessed by the Stroop and flanker tasks. Executive functioning paradigms such as the Stroop Our findings are consistent with previous studies that and Flanker task are designed specifically to assess control reported non-significant correlations between inhibition- over pre-potent impulses (cf., Botvinick et al., 2001; related executive functions and self-report measures Miyake et al., 2001). Evidence that these tasks assess the of impulsivity (Eisenberg et al., 2018; Nęcka, Gruszka, ability to overcome inappropriate impulses has been Art. 39, page 12 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function suggested both by behavioural and psychophysiological has emerged in self-control research, where two types of investigations (Kopp et al., 1996; Verleger et al., 2009). task (i.e., behavioural and self-report) that are commonly Similarly, the concept of inhibiting unwanted impulses identified as operationalisations of one psychological was central to the development of the Self-Control Scale construct (i.e., self-control), bear little-to-no empirical (Tangney et al., 2004), and many of the items in this scale relationship with each other. assess inhibition-like content (e.g., “I am good at resisting temptation”). Despite these logical similarities, it is Limitations and future directions reasonable to conclude from our results that self-report The current studies should be considered in light of some measures of control and laboratory tests of inhibition- important limitations and questions for future research. related executive functions assess different underlying First, we focused on a relatively constrained range of self- processes. These findings should be of great concern to report and performance measures that reflect canonical psychological scientists interested in self-control: Despite measures of self-reported self-control and inhibition- theoretical suggestions to the contrary (Hofmann et al., related executive functions. However, self-control and 2012), our results suggest that the field’s most widely used self-discipline can be measured both by longer and trait measure of self-control is uncorrelated with two of shorter-form scales (Goldberg, 1992; Gosling, Rentfrow, the field’s most commonly used executive functioning & Swann, 2003; Duckworth & Quinn, 2009), and also measures of self-control. by observer reports (Jackson et al., 2010; Moffitt et al., We should be clear that our results do not undermine 2011). Similarly, a wide range of reaction timed tasks the validity of the Self-Control Scale (or other self-report are available to measure inhibition-related executive measures like it) as a predictor of real-world outcomes. functions (e.g., antisaccade task, the Stop-signal task). Scale measures of self-control are consistently related Given this diversity of measures, it is clear that on-going to multiple indices of wellbeing (de Ridder et al., 2012; research should test the generalisability of our findings to Moffitt et al., 2011). In fact, initial validation work focused other measures in order to further explore the structure on associations between the Self-Control Scale and of self-control. relevant outcomes (e.g., less binge eating, alcohol abuse, As mentioned when introducing our measures, there better relationships, and good psychological adjustment), also exists a unity and diversity among established rather than exploring associations with other established measures of executive functioning (Miyake et al., 2000; self-control measures (Tangney et al., 2004). Instead, Miyake & Friedman, 2012). Future research could it appears that the Self-Control Scale and performance explore associations among self-report measures of measures of inhibition-related executive functioning self-control and other aspects of executive functioning, might be largely non-overlapping, despite these tasks such as updating and task switching. One recent both being framed as assessments of the ability to investigation indicated that the personality dimension of override impulses. conscientiousness was associated with shifting, but not The current results are illustrative of a general with the inhibition of prepotent responses or working conceptual and definitional ambiguity that may hinder memory updating (Fleming, Heintselman, & Bartholow, the empirical validation self-control as a psychological 2016). These findings suggest that rather than reflecting construct. Self-control is typically defined as the ability the ability to overcome impulses, conscientiousness might to override unwanted impulses (Baumeister et al., 2014), be more closely associated with control processes that and this ability is assessed using multiple measures. This allow people to flexibly respond to changing contexts and heterogeneity of assessment is particularly challenging for environments. Similar patterns might be expected with construct validity as there currently exists no gold-standard reported self-control given the high degree of empirical criterion measure against which the validity of other self- and conceptual overlap between self-control and control measures can be assessed. If I hypothesize that conscientiousness (Roberts et al., 2014). Such a finding measure A (e.g., handgrip strength, the Stroop task, or a would be consistent with theoretical perspectives in new self-report scale) is a measure of self-control, there is which self-control involves adaptively managing priorities no agreed benchmark assessment of self-control that I can between activities and goals (Inzlicht et al., 2014). correlate with handgrip strength. Following this logic, it Related to the diversity of self-control measures, the is impossible to conclude from our results if the observed exact role of an inhibitory mechanism (vs. selective lack of significant correlations points to validity issues attention/attentional control) has been questioned in with any one of our measures, or if there are broader regard to many behavioural measures of inhibition- problems with the construct space of self-control (cf., related executive functioning (Egner & Hirsch, 2005). Cronbach & Meehl, 1955). Furthermore, while some analyses have suggested that Conclusions complementary to our own were drawn a common latent factor unites measures of inhibitory in recent analyses in which task/performance based executive functioning (Miyake et al., 2000; Miyake & measures of self-regulation (e.g., go/no-go task, delay Friedman, 2012), other research indicates a lack of discounting, and 35 other behavioural tasks) predicted convergence between these measures (Egner, 2008). As other task-based measures, but were not associated with each of our studies assesses the link between reported 27 self-report based measures of self-regulation (and self-control and one executive functioning task, we are vice versa; Eisenberg et al., 2018). Together with our own not able to assess links between introspective reports of results, these studies indicate a so-called jingle-fallacy the ability to control impulses and any latent executive Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Art. 39, page 13 of 16 Executive Function functioning factor that is common to the Stroop and majority of conditions (all rs < –.144). There was a small Flanker tasks. Future work should explore this possibility. negative correlation between the Stroop effect and Another limitation is the previously mentioned reliability error rates during the majority compatible condition paradox (Hedge et al., 2017): Robust cognitive tasks do not and the self-discipline composite (r = –.167, p = .044). produce reliable individual differences, making their use However, this correlation should be interpreted with as trait-level correlational tools problematic. Behavioural some caution given both the high p-value and the tasks only become well established when between-subject considerable number of comparisons undertaken in variability is low; however, low between-subject variability this analysis. hurts reliability for individual differences and deflates 2 We replicated our Bayesian analyses using the correlations. One potential solution, albeit controversial, non-differences scores (i.e., RT and error-rates on is to disattenuate correlations undermined by low compatible and incompatible trials), and these are reliabilities (Muchinsky, 1996). When we disattenaute presented in the online supplementary materials. the meta-analytic correlations, which range from r = The results for the non-difference scores were mixed. .03–06, they increase somewhat to r = .05–.08. Though In reaction times, we tended to see small positive slightly increased, this disattenuation suggests that the correlations between reported self-control and correlations between reported self-control and inhibition- performance on both compatible and incompatible related executive function is not low because of poor trials, whereas smaller negative correlations were reliabilities, but because they are actually uncorrelated, observed for error rates on the same trial types. This with less than 0.7% in shared variance. Furthermore, it pattern of results is more consistent with reported should be noted that while non-difference scores from self-control correlating with a slight speed-accuracy the Stroop task (e.g., mean reaction time on incompatible trade-off (slower overall RT and reduced error-rates), trials) demonstrated good reliability, Bayesian correlations rather than an increase in inhibition-related executive supported a null relationship with reported self-control functioning. (see online supplemental materials). Together with the 3 A sensitivity analysis using other reasonable choices of disattenuated correlations, these results suggest that the priors revealed no concerns that affect the conclusions current findings are unlikely a direct result of the poor of the present analyses. reliability in the executive functioning tasks. 4 One previous factor analytical investigation suggested that the brief Self-Control Scale can be further divided Conclusion into subscales with of items reflecting initiatory and The current findings are consistent with a null relationship inhibitory self-control (de Ridder, de Boer, Lugtig, between performance measures of inhibition-related Bakker, & van Hooft, 2011). Bayesian correlations executive functioning (the Stroop and flanker tasks), conducted in JASP supported a null relationship the Self-Control Scale, and related measures of self- between the inhibition related items and the Stroop discipline (Grit, Conscientiousness). Our results highlight effect in RT(r = .022, BF01 = 14.93) and error-rates (r = empirical and conceptual problems with self-control .077, BF01 = 23.86). as a psychological construct, where widely used and 5 All results were identical when no exclusions were established measures of self-control are largely unrelated applied (see supplemental materials, https://osf.io/ to each other. jws4x/). 6 Studies 2b and 2c had high rates of exclusion. While Data Accessibility Statement we cannot tell exactly why exclusion rates were so high The data and scripts are available on our OSF page in these studies, we think that the poor data quality (https://osf.io/8etus/). likely arose from the use of online survey companies in which participant panels likely had less experience Notes with behavioural tasks than Mturk participants. 1 In addition to the classic Stroop effect, we manipulated It is important to note, however, that our overall the proportion of congruent to incongruent trials in conclusions were already supported from the results the Stroop task (cf., Logan & Zbrodoff, 1979). However, of study 1 and 2a without including studies 2b–c. this manipulation was not modelled as it is not central However, we opted to include the later studies (after to the present work. The task comprised 576 trials removing poorly performing participants) for the sake divided equally into 9 blocks. Blocks were divided into of transparent reporting. groups of three that varied in the ratio of compatible 7 Bayesian correlations conducted in JASP supported to incompatible trials (75% compatible/25% a null relationship between the inhibition related incompatible; 50% compatible/50% incompatible; items of the brief Self-Control Scale (cf., de Ridder et

25% compatible; 75% incompatible). Condition order al., 2011) and the Stroop effect in RT(r = –.109, BF01 =

was counterbalanced between participants, while 2.041) and error-rates (r = –.052, BF01 = 5.581). the three blocks with equal proportions were always presented together. Proportion congruency conditions Funding Information were collapsed for all analyses. It should be noted that Alexander Etz received funding from the National Science neither the self-control scale nor the self-discipline Foundation Graduate Research Fellowship Program composite measure predicted the Stroop effect in the #DGE-1321846, and grant #1534472 from the National Art. 39, page 14 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function

Science Foundation’s Methods, Measurements, and de Ridder, D. T., de Boer, B. J., Lugtig, P., Bakker, A. B., Statistics panel. & van Hooft, E. A. (2011). Not doing bad things is not equivalent to doing the right thing: Distinguishing Competing Interests between inhibitory and initiatory self-control. The authors have no competing interests to declare. Personality and Individual Differences, 50(7), 1006– 1011. DOI: https://doi.org/10.1016/j.paid.2011.01.015 Author Contributions de Ridder, D. T., Lensvelt-Mulders, G., Finkenauer, C., Saunders, Milyavskaya, Randles, and Inzlicht conceived Stok, F. M., & Baumeister, R. F. (2012). Taking stock the study. Saunders, Randles, Inzlicht and Milyavskaya of self-control a meta-analysis of how trait self-control oversaw the design and data-collection for each of the relates to a wide range of behaviors. Personality and studies. Data was structured and aggregated by Saunders Social Psychology Review, 16, 76–99. DOI: https://doi. and Randles. Data analyses were conducted by Saunders, org/10.1177/1088868311418749 Milyavskaya, and Etz. Saunders drafted the initial Duckworth, A., & Gross, J. J. (2014). Self-control and grit manuscript and all authors provided critical feedback and related but separable determinants of success. Current edits on the manuscript. Directions in Psychological Science, 23, 319–325. DOI: https://doi.org/10.1177/0963721414541462 References Duckworth, A. L., & Kern, M. L. (2011). A meta-analysis Allan, J. L., Johnston, M., & Campbell, N. (2010). of the convergent validity of self-control measures. Unintentional eating. What determines goal- Journal of Research in Personality, 45, 259–268. DOI: incongruent chocolate consumption? Appetite, https://doi.org/10.1016/j.jrp.2011.02.004 54(2), 422–425. DOI: https://doi.org/10.1016/j. Duckworth, A. L., Peterson, C., Matthews, M. D., & appet.2010.01.009 Kelly, D. R. (2007). Grit: Perseverance and passion Allom, V., Panetta, G., Mullan, B., & Hagger, M. S. for long-term goals. Journal of Personality (2016). Self-report and behavioural approaches to the and Social Psychology, 92 DOI: https://doi. measurement of self-control: Are we assessing the org/10.1037/0022-3514.92.6.1087 same construct? Personality and Individual Differences, Duckworth, A., & Seligman, M. E. (2006). Self-discipline 90, 137–142. DOI: https://doi.org/10.1016/j. outdoes IQ in predicting academic performance of paid.2015.10.051 adolescents. Psychological Science, 16, 939–944. DOI: Baumeister, R. F. (2014). Self-regulation, ego https://doi.org/10.1111/j.1467-9280.2005.01641.x depletion, and inhibition. Neuropsychologia, Duckworth, A. L., & Quinn, P. D. (2009). Development 65, 313–319. DOI: https://doi.org/10.1016/j. and validation of the Short Grit Scale (GRIT–S). Journal neuropsychologia.2014.08.012 of Personality Assessment, 91, 166–174. DOI: https:// Baumeister, R. F., Heatherton, T. F., & Tice, D. M. (1994). doi.org/10.1080/00223890802634290 Losing control: How and why people fail at self- Edmonds, G. W., Bogg, T., & Roberts, B. W. (2009). regulation. Academic press. Are personality and behavioral measures of impulse Bogg, T., & Roberts, B. W. (2004). Conscientiousness control convergent or distinct predictors of health and health-related behaviors: A meta-analysis of behaviors? Journal of Research in Personality, 43, 806– the leading behavioral contributors to mortality. 814. DOI: https://doi.org/10.1016/j.jrp.2009.06.006 Psychological Bulletin, 130, 887. DOI: https://doi. Egner, T. (2008). Multiple conflict-driven control org/10.1037/0033-2909.130.6.887 mechanisms in the human brain. Trends in Cognitive Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Sciences, 12, 374–380. DOI: https://doi.org/10.1016/j. Cohen, J. D. (2001). Conflict monitoring and cognitive tics.2008.07.001 control. Psychological Review, 108, 624–652. DOI: Egner, T., & Hirsch, J. (2005). Cognitive control mechanisms https://doi.org/10.1037/0033-295X.108.3.624 resolve conflict through cortical amplification of task- Carpenter, B., Gelman, A., Hoffman, M., Lee, relevant information. Nature Neuroscience, 8, 1784– D., Goodrich, B., Betancourt, M., Riddell, 1790. DOI: https://doi.org/10.1038/nn1594 A., et al. (2016). Stan: A probabilistic programming Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise language. Journal of Statistical Software, 20, 1–37. letters upon the identification of a target letter in a Crump, M. J., McDonnell, J. V., & Gureckis, T. M. (2013). nonsearch task. Attention, Perception, & Psychophysics, Evaluating Amazon’s Mechanical Turk as a tool for 16, 143–149. DOI: https://doi.org/10.3758/ experimental behavioral research. PloS One, 8, e57410. BF03203267 DOI: https://doi.org/10.1371/journal.pone.0057410 Etz, A., & Vandekerckhove, J. (2016). A Bayesian Davidson, M. C., Amso, D., Anderson, L. C., & perspective on the reproducibility project: Diamond, A. (2006). Development of cognitive Psychology. PLoS One, 11(2), e0149794. DOI: https:// control and executive functions from 4 to 13 doi.org/10.1371/journal.pone.0149794 years: Evidence from manipulations of memory, Etz, A., & Vandekerckhove, J. (In press). Introduction inhibition, and task switching. Neuropsychologia, to Bayesian Inference for Psychology. Psychonomic 44, 2037–2078. DOI: https://doi.org/10.1016/j. Bulletin & Review. DOI: https://doi.org/10.3758/ neuropsychologia.2006.02.006 s13423-017-1262-3 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Art. 39, page 15 of 16 Executive Function

Fleming, K. A., Heintzelman, S. J., & of the Behavioral Indicators of Conscientiousness Bartholow, B. D. (2016). Specifying associations (BIC). Journal of Research in Personality, 44, 501–511. between conscientiousness and executive functioning: DOI: https://doi.org/10.1016/j.jrp.2010.06.005 Mental set shifting, not prepotent response inhibition John, O. P., & Srivastava, S. (1999). The Big Five trait or working memory updating. Journal of Personality, taxonomy: History, measurement, and theoretical 84, 348–360. DOI: https://doi.org/10.1111/ perspectives. Handbook of personality: Theory and jopy.12163 research, 2, 102–138. Fraley, R. C., & Vazire, S. (2014). The N-pact factor: Johnson, J. A. (2014). Measuring thirty facets of the Evaluating the quality of empirical journals with Five Factor Model with a 120-item public domain respect to sample size and statistical power. PloS One, inventory: Development of the IPIP-NEO-120. Journal 9, e109019. DOI: https://doi.org/10.1371/journal. of Research in Personality, 51, 78–89. DOI: https://doi. pone.0109019 org/10.1016/j.jrp.2014.05.003 Gailliot, M. T., Baumeister, R. F., DeWall, C. N., Kern, M. L., & Friedman, H. S. (2008). Do conscientious Maner, J. K., Plant, E. A., Tice, D. M., Schmeichel, individuals live longer? A quantitative review. B. J., et al. (2007). Self-control relies on glucose Health Psychology, 27, 505. DOI: https://doi. as a limited energy source: Willpower is more org/10.1037/0278-6133.27.5.505 than a metaphor. Journal of Personality and Kopp, B., Rist, F., & Mattler, U. W. E. (1996). N200 in the Social Psychology, 92, 325–336. DOI: https://doi. flanker task as a neurobehavioral tool for investigating org/10.1037/0022-3514.92.2.325 executive control. Psychophysiology, 33, 282–294. DOI: Goldberg, L. R. (1990). An alternative “description of https://doi.org/10.1111/j.1469-8986.1996.tb00425.x personality”: The big-five factor structure. Journal Logan, G. D., & Zbrodoff, N. J. (1979). When it helps to be of Personality and Social Psychology, 59, 1216. DOI: misled: Facilitative effects of increasing the frequency https://doi.org/10.1037/0022-3514.59.6.1216 of conflicting stimuli in a Stroop-like task. Memory & Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A Cognition, 7, 166–174. DOI: https://doi.org/10.3758/ very brief measure of the Big-Five personality domains. BF03197535 Journal of Research in Personality, 37, 504–528. DOI: Ly, A., Etz, A., Marsman, M., & Wagenmakers, https://doi.org/10.1016/S0092-6566(03)00046-1 E. -J. (2017). Replication Bayes factors from Evidence Gronau, Q. F., Sarafoglou, A., Matzke, D., Ly, A., Updating. PsyArXiv preprints. Retrieved from: https:// Boehm, U., Marsman, M., Steingroever, H., et al. osf.io/preprints/psyarxiv/u8m2s/. DOI: https://doi. (2017). A tutorial on bridge sampling. arXiv preprint org/10.31234/osf.io/u8m2s arXiv:1703.05984. Mischel, W., Shoda, Y., & Rodriguez, M. I. (1989). Delay Gronau, Q. F., Singmann, H., & Wagenmakers, E. -J. of gratification in children. Science, 244, 933–938. (2017). Bridgesampling: Bridge sampling for marginal DOI: https://doi.org/10.1126/science.2658056 likelihoods and Bayes factors. Retrieved from: https:// Miyake, A., & Friedman, N. P. (2012). The nature and github.com/quentinngronau/bridgesampling (R organization of individual differences in executive package version 0.2-2). functions: Four general conclusions. Current Directions Hall, P. A. (2012). Executive control resources and in Psychological Science, 21, 8–14. DOI: https://doi. frequency of fatty food consumption: Findings from an org/10.1177/0963721411429458 age-stratified community sample. Health Psychology, Miyake, A., Friedman, N. P., Emerson, M. J., 31, 235. DOI: https://doi.org/10.1037/a0025407 Witzki, A. H., Howerter, A., & Wager, T. D. (2000). Hedge, C., Powell, G., & Sumner, P. (2017). The reliability The unity and diversity of executive functions and their paradox: Why robust cognitive tasks do not produce contributions to complex “frontal lobe” tasks: A latent reliable individual differences. Behavior Research variable analysis. Cognitive Psychology, 41, 49–100. Methods, 1–21. DOI: https://doi.org/10.1006/cogp.1999.0734 Hofmann, W., Schmeichel, B. J., & Baddeley, A. D. Moffitt, T. E., Arseneault, L., Belsky, D., Dickson, N., (2012). Executive functions and self-regulation. Trends Hancox, R. J., Harrington, H., Sears, M. R., et al. in Cognitive Sciences, 16, 174–180. DOI: https://doi. (2011). A gradient of childhood self-control predicts org/10.1016/j.tics.2012.01.006 health, wealth, and public safety. Proceedings of the Inzlicht, M., & Gutsell, J. N. (2007). Running on National Academy of Sciences, 108, 2693–2698. DOI: empty: Neural signals for self-control failure. https://doi.org/10.1073/pnas.1010076108 Psychological Science, 18, 933–937. DOI: https://doi. Molden, D. C., Hui, C. M., Scholer, A. A., Meier, B. P., org/10.1111/j.1467-9280.2007.02004.x Noreen, E. E., D’Agostino, P. R., & Martin, V. (2012). Inzlicht, M., Schmeichel, B. J., & Macrae, C. N. (2014). Motivational versus metabolic effects of carbohydrates Why self-control seems (but may not be) limited. Trends on self-control. Psychological Science, 23, 1137–1144. in Cognitive Sciences, 18, 127–133. DOI: https://doi. DOI: https://doi.org/10.1177/0956797612439069 org/10.1016/j.tics.2013.12.009 Muchinsky, P. M. (1996). The correction for Jackson, J. J., Wood, D., Bogg, T., Walton, K. attenuation. Educational and Psychological E., Harms, P. D., & Roberts, B. W. (2010). What do Measurement, 56, 63–75. DOI: https://doi. conscientious people do? Development and validation org/10.1177/0013164496056001004 Art. 39, page 16 of 16 Saunders et al: Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function

Nigg, J. T. (2001). Is ADHD a disinhibitory disorder? Tangney, J. P., Baumeister, R. F., & Boone, A. L. (2004). Psychological bulletin, 12, 571. DOI: https://doi. High self‐control predicts good adjustment, less org/10.1037/0033-2909.127.5.571 pathology, better grades, and interpersonal success. Pronk, T. M., Karremans, J. C., & Wigboldus, D. H. Journal of Personality, 72, 271–324. DOI: https://doi. (2011). How can you resist? Executive control helps org/10.1111/j.0022-3506.2004.00263.x romantically involved individuals to stay faithful. Verleger, R., Kuniecki, M., Möller, F., Fritzmannova, M., Journal of personality and social psychology, 100, 827. & Siebner, H. R. (2009). On how the motor cortices DOI: https://doi.org/10.1037/a0021993 resolve an inter‐hemispheric response conflict: Roberts, B. W., Jackson, J. J., Fayard, J. V., Edmonds, G., & An event‐related EEG potential‐guided TMS Meints, J. (2009). Conscientiousness. In: Leary, M. R., study of the flankers task. European Journal of & Hoyle, R. H. (Eds.), Handbook of individual differences Neuroscience, 30, 318–326. DOI: https://doi. in social behaviour, 369–381. New York, NY, US: org/10.1111/j.1460-9568.2009.06817.x Guilford Press. Von Culin, K. R., Tsukayama, E., & Duckworth, A. L. Roberts, B. W., Lejuez, C., Krueger, R. F., Richards, (2014). Unpacking grit: Motivational correlates of J. M., & Hill, P. L. (2014). What is conscientiousness and perseverance and passion for long-term goals. The how can it be assessed? Developmental Psychology, 50, Journal of Positive Psychology, 9, 306–312. DOI: 1315. DOI: https://doi.org/10.1037/a0031109 https://doi.org/10.1080/17439760.2014.898320 Stan Development Team. (2017). Stan: A C++ library Wagenmakers, E. J., Verhagen, J., & Ly, A. (2015). How to for probability and sampling. Online: http://mc-stan. quantify the evidence for the absence of a correlation. org. Behavior Research Methods, 48, 413–426. DOI: https:// Streeter, C. C., Terhune, D. B., Whitfield, T. H., doi.org/10.3758/s13428-015-0593-0 Gruber, S., Sarid-Segal, O., Silveri, M. M., Renshaw, Wöstmann, N. M., Aichert, D. S., Costa, A., Rubia, K., P. F., et al. (2008). Performance on the Stroop Möller, H. J., & Ettinger, U. (2013). Reliability and predicts treatment compliance in cocaine-dependent plasticity of response inhibition and interference individuals. Neuropsychopharmacology, 33, 827. DOI: control. Brain and Cognition, 81, 82–94. DOI: https:// https://doi.org/10.1038/sj.npp.1301465 doi.org/10.1016/j.bandc.2012.09.010

Peer review comments The author(s) of this paper chose the Open Review option, and the peer review comments are available at: http://doi.org/ 10.1525/collabra.134.pr

How to cite this article: Saunders, B., Milyavskaya, M., Etz, A., Randles, D., & Inzlicht, M. (2018). Reported Self-control is not Meaningfully Associated with Inhibition-related Executive Function: A Bayesian Analysis. Collabra: Psychology, 4(1): 39. DOI: https://doi.org/10.1525/collabra.134

Editor/Senior Editor: Simine Vazire

Submitted: 19 January 2018 Accepted: 24 September 2018 Published: 20 November 2018

Copyright: © 2018 The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.

Collabra: Psychology is a peer-reviewed open access

journal published by University of California Press. OPEN ACCESS