Running head: GENERAL MISCONCEPTIONS AND THINKING STYLES 1
A new measure of general misconceptions: Relations with rational, intuitive, critical thinking,
and monetary incentives
Marle Wortelboer
Master thesis Economic Psychology
Tilburg School of Social and Behavioral Sciences
1st Supervisor: W.W.A. Sleegers
2nd Supervisor: F. van Leeuwen
March 22th 2019
GENERAL MISCONCEPTIONS AND THINKING STYLES 2
Abstract
Past research indicated several issues regarding measurements of (psychological) misconceptions. In this paper, it was aimed to develop a new Test of General Knowledge and
Misconceptions (TOGKAM) based on recent suggestions to address previous limitations of
Bensley et al. (2014). In order to validate this TOGKAM, relations with rational- and intuitive thinking styles and critical thinking dispositions were tested. It was hypothesized that an
Intuitive-Experiential thinking style was positively correlated with TOGKAM scores, whereas a
Rational-Analytic thinking style and Critical Thinking dispositions were negatively correlated with TOGKAM scores. In addition, it was examined whether monetary incentives decreased the endorsement of misconceptions. Hypothesized was, that incentivized participants were more likely to use rational and analytic thinking styles and subsequently endorsed fewer misconceptions. Results of this study supported all the hypotheses.
Keywords: misconceptions, intuitive- and rational thinking styles, critical thinking, dual- processing.
GENERAL MISCONCEPTIONS AND THINKING STYLES 3
A new measure of general misconceptions: Relations with rational, intuitive, critical thinking,
and monetary incentives
Misconceptions are widely held beliefs that contradict established scientific evidence
(Gardner & Brown, 2013; Sak, 2011). Misconceptions, or sometimes called myths, are stable and often deeply rooted beliefs about the world. Common misconceptions such as ‘bats are blind and navigate through echolocation’ or ‘the Chinese wall is the only man-made structure visible from space with the naked eye, are often part of ‘folk wisdom’ (McCutcheon, 1991). Despite the amount of research on the topic, implying sufficient knowledge is available regarding misconceptions, more information concerning the prevalence, origin, prevention and previous test assessments is required.
Several studies demonstrated that misconceptions appear within all age groups and do not only prevail among lay people but also among experts of competence, or intelligence, such as with cardiologists and other healthcare workers (Cross, 2005; Sternberg, 1996; Winner, 1996).
However, while misconceptions can be found throughout the entire population and within each field of knowledge, their exact prevalence rates are not well known (Garrett & Fisher, 1926;
Crowe & Miura, 1995; Swami, Stieger, Pietschnig, Nader, & Voracek, 2012). For instance, within the field of psychology, the prevalence rate of psychological misconceptions is found to range from 40–70% among students (Arntzen, Lokke, Lokke, & Eilertsen, 2010). Moreover, a substantial increase was found from student’s myth recognition scores, from 38.5% to 66.3%
(Kowalski & Taylor, 2009). Lastly, Furnham and Hughes (2014) showed that 35% of the 249 psychological myths were believed by over 50% of the respondents to be true. In conclusion, there is argued that these discrepancies in the prevalence estimations are assumed to be partly due to methodological issues (Bensley et al., 2014).
GENERAL MISCONCEPTIONS AND THINKING STYLES 4
It is necessary to prevent misconceptions because they tend to pose a threat to society.
First, misconceptions form obstacles to learning by impeding the acquisition of new and accurate knowledge (Gardner & Brown, 2013). This is particularly an issue for teachers as students already possess misconceptions before attending science courses (Mestre, 2001; Hammer, 1996).
Secondly, misconceptions consist of misleading information which could influence people to make wrong decisions in their everyday lives (Lilienfeld, Lynn, Ruscio, & Beyerstein, 2011).
For example, people’s inaccurate belief that vaccinations cause autism, prevent them from seeking vaccinations for themselves and their children. As a result, the vaccination-rate against contagious diseases such as measles had strongly decreased (Madsen et al., 2002). In addition, even among experts misconceptions are prevalent with disastrous consequences. For example, it is found that when healthcare workers hold misconceptions as ‘exercise is ‘bad’ for the heart’, which could eventually lead to patients having a slower recovery and reduced rate of return to work (Lin, Furze, Spilsbury, & Lewin, 2008; Angus et al., 2012). In summary, misconceptions could prompt serious challenges for laypersons, policy makers, journalists, attorneys, and others
(Bensley & Lilienfeld, 2017).
Psychological Origins of Misconceptions
One important source for misconceptions consists of a variety of psychological dispositions. In other words, an individual's personal attitudes, motivations, and opinions can shape their beliefs. Moreover, once misinformation becomes fixed in a person's knowledge base, new information is frequently distorted or ignored, resulting in strengthening or the retention of the inaccurate beliefs (Taylor & Kowalski, 2004). Therefore, strongly held, but incorrect beliefs are particularly difficult to change (Vosniadou, 2001; Lewandowsky, Ecker, Seifert, Schwarz. &
Cook, 2012).
GENERAL MISCONCEPTIONS AND THINKING STYLES 5
Lewandowsky et al. (2012) argued that awareness, motivation or the availability of cognitive sources are required to be able to reject incorrect information. There are various psychological traits, such as habitual patterns of behavior, thoughts, and emotions that affect whether misconceptions are adopted or rejected. In terms of misconception, critical thinking is the most obvious factor in affecting the endorsement. It is the disposition and ability to retrieve and use information to evaluate knowledge claims. For example, Taylor and Kowalski (2004) showed that students who actively employed critical thinking (CT) skills to evaluate newly encountered information were less susceptible to misconceptions. However, people often fail in signaling difficulties and activate critical thinking dispositions due to people often rely on heuristics (Bensley & Lilienfeld, 2016).
Moreover, the acceptance of misconceptions versus scientifically supported beliefs relates to differences in information processing. The dual process theory posits that people have an intuitive system that automatically learns from experience and is largely unconscious (system
1), and a rational system for engaging in verbal reasoning that is conscious, deliberate, and analytic (system 2), (Evans & Stanovich, 2013). For people who tend to endorse psychological misconceptions, an intuitive process may be dominant, whereas, for those who hold fewer misconceptions, a rational process may be dominant (Bensley et al., 2014). To illustrate this, the following misconception can be used ‘Ostriches stick their heads in the sand to hide when they sense danger’. Here, it is possible that people simply recognize this information and as a somewhat social proof argumentation, intuitively accept the misconception to be true. However, there is also a possibility that people more deeply analyze the information and conclude that there is no benefit for ostriches sticking their head in the sand, and reject this information to be
GENERAL MISCONCEPTIONS AND THINKING STYLES 6 true. This illustrates that these two systems of information processing affect the determination of misconception endorsement.
In essence, within misconception research, the inference of the human brain processes information in two different systems should be taken into account. Pacini and Epstein (1999) came up with a Cognitive-Experiential self-theory as a dual-process model of cognition. They suggested that human behavior is controlled by two distinct information processing systems (1) a
Rational-Analytic system that is conscious, controlled, logic-based, and largely affect free, and
(2) An Intuitive-Experiential system that is predominantly preconscious, automatic, and tied to intuition and affect. Since these systems are closely related to the dual-process approach, similar assumptions regarding a link with misconception endorsement could be established. From these approaches, explanations regarding the prevalence of misconceptions could be provided. First, it is argued that people’s intuition can interfere with the process of determining information correct or incorrect, resulting in more errors (Myers, 2002). Therefore, effortful reflective thinking is required in order to be able to reject these misconceptions.
Measurement Issues
A full understanding of misconceptions relies on a reliable and valid assessment of misconceptions. However, despite the importance of misconceptions research, popular measures of misconceptions suffer from severe methodological problems (Crowe & Miura, 1995; Swami et al., 2012).
One methodological issue refers to the response format. Misconception measures often consist of a true/false (T/F) format. Participants have to indicate whether they believe a particular misconception is either true or false. However, the T/F format is problematic since (1) responses could be influenced by an acquiescence bias (yea-saying bias), (2) the format of T/F often
GENERAL MISCONCEPTIONS AND THINKING STYLES 7 constraints responses to be either completely true or false although misconceptions are not always completely false or correct, and (3) when the correct answers in the T/F format are always keyed false it could make it easier to guess correctly.
There are several reasons to think these methodological shortcomings lead to an over- or underestimation of misconception rates. First, the acquiescence bias is problematic because people, when in doubt, have a tendency to agree with information they encounter, while in fact, they should indicate they do not know whether information is correct or not. Due to this acquiescence bias, higher rates of misconceptions will be obtained (Podsakoff, MacKenzie, Lee,
& Podsakoff, 2003). Relatedly, misconception statements are often somewhat ambiguous, which means that depending on the interpretation they could be true or false. So, when participants have an acquiescence bias, they might be again more favorable towards the interpretation they can agree with (Hughes, Lyddy, & Kaplan, 2013). Third, when patterns of correct answers can be recognized, in this case by recognizing that each item refers to a misconception, participants can easily give correct answers and judging each item as false (Bensley et al., 2014). Lastly, the
T/F format does not allow respondents to indicate their certainty about a given answer, which complicates the assessment of the veracity of participants’ knowledge.
With the purpose to remedy these measurement issues, Bensley, Lilienfeld, and Powell
(2014) developed the Test of Psychological Knowledge and Misconceptions (TOPKAM). They addressed both, the temporary status of knowledge and the fact that knowledge is rarely completely true or false by presenting two statements at the same time. A common misconception was paired with an evidence-based response option contradicting the misconception. For example, the misconception ‘People use about 10% of their brains’, was paired with the evidence-based response ‘People use the entirety of their brains, but not all at
GENERAL MISCONCEPTIONS AND THINKING STYLES 8 once. Here, participants were asked to base their answer on the option that was “most true”. Due to the design of two prepositions presented at each question, the issue of ambiguity was resolved.
They added a confidence rating after each question in order to take the effect of the acquiescence bias on actual scores into account. Participants need to rate the certainty of the correctness of their answers after each question. Moreover, all the items were randomized with an equal number of ‘a’ and ‘b’ response options correct.
Bensley et al. (2014) conducted multiple studies, in which they validated their TOPKAM measure. First, by assessing the test-retest reliability whereby they found a reasonable test- reliability. Next, they validated the TOPKAM by linking it to intuitive and rational thinking styles and critical thinking dispositions (Shiloh, Salton, & Sharabi, 2002; Kahneman, Slovic,
Slovic, & Tversky, 1982). They found modest relationships with intuitive and experiential thinking styles, and adequate internal consistency. Their results showed that being more disposed to reflect on psychological claims in combination with greater critical thinking skills predict more successful rejection of psychological misconceptions.
A General Measure of Misconceptions
The purpose of the present study is to design a new test of general misconceptions and report on the development and initial validation. Different from the study of Bensley et al.
(2014), we aim to design a measurement of misconceptions from a variety of domains, rather than only psychology-related misconceptions. A reason to shift from a psychological to a more general focus on knowledge and misconceptions is based on findings suggesting that endorsement of misconceptions is dependent on someone’s field of knowledge (Hughes, Lyddy,
& Kaplan, 2013). Hence, people tend to be more likely to agree with common misconceptions than with topic-specific misconceptions. Therefore, it could be assumed that general
GENERAL MISCONCEPTIONS AND THINKING STYLES 9 misconceptions are even more prevalent than topic-related misconceptions, such as psychological misconceptions. Secondly, the extent to which psychological misconceptions are related to critical thinking and intuitive thinking should also apply to general misconceptions, yet this has not been empirically demonstrated as most research on misconceptions has relied on psychological misconceptions only. In other words, shifting from a specific to more general misconceptions measurement will provide relevant insights.
The validity of the new measurement will be assessed by administering the new test with measures of thinking dispositions such as critical thinking, intuitive, and rational thinking styles.
Eventually, assessment of these relationships could provide us information on how to predict prevalence ratings regarding general misconceptions.
Moreover, next to thinking styles, Lewandowsky et al. (2012) argued that motivation could also serve as a predictor of accepting misconceptions. It is assumed that the ease for people to accept misconceptions due to intuitive thinking, can be diminished by motivating people to think critically by rewarding them based on their performance. A heretofore untested application of this idea is to use monetary incentives to improve on the validity testing of misconception measurements. Participants who are financially incentivized should engage in more effortful and critical thinking, compared to participants whose answers have no true consequences for themselves, at least in terms a financial payoff.
In summary, as part of creating a valid assessment, the TOGKAM is expected to reveal negative relationships with rational-analytic thinking styles and Critical thinking dispositions, and positive relationships with intuitive-experiential thinking styles. Participants who use their rational-analytic system (system 2) or have critical thinking dispositions endorse few misconceptions (Hypothesis 1a, 1b), consistent with dual process theory. Additionally, again
GENERAL MISCONCEPTIONS AND THINKING STYLES 10 consistent with the dual-process theory, it is expected that when participants use their intuitive- experiential system (system 1) endorse more misconceptions (Hypothesis 2). Lastly, it is hypothesized that monetary incentives will elicit more rational-analytic thinking styles and therefore result in a decrease of misconception endorsement (Hypothesis 3).
GENERAL MISCONCEPTIONS AND THINKING STYLES 11
Method
Design
Participants filled in a series of questionnaires by which relationships between thinking styles and susceptibility to common general misconceptions could be explored. In addition, participants were randomly assigned to one of two conditions; a non-incentive condition where participants received merely a fixed reward (course credits), and an incentive condition, where next to course credits, monetary rewards were provided based on participants’ performance on the TOGKAM.
Participants
During the first two weeks of February 2019, 445 participants participated in the experiment. The participants were recruited via an online portal, only available to first years
Psychology students of Tilburg University. After excluding some participants because they had used the internet to search for answers (n = 3) or they had reported being poorly skilled in the
English language (n = 2), the final sample consisted of 440 participants of which 69 males and
370 females with ages ranging from 18 to 44 years. A G*power analysis showed that, with this sample size and a power of 80 percent, an effect size of .12 can be detected.
Procedure and Materials
Incentive and non-incentive conditions
The participants were welcomed in the laboratory of the Tilburg University, by one of two female experimenters. They were asked to take place in an individual cabin in which they had to fill in a questionnaire on a computer. Prior to the general misconception test, a consent form was presented to the participants. Additionally, participants either received instructions corresponding to the condition they were randomized to. For example, to participants in the
GENERAL MISCONCEPTIONS AND THINKING STYLES 12 incentive-condition, the following was presented “Depending on your performance, you can win a monetary reward. After the completion of this study, we will select the top five participants who answered most questions correctly. Each of those participants will receive a monetary reward of €20.” The participants within the incentive condition were asked to indicate their e- mail address if they were willing to win the €20,-.
Test of General Knowledge and Misconceptions
Next, participants had to carry out the new developed Test of General Knowledge and
Misconceptions (TOGKAM) that, to ensure content validity, consisted of 25 items and 5 filler items (Bensley et al., 2014). These filler items were added to the test so that it became less conspicuous to the participants what the test actually measured. The 25 misconceptions used in the TOGKAM were selected in order to represent general knowledge, organized within five categories; human body, animals, science, history and society and were retrieved from a website and books on general misconceptions (https://geekwrapped.com; Maanen, 2004; Boel, 2013). All items of the TOGKAM were randomized with an equal number of ‘a’ and ‘b’ response options correct.
Similar to the setup of the TOPKAM, the test was written in everyday language combined with scientific language while avoiding jargon (Bensley et al., 2014). Moreover, participants received the instruction to choose the option that was most likely to be true, according to their knowledge. An example of a question was “Which option do you believe is most likely to be true?” followed by two response options: “a. ‘Bats are blind and can only navigate through echolocation’, b. ‘Bats have eyes and are capable of sight’”. After each item, participants were asked to rate their certainty of correctness, by using a 5-point Likert scale ranging from 1 = not at all certain to 5 = completely certain. The total scores on the general
GENERAL MISCONCEPTIONS AND THINKING STYLES 13 misconception test ranged from 0 to 25 correct, in which higher scores reflect a greater endorsement of misconceptions. After completing the test, participants were asked a manipulation check of the incentive manipulation by rating to what extent they were motivated to perform well on the test.
Measures of Thinking Styles and Dispositions
Next, thinking styles and critical thinking dispositions were measured by first, using the
Rational-Analytic scale (20 items) and the Intuitive-Experiential scale (20 items), both acquired from the Rational-Experiential Inventory (REI-40) of Pacini and Epstein (1999). And second, by using a shortened version (25 items) of the Scottsdale Critical Thinking test of Ricker (2003).
Lastly, demographics such as; gender, language and language skills were asked. The average duration of completing the entire questionnaire equalized approximately 44 minutes. At the end of the experiment, all participants were debriefed and received course credit for their participation.
Results
Descriptives
An overview of the total scores per item of the TOGKAM is presented in Table 1. Here, each first statement of an item refers to the misconception and each second statement refers to the correct alternative. For example, in item 21 the first statement ‘You need to wait 24 hours before you can report someone as missing’ is chosen by 31.8 percent of the participants as the correct answer. Another example is the one presented in item 16 in the category; History. Here, almost 80 percent incorrectly believed that Christopher Columbus discovered North-America
(See Table 1, on the next page). The prevalence of misconceptions endorsements ranges from
11.4% to 90.1%. So from this, there could be stated that differences exist in endorsements
GENERAL MISCONCEPTIONS AND THINKING STYLES 14 between the misconceptions. The average prevalence rating from the categories; Body, Animals,
Science, History, and Society are respectively, 30.56%, 65.92%, 54%, 78.84%, and 60.2%.
Resulting in an overall total average of 58.5% across the 25 misconceptions.
Table 1 Answers given on the TOGKAM, per item by all participants (N=440). Category Item Statements N % 1 People only use about 10% of their brains 50 11.4 People use the entirety of their brains, but not all at once 390 2 Cracking one's knuckles may cause arthritis and other harm to your joints 223 50.7 Cracking one's knuckles causes the release of gas bubbles and is not harmful 217 3 Shaving causes hair to grow back faster, stiffer, and darker 184 41.8 Body Shaving cuts hair tips, causing them to feel coarser 256 4 Eating shortly before swimming increases the risk of experiencing muscle cramps and 256 58.8 can be dangerous Eating shortly before swimming can be unpleasant, but it is not dangerous 209 5 It takes seven years to digest chewing gum 50 11.4 Chewing gum is mostly indigestible, but passes through the digestive system at the same 390 rate as other matter 6 Ostriches stick their heads in the sand to hide when they sense danger 281 63.9 Ostriches who sense danger flop down with their head and neck flat against the ground 159 7 Bulls are enraged by the red color of the matador’s cape 197 44.8 Bulls are enraged by the movement of the matador’s cape 243 Animals 8 Color change in chameleons is predominantly used for camouflage to hide from possible 351 79.8 danger Color change in chameleons is predominantly used for social signaling and temperature 89 regulation 9 Bears hibernate during the winter to save energy 286 65 Bears rest more during the winter, but do not hibernate 154 10 Bats are blind and can only navigate through echolocation 335 76.1 Bats have eyes and are capable of sight 105 11 The seasons are determined by the Earth’s distance from the Sun 136 30.9 Seasons are caused by the Earth's axial tilt 304 12 People have five senses 309 70.2 People have around 20 senses 131 Science 13 Lightning never strikes twice in the same spot 123 28 Lighting can strike more than one time in the same spot 317 14 Bananas grow on trees 400 90.9 Bananas grow on bushes 40 15 When a coin falls from a high altitude, it reaches a high enough speed to be able to kill 220 50 someone 220 Coins dropped from a high altitude are not able to kill someone due to wind resistance 16 Christopher Columbus discovered North-America 347 78.9 Leif Erikson discovered North-America 93 17 During the Middle-Ages, people were expected to live until about 30 years old 222 50.5 During the Middle-Ages, people lived to be older than 30 years, but many children died 218 during childhood History 18 Vikings wore horned helmets in battle 292 66.4 Vikings wore round helmets in battle, without horns 148 19 Napoleon was an exceptionally small Frenchman 328 74.5 Napoleon was of average height for French men 112 20 Medieval harnesses were so heavy that knights had to be hoisted on their horses 369 83.9 Medieval harnesses were light enough for knights to run and mount horses without help 71
GENERAL MISCONCEPTIONS AND THINKING STYLES 15
21 You need to wait 24 hours before you can report someone as missing 140 31.8 If there is evidence of violence or unusual absence you can immediately inform the 300 police about a missing person 22 Eskimos languages have an exceptional number of words for snow 361 82 Eskimo languages do not have more words for snow than the English language has 79 Society 23 ‘SOS’ stands for ‘Save our Souls’ 302 68.6 ‘SOS’ is no abbreviation 138 24 The Great Wall of China is the only man-made structure visible from space with the 268 60.9 naked eye Man-made structures such as highways, dams, cities are visible from space with the 172 naked eye 25 Big Ben is the name of the clock tower at the Palace of Westminster in London 254 57.7 Big Ben is the name of the great bell hanging inside the clock tower in London 186
Note. TOGKAM = Test of General Knowledge and Misconceptions, consisted of 25 items. The first statement of each item refers to a misconception, the second statement of each item is the correct answer. N = number of participants, % = percentage of participants who accepted the misconception, out of 100% in total. In total, 17 of the 25 misconceptions were accepted to be true by >50% of the participants.
Internal consistency
First, just like in Bensley et al. (2014) analyses were conducted in order to test the internal consistency of the new test. The internal consistency of the TOGKAM across the entire sample (N=440) using the Cronbach’s alpha was α = .58, suggesting poor internal consistency.
However, a weak reliability was presupposed as a wider range of items was used in the
TOGKAM. To illustrate this, when someone incorrectly believes that humans only use 10% of the brain, this does not have to imply an incorrect belief in the fact that ostrich will bury its head in the sand in danger. Simply because there is no relationship between the categories of general knowledge, or even between the propositions within a category, in contrast with the items and categories used in Bensley et al. (2014). Nevertheless, the idea is that for example the personality of a person predicts the probability of accepting misconceptions. Therefore, the items on the
TOGKAM should be somewhat related to each other. Similarly, the internal consistency of the
Critical Thinking scale was questionable α = .65, however again presupposed for similar reasons.
It is not necessarily expected that answering correctly on one item predict also a correct answer on another item. However, the reliability of the subscales of the REI, Rational-Analytic, and
GENERAL MISCONCEPTIONS AND THINKING STYLES 16
Intuitive-Experiential were respectively, α = .87 and α = .86, suggesting good internal consistencies.
Concurrent validity
Next, the concurrent validity was evaluated by examining the correlations between the
TOGKAM and the three measures of thinking dispositions. As shown in Table 2, scores on the
TOGKAM exhibit a significant, negative correlation with scores on the Rational-Analytic scale, r = -.228, p < .01, t (439) = -106.10, n = 440, a significant, positive correlation with scores on the
Intuitive-Experiential scale, r = .125, p < .01, t (439) = -111.61, n = 440, and a significant, negative correlation with Critical Thinking, r = -.267, p < .01, t (439) = -14.90, n =440. These significances of correlations across all measures of thinking dispositions and the TOGKAM supported the validity of the test.
Table 2 Descriptive statistics and bivariate correlations for the test of general knowledge and misconceptions and measures of thinking dispositions. Measure 1 2 3 4 1. TOGKAM - -.228** .125** -.267** N = 440 440 440 2. Rational-Analytic scale - 0.17 .217** N = 440 440 3. Intuitive-Experiential scale - -.083 N = 440 4. CT - N = M 14.01 70.76 65.24 17.92 SD 3.35 9.97 9.46 3.57 Note. TOGKAM = Test of General Knowledge and Misconceptions; Rational-Analytic scale and Intuitive- Experiential scale are the subscales adapted from the Rational-Experiential Inventory (REI) of Pacini and Epstein (1999); CT = The Scottsdale Critical Thinking test from Scottsdale Ricker (2003). * p <.05. ** p <.01. *** p <.001.
GENERAL MISCONCEPTIONS AND THINKING STYLES 17
Monetary incentive
Results of the manipulation check showed that participants in the monetary incentive condition (M = 3.8, SD = 0.73) did report a higher, although not significant, level of motivation to perform well on the test compared with participants in the non-incentive condition (M = 3.67,
SD = 0.71), t (438) = -1.82, p = .07, two-tailed. The effect of a monetary incentive to the total score on the TOGKAM was tested using an independent t-test. There was a significant, but small difference between scores on the TOGKAM for participants in the non-incentive condition (M =
14.40, SD = 3.13) and participants in the incentive condition (M = 13.61, SD = 3.51), t (438) =
2.51, p = .013. These results suggest that adding a monetary incentive to the questionnaire has a small but significant effect on the score on the TOGKAM (See Figure 1).
GENERAL MISCONCEPTIONS AND THINKING STYLES 18
Relative scores on the TOGKAM
Bensley et al. (2014) mentioned that the certainty ratings needed to be taken into account.
Therefore, analyses regarding the correlations of the TOGKAM with rational, intuitive and critical thinking were repeated, but at this time taking into account the certainty ratings of each response. These new relative scores on the TOGKAM were calculated as follows. Each correct answer on the TOGKAM (i.e., rejection of the alternate misconception option), was coded as ‘0’ and each incorrect answers on the TOGKAM (i.e., acceptation of the alternate misconception option) was coded as ‘1’. Subsequently, scores on TOGKAM were multiplied by the certainty ratings, ranging from 0-5. Herewith, the new relative scores of each item ranged from 1-5, and the total scores ranged from 0-100. Analyzing these certainty ratings could provide possible underlying thoughts in the decision process or could propose future behavior. For instance, participants who reported to be extremely certain, might stick more to a misconception and are therefore more resistant to change, according to Epstein (2008). The other way around, participants who have gambled or had no idea of the correct answer, might perhaps be more open to change or conviction of the truth. Initially, no hypotheses had been formulated, despite this interesting notion. Results in Table 3, show that the correlation with CT remained somewhat similar (r = -.274, p < 0.01, n = 440). Next, the correlation with the Rational-Analytic scale decreased (r = -.100, p < 0.01, n = 440). And lastly, most notable the correlation between the
Intuitive-Experiential became stronger (r = -.213, p < 0.01, n = 440).
GENERAL MISCONCEPTIONS AND THINKING STYLES 19
Table 3 Descriptive statistics and bivariate correlations for the TOGKAM scores established with the corresponding certainty ratings and measures of thinking dispositions. Measure 1 2 3 4 1. TOGKAM * Certainty rate - -.100* .213** -.274** N = 440 440 440 2. Rational-Analytic scale - 0.17 .217** N = 440 440 3. Intuitive-Experiential scale - -.083 N = 440 4. CT - N = M 47.05 70.76 65.24 17.92 SD 14.89 9.97 9.46 3.57 Note. TOGKAM * Certainty rate = total score on the Test of General Knowledge and Misconceptions established with the corresponding certainty ratings; Rational-Analytic scale and Intuitive-Experiential scale are the subscales adapted from the Rational-Experiential Inventory (REI) of Pacini and Epstein (1999); CT = The Scottsdale Critical Thinking test from Ricker (2003). * p <.05. ** p <.01.
Explorative analyses
Subsequently, some explorative analyses were conducted. In the first exploratory analysis the relation between the subscales of the REI, and the scores on the TOGKAM had been explored (See Table 4). Results revealed that the correlations between the RA and RE did not significantly differ from each other (z = 1.003, p = .316), whereas the correlations between the
EA and EE did significantly differ from each other (z = 2.177, p = .030).
GENERAL MISCONCEPTIONS AND THINKING STYLES 20
Table 4 Descriptive Statistics and Bivariate correlation comparisons between TOGKAM and subscales of the Rational Experiential Inventory (REI). Scale TOGKAM M SD N 1. Rational Ability (RA) -.174** 35.34 5.44 440 2. Rational Engagement (RE) -.220** 35.42 6.02 440 3. Experiential Ability (EA) .159** 33.36 5.22 440 4. Experiential Engagement (EE) .070 31.88 5.07 440 Note. RA= Rational Ability, a subscale of the Rational-Analytic scale of the REI. RE = Rational Engagement, a subscale of the Rational-Analytic scale of the REI. EA = Experiential Ability, a subscale of the Experiential- Intuitive scale of the REI. EE = Experiential Engagement, a subscale of the Experiential-Intuitive scale of the REI. * p <.05. ** p <.01.
Lastly, since a remarkable number of international students (n = 140) participated to the experiment alongside Dutch students, it was examined whether significant differences in test scores exists between these two groups (‘Dutch nationality’ and ‘Other nationality’). For this purpose, an independent-samples t-test was conducted to compare the scores of the TOGKAM for Dutch nationalities and other nationalities. The results showed no significant difference in scores on the TOGKAM for Dutch nationalities (M = 14.08, SD = 3.23, n = 299) and other nationalities (M = 13.83, SD = 3.58, n =140; t (439) = -.734, p = .46, two-tailed). Additionally,
Hedges’ g was used for determining the effect size, since the two groups consist of different sample sizes, that turned out to be very small, g = 0.061.
GENERAL MISCONCEPTIONS AND THINKING STYLES 21
General discussion
In this study, a new Test of General Knowledge and Misconceptions (TOGKAM) was designed to address several shortcomings of previous tests of (psychological) misconceptions.
The aim was to validate the test by linking the endorsement of misconceptions to several thinking- styles, and dispositions. Additionally, it was investigated whether monetary incentives would increase motivation and therefore result in fewer endorsements of misconceptions due to more rational and critical thinking processes. The results supported the measure of general knowledge and misconceptions as being a valid measure. There was a negative correlation found between the TOGKAM and a rational thinking style and critical thinking dispositions, and a positive correlation found between the TOGKAM and an intuitive thinking style. Moreover, a positive, significant effect of the monetary incentive on TOGKAM scores was found, suggesting that motivation elicit rational thinking styles. In general, our results are consistent with dual- process theories; higher endorsement of misconceptions is related to an intuitive thinking style, whereas fewer endorsement of misconceptions is associated with rational-analytic processes.
In spite of the statistical significance of the monetary incentive, this result might not appear to be practically relevant. In order to evaluate the strength of this statistical claim, the
Cohen’s d effect size is required. G*Power analysis showed an effect size of d = 0.314, suggesting a small effect size. So although this effect reaches statistical significance, the actual difference in the mean scores between the two groups (non-incentivized and incentivized) is very small. This means that monetary incentives have an almost unobservable effect on rejecting misconceptions due to rational-analytic thinking styles. Additionally, according to R.A. Fisher, the significance found in studies is merely worthy of attention in the form of meriting more experimentation but not proof itself. Ironically in the context of this study, it appeared that many
GENERAL MISCONCEPTIONS AND THINKING STYLES 22 misconceptions exist regarding p-values. In summary, we should be careful when interpreting statistical significance by means of practical relevance.
Moreover, analyses regarding the relative scores of the TOGKAM, taking the certainty ratings into account provided additional insights. The overall internal consistency of these new relative scores on the TOGKAM had increased to an alpha of α = .70 compared to an alpha of α
= .58 for the original TOGKAM scores, suggesting that these relative scores reflect a more adequate measurement of the general misconception endorsement. This suggests that with these relative scores, the chance that scores on TOGKAM could be due to random factors such as differences expertise or attention among participants has reduced. As a consequence, similar results could be found in a test-retest analysis.
Furthermore, a higher internal consistency could also suggest stronger correlations between the relevant thinking measures. First, the strength of the correlation with the Intuitive-
Experiential scale increased as expected, suggesting less distribution of scores (See Appendix
A). The total scores increased and the distribution between the scores decreased. Thus, people who use an intuitive and experiential thinking style, and thus mainly reasoning based on intuition, accept more misconceptions and, in addition, they do not opt for extreme values on the certainty ratings (are on average moderately certain). Second, the strength of the correlation with the Critical Thinking test remained somewhat the same, suggesting people who scored high on the Critical thinking dispositions rejected misconceptions by which the certainty rates had no effect on the TOTAL scores. Moreover, when they accepted misconceptions to be true, they were moderately certain (See Appendix A). Lastly, the strength of the correlation with the
Rational-analytic scale decreased suggesting more distribution of scores. Since the overall internal consistency had increased, this decrease in correlation was remarkable (See Appendix
GENERAL MISCONCEPTIONS AND THINKING STYLES 23
A). There could be tentatively argued that people who engage in rational-analytic thinking styles or possess critical thinking skills endorse fewer misconceptions and when they do endorse misconceptions they are either not certain at all or completely certain (report extreme values on the certainty ratings) about knowledge-based choices they make.
The relevance of the current research was based on assumptions that (1) someone’s field of knowledge decrease misconception endorsement, for instance, when being an expert in a psychology domain, fewer psychology related misconceptions will be endorsed. Therefore, the current study created a general measure of misconceptions to assess the prevalence of misconceptions not just within the field of psychology but across multiple domains in life where an even greater prevalence of misconceptions can be expected. And subsequently, since (2) relationships between misconceptions endorsement and different thinking styles and dispositions were assumed, we validated the TOGKAM by relating it to intuitive and rational thinking styles and critical thinking dispositions. As mentioned at the beginning of this section, hypotheses were confirmed and thus, the Rational-Experiential Inventory (REI) and the Critical Thinking (CT) test appeared to be reliable predictors of misconception endorsement.
Limitations and future research
In spite of, our successful development of the new measurement of general misconceptions, some remarks can be made. Firstly, only psychology students from Tilburg
University have participated in the current study. Herewith, it is not yet known whether similar effects can be found among a wider scope of people. Despite the items of the TOGKAM consisted of a wide range of domains, still, some difference could occur in scores between different populations. Future research could include a more varied group of people, in order to see whether our results could be generalized to a more general population.
GENERAL MISCONCEPTIONS AND THINKING STYLES 24
Furthermore, the measurement is not entirely flawless since we cannot conclude from our test that all misconceptions are preventable by using Critical Thinking. This has to do with the fact that there was no distinction made between the types of misconceptions. According to
Hughes, Lyddy, and Lambe (2013), two types of misconceptions can be distinguished. On the one hand, there are factual misconceptions: beliefs that arise from incorrect or incomplete information encountered in popular media or the everyday environment. The belief that
Napoleon was short is one such misconception. On the other hand, there are ontological misconceptions which reflect naive or common sense theories about worldly phenomena. For example, the myth of ‘heavier objects fall faster than lighter objects’, is more likely to be derived from intuitive theories of the mind rather than external sources. Future research would benefit, by taking these different types of misconceptions into account, to be able to draw more specific conclusions. For instance, it could be examined whether the endorsement of ontological misconceptions depends on to what extent someone relies on intuition. Or whether the endorsement of factual misconception depends on the fact if someone has seen relevant information on the topic.
The current study foresaw that people possess dispositions to think either in an intuitive or rational way. However, in the context of factual misconceptions, it does not make any sense whether someone is more rational or intuitive, there is no logical reasoning required to find out whether Napoleon was short or not. In contrast, with ontological misconceptions, people either just (1) rely on their intuition or (2) deeply analyze information and then conclude to an outcome. For example, for the misconception of ‘ostriches stick their heads in the sand to hide when they sense danger’ it is assumable that people think ‘I’ve heard from this before, so I rely on my intuition’. But at the same time, it could be possible that people try to logically figure out
GENERAL MISCONCEPTIONS AND THINKING STYLES 25 what the advantages of ostriches could be and then conclude that it does not make sense for ostriches to stick their head into the sand. To summarize, psychological dispositions are more likely to have an effect on ontological misconceptions because they can be rationally analyzed.
While with factual misconceptions deeply analyzing will not make sense. Altogether, it is essential for future studies to distinguish between the types of misconceptions in order to draw more specific conclusions.
With this, we arrive at another recommendation. From previously mentioned findings and hypotheses, it can be assumed that it is also insightful to find out from which source, information is gathered. Moreover, adding more time by creating a moment of awareness before definitive answering the question could elicit more rational and critical thinking. For example, simply by adding a ‘pop-up’ that asks participants if they are sure about their given answer. Future studies should add questions regarding information sources and questions that elicit more awareness, to see whether this will decrease misconception endorsement.
Next, it is recommending for future studies to use a version of the REI specially designed for adolescents (REI-A), (Marks, Hine, Blore, & Phillips, 2008). Although the REI-40, used in the current study, is often recommended over other versions, the REI-A might reveal even more accurate results (Marks et al., 2008). Furthermore, the Cognitive Reflection Test (CRT) could also be a valuable addition to the current questionnaire, since it measures a person's tendency to override an incorrect "gut" response and engage in further reflection to find a correct answer
(Toplak, & Stanovich, 2011). Study of Bialek and Pennycook (2017) shows that people who score high on the CRT are more successful distinguishing fake news from real news. Thus, in the contexts of misconceptions, the CRT is appropriate to apply. Besides, other than the REI, scores on CRT are not based on self-reporting. Lastly, the CRT is proved to be robust to multiple
GENERAL MISCONCEPTIONS AND THINKING STYLES 26 exposures, so that despite the raw score increases for experienced participants, its correlations with other variables remain unaffected.
Conclusion
In this paper, a new Test of General Knowledge and Misconceptions (TOKAM) was designed and linked to thinking styles and disposition with the aim of validation. The results of the study showed that the TOGKAM has yielded a poor, although less relevant, internal consistency and adequate concurrent validity. The monetary incentive condition had positive effects on TOGKAM score, however, due to a small effect, it cannot be considered as practical relevance. Our findings are consistent with the dual-process theory, in terms of different systems of information processing. Those who endorsed more misconceptions showed differences in thinking style from those who held fewer misconceptions. In particular, having critical thinking skills or high scores on the Rational-Analytic scale (associated with system 2) predicted lower scores on the TOGKAM, which means lower misconception endorsement and more accurate knowledge. While high scores on the Intuitive-Experiential scale (associated with system 1) predicted higher scores on the TOGKAM. Future research should apply our recommendations regarding, the scope of the sample, distinguish the two types of misconceptions and add the
Cognitive Reflection Test (CRT) as well as questions regarding someone’s source of information in order improve the Test of General Knowledge and Misconceptions.
GENERAL MISCONCEPTIONS AND THINKING STYLES 27
References
Angus, N., Patience, F., Maclean, E., Corrigall, H., Bradbury, I., Thompson, D. R., Atherton, I.,
& Leslie, S. J. (2012). Cardiac misconceptions in healthcare workers. European Journal
of Cardiovascular Nursing, 11(4), 396-401.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of
Personality and Social Psychology, 51, 1173-1182.
Bensley, D. A., & Lilienfeld, S. O. (2015). What is a psychological misconception? Moving
towards an empirical answer. Teaching of Psychology, 42, 282–292.
Bensley, D. A., & Lilienfeld, S. O. (2016). Critical thinking dispositions, confidence, and
availability in endorsement of psychological misconceptions. Unpublished manuscript
Bensley, D. A., & Lilienfeld, S. O. (2017). Psychological misconceptions: Recent scientific
advances and unresolved issues. Current Directions in Psychological Science, 26(4),
377-382.
Bensley, D. A., Lilienfeld, S. O., & Powell, L. A. (2014). A new measure of psychological
misconceptions: Relations with academic background, critical thinking, and acceptance
of paranormal and pseudoscientific claims. Learning and Individual Differences, 36,
9–18.
Boel, H. (2013). 666 leugens die iedereen gelooft. Lannoo.
Crowe, R. A., & Miura, C. K. (1995). Challenging pseudoscientific beliefs: Surveying evidence
for exotic claims. Psychological reports, 77(3_suppl), 1263-1282.
Epstein, S. (2008). Intuition from the perspective of cognitive experiential self-theory. Intuition
in judgment and decision making, 23, 37.
GENERAL MISCONCEPTIONS AND THINKING STYLES 28
Evans, J. S. B., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing
the debate. Perspectives on psychological science, 8(3), 223-241.
Garrett, H. E., & Fisher, T. R. (1926). The prevalence of certain popular misconceptions. Journal
of Applied Psychology, 10(4), 411.
Furnham, A., & Hughes, D. J. (2014). Myths and misconceptions in popular psychology:
Comparing psychology students and the general public. Teaching of Psychology, 41(3),
256-261.
Gardner, R. M., & Brown, D. L. (2013). A test of contemporary misconceptions in psychology.
Learning and Individual Differences, 24, 211-215.
Hammer, D. (1996). More than misconceptions: Multiple perspectives on student knowledge and
reasoning, and an appropriate role for education research. American Journal of Physics,
64(10), 1316-1325.
Hughes, S., Lyddy, F., & Kaplan, R. (2013). The impact of language and response format on
student endorsement of psychological misconceptions. Teaching of Psychology, 40(1),
31-37.
Hughes, S., Lyddy, F., & Lambe, S. (2013). Misconceptions about psychological science: A
review. Psychology Learning & Teaching, 12(1), 20-31.
Kahneman, D., Slovic, S. P., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment under
uncertainty: Heuristics and biases. Cambridge university press.
Kowalski, P., & Taylor, A. (2009). The effect of refuting misconceptions in the introductory
psychology class. Teaching of Psychology, 36, 153–159.
Landau, J. D., & Bavaria, A. J. (2003). Does deliberate source monitoring reduce students’
misconceptions about psychology. Teaching of Psychology, 30(4), 311-314.
GENERAL MISCONCEPTIONS AND THINKING STYLES 29
Lewandowsky, S., Ecker, U. K., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation
and its correction: Continued influence and successful debiasing. Psychological Science
in the Public Interest, 13(3), 106-131.
Lilienfeld, S. O., Lynn, S. J., Ruscio, J., & Beyerstein, B. L. (2011). 50 great myths of popular
psychology: Shattering widespread misconceptions about human behavior. John Wiley
& Sons.
Lin, Y. P., Furze, G., Spilsbury, K., & Lewin, R. J. (2008). Cardiac misconceptions: comparisons
among nurses, nursing students and people with heart disease in Taiwan. Journal of
Advanced Nursing, 64(3), 251-260.
Lynn, R., Irwing, P., & Cammock, T. (2001). Sex differences in general knowledge. Intelligence,
30(1), 27-39.
Maanen, van, H. (2004). Encyclopedie Van Misvattingen. Boom Koninklijke Uitgevers.
Madsen, K. M., Hviid, A., Vestergaard, M., Schendel, D., Wohlfahrt, J., Thorsen, P., Olsen, J., &
Melbye, M. (2002). A population-based study of measles, mumps, and rubella
vaccination and autism. New England Journal of Medicine, 347(19), 1477-1482.
Marks, A. D., Hine, D. W., Blore, R. L., & Phillips, W. J. (2008). Assessing individual
differences in adolescents’ preference for rational and experiential cognition. Personality
and Individual Differences, 44(1), 42-52.
McCutcheon, L. E. (1991). A new test of misconceptions about psychology. Psychological
Reports, 68(2), 647-653.
Mestre, J. P. (2001). Cognitive aspects of learning and teaching science. University of
Massachusetts Physics Education Research Group. 1-18.
Myers, D. G. (2002). The powers & perils of intuition. Psychology Today, 35(6), 42-52.
GENERAL MISCONCEPTIONS AND THINKING STYLES 30
Pacini, R., & Epstein, S. (1999). The relation of rational and experiential information processing
styles to personality, basic beliefs, and the ratio-bias phenomenon. Journal of personality
and social psychology, 76(6), 972.
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method
biases in behavioral research: A critical review of the literature and recommended
remedies. Journal of applied psychology, 88(5), 879.
Ricker, J. (2003). Scottsdale Test of Critical Thinking. Unpublished instrument, Scottsdale
Community College.
Sak, U. (2011). Prevalence of misconceptions, dogmas, and popular views about giftedness and
intelligence: A case from Turkey. High Ability Studies, 22(2), 179-197
Shiloh, S., Salton, E., & Sharabi, D. (2002). Individual differences in rational and intuitive
thinking styles as predictors of heuristic responses and framing effects. Personality and
Individual Differences, 32(3), 415-429.
Silva-Ayçaguer, L. C., Suárez-Gil, P., & Fernández-Somoano, A. (2010). The null hypothesis
significance test in health sciences research (1995-2006): statistical analysis and
interpretation. BMC medical research methodology, 10(1), 44.
Swami, V., Stieger, S., Pietschnig, J., Nader, I. W., & Voracek, M. (2012). Using more than 10%
of our brains: Examining belief in science-related myths from an individual differences
perspective. Learning and Individual Differences, 22(3), 404-408.
Taylor, A. K., & Kowalski, P. (2004). Naive psychological science: The prevalence, strength,
and sources of misconceptions. The Psychological Record, 54(1), 15-25.
Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The Cognitive Reflection Test as a
predictor of performance on heuristics-and-biases tasks. Memory & cognition, 39(7),
GENERAL MISCONCEPTIONS AND THINKING STYLES 31
1275.
Vosniadou, S. (2001). What can persuasion research tell us about conceptual change that we did
not already know?. International Journal of Educational Research, 35(7-8), 731-737.
Retrieved from https://www.geekwrapped.com/myth-infographic
GENERAL MISCONCEPTIONS AND THINKING STYLES 32
Appendix
Appendix A.
Scatterplots of (1) correlations between TOGKAM scores and score on the Intuitive-Experiential scale on the left and (2) correlations between relative TOGKAM scores and score on the Intuitive-Experiential scale on the right.
Scatterplots of (1) correlations between TOGKAM scores and score on the Critical Thinking test on the left and (2) correlations between relative TOGKAM scores and score on the Critical Thinking test on the right.
GENERAL MISCONCEPTIONS AND THINKING STYLES 33
Scatterplots of (1) correlations between TOGKAM scores and score on the Rational-Analytic scale on the left and (2) correlations between relative TOGKAM scores and score on Rational- Analytic scale on the right.