ASSESSING STUDENT UNDERSTANDING OF THE CONNECTION BETWEEN DNA AND

Jill M. Jaksetic

A Thesis

Submitted to the Graduate College of Bowling Green State University in partial fulfillment of the requirements for the degree of

MASTER OF

August 2012

Committee:

Karen Sirum, Advisor

Paul Morris

Eileen Underwood

© 2012

Jill M. Jaksetic

All Rights Reserved

iii

ABSTRACT

Karen Sirum, Advisor

Many have investigated students’ understanding and acceptance of evolution but have not found a robust relationship between the two. Arguably, before a relationship between understanding and acceptance of evolution can be analyzed, there must be a clear definition of what is meant by “understanding.” Most measures of evolutionary understanding have largely ignored the molecular components of evolution while concentrating primarily on . These molecular understandings may be integral to alleviating student misconceptions, aiding overall understanding of evolution, and acceptance. To test this idea, a strategy was developed that is capable of assessing students’ ability to explain evolutionary concepts that span from the DNA-level to the level of . Two new open-ended questions were designed to assess students’ abilities to make the conceptual connection between the of DNA, protein shape, and protein function. These two questions were coupled with an existing natural selection prompt regarding the evolution of speed in cheetahs (Bishop and

Anderson, 1990) and supplementary and evolutionary literacy items from the

Evolutionary Attitudes and Literacy Survey (EALS, Hawley et al., 2011). At the beginning and end of a semester, our survey was administered alongside an instrument that measures acceptance of the of evolution (MATE, Rutledge and Warden, 1999) to a variety of undergraduate biology students ranging from non-majors to upper-level majors. A scoring rubric was developed for the new molecular biology questions, which

iv serves as a diagnostic tool for identifying gaps in understanding. This new rubric was used alongside a previously existing rubric developed by Nehm (2010) for the question regarding natural selection. Student responses were scored based on the number of key concepts and alternative conceptions present. Student performance for each rubric item on the open-ended questions and for the supplementary EALS items was compared, and measures of understanding were also compared to student acceptance of evolution.

Results revealed that as predicted, many students in our sample seem to have a gap in understanding regarding how changes in DNA are relevant to other evolutionary ideas, and there are weak to moderate correlations between understanding of underlying genetics concepts and acceptance of evolution.

v

A curious aspect of the theory of evolution is everybody thinks he understands it.

- Jacques Monod, On the Molecular Theory of Evolution (1974)

vi

ACKNOWLEDGMENTS

I am profoundly grateful to my advisor, Dr. Karen Sirum, for her guidance and encouragement during the course of this . Her creativity and dedication were integral to this process. Special thanks to Dr. Paul Morris and Dr. Eileen Underwood, for serving on my committee and providing valuable insight regarding this project. I would also like to thank the faculty (especially Dr. Matthew Partin), staff, and graduate students in the department who aided in the collection of survey data. Many thanks go out to

DeeDee Wentland, the department’s incredibly patient and helpful graduate secretary. I am also very grateful to Alexis Majorczyk and Alfred Andrews for serving as a helpful and friendly audience during our weekly research meetings. I would also like to recognize

Dr. Kenneth Ryan and Mary Paler at the BGSU Center for Business Analytics for aiding in statistical analyses.

This research is indebted to my parents who have always nurtured my curiosity and believed in my ability to achieve whatever goals I choose to set for myself. Thanks must also be extended to my big sister, Rhonda, for serving as a lifelong role model and inspiration. Thank you to my little brother, Connor, and my nephew and niece, Ethan and

Eva, as the wish to make you proud has certainly provided further inspiration to help me complete this work. To my best friend, Alicia, thank you for always providing a constant supply of whimsy, as graduate school has surely not been without its’ stresses. Last, but certainly not least, thank you to my partner, Bret, for your unwavering support, encouragement, and love.

vii

TABLE OF CONTENTS

Page

INTRODUCTION...... 1

METHODS…………… ...... 13

RESULTS AND DISCUSSION ...... 19

LITERATURE CITED ...... 34

TABLES……….……………………………… ...... 36

FIGURES……………...... 50

APPENDIX A: ITEMS FROM THE CONCEPT INVENTORY OF NATURAL

SELECTION (CINS) ...... 67

APPENDIX B: OPEN-ENDED QUESTIONS INCLUDED IN SURVEY ...... 68

APPENDIX C: ITEMS INCLUDED FROM THE EVOLUTIONARY ATTITUDES

AND LITERACY (EALS) SURVEY…………………………………………… ... 69

APPENDIX D: MEASURE OF ACCEPTANCE OF THE THEORY OF EVOLUTION

(MATE) INSTRUMENT...... 70

APPENDIX E: SUBJECTS REVIEW BOARD (HSRB) APPROVAL……….. . 71 viii

LIST OF TABLES

Table Page

1 Distribution of Biology Students Surveyed with All Instruments at the Beginning

(Pre) and at the End (Post) of the Fall 2011 semester...... 36

2 Number of Student Responses from the Various Courses and Sections in the Pre-

Cohort that were Sampled for Initial Scoring of the Three Open-ended Questions... 37

3 Distribution of Student Correct Responses on Open-Ended Questions Using the

Initial Simple Correct or Incorrect Scoring Protocol, (N=191)...……… ...... 37

4 Distribution of Students Across Courses and Sections in the Sample of Matched-

Pairs Used for Scoring of Student Responses to the Three Open-Ended Questions,

(N=158)………………………...... 38

5 Scantron Response Conversion Key for Demographic, MATE, and EALS Items. .. 39

6 Z-test Results Showing Key and Alternative Concepts Present in Significantly

Different Percentages in the Total Pre vs. Post Sample...... 41

7 Z-test Results Showing Key and Alternative Concepts Present in Significantly

Different Percentages in Pre vs. Post Data by Course...... 42

8 Results of Binomial Regression Analysis of the Relationships Between Specific Key

and Alternative Concepts...... 43

9 Z-test Results Showing Statistically Significant Pre/Post Increases in the Proportions

of Students Correctly Responding to a Particular EALS Item. ………… ...... 44

10 Z-test Results Showing Statistically Significant Increases in the Proportion of

Students in Different Courses Correctly Responding to Particular EALS Items...... 45 ix

11 Z-test Results Comparing Statistically Significant Pre/Post Increases in EALS

Performance for Students in a Traditional vs. Active Learning and Integrated

Molecular Concepts Version of Course A...... 46

12 Z-test Results Comparing Statistically Significant Pre/Post Increases in EALS

Performance for Students in a Traditional vs. Active Learning Course B Sections… 47

13 Correlation Table Displaying Relationships Between LPQ Scores, MATE Scores,

and EALS Scores ………………...... 48

14 Correlations Between LPQ Scores and MATE Scores with Individual EALS items.. 49

x

LIST OF FIGURES

Figure Page

1 Rubric for Scoring Student Responses to Open-Ended Questions ...... 50

2 Pre/Post Comparison of the Percentage of Students Including Key Concepts in

Their Responses to the Three Open-Ended Questions...... 55

3 Pre/Post Comparison of the Percentage of Students Including Between 0-7

Alternative Concepts in Responses to Open-Ended Questions...... 55

4 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the First Open-Ended Question...... 56

5 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Second Open-Ended Question...... 56

6 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Third Open-Ended Question...... 57

7 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in Course B with

Specific Molecular Emphasis...... 57

8 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in Course E...... 58

9 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in Course A, All

Sections Combined...... 58 xi

10 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in the Course A Section

with Active Learning and Specific Molecular Emphasis...... 59

11 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in the Course A

Section without Active Learning and Specific Molecular Emphasis...... 59

12 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in Course D...... 60

13 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in Course F...... 60

14 Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts

Present in Student Responses to the Open-Ended Questions in Course G...... 61

15 Pre/Post Comparison of the Proportion of Students with a Correct Response to Each

Individual EALS Item Numbered 1-16...... 61

16 Pre/Post Comparison of the Proportion of Students with a Correct Response to Each

Individual EALS Item Numbered 1-16 in Course A, All Sections Combined...... 62

17 Pre/Post Comparison of the Proportion of Correct Student Responses to Individual

EALS items in the Course A Section Taught with Active Learning and Specific

Molecular Emphasis...... 62

18 Pre/Post Comparison of the Proportion of Correct Student Responses to Individual

EALS Items in Course A Sections without Active Learning and Specific Molecular

Emphasis………...... 63 xii

19 Pre/Post Comparison of the Proportion of Correct Student Responses to Individual

EALS items in Course B All Sections Combined...... 63

20 Pre/Post Comparison of the Proportion of Correct Student Responses to Individual

EALS items in the Course B Section with Active Learning and Specific Molecular

Emphasis…… ...... 64

21 Pre/Post Comparison of the Proportion of Correct Student Responses to Individual

EALS items in the Course B Sections without Active Learning and Specific

Molecular Emphasis...... 64

22 Pre/Post Comparison of Proportion of Correct Student Responses to Individual EALS

Items in Course D...... 65

23 Pre/Post Comparison of the Proportion of Correct Student Responses to Individual

EALS Items in Course E...... 65

24 Pre/Post Comparison of Proportion of Correct Student Responses to Individual EALS

Items in Course F...... 66

1

INTRODUCTION

It is widely acknowledged that evolution is a central and unifying theme in the field of biology. famously stated in a 1973 paper, “Nothing in biology makes sense except in the light of evolution.” The theory of evolution provides us with an evidence- based and eloquent explanation for the incredible diversity of life on our planet. Not only can it lead us in understanding how different forms of life evolved, but it also paves the way for innovative scientific techniques and groundbreaking research that can better the human experience (Dobzhansky, 1973). However, in spite of the overwhelming acceptance of the validity of evolution in the , most Americans continue to either feel unsure about evolution or even outright deny its existence. Since 1985, national samples of U.S. adults have been asked to respond with “true”, “false”, “not sure”, or “don’t know” to the following statement: Human beings, as we know them, developed from earlier of . In 1985,

7% of U.S. adults answered that they were “unsure.” In 2005, the percentage of U.S. adults who felt unsure rose to 21%. This rise in the number of U.S. adults feeling unsure about evolution corresponded with a decrease in both overt acceptance (from 45% to 40%) and overt rejection

(from 48% to 39%). Public acceptance of evolution in the was also compared to 34 other developed nations based on responses to the above statement regarding human origins. The

United States ranked second to last out of the 34 countries, followed only by Turkey (Miller et al., 2006).

Often, or other proponents of evolutionary theory are quick to speculate that the main cause for a lack of acceptance of the theory of evolution is religious belief. Although religious belief is certainly a factor that must be considered, it cannot be ignored that there are many people who are both devoutly religious and accepting of the theory of evolution. There are 2 those that accept the theory of evolution by considering it a means in which has created man.

However, much of the focus on a lack of evolutionary acceptance has been placed on fundamentalist Christians who accept a literalist interpretation of the that is rendered incompatible with evolution. In 1992, Lawson and Worsnop found that strength of religious commitment in a sample of pre-secondary students was negatively correlated (r = -0.45) with initial belief in evolution and with a change in belief toward evolution (Lawson and Worsnop

1992). Some studies have found that educational experiences can influence both student knowledge and acceptance of evolution (Paz-y-Mino and Espinosa, 2009; Bishop and Anderson,

1990; Ingram and Nelson, 2006). These findings contrast others which have suggested that acceptance of evolution is a more stable construct (Rutledge and Warden, 1999). Some researchers believe that there may be a “critical threshold” of course work which must be achieved in order to impact levels of understanding or acceptance of evolution in a significant way (O’Brien et al., 2009).

The matter of accepting or not accepting evolution is not black and white, as there is an entire spectrum of acceptance that falls between Biblical creationist and atheist evolutionist.

Some people feel very strongly one way or another, but many people are, frankly, unsure how they feel about evolution’s validity. Researchers have been trying to uncover the relationships that exist between specific variables and acceptance of evolution. Many factors that could contribute to a lack of acceptance of evolution have now been considered, including a person’s understanding of the of science, knowledge about natural selection, and macroevolutionary concepts such as an understanding of deep time and speciation. The results from many of the studies concerning this issue are disheartening for biology educators as they indicate that students do in misunderstand much about the nature of science and evolution, 3 and students may simultaneously hold pre-conceptions or misconceptions regarding these concepts, thus hindering their learning of correct conceptions.

Even though natural selection is touted as a simple and elegant concept, many people have had a notoriously difficult time truly grasping the concept due to erroneous preconceptions.

It has been found that not only do the public and non-biology majors struggle with the concept of natural selection, but even biology-majors and biology educators have trouble overcoming their misconceptions about natural selection (Nehm and Reilly, 2007; Nadelson, 2009). Lawson and

Thompson (1988) defined misconceptions as “knowledge spontaneously derived from extensive personal experience that is incompatible with established ”. Biology educators face the challenge of not only teaching the scientific conceptions of the theory of evolution, but also of combating student misconceptions. In order to do this, educators must first ascertain what misconceptions exist. Once common misconceptions are known, care must be taken to avoid causing or propagating them. One strategy that can be used to correct misconceptions is to encourage cognitive conflict in the minds of students. This cognitive conflict or dissonance is formed when students are presented with information that cannot be adequately explained by misconceptions held by the students (Kalinowski et al., 2010). Research has indicated that one of the most difficult misconceptions to overcome is that concerning the “chance” involved in evolution. The confusion here has much to do with how aspects of natural selection are both random and non-random. The variation that exists between organisms is due to the largely

“random” process of . Here, “random” refers to the fact that in DNA are not

“preferred” but happen due to replications errors, mutagens, recombination events, etc. However, selection is “non-random” in that certain variants of genes are more or less likely to be passed on to offspring. Because of this complicated relationship between random and non-random events, it 4 is easy to see how misconceptions about this particular concept can arise.

Misconceptions are not simply considered naïve or uninformed ideas, but they are often recognized as views that have previously been held by other prominent and otherwise intelligent individuals (Kalinowski et al., 2010). The particular misconception concerning the “random” nature of natural selection can be termed “Lamarckian.” Jean-Baptiste Lamarck is well-known as being a proponent for the inheritance of acquired characteristics. Recently, new revelations at the epigenetic level have revealed that some characteristics acquired over an organism’s lifetime, such as methylation of particular genes which affect expression, can in fact be passed on to offspring (Stansfield, 2011). Though our increasing understanding of epigenetics sheds light on how there is some truth to the inheritance of acquired characteristics, Lamarckian thinking may still impede student understanding of natural selection.

The inheritance of acquired characteristics was once a leading competitor in mechanisms for explaining the diversity of life on . Lamarckian thinking proposes that organisms change in order to adapt to their environment. The most famous example of a Lamarckian explanation is the idea that giraffes have long necks because they strained their necks to reach food at the top of the trees and their offspring thus had progressively longer necks. This process is not random since it implies that the giraffes were able to directly change themselves in some way due to a

“need.” Darwin’s theory differs as it does not suppose that creatures can directly adapt to their environment based on “need.” Instead, natural variations already exist between organisms of a particular species and those already existing variants differentially fare in a particular environment. Even after semester-long courses incorporating active learning in order to target evolutionary misconceptions such as Lamarckian thinking, students are known to still maintain these alternative conceptions in large numbers (Frasier and Roderick, 2011). 5

Misconceptions often arise when everyday experiences seem to be at odds with scientific understandings. For example, our own bodies are affected by use and disuse and we also observe that children are similar to their parents. When these two observations are coupled with each other, it can suggest that traits which are acquired during a person’s lifetime can be passed on to future generations. Again, although new understanding suggests that some degree of soft inheritance does exist in nature in the form of epigenetics, it is important for students to be able to understand and articulate how natural selection works.

Since natural selection is too slow of a process to be observed in the classroom and genetic material cannot be observed with the naked eye, it is not surprising that misconceptions exist regarding the nature of variation and inheritance (Kalinoski et al., 2010). Student difficulty with abstract concepts has also been documented in regards to an understanding of atoms and molecules in chemistry classrooms, which contributes to the perpetuation of misconceptions

(Griffiths and Preston, 1992).

Evolution can be a particularly difficult concept for people to grasp for many reasons. As stated above, evolutionary concepts can be at odds with an understanding of the world that relies on our experiences. Obfuscating the matter further, the evidence supporting evolution can be rather abstract. Although we have ample physical evidence of evolution as seen in the record, some of the best and most convincing evidence comes from genetics. While the relatedness of organisms can be seen by comparing phenotypes, it can also be seen by comparing genotypes. However, without an understanding of basic genetics concepts and how they relate to evolution, many students may not be able to adequately assess this evidence. In short, how evolution works and the evidence supporting it may be reduced to a “black box.” If the contents of the black box are unknown, evolution may be reduced to a concept that is either to be believed 6 or not believed, instead of known, and students can substitute in their own explanations.

How an understanding of basic genetic principles may relate to a student’s ability to comprehend or accept the theory of evolution has not yet been sufficiently explored. It has been documented that people largely misunderstand basic genetic principles. For example, a 2004 study of public understanding of genetics revealed that less than half of the study participants knew that genes were located in all cells of the body, whereas the other half of the survey participants believed that genes were isolated in specific locations such as the brain, blood, or reproductive system. Furthermore, when faced with the task of describing what the word

“genetic” means, more than one-third of survey participants openly mentioned that such a task was very difficult for them to do (Lanie et al., 2004). One of the principle components of evolution is the existence of between organisms. If students have a poor understanding of what genes are and what genes do, are they capable of having a solid understanding of evolution?

Though evolution is an overarching theme that ties together biological knowledge, it is usually presented to students and taught in such a way that makes it seem like an isolated topic that has little relevance to other biology content. An example of this can be seen in the way

Introductory Biology courses have traditionally divided content. For example, at BGSU Biology majors are required to take both BIOL2040: Concepts in Biology I and BIOL2050: Concepts in

Biology II. BIOL2040 is an introduction to ecological and , whereas

BIOL2050 is an introduction to molecular and cellular biology. A similar separation of content exists in the BIOL1010 and BIOL1040 courses for non-majors. A separation of content does not imply that instructors are not teaching evolutionary concepts in molecular courses, but simply that students may have difficulty seeing how molecular ideas concerning evolution relate to non- 7 molecular ideas. Evolution oftentimes has its own chapter or section in a , although a true understanding of evolution suggests that evolutionary ideas should be incorporated into every aspect of a biology curriculum. Recommendations have been made for the adjustments of and curricula in order to resolve this discrepancy (Nehm et al., 2009; Kalinoski et al.,

2010).

With evolution commonly being taught separately from molecular biology content and with the topics separated as such in textbooks, it would be unfair to assume that most undergraduate students readily reach the kind of insight that links their understanding of molecular biology with their understanding of evolution. However, we believe that kind of insight may be necessary for assessing the tremendous supporting evidence for evolution as well as for aiding in dispelling misconceptions and alternative conceptions.

In reference to Dobzhansky’s statement about evolution providing sense to our understanding of biology, Kalinowski et al. published a 2010 essay with the following title:

Nothing in Evolution Makes Sense Except in the Light of DNA. The authors argued that the absence of DNA-level teaching in evolution curricula perpetuate student misconceptions and that students would develop a better understanding of evolution if the genetic basis of evolution was made more clear at the molecular level. Due to evolution being treated as an isolated area of interest in biology, we posit that most students are unlikely to understand evolution in the context of something traditionally taught separately from it: the Central of Molecular Biology.

The Central Dogma describes information transfer in a cell. It states that the information to build proteins is encoded in DNA, which must be replicated. The DNA serves as a template for the formation of mRNA (transcription), which is later used as a guide for stringing together amino acids in a specific sequence to create a specific protein (translation.) Admittedly, there is much to 8 know about the Central Dogma that is beyond the scope of what many biology students need to know. However, the basics of the Central Dogma can provide students with a framework to call upon to understand natural selection and evolution in general.

It is helpful to break the concept of natural selection down into smaller components in order to consider difficulties students may have with understanding specific parts of the process.

According to Jensen et al. (2007), understanding the process of natural selection can be simplified into four separate conceptual components: 1) Variation, 2) Genetics, 3) Differential

Survival and Reproduction, and 4) Time. The component of variation states that organisms of the same species have naturally occurring differences between them. Genetics refers to the fact that sexually-reproducing organisms can pass on their traits to their offspring (heritability).

Differential survival and reproduction means that different variants have differing possibilities of passing on their genes. Lastly, time is necessary for the above three processes to result in noticeable changes that can cause speciation.

Particularly, the underlying ideas in the Central Dogma explain the nature of variation in a way that is at odds with Lamarckian misconceptions. The process of DNA replication has a known error rate, which accounts for some of the initial variation that is present between different organisms within the same species. Changes to DNA can affect the survivability of an organism drastically or not at all, depending on where the changes occurred. Since proteins are built into specific functional shapes based on the DNA instructions, a change in the instructions can alter the structure and consequently the activity of the proteins. This allows for the differential survival of members of a species with particular types of variation in the DNA. If the mutations are heritable, then this differential survival of variants is perpetuated in offspring. If this process occurs for a long enough time, speciation results. It is important to note that changes 9 in gene expression, such as the regulation of the timing of gene expression or the amount a gene is expressed majorly contribute to the differences between species. However, one cannot fully comprehend the importance of the changes in gene expression without understanding the fundamental role of a gene.

Clearly, the process of speciation is a separate issue for students and it is being argued by some that an understanding of macroevolutionary events is also important regarding accepting evolution. A popular creationist argument is that a person understands or believes in , but fails to see how can occur. The division between microevolution and macroevolution arbitrarily separates evolution into short-term

(microevolutionary) and long-term (macroevolutionary) processes. Nadelson and Southerland identified five essential concepts that they regarded as necessary for a comprehension of macroevolution: deep time, , speciation, , and the nature of science

(Nadelson and Southerland, 2010). Although macroevolutionary ideas are indeed challenging for students, the literature also shows that most students actually fall short of a complete understanding of microevolution as well.

Without an underlying understanding of microevolution, an understanding of macroevolution is not possible. To understand natural selection, a student must have an understanding of each component that comprises the process. Understanding just differential survival and is not enough, and the research literature indicates that is the kind of incomplete understanding that is typically shown by students (Ferrari and Chi, 1998; Nehm and

Reilly, 2007). Additionally, an emphasis on DNA-level understanding of evolutionary concepts would directly contribute to a student’s ability to understand the phylogenetic and speciation components of macroevolution. Kalinowski et al. argued that although the precise genetic basis 10 for phenotypic differences between species is seldom known, it is clear that species are different because their are different. However, even distantly related species have remarkable similarities in some portions of their DNA. The authors suggest that showing students actual

DNA sequences from different species can demonstrate to a student how there is “no reason why mutation and selection, operating gradually over a long time period, could not transform a species” (Kalinowski et al., 2010). Preliminary data from Kalinowski et al. indicate that incorporating DNA sequences into traditional lectures about natural selection is effective at reducing student misconceptions in an introductory biology course. The data concerning the effectiveness of integrating DNA sequences into the natural selection curriculum is limited as of yet. In order to explore this idea further, it is necessary to determine how we will assess understanding of these concepts.

In recent years, biology education researchers have been developing concept inventories in order to assess student understanding of specific biology topics. The idea of the concept inventory was born when Physics instructors felt the need for a comprehensive instrument to assess student understanding of force (Hestenes and Swackhamer, 1992). The Force Concept

Inventory (FCI) has inspired the creation of concept inventories in other fields. Biologists are still in the process of developing a Biology Concept Inventory (BCI) in order to assess student understanding of a variety of general biological concepts (Klymkowsky and Garvin-Doxas,

2008) and additional concept inventories have been created for specific biological topics, such as the Concept Inventory of Natural Selection (CINS) developed by Anderson et al. (2002) and the

Genetic Literacy Assessment Instrument (GLAI) developed by Bowling et al. (2008), among others. Concept inventories are often developed by administering open ended instruments to students in order to determine common misconceptions students may have about particular 11 concepts. Once student responses are gathered and evaluated, multiple choice instruments can be created using popular student misconceptions in distractor items. Concept inventories are useful for diagnosing particular misconceptions and evaluating the effectiveness particular teaching strategies have at reducing misconceptions (D’Avanzo, 2008). Much research exploring student understanding of natural selection relies on the CINS (Nadelson and Southerland, 2010;

Kalinowski et al., 2010). Though many have found the CINS to be useful, the instrument does not contain questions that require students to display their potential understanding of the nature of variation at the DNA level. Instead, understanding of variation is measured using questions that refer to “genetics.” [See Appendix A for examples.] Other known and previously used measures of understanding of natural selection are similarly flawed. The Genetics Literacy

Assessment Instrument (GLAI) has questions which link DNA mutations with the formation of novel genes, but there aren’t any questions which link this idea to phenotypic variation

(Kalinowski et al., 2010).

Before conclusions can be made concerning student understanding of evolution, we must have a reliable and valid way to measure this understanding. Understanding of evolution in the education literature has largely been synonymous with an understanding of natural selection.

Although natural selection is a key aspect of the theory of evolution, a student’s understanding of natural selection does not necessarily indicate a deep understanding of evolution. Furthermore, the possibility of a relationship between understanding and acceptance cannot be dismissed if understanding is being inadequately measured.

The separation between evolution and its molecular underpinnings is not limited to textbooks and curriculums; it also seems to exist in the realm of assessment. It is possible that students may possess the ability to succeed on assessments that measure either side of the 12 proposed evolutionary knowledge gap, but they are unable to pass an assessment that integrates molecular genetic understanding with evolution. There also may be students who understand key concepts regarding evolution but are unable to pass assessments due to barriers presented by vocabulary. As known existing instruments may not reasonably measure the proposed gap in student understanding, the development of new questions to specifically investigate this is necessary.

13

METHODS

Measuring Understanding

Two open-ended questions were developed in order to use in conjunction with a third, previously existing question in order to ascertain student understanding of the nature of variation, natural selection, and how the two concepts are related (Appendix B). An open-ended format was chosen as opposed to close-ended formats such as multiple choice or true/false questions because open-ended questions are better suited for research designs in which possible responses by students cannot be specified in advance. Responses to open-ended questions also provide greater insight into the thought processes of the students and what they really know. The two new open-ended questions were written to be as free of scientific jargon as possible to minimize any barrier that could prevent students from understanding the questions. The preexisting question about natural selection was also chosen with these ideas in mind, as well as to be able to compare our data with previous published work using that same question.

Our previous pilots of Question 3, the “cheetah question” (Bishop and Anderson, 1990) and questions similar to it revealed that when describing natural selection, students are not inclined to respond in a way that reveals their knowledge about the origin of variation. Instead, students will typically state that variation exists within a population, but they will not describe where the variation initially arose. Thus, Questions 1 and 2 were developed in order to prompt students to reveal their possible understanding of the origins of variation (Question 1) and how genetic variation can effect phenotypic variation (Question 2). Question 2 is of particular significance because it is the question that is meant ascertain whether or not a student is able to bridge an understanding of variation at the DNA level with their understanding of natural selection. In other words, since genetic variation is an integral component to natural selection, 14 we developed Questions 1 and 2 to help us discern the depth of a student’s knowledge about what genes are and what they do. We hypothesized that many students would be capable of correctly describing natural selection without having this base understanding of genetic variation.

Select items from the recently developed Evolutionary Attitudes and Literacy Survey

(EALS) were also included in the survey administration in order to gather additional relevant data about student understanding. The EALS was developed by Hawley et al. in order to meet the need for a more comprehensive instrument which measures a broader array of explanatory variables concerning understanding and acceptance of evolution than other existing measurement tools. The EALS contains items from 16 categories such as political activity, religious activity, attitudes toward life, moral objections to evolution, exposure to evolution, genetic literacy, and evolutionary knowledge, etc. Items from the original EALS survey were rated by survey participants using a Likert-style scale which contains responses ranging from strongly agree to strongly disagree (Hawley et al., 2011).

Student responses to the items from the EALS survey were used to complement their responses to the open-ended questions and provide us with greater detail regarding what the students know, but may not have revealed in their written responses. Sixteen items from the genetic literacy and evolutionary knowledge sections of the EALS were chosen as these items were the most relevant to our research questions (Appendix C). Instead of a Likert-style scale, students were asked to respond with “true,” “false,” or “not sure” for each item due to the items being a measure of understanding instead of acceptance. Correct responses were awarded one point. Incorrect responses or responses of “not sure” were awarded zero points. Student performance on individual items was logged, as well as a score regarding how many of the sixteen items each student got correct. 15

Measuring Acceptance

In addition to assessing the gap in understanding that we believe exists in many students, student acceptance of the theory of evolution was also assessed using the Measure of Acceptance of the Theory of Evolution (MATE) Instrument. The MATE was originally developed in order to measure evolution acceptance in public high school biology teachers. The MATE has since been shown to be a valid and reliable measure of evolution acceptance in students and has already been used as a measure of student acceptance of evolution in numerous studies (Rutledge and

Warden, 1999; Rutledge and Sadler, 2007; Nadelson and Southerland, 2010; Ha et al., 2012).

The MATE is a 20-item Likert-style survey which asks participants to rate statements about evolution on a 5-point scale ranging from strongly agree to strongly disagree. Scores on the

MATE can range from 20 points (lowest possible acceptance) to 100 points (highest possible acceptance). The large score range is useful in its ability to reveal a broad range of acceptance to the theory of evolution which can be obscured on shorter instruments or instruments that only use true/false statements to measure acceptance. The MATE also places a minimal emphasis on young earth creationist beliefs which again makes it more suitable for measuring the known diversity of attitudes toward evolution as compared to other instruments of its kind (Appendix

D).

Survey Administration

Our three-open ended questions and the accompanying EALS items and MATE survey were administered at the beginning and end of the Fall 2011 semester as part of a department- wide assessment of students enrolled in a variety of biology courses at BGSU (Table 1). Students 16 were given approximately twenty minutes to respond to the open-ended questions as well as the additional EALS items and MATE survey. Reponses to the open-ended questions were written on the questionnaires, while responses to both the EALS and MATE items were collected using

Scantron sheets. Students were asked to include their names on the answer sheets for the open- ended questions as well as on the Scantron sheets. Survey administrators were asked to inform the students that their responses would be kept confidential and would not affect their grades in any way. Demographic information was also collected using the Scantron sheets, including age, year in college, number of biology courses taken in high school, number of biology courses taken in college, number of non-biology science courses taken in college, and major or college.

Survey administration was approved by the Human Subjects Review Board (HSRB) and each participating student received an HSRB consent form with the survey (APPENDIX E).

Data Coding and Scoring

The first step in analyzing the data we collected was to begin scoring student responses to the three open-ended prompts. Samples were initially taken from each course for scoring using the pre-data since we began the scoring process before the post-data set had been collected. Our

Pre- sample consisted of students from a variety of different biology courses, including courses for majors, non-majors, and honors students (Table 2). At the outset, a sample of responses across all courses (N=191) was scored in a binary fashion as either correct or incorrect based on the presence or absence of particular ideas. For example, for the first open-ended question which prompted students to describe why the same protein, elastin, could have two different shapes responses containing phrases such as “different genes/alleles/DNA” were considered correct. An 17 example of an incorrect response was a student simply stating that the elastin proteins have different shapes in people because “everyone’s different.” A correct student response to the first prompt was coded as “1” while responses considered incorrect were coded as “0”. Because there were three open-ended prompts, there were three total points possible. Doing so enabled the enumeration of the total number of correct and incorrect responses for each prompt, revealing which prompts were easier or more difficult for students. Utilizing consistent scoring criteria, it became apparent that students had an easier time with the first and third prompts as compared to the second prompt (Table 3).

Scoring the responses in the way outlined above turned out to be quite challenging, as many students included a blend of correct and incorrect ideas in their responses making it difficult to score the responses in a consistent binary fashion. To account for this, the scoring system was reformatted and student responses were subsequently scored based on the presence or absence of particular key concepts or alternative conceptions. Based on the types of responses given by students, a coding rubric was created to capture key concepts students used in their responses along with alternative concepts that were used commonly enough to warrant their own category (Figure 1). Using Microsoft Excel, a spreadsheet was created in order to track the number of key and alternative concepts included by each student for each of the three open- ended questions. This set-up enabled the discernment of gains and reductions in particular key and alternative concepts for pre/post data analysis. Samples from the post data set (Table 4) were scored using this method. Many students from the pre-data set did not complete the post-testing, so additional samples were taken to increase the size of the pre/post matched-pair data set to allow for better comparisons and statistical analysis. Refer to Table 4 to see the distribution of students across the courses and sections (N=158, matched-pairs) whose open-ended responses 18 were scored in both pre and post-testing.

Across the three questions, there were ten total possible key concepts and thirteen total alternative concept types that were employed by students. An ideal response contained the maximum number of key concepts with zero alternative conceptions. Many responses had a mix of key and alternative concepts. Some students included only alternative concepts in their responses. To be able to compare student performance on the open-ended portion of the instrument, a Learning Performance Quotient (LPQ) was calculated for each student based on the methods of Nehm and Reilly, 2007. The LPQ creates a grade-like performance score and was calculated using the following formula:

Key Concept Diversity X Key Concept Diversity Key + Alternative Concept Diversity Total Possible Key Concepts

The left side of the formula expresses how much of the student’s response was “correct,” while the right side of the formula expresses how “correct” the student’s response was compared to how “correct” it could have been. An LPQ score was tabulated for each of the individual open-ended questions as well as across all three questions. This allowed comparison of student performance on specific open-ended questions to performance on particular genetic and evolutionary literacy items from the EALS instrument.

Scantron outputs for student responses to the demographic, EALS, and MATE items were converted from the default Scantron alphabetical coding to numerical coding in order to accomplish scoring and statistical analysis (Table 5). Statistical analysis of the data collected for this research was conducted using IBM SPSS Statistics (Version 19.0, August 2010) and

Microsoft Excel 2010 (z-tests only). 19

RESULTS AND DISCUSSION

Student Use of Key and Alternative Concepts

The rubric we developed (Figure 1) was utilized to score matched-pairs from the sample

(Table 4) with the goal of getting an overall picture of student understanding based on the number of key and alternative concepts used by students in their responses to the open-ended questions. Analysis of the data revealed that over the fall 2011 semester, there was an overall increase in the number of key concepts sampled students included in their responses to the three open-ended prompts. Between 30-35% of students did not include any key concepts in their responses during pre-testing, while between 15-20% of students did not include any key concepts during post-testing (Figure 2). Similarly, there was a reduction in the number of alternative concepts students employed in their responses at the end of the semester. Between 25-30% of students had zero alternative conceptions in their responses during pre-testing compared to about

40% during post-testing (Figure 3).

Pre/post comparisons of the proportion of students employing particular key or alternative concepts were made for each of the three open-ended questions (Figures 4-6). Z-tests were conducted in order to assess whether pre/post changes in student usage of particular key concepts or alternative concepts were statistically significant (Table 6). In the first open-ended question about one protein (elastin) having two different shapes, more students discussed mutations in their responses in the post-testing than in the pre-testing. In conjunction with this, fewer students cited demographic, general differences between people, dietary differences, or disease-states as reasons for differences in protein shape. For the second question, a significantly higher number of students suggested the key concept that a single nucleotide base change might result in a shape change to the dystrophin protein, which would cause muscle weakness. Also, 20 fewer students included the alternative conception that all mutations are deleterious. Regarding the third open-ended question, significant gains were made regarding the number of students discussing the heritability of variation.

Due to the differences in the content and format of the courses we sampled from, we analyzed changes in the proportions of students employing specific key or alternative concepts by course (Table 4 and Figures 7-14). Table 7 shows the significant changes in key or alternative concepts by course, as detected by Z-tests. Z-tests were conducted for all courses, but only courses with significant results are shown. The Course B sample was from one section of the course that was conducted with extensive active learning strategies and with a particular emphasis on concepts important in this study. By the end of the semester, students in this course were more likely to mention mutation as being a reason for the shape difference in elastin

(KC1b) increasing from 17% to 52% (Figure 7). In Question 3, students from this section made gains in their ability to mention the variation among cheetahs in a population (KC3a).

A random sample of 25 students in 1 section of Course E experienced a gain from 19% to

46% in KC1b, which indicates the possible role of mutation in the difference in elastin shape

(Figure 8). Course E students in the sample also experienced reductions from 58% to 31%

(Figure 8) for the alternative concept in Question 3 in which students use “essentialist” language

(See Figure 1) to describe the evolution of cheetahs (AC3a).

A combined sample of 48 students from two sections of Course A did not experience any significant changes in numbers of key or alternative concepts (Figure 9). However, when the section with extensive active learning along with emphasis of specific molecular concepts

(Figure 10) was separated from a more traditional Course A (Figure 11, N = 27) section, the

Course A section with active learning (N = 21) experienced significant increases from 19% to 21

76% in the key concept: KC1b, which cites mutation as a possible reason for the difference in elastin shapes. Significant gains were not found in the sample of 27 students from 2 sections of

Course D (Figure 12), the sample of 19 students from Course F (Figure 13), or the sample of 15 students from 1 section of Course G (Figure 14). The lack of statistically significant gains measured in some of the courses may be attributed to too few student responses scored as well as some courses having lower student-enrollment compared to other courses. Scoring of additional responses in the future would be valuable for raising the sample sizes per course, which may result in better detection of differences between sections in student usage of key and alternative concepts.

The average LPQ score at the beginning of the semester for the sample of 158 students was 0.21, while at the end of the semester the average was 0.29. The range of LPQ scores in the sample spanned the entire range of possible LPQ scores (from 0.0 to 1.0). At the beginning of the semester, one student in the sample of 158 had a perfect LPQ score of 1.0. The number of students with a perfect LPQ score increased to five students at the end of the semester.

Binomial regression (stepwise-forward) was also performed to see if individual key or alternative concepts were predictive of each other or of specific EALS items. No substantial models were found in which student responses to specific EALS items could be predicted by their inclusion of key or alternative concepts in their responses to the open-ended questions.

Some interesting models were found which were predictive of student usage of specific key or alternative concepts based on other key or alternative concepts students employed in their responses (Tables 8 a-f). Tables 8a and 8b indicate how some key concepts are related across all three of the open-ended questions. Note: the key concept “DNA contains the instructions for building proteins” appears twice, as it is considered a key concept for both Question 1 and 22

Question 2. The data in Tables 8(a-b) indicated that students are more likely to employ this key concept in one of the questions if they also employed it in the other, even though the two prompts are quite different. Tables 8(c-e) indicates that students are more likely to use a key concept in Question 2 if they already used a different key concept in that same question. Table 8f indicates that students mentioning of the heritability of variation among cheetahs often went hand-in-hand with a mention of differential survival and reproduction. These results may signify that some students in the sample have an understanding that allows for significantly bridging the concepts in the open-ended questions.

Student Response to Supplementary Genetic and Evolution Literacy Items

To create an overall snapshot of student understanding of the supplementary genetic and evolutionary literacy items, preliminary pre/post comparisons of student performance on the

EALS items were conducted using unmatched pairs from the entire pre/post data sets (Table 1).

See Figure 15 for a bar graph that summarizes these results. Z-tests were conducted to assess which changes in student performance on EALS items were statistically significant. Significant increases occurred over the course of the semester for nine of the sixteen EALS items when data from all courses was combined (Table 9). We further analyzed changes in student performance on the sixteen EALS items by course (see Figures 16-24). Larger increases in the percentage of students correctly responding to EALS items occurred in Course B sections and Course E sections as compared to Course A and Course D sections (Table 10). This result is expected as

Course B and Course E are courses designed to contain more content about molecular biology concepts as compared to Course A and Course D. The only EALS item which experienced no significant increase over the course of the semester for any course was the item stating “ developed from earlier life forms.” 23

It should be noted that a when separated from the rest of the Course A cohort, one section of the course in which active learning teaching strategies were used and molecular concepts were intentionally emphasized in the context of evolution and natural selection, had significant increases in correct responses for six of the EALS items, while the rest of the Course A cohort experienced no significant increases on any of the EALS items (Table 11). Instruction in the

Course A section with specific molecular emphasis was achieved through the use of homework assignments based on assigned readings and in-class discussions regarding what mutations are and how they can contribute to the differential of organisms. For example, students were exposed to how a single nucleotide change can result in changes in the shape and function of specific proteins responsible for antibiotic or pesticide resistance. The aforementioned concepts are traditionally not emphasized in many Course A sections and are typically reserved for inclusion in Course B. Future matched-pair analysis is needed in the future to shed light on the differences in student understanding of specific EALS items in traditionally taught Course A sections vs. the sections that include these molecular concepts, as well as the role of the pedagogical strategies in promoting understanding of these concepts.

Table 12 shows a comparison of the differences in student performance on specific EALS items in the active learning vs. traditional sections of Course B, a non-majors course with content including molecular genetics concepts as well as instruction on evolution. There were statistically significant increases in the number of correct EALS responses in the post-testing for both active learning and traditional sections. A few small differences exist between the active and non-active learning Course B sections. The active learning section had statistically significant increases in EALS11, while the non-active learning section did not. Further, the non- active learning section had statistically significant increases in EALS7, EALS10, and EALS14, 24 while the active learning section did not. The percentage of students getting the correct answer on EALS items in the active learning section was higher than the more traditional sections, but further statistical analysis is required to determine if this difference is significant. These differences may reflect differences in pedagogy or content emphasis in the different sections.

The use of unmatched pairs in the analysis of EALS survey samples from all the courses limits our ability to know if the increases in correct EALS responses were due to learning gains due to the possibility that the post cohort contained stronger students than the pre cohort due to struggling students dropping courses or other unknown factors. Future matched pairs analysis of

EALS data can shed more light on where in the curriculum students are developing their understanding of these concepts and the type of interventions needed.

Though the research here pertains to how the degree of genetic literacy impacts evolutionary understanding and acceptance, general genetic literacy is becoming increasingly important for everyone due to advances in sequencing and subsequently, healthcare decisions (Lanie et al., 2004). Thus, it is important to have the ability to measure the effectiveness of the learning activities and experiences that are conducive to increasing the genetic literacy of non-majors.

Factors Affecting Student Acceptance of Evolution

As found in other previous studies (Paz-y-Mino and Espinosa, 2009), a one-way ANOVA analysis using our data revealed that student acceptance of evolution was significantly affected by year in college, F=5.49, p=0.002, DF=3. Analysis of the demographic characteristics that we assessed revealed no other characteristic that was significantly related to acceptance. Two sample T-Tests were used to compare pre and post data to see if any statistically significant 25 changes occurred in MATE scores or performance on EALS items when looked at in aggregate across all the courses that returned these assessments at the beginning and end of the semester.

There was no significant change in MATE scores, T-test value: -0.58, p=0.56, DF = 220. There was a small, but significant change in student performance on the EALS items over the course of the semester, T-test value: -2.61, p=0.01, DF=234.

MATE scores from the entire sample (Table 1) and the smaller sample for which the open-ended questions were scored (Table 4) were the same, with the average score being 74.

These averages can be classified as moderate based on the range of acceptance outlined by

Rutledge and Sadler, 2007. The average score of 74 falls between average scores reported for strictly non-majors and biology teachers in the United States, which seems to reflect the fact that our sample contained a mixture of students ranging from non-majors in introductory level courses to biology majors in upper-level courses (Ha et al., 2012).

A correlation analysis was conducted in order to determine the presence and strength of relationships among student understanding and acceptance of evolution. The post-testing data set from the combined sample of 158 subjects was used for the analysis (Table 4). Moderate correlations were found between student performance on open-ended questions as measured by

LPQ score and student performance on the EALS portion of instrument, r = 0.438, p < .000, as well as between student performance on EALS items and acceptance of evolution, as measured by MATE score, r = 4.75, p < .000. A significant, but weak correlation was found between cumulative LPQ score for the three open-ended questions and MATE score, r = 0.204, p < 0.05

(Table 13). Individual LPQ scores for the three open-ended questions (LPQ1, LPQ2, and LPQ3) were also included in the correlation analysis. The only question-specific LPQ score significantly related to MATE score was for the second open-ended question (LPQ2), r = 0.196, p < 0.05. 26

Thus, students who were able to correctly respond to the question linking an understanding of mutations with natural selection were slightly more likely to be more accepting of evolution

(Table 13). This weak but significant relationship adds support to our that a deep understanding of the underlying molecular components of evolution may aid in students’ abilities to evaluate the evidence for evolution and thus influence acceptance.

One drawback to using the LPQ score as a measure of understanding is that the set of

“alternative conceptions” contained a mixture of misconceptions and also ideas which may not necessarily be “wrong,” but were not along the lines of what we designed the prompts to elicit. It is possible that if students included more details in their responses that some alternative conceptions would come to more closely resemble key concepts. For example, AC1e is the alternative concept in which a student mentions a disease or condition in their response to the question about why elastin can have two different shapes. If the student were to include further details which perhaps elaborated that it may be a disease state caused by a mutation in the DNA, etc., the response would subsequently be coded differently, resulting in a higher LPQ score.

Therefore, some incomplete ideas included by students in their responses may have lowered their

LPQ score even if the ideas were not misconceptions per se. However, the benefits of being able to use the LPQ score to quantify student responses and thus be able to better compare the responses outweighed the drawbacks.

Table 14 summarizes the correlations found between specific EALS items and MATE or

LPQ scores in the post-testing data set from the combined sample of 158 subjects (Table 4). Both individual and cumulative LPQ scores were almost universally related to individual EALS items, however no strong correlations were found. Nearly half of the individual EALS items were significantly related to MATE score (EALS items 1, 6, 7, 8, 11, 14, 15). The item “Humans 27 developed from earlier life forms” (EALS8) had the largest correlation with MATE score compared to all other items in our analysis, r = 0.651, r2 = 0.424, p < .000. This result was particularly interesting considering only two of the twenty items on the MATE survey are specifically about humans (See Appendix D, Items 3 and 15), but a student’s response to EALS item 8 explains 42.4% of the variance in MATE score. The finding that student agreement with

EALS 8 item “Humans developed from earlier life forms” was moderately predictive of overall

MATE score was surprising, as the MATE instrument is constructed in a way to measure student acceptance of evolution in general, not specifically their beliefs regarding . This result also corresponds with the statistical results from Z-tests conducted on individual EALS items. The EALS 8 item is the only genetic and evolutionary literacy item that experienced no significant gains for the cohort as a whole or by course (Refer to Table 10). This result supports the idea that students’ views regarding human evolution are relatively stable. This finding is particularly important due to the wide use of the MATE instrument in studies pertaining to student acceptance of evolution. The fact that student views regarding human evolution as measured by student response to EALS item 8 are substantially related to MATE score could potentially inhibit proper interpretation of MATE score results in such studies. This finding is also interesting considering the fact that many people claim to accept evolution for all other organisms except for humans. What this finding may suggest is that, instead, beliefs regarding human evolution may have a large effect on beliefs regarding evolution occurring in other organisms.

Student responses to the EALS item that states “you can see traces of our evolutionary past in human embryos” (EALS7) were also moderately correlated with MATE score, r = 0.465, r2 = 0.216, p < .000. A stepwise regression analysis revealed that student responses to both 28

EALS7 and EALS8 are slightly more predictive of MATE score than EALS8 alone, r = 0.687, r2

= 0.472, p < .000. Although significant pre/post changes for EALS8 were not found among any of the samples across all courses, EALS7 experienced significant increases, particularly in

Course B and Course D (refer to Table 10).

The EALS items were originally included as an additional measure of student understanding. On the actual EALS instrument itself, the questions are from two sections entitled

“Genetic Literacy” and “Evolutionary Knowledge.” However, the complicated relationship between knowledge and belief permits students to respond to such items in a similar way as they might respond to items regarding acceptance of evolution. It is possible that that some students are more likely than others to view some of these “knowledge” questions as “belief’ questions.

Perhaps the fact that EALS7 exhibited pre/post differences while EALS8 did not reveals that students may be likely to view the statement “you can see traces of our evolutionary past in human embryos” as something to be known, whereas they may be more likely to view the statement “humans developed from earlier life forms” as something to be believed.

In order to see if any particular key or alternative concepts were related to MATE score, a one-way ANOVA was performed. Results revealed that in the post test sample of matched-pair student responses across all the courses (Table 4), significant differences in MATE score were not found to correlate with any of the key or alternative concepts except AC1c. Students’ inclusion or exclusion of AC1c (other non-molecular explanations/demographic criteria in response to the question about the shape of elastin) corresponded with significantly different

MATE scores, F = 4.97, p = 0.027, DF = 1. Demographic type AC1c explanations to the first open-ended question included suggesting a person’s age, body type, sex, ethnicity, or flexibility resulted in the changed shape of elastin. Another way of thinking about this particular alternative 29 conception is that the student explanations pertain to physically observable traits, whereas

Question 1 was designed to prompt students to discuss abstract concepts which students cannot readily see (genes, mutations, etc.) Presenting these kinds of explanations is not necessarily indicative of a student not knowing about genes in general, but we do know that for whatever reason, our prompt did not elicit a response about genes from all of the students sampled. It is possible that some students lacked prior exposure to learning about genes, or that for some reason they did not draw from their prior exposure to genes in order to respond to this question.

In the future, interviews could be conducted in order to help understand why students may respond to the prompt in this way and what degree of genetic or molecular understanding they may have, but do not feel compelled to discuss.

A stepwise regression analysis was run on the data to see if any particular EALS items from our survey were predictive of student performance on the open-ended questions. We also included student performance on other individual open-ended items in the analysis. The most notable model developed was predictive of LPQ3 scores (LPQ score for “cheetah” question). R- square: 0.437.

Predictors:

LPQ2 (Student performance on dystrophin question)

EALS6 – “Humans have somewhat less than half the DNA in common with chimpanzees.”

EALS4 – “Today it is not possible to transfer genes from one species to another.”

EALS1 – “Humans share a majority of their genes with chimpanzees.”

EALS16 - “Mutations occur all the time.”

This model indicates that genetic literacy is particularly important to a student’s ability to describe how natural selection works. The LPQ2 score summarizes a student’s ability to 30 understand the way DNA changes are implicated with conformational and functional changes in proteins. Thus, LPQ2 scores indicate that a student has a relatively deep understanding of what is meant by the word “gene.” However, only 43% of the variance in LPQ3 scores can be explained by responses to an understanding of genes. This indicates that many students are able to correctly respond to the question about natural selection without having this background understanding.

Though they may have an understanding of natural selection that is sufficient enough to be scored as correct, they may not have a level of understanding that aids them in seeing how the mechanisms underlying natural selection work. This lack of understanding may contribute to feelings regarding the believability of evolution.

Conclusions

The complex question of why some students accept evolution and others do not remains open. However, the relationships found between LPQ, LPQ2, and EALS scores and MATE scores (Tables 13 and 14) indicate that student understanding of the molecular underpinning of evolution is significantly, but weakly related to levels of acceptance. The relationship between acceptance and a student’s inclusion of an alternative concept (AC1c) in a question about protein shape (Question 1) adds additional support. Though our definition of a deep understanding of evolution goes a step farther than equating an understanding of evolution with only an understanding of natural selection, there are still many more concepts regarding evolution that we left out of our analysis. However, the new assessments developed and used in this study can serve as additional valuable tools that can be used for assessing evolution in a more comprehensive fashion, more reflective of the topic’s unifying nature.

Additionally, the revelation that students’ feelings about human evolution (EALS8) are 31 moderately predictive of MATE score (Table 14) is important to future studies which seek to measure the possible relationships between student understanding of particular concepts and their acceptance of evolution. We now know more about what a student’s score on the MATE instrument may tell us about their beliefs.

In subsequent studies, it may be helpful to create two sets of open-ended prompts: a set containing only questions concerning humans and a set containing equivalent questions about other organisms. Responses from the same students to both the cheetah and human prompts could then be compared. It would be interesting to see if students perform similarly on the two types of prompts, or if students are less likely to include a description of natural selection in their responses when faced with the prompts about human evolution.

Furthermore, the rubric that was created in order to score student responses to the open- ended questions can serve as a valuable diagnostic tool. In the future, instructors can use the coding rubric in order to assess students’ prior knowledge at the beginning of a course or lesson, and both if students are making substantial learning gains as measured by key concepts and also if reductions in alternative conceptions are being exacerbated or reduced at the end of the course.

Thus, the use of the rubric can help instructors to know if particular learning activities or interventions were effective or if they need further modification.

Additionally, our prompts are currently only capable of measuring if students have an understanding that spans from a molecular understanding of variation to natural selection. Our prompts are not currently equipped to assess whether or not students themselves realize how these different understandings are connected. For example, a student may perform very well on all three open-ended questions, but may not be able to explain how the three questions are related to each other. Therefore, it may be helpful to actually include a fourth question to ask the student 32 to explain how their responses to each of the open-ended questions are connected.

In the future, the information gleaned from coding and scoring student responses to the open-ended prompts could be used to develop multiple choice questions that are less-time consuming to score. The most common alternative conceptions found in student responses could be used alone or in conjunction with key concepts as distractors options. This would be particularly helpful for students who may have understood the prompts but had difficulty articulating their responses in the open-ended format. Giving students the option of choosing the best response to a prompt out of an assortment of other prompts containing alternative conceptions would make data analysis less subjective. It is also possible that some students simply did not correctly interpret the prompts and thus were not compelled to give the kind of response we were looking for. Interviews could be used in order to determine the degree to which this occurs and if particular students (for example, majors or non-majors) are more or less likely to misunderstand the expectations of the prompts. One other important aspect which could be improved upon in the future is addressing the issue of getting students to employ their best effort when responding to the open-ended prompts. In the current study, the broad scope of the sample across many courses made this problematic. In future studies, having the same person administering all surveys in order to give students the same directions and motivation might increase students’ efforts when responding to survey items.

It would be helpful to include questions that ask students if they view certain items as something to be “known” or “believed” and why. This could provide further insight regarding the complex relationship between knowledge and belief. Further, this data could guide instructors in implementing curriculum changes that place more emphasis on the evidence regarding particular concepts. 33

It is worthy of note that a recently published study indicates that students’ perceptions of their performance on a natural selection assessment instrument (CINS) are better predictors of their acceptance, as measured by MATE score, than their actual performance on the instrument

(Ha et al., 2012). It is possible that a student’s gut feeling regarding the topic of evolution or how much they think they understand evolution plays a substantial role in their acceptance of it. With this in mind, though our assessment instrument was used in a summative fashion for this research, it could also be of valuable use as formative assessment in order to challenge students and help them develop a more rigorous and solid understanding of evolution.

34

LITERATURE CITED

Anderson, D., Fisher, K., and Norman, G. (2002). Development and Evaluation of the Conceptual Inventory of Natural Selection. Journal of Research in Science Teaching. 39(10), 952-978.

Bishop, B., and Anderson, C. (1990). Student Conceptions of Natural Selection and Its Role in Evolution. Journal of Research in Science Teaching. 27(5), 417-425. Bowling, B., Acra, E., Wang, L, Myers, M., Dean, G., Markle, G., Moskalik, C., and Huether, C. (2008). Development and Evaluation of a Genetics Literacy Assessment Instrument for Undergraduates. Genetics. 178, 15-22. D’Avanzo, C. (2008). Biology Concept Inventories: Overview, Status, and Next Steps. BioScience. 58(11), 1079-1085.

Dobzhansky, T. (1973). Nothing in Biology Makes Sense except in the Light of Evolution. The American Biology Teacher. 35(3), 125-129. Ferrari, M., and Chi, M. (1998). The Nature of Naïve Explanations of Natural Selection. International Journal of Science Education. 20(10), 1231-1256. Frasier, T., and Roderick, C. (2011). Improving How Evolution is Taught: Facilitating a Shift from Memorization to Evolutionary Thinking. Evolution Education and Outreach. 4, 298-307. Griffiths, A., and Preston, K. (1992). Grade-12 Students’ Misconceptions Relating to Fundamental Characteristics of Atoms and Molecules. Journal of Research in Science Teaching. 29(6), 611-628. Hawley, P., Short, S., McCune, L., Osman, M., and Little, T. (2011). What’s the Matter with Kansas?: The Development and Confirmation of the Evolutionary Attitudes and Literacy Survey (EALS). Evolution Education and Outreach. 4, 117-132. Hestenes, D., Wells, M., and Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher. 30, 141-158. Ingram, E., and Nelson, C. (2006). Relationship between Achievement and Students’ Acceptance of Evolution or Creation in an Upper-Level Evolution Course. Journal of Research in Science Teaching. 43(1), 7-24. Jensen, M., Moore, R., Hatch, J., and Hsu, L. (2007). A Scoring Rubric for Students’ Responses to Simple Evolution Questions: Darwinian Concepts. The American Biology Teacher. 69(7), 394-399. Kalinowski, S., Leonard, M., and Andrews, T. (2010). Nothing in Evolution Makes Sense Except in the Light of DNA. CBE— Life Sciences Education. 9, 87-97. Klymkowsky, M., and Garvin-Doxas, K. (2008). Recognizing Student Misconceptions through Ed’s Tools and the Biology Concept Inventory. PLoS Biology. 6(1), 0014-0017. 35

Lanie, A. , Jayaratne, T., Sheldon, J. , Kardia, S., Anderson, E., Feldbaum, M., and Petty, E. (2004). Exploring the Public Understanding of Basic Genetics Concepts. Journal of Genetic Counseling. 13(4), 305-320. Lawson, A. E, and Thompson, L. (1988). Formal Reasoning Ability and Misconceptions Concerning Genetics and Natural Selection. 25(9), 733-746. Lawson, A. E., and Worsnop, W. A. (1992). Learning about Evolution and Rejecting a Belief in Special Creation: Effects of Reflective Reasoning Skill, Prior Belief, and Religious Commitment. Journal of Research in Science Teaching. 29(2), 143-166. Miller, J. D., Scott, E. C., and Okamoto, S. (2006). Public Acceptance of Evolution. Science. 313, 765-766.

Nadelson, L. (2009). Preservice Teacher Understanding and Vision of How to Teach Biological Evolution. Evolution Education and Outreach. 2, 490-504. Nadelson, L., and Southerland, S. (2010). Development and Preliminary Evaluation of the Measure of Understanding of Macroevolution: Introducing the MUM. The Journal of Experimental Education. 78, 151-190. Nadelson, L., and Southerland, S. (2010). Examining the Interaction of Acceptance and Understanding: How Does the Relationship Change with a Focus on Macroevolution? Evolution Education and Outreach. 3, 82-88. Nehm, R., and Reilly, L. (2007). Biology Major’s Knowledge and Misconceptions of Natural Selection. Bioscience. 57(3), 263-272. Nehm, R., Poole, T., Lyford, M., Hoskins, S., Carruth, L., Ewers, B., and Colberg, P. (2009). Does the Segregation of Evolution in Biology Textbooks and Introductory Courses Reinforce Students’ Faulty Mental Models of Biology and Evolution? Evolution Education and Outreach. 2, 527-532. O’Brien, D., Wilson, D., and Hawley, P. (2009) Evolution for Everyone: A Course that Expands Evolutionary Theory Beyond the Biological Sciences. Evolution Education and Outreach. 2:445- 447.

Paz-y-Mino, and G., Espinosa, A. (2009). Acceptance of Evolution Increases with Student Academic Level: A Comparison Between a Secular and a Religious College. Evolution Education and Outreach 2, 655-675. Rutledge, M., and Sadler, K. (2007) Reliability of the Measure of Acceptance of the Theory of Evolution (MATE) Instrument with University Students. The American Biology Teacher. 69(6), 332-335. Rutledge, M., and Warden, M. (1999) The Development and Validation of the Measure of Acceptance of the Theory of Evolution Instrument. School Science and Mathematics. 13-18. Stansfield, W. (2011). Acquired Traits Revisited. The American Biology Teacher. 73(2), 86-89. 36

TABLES

Table 1. Distribution of Biology Students Surveyed with All Instruments at the Beginning (Pre) and at the End (Post) of the Fall 2011 semester.

Course Section ID Pre Post Code Course Description Number N N A Introduction to Ecology 1 115 82 and Environmental Biology 2 145 105 (1000 level, Non-Majors) 3 27 8 4* 147 88 Total 434 283 B Introduction to Cell and 1 203 164 Molecular Biology 2 232 115 (1000 level, Non-Majors) 3** 167 136 Total 602 415 C Introduction to Marine 1 82 ND Biology 2 138 123 (1000 level, Non-Majors) Total 220 123 D Introduction to Ecology 1 150 82 and Evolution 2 145 87 (2000 level, Majors) 3H 8 8 Total 303 177 E Introduction to Cell and 1 98 74 Molecular Biology 2 104 ND (2000 level, Majors) 3 148 76 Total 350 150 F General Genetics 1 45 ND (3000 level, Majors) G Developmental Biology 1 18 16 Overview (4000 level, Majors) H Seminar Course (4000/5000 level, Majors) 1 ND 4 Total 1972 1176 * Section in which molecular concepts were intentionally emphasized in an active learning environment. ** Section of conducted in active learning environment with specific molecular emphasis. H Honors section ND: Not determined as data was not available.

37

Table 2. Number of Student Responses from the Various Courses and Sections in the Pre- Cohort that were Sampled for Initial Scoring of the Three Open-ended Questions.

Course Sections Pre Code Course Description N A Introduction to Ecology and 1 27 Environmental Biology (Non- 2* 31 Majors) B Introduction to Cell and Molecular 1** 31 Biology (Non-Majors) D Introduction to Ecology and 1 20 Evolution (Majors) 2H 8 E Introduction to Cell and Molecular 1 28 Biology (Majors) F General Genetics (Majors) 1 28 G Developmental Biology Overview 1 18 (Majors) Total 191 * Section in which molecular concepts were intentionally emphasized in an active learning environment. ** Section conducted in active learning environment with specific molecular emphasis. H Honors section

Table 3. Distribution of Student Correct Responses on Open-Ended Questions Using the Initial Simple Correct or Incorrect Scoring Protocol, (N=191).

Number of Questions Question ID# # of Students Each Student Got Correct (out of 3) 0 N/A 82 1 1 52 2 0 3 14 2 1 + 2 6 1 + 3 22 2 + 3 2 3 1 + 2 + 3 13

38

Table 4. Distribution of Students Across Courses and Sections in the Sample of Matched- Pairs Used for Scoring of Student Responses to the Three Open-Ended Questions, (N=158).

Course Code Course Description Sections Number Scored

A Introduction to Ecology and 1 27 Environmental Biology 2* 21 (1000 level, Non-Majors) B Introduction to Cell and Molecular 1** 23 Biology (1000 level, Non-Majors) D Introduction to Ecology and 1 20 Evolution 2H 7 (2000 level, Majors) E Introduction to Cell and Molecular 1 25 Biology (2000 level, Majors) F General Genetics 1 20 (3000 level, Majors) G Developmental Biology Overview 1 15 (4000 level, Majors) Total 158

* Section in which molecular concepts were intentionally emphasized in an active learning environment. ** Section of conducted in active learning environment with specific molecular emphasis. H Honors section

39

Table 5. Scantron Response Conversion Key for Demographic, MATE, and EALS Items. Note: SA = Strongly Agree, A = Agree, N = Neutral, D = Disagree, and SD = Strongly Disagree.

Item Item A B C D E # 1 Age in Years < or = 17 18-19 20-21 22-23 > or = 24 2 Year in College Freshmen Soph. Junior Senior Other 3 # of Biology 0 1 2 3 4+ courses taken in high school 4 # of Biology 0 1 2 3 4+ courses taken in college 5 # of non-Biology 0 1 2 3 4+ science courses taken in college 6 Major or College Biology Arts & Education Business Other Sciences 7 MATE ITEM 1 SA = 5 A = 4 N = 3 D = 2 SD =1 8 MATE ITEM 2 SA = 1 A = 2 N = 3 D = 4 SD = 5 9 MATE ITEM 3 SA = 5 A = 4 N = 3 D = 2 SD =1 10 MATE ITEM 4 SA = 1 A = 2 N = 3 D = 4 SD = 5 11 MATE ITEM 5 SA = 5 A = 4 N = 3 D = 2 SD =1 12 MATE ITEM 6 SA = 1 A = 2 N = 3 D = 4 SD = 5 13 MATE ITEM 7 SA = 1 A = 2 N = 3 D = 4 SD = 5 14 MATE ITEM 8 SA = 5 A = 4 N = 3 D = 2 SD =1 15 MATE ITEM 9 SA = 1 A = 2 N = 3 D = 4 SD = 5 16 MATE ITEM 10 SA = 1 A = 2 N = 3 D = 4 SD = 5 17 MATE ITEM 11 SA = 5 A = 4 N = 3 D = 2 SD =1 18 MATE ITEM 12 SA = 5 A = 4 N = 3 D = 2 SD =1 19 MATE ITEM 13 SA = 5 A = 4 N = 3 D = 2 SD =1 20 MATE ITEM 14 SA = 1 A = 2 N = 3 D = 4 SD = 5 21 MATE ITEM 15 SA = 1 A = 2 N = 3 D = 4 SD = 5 22 MATE ITEM 16 SA = 5 A = 4 N = 3 D = 2 SD =1 23 MATE ITEM 17 SA = 1 A = 2 N = 3 D = 4 SD = 5 24 MATE ITEM 18 SA = 5 A = 4 N = 3 D = 2 SD =1 25 MATE ITEM 19 SA = 1 A = 2 N = 3 D = 4 SD = 5 26 MATE ITEM 20 SA = 5 A = 4 N = 3 D = 2 SD =1 27 EALS ITEM 1 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 28 EALS ITEM 2 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 29 EALS ITEM 3 TRUE = 0 FALSE = 1 Don't Know N/A N/A 40

= 0 30 EALS ITEM 4 TRUE = 0 FALSE = 1 Don't Know N/A N/A = 0 31 EALS ITEM 5 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 32 EALS ITEM 6 TRUE = 0 FALSE = 1 Don't Know N/A N/A = 0 33 EALS ITEM 7 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 34 EALS ITEM 8 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 35 EALS ITEM 9 TRUE = 0 FALSE = 1 Don't Know N/A N/A = 0 36 EALS ITEM 10 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 37 EALS ITEM 11 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 38 EALS ITEM 12 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 39 EALS ITEM 13 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 40 EALS ITEM 14 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0 41 EALS ITEM 15 TRUE = 0 FALSE = 1 Don't Know N/A N/A = 0 42 EALS ITEM 16 TRUE = 1 FALSE = 0 Don't Know N/A N/A = 0

41

Table 6. Z-test Results Showing Key and Alternative Concepts Present in Significantly Different Percentages in the Total Pre vs. Post Sample (N=158, matched-pairs). NOTE: Z-tests were conducted for all Key and Alternative concepts, but only items significantly higher for KCs and lower for ACs, with a Z-value confidence level above 95%, are shown.

Concept Z-Value Key Concept 1b: The different alleles arose because of mutation. 3.90**

Alternative Concept 1a: Dietary changes cause differences in elastin shapes. 1.98*

Alternative Concept 1c: Other non-molecular explanations or demographic criteria. 2.91*

Alternative Concept 1d: Simply stating that the proteins are different shapes 2.19* because people in general are different.

Alternative Concept 1e: Stating that a disease causes the shape difference. 2.21*

Key Concept 2c: The mutation could lead to a shape change in the protein. 2.17*

Alternative Concept 2a: All mutations are bad. 2.31*

Key Concept 3b: The variation is heritable. 2.01*

**Z-value significant at 99% confidence level *Z-value significant at 95% confidence level

42

Table 7. Z-test Results Showing Key and Alternative Concepts Present in Significantly Different Percentages in Pre vs. Post Data by Course. NOTE: Z-tests were conducted for all Key and Alternative concepts, but only items significantly higher for KCs and lower for ACs, with a Z-value confidence level above 95%, are shown.

COURSE CODE A+ B++ E N 21 23 25 KC1b: The different alleles arose 3.71** 2.48* 2.07* because of mutation. KC3a: Variation exists in the cheetah 2.14* population. AC3a: Essentialism 2.85** + Active learning section and specific emphasis on molecular concepts. ++ Section with extensive active learning and specific molecular emphasis. ** Z-value significant at 99% confidence level * Z-value significant at 95% confidence level

43

Tables 8 (a-f). Results of Binomial Regression Analysis of the Relationships Between Specific Key and Alternative Concepts. Note: Binomial Regression Analysis was conducted with each KC and AC serving as the dependent variable, but only models with a significance of < 0.05 are shown. R2 values in these tables are Nagelkerke pseudo-R2 values.

Table 8a. Dependent Variable: KC1c - DNA contains the instructions for building proteins. Predictors: Step R2 Sig. KC2c - Mutation can lead to shape change in protein 1 0.278 .000 KC1a - Different genes/alleles 2 0.410 .003 KC2a - DNA contains the instructions for building proteins 3 0.463 .012 KC3a - Variation among individuals 4 0.498 .018

Table 8b. Dependent Variable: KC2a - DNA contains the instructions for building proteins. Predictors: Step R2 Sig. KC2b - Amino acid change in protein 1 0.251 .000 KC3a - Variation among individuals 2 0.370 .000 KC1c - DNA contains the instructions for building proteins 3 0.418 .000

Table 8c. Dependent Variable: KC2b - Amino acid change in protein Predictors: Step R2 Sig. KC2d - protein function change 1 0.383 .000 KC2a - DNA contains the instructions for building proteins 2 0.539 .000

Table 8d. Dependent Variable: KC2c - protein shape change Predictors: Step R2 Sig. KC2d - protein function change 1 0.555 .000 KC2a - DNA contains the instructions for building proteins 2 0.619 .000

Table 8e. Dependent Variable: KC2d - protein function change Predictors: Step R2 Sig. KC2c - protein shape change 1 0.539 .000 KC3b – heritability of variation 2 0.607 .000

Table 8f. Dependent Variable: KC3b - heritability of variation Predictors: Step R2 Sig. KC3c - differential survival/reproduction 1 0.801 .036

44

Table 9. Z-test Results Showing Statistically Significant Pre/Post Increases in the Proportions of Students Correctly Responding to a Particular EALS Item. Pre N = 1741, Post N = 1103 (unmatched pairs). NOTE: Z-tests were conducted for all EALS items but only items with Z-Values significant above the 95% confidence level are shown.

EALS Item Z-Value

2: Humans share more than half of their genes with mice. 9.71** 3: Ordinary tomatoes do not have genes, whereas genetically modified tomatoes do. 4.05** 4: Today it is not possible to transfer genes from species of to another. 2.90** 7: You can see traces of our evolutionary past in human embryos. 3.80** 9: Mutations are never beneficial. 4.19** 11: Individuals don't evolve, species do. 3.66** 13: Increased genetic variability makes a population more resistant to . 3.55** 15: Natural selection is the only cause of evolution. 3.50** 16: Mutations occur all the time. 2.49* **Z-value significant at 99% confidence level *Z-value significant at 95% confidence level

45

Table 10. Z-test Results Showing Statistically Significant Increases in the Proportion of Students in Different Courses Correctly Responding to Particular EALS Items. Course F was not included in this analysis as post- EALS scores were not available. Course G was not included on this table due to no significant differences in EALS items pre/post (N=15). The Course B sample included the Active Learning section (Pre N = 167, Post N = 136) and the Course A sample included the Active Learning section (Pre N = 147, Post N = 88). NOTE: Z- tests were conducted for all EALS items, but only items with Z-Values significant above the 95% confidence level are shown.

COURSE CODE: A B D E PRE N: 396 549 199 337 POST N: 269 392 171 142

EALS ITEM 1: Humans share a majority of their genes 4.28** 2.54* with chimpanzees. 2: Humans share more than half of their 2.31* 11.17** 2.34* 5.24** genes with mice. 3: Ordinary tomatoes do not have genes, 2.72* 5.99** 3.31** whereas genetically modified tomatoes do. 4: Today it is not possible to transfer 7.00** 3.62** genes from species of animal to another. 5: All plants and animals have DNA. 2.50** 4.69** 2.14* 6: Humans have somewhat less than half 5.26** of the DNA in common with chimpanzees. 7: You can see traces of our evolutionary 3.03** 5.16** past in human embryos. 9: Mutations are never beneficial. 3.53** 8.37** 2.21* 10: In most populations, more offspring 2.98** are born than can survive. 11: Individuals don't evolve, species do. 2.86** 4.46** 2.19* 12: Mutations can be passed down to the 6.93** next generation. 13: Increased genetic variability makes a 5.74** 2.95** population more resistant to extinction. 14: The more recently species share a 2.03* 3.11** common ancestor, the more closely related they are. 15: Natural selection is the only cause of 2.08* 7.50** 2.46* evolution. 16: Mutations occur all the time. 6.57** 2.80** **Z-value significant at 99% confidence level *Z-value significant at 95% confidence level

46

Table 11. Z-test Results Comparing Statistically Significant Pre/Post Increases in EALS Performance for Students in a Traditional vs. Active Learning and Integrated Molecular Concepts Version of Course A. NOTE: Z-tests were conducted for all EALS items, but only items with Z-Values significant above the 95% confidence level are shown.

EALS ITEM A+ A

2: Humans share more than half of their genes with mice. 2.31* 3: Ordinary tomatoes do not have genes, whereas genetically modified tomatoes do. 2.72** 9: Mutations are never beneficial. 3.47** 12: Mutations can be passed down to the next generation. 2.14* 15: Natural selection is the only cause of evolution. 1.97* 16: Mutations occur all the time. 3.11** +section taught with a molecular emphasis and extensive active learning **Z-value significant at 99% confidence level *Z-value significant at 95% confidence level

47

Table 12. Z-test Results Comparing Statistically Significant Pre/Post Increases in EALS Performance for Students in a Traditional vs. Active Learning Course B Sections. NOTE: Z-tests were conducted for all EALS items, but only items with Z-Values significant above the 95% confidence level are shown.

EALS ITEM B+ B 1: Humans share a majority of their genes with 3.65** 2.49* chimpanzees. 2: Humans share more than half of their genes with mice. 7.76** 7.94** 3: Ordinary tomatoes do not have genes, whereas 5.02** 3.44** genetically modified tomatoes do. 4: Today it is not possible to transfer genes from species of 5.52** 4.49** animal to another. 5: All plants and animals have DNA. 2.95** 3.51** 6: Humans have somewhat less than half of the DNA in 2.50* 4.62** common with chimpanzees. 7: You can see traces of our evolutionary past in human 3.84** embryos. 9: Mutations are never beneficial. 5.91** 5.85** 10: In most populations, more offspring are born than can 3.82** survive. 11: Individuals don't evolve, species do. 3.20**

12: Mutations can be passed down to the next generation. 4.41** 5.14** 13: Increased genetic variability makes a population more 4.38** 3.64** resistant to extinction. 14: The more recently species share a common ancestor, 1.97* the more closely related they are. 15: Natural selection is the only cause of evolution. 2.56** 7.16** 16: Mutations occur all the time. 5.44** 4.05** +Section taught with a specific molecular emphasis and extensive active learning **Z-value significant at 99% confidence level *Z-value significant at 95% confidence level

48

Table 13. Correlation Table Displaying Relationships Between LPQ Scores, MATE Scores, and EALS Scores (Matched surveys from the post data set indicated in Table 4 were used).

LPQ1 LPQ2 LPQ3 LPQ MATE EALS

Pearson Correlation 1 .286** .192* .551** .166 .320**

LPQ1 Sig. (2-tailed) .000 .016 .000 .093 .001

N 158 158 158 158 103 103

Pearson Correlation .286** 1 .437** .797** .196* .317**

LPQ2 Sig. (2-tailed) .000 .000 .000 .047 .001

N 158 158 158 158 103 103

Pearson Correlation .192* .437** 1 .803** .107 .337**

LPQ3 Sig. (2-tailed) .016 .000 .000 .282 .000

N 158 158 158 158 103 103

Pearson Correlation .551** .797** .803** 1 .204* .438**

LPQ Sig. (2-tailed) .000 .000 .000 .039 .000

N 158 158 158 158 103 103

Pearson Correlation .166 .196* .107 .204* 1 .475**

MATE Sig. (2-tailed) .093 .047 .282 .039 .000

N 103 103 103 103 101 103

Pearson Correlation .320** .317** .337** .438** .475** 1

EALS Sig. (2-tailed) .001 .001 .000 .000 .000

N 103 103 103 103 101 103

**. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed).

49

Table 14. Correlations Between LPQ Scores and MATE Scores with Individual EALS items

(Matched surveys from the post data set indicated in Table 4 were used).

EALS EALS EALS EALS EALS EALS EALS EALS 1 2 3 4 5 6 7 8

LPQ1 Pearson Correlation .008 .150 -.034 .019 .016 .041 .116 .174

Sig. (2-tailed) .929 .097 .711 .834 .861 .657 .201 .054

N 124 123 122 121 124 122 124 124

LPQ2 Pearson Correlation -.050 .219* .101 .055 -.108 .090 .032 -.024

Sig. (2-tailed) .581 .015 .267 .548 .232 .324 .725 .787

N 124 123 122 121 124 122 124 124

LPQ3 Pearson Correlation -.082 .101 .101 .174 -.069 .251** -.047 .030

Sig. (2-tailed) .368 .264 .268 .056 .448 .005 .601 .738

N 124 123 122 121 124 122 124 124

LPQ Pearson Correlation -.068 .211* .091 .135 -.075 .193* .032 .069

Sig. (2-tailed) .453 .019 .318 .140 .406 .033 .723 .446

N 124 123 122 121 124 122 124 124

MATE Pearson Correlation .218* .161 .096 .036 .189 .270** .465** .651**

Sig. (2-tailed) .027 .106 .341 .724 .056 .006 .000 .000

N 103 102 101 100 103 101 103 103

EALS EALS EALS EALS EALS EALS EALS EALS 9 10 11 12 13 14 15 16

LPQ1 Pearson Correlation .103 .014 .060 .085 .161 .052 .044 .154

Sig. (2-tailed) .260 .874 .507 .349 .074 .567 .638 .090

N 121 124 124 123 123 123 119 123

LPQ2 Pearson Correlation .035 -.115 .060 .071 .057 .028 .001 .053

Sig. (2-tailed) .706 .203 .510 .434 .533 .757 .995 .559

N 121 124 124 123 123 123 119 123

LPQ3 Pearson Correlation .096 -.013 .108 .052 .089 -.001 -.004 .127

Sig. (2-tailed) .295 .883 .232 .569 .330 .991 .963 .163

N 121 124 124 123 123 123 119 123

LPQ Pearson Correlation .103 -.049 .106 .079 .121 .032 .030 .136

Sig. (2-tailed) .261 .586 .241 .385 .184 .728 .746 .134

N 121 124 124 123 123 123 119 123

MATE Pearson Correlation .118 .191 .296** .028 .160 .219* .310** .194

Sig. (2-tailed) .244 .053 .002 .781 .108 .027 .002 .051

N 100 103 103 102 102 102 98 102

**. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed).

50

FIGURES

Figure 1. Rubric for Scoring Student Responses to Open-Ended Questions

Concept Key Words Example 1 Example 2 Key Concept 1a: Different “The shape is affected by “Proteins are different in The alleles encoding genes/alleles/DN a person's genetics. each individual. elastin are different. A, mutation While most people have Depending on your the gene for straight as genetic make-up, you dominant some have only will have a straight or the bent gene.” bent protein.” Key Concept 1b: DNA change, “Genetic mutation of the “Depending on the The different alleles mutation, genetic genetic material of person's gene history or arose because of change proteins.” a mutation, that could mutation. explain why the elastin could have two different shapes.” Key Concept 1c: instructions or “Their DNA gave their “The genes in one DNA contains the directions for body instructions to person may vary from instructions for building the make their elastin those in another. The building proteins. body, encoding, proteins in that shape.” differences in the genes coding may lead to different coding for the RNA that attaches to the ribosomes creating a different shape for the protein molecule in different human subjects.” Alternative Concept protein intake or “Certain protein “The amount of protein 1a: Dietary changes deficiency, diet deficiencies or mutation they consume.” cause differences in can cause abnormal elastin shapes. elastin shape.” Alternative Concept amount of “Elastin may have “Elastin can have a 1b: Abundance of elastin, how different shapes in different shape in other elastin in the body much elastin different people because people because no one can alter a person's some people may have has the same body. protein shapes. more or less elastin Another reason the protein in their tissues elastin is different is that than others.” some people don't have enough or have too much elastin in their body. Elastin could also bend that way if a 51

person's body does not recognize the elastin.” Alternative Concept age, body shape, “Elastin could have “Elastin can have 1c: Other non- sex, blood type, different shapes in different shapes in molecular ethnicity, different people because different people by explanations/demo- flexibility, etc. it may depend on the racial background or graphic criteria person’s body shape. If skin diseases. For people someone is larger their of different racial elastin protein may be background their skin is bent because it is already adapted for different being stretched so the reasons like climate. shape may change. It Skin diseases can cause could also be genetics. a change in the genetic makeup of skin cells.” Alternative Concept everyone’s “Everybody is not the “The shape of proteins 1d: Simply stating different; no one same. People have can vary from person to that the proteins are is the same different bodies. People person.” different shapes are very different from because people in each other.” general are different. Alternative Concept Disease, “Elastin can have “Elastin can have 1e: Stating that a disorder, different shapes in different shapes in disease causes the condition different people by racial different people because shape difference. background or skin some tissue can form diseases.” differently due to a genetic disorder or because that person may have been using their muscle tissue in a different way.” Alternative Concept accept, reject, “The elastin has to “Elastin can have a 1f: Body must recognize change shapes so that the different shape in other accept/recognize the body will accept them.” people because no one elastin shape has the same body. Another reason the elastin is different is that some people don't have enough or have too much elastin in their body. Elastin could also bend that way if a person's body does not recognize the elastin.” Key Concept 2a: instructions or “The encoding of the “The G --> A nucleotide DNA contains the directions for DNA chain consists of change could affect the 52

instructions for building the "instructions" for certain process of building building proteins. body, encoding, parts of the body to dystrophin, causing coding, perform.” weakened muscle expression fibers.” Key Concept 2b: amino acids, use “The functional protein “The amino acid The mutation could of a triplet code contains the sequence sequence is changed and cause the amino in response (such "GAA", and the non- causes dystrophin to acids in the resulting as AAA, etc.) functional protein have less affinity to the protein to change contains "AAA", which muscle fibers.” codes for a different polypeptide. A change in the sequence causes a change in the polypeptide produced, thus producing a defective or different protein altogether.” Key Concept 2c: shape change, “The nucleotide “Because DNA coding The mutation could different differences can account is in sets of three ATT - lead to a shape conformation, for different RNA strands CGC etc. Though some change in the different protein used to synthesize codes are for the same protein. produced proteins such as protein, some are not dystrophin, and therefore and can cause drastic different molecular changes in an shapes that inhibits organism.” normal functioning of the muscles, similar to if red blood cells did not have their distinct dimpled shape; they would be less able to fulfill their requirements, which is to transport .” Key Concept 2d: effectiveness, “A change in the DNA “The function of The mutation could "working" or could cause the dystrophin is to bind lead to a change in not, dystrophin to not be able muscle fibers. If this the protein's functionality, to connect to other protein is compromised function. ability (in muscle fibers.” by a single nucleotide reference to change it can lose its dystrophin ability to function.” protein) Alternative Concept perfect, exact, “A single change can “Because it's altering 2a: All mutations expressing lead to weakened your DNA to a different are bad that any muscles because our one which is the change will DNA is a certain way for opposite like GA and definitely be a reason. Small changes TA go together. If you negative for the change your genetics switch it then the 53

organism cause you to be weaker.” opposite happen. So if you have ‘G’ as weak then ‘A’ is strong.” Alternative Concept reference to a “A person's DNA is a “The change in the 2c: Incorrect use of person's "genetic complete map and puzzle genetic code alters the the phrase "genetic code" in of their genetic make-up. "function" that code" reference to their Like a puzzle, all the dystrophin is meant to DNA pieces need to fit together perform. You might say just right in order to that the DNA gives the work. If not, there will be protein the wrong a problem in the genetic orders. Because the code code.” is incorrect, the protein cannot do the job - thus weakening rather than strengthening the muscles.” Alternative Concept lack of nutrition, “Maybe the parents did “A single dystrophin 2d: Mutation is amount of not feed their child change can be caused by caused by a nutrient nutrients, diet proper nutrients or the lack of health. The cause deficiency. child was born with this of weakened muscles disease.” can be a part of staying healthy and taking in the right amount of nutrition.” Key Concept 3a: mutation, faster, “Natural selection over “Individual cheetahs that Variation exists in slower, many years favored the were able to run faster the cheetah less/more able, faster of the cheetahs, the than the average cheetah population. better (in slower ones died off due to a genetic reference to faster leaving the faster mutation in its muscle different cheetahs to reproduce at fibers would have been cheetahs) a higher rate.” more successfully at catching prey.” Key Concept 3b: passing on genes “Faster individuals would “The fastest cheetahs got The variation is or traits, be healthier, live longer the most food so they heritable. production of lives, therefore producing survived and bred.” offspring more offspring (passing on the mutated gene) as well as being able to provide more food to its offspring (ensuring the survival of the mutated genes).” Key Concept 3c: better/more fit, “The cheetahs that could “Natural selection over Differential advantage, more not run fast enough to many years favored the survival/reproductio offspring, more catch their prey would faster of the cheetahs, n of individuals. likely to get food die, so the faster cheetahs the slower ones died off 54

survived which allowed faster leaving the faster them to eventually cheetahs to reproduce at evolve from running at a higher rate.” 20mph to 60mph.” Alternative Concept the species of “Over generations of “Over time the cheetah 3a: Essentialism cheetahs mutated cheetahs, the DNA would mutate causing it (ignores encoded has improved to be faster. And due to variation in causing a more satisfying the cheetah improvement in the way the old cheetah would population) in which they ran.” die out causing the new faster cheetah to live today.” Alternative Concept In order to, “so “Maybe the cheetah's “A could 3b: Intentionality that” prey became faster so in explain by saying order for the cheetah to cheetahs DNA became survive, the cheetahs more and more complex, evolved to run faster to or that the cheetahs, with catch their prey.” time realized they had the ability to run faster.” Alternative Concept Had to, needed “Cheetahs had to develop “The cheetahs had to 3c: Teleological to, forced to, stronger muscles because evolve when their prey Explanations (Need were required their prey was faster than changed. The ancestors causes a trait to them and they had to be may have chased after occur) able to catch them in the slower animals that have open savannah in order to now become extinct or survive.” maybe there was a more plentiful amount of prey years ago. The cheetah now runs fast to attack the prey they could not get, otherwise they would end up starving.”

55

30.00%

25.00%

20.00%

15.00% PRE

10.00% POST theScored Sample Percentage of Studentsin 5.00%

0.00% 0 1 2 3 4 5 6 7 8 9 10 Number of Included Key Concepts

Figure 2. Pre/Post Comparison of the Percentage of Students Including Key Concepts in Their Responses to the Three Open-Ended Questions. N=158 (matched pairs, see Table 4.)

45.00% 40.00%

35.00%

30.00% 25.00% 20.00% PRE 15.00% POST theScored Sample 10.00% Percentatge of Studentsin 5.00% 0.00% 0 1 2 3 4 5 6 7 Number of Included Alternative Concepts

Figure 3. Pre/Post Comparison of the Percentage of Students Including Between 0-7 Alternative Concepts in Responses to Open-Ended Questions. N=158 (matched pairs, see Table 4). There were no student responses in the scored samples that included more than seven of the 13 possible alternative concepts. 56

60.00%

** 50.00%

40.00%

30.00% PRE 20.00% POST theScored Sample

Percentage of Studentsin 10.00% * * * * 0.00% KC1a KC1b KC1c AC1a AC1b AC1c AC1d AC1e AC1f Concept

Figure 4. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the First Open-Ended Question. N=158 (matched pairs, see Table 4; see Figure 1 for Concept List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

35.00%

30.00%

25.00% * 20.00%

15.00% PRE POST

theScored Sample 10.00%

Percentage of Studentsin 5.00% * 0.00% KC2a KC2b KC2c KC2d AC2a AC2b AC2c AC2d Concept

Figure 5. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Second Open-Ended Question. N=158 (matched pairs, see Table 4; see Figure 1 for Concept List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

57

50.00% 45.00%

40.00%

35.00% * 30.00% 25.00% PRE 20.00% 15.00% POST theScored Sample 10.00% Percentage of Studentsin 5.00% 0.00% KC3a KC3b KC3c AC3a AC3b AC3c Concept

Figure 6. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Third Open-Ended Question. N=158 (matched pairs, see Table 4; see Figure 1 for Concept List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

60.00% * *

50.00%

40.00%

30.00% Pre

in thein Sample 20.00% Post

Percentage of Students 10.00%

0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 7. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in Course B with Specific Molecular Emphasis. (N=23, matched-pairs, see Table 4; see Figure 1 for Concept List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05. 58

70.00%

60.00% *

50.00% * 40.00%

30.00% Pre in thein Sample 20.00% Post Percentage of Students 10.00%

0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 8. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in Course E. (N=25 matched-pairs, see Table 4; see Figure 1 for Concept List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

60.00%

50.00%

40.00%

30.00% Pre

in thein Sample 20.00% Post Percentage of Students 10.00%

0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 9. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in Course A, All Sections Combined. (N=48, matched-pairs, see Table 4; see Figure 1 for Concept List). 59

80.00% ** 70.00%

60.00%

50.00%

40.00% Pre 30.00% in thein Sample Post 20.00% Percentage of Students 10.00%

0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 10. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in the Course A Section with Active Learning and Specific Molecular Emphasis. (N=21 matched-pairs, see Table 4; see Figure 1 for Concept List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

60.00%

50.00%

40.00%

30.00% Pre

in thein Sample 20.00% Post Percentage of Students 10.00%

0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 11. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in the Course A Section without Active Learning and Specific Molecular Emphasis. (N=27 matched-pairs, see Table 4; see Figure 1 for Concept List).

60

60.00%

50.00%

40.00%

30.00% Pre

in thein Sample 20.00% Post

Percentage of Students 10.00%

0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 12. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in Course D. (N=27 matched-pairs, see Table 4; see Figure 1 for Concept List).

90.00% 80.00%

70.00%

60.00% 50.00%

40.00% Pre

in thein Sample 30.00% Post

Percentage of Students 20.00% 10.00% 0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 13. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in Course F. (N=20, matched-pairs, see Table 4; see Figure 1 for Concept List).

61

70.00%

60.00%

50.00%

40.00%

30.00% Pre in thein Sample 20.00% Post Percentage of Students 10.00%

0.00% AC1f KC1c KC2c KC3c KC1a KC2a KC3a AC1c AC2c AC3c KC1b KC2b KC2d KC3b AC1a AC2a AC3a AC1e AC1b AC1d AC2b AC2d AC3b Concept

Figure 14. Pre/Post Comparison of the Percentage of Particular Key or Alternative Concepts Present in Student Responses to the Open-Ended Questions in Course G. (N=15 matched-pairs, see Table 4; see Figure 1 for Concept List).

100.00%

90.00% 80.00% ** 70.00% ** ** * 60.00% ** ** ** ** 50.00% Pre 40.00% ** Post 30.00% 20.00%

Percentage of Students Samplein 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 15. Pre/Post Comparison of the Proportion of Students with a Correct Response to Each Individual EALS Item Numbered 1-16. Pre N = 1741, Post N = 1103 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05. 62

100.00% * 90.00%

80.00% **

70.00% 60.00% ** 50.00% * PRE 40.00%

in thein Sample POST 30.00% *

Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 16. Pre/Post Comparison of the Proportion of Students with a Correct Response to Each Individual EALS Item Numbered 1-16 in Course A, All Sections Combined. Pre N = 434, Post N = 283 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

100.00% ** 90.00% * ** 80.00%

70.00% 60.00% ** * 50.00% PRE 40.00% **

in thein Sample POST 30.00%

Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 17. Pre/Post Comparison of the Proportion of Correct Student Responses to Individual EALS items in the Course A Section Taught with Active Learning and Specific Molecular Emphasis. Pre N = 147, Post N = 88 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05. 63

100.00% 90.00%

80.00% 70.00%

60.00% 50.00% PRE 40.00%

in thein Sample POST 30.00%

Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 18. Pre/Post Comparison of the Proportion of Correct Student Responses to Individual EALS Items in Course A Sections without Active Learning and Specific Molecular Emphasis. Pre N = 287, Post N = 195 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

100.00% ** 90.00% ** ** 80.00% ** **

70.00% ** ** ** ** 60.00% ** ** ** * ** 50.00% ** PRE 40.00%

in thein Sample POST 30.00%

Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 19. Pre/Post Comparison of the Proportion of Correct Student Responses to Individual EALS items in Course B All Sections Combined. Pre N = 602, Post N = 415 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

64

100.00% ** ** ** 90.00% ** ** ** 80.00% ** ** 70.00% ** * 60.00% ** * 50.00% PRE 40.00%

in thein Sample POST 30.00% Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 20. Pre/Post Comparison of the Proportion of Correct Student Responses to Individual EALS items in the Course B Section with Active Learning and Specific Molecular Emphasis. Pre N = 167, Post N = 136 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

100.00% ** 90.00% ** 80.00% ** ** ** **

70.00% ** ** ** ** ** 60.00% ** * 50.00% ** PRE 40.00% **

in thein Sample POST 30.00%

Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 21. Pre/Post Comparison of the Proportion of Correct Student Responses to Individual EALS items in the Course B Sections without Active Learning and Specific Molecular Emphasis. Pre N = 435, Post N = 279 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05. 65

100.00% 90.00% ** ** ** 80.00% ** 70.00%

60.00% 50.00% ** PRE 40.00%

in thein Sample POST 30.00% Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 22. Pre/Post Comparison of Proportion of Correct Student Responses to Individual EALS Items in Course D. Pre N = 303, Post N = 177 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05.

100.00% * 90.00% * ** * 80.00% ** ** 70.00% ** * 60.00% 50.00% ** PRE 40.00%

in thein Sample POST 30.00%

Percentage of Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 23. Pre/Post Comparison of the Proportion of Correct Student Responses to Individual EALS Items in Course E. Pre N = 350, Post N = 150 (unmatched pairs, see Table 1; see Appendix C for Item List). Post percentages that are significantly different from pre are indicated by asterisks: **p<0.01; *p<0.05. 66

100.00% 90.00% 80.00%

70.00%

60.00% 50.00% PRE 40.00%

in thein Sample POST 30.00% Perecentof Students 20.00% 10.00% 0.00% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EALS Item

Figure 24. Pre/Post Comparison of Proportion of Correct Student Responses to Individual EALS Items in Course F. Pre N = 18, Post N = 16 (unmatched pairs, see Table 1; see Appendix C for Item List).

67

APPENDIX A:

Example items from the Concept Inventory of Natural Selection (CINS). Anderson et al., 2002.

68

APPENDIX B:

OPEN-ENDED QUESTIONS INCLUDED IN SURVEY

Please answer the following three questions to the best of your ability:

1. Elastin is the name of a protein found in human tissues. Most people have elastin proteins that have a shape that is straight, like this

In some people, the elastin proteins have a shape that is bent, like this:

Explain how elastin can have different shapes in different people.

2. Certain diseases in humans are caused by a single change in a person’s DNA sequence. In one such disease, people have extreme muscle weakness. Individuals with this disease have a single nucleotide base change in the DNA sequence that encodes the protein called dystrophin.

For example: ATTCGCCCGGAATTAAGAGTGAGG is changed to: ATTCGCCCGAAATTAAGAGTGAGG

The function of dystrophin is to bind to muscle fibers. This helps to stabilize and strengthen the muscle fibers. How can a single change in the dystrophin DNA sequence lead to weakened muscles?

3. Cheetahs (large African wild cats) are able to run faster than 60 miles per hour when chasing prey. How would a biologist explain how the ability to run fast evolved in cheetahs, when the cheetahs’ ancestors could run only 20 miles per hour?

69

APPENDIX C:

Evolutionary Attitudes and Literacy Survey (EALS) items included in our assessment. Hawley et al., 2011.

Genetic Literacy 1) Humans share a majority of their genes with chimpanzees. 2) Humans share more than half of their genes with mice. 3) Ordinary tomatoes do not have genes, whereas genetically modified tomatoes do. 4) Today it is not possible to transfer genes from species of animal to another. 5) All plants and animals have DNA. 6) Humans have somewhat less than half of the DNA in common with chimpanzees. 7) You can see traces of our evolutionary past in human embryos. 8) Humans developed from earlier life forms. 9) Mutations are never beneficial.

Evolutionary Knowledge 10) In most populations, more offspring are born than can survive. 11) Individuals don't evolve, species do. 12) Mutations can be passed down to the next generation. 13) Increased genetic variability makes a population more resistant to extinction. 14) The more recently species share a common ancestor, the more closely related they are. 15) Natural selection is the only cause of evolution. 16) Mutations occur all the time.

70

APPENDIX D

Measure of Acceptance of the Theory of Evolution (MATE) Instrument, Rutledge, M., and Warden, M., 1999.

1) Organisms existing today are the result of evolutionary processes that have occurred over millions of years. 2) The theory of evolution is incapable of being scientifically tested. 3) Modern humans are the product of evolutionary processes that have occurred over millions of years. 4) The theory of evolution is based on speculation and not valid scientific observation and testing. 5) Most accept evolutionary theory to be a scientifically valid theory. 6) The available data are ambiguous (unclear) as to whether evolution actually occurs. 7) The age of the earth is less than 20,000 years. 8) There is a significant body of data that supports evolutionary theory. 9) Organisms exist today in essentially the same form in which they always have. 10) Evolution is not a scientifically valid theory. 11) The age of the earth is at least 4 billion years. 12) Current evolutionary theory is the result of sound scientific research . 13) Evolutionary theory generates testable predictions with respect to the characteristics of life. 14) The theory of evolution cannot be correct since it Ds with the Biblical account creation. 15) Humans exist today in essentially the same form in which they always have. 16) Evolutionary theory is supported by factual historical and laboratory data. 17) Much of the scientific community doubts if evolution occurs. 18) The theory of evolution brings meaning to the diverse characteristics and behaviors observed in living forms. 19) With few exceptions, organisms on earth came into existence at about the same time. 20) Evolution is a scientifically valid theory.

71

APPENDIX E:

Human Subjects Review Board (HSRB) Consent Form