JOURNAL OF GEOSCIENCE EDUCATION 60, 393–407 (2012) Utilizing Xenarthra (Tree Sloth, Anteater, Armadillo, Ground Sloth, Glyptodont, and Pampathere) Cranial Material to Evaluate Students’ Understanding of This Thing Called Science Barbara J. Shaw1,2,a and Luis A. Ruedas2,3 ABSTRACT Two-thirds of U.S. citizens do not understand the scientific process. There is a clear misunderstanding about what science is— and is not—both in our society and in the classroom. Furthermore, students below basic proficiency are locked into an achievement gap. In response, the No Child Left Behind Act was passed in 2001. Since then, there has been some progress in decreasing the achievement gap. However, according to The Nation’s Report Card, 34% of fourth grade and 43% of eighth grade students sampled by the National Assessment for Educational Progress still fall below a basic level of proficiency in science. To evaluate what is misunderstood about the scientific process, third through eighth graders were guided to discern science from pseudoscience, and form testable questions by using 45 animal skulls and design experiments, and to then collect and analyze data to answer their questions based on the graphs they developed. They were given a pre-assessment at the beginning and a postassessment the end of a 12-h unit to determine changes in learning. These data were analyzed by a paired Student’s t-test. The results show that students gained significantly in memorizing facts and making objective observations about xenarthrans. Students were not able, however, to transfer the skills gained to make objective observations about dinosaurs. In addition, they had difficulty differentiating between scientific questions (objectively testable) from nonscience questions. Ó 2012 National Association of Geoscience Teachers. [DOI: 10.5408/10-211.1] Key words: inquiry, science education, testable question, pseudoscience INTRODUCTION (TIMSS) assessment was developed by the International Voters and politicians both rate education among the Association for the Evaluation of Educational Achievement top 10 issues in the current sociopolitical situation of this (IEA) to measure students’ achievements in mathematics country (Polling Report, 2007). Youth education is manda- and science. The Institute of Education Sciences of the U.S. tory in all states, requiring attendance from ages 4 or 5 to Department of Education has a series of directives including usually at least age 16 years (U.S. Department of Education, participating in and maintaining the statistics of the TIMSS 2007). It is a national goal for all children to obtain a specific assessments (IEA, 2007). TIMSS provides participating level of understanding—or standard—in English, math countries with an unprecedented opportunity to evaluate skills, social science, and science, as expressed in and passed students’ progress in mathematics and science achievement by the No Child Left Behind (NCLB) Act of 2001 (U.S. on a regular 4-year cycle, which began in 1995, with the Department of Education, 2001) as well as in the science, most current results being from 2007 (IEA, 2007). Through technology, engineering, and mathematics (STEM) educa- participation in TIMSS, the United States has obtained tion initiatives of the National Science Foundation (NSF). reliable and timely data on the mathematics and science The mission statement of the U.S. Department of Education achievement of U.S. students compared with those of encapsulates the importance of a solid education: The students in other countries (Martin et al., 2004). One trend department’s mission is ‘‘to promote student achievement observed in U.S. science education is that as students and preparation for global competitiveness by fostering progress through U.S. schools, their science scores are educational excellence and ensuring equal access.’’ Scientists relatively highest in fourth grade (compared with those of and educators are, however, failing at the basics in science: other countries) and relatively lowest in 12th grade (IEA, More than two-thirds of Americans do not understand 1995, 2007). science or the scientific process (NSF, 2004). In each of the previous TIMSS (1995, 1999, 2003, and If education is the key to remain competitive in the 2007), U.S. students in the fourth grade also improved in global arena, then the United States is not meeting its stated each assessment and statistically were fourth, after Singa- objectives. The Trends in Mathematics and Science Study pore, Taiwan, Hong Kong, and Japan. The Russian Federation, Latvia, and England had higher mean scores, Received 20 November 2009; revised 11 December 2011; accepted 14 July 2012; but were not significantly different from those of U.S. fourth published online 6 November 2012. graders. 1Colorado State University Extension, 1001 N. 2nd Street, Montrose, The eighth grade U.S. students likewise improved Colorado 81401, USA significantly from 1997 to 2007 in both mathematics and 2Department of Biology, Portland State University, Portland, Oregon science scores. However, by the 2007 TIMSS, 9 of 47 97201, USA countries assessed countries had statistically higher mean 3Museum of Vertebrate Biology, Portland State University, Portland, Oregon 97201, USA scores in science than U.S. eighth graders. The top countries aAuthor to whom correspondence should be addressed. Electronic mail: were Singapore, Taiwan, Japan, the Republic of Korea, [email protected] England, Hungary, the Czech Republic, Slovenia, and the 1089-9995/2012/60(4)/393/15 393 Q Nat. Assoc. Geosci. Teachers 394 B. J. Shaw and L. A. Ruedas J. Geosci. Educ. 60, 393–407 (2012) Russian Federation. The United States’ and Hong Kong’s transform—what they learn in an unstructured classroom scores were not significantly different, although Hong setting into the standardized test format (Jordan et al., 2000). Kong’s score was higher than the United States’ score. A potential remedy for this failure might be to help students The highest assessment, TIMSS-A[dvanced], is given in develop metacognitive and critical thinking skills, such that the final year of secondary school students. This corresponds they are able to apply information from one area to distinct to 12th grade in the United States, although other countries scenarios (Jordan et al., 2000), and for this, teachers need have different number of required years of schooling. The consistent and repetitive training in these areas (Abd-El- United States participated in 1995, and the results were Khalick and Akerson, 2009). dismal: Of 21 countries, the U.S. ranked 16th in science and Thus, it would appear that there is more to the failure of 19th in math, with an overall mean significantly below meeting the standards than merely students not doing well average. In 2008, TIMSS-A was offered again. The Bush on standardized tests. U.S. students simply are not engaged Administration decided not to have U.S. high school seniors in science because of a multitude of reasons—from language participate, reasoning that other countries test students older nuances or English proficiency, through perceiving science than those tested in the United States, and that many of learning as ‘‘white,’’ to even having teachers unfamiliar with these countries begin specializing in high school in different science (Secada, 1992; Koba, 1996; Poliquin, 1997; Visone, core areas such as physics or math (Mervis, 2007). Educators 2010). This achievement gap is therefore clearly not just an disagree, feeling that much can learned from these scores. artifact of standardized testing, a true gap exists (Olszewski- The educational reforms over the past 6 years (2002, when Kubilius, 2006). Not only is there a gap in achievement NCLB went into effect, until 2008, when the TIMSS-A was based on ethnic traits, but also a similar achievement gap administered to participating countries) include high school occurs based on socioeconomic status (SES). Students from students taking more math and more advanced-placement low-SES households score much lower than students from science classes. high-SES households on the NAEP assessment (average At this time, individual states have the responsibility to scale scores were 142 and 159, respectively). develop standards for measuring student learning in To improve standardized scores, we must start closing mathematics and science. A direct comparison among the gap starting as early as preschool and kindergarten students from different states is therefore impossible. In (Chapin, 2007; Johnston, 2009; Akerson and Donnelly, order to evaluate state efforts, the National Assessment of 2010). Our students need to engage in science activities in Educational Progress (NAEP) was founded in 1969 as a part order to discern what science is and appreciate it in a of the U.S. Department of Education, under the National nonjudgmental environment. However, as in any endeavor, Center for Education Statistics. The NAEP sampled students if the objectives are not clear, then the outcome will not be from across the nation, and correlated demographics clear (Chapin, 2006; Brown and Abell, 2007; Sarkar and together with these scores to formulate The Nation’s Report Frazier, 2008). The objectives of the learning exercises must Card, with 2005’s being the most current data available. The therefore be made clear to the students in order that learning findings are shocking: 34% of fourth and 43% of eighth be achieved, in explicit instruction (Khishfe and Lederman, graders were not meeting proficiency in basic science as of 2007).