Teaching the Process of Science

Submission to: U. S. Department of Education Fund for the Improvement of Postsecondary Education (FIPSE) The Comprehensive Program, CFDA 84.116B PIs: Anne Egger, Stanford University, and Anthony Carpi, John Jay College, CUNY ABSTRACT Many social issues have become “socioscientific” issues, requiring individuals to make decisions regarding diet, the environment, or other areas of their life based on an understanding of the nature of scientific research. Yet the majority of people are ill-equipped to make these decisions because they lack an understanding of the scientific process (Sadler et al., 2004).

The proposed project is a collaboration between the City University of New York

(CUNY), Stanford University (SU), and the University of California Museum of Paleontology

(UCMP) that is designed to expose a greater number of students, science majors and non-science majors alike, to the scientific process and scientific research by: 1) assembling an interdisciplinary panel of experts in science and science education to develop a one-semester, undergraduate curriculum on the Process of Science; 2) creating a set of freely available online curricular modules and associated teaching materials to be used in Process of Science courses; 3) evaluating these materials in a Process of Science course to be mounted at CUNY, in a disciplinary science course at SU, and at four additional institutions; and 4) disseminating the

Process of Science teaching materials into a national program for science education, and creating a community of professionals to promote their use and continued assessment.

This proposal draws on an existing national-scale project that has developed an innovative and successful model for the delivery of high-quality teaching content for interdisciplinary science education. Based on this success and the near-consensus among educational researchers that study about the nature of science improves scientific understanding, we now seek to expand this model to catalyze a shift in the way undergraduate introductory science courses are taught. Teaching the Process of Science

TABLE OF CONTENTS

GOALS AND

OBJECTIVES...... 1

THE STATE OF SCIENCE EDUCATION...... 1

TEACHING THE PROCESS OF SCIENCE...... 4

PROPOSED WORK...... 6

Objective 1: Assemble an interdisciplinary panel of experts...... 7

Objective 2: Create a set of curricular modules...... 9

Table 1: Key Scientific Concepts and Possible Module Topics...... 9

Objective 3: Test and evaluate these materials...... 12

Table 2: Examples of assessment instrument questions...... 13

Objective 4: Disseminate and develop these materials...... 16

WHY VISIONLEARNING?...... 17

EXPECTED OUTCOMES...... 19

REFERENCES CITED...... References pg. 1

TOC Teaching the Process of Science

GOALS AND OBJECTIVES

This project aims to catalyze a shift in the way introductory science courses are taught by moving them away from portraying science as an exercise in memorizing concepts and toward teaching the process of science, as championed by the educational literature (Reif, 1995;

Schmidt, 1996; Dagher et al, 2004; Flick & Lederman, 2004). This work is a collaboration of major public and private institutions, and existing national-scale educational projects, thus assuring its broad dissemination. Toward this goal, this work has four specific objectives:

Objective 1: Assemble an interdisciplinary panel of experts in science and science education to

develop a one-semester, undergraduate curriculum on the Process of Science.

Objective 2: Create a set of curricular modules and associated teaching materials to be used in

Process of Science courses and launch the materials on the Visionlearning website.

Objective 3: Evaluate these materials in a Process of Science course to be mounted at the City

University of New York, in a disciplinary science course at Stanford University, and at four

other institutions.

Objective 4: Disseminate the Process of Science teaching materials into a national program for

science education, and create a community of professionals to promote their use and

continued assessment.

This project addresses FIPSE Invitational Priority A by providing all undergraduate students access to scientific research through freely available, high quality materials that elucidate the process of science and highlight modern research questions.

THE STATE OF SCIENCE EDUCATION

The poor state of science education in the United States is all too well known and has serious consequences on American culture and society (Dye, 2000; Augustine, 1998). The most

1 The Process of Science Carpi recent Science and Engineering Indicators study shows scientific knowledge among adults in the

United States is declining, and over 61% of respondents lack a clear understanding of the nature of scientific inquiry (NSB, 2006). On a local level, the lack of public understanding of the scientific process has enabled school boards in Kansas, Dover, PA, and other regions to argue, at least temporarily, that the teaching of science should include a discussion of belief systems such as Intelligent Design (NSB, 2006; NCSE, 2005; Allen, 2005). On a global level, the U.S. is the only developed country to reject the limits described by international treaties on climate change because a vocal minority has convinced the public that scientific debate means that the scientific basis for this change is uncertain, unreliable, or unproven (Oreskes, 2004).

While responses by the public to scientific issues often perplex scientists and educators, they are a natural product of the manner in which science is taught in the U.S. Traditional college lecture courses present science as a simple series of facts and conclusions made by long-dead scientists, overlooking the natural course of debate and discussion that delimited those conclusions and denying students the opportunity to understand the process by which science derives its authority (Allchin, 2003). As a result, the discourse that accompanies current scientific theories is seen as surprising and out-of-character to students, and elicits a natural response to question the very validity of that which is being debated, rather than understand that the debate is itself crucial to scientific understanding (Rudolph & Stewart, 1998).

Almost every discipline presents itself through both process and application courses:

English pairs creative writing and effective speaking courses with literature and poetry classes; mathematics, possibly to a fault, focuses on process as opposed to its application in accounting, statistics, or other areas; foreign languages teach the process of speaking, writing, and understanding the language in addition to foreign literature and culture. And yet the sciences are almost completely devoid of courses on the process of science at the introductory level.

Applications of the subject in various fields – chemistry, physics, biology, etc. – are taught at

2 The Process of Science Carpi nearly every undergraduate institution in the country; however, courses on the process and nature of science, which convey the common principles of scientific research, are a rarity.

Some faculty believe that students pick up an understanding of the process of science by performing laboratory experiments. Yet the common laboratory course, in which precise measures and exact endpoints are already known, does little to convey the process of science

(Hofstein & Lunetta, 2004). Others have shown that students do begin to understand the nature of science by participating in faculty-mentored research (Ryder et al., 1999). As a result, the U.S.

Department of Education and the National Science Foundation are staunch supporters of undergraduate research experiences. Unfortunately, traditional research experiences are reserved for undergraduate students who major in the sciences at a limited number of institutions: commonly four-year institutions that grant graduate degrees or have access to research facilities.

Non-science majors or low-performing students rarely have access to undergraduate research internships. Further, minority- and Hispanic-serving colleges and universities are among the least prepared to offer these experiences to students (NSF, 1998). Additionally, those students who do have access to research opportunities often do not participate in these experiences until their final year (Ryder et al., 1999).

Teaching the process of science is critical at the introductory undergraduate level, however, because many students enter college with minimal understanding of the scientific endeavor beyond methodical procedure (Moss et al., 2001; Bell et al., 2003). Unfortunately, the vast majority of students are never taught the scientific process; instead, science is presented to them as a system of discipline-specific facts to be memorized – comparable to memorizing a poem in a foreign language without understanding the vocabulary. As a result, non-science majors express high anxiety and low motivation when confronted with decisions that involve scientific information (Gogolin & Swartz, 1992). Even science majors express limited understanding of the creativity involved in the process of science (Ryder et al., 1999). This may

3 The Process of Science Carpi contribute to the high attrition rates documented in the first two years of the science major

(Seymour & Hewitt, 1997).

TEACHING THE PROCESS OF SCIENCE

Though few studies have been conducted on the effectiveness of teaching the nature and process of science, the results of those studies are encouraging. Misconceptions regarding the scientific process often rest on language or terminology that can be easily clarified: Dagher and

Boujaoude (1997) have concluded that one cause of the common student opinion that evolution is a very tentative theory is a lack of explicit teaching about the nature of the word “theory.”

When instruction in the nature of theories is explicitly included in the study of scientific theories themselves, student learning is improved (Dagher et al, 2004). When students are explicitly taught that science is a process and a product of debate, they begin to understand the process in a more intelligent manner, giving them the perspective to better appreciate current socioscientific issues such as global climate change (Schmidt, 1996; Monk & Osborne, 1998).

These results suggest that it is important to introduce students to the nature and process of science early in their undergraduate careers. All students are generally required to take at least one introductory-level science course in college. These courses have tremendous potential to provide them with an understanding of the nature and process of science through inquiry-based experiences and exposure to ongoing research in the literature. Instead, in the vast majority of introductory courses, the traditional lecture format dominates and science is presented as a collection of facts determined by content in a textbook (Astin & Astin, 1992; Moulton, 1994).

Textbooks, however, have proven to be poor purveyors of scientific understanding

(Blosser & Helgeson, 1988). Few contain explicit information about the process of science beyond an introductory chapter about the scientific method; though there are notable exceptions

(see Smith and Pun, 2005). Among other cited shortcomings, many science textbooks have been

4 The Process of Science Carpi identified as "low-cohesion," presenting science as a string of disconnected facts that make it difficult for students to see the relationships between concepts (Best et al., 2005). Textbooks commonly present watered-down, popular versions of science that ignore controversy or uncertainty, and fail to provide any content to convey the process of scientific discovery (Brush,

1974). They focus on solving isolated problems, ignoring the research that suggests that discussing the nature of scientific experimentation provides students with a deeper appreciation of the subject matter and builds better concept recall (Nurrenbern & Pickering, 1987).

The few textbooks dedicated to the process of science are targeted to primary and secondary school teachers (Rezba et al., 2003), or to primary and secondary school children themselves (Pearson, 2005). Unfortunately, these books often fare poorly when externally reviewed (Bennetta, 1993). Available online resources on the nature of science tend to target the

K-12 audience (NAP, 1998) or are limited to a brief discussion of the scientific process (Farabee,

2002). Thus, the obligation falls upon the instructor to include explicit instruction about the nature of science in his or her class with little textbook support (Hurd, 1998).

Some guidance for teaching the process of science does exist. Flick and Lederman (2004) edited a collection of essays on teaching the nature of science. Frederick Reif’s speech and paper following his receipt of the Millikan Medal for “creative contributions to the teaching of physics,” provides a discussion of the processes critical for students to properly interpret scientific concepts and principles (Reif, 1995). And Tobin (1990) has discussed the principles that should be conveyed in laboratory class to foster inquiry and an understanding of the scientific process. Yet there is still no widespread integration of these concepts into the typical introductory science classroom where the vast majority of students could benefit the most.

5 The Process of Science Carpi

PROPOSED WORK

The overall goal of this project is to develop, test, and refine a one-semester, undergraduate curriculum for teaching the nature and process of science based on collaborative input from a variety of experts in scientific disciplines, education, and cognitive science. Such a curriculum has the potential to revolutionize the way science is taught to science majors and non- majors alike, providing access to high quality science education for students at a variety of institutions. By integrating the proposed materials into an established and nationally known website for teaching interdisciplinary science, these materials can be assembled as a stand alone course, or they can be integrated into introductory science courses in chemistry, earth science, and other disciplines. As such, this project will not only catalyze a new agenda in science education reform, but it will present this agenda in a way such that it can be integrated gradually into existing curricula, avoiding the barriers to adoption that radical curricular change may encounter. Further, this will allow these reforms to be reinforced in several courses across a variety of disciplines, as promoted in the literature (Dagher et al, 2004).

The work proposed is a collaboration between John Jay College of CUNY, Stanford

University, and the University of Eth n ic C o m p o s itio n o f th e J o h n J a y S tu d e n t B o d y O th e r Africa n 1 % Am e rica n California Museum of Paleontology. H is p a n ic 2 9 % 3 9 % John Jay College is a leading minority- As ia n 4 % and Hispanic-serving institution (Fig. C a u ca s ia n 2 7 % 1), thus this project connects a diverse Figure 1 set of influential institutions to assure its wide dissemination and broad impact. Further, this project brings together the Visionlearning project at CUNY and the Understanding Science project at UCMP. Visionlearning was funded in

2000 and again in 2002 by the National Science Foundation to produce undergraduate curricular materials for teaching interdisciplinary science. Understanding Science was funded in 2006 by

6 The Process of Science Carpi the National Science Foundation to identify critical principles for teaching the process of science, and produce web-based resources that convey these principles. As such, this project is uniquely positioned to achieve the stated goal of developing a curriculum for teaching the process of science to all undergraduates and mounting this curriculum so that it can be taught as a stand- alone course or integrated into existing disciplinary science courses. Toward achieving this goal, this project has four specific objectives.

Objective 1: Assemble an interdisciplinary panel of experts in science and science education to develop a one-semester, undergraduate curriculum on the Process of Science.

Experts in a wide variety of fields have no problem identifying what science is (and what it is not), despite the fact that the actual procedures may differ significantly between fields.

However, existing studies that evaluate student gains from research projects define the nature of science only very vaguely (see Seymour et al., 2004, for summary). Our objective is to define the nature of science such that a core undergraduate course can be developed.

As part of their funded grant, Understanding Science will convene an Advisory Board in

September 2006 that will consist of experts in several scientific disciplines. The Understanding

Science Advisory Board will be charged with defining the concept of natural science, and establishing a set of characteristics critical to understand the nature of the scientific process science and scientific research. This work will serve as a foundation upon which we will build, and in preparation for the proposed work, the Project Coordinator for this proposal (Ms. Egger) will attend this meeting. Thus, even before a grant is made under the FIPSE program, this collaboration will have created an extensive outline of key scientific concepts important to understanding the nature of science.

As soon as a funding notification is made, we will assemble our own panel of interdisciplinary experts who will be charged with using the Understanding Science outline and

7 The Process of Science Carpi the existing literature (Tobin, 1990; Reif, 1995; Flick & Lederman, 2004) to develop an undergraduate curriculum on the process of science and outline a series of teaching modules

(described further in objective 2) to be used as course readings to support faculty teaching this course. Our expert panel will meet virtually to discuss their objectives, and will then convene for a 2.5 day face-to-face conference to outline these materials. We have already contacted and confirmed the participation of several key educators for this panel. Panel participants include

(full Biographical Sketches and Letters of Support are included in the appendices):

Dr. Anthony Carpi (Project Director) is the founder of Visionlearning, recipient of two NSF grants to develop innovative educational materials, and has published broadly on the effect of novel online resources on improving undergraduate science education (Carpi, 2001; Carpi, 2003; Carpi & Mikhailova, 2003). Ms. Anne Egger (Project Coordinator) is the Undergraduate Program Coordinator in the Department of Geological and Environmental Sciences at Stanford University, where she has redesigned the undergraduate curriculum and directs the undergraduate research program. Dr. Norman Lederman is internationally known for his research and scholarship on the development of students' and teachers' conceptions of the nature of science and has published widely on methods used to assess student and teacher understanding of the nature of science (Lederman, 1992; Lederman, 1999, Lederman, 2002). Dr. Cathy Manduca is Director of the Science Education Resource Center (SERC) at Carleton College and has earned a national reputation for her work with projects that support improvements in geoscience and science education. Ms. Judy Scotchmoor is Assistant Director of the University of California Museum of Paleontology, and the Project Coordinator of the NSF-funded Understanding Science project. Dr. Natalie Kuldell is the designer and coordinator of the Biological Engineering program at MIT, and has written on the use of research in the teaching of undergraduate science Ms. Sandra Swenson (Evaluation Coordinator) has extensive experience teaching and evaluating the impact of science teaching resources; she is a doctoral student in science education at Columbia University.

To assure that the objectives of the Department of Education are fully considered in this work, we will consult with our FIPSE program director to determine three additional panel members with experience in science research and education. The panel will have the following charges:

1. Compare and contrast how the scientific method is practiced in diverse disciplines. The

scientific method is perhaps one of the most commonly misinterpreted concepts in

undergraduate science education (Bauer, 1992). Current texts contain major errors:

portraying the scientific method as absolute, suggesting that a single, linear scientific method

exists, and suggesting that the acceptance of scientific knowledge is straightforward

(McComas, 1998). Our goal is to portray the scientific method accurately as it is practiced in

diverse fields of science as a non-linear, self-reinforcing process.

2. Describe key aspects of the process of science. The panel will be asked to develop the key

characteristics of science identified by the Understanding Science Advisory Board into a full

set of curricular materials for undergraduate education.

3. Identify major scientific revolutions, important scientists, major scientific controversies,

and key technological breakthroughs within and across disciplines. Scientific advances are

not always gradual, and understanding the contribution of individuals and the nature of

scientific revolution is key to understanding the process of science.

Objective 2: Create a set of curricular modules and associated teaching materials to be used in Process of Science courses and launch the materials on the Visionlearning website.

A major strength of this proposal is that it builds on an established national-scale project for creating and disseminating undergraduate science teaching content. The Visionlearning project has developed a teaching module design that has proven effective and is widely used by instructors (described in detail later in this proposal). These modules consist of 3-4 page explorations of a single concept that support classroom instruction. While specific module topics will be outlined by our panel, a draft outline of possible module topics is included in Table 1 below to provide a sense of the nature of the materials we will create.

Table 1: Key Concepts and Possible Module Topics to be Developed in this Project Key Concept Specific Module Topics Scientific methods o Characteristics of non-linear scientific methods o Variations in the scientific method between disciplines The evolution of science through time o The development of scientific thinking o The role of key scientists

o Instrumentation and analytical advances o What makes a scientific revolution o What makes a scientific controversy The practice of science o Experimentation o Analyzing data o Error, uncertainty, and statistics o Modification of theories Communicating science: o Scientific writing and peer review o Science and policy o Science and law o Science and belief systems o The language of science: the meaning of terms like theory, data, etc. o Public perception of science o Scientific ethics: fraud and objectivity Science today o Scientific research profiles o Critical reading of the literature

Five key concepts are currently envisioned as making up this work, which will be later refined by our panel of experts. 1) Scientific methods: will consist of two to three modules that convey the scientific method as it is practiced - in a non-linear fashion that differs between different disciplines. 2) The evolution of science through time: will consist of five modules that describe the origin of scientific thinking, what can cause a scientific revolution or controversy, and the role of key scientific figures and instrumentation discoveries in the advancement of scientific thought. 3) The practice of science: will consist of four to five modules that describe key aspects of scientific research as it is broadly defined across multiple disciplines, with particularly detailed explanations of language where appropriate (such as the words “theory” and

“hypothesis”). 4) Communicating science: will consist of seven modules that describe the major modes in which scientific theories and developments are communicated to other scientists and to the public, and how they can affect policy and law. Within this context we plan to explicitly address the difference between science and belief systems. Finally, 5) Science today: will consist of a series of supplemental resources that will be linked to from the core modules. These

10 The Process of Science Carpi resources will provide profiles of scientists and scientific research, and links to published research papers to provide a more detailed view of the process of science.

As a proof-of-concept of this proposal, we have already developed and published three modules on Visionlearning that focus on scientific process instead of content. Visualizing

Scientific Data: An Essential Component of Research, explores methods for the graphical presentation of data and the importance of these presentation modes in data analysis.1 The module presents examples of data from the fields of chemistry, earth science, and sociology and explores the relative effectiveness of various data presentation methods. Additionally, the module provides a data set on atmospheric CO2 concentrations from the Mauna Loa observatory that the user can download and graph to experiment with presenting data in a useful manner.

A second module, Scientific Writing: Peer Review and Scientific Journals, describes the process of peer-review and discusses the differences between popular magazines and scientific journals.2 The module presents a generalized format of scientific articles and discusses tips for reading and understanding scientific literature. To provide readers direct experience with the scientific literature, we have negotiated a relationship with Science magazine, and the research section of this module provides direct access to the research paper “Ivory-billed Woodpecker

(Campephilus principalis) Persists in Continental North America,” regardless of whether or not the user has a subscription to the journal (Fitzpatrick et al., 2005). When accessing this article through our site, teachers and students first pass through a page of targeted questions meant to focus their reading and stimulate a follow-up discussion of the material in the classroom.

For the “critical reading” component of the “Science Today” concept listed in Table 1, we plan to expand this relationship with Science and seek similar relationships with Nature,

Geology, and other journals in to provide students broad and seamless access to scientific research directly in the classroom. This represents a significant benefit to students at smaller

1 http://www.visionlearning.com/library/module_viewer.php?mid=109&l=&c3= 2 http://www.visionlearning.com/library/module_viewer.php?mid=123&l=&c3=

11 The Process of Science Carpi schools and community colleges. Subscriptions to these journals average $1660 per year, and are among the first expenses to be eliminated when budgets are cut (Willinsky, 2006). By connecting individual journal articles to content, we will provide all students with guided exploration of the current research.

A third module displays the type of supporting content that will be developed for the

“scientific research profiles” component of the “Science Today” concept listed in Table 1. Our biology module Adaptation: The Case of Penguins discusses the scientific concept of organism adaptation to the environment. The research section of this module links to a series of research diary entries written by Ms. Iris Saxer, an NSF-funded penguin biologist.3 These diary entries provide a candid and insightful view of the practice of scientific research. In the proposed grant period, we will produce a series of profiles about practicing scientists and their work to give students a more personal perspective on scientific research. These materials will be linked to from research section of the proposed Process of Science teaching modules.

Objective 3: Evaluate these materials in a Process of Science course to be mounted at

CUNY, in a disciplinary science course at Stanford, and at four other institutions.

This project includes a three-phase evaluation program (outcomes and attainment measures are further detailed in the evaluation table in the appendices). To assure high quality, evaluations will be administered by a semi-independent Evaluation Coordinator, Ms. Sandra

Swenson, who will also serve on our expert panel. An independent External Evaluator will conduct yearly formative evaluations to coincide with our annual reports to the Department of

Education, and will review and oversee all evaluations to assure that they are reliable, valid, and unbiased. Evaluations will draw on the National Research Council’s report Evaluating and

Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics and the assessment instruments referenced therein (NRC, 2003).

3 http://web.visionlearning.com/custom/diary/index.htm

The first phase of our project evaluation will focus on assessing the impact of the resources developed on student understanding of the nature of science at the collaborating institutions. Evaluation instruments will be produced in consultation with our expert panel and will be adapted from the published Views of Nature of Science (VNOS) Questionnaire, the

College Student Report from Carnegie Mellon’s Eberly Center for Teaching Excellence, and other validated instruments (Ryder et al., 1999; NRC, 2002; Lederman et al., 2002). This project will benefit significantly from Dr. Lederman’s direct participation on our expert panel. Table 2 contains four sample questions that might be used in our assessment tools to provide an example of the instruments we will develop.

Table 2: Examples of possible pre- and post-assessment instrument questions  After scientists have developed a theory, does the theory ever change? If you believe theories change, why do we bother to teach scientific theories?  [For GES 1] Explain the concept of plate tectonics. How certain are scientists about plate tectonics? What specific evidence supports plate tectonic theory?  [For NSC107] Explain a scientific concept you recently became aware of outside of class.* Is there evidence to support this concept? If so, what specific evidence exists?  To what extent did† the work in this course emphasize the following activities? Please rank your answer from 1 to 5, 1 being very little emphasis, 5 being very much emphasis. a) Memorizing b) Analyzing c) Synthesizing information d) Making judgments e) Applying concepts in new situations * The ability to cite a valid scientific concept may itself indicate an understanding of scientific process. † “Did” will be replaced by “do you expect that” in the pre-test.

John Jay College has committed to pilot the implementation of a set of the Process of

Science materials in the College’s interdisciplinary, non-majors laboratory science course

NSC107: An Introduction to Science in Society (see enclosed Letter of Support from the College

President). Once the core teaching materials are complete, two to three lecture sections (n ~ 30 per section) will be taught using these materials as their core readings. The adapted VNOS instrument described above will be administered as a pre- and post-test in these sections as well as in an equivalent number of control sections of the NSC107 course taught with traditional

13 The Process of Science Carpi science content resources. Results will be analyzed by our Evaluation Coordinator, submitted to our External Evaluator for validation, and presented to our expert panel for review.

Research has suggested that integrating the process of science into traditional disciplinary courses is a powerful way of building student understanding of the nature of science (Dagher et al, 2004). Non-project instructors are more likely to incorporate new process of science materials gradually into their curriculum before committing to a major teaching revision. As such, we will also evaluate the effectiveness of a subset of the materials we are developing when integrated into the disciplinary course GES 1: Fundamentals of Geology, at Stanford University. Because only one section of this course is taught in any given quarter, which precludes using the case- control design, baseline data will be collected for two quarters prior to the materials implementation as soon as a funding notification is made. The adapted VNOS will be administered as a pre- and post test to assess student understanding of the process of science and comprehension of core theories taught in the course.

In addition, all students enrolled in the sections of NSC107 and GES 1 in which process of science modules are used will complete a short assessment paired with each new module, a functionality already in place in the Visionlearning website. We will develop individual quizzes that contain several questions designed to gauge student understanding of the content presented in each module, and additional questions to assess opinions regarding the clarity and utility of each module. This will provide us a measure of the relative strength and utility of each module developed for the curriculum.

In the second phase of evaluation, we will expand to additional institutions by providing a small stipend to the first four instructors on our site who teach a course using at least six of these modules and who agree to participate in the evaluation. The adapted VNOS instruments will be placed online to facilitate their use. Participating faculty will be asked to administer the validated instrument as a pre- and post-test. Our Evaluation Coordinator will direct these

14 The Process of Science Carpi assessments to test the progression in thinking about the nature of science that occurred as a result of using the materials created in this project.

In the third phase of our evaluation, we will initiate an assessment of teachers to determine the impact of these materials on teaching strategies. An online questionnaire will be given to teachers, this questionnaire will be designed to:

1. Evaluate the impact of the developed materials on teachers’ understanding of the process and nature of science;

2. Study how teachers use these materials; and

3. Examine how teachers perceive that the resources affected their students’ understanding of the process of science.

This aspect of the evaluation will be used to guide development of support materials for teachers that will describe how the Process of Science materials can be used in the classroom.

Visionlearning has a successful model for creating faculty “help” modules designed like our teaching modules (www.visionlearning.com/teach_tech.php). We will draw on our collaboration with SERC to develop similar modules into a series of gateway materials to integrate into their

“Starting Point” collection. For many college science faculty, the concept of science as a process is so inherent to their thinking that they neglect to address it explicitly in the classroom. When awakened to the learning gains that can be made, many of these same faculty adopt new practices to engage their students. For this reason, a key component of this project is to give faculty the tools they need to integrate these materials into their classrooms.

All evaluation results will be submitted to the peer-reviewed literature for publication and dissemination. In addition, evaluation data will be reviewed by our expert panel to allow them to prepare a series of recommended revisions to our curriculum that will maximize its effectiveness.

This iterative process of evaluation will assure that the products are successful and will be widely adopted once they are fully disseminated.

Objective 4: Disseminate the Process of Science teaching materials into a national program for science education, and create a community of professionals to promote their use and continued assessment.

This proposal represents a significant innovation in the teaching of introductory science.

As with any radical change in curriculum, this may result in barriers to its adoption as a consequence of its modification in teaching content. To overcome these barriers, we will publish these resources on the existing Visionlearning website in context of existing science content modules. This will permit their gradual adoption into courses by allowing them to be paired with science content modules in many different disciplinary course settings.

The Visionlearning project (http://www.visionlearning.com) has a significant existing user base: with almost 4,000 instructors and more than 19,000 students currently registered with the site, it receives over 6 million hits per month (Visionlearning, 2006). As of spring 2006, over

300 online classrooms were active on the site, representing 80 different institutions. Thus the materials will be available to a wide audience as soon as they are published. To facilitate their adoption and use, we plan to notify all instructors registered with Visionlearning via email when modules are published (as we currently do for all new content modules). In the past, this has resulted in near-immediate adoption of these materials.

This project has already negotiated collaborations with digital libraries and science education centers around the country to help aid in its dissemination. In particular, our collaboration with the Science Education Resource Center at Carleton College will significantly aid in the project’s dissemination through their “Starting Point” collection

(http://serc.carleton.edu/introgeo/). Our resources are catalogued in the National STEM Digital

Library (NSDL), the Digital Library for Earth Systems Education (DLESE), and others.

Visionlearning has also negotiated a relationship with Kendall-Hunt Publishers to distribute an inexpensive textbook based on the learning modules available on the site for teaching

16 The Process of Science Carpi interdisciplinary science courses. This relationship will be extended in the proposed period to distribute an affordable text based on the Process of Science materials.

The broadest dissemination of these materials is most likely to occur through the development of a Community of Practice of science educators focused on teaching the process of science (Schlager et al., 2000). To facilitate the development of such a community, we plan to host workshops at major national science meetings (such as the American Geophysical Union,

American Chemical Society, etc.). During these workshops, the benefits of teaching the process of science will be discussed, and the materials developed in this proposal will be presented as a means of incorporating the process of science into the undergraduate classroom. Again, we will draw on the experience of our collaborator Cathy Manduca of SERC in developing and running similar workshops. In order to allow faculty to continue to stay in contact after these workshops are complete, we plan to develop a series of teacher-to-teacher communication tools during this project to facilitate the sharing of knowledge and practices among our users. These tools will include direct email, teacher blogs, bulletin boards, and chat areas.

WHY VISIONLEARNING?

The Visionlearning project was funded by the National Science Foundation to create an innovative system for providing interdisciplinary science content to college non-major courses.

This project has achieved or exceeded all of the objectives originally proposed (Boyd, 2006).

The current proposal seeks to expand this successful undergraduate teaching tool into an innovative new area – the explicit teaching of the nature and process of science. Visionlearning is particularly well-suited to the development and distribution of these materials.

The Visionlearning website hosts a library of more than 60 original teaching modules - concise descriptions of scientific concepts that integrate the history of discovery and promote interdisciplinary learning (http://www.visionlearning.com/library). These modules draw on the

17 The Process of Science Carpi pedagogical research to take a hybrid approach to the 30-page chapter seen in textbooks and the

<1 page outline/summary approach often seen on the web. They provide a 3-4 page concise description of a concept that promotes reading and is modeled after the scientific research paper.

Each module integrates a concept description with topical news stories, scientist biographies, and ongoing research through a series of categorized hyperlinks. Additionally, modules include an online assessment to provide students a measure of their understanding of the concept and to provide feedback to the instructor. Modules are written by collaborating scientists and educators, and undergo extensive review to assure accuracy and clarity before their publication.

The site incorporates a customizable content management system called MyClassroom

(http://www.visionlearning.com/myclassroom). This system contains standard features available with course management software like WebCT™, including an online syllabus, announcement system, and email system. Unlike other systems, MyClassroom focuses on content: allowing instructors to select a custom set of modules for use in their classroom, annotate and customize these modules to their needs, and create targeted online assessments to gauge student comprehension. This system will be especially valuable in the proposed project as it will not only allow faculty to select the specific order in which they choose to present the process of science modules, but it will also allow them to directly annotate these modules and, of particular significance, combine process modules with science content modules. Several researchers have concluded that content about the nature of science should be integrated into disciplinary content, allowing instructors to discuss the nature of scientific theories while students are learning about the theory of plate tectonics, for example (Ryder et al., 1999; Brickhouse et al., 2000; Dagher et al., 2004; Dagher and Boujaoude, 2004). Others have suggested that socioscientific issues (such as global warming) are the ideal context in which to study the nature of science (Sadler et al.,

2004). Thus, by integrating the proposed modules with the existing site content, this project will create an exceptionally useful resource for teaching the process of science.

This project has a demonstrated history of rigorous and high-quality evaluation (Carpi,

2001; Carpi & Mikhailova, 2003). Early stages of the project evaluated prototype teaching modules against traditional resources. The use of these tutorials reduced attrition in the target course from over 11% to less than 5%, increased student-instructor communication, and increased satisfactory performance (a final grade of C or better) from 68% to >80% (Carpi,

2001). In later tests of the Visionlearning modules, students in three sections of an undergraduate non-major science course had an average score of 77 on a standard science comprehension assessment, significantly higher (p<0.001) than the average of 62 for students in five other sections of the course who used a traditional textbook (Carpi & Mikhailova, 2003). When asked how they felt the Visionlearning materials presented science, almost 70% of students and 80% of instructors responded that the materials conveyed science as a process of inquiry, as opposed to a list of facts, and 63% of students stated that Visionlearning helped them to understand the practice of scientific research (Boyd, 2006). As a result, 88% of instructors stated that they have recommended our site to colleagues.

EXPECTED OUTCOMES

This is an innovative project that will undertake a major effort to catalyze reform in undergraduate science instruction. Teaching the process of science is widely cited by experts as leading to a significant increase in student understanding of the nature of the scientific process

(Reif, 1995; Schmidt, 1996; Dagher & Boujaoude, 1997; Dagher et al, 2004; Flick & Lederman,

2004). And yet, it is not widely taught in undergraduate introductory science courses as a result of the lack of teaching support materials. This problem is exacerbated at community colleges and small schools where the cost of supplementary teaching materials, innovative resources, and subscriptions to research journals may be beyond their budget (Willinsky, 2006). Thus students

19 The Process of Science Carpi are denied not only access to authentic research experiences, but access to an understanding of the research process as well. We anticipate the following outcomes from this project:

 By drawing on a collaboration of leading institutions and assembling a panel of experts in

the field, we will produce a curriculum and content support materials on the Process of

Science to enable the teaching of such courses at two- and four-year institutions.

 By incorporating these resources into a widely used science teaching website, we will

facilitate instruction in the process of science as a stand-alone course and as a supplement

to existing disciplinary science courses.

 By publishing our evaluation data in the peer-reviewed literature, we will add

significantly to the research base on the nature of science and student understanding.

 By teaching the process of science, we will create a more informed public who can base

their decisions on understanding the complex socioscientific issues we face in the world.

By the end of the proposed three-year project period, we are confident that this work will be a widely used and broadly cited success, further contributing to the reputation of the FIPSE program in catalyzing educational innovation.

REFERENCES CITED Allen (2005, August 11) “Kansas Moves Closer to Intelligent Design Curriculum,” National Public Radio, available at: http://www.npr.org/templates/story/story.php?storyId=4795205 Allchin, D. (2003) “Scientific Myth-conceptions,” Sci. Ed. 87(3):329-351. Astin, A.W., Astin, H.S. (1992) Undergraduate Science Education: The Impact of Different College Environments on the Educational Pipeline in the Sciences, Higher Education Research Institute, UCLA, Los Angeles. Augustine, N. (1998) Essays on Science and Society: What We Don't Know Does Hurt Us. How Scientific Illiteracy Hobbles Society, Science 279 (5357): 1640 – 1641. Bauer, H.H. (1992) Scientific Literacy and the Myth of the Scientific Method, University of Illinois Press, Urbana. Bell, R.L., Blair, L.M., Crawford., B.A., and Lederman, N.G. (2003) “Just do it? Impact of a science apprenticeship program on high school students’ understandings of the nature of science and scientific inquiry,” J. Res. Sci. Teach. 40 (5): 487-509 Bennetta, W.J. (1993) “Reviewing a middle-school book in the Prentice Hall Science Series: The Nature of Science,” The Textbook Letter, http://www.textbookleague.org/35natur.htm. Best, R.M., Rowe, M., Ozuru, Y., McNamara, D. (2005) “Deep-Level Comprehension of Science Texts: The Role of the Reader and the Text,” Top. Lang. Dis. 25 (1):65. Blosser. P.E, Helgeson, S.L (1988) “Textbooks Contribute to Student Misunderstandings,” Invest. Sci. Ed. 14 (3). Boyd, H.H (2006) “External Evaluation of Visionlearning Web Resources,” Final Report for Grant 0127246, prepared for the National Science Foundation. Brickhouse, N., Dagher, Z., Letts, W., and Shipman, H. (2002) “Evidence and warrants for belief in a college astronomy class,” Science and Education 11: 573-588 Brickhouse, N., Dagher, Z., Letts, W., and Shipman, H. (2000) “Diversity of Students’ Views about Evidence, Theory, and the Interface between Science and Religion in an Astronomy course,” J. Res. Sci. Teach.37 (4): 340-362 Brush, S.G. (1974) “Should the History of Science be Rated X?” Science, 183:1164-1172. Carpi, A. (2001) “Improvements in Undergraduate Science Education Using Web-Based Instructional Modules: The Natural Science Pages,” J. Chemical Education 78(12):1709. Carpi, A. (2003) “Designing Effective Instructional Web Pages,” in Simon, E. J. (ed.) Teachers Using Technology: Practical Ideas from the Classroom, Kluwer Academic, New York. Carpi, A., Mikhailova, Y. (2003) “The Visionlearning Project: Evaluating the Design and Effectiveness of Interdisciplinary Science Web Content,” J. College Science Teaching 23(1):12-15. Dagher, Z.R. and Boujaoude, S. (1997) “Scientific views and religious beliefs of college students: The case of biological evolution,” J. Res. Sci. Teach.34: 429-445 Dagher, Z.R. and Boujaoude, S. (2004) “Students’ Perceptions of the Nature of Evolutionary Theory,” Wiley InterScience Dagher, Z.R., Brickhouse, N., Shipman, H., and Letts., W. (2004) “How some college students represent their understanding of tscientific theories,” Int. Jnl of Sci. Ed., 26 (6): 735-755 Dye, L. (2000) Science Miseducation: Focusing on Facts Misses the Wonder of Exploration, ABC News, March 22, at: http://www.math.buffalo.edu/~pitman/science_education.html.

References pg. 1 The Process of Science Carpi

Farabee, M.J. (2002) Online Biology Book, Estrella Mountain Community College, http://www.emc.maricopa.edu/faculty/farabee/BIOBK/BioBookTOC.html. Fairchild, M. (2004) “Ripoff 101: How the current practices of the textbook industry drive up the costs of college textbooks,” CALPIRG Fitzpatrick et al. (2005) “Ivory-billed Woodpecker (Campephilus principalis) Persists in Continental North America ,” Science 308:1460-1462. Flick, L.B., Ledereman, N.G. (2004) Scientific Inquiry and the Nature of Science: Implications for Teaching, Learning, and Teacher Education, Kluwer Academic, Dordrecht. Gogolin, L., Swartz, F. (1992) “A Quantitative and Qualitative Inquiry into the Attitudes toward Science of Nonscience College Students,” J. Res. Sci. Teach. 29(5):487-504. Hofstein, A., Lunetta, V.N. (2004) “The laboratory in science education: Foundations for the twenty-first century.” Sci. Ed. 88:28-54 Hurd, P. D. (1998) “Science Literacy: New Minds for a Changing World,” Sci. Ed. 82(3):407- 416. Lederman, N.G. (1992) “Students' and Teachers' Conceptions of the Nature of Science: A Review of the Research,” J. Res. Sci. Teach. 29(4):331-359. Lederman, N.G. (1999) “Teachers' understanding of the nature of science and classroom practice: Factors that facilitate or impede the relationship,” J. Res. Sci. Teach. 36(8): 916 – 929. Lederman, N.G., Abd-El-Khalick, F., Bell, R.L., and Schwartz, R.S. (2002) Views of Nature of Science Questionnaire: Toward Valid and Meaningful Assessment of Learners’ Conceptions of Nature of Science, Journal of Research in Science Teaching 39(6):497-521 McComas, W.F. (1998) “The Principal Elements of the Nature of Science: Dispelling the Myths of Science,” In W.F. McComas (ed.) The Nature of Science in Science Education: Rationales and Strategies. Kluwer (Springer) Academic Publishers (pg. 53-70). Moulton, J. (1994) “How Do Teachers Use Textbooks and Other Printed Materials? A review of the literature,” Report to the Improving Education Quality Project, USAID, Arlington, VA, http://www.pitt.edu/~ginie/ieq/pdf/textbook.pdf. Monk, M., Osborne, J. (1998) “Placing the History and Philosophy of Science on the Curriculum: A Model for the Development of Pedagogy,” Sci. Ed., 81(4): 405-424. Moss, D. M., Abrams, E.D., and Robb, J., (2001) “Examining student conceptions of the nature of science,” Int. J. Sci. Educ. 23(8): 771-790 NAP (1998) Teaching About Evolution and the Nature of Science, National Academy Press, Washington. NCSE (2005) “Kitzmiller v. Dover Area School District: Legal Documents, Trial Materials and Updates,” National Center for Science Education, available at: http://www2.ncseweb.org/wp/ NSB (2006) Science and Engineering Indicators 2006, National Science Board, National Science Foundation, Washington, DC, available at: http://www.nsf.gov/statistics/seind06/ NRC (2003) Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics, National Academy Press, Washington, DC. NSB (2004) An Emerging and Critical Problem of the Science and Engineering Labor Force. Companion to the Science and Engineering Indicators 2004, National Science Board, National Science Foundation, available at: http://www.nsf.gov/statistics/nsb0407/ NSF (1998) Scientific and Engineering Research Facilities at Colleges and Universities, National Science Foundation #01-301, http://www.nsf.gov/statistics/nsf01301/c7.htm.

References pg. 2 The Process of Science Carpi

NSF (2002) “Visionlearning: An Interdisciplinary Science Education Web Portal,” NSF, CCLI- EMD grant #0127246. NSF (2006) Research Experiences for Undergraduates, National Science Foundation, http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5517&from=fund. Nurrenbern, S.C., Pickering, M. (1987) “Concept Learning versus Problem Solving: Is There a Difference?” J. Chem. Ed. 64(6):508-10. Oreskes, N. (2004) “Science and Public Policy: What’s Proof Got to Do with It?” Environ. Sci. & Policy 7:369-383. Pearson (2005) The Nature of Science and Technology, Science Explorer Series, Pearson Education, Upper Saddle River. Reif, F. (1995) “Understanding and teaching important scientific thought processes,” J. Sci. Ed. Tech. 4(4):261-282. Rezba, R.J., Sprague, C.S., Fiel, R.L. (2003) Learning and Assessing Science Process Skills, Kendal-Hunt, Dubuque. Rudolph, J.L., Stewart, J. (1998) “Evolution and the nature of science: On the historical discord and its implications for education,” J. Res. Sci. Teach. 35(10):1069-1089. Ryder, J., Leach, J., and Driver, R. (1999) “Undergraduate Science Students’ Images of Science,” J. Res. Sci. Teach. 36(2):201-219. Sadler, T.D., and Zeidler, D.L. (2004) “Sudent conceptualizations of the nature of science in response to a socioscientific issue,” Int. J. Sci. Educ. 26 (4): 387-409 Schlager, M.S., Fusco, J., Schank, P. (2000) “Evolution of an On-Line Education Community of Practice,” in K.A. Renniger & W. Shumar (Eds.) Building Virtual Communities: Learning and Change in Cyberspace. Cambridge University Press, NY. Schmidt, K. (1996) “Creationists Evolve New Strategy,” Science, 273(5274):420-422. Seymour, E., Hewitt, N.M. (1997) Talking about Leaving : Why Undergraduates leave the Sciences, Westview Press, Boulder, CO. Seymour, E., Hunter, A-B., Laursen, S.L., and Deantoni, T. (2004) “Establishing the Benefits of Research Experiences for Undergradutes in the Sciences: First Findings from a Three-Year Study,” Wiley InterScience Smith, G.A. and Pun, A. (2006) How Does Earth Work? Physical Geology and the Process of Science, Pearson Prentice Hall, Upper Saddle Rive, NJ. Tobin, K. G. (1990). “Research on science laboratory activities. In pursuit of better questions and answers to improve learning.” School Science and Mathematics, 90, 403–418. Visionlearning (2006) Site use statistics as compiled on July 01, 2006. Willinsky, J. (2006) The Access Principle: The Case for Open Access to Research and Scholarship, The MIT Press, Cambridge, MA

References pg. 3