Advanced Placement Exam Scores As a Predictor of Performance In

Philip M. Sadler, Robert H. Tai

Advanced Placement Exam Scores as a Predictor of Performance in Introductory College Biology, Chemistry and Physics Courses Survey data from 8,594 students in 55 randomly chosen colleges and universities finds that those having passed an AP science exam earn somewhat higher college science grades, but not enough to assume prior mastery. Moreover, half of this performance difference appears to be related to demographics and high school coursework and not to students’ AP coursework.

Introduction grades of 3 are equivalent to the outcome of AP course-taking itself Students who take Advanced a range of work representing may be overestimated. Dougherty, Placement (AP) examinations in high mid-level B to mid-level C Mellor and Jian (2006a) offered the school seek the option of receiving performance in college (College following insight: 1 credit for and bypassing introductory Board, 2006, p. 1). Much of those [AP] students college level courses. These exams are The purpose of this study is to ap- later success in college may constructed and graded by the College praise the College Board’s claim of the be due not to the AP classes Entrance Examination Board (College equivalence of Advanced Placement themselves, but to the personal Board) to “allow participating col- courses as measured by AP exam characteristics that led them leges and universities to award high scores with the comparable college to participate in the classes school students college credit, ad- course in three science disciplines: in the first place – better vanced placement, or both” (College biology, chemistry, and physics. This academic preparation, stronger Board, 2004, p. 6). While the granting is a complex issue because, to some motivation, better family of credit or placement remains at the degree, AP course-taking is a sorting advantages, and so on. These discretion of each individual college process that draws primarily highly selection effects will affect any or university, the College Board is motivated, high achieving students. comparison of AP and non-AP unequivocal in its interpretation of If this self-selection effect is ignored, students (p. 3). AP exam scores: Colleges vary considerably in their AP Exam grades of 5 are policies and in their advice to their equivalent to the top A-level Colleges vary considerably students concerning enrollment in work in the corresponding in their policies and introductory science courses. As a college course. AP Exam grades in their advice to their result, there exists a sizable population of 4 are equivalent to a range students concerning of students taking introductory science of work representing mid-level enrollment in introductory courses who have done well in AP A to mid-level B performance science courses while in high school, in college. Similarly, AP Exam science courses.

1. In this research we refer to the numerical score earned on the Advanced Placement exam (1, 2, 3, 4, or 5) as the “AP exam score.” We refer to the teacher’s assessment of the AP students’ coursework as the “AP course grade.” In its literature, the College Board generally refers to the AP exam score as the “AP grade.”

Fall 2007 Vol. 16, No. 2 yet do not “place out” in college. Since With tests scores in hand, students the AP exam is taken prior to college are assured by the College Board entry, one would expect that these High school AP courses that “Over 90 percent of the nation’s students, whom the College Board has corresponding to the 35 AP colleges and universities have an AP awarded passing exam scores, would examinations administered policy granting incoming students earn substantially higher grades, on the by the College Board in both credit, placement, or both, for qualify- average, upon re-taking an introduc- science and non-science ing AP Exam grades.”7 However, the tory college course in the same sub- disciplines are taught by fine print is more complex. Having ject. The apparent advantage is clear. 110,000 teachers in the US. an “AP policy” does not mean that Students exhibiting higher levels of colleges and universities follow the AP performance would most certainly College Board’s recommendation that be expected to have a stronger grasp and offer a means of certifying each an AP score of 3 should garner course of the relevant college content. They student’s mastery of course content at credit or advanced placement. Many should do better than students who the college level. In the sciences, stu- times, colleges only offer credit for have not had the advantage of what dents sat for a total of 311,595 exams scores of 4 or even 5. Some college is typically a second year of study in in environmental science, biology, policies do not grant either to incom- 3 the subject while in high school. The chemistry, or physics in 2005. Among ing students.8 intent of this paper is to characterize the these science test-takers, 59% earned While the College Board puts much performance of these students as both scores of 3 or higher (5-point scale). energy into the construction, monitor- an independent approach to the valida- By comparison, 62% of mathematics ing, and scoring of AP exams in an tion of Advanced Placement exams in test-takers earned scores of 3 or better. attempt to attain a measure of college biology, chemistry, and physics and as As a result, many of these students course equivalency, there is a consider- an estimate of the contribution of these would have grounds to expect their able range of opinion concerning the advanced high school courses. prospective colleges to offer them value of AP courses in the sciences and course credit in these disciplines when mathematics. Many professors believe 4 Background they matriculate. This outcome would that all students can benefit from taking The AP Program of the College be consistent with the original purpose introductory science courses in college Board provides “motivated high of Advanced Placement envisioned (National Research Council, 2002, p. school students with the opportunity by representatives of the faculties of 58). Others argue that college-level to take college-level courses in a high the three private academies (Andover, courses require the efforts of college- school setting.”2 High school AP Exeter, and Lawrenceville) and level faculty (Juillerat, Dubowsky, courses corresponding to the 35 AP three private universities (Harvard, Ridenour, McIntosh & Caprio, 1997). examinations administered by the Princeton, and Yale) responsible for Some colleges administer their own 5 College Board in both science and formulating the program. To promote placement exams. Others allow credit non-science disciplines are taught by this policy, the College Board makes only for students not majoring in that 110,000 teachers in the US. These available research studies that support specific field or award credit only after 6 exams are central to the AP Program its contentions. a student has taken and passed a higher-

2. College Board AP Central website. http://apcentral.collegeboard.com. 3. http://apcentral.collegeboard.com/members/article/1,3046,152-171-0-39036,00.html 4. “ACE [American Council on Education] notes that the practice of granting provisional credit for a grade of 2 is becoming more frequent because comparabil- ity studies have shown that many of these students are potentially qualified for college-level work -- credit is made permanent if the student satisfies another criterion, such as successfully completing the next course in the discipline.” http://apcentral.collegeboard.com/apc/public/colleges/setting_policy/index.html 5. “[For] the student of marked mathematical and scientific ability and interest, who is probably heading for a college major in science and a career in science or engineering … could take a one-and-a-half or two-year school course in chemistry, physics, or biology, designed to prepare him for admission to the sophomore college course in the field chosen for concentration in school [i.e. in college]. (PhillipsAcademy, 1952, p. 65.) 6. http://apcentral.collegeboard.com/article/0,3045,152-167-0-11592,00.html However, research that can be considered somewhat critical of the claims of the College Board concerning the AP program does not appear to be included at their website (e.g. Geiser & Santelices, 2004; Lichten, 2000). 7. http://apcentral.collegeboard.com/program/ 0,3060,150-0-0-0,00.html 8. Lichten’s (2007) sampling of colleges and universities found that only 30% of colleges accept an AP exam score of 3 for advanced placement, while 89% accept a score of 4, and 95% accept a score of 5.

Science Educator level course in the subject (National addition to attracting potentially more lowest score of 1 (out of 5) on the AP Research Council, 2002, p. 58). Some highly motivated and gifted students, exam have not demonstrated content universities offer special transitional there are other potential covariates: mastery. However, this outcome leads courses for those students who place AP courses are more prevalent in one to wonder whether this year for out of introductory courses, but do wealthier communities, are typically these students of study could have been not advance to the next level course taught by more experienced teachers better spent in a course with a differ- (National Research Council, 2002, p. (Burdman, 2000) and predominantly ent format. Also troubling are the low 59).9 Lichen (2000) argues that the enroll students who have already AP science exam passing rates among quality of AP courses has diminished studied the subject for a year. Much under-represented groups.12 to the point that the College Board’s of the existing AP research gives no recommendations are no longer ac- recognition to these factors. Studies Prior Research cepted by a large number of colleges, lacking controls suffer the potential Research on the value of AP particularly those considered highly weakness of overestimating the effects courses in the sciences has primarily selective (e.g. Harvard, Yale, MIT, of AP courses, because they conflate focused on accounting for differences University of Pennsylvania).10 An the influence of numerous factors and in freshman grade point average or NRC committee recommends that col- fail to account for the relationship of persistence to graduation, both rather leges should be strongly discouraged AP courses net of the covariates. While indirect measures of the equivalence “…from using college scores on AP an many assume that students who do of AP courses and introductory col- IB exams as the sole basis for granting well in AP courses in high school owe lege science courses. Others judge the automatic advanced placement out of their success to their AP studies, other performance of students who “place specific courses for majors, or out of factors may well be involved, such as out of” one or two semesters of the biology distribution requirements for coming from more advantaged homes introductory course in a scientific field non majors (Wood 2002, p. 125.)” and attending schools that emphasize and then enroll in a second or third The population of students who college preparation (Willingham & semester science course equivalent have taken AP courses in high school Morris 1986). Moreover, the fast pace to students who take the prerequisite and retake introductory courses has of AP courses, due in large measure course in college. Such studies have been has been largely neglected by re- to the focus on test preparation, may not controlled fully for the fact that searchers.11 Surprisingly, researchers not serve students as well as courses AP students generally have stronger have rarely structured past studies to that progress at a more moderate pace, high school preparation in math and reveal the degree to which AP courses focusing on topics in greater depth and English, contributing to the ability to bestow an added benefit upon students detail. This supposition led a recent na- “skip ahead.” Moreover, second and who take them, “… the existing em- tional study to posit that, “It is possible third semester science courses can pirical evidence regarding the benefits that AP students were at a disadvantage cover content unrelated to the pre- of AP experience is questionable in some classes or at some colleges” requisite course (e.g. second semester (Klopfenstein and Thomas, 2005, p. (National Research Council, 2002, p. physics usually entails a study of elec- 5).” Klopfenstein and Thomas refer to 194). While most assume that such an tricity, magnetism, and light, topics the body of AP research, a majority of outcome is unlikely, we are willing to distant from first semester kinematics which lack or include insufficient con- subject such hypotheses to scrutiny. and Newtonian mechanics). In most trols. We should keep in mind that, in For example, students reporting the cases, studies examine the perfor-

9. None of the courses participating in this study have that profile. 10. USA Today, 3/20/2006, Advanced Placement: A detour for college fast track? Mary Beth Marklein 11. Studies comparing student performance in second-year college courses typically compare those who have “placed out” with high AP exam scores to those with students who take the introductory level college course with either a failing AP exam score or no AP experience (Dodd, Fitzpatrick, De Ayala, & Jen- nings, 2002). No mention is found in the literature of the experience of those students who chose to retake the introductory college course with a passing AP exam score. 12. Fewer than half of under-represented minorities are “passing” the AP exam with scores of 3 or higher, while scores of 1 (the lowest score possible) are awarded to more than half of African American and Chicano students. Put in another way, students not “passing” the exam constitute 70% of Chicano and African-American students, while more than 50% of Native Americans and Puerto Rican students score below 3. Based on this measure, AP courses do not appear to be an effective means of AP exam preparation for many students. For more information, please consult the AP National Report 2005, http://www. collegeboard.com/student/testing/ap/exgrd_sum/2005.html

Fall 2007 Vol. 16, No. 2 Shellhammer, 1978). Willingham difference was reported to be statisti- The élan of having AP and Morris’ (1986) found that cally significant for one year of the about half of the grade differen- study. The AP-Class group earned courses, especially in tial (those with a college grade slighted higher grades in the second science and mathematics, average of B or above) of AP semester course than the AP-CR is well known. students over non-AP students group. Reported GPA’s in other college was accounted for by matching biology courses were also higher for mance of students at a single college pairs of 1878 students on their Non-AP students. or university or within a single state, academic background. Klopfen- • Persistence to college gradua- making generalization to the entire stein & Thomas (2005) found that tion – The combined number of U.S. problematic. These studies also of 28,167 students, those taking AP and honors courses taken in fall into two general categories: studies AP courses in high school did high school was not a significant with adequate control variables and not earn higher freshman grades predictor for persistence through those without. The latter group typi- in college. freshman to sophomore years in cally compared grand means between • College Science Course Grades. college (Geiser & Santelices, samples of AP and non-AP students or Ruch (1968) did not find a sta- 2004); Klopfenstein & Thomas correlated the performance of AP stu- tistically significant difference (2005) found students taking AP dents with other performance-related between matched pairs of AP courses in high school have no variables. We found that these studies and non-AP students on college higher probability of persisting typically appear as reports that are not grade in the courses related to the to a second year of college. Low- published in peer-reviewed journals. AP course taken or overall fresh- income students who take AP Often the source of funding for much man GPA. Dodd, Fitzpatrick, courses in high school graduate of this work is unclear. On the other and Jennings (2002) studied the college at a greater rate (Dough- hand, the “with controls” studies more performance of three groups of erty, Mellor & Shuling, 2006a) frequently appear in peer-reviewed 831 students in a second semester • Choice of Further Study – Ruch journals, having been subjected to biology course at the University (1968) used 21 matched-pairs more scrutiny and review. of Texas at Austin. They compare of students and found that that While we reviewed both forms of the mean grades of three student AP students were more likely to research, we summarized only the groupings: continue with the same subject findings of the “with-controls” studies 1. Those who earned credit for the in college. Chamberlain, Pugh, below, grouping the findings under prerequisite course with a score & Shellhammer (1978) found four commonly analyzed outcome of 3 or above on their AP exam academic progress, college GPA, variables. The 1986 Willingham and (AP-CR). and course-taking measures were Morris report was especially useful in 2. Those who earned lower than superior for AP students over 344 constructing this list. a 3 in their AP course and took matched sets of students (Cahow, • College Grade Point Average the prerequisite (AP-Class). Christensen, Gregg, Nathans, (GPA)– The total number of AP Strobel, & Williams, 1979). Will- 3. Those who did not take an AP courses taken in high school has ingham and Morris’ (1986) found class and enrolled in the perqui- little effect on predicting fresh- that of 1878 students, AP students site class (Non-AP). man college GPA, but AP exam were no more likely to take more scores are a strong predictor of AP-CR and Non-AP groups were than one year’s coursework in sophomore grades (Geiser & matched using high school class rank biological or physical science, Santelices, 2004). Using matched and SAT and ACT scores. Over a nor to major in those fields. sets totaling 688 students at four-year period, the Non-AP group All of the studies above controlled Indiana University, AP students earned higher mean grades than either for students’ high school performance were found to earn higher college the AP-Class or the AP-CR groups (i.e. rank, HSGPA, or coursework GPA’s (Chamberlain, Pugh, & in the second semester course. This

Science Educator completed) except for Chamberlain in order to be considered robust. For 2.1 million entering freshmen enrolled et al. (1978) and Dougherty et al. example, inclusion of SAT scores in degree granting institutions in Fall (2006a). Geiser & Santelices (2004), alone, entered as a control for high 2003. One might assume that on the Klopfenstein & Thomas (2005), and school-level learning, is insufficient average nearly half of all entering col- Dougherty et al. (2006a) included to account for prior science knowl- lege freshmen have taken at least one measures of students’ socio-economic edge learned in first-year high school AP course in high school.14 In reality, status. Ruch (1968), Dougherty et al. science courses. It is incumbent on many have taken more than one course. (2006a) and Klopfenstein & Thomas the scholar employing regression to Our estimate is that in 2004, those (2005) included high school quality explore alternative relationships that taking AP science exams represented ratings, while Willingham & Morris are potentially causal and to take great one of five college freshmen (Digest (1986) matched students by college care in considering, measuring, and of Educational Statistics, 2004).15 attended. including factors that could account The expansion of AP Programs is From this short list of controlled for additional variance. Put another also currently propelled by economic studies, it is apparent that the measure way, it is the attention that scholars and political support since several of AP coursework value varies with the pay to controlling for alternative states currently extend financial in- outcome variable considered and the hypotheses that is often a measure of centives to high schools that offer AP control variables included. We did not quality research. courses16, while others require all high find any studies that measured perfor- But, why is research on the direct schools to participate (Hershey, 1990; mance in introductory college science comparison of AP and non-AP students Willingham & Morris, 1986).17 Many courses while controlling for relevant important? Conventional wisdom states have mandated through legisla- covariates such as performance in high ascribes clear and overwhelming ben- tion the inclusion of AP enrollment in school courses considered prerequi- efits to Advanced Placement classes admissions decisions or the granting sites for AP science courses. Given in high schools. The élan of having of college credit for high AP exam that performance in prior coursework AP courses, especially in science and scores in public institutions of higher and is widely regarded as the strongest mathematics, is well known. As a re- education (Lichten, 2000). The U.S. predictor of subsequent performance, sult, the College Board’s AP program president has called for an addition of it is essential that this factor should has grown dramatically over the last 70,000 Advanced Placement teachers be included in any analysis seek- 20 years, with a sustained increase in science and mathematics, a near ing to isolate the relationship to AP in the number of exams including all doubling of current numbers.18 In addi- coursework itself, rather than merely subject areas of 9.3% (with an 8.8% tion, the U.S. Department of Education considering an aggregated analysis increase in number of students) annu- subsidizes AP programs for low-in- that lacks both detail and is unable ally. To offer some perspective, this come students (Klopfenstein, 2004).19 to estimate the relative contributions growth is more than five times faster This expenditure of public funds, in of various factors. The inclusion of than the growth in the US high school effect, subsidizes one particular edu- such controls provides an opportunity population.13 The ratio of students to cational program over others. Hence, it for refinement of models that more exams tracked closely until recently; is desirable that Advanced Placement accurately estimate the relationship their separation is indicative of an policies and programs be rigorously of AP coursework relative to other increase in the number of students tak- assessed so that scarce resources can factors. Regression models must be ing multiple AP exams (1.29 in 1970 be spent the most wisely. constructed using variables represent- vs. 1.72 in 2005). In 2005, 1.2 million A powerful motivating factor for ing viable alternative hypotheses to students took 2.1 million AP exams. students is the fact that one may enter the main relationship being studied According to the latest available data, college with course credits and in some

13. http://apcentral.collegeboard.com/members/article/1,3046,152-171-0-47040,00.html. 14. http://nces.ed.gov/programs/digest/d05/tables/dt05_180.asp. 15. http://apcentral.collegeboard.com/members/article/1,3046,152-171-0-39036,00.html. 16. Florida, Louisiana, and Utah. 17. South Carolina. 18. George W. Bush, State of the Union Address, 1/31/2006. 19. Higher Education Act Amendments of 1998, Title VIII, Part B, P. L. 105-244.

Fall 2007 Vol. 16, No. 2 cases with enough credits to bypass the analyzed survey data gathered from of participants, provided they are from freshman year, saving a year’s tuition students in 124 different first semester a well-selected random sample. (MacVicar, 1988; Pushkin, 1995). introductory college biology, chemis- Choosing to focus only on under- Troubling to many professors is that try, and physics courses in 55 colleges graduates, we omitted cases of 103 students may use AP credit in many and universities. These institutions graduates and 400 non-traditional colleges to satisfy their requirements were from a stratified random sample students. We also omitted 397 students for science courses and never take reflecting a nationally representative who spent their high school years another science course when in college distribution of institutional size and outside of the United States. Allowing (National Research Council, 2002, selectivity, from small liberal arts for some students in the intersection of p. 59). For example, Lichten (2000) colleges to large state universities. these various groups, our final sample found that only 22% of students who All participating introductory science included 8594 undergraduates. had scored “3” on their AP calculus courses met in lecture-style classes Of this total, 31% of the sample was exam enrolled in an advanced calculus with weekly recitations sessions and enrolled in biology, 42% in chemistry, course in college, while 17% took a laboratories and had enrollments of and 27% in physics. Females made up remedial calculus course. Though 10 students or more. 60% of the biology students, 55% of high school AP teachers may be very Of 9354 collected surveys in this the chemistry, and 42% of the physics talented, they generally do not have subsample, 1029 respondents reported students. Parental education was relat- the breadth of knowledge nor the most having taken an AP course in the sub- ed to the highest level of mathematics recent and relevant content knowledge ject in high school in which they were taken by students, with representation possessed by college faculty (Lichten currently enrolled in college. Of the of students taking calculus in high 2000). By accepting AP exam-related students who took AP courses, 316 school ranging from 33% of student course credits, students may be forego- earned scores of 3 or higher on the with parents having a high school di- ing the opportunity to learn about the AP exam, roughly 3% of the entire ploma or less to 49% of students with a most topical findings and issues from sample. While this can be considered parent holding a graduate degree. The college instructors whose “other” job a small number compared to the sample identified themselves as 77% it is to do research. 170,000 students who earned scores White, 7% Asian/Pacific Islander, 6% The striking expansion of the AP of 3 or higher on AP exams in these black, 5% Hispanic, 3% multi-racial, program offers an opportunity to subjects in 2005, the data used in this and 2% other or unreported. gauge the pedagogical influence of This study, by its very nature, may AP coursework. Merely presuming only examine students who take intro- that AP courses are comparable with … many students feel that while ductory-level college science courses. college courses ignores the opportu- AP is a good preparation for Students enrolled in colleges that offer nity we have to analyze the degree of college science, many also feel the option of bypassing introductory- this purported similarity as a means to they benefited from taking the level science courses with sufficient improve college preparation through AP exam scores and who also make actual college science course. high school instruction. the decision to take advantage of this option to place out are not included. Methods analysis represents a national sample Yet, since prior research studies have The Sample of students who have both taken part generally ignored this interesting Factors Influencing College Science in Advanced Placement courses and population, studying these students Success (Project FICSS) is a large- exams in high school and also take can make a valuable contribution to scale national study of introductory introductory level college science understanding the value of Advanced college students and their high school courses. The theory and practice of Placement programs. science experiences, which collected a inferential statistics has established Students who did poorly on AP total of nearly 18,000 surveys. The data that valid inferences (within ap- exams in a given science discipline includes a series of subsamples of col- propriate confidence limits) can be may choose not to take the college lege students in various college science drawn from relatively small number course. By comparing the national courses. In this particular analysis, we statistics for AP exam scores with the

Science Educator AP Exams and College Performance in Science

FigureFigure 1 1:. Distribution Distribution of of AP AP ExamExam ScoresScores in in Sample Sample vs. vs. National National Norms Norms

40%

35%

30%

25%

Sample 20% National Norms

15%

10%

5% Error Bars ±1 SE 0% AP 1 AP 2 AP 3 AP 4 AP 5 distribution in We our sample,were also one careful can see to checka relative for anydearth anomalies of students in earningthe makeup Placement of students exam who in the discipline of earned scores of 3, 4, or 5 on their AP exams. Table 1 displays a comparison across differences (Figure 1).20 The five AP low AP exam scores. their introductory college science seven measures of academic performance among nine high school science preparation 22 score categoriescategories eachranging average from about Computed We grades were also represent careful theto check mean for grade course for a andparticular received a score of 5. In one-fifth of the students who take the any anomalies in the makeup of stu- general, AP students outperformed group21 (on a 100-point scale where A=95, B=85, etc.). “Not Taken” for students reporting exam.thatIt appears they had that not students taken at a the high schooldents who course earned in thescores discipline of 3, 4, or of 5 their honors introductory students, who in turn outper- low endcollege (AP exam science scores course of 1 andto “AP 2) 5”on for their students AP exams. reporting Table 1they displays had takenformed the Advanced regular students, who in turn and atPlacement the high end exam (AP examin the scores discipline a comparison of their introductory across seven college measures science outperformed course and students with no high of 4 received and 5) are a score under-represented of 5.22 In general, of AP academic students performance outperformed among honors school students, coursework who in in the correspond- in theturn sample. outperformed Presumably regular a larger students, nine high who school in turn science outperformed preparation students ing college with scienceno high discipline. Within fractionschool of those coursework who scored in below the corresponding 3 categories college ranging science from Computed discipline. Withinthe AP group,the AP we see a clear pattern of chose group,not to enroll we seein a collegea clear course.pattern ofgrades higher represent performance the mean corresponding grade for a tohigher higher performance AP exam corresponding to Many scores.at the higher High end scoring arguably AP took 4 and APparticular 5 students group consistently (on a 100-point reported scale higherhigher APSAT exam scores, scores. High scoring advantagemathematics of placement grades, to enterin-discipline a where science A=95, grades B=85, etc.). for regular “Not Taken” or honors AP courses,4 and AP and 5 students consistently higher-levelgrades course in other or received science credit courses. for Higher students scori reportingng students that theyin our had sample reported also reported higher SAT scores, math- higher grades in their AP courses, on average. Students in our sample who chose not to for the introductory course. The mean not taken a high school course in the ematics grades, in-discipline science AP exam score for the national data is discipline of their introductory college grades for regular or honors courses, slightly lower (2.8 1 ) than the sample science course to “AP 5” for students and grades in other science courses. used in this analysis (2.99) reflecting reporting they had taken the Advanced Higher scoring students in our sample 22 Regular signifies that the highest level of course taken in high school was neither AP nor honors level. Honors signifies an honors course. AP courses are broken down by AP exam scores of 1 through 5 or the 20. The categorynational statistics of students for AP biology, not taking chemistry, an AP and examination.physics have been Means averaged. are reported for the SAT math or SAT verbal 21. Onescores should note(or thethat theequivalent distribution from of scores the for ACT), the AP the exam fraction is not Gaussian completing as is reported a calculus for most course national in standardized high school, exams the such last as the SAT or ACT. Whilegrade those inat themathematics, center of the distribution the mean can grade be thought in the of relevantas earning closescience to the preparatory mean score, those course at the (if 4 leveltaken), or above the mean(35%) or at the 5 level (16%) occurgrade with much in the greater AP coursefrequency (if than taken), students and at one the standard average deviation of science above grades(16%) or in two fields standard other deviations than abovethe college (2%). science 22. Regularcourse signifies of interest. that the highest level of course taken in high school was neitherAP nor honors level. Honors signifies an honors course. AP courses are broken down by AP exam scores of 1 through 5 or the category of students not taking an AP examination. Means are reported for the SAT math or SAT verbal scores (or the equivalent from the ACT), the fraction completing a calculus course in high school, the last grade in mathematics, the mean grade in the relevant scienceSadler preparatory and Tai course (if taken), the meanDraft grade 8/13/07 in the AP — course Do Not(if taken), Duplicate and the averageor Quote of science grades in fields otherPage than the10 college science course of interest.

Fall 2007 Vol. 16, No. 2 also reported higher grades in their AP various high school science courses students in the field or their academic courses, on average. Students in our including AP, regular, or honors). On advisors. Five took the course because sample who chose not to take the AP the contrary, we found in our sample of department requirements in their exam, even after taking the AP course, that students who reported an AP disciplines. Three reported AP exam reported academic performance mea- exam score of 5 also averaged 1300 scores of 4, while their college required sures similar to the students in the AP on their SAT’s, with three-quarters a score of 5 for credit. Two reported 3 category. having taken calculus in high school, low scores on departmental placement In our analysis, we were particu- and also reported earning an “A” on exams required by their colleges to larly concerned with the academic average in their high school science earn course credit. All but one felt performance in college of students who and mathematics courses. that their AP course prepared them reported receiving AP Exam scores of Who are the group of students who well for introductory college science. 3, 4, or 5. We wondered if the students have taken an AP course in high school However, eleven reported that taking in our sample were particularly weak in and have earned an AP exam score the introductory college course as a these science disciplines despite theirAP Examsgreater and than College 3? Why Performance are they taking in Sciencean very worthwhile experience that paid high AP exam scores. Might a weak- introductory course in college? Were off for them later in their college sci- ness in their academic background they advised or required to take this ence course work. Four noted, in ret- have ledtake them the toAP pursue exam, additional even after coursetaking by the their AP college? course, To gain reported a better academicrospect, after performance taking advanced courses, coursemeasures work at the similar introductory to the col students- understanding in the AP of possible3 category. motivations, that they should have moved directly lege level?In our We analysis,imagined that we students were particularly we contacted concerned 44 students with who the both academic to higher-level performance course in work. with highcollege AP exams,of students but relatively who reported scored receiving well on AP AP exams Exam and scores took of 3, 4,As or one 5. of We those wondered students noted: if weak the academic students backgrounds in our sample may wereintroductory particularly college weak science in coursesthese science • Idisciplines very much despite should have been not betheir fully fluenthigh APin English,exam scores.or may Mightin these a disciplines. weakness These in their students academic had background[granted] credit. have Theled class was have weakerthem to math pursue backgrounds, additional or coursevolunteered work their at the contact introductory information college level?pretty much We imagineda waste of my time. may have earned lower overall high in response to a question asking if they that students with high AP exams, but relatively weak academic backgroundsIn contrast, others may offered not more schoolbe grades. fully To fluent address in theseEnglish, pos- or maywould have agree weakerto future contactsmath backgrounds, in light or may have earned sibilities, we analyzed these various of survey related queries. Among the positive reports of their college science lower overall high school grades. To address these possibilities,course we analyzed experiences: these backgroundvarious measures background and did not measures find responses, and did fifteen not find students that, compared provided with other students, • It was worthwhile and I am glad that, comparedhigh AP withexam other scorers students, exhibited detailed any replies. weaknesses While these in languageresponses proficiency (SAT Verbal I chose to take the intro level high APscore), exam mathematicsscorers exhibited proficiency any should (SAT not Mathbe considered score, highrepresenta calculus- course taking patterns, course. I recommend it to fresh- weaknesseshigh school in language mathematics proficiency course tive, grades),they do offer or overall some insight high intoschool grades (grades in man coming in who face a similar (SAT various Verbal score),high school mathematics science coursesthe diversity including of students’ AP, experiences regular, or honors). On the contrary, situation. Taking the introductory proficiencywe found (SAT in our Math sample score, that high andstudents opinions. who Five reported made the personalan AP exam score of 5 also course refreshes my memory on calculusaveraged course taking 1300 patterns, on their high SAT’s, decision with three-quar to take the introductoryters having course taken calculus in high school, the subject, provides deeper schooland mathematics also reported course grades), earning or aneven “A” though on average they could in their have highplaced school science and insight into the subject that just overallmathematics high school grades courses. (grades in out, citing advice from more advanced Table 1: 1. Comparison of of High High School School Performance Performance Measures Measures by Group by Group

Highest HS Other Grade in Course or AP SAT HS Last Math Science Honors or Score Sat Math Verbal Calculus Grade Grades Regular AP Grade not taken 475 454 27% 86.0 88.9 regular 529 510 32% 87.7 89.4 88.8 honors 596 564 60% 89.4 90.8 90.4 AP, no exam 610 574 70% 89.6 91.1 92.4 87.6 AP 1 543 508 47% 85.2 88.4 90.0 85.0 AP 2 592 541 54% 90.1 90.5 91.5 88.1 AP 3 595 568 61% 90.1 90.9 91.8 88.9 AP 4 631 628 64% 91.0 92.1 93.9 92.2 AP 5 666 654 75% 92.5 93.0 94.0 94.1 Who are the group of students who have taken an AP course in high school and have earned an AP exam score greater than 3? Why are they taking an introductory courseScience inEducator college? Were they advised or required to take this course by their college? To gain a better understanding of possible motivations, we contacted 44 students who both scored well on AP exams and took introductory college science courses in these disciplines. These students had volunteered their contact information in response to a question asking if they would agree to future contacts in light of survey related queries. Among the responses, fifteen students provided detailed replies. While these responses should not be considered representative, they do offer some insight into the diversity of students’ experiences and opinions. Five made the personal decision to take the introductory course even though they could have placed out, citing advice from more advanced students in the field or their academic advisors. Five took the course because of department requirements in their disciplines. Three reported AP exam scores of 4, while their college required a score of 5 for credit. Two reported low scores on departmental placement exams required by their colleges to earn course credit. All but one felt that their AP course prepared them well for introductory college science.

Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote Page 11 AP Exams and College Performance in Science

FigureFigure 2:2. Survey Survey QuestionsQuestions Relating Relating to Highto High School School Course Course Taking Taking in Science in Science and and MathematicsMathematics

Two major concerns loom for any large-scale self-report study, accuracy and reliability. wasn'tThese there in concerns high school, stem and from to a complete report theseby Brad surveysburn, while Rips, in their and Shevelland unreliable. (1987) However, that appeared in research preparesin the me journal for higher Science level. Thecollege study classes. concluded Using thatthe names self-report and datathat has were followed inaccurate in the interveningand coursesunreliable. in the subject. However, in researchID numbers that provided has followed by students in onthe interveningyears, which years,includes which further research includes further research by Bradburn himself, these conclusions have been qualified • It was definitely a worthwhile the cover, college instructors entered by Bradburn himself, these conclu- and mollified to portray astudents’ complex course and more grades. responsible After this conclusion:sions have been that qualified the accuracy and mollified experienceand reliabilityto retake the of classes, self-report likely depend on several factors including context, as it gives you a second chance process was completed, the perforated to portray a complex and more respon- relevance, and survey claritycover Pace, with student Barahona identification & Kaplan, was 1985; sible Bradburn, conclusion: 2000; that the Niemi accuracy & and to absorbSmith, the 2003). material—which In particular, self-reports of course taking, grades earned, and can never hurt. torn away and discarded, leaving the reliability of self-report likely depend standardized test scores madesurveys by anonymous, college stud exceptents for tend any to beon highly several accuratefactors including (Baird, context, • I was at least familiar with many 1976; Anaya, 1999). Enrollmentcontact informationreports are students especially may have accurate relevance, for courses and survey with clarity Pace, of the topics covered in my col- unambiguous names andvolunteered for high-achieving in response studentsto a “further (Sawyer, Barahona Laing & Kaplan,& Houston, 1985; Bradburn, 1989). lege class. Many concepts were In a recent review of existingcontact” research request. on Students self-report, also Kuncel, re- 2000; Credé, Niemi & & Thomas Smith, 2003). (2005) In par- covered with greater detail in conclude that self-report portedmay be on characterized a variety of demographic as reasonably ticular, accurate self-reports in samples of course where taking, college. I felt well prepared for the surveys address issuesvariables relevant (parents’ to the education, respondents. profes -Ourgrades study earned, surveyed and standardized test the college class, but I am glad I introductory college sciencesion, students etc). Home in ZIPthei codesr fall allowedsemester collegescores made science by college classes students where tend took the class instead of testing reflection on their prior experienceus to match thewould surveys be withcommonplace. existing to In be addition, highly accurate the students’ (Baird, 1 976; out. own instructors administereddata on the median surveys. income, home value, Anaya, 1999). Enrollment reports are The studentWe entered comments this indicate project withand averageprevious educational survey researchbackground experience especially in accuratewhich wefor surveyedcourses with that both motivationsover 2000 and college experiences physics of students each home in over locale. 20 different The survey college unambiguous and universities names and (Sadler for high- & associated Tai,with 2001).introductory The designcollege of thequestions current themselves survey wereincluded carefully the useachieving of student students focus (Sawyer, groups Laing to & science coursescomment differ widely.on question formats,designed two to fitpilot on surveysthe front andthat back included Houston, 304 college1989). In science a recent review The Surveystudents, a review by an advisoryof the remaining panel of three college sheets. professors See of and existing high research school onteachers, self-report, The surveyand instrumenta separate was reliability con- Figurestudy 2.where 113 college chemistryKuncel, students Credé, took & the Thomas survey (2005) structed to collecttwice, data two concerning weeks apart aca- andTwo where major their concerns responses loom for were any compared.conclude Much that self-report can be done may to be demic experiencesenhance and accuracy performance and recall:large-scale careful self-report design, study, contextual accuracy cues, characterized and participant as reasonably relevance accurate measures closelyall play associated important with high roles. and reliability. These concerns stem in samples where the surveys address from a report by Bradburn, Rips, and issues relevant to the respondents. Our school scienceIn large-scale course work thatsurveys, might missing responses are not uncommon. In dealing with missing Shevell (1987) that appeared in the study surveyed introductory college predict performanceresponses, in list-wise introductory deletion is the simplest and most commonly applied option. journal Science. The study concluded science students in their fall semester college scienceHowever, classes. list-wiseIn the form deletion of assumes that the data are missing at random. Should a that self-report data were inaccurate college science classes where reflec- a 4-sheet booklet,large studentspercentage were askedof the sample exhibit missing responses, this approach may not be the

Fall 2007 Vol. 16, No. 2 Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote Page 13 tion on their prior experience would ing responses and found that, for the AP course should be included. Table be commonplace. In addition, the variables of concern, only about 1-2% 2 shows the Pearson correlations be- students’ own instructors administered were missing for each. Given this tween academic preparation variables the surveys. characteristic, we chose to use list-wise and college course grade. We entered this project with pre- deletion. The two exceptions were Among the nine academic perfor- vious survey research experience in that no standardize test score (SAT mance measures, AP Score exhibits which we surveyed over 2000 college or ACT) was entered for 347 cases the highest correlation with College physics students in over 20 different (quantitative) and 386 cases (qualita- Grade in introductory science courses, college and universities (Sadler & tive). A mean value for such scores though other measures with similar Tai, 2001). The design of the current was substituted in these cases since correlations include: SAT Math, survey included the use of student we did not want to eliminate students “other” science GPA, and last high focus groups to comment on question from the dataset who had not taken school math grade. A central issue formats, two pilot surveys that in-AP Examsthese exams.and College Performance in Sciencefaced by those investigating the asso- cluded 304 college science students, a Choice of controls ciation between AP exam performance review by an advisory panel of college Our initial analysis compares the and college performance is that many professorsmost and appropriate high school teachers,option. Wemean analyzed college the grades percentage for students of missing other responses measures and of academic found perfor- and a separatethat, for reliability the variables study where of concern, with differentonly about levels 1-2% of high were school missing mance for each. are also Given positively this associated 113 collegecharacteristic, chemistry studentswe chose took to usecoursework list-wise de in letion. biology, The chemistry two exceptions with college were grades. that no the surveystandardize twice, two weekstest score apart and(SAT orand ACT) physics. was This entered particular for analysis347 cases (quantitative)Figure 3 offers and a graphical 386 repre- where theircases responses (qualitative). were compared. A mean valueonly shows for such the predictive scores was value substituted of AP sentation in these using cases the resultssince weproduced Much candid be not done want to enhance to eliminate accuracy studentsexam scores from on the college dataset course who grades, had notfrom taken our these sample. exams. College science and recall: careful design, contextual grades vary with AP exam score, Choice of controls quantifying the magnitude of the dif- cues, and participant relevance all play ference between students who take AP but they also vary with the other five importantOur roles. initial analysis comparesor the not, mean and those college who score grades at different for students measures with of different academic levels preparation. In large-scaleof high school surveys, coursework missing inlevels biology, on the chemistry AP exam. This and first physics. step ThisAs expected, particular more analysis rigorous levels of responsesonly are shows not uncommon.the predictive In valuedescribes of AP the exam differences scores that on need college to coursehigh school grades, science and mathematics dealingquantifying with missing theresponses, magnitude list- ofbe the explained difference by competing between hypotheses students whocourses take match AP with or highernot, and college sci- wise deletionthose iswho the simplestscore at and different most levels(Dougherty, on the Mellor AP exam.& Jian 2006a).This first stepence describes grades. Each the graph shows a clear commonlydifferences applied option. that need However, to be explainedHowever, by attempting competing to account hypotheses for monotonically (Dougherty, increasing Mellor &association list-wiseJian deletion 2006a). assumes that the the potential of taking an AP course between college grades and the six data are missing at random. Should a high school measures of academic However, attempting to accountrequires for the controls potential for demographic of taking an AP course requires large percentage of the sample exhibit achievement. It should be noted that controls for demographic andand academic academic preparationpreparation differences differences among students as missing responses, this approach may the average grade attained by students well. Far from being tabula rasaamong, students’ students asskill well. and Far fromknowledge being prior to, or unrelated to, not be the most appropriate option. who earn an AP score of 3 is equaled taking an AP course should betabula included. rasa, students’ Table skill 2 shows and knowl the- Pearson correlations We analyzed the percentage of miss- by students: earning a grade of “A” in between academic preparationedge variables prior to, or and unrelated college to, taking course an grade. Table 2: 2. CorrelationCorrelation Coefficients Coefficients of Academic of Academic Preparation Preparation Covariates Covariates Regular Last or College SAT SAT Math Last Eng Science Honors AP Grade Math Verbal Grade Grade GPA Grade Honors Calculus Score College Grade 1.000 SAT Math 0.315 1.000 SAT Verbal 0.200 0.330 1.000 Last Math Grade 0.317 0.288 0.108 1.000 Last Eng Grade 0.219 0.134 0.200 0.343 1.000 Science GPA 0.306 0.254 0.190 0.387 0.378 1.000 Regular or Honors Grade 0.269 0.238 0.166 0.306 0.265 0.464 1.000 Honors 0.128 0.199 0.140 0.107 0.098 0.129 0.063 1.000 Calculus 0.244 0.415 0.203 0.192 0.159 0.247 0.196 0.281 1.000 AP Score 0.328 0.311 0.404 0.216 0.189 0.277 0.207 0.004 0.157 1.000 Among the nine academic performance measures, AP Score exhibits the highest 10 correlation with College Grade in introductory science courses, though other measuresScience E ducator with similar correlations include: SAT Math, “other” science GPA, and last high school math grade. A central issue faced by those investigating the association between AP exam performance and college performance is that many other measures of academic performance are also positively associated with college grades. Figure 3 offers a graphical representation using the results produced from our sample. College science grades vary with AP exam score, but they also vary with the other five measures of academic preparation. As expected, more rigorous levels of high school science and mathematics courses match with higher college science grades. Each graph shows a clear monotonically increasing association between college grades and the six high school measures of academic achievement. It should be noted that the average grade attained by students who earn an AP score of 3 is equaled by students: earning a grade of “A” in an honors or regular science course, scoring a 750 on their quantitative

Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote Page 14 AP Exams and College Performance in Science

SAT, who take an AP course in calculus, or whose last high school mathematics course grade was an “A.” Figure 3. Course Grade and Values of Key Covariates. Figure 3: Course Grade and Values of Key Covariates. 100

80 College Science

70 Error Bars=± 1SE 65 B A B A D C D C AP 250 350 450 550 650 750 AP 1 AP 2 AP 3 AP 4 AP 5 honors regular Calculus not taken Algebra II precalculus Calculus AB Highest Honors or Calculus BC Science Regular SAT Math Score Highest Math Highest Math AP Exam Score Course Course Course Level CourseGrade Type Grade Results an honors or regular science course, The inferential analysis offers a and English, typical of many studies scoringDescriptive a 750 on their quantitativestatistics SAT, series of five multiple linear regres- accounting for high school grade point who takeTable an AP 3 displays course in calculus,a cross ortabulation sion models. between Model college A includes science only grades average and (HSGPA) the various or its correlate, high whoseacademic last high school performance mathematics levels thedisplayed AP status inof Tablestudent: 1. if Approximately they took school 13% rank-in-class. of students Model in E adds the courseintroductory grade was an “A.” courses did not AP,take if athey correspon did not takeding the course exam, orin highkind school. of course Roughly and performance half in any of the students in our sampleexam reported scores taking if they did only take the the exam.regular levelpreparatory high schoolcourse prior course to taking the in the disciplineResults correspondingIt explains to college only science2% of the course, variance with in aAP quarter course enrollingin the subject. at theThis model Descriptivehonors statistics level in high school andcollege 11% grade. enrolling This numberin AP coursesis to be of seeksthe corresponding to control for the influence of Tablescience. 3 displays Of athe cross students tabulation who expectedreported since enrolling the AP status in AP is only courses, one 433different took levels the APof preparation exam in biol- betweenand college reported science their grades scores. and the factor in the background of students. ogy, chemistry or physics. Model E various academic performance levels Model B includes a dummy variable can be interpreted as presenting the displayed in Table 1. Approximately for each professor (each separate difference in college performance 13% of students in introductory college course) and accounts for dif- that is related to taking an AP course courses did not take a corresponding ferences in grading severity between or to a certain performance on the AP course in high school. Roughly half of courses. This shows the predicted test between hypothetically identical the students in our sample reported tak- grades as if all the students enrolled students in all other variables, i.e. ing only the regular level high school in the same university course. This between students who had the same course in the discipline correspond- model shows a small upward change preparation and background in prior ing to college science course, with a in the predicted mean grade earned by years. It generates an estimate of the quarter enrolling at the honors level in each AP grouping. Model C adds in incremental effect of taking an AP high school and 11% enrolling in AP students SAT scores, or ACT scores course or having a certain performance courses of the corresponding science. converted using a concordance table on the AP test on the college grade in Of the students who reported enrolling (Dorans, Pommerich & Lyu, 1999) the same subject. Included for each in AP courses, 433 took the AP exam for both mathematics and composite model are the standardized beta (β) and reported their scores. scores. Model D includes variables coefficients that allow a comparison Inferential Analysis that account for general high school between the magnitude of the relative performance in science, mathematics, contribution of each variable to the Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote Page 15

Fall 2007 Vol. 16, No. 2 11 AP Exams and College Performance in Science

Table 3. Distribution of College Grades by Most Rigorous High School Course in the SubjectTable 3: Distribution of College Grades by Most Rigorous High School Course in the Subject

College Prerequisite HS Course Advanced Placement Course total Science not no took Grade taken regular honors e x a m exam AP1 AP2 AP3 AP4 AP5 55F 106 356 117 12 12 4 3 4 0 1 603 61D- 4 36 11 0 3 1 1 1 0 0 54 65D 102 381 127 22 11 3 4 2 2 0 643 68D+ 12 83 21 0 5 0 3 1 0 1121 71C- 48 163 57 5 15 2 5 5 3 0 288 75C 226 957 382 78 56 9 17 23 6 1 1699 78C+ 46 245 61 12 22 3 8 5 5 1 386 81B- 56 225 83 22 28 7 8 5 7 1 414 85B 243 934 518 134 79 6 17 38 12 6 1908 88B+ 44 226 102 27 56 5 16 14 10 11 455 91A- 36 202 90 22 39 1 6 15 10 7 389 95A 179 659 447 165 101 4 17 31 27 22 1551 98 A+ 1 31 40 5 6 0 1 1 2 2 83 total 1103 4498 2056 504 433 45 106 145 84 53 8594 Mean 78.79 79.18 81.98 85.21 84.35 78.11 82.11 84.30 87.15 89.77 80.42 SE 0.36 0.17 0.25 0.43 0.47 1.63 0.96 0.78 0.88 1.05 0.12 Inferential Analysis explained variance. The Noteinferential that the finalanalysis a reductionoffers a series in the of magnitude five multiple of all linear• Meanregression Ed Level models. – The average regressionModel model A accountsincludes for only 35% theof APAP-related status of coefficients. student: if they took AP, if educationalthey did not level take of those the in the the overallexam, variance. or exam The scores lack of highif they didModel take E, the deemed exam. the It “full explains model,” only 2%student’s of the variance home ZIP in code, based correlationcollege coefficients grade. This in Table number 2 as isis to particularly be expected interesting, since thegiven AP that status is onlyon the one categories factor usein the for Hipa- well asbackground other tests are of evidence students. of aModel it accounts B includes for severala dummy factors variable not for eachred. professor (each lack ofseparate colinearity college in the models. course) and accountseasily disaggregated for differences from students’ in grading severity between A graphicalcourses. comparison This shows of Modelthe predicted and teachers’ grades experiences. as if all the This students model enrolled inDiscussion the same A anduniversity Model E is course. shown in This Figure model includes shows four a small significant upward demographic change in theThe predicted initial model mean which does 4. (Errorgrade bars earnedrepresent by one each standard AP grouping.variables Model that together C adds may inbe thoughtstudents SATnot control scores, for or student ACT backgrounds error fromscores the mean converted for each usingcategory.) a concordance of as measures table of(Dorans, socio-economic Pommerich and & academic Lyu, 1999) performance for both prior to, The twomathematics models show theand comparison composite scores.status (SES). Model Adjustments D includes for variables such and that independent account for of, general any AP course of meanhigh college school science performance grades by in science,variables mathematics, can never insure and that English, SES experiences,typical of manyshows studieslarge differences simpleaccounting calculation fromfor high raw gradesschool gradeis controlled point averagefor fully, since(HSGPA) it cannot or itsbetween correlate, students high whoschool enroll in AP in Modelrank-in-class. A and by controlling Model E adds for thebe measured kind of withoutcourse error:and 23performanceand in others,any preparatory earning college course the collectioncourse prior of important to taking covari the- AP course• Race in – theAccounting subject. for This the modelself gradesseeks toabout control four points for the higher than ates ininfluence Model E. The of different course mean levels is of preparationidentified racein biology, or ethnicity chemistry of the or course physics. average. Model There E is little dif- representedcan be by interpreted a horizontal as dashed presenting thestudents: difference White: Black, in college Hispanic, performance ference between that is those related reporting to taking line. Thetaking magnitudes an AP ofcourse coefficients or to a certainAsian, performance multi-ethnic, on or the other. AP test thebetween AP exam hypothetically and those who choose for AP experience are clearly muted not to in AP courses. However, when identical students in all other variables,• Hipared i.e. – the between highest students level of who had the same when important background factors demographics and prior academic preparation and background in prioreducation years. of either It generates parent: less an or estimate of the incremental not associated with AP are accounted achievement are accounted for, the effect of taking an AP course or havingequal to aa certainhigh school performance diploma, on the AP test on the for in subsequent models. The average apparent advantage held by students college grade in the same subject.some Included college, for 4-yeareach model degree, are the standardized beta slope of the data relating AP score with with AP experiences in high schools (��coefficients that allow a comparisongraduate or between professional the degree. magnitude of the relative coursecontribution grade is considerably of each reduced variable to the explained variance. Note thatare theroughly final cut regression in half. This reduction • Hstype’ – The kind of high school as shownmodel in the accounts third panel for of 35%Figure of the overall variance. The lack of highis correlationsimilar to findings coefficients by Dougherty et last attended: private, public or 4. Accountingin Table for 2 covariatesas well as leads other to tests are evidence of a lack of colinearityal. (2006a) in the in models. which the introduction other. of controls in their regression analysis A graphical comparison of Model A and Model E is shown in Figure 4. (Error bars 23. Thisrepresent point suggested oneby an anonymousstandard reviewer error of this from paper. the mean for each category.) The two models show

12 Sadler and Tai Draft 8/13/07 — Do Not Duplicate or Quote SciePagence 16 Educator AP Exams and College Performance in Science

the comparison of mean college science grades by simple calculation from raw grades in Model A and by controlling for the collection of important covariates in Model E. The course mean is represented by a horizontal dashed line. The magnitudes of coefficients for AP experience are clearly muted when important background factors not associated with AP are accounted for in subsequent models. The average slope of the data relating AP score with course grade is considerably reduced as shown in the third panel of Figure 4. Accounting for covariates leads to a reduction in the magnitude of all AP- related coefficients. Table 4: 4. Regression Models Models Explaining Explaining Variance Variance in College in College Science Science Grade Grade

DoF Model A SE Model B SE Model C SE Beta Model D SE Beta Model E SE Beta Constant 183.79*** 84.76*** 0.4559.07*** 1.26 37.73*** 1.61 42.84*** 1.68 Prof ID 123 *** *** *** *** Race 5 *** *** *** hipared 3 *** *** *** Hstype' 2 ** *** *** Mean Ed Level' 1 - *** 0.61 *** 0.19 0.03 SATM_CAL' 4 0.01*** 0.00 0.09 0.01*** 0.00 0.09 0.02*** 0.00 0.13 SATV_CAL' 1 0.03*** 0.00 0.27 0.01*** 0.00 0.06 0.01*** 0.00 0.06 himath 4 *** 0.13 *** 0.11 Algebra II -2.69 0.28 -2.31 0.28 precalculus -1.33 0.19 -1.16 0.19 Calculus 0.36 0.25 0.27 0.25 Calculus AB 1.38 0.23 1.17 0.23 Calculus BC 2.28 0.37 2.03 0.37 Math Grade 1 2.51 *** 0.15 0.17 2.27 *** 0.15 0.16 English Grade 4 0.93 *** 0.19 0.05 0.75 *** 0.19 0.04 Science Grades 1 2.99 *** 0.23 0.14 2.13 *** 0.24 0.10 Prep Course 4 *** 0.08 not taken 0.19 0.29 not reported -0.41 0.39 C or lower -1.63 0.33 B -0.28 0.22 A 2.12 0.21 Honors 1 0.97 *** 0.26 0.04 AP Status 6 *** *** *** 0.19 *** 0.14 *** 0.15 no AP -3.91 0.45-4.66 0.44-3.57 0.41 -2.69 0.39 -2.26 0.40 No AP Exam 1.42 0.61 0.14 0.60 0.07 0.56 0.11 0.53 0.16 0.53 AP 1 -5.68 1.49-5.22 1.41-3.32 1.33 -2.19 1.26 -2.47 1.25 AP 2 -1.68 1.02-1.34 0.97-0.80 0.91 -0.94 0.86 -1.02 0.86 AP 3 0.51 0.90 0.78 0.86 0.36 0.81 0.18 0.76 0.16 0.76 AP 4 3.36 1.13 4.22 1.07 2.93 1.01 2.22 0.95 2.12 0.94 AP 5 5.98 1.38 6.08 1.32 4.32 1.25 3.31 1.18 3.30 1.18 Level of AS Raw Data Full Model no AP 79.88 765780.11 7657 79.05 7657 80.18 7575 79.46 7575 No AP Exam 85.21 50484.90 50482.69 504 82.98 502 81.88 502 AP 1 78.11 45 79.55 45 79.30 45 80.68 45 79.25 45 AP 2 82.11 10683.42 10681.82 106 81.93 105 80.70 105 AP 3 84.30 14585.54 14582.98 145 83.05 145 81.88 145 AP 4 87.15 84 88.98 84 85.55 84 85.09 84 83.84 84 AP 5 89.77 53 90.84 53 86.94 53 86.19 52 85.02 52

Total Cases 8594 8594 8594 8594 8594 Cases Missin 0 0 0 86 86 Cases Used 8594 8594 8594 8508 8508

r^2= 0.022 0.153 0.252 0.336 0.347 Model E, deemed the “full model,” is particularly interesting, given that it accounts for resultedseveral in reductions factors in thenot modeled easily disailluminatingggregated (Figurefrom students’ 4). We found and teachers’by many experiences. that it is always This better to take outcomemodel variable, includes college fourgraduation significant that demographicstudents who are variables awarded an that AP togethera more rigorousmay be course thought and earnof a low rate, fromas measures 39% to 26% of forsocio-economic students exam status score (SESof 1 earned). Adjustments college science for suchgrade variables than an easier can course never and earn a passinginsure an AP thatexam. SES Hence, is controlled in that gradesfor fully, no better since than it did cannot those studentsbe measured high withoutgrade (Adelman, error:23 1999). study, a third of the apparent effect of who entered the college course after Based on the results from the Full passing an AP exam was explained by having taken a regular course in the Model, those students who enroll in the two covariates used: eighth-grade subject. Similarly, students who have AP and have not taken the AP exam mathematics achievement and high an AP exam score of 2 do no better than perform in college science about as school 23 characteristics. This point suggested Our findings by an anonymous those who reviewer have completed of this paper. an honors well as AP 3 students: about 1.5 points are similar to those of Willingham & course. Since students usually com- higher than the student average. AP 4 MorrisSadler (1986) and who Tai also found that Draftplete 8/13/07 either —a regular Do Not or Duplicate honors course or Quote students are predicted toPage earn 17 college half of the “AP advantage” was ac- prior to enrolling in an AP course, it science grades 3.4 points above aver- counted for by matching students by appears as if students reporting low age. AP 5 students are predicted to earn background. AP exam scores may have gained no grades 4.6 points higher than average. Analyzing college grade by differ- benefit from their AP course. This find- Though these predicted scores appear ent levels of AP exam score is also ing contrasts with the belief espoused to be fairly impressive, we should con-

Fall 2007 Vol. 16, No. 2 13 AP Exams and College Performance in Science

• Race – Accounting for the self identified race or ethnicity of students: White: Black, Hispanic, Asian, multi-ethnic, or other. • Hipared – the highest level of education of either parent: less or equal to a high school diploma, some college, 4-year degree, graduate or professional degree. • Hstype’ – The kind of high school last attended: private, public or other. • Mean Ed Level – The average educational level of those in the student’s home ZIP code, based on the categories use for Hipared. FigureFigure 4.4: Comparison Comparison of of Model Model A (“RawA (“Raw Data”) Data”) and Modeland Model E (“Full E Model”)(“Full Model”)

95 Preparatory Advanced Placement Coursework Coursework

90 Raw Data

Accounting for 85 Background Course Mean

80 College Science Grade

AP1 AP2 AP3 AP4 AP5 regular honors not taken AP, exam AP, no exam

Discussion sider that the student average sits just and average science grade (in other tend to invalidate this claim. More above 80The points, initial amounting model whichto about does subjects not control (β = 0. 1for0). Takenstudent as abackgrounds group, important and toacademic success in college science a gradeperformance of B-. An additional prior 4.6 to, points and independentthese are arguably of, any stronger AP predictorscourse experiences, appear to beshows general large academic skills, would onlydifferences elevate the between course grade students to of who college enroll grade in than AP performance and others, in earningparticularly college in mathematics. course AP exam roughlygrades 85 points, about suggesting four points that the higherhigh than school the courses course preparatory average. There to score is little had adifference correlation of 0.404 with averagebetween AP 5 student those in introductoryreporting taking an AP the course AP exam (β = and 0.08). those This maywho chooseSAT verbal not toscores in AP (Table 2). This can collegecourses. science would However, be predicted when to demographicsreflect the importance and prior of academicmathemati- achievementbe interpreted are as accountedevidence that high earn a letterfor, thegrade apparent of B if one advantage accounts heldcal knowledge by students and reasoning with AP ability experiences verbal in functions high schools (both in are reading and for thoseroughly background cut variables in half. that This are reduction for college is similar students to in findings the sciences by Doughertywriting) are et necessaryal. (2006a) to getin the most differentwhich from themost introduction students in their of controls(Burton in & theirRamist, regression 2001). analysis resultedfrom an AP in course reductions (particularly in read- sciencethe course. modeled outcome variable,Poor college performance graduation in AP rate, courses from 39%ing theto textbook)26% for studentsand to perform well Nextpassing we considered an AP the exam. relative Hence, im- isin often that attributedstudy, a tothird a concern of the over apparent on the effect AP exam. of passing an portanceAP of exam variables was in theexplained explained by thethe two“lack covariates of academic used: readiness” eighth-grade for There mathematics are many possible explana- varianceachievement of the full model and based high on theschool advanced characteristics. placement Our courses, findings particu -are similartions for towhy those AP exam of high-scorers standardized β (in units of the fraction larly a deficit in the “possession of do not consistently attain levels of of a standard deviation of college grade prior relevant knowledge (Dougherty performance commensurate with earnedSadler for one and standardTai deviation Draftet 8/13/07al., 2006b, — p.Do 6).” Not While Duplicate one mayor Quote stated College Board expectations.Page 18 We change in the independent variable). assume that such knowledge could be discuss some of these below: As one might expect, AP exam score acquired from a preparatory course • Non-equivalency of the AP has a substantial role (β = 0.15). Yet, (e.g. regular or honors chemistry for exams and college science at- four other variables have similar rat- an AP chemistry course), the relatively tainment measures. AP exams ings: high school math grade (β = low standardized coefficients related to may not fully reflect content of 0.16), SAT Math (β = 0.13), highest theses courses (0.04 for honors level the college courses in this study, math course taken in HS (β = 0.11), and 0.08 for the grade in this course) even though the College Board’s

14 Science Educator efforts are designed to accurately website.24 An example is given student’s knowledge and ability. reflect the college curriculum. for the Physics C (Electricity and The accuracy of these claims must Certainly there is content at the Magnetism Exam). The cut-off be clear and unequivocal, since college level that is uniquely pre- score over which students earn decisions for an individual are sented by college professors that a “5” is 19.7 out of 35 problems made on the weight of a single cannot show up in a standardized correct or 56%. The cut-off score AP exam score. A decision to AP test. Also, more goes into a over which students earn a “4” is bypass an introductory course college course grade than a single 40% and over which they earn does have potential repercussions final exam. Students often take a “3” is 31%.25 While assigning for individual students. additional tests (e.g. mid-terms), final “grades” is a function of item • Weak Methodology in AP Score are graded on their laboratory difficulty, such a low percentage Validation. For the process of work and problem sets, and may of items correct translating into validation carried out by the be required to pursue a project or a top score is unusual for a test. College Board, items from the rel- paper. No matter how close the As a criterion referenced exam, evant AP exam are administered AP exam is to the college-level the criteria may be set too low to college science students at the exam, there may be missing ele- (Lichten, 2000). end of their course. Any laxity of ments that do not make the AP • Lack of Sufficient Content effort on the part of these students test a valid substitute for a college Coverage in AP Exams. A to do well would invalidate the grade in science, however similar fundamental limitation of all equating of AP scores to college it may be to the final exam. standardized exams is the amount grades (Lutkus, Weiss, Campbell, • Over-generosity in AP Exam of content each exam may in- Mazzeo & Lazer 1999). The Col- Scoring. “In general, an AP clude. Although exams may be lege Board is silent in its technical grade of 3 or higher indicates “comprehensive” in some ways, manual as to what motivation sufficient mastery of course a single Advanced Placement these students have to do their best content to grant a student ex- exam has no hope of including the on these exams. Also, extensive emption from a college course, degree of comprehensive content preparation of AP students with credit, or both (College Board, coverage typically included in practice exams familiarizes them 2004, p. 6.)” The College Board college coursework. As a result, with the format and probable is clear in its guidelines, adjust- AP exams are at a fundamental content. Participating college ing the cut-points (how exam disadvantage, depending largely students enjoy no such “prep- scores are assigned) of each of on an assessment of student ping” for the questions that they its AP examinations so that ”… content knowledge based on a answer drawn from AP exams the lowest AP 5 is equivalent to sampling of student performance. and may be unaccustomed to their the average “A” in college, the This approach is based on the formulation, earning them lower lowest AP 4 is equivalent to the same premise as this large-scale scores. There is a lack of parity average “B,” and the lowest AP 3 study; however, the difference since the College Board does not is equivalent to the average “C” lies in its application. While we administer actual college science (College Board, 2004, p. 11).” make no claims as to the perfor- final exams to AP students while The so-called “composite score mance of individuals based on in high school to see if they score boundaries” set the cut-offs for our analysis, but rather limit our at levels reflective of high college AP exam “grades” of 1, 2, 3, 4, conclusions to generalizations for grades. or 5 from the raw scores or “for- large groups, AP exams do make • Sample Size of the Current mula scores.” The setting of these the claim of providing a highly ac- Study. While 8594 students were boundaries is explained on the AP curate appraisal of an individual included in this study, only 937

24. “An AP grade may also indicate that the student should receive credit for a college course that he or she has not taken. To check the validity of such decisions, studies are conducted in which the AP Exam is administered to students taking the college courses in question. The college students’ grades on the AP Exam are then compared with their grades in the college course.” (http://apcentral.collegeboard.com/article/0,3045,152-167-0-2052,00.html#key) 25. http://apcentral.collegeboard.com/members/article/1,3046,154-181-0-11583,00.html

Fall 2007 Vol. 16, No. 2 15 reported taking the relevant AP and, hence, do not take it. Many, Placement exams as predictors of course in high school. Of these perhaps most, of the students college science performance and the students 433 reported taking in colleges that we study who “value-added” of taking AP science the exam with 282 reporting AP earn a 3, 4, or 5 are simply not courses in high school. AP courses act exam scores of 3 or higher. While in our sample. There is no direct as both a filter for the most motivated this study is smaller than several way to know how these students and brightest high school students, concerning Advanced Placement, would fare in an introductory while offering an opportunity to learn it is the largest investigation of course, one can only analyze the science in a rigorous, fast-paced en- its kind that controls for relevant grades of those who are actu- vironment modeling an introductory background factors such as prior ally enrolled in an introductory college course. We have chosen to mathematics and science course- course, as we have. However, we examine the performance of college taking and performance. This have established that these high students taking introductory college study offers a snapshot of the scoring AP students have no ap- biology, chemistry, and physics. actual performance of students parent deficiencies in other areas. These students have varied academic in introductory college science Through our follow-up survey we backgrounds that allow for the use of in U.S. colleges and universi- have also found that most thought multiple regression to model the de- ties. While studies with larger retaking the introductory course gree to which specific variables explain samples of AP students abound, was beneficial. These students differences in college performance. most control for few or no back- strive for high grades and work We find that the students in our ground variables. These results as hard or harder than their class- sample who reported AP exam scores offer little to no information about mates. of 3 or higher earned college grades the disaggregated relationship of • AP students were bored retak- that were higher than the student av- Advanced Placement experiences ing introductory courses. One erage. However, many AP 5 students and student performance in col- explanation may be boredom performed at levels below the College lege. After controlling for several among AP exam takers since Board’s claims of excellence even covariates, our study found that the college course may merely after taking a semester of the college AP course participation and be a review of concepts covered science. While 73% of AP 4 students performance on AP tests were in high school. With this issue earned college science grades above indeed related to performance in mind, we queried students in the College Board’s stated “mid-level in college science courses, but our survey who offered “further B performance in college,” about that the magnitude of the effect contact” information and who half of the AP 5 students missed the was about half that found with- reported high AP exam scores. A level performance cited by the out controls. Moreover, we do Our follow-up inquiries found College Board. We should be clear not know if AP experiences help that a majority of the students on one very important point, all the students or if better students help contacted reported the “re-tak- students in our survey experienced a AP outcomes. Given the clear dis- ing” of the college science course semester of actual introductory college parities of educational resources to be beneficial. In addition, we science course work. Thus, given that in American education, the latter did not find significant differences high school AP courses are intended may not be further from the truth (at p≤0.05 for χ2 test) in level of to stand in place of college courses, it than the former. effort reported in college science can be argued that AP students have • Missing students who “place between AP students who passed taken the introductory college course out” in the current study. This or failed the AP exam in high twice, and despite this clear advantage, study, by its very design, does not school. their performance is by no means include any students who have indicative of the benefit many would earned high enough AP exam Conclusions have us believe Advanced Placement scores to receive credit for an in- This study seeks to disentangle courses would impart to high school troductory college science course two issues: the validity of Advanced students who take them. As a measure

16 Science Educator of introductory college science content course in their school offered little or knowledge mastery, the performance … our study has found no benefit to students beyond regular or of a considerable fraction of AP takers honors level science courses. Schools does not appear to adequately reflect indications that the AP in which the majority of students score the professed notions of the College program, while certainly in this range should examine whether Board. In fact, in our qualitative of value to many students, exposure to AP-level rigor holds any probe, many students feel that while may lack some of the benefits apart from the “academic pad- AP is a good preparation for college evidence necessary ding” coined by Lichten that aids them science, many also feel they benefited to support its claim of in the college admission process. from taking the actual college science academic rigor equal to Advanced Placement has become course. one of the most highly respected Based on our analysis, it appears that of introductory college “brand-names” in secondary/post- that about half of the advantage at- and university courses in secondary education. However, our tributed to AP experience can be ac- science. study has found indications that the counted for by variables representing AP program, while certainly of value the academic abilities and experiences Our findings suggest that the edu- to many students, may lack some of possessed by AP students prior to, or cators, teachers, parents, and students the evidence necessary to support its independent of, their AP course expe- would be better served by: claim of academic rigor equal to that riences. Students’ backgrounds, par- −An acknowledgment from the of introductory college and university ticularly their mathematical and verbal College Board that evidence exists courses in science. While this current skills, appear to contribute mightily to indicating the non-equivalence of study certainly suffers from limita- their performance in college science AP courses and college courses in tions, as do all research studies, what courses. While the AP examination biology, chemistry, and physics. is clear from our findings is that the program is an elaborate system, with −A more balanced mix of both sup- claims of the College Board regard- its professional development program portive and critical research at the ing the interpretation and validity of rivaling its assessment program in Advanced Placement website.26 AP exam scores are problematic. Our size and productivity, the Advanced findings indicate the need for further Placement exams themselves appear −The College Board clearly stating investigations of greater size, scope, to fall short of the predictive validity their role in funding research stud- and intensity, especially given the po- claimed by the College Board. Based ies cited as evidence for AP course tential being ascribed to the Advanced on the findings from our study, AP effectiveness. Placement program as part of the exams scores of 3 do not appear to −AP exam scoring made more strin- cure for what ails American science warrant the granting of college credit gent so that the highest exam score education. over those students who take an AP truly represents achievement at the course in high school, but do not take highest level of college work. References the exam. In addition, we found that an −The College Board acknowledg- Adelman, C. (1999) Answers in the Tool- AP exam score of 1 or 2 offered little ing that the later performance of box: Academic Intensity, Attendance evidence of any benefit derived from students who pass the AP exams Patterns, and Bachelor’s Degree At- the AP course work experienced by the appears to be explained as much tainment. Jessup, MD: Educational students in high school. In addition, by their prior background as their Publications Center. students passing an AP exam might AP course enrollment. Anaya, G (1999) Accuracy of Self-Re- ported Test Scores, College and Uni- well consider retaking introductory For high schools seeking to evaluate versity, 75(2), 13-19. courses in college to more completely their AP science courses, a preponder- Baird, L. (1976) Using Self –Reports to master the content, as many colleges ance of AP exam scores of 1 or 2 may Predict Student Performance. Research and universities already require. be interpreted to mean that the AP Monograph No. 7., New York, NY: Col- lege Entrance Examination Board.

26. http://apcentral.collegeboard.com/apc/public/colleges/research/index.html

Fall 2007 Vol. 16, No. 2 17 Bradburn, N., Rips, L. and S. Shevell Dougherty, C., Mellor, L. and Jian, S. MacVicar, R. (1988) Advanced Place- (1987) Answering autobiographical (2006b) Orange Juice or Orange ment: Increasing Efficiency in High questions: The impact of memory Drink? Ensuring that “Advanced School--University Articulation. Phoe- and inference on surveys. Science. Courses” Live Up to Their Labels. nix, AZ: Arizona Board of Regents. 236(4798): 157–61. National Center for Educational Ac- (ED306835) Bradburn, N. (2000). Temporal represen- countability. National Research Council (2002) Learn- tation and event dating. A. A. Stone, Geiser, S. and Santelices, V. ( 2004) The ing and Understanding: Improving J.S. Turkan, C.A. Bachrach, J.B. Jobe, Role of Advanced Placement and Hon- Advanced Study of Mathematics and H.S. Kurtzman, V.S. Cain (Eds.), . ors Courses in College Admissions. Science in US High Schools, Commit- The Science of Self-Report. Mahwah, Center for Studies in Higher Education. tee on Programs for Advanced Study of NJ: Lawrence Erlbaum Associates, Paper CSHE-4-04. http://repositories. Mathematics and Science in American pp. 49-61. cdlib.org/cshe/CSHE-4-04/ High Schools. J. P. Gollub, M. W. Ber- Burdman, P. (2000). Extra credit, extra Hershey, S. (1990) “College Admission tenthal, J. B. Labov, and P. C. Curtis, Jr., criticism. Black Issues in Higher Edu- Practices and the Advanced Placement eds. Center for Education, Washington, cation 17(18), 28-33. Program.” Journal of College Admis- DC: National Academy Press. Burton, N. W. and Ramist, L. (2001) Pre- sion, 128, 8-11. Niemi, R. and Smith, J. (2003) The ac- dicting Success in College: SAT Studies Juillerat, F., Dubowsky, N., Ridenour, curacy of students’ reports of course of Classes Graduating Since 1980. New N., McIntosh, W., and Caprio, M. taking in the 1994 National Assessment York, NY: College Board. (1997) Advanced placement science of Educational Progress. Educational Cahow , C. R., Christensen, N. L., Gregg, courses: high school--college articula- Measurement: Issues and Practice, J. R., Nathans, E. B., Strobel, H. A. & tion issues, Journal of College Science 22(1), 15-21. Williams, G. W. (1979) Undergraduate Teaching, 27(1), 48-52. Pace, C., Barahona, D., and Kaplan, D. Faculty Council of Arts and Sciences, Klopfenstein, K. and Thomas, K. (2005) (1985) The credibility of student self-re- committee on Curriculum, Subcommit- The Advanced Placement Performance ports. Los Angeles, CA: UCLA Center tee on Advanced Placement Report. Advantage: Fact or Fiction? Accessed for the Study of Evaluation Durham, NC: Duke University. from http://www.aeaweb.org/annual_ Phillips Academy (1952) General educa- Chamberlain, P., Pugh, R. and Shellham- mtg_papers/2005/0108_1015_0302. tion in school and college; a committee mer, J. (1978) Does Advanced Place- pdf report by members of the faculties of ment Continue Through the Under- Klopfenstein, K. (2004) The advanced Andover, Exeter, Lawrenceville, Har- graduate Years? College & University, placement expansion of the 1990s: vard, Princeton, and Yale. Cambridge, 53, 195-200. How did traditionally underserved stu- MA: Harvard University Press. College Board. (2004) Interpreting and dents fare? Education Policy Analysis Pushkin, D. (1995) The AP Exam and Using AP Grades. New York, NY. Archives, 12(68), 1- 14. the Introductory College Course. The College Board (2006) Advanced Place- Kuncel, N., Credé, M., & Thomas, L. Physics Teacher, 33(8), 532-535. ment Report to the Nation. New York, (2005) The validity of self-reported Ruch, C. (1968) A Study of the Collegiate NY. grade point averages, RICs, and test Records of Advanced Placement and Dodd, B., Fitzpatrick, R. and Jennings, J. scores: A meta-analysis and review of Non-Advanced Placement Students. (2002) An Investigation of the Validity the literature. Review of Educational College & University, 44(2), 207- of AP Grades of 3 and a Comparison of Research, 75(1), 63 – 82. 210. AP and Non-AP Student Groups. New Lichten, W. (2000) Whither Advanced Sadler, P.M. & Tai, R.H.(2001) Success York, NY: College Board. Placement? Education Policy Analy- in College Physics: The Role of High Dorans, N. (1999) Correspondence Be- sis Archives, 8(29). http://epaa.asu. School Preparation, Science Education. tween Act and SAT I Scores (College edu/epaa/v8n29.html 85(2), 111-136. Board Research Report 99-1), New Lichten, W. (2007) Equity and Excel- Sawyer, R., Laing, J. and Houston, W. York: College Board. lence in the College Board Advanced (1989) Accuracy of Self-Reported Dougherty, C., Mellor, L. and Jian, S. Placement Program, Teachers College High School Courses and Grades of (2006a) The Relationship Between Record, 1/16/07. College-Bound Students. College and Advanced Placement and College Lutkus, A., Weiss, A., Campbell, J., University, 64(3), 288-299. Graduation. National Center for Edu- Mazzeo, J., and Lazer, S. (1999) NAEP Willingham, W. and Morris, M. (1986). cational Accountability. 1998 Civics Report Card for the Nation, Four Years Later: A Longitudinal Washington, DC: National Center for Study of Advanced Placement Students Educational Statistics, 132. in College. New York: College Board Publications.

18 Science Educator Wood, W.B. (2000) Advanced High School The authors would like to acknowledge the the field: Michael Neuschatz of the American Biology in an Era of Rapid Change: A people who helped make this large research Institute of Physics, William Lichten of Yale Summary of the Biology Panel Report project possible: Janice M. Earle, Finbarr C. University, Trevor Packer of the College from the NRC Committee on Programs Sloane, and Larry E. Suter of the National Board, Saul Geiser of University of California, for Advanced Study of Mathematics Science Foundation for their insight and Paul Hickman of Northeastern University, and Science in American High Schools. support; James H. Wandersee, Joel J. Mintzes, William Fitzsimmons, Marlyn McGrath Lewis, Cell Biology Education, 123-7. Lillian C, McDermott, Eric Mazur, Dudley R. Georgene Herschbach, and Rory Browne of Herschbach, Brian Alters, and Jason Wiles of Harvard University, and Kristen Klopfenstein Philip M. Sadler is director of the science the FICCS Advisory Board for their guidance; of Texas Christian University. We are indebted education department at the Harvard- and Nancy Cianchetta, Susan Matthews, Dan to the professors at universities and colleges Smithsonian Center for Astrophysics and Record, and Tim Reed of our High School nationwide who felt that this project was worth F.W. Wright Senior Lecturer in Harvard’s Advisory Board for their time and wisdom. contributing a piece of their valuable class Department of Astronomy, Cambridge, MA This research has resulted from the tireless to administer our surveys and their students’ 02138. Correspondence concerning this article efforts of many on our research team: Michael willingness to answer our questions. This work may be sent to [email protected]. Filisky, Hal Coyle, Cynthia Crockett, Bruce has been carried out under a grant form the Ward, Judith Peritz, Annettte Trenga, Freeman Interagency Educational Research Initiative Robert H. Tai is assistant professor of Deutsch, Nancy Cook, Zahra Hazari, Jamie (NSF-REC 0115649). Any opinions, findings science education, Curry School of Education, Miller, Marc Schwartz, and Gerhard Sonnert. and conclusions or recommendations expressed University of Virginia, Charlottesville, VA Matthew H. Schneps, Nancy Finkelstein, Alex in this material are those of the authors and do 22904Acknowledgements Griswold, Tobias McElheny, Yael Bowman, not necessarily reflect the views of the National and Alexia Prichard of our Science Media Science Foundation, The U.S. Department Group constructed of our dissemination of Education, or the National Institutes of website (www.ficss.org). We also appreciate Health. advice and interest from several colleagues in

Fall 2007 Vol. 16, No. 2 19