Validating the Use of AP® Exam Scores for College Course Placement by Brian F
Total Page:16
File Type:pdf, Size:1020Kb
RESEARCH REPORT 2013-2 Validating the Use of AP® Exam Scores for College Course Placement By Brian F. Patterson and Maureen Ewing VALIDITY Brian F. Patterson is an assistant research scientist in the College Board’s Research department. Maureen Ewing is a senior director in the College Board’s Research department. Acknowledgments We would like to acknowledge Priyank Patel’s assistance in the review of introductory credit and placement–granting policy data and Mylene Remigio’s thoughtful review of the data and analyses. We also appreciate Michael Chajewski’s perspective on the validation of propensity score models. This manuscript also benefited from the helpful feedback of Ben Kelcey, and we are grateful for his contributions. About the College Board The College Board is a mission-driven not-for-profit organization that connects students to college success and opportunity. Founded in 1900, the College Board was created to expand access to higher education. Today, the membership association is made up of over 6,000 of the world’s leading educational institutions and is dedicated to promoting excellence and equity in education. Each year, the College Board helps more than seven million students prepare for a successful transition to college through programs and services in college readiness and college success — including the SAT® and the Advanced Placement Program®. The organization also serves the education community through research and advocacy on behalf of students, educators and schools. For further information, visit www.collegeboard.org. © 2013 The College Board. College Board, Advanced Placement Program, AP, SAT and the acorn logo are registered trademarks of the College Board. ACES, Admitted Class Evaluation Service, AP Potential, AP Potential and SAT Subject Tests are trademarks owned by the College Board. PSAT/NMSQT is a registered trademark of the College Board and National Merit Scholarship Corporation. All other products and services may be trademarks of their respective owners. Visit the College Board on the Web: www.collegeboard.org. For more information on College Board research and data, visit VALIDITYresearch.collegeboard.org. Contents Executive Summary ................................................................................................ 5 Introduction ............................................................................................................. 6 Method ..................................................................................................................... 9 Sample .............................................................................................................. 9 Measures .........................................................................................................10 Analyses ..........................................................................................................11 Results .....................................................................................................................16 Selection of AP® Exams for Presentation .....................................................16 Subsequent Mathematics Course Performance ..........................................18 Subsequent Natural Science Course Performance .....................................22 Subsequent Social Science and History Course Performance ...................28 Summary of Subsequent Course Performance Analyses ...........................38 Discussion ...............................................................................................................39 Limitations and Directions for Future Research .........................................41 Conclusions .............................................................................................................41 References ...............................................................................................................42 Appendix .................................................................................................................45 Tables Table 1. Summary of Colleges’ AP Introductory Credit- and Placement-Granting Policies ....12 Table 2. Summary of Institutional Characteristics ......................................................................13 Table 3. Summary of Sample Sizes Before Matching..................................................................17 Table 4. Summary Statistics for AP Calculus AB and Non-AP Students, Before and After Matching ............................................................................................................19 Table 5. Summary Statistics for AP Calculus BC and Non-AP Students, Before and After Matching ............................................................................................................21 Table 6. Summary Statistics for AP Biology and Non-AP Students, Before and After Matching ............................................................................................................23 Table 7. Summary Statistics for AP Chemistry and Non-AP Students, Before and After Matching ............................................................................................................25 Table 8. Summary Statistics for AP Physics C: Mechanics and Non-AP Students, Before and After Matching ............................................................................................................27 Table 9. Summary Statistics for AP Microeconomics and Non-AP Students, Before and After Matching ............................................................................................................29 Table 10. Summary Statistics for AP Macroeconomics and Non-AP Students, Before and After Matching ............................................................................................................31 Table 11. Summary Statistics for AP Psychology and Non-AP Students, Before and After Matching ............................................................................................................33 Table 12. Summary Statistics for AP U.S. Government and Politics and Non-AP Students, Before and After Matching ....................................................................35 Table 13. Summary Statistics for AP U.S. History and Non-AP Students, Before and After Matching ............................................................................................................37 Table 14. Summary Statistics for AP and Non-AP Students on Subsequent Course Grades, After Matching ....................................................................................................39 Table A1. Student Characteristics of Propensity Score Model Sample.....................................45 Table A2. Academic Characteristics of Propensity Score Model Sample.................................46 Table A3. AP Exam Participation Rates : Overall and Extreme High School Proportions and Test of Heterogeneity .............................................................................................................47 Table A4. Propensity Score Model for AP Calculus AB Exam Participation.............................48 Table A5. Propensity Score Model for AP Calculus BC Exam Participation .............................50 Table A6. Propensity Score Model for AP Computer Science A Exam Participation ..............52 Table A7. Propensity Score Model for AP Biology Exam Participation .....................................54 Table A8. Propensity Score Model for AP Chemistry Exam Participation ................................56 Table A9. Propensity Score Model for AP Physics B Exam Participation .................................58 Table A10. Propensity Score Model for AP Physics C: Mechanics Exam Participation ..........60 Table A11. Propensity Score Model for AP Microeconomics Exam Participation ...................62 Table A12. Propensity Score Model for AP Macroeconomics Exam Participation ..................64 Table A13. Propensity Score Model for AP Psychology Exam Participation ............................66 Table A14. Propensity Score Model for AP U.S. Government and Politics Exam Participation .........................................................................................................................68 Table A15. Propensity Score Model for AP U.S. History Exam Participation ...........................70 Table A16. Propensity Score Model for AP English Language and Comp. Exam Participation .........................................................................................................................72 Table A17. Propensity Score Model for AP English Literature and Comp. Exam Participation .........................................................................................................................74 Table A18. Propensity Score Model for AP Spanish Language Exam Participation ................76 Figures Figure 1. Standardized differences for AP Calculus AB and non-AP students ........................18 Figure 2. Standardized differences for AP Calculus BC and non-AP students ........................20 Figure 3. Standardized differences for AP Biology and non-AP students ................................22 Figure 4. Standardized differences for AP Chemistry and non-AP students ...........................24 Figure 5. Standardized differences for AP Physics C: Mechanics and non-AP students .......26 Figure 6. Standardized differences for AP Microeconomics and non-AP students.................28 Figure 7. Standardized differences for AP Macroeconomics and non-AP students