Is the United States Really Losing the International Horse Race in Academic Achievement?

University of Pennsylvania ScholarlyCommons GSE Faculty Research Graduate School of Education 5-1-2005 Is the United States Really Losing the International Horse Race in Academic Achievement? Erling E. Boe University of Pennsylvania, [email protected] Sujie Shin University of Pennsylvania Follow this and additional works at: https://repository.upenn.edu/gse_pubs Part of the Education Commons Recommended Citation Boe, E. E., & Shin, S. (2005). Is the United States Really Losing the International Horse Race in Academic Achievement?. Retrieved from https://repository.upenn.edu/gse_pubs/5 Copyright Phi Delta Kappa International. Reprinted from Phi Delta Kappan, Volume 86, Issue 9, May 2005, pages 688-695. This material is posted here with permission of Phi Delta Kappa. Content may not be copied or emailed to multiple sites or posted to a listserv or website without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This paper is posted at ScholarlyCommons. https://repository.upenn.edu/gse_pubs/5 For more information, please contact [email protected]. Is the United States Really Losing the International Horse Race in Academic Achievement? Abstract The results of international assessments of student achievement are far more nuanced than the headlines lead us to believe. Having examined six comparisons of performance — in various subjects and at various levels — by students in the U.S. and other industrialized nations, Mr. Boe and Ms. Shin conclude that the dire pronouncements about America’s standing are greatly exaggerated. Disciplines Education Comments Copyright Phi Delta Kappa International. Reprinted from Phi Delta Kappan, Volume 86, Issue 9, May 2005, pages 688-695. This material is posted here with permission of Phi Delta Kappa. Content may not be copied or emailed to multiple sites or posted to a listserv or website without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This journal article is available at ScholarlyCommons: https://repository.upenn.edu/gse_pubs/5 EXAMINING TEST SCORES Is the United States Really Losing the International Horse Race in Academic Achievement? The results of international assessments of student achievement are far more nuanced than the headlines lead us to believe. Having examined six comparisons of performance — in various subjects and at various levels — by students in the U.S. and other industrialized nations, Mr. Boe and Ms. Shin conclude that the dire pronouncements about America’s standing are greatly exaggerated. BY ERLING E. BOE AND SUJIE SHIN T IS widely believed and lamented that U.S. students perform poorly on international comparisons of academic achievement. For example, Edward Silver reports that U.S. seventh- and eighth-grade students performed poorly on the mathematics section of the Third International Mathematics and Science Study (TIMSS 1995) and that this indicates “a pervasive and intol- Ierable mediocrity in mathematics teaching.”1 Like- wise, the Institute of Education Sciences (IES) in the U.S. Department of Education attributed the report- edly poor performance of U.S. middle-grade students on the TIMSS 1995 mathematics assessment to the ineffectiveness of mathematics education.2 Such per- ceptions have led to grave concerns about the future economic competitiveness of the U.S. For example, Rita Colwell, the former director of the National Sci- ence Foundation, has stated that if the U.S. is to main- tain its position in the world economy, it is critical for the nation’s students to achieve at high levels in mathematics and science.3 ERLING E. BOE is a professor in the Graduate School of Education and a co-director of the Center for Research and Evaluation in Social Policy, University of Pennsylvania, Philadelphia, where SUJIE SHIN is a research assistant. 688 PHI DELTA KAPPAN Given the concern about the strength of the U.S. econ- of achievement. Therefore, we have assembled and organ- omy, it is often said that the nation’s students perform poor- ized the results of all major international achievement sur- ly compared with students in “many” other industrialized veys conducted from 1991 through 2001 (2003 results were nations. This indeed is true. One can pick a particular sur- published subsequently) in order to determine how well U.S. vey (e.g., TIMSS 1995), subject matter (e.g., mathematics), students have performed compared with their peers from and grade level (e.g., grade 8) and find “many” industri- other industrialized nations. More particularly, we have sought alized nations that scored significantly higher than the U.S. to address the following questions: (e.g., France, Japan, and Switzerland).4 Yet it is also true • How do U.S. achievement scores overall compare with that U.S. students perform better than students in many in- scores from other industrialized nations? dustrialized nations. For example, the U.S. scored signifi- • How does U.S. achievement by subject matter com- cantly higher than many industrialized nations (e.g., France, pare with that of other industrialized nations? Germany, and Switzerland) in the 1991 Reading Literacy • How does U.S. achievement by grade level compare Study at grade 4.5 (Throughout this article, “significantly” with that of other industrialized nations? refers to statistical significance.) • How does U.S. achievement compare with that of its Thus, depending on one’s interest or agenda, a partic- major economic competitors — the other G7 nations (Can- ular survey result can be selected to support almost any ada, France, Germany, Italy, Japan, and the United Kingdom)? conclusion about how the U.S. stands in the internation- • How does the achievement of U.S. students from vari- al achievement horse race. For instance, in order to sup- ous racial and ethnic groups compare with that of other port the conclusion that the U.S. was at risk of falling well G7 nations? behind other nations in economic competitiveness, the first item of evidence cited in A Nation at Risk was “that BRINGING IT ALL TOGETHER on 19 academic tests American students were never first or second and, in comparison with other industrialized The first step in our analysis was to assemble the achieve- nations, were last seven times.”6 No mention was made of ment results for reading, mathematics, science, and civics which international tests, grade levels, or subjects were of all international surveys conducted from 1991 through selected for these comparisons, and nothing was said 2001 under the auspices of either the International Associ- about how U.S. achievement on average compared with ation for the Evaluation of Educational Achievement (IEA) that of other industrialized nations.7 or the Organisation for Economic Co-operation and Devel- A much more objective and accurate assessment of the opment (OECD).8 National mean achievement scores for standing of the U.S. in international comparisons would 22 industrialized nations that participated in two or more be based on the results of multiple surveys, multiple sub- of the six surveys analyzed were obtained from primary ject matters, and multiple grade levels. Fortunately, another sources for each of the surveys.9 These sources also pro- generation of international achievement surveys has been vided information on the statistical significance of the dif- completed since 1990, and it is worth looking at the U.S. ference between the mean achievement scores of the U.S. performance on these more recent assessments. In addition and the mean achievement scores for each industrialized to being of high technical quality, these international as- nation participating in a given survey. sessments are not limited to mathematics and science, as Achievement results from the six surveys were aggre- were surveys from previous years, but also include subjects gated as appropriate to address the research questions.10 such as reading and civics. Cross-national comparisons were based on national com- The perception of poor performance by U.S. students on prehensive scores (e.g., full scale scores in mathematics) international comparisons is typically attributed to the in- rather than on subscale scores (e.g., score in algebra). effectiveness of American public education. Educators and policy makers of widely different perspectives embrace this U.S. ACHIEVEMENT BY SUBJECT MATTER conclusion because it creates enormous pressure for change. It is useful to those who are dedicated to reforming public The perception that the U.S. performs poorly on inter- education in various ways, as well as to those who seek to national assessments of achievement is typically associated diminish public schooling through strategies such as voucher with mathematics. This perception may not be accurate, programs that would increasingly privatize the system. With and it ignores U.S. performance in other subjects. so much at stake, it is important to know just how well U.S. Figure 1 presents U.S. achievement compared with that students have performed on recent international assessments of other industrialized nations in four subjects aggregated MAY 2005 689 across six surveys and the fourth, eighth, ninth, and 10th Cyprus, and South Africa.12 Hanushek based this observa- grades. In mathematics, less than half the nations (44%) tion on an analysis of TIMSS 1995 results in which he judged scored significantly above the U.S., while 37% of the na- high school performance to be represented by the achieve- tions had scores equal to those of U.S. The remaining 19% ment of students at age 17 (i.e., grade 12). This interpreta- scored significantly below the U.S. Viewed this way, the ag- tion is incorrect, however, because the TIMSS 1995 tests gregated results make it clear that the U.S. did not perform “poorly.” Instead, the U.S. mathematics score was FIGURE 1. somewhat below average. To say that the U.S.

Load more